Get the latest tech news
A/B testing mistakes I learned the hard way
Bad experiments are worse than none at all
In fact, while the new flow worked great on mobile, conversion was lower on desktop – an insight we missed when we combined these metrics. Creating a list of product health metrics you want to monitor, such as retention, session duration, or daily active users, and check how they change during and after your experiment. Another option is to run a holdout test, where a small group of users is not shown your changes for a long period of time – e.g. weeks or months after your experiment ends.
Or read this on Hacker News
