Analysis of Binary Outcome A/B Tests – New Metrics

As a user (Product Manager, Analyst, HiPPO) of AB tests or a
designer/builder of AB tests and systems, do you have a nagging
feeling that though A was statistically better than B, the
difference wasn’t large enough to be important? Do you have
doubts about the business impact of the test outcome? Do you want to
set a single-valued business or revenue goal (e.g. 10% better) and have a definite recommendation
made to you instead of dealing with “we are 90% confident that A is
11.1% better than B”? Do you feel overwhelmed by the “test
everything” approach becoming increasingly prevalent in Tech? If you
answered yes to any of these questions, then …

Read More: ab_stats_blog

from NerdWallet
https://www.nerdwallet.com/blog/engineering/analysis-binary-outcome-ab-tests-new-metrics/