The power of A/B testing – @HBR

The power of properly constructed online experiments is explored in September-October’s Harvard Business Review. Microsoft’s Ron Kovani and Harvard’s Stefan Thomke include some powerful examples of results achieved as well as cautionary tales on the dangers of poorly constructed tests and misinterpreted results.

Highlights include:

• How a subtle change to online text colours (against the designers’ recommendations) drove increased revenue of $10m per year for Bing.
• How online tests showed that  every 100-millisecond difference in performance had a 0.6% impact on revenue, equal to $18 million in annual incremental revenue.

Interesting insights.

Photo by Matthieu Joannon on Unsplash