AI-Driven A/B Testing: The Next Evolutionary Leap
2:05 - 2:35 | Thursday, May 23
AB testing is one of the most powerful techniques in the data science toolbox, enabling practitioners to attribute causality to individual product changes. This technique has enabled companies as diverse as Amazon, Facebook, and Google to grow into market-straddling colossi.
However, A/B testing does have challenges, especially for smaller players in the market. One challenging problem is local maxima – never quite knowing if sequential A/B tests have reached the apogee of their potential, or if they are residing on a lower peak. The traditional answer to this challenge has been multi-variate testing (MVT) – armed with sampling and statistical heuristics to find the best peak by testing many variants at once. But this takes lots of traffic. Lots and lots of traffic – a luxury not available to all. To solve the problem of local maxima without the availability of massive traffic, we have developed a testing strategy we call “Neo-Darwinian testing”.
Why does Neo-Darwinian testing represent a leap forward? By using genetic algorithms, we’ve found a way to intelligently look at a population of tests, and rapidly breed new generations of variants until the ultimate ‘genome’ wins the test – each generation shaped by the external pressures of real customer behaviours. The power of this technique enabled us to run an MVT test so large that they would normally take 10 years to produce a statistically valid result, and get results in 30 days…
We’ll show you how…
Director, Global Digital Business, LexisNexis
Global Director, Data and Insights, LexisNexis
Director, Global Product Testing, LexisNexis