A recent Seattle SMX conference featured Ayat Shukairy, a key player in the investment optimization field, distilling years of experience for the benefit of those taking notes. Gems included the advice not to get too excited when conversion rates grow suddenly buoyant, as it may be a sign that testing has not been allowed to go on long enough for true and meaningful results. Other points touched included, remembering that a website is only a piece of a brand panorama created by the seller, never assuming there isn’t more to learn about customers, not rejecting poor testing results, as such results can teach experts as much as winning results, and to be very vigilant about ways that testing can be contaminated.
- We often focus on the wrong thing. Stop focusing on the short term. We need to keep our long-term testing plans in mind when executing our strategies.
- Don’t assume you know everything about customers. Listen to customers through qualitative research.
- Embrace failure. More often than not, your tests will fail to drive meaningful growth. That means you must push through and innovate on your failure.
“if you see a huge lift in conversion rates (CVR), then check to make sure that (a) you’ve run the test long enough to produce meaningful results, and (b) this test is not an outlier.”