Recently our team spent several days in San Francisco with our partners from Optimizely at Opticon, their annual conference. Between exciting Optimizely announcements, spending time with clients and partners, and attending information sessions, it was a successful trip to the west coast.
Dan Siroker, Co-Founder & CEO of Optimizely, took the stage to make several announcements. The crowd was really buzzing over the new Optimizely Personalization tool. It is in private beta now, with general availability coming this fall.
How is this new tool different from Optimizely’s widely-adopted optimization platform? Personalization essentially allows one to target segments of visitors based on behavior (for example, visited product page “A” two times) or demographics to offer them personalized experiences. On the other hand, conversion testing focuses on determining which changes, or combination of changes, result in optimal on-site performance.
At Blue Acorn, we believe that the best approach is to begin with a conversion testing strategy to ensure that your site is functioning to the best of its ability. Then, once the site is in order, layer a personalization strategy on top of it to really hit it home.
Even though the intention of personalization isn’t to “test” per se, please note that Optimizely will initially be holding back 5% of any given segment to create a baseline for performance comparisons. The remaining 95% will be exposed to the personalized experience.
While visiting with clients and partners at our booth is always a good time, I was able to sneak away on a few occasions to attend several of the information sessions organized by Optimizely. As a conversion optimization consultant, I use the platform on a daily basis. I get deep into the weeds, seeing both the limitations and the vast possibilities of conversion testing, so I am always on the lookout for opportunities to build on my knowledge of existing and new capabilities.
Here’s a quick recap of three sessions I attended.
Testing with Stats Engine: Why it’s Harder to Find Winners with Optimizely: Prior to Optimizely’s Stats Engine, an analyst was responsible for determining whether a sample size was large enough for results to be considered relevant. With Stats Engine, this is no longer necessary. The biggest takeaway that they emphasized on several occasions was the importance of setting a primary goal for every test. The primary goal, above all secondary goals recorded through Optimizely, receives a greater amount of statistical power behind its calculation. Setting the primary goal prior to launch is key to obtaining optimal (and hopefully more timely) results for your test’s primary KPI. Check out this Optiverse article to learn more about setting a primary goal.
Early Lessons in Personalization Strategy: I am quite excited to start working with Optimizely’s new personalization tool. One feature that really stood out to me was the Audience Explorer. As an alternative to digging through site analytics to determine optimal segments to test, Optimizely provides suggestions for which segments to target using predictive analytics. Panelists who shared their own stories recommended using a combination of data and gut instinct to get started with a personalization strategy. Fair enough.
Beyond all of the inspiring optimization talk, the view from Opticon’s location at Pier 27 in San Francisco wasn’t too shabby either. Thanks for a lovely week, Optimizely!