Sep 13 2018

The Culture of Experimentation – Opticon ’18 Keynote

Since before the age of digital transformation, experimentation has always been a critical tool for understanding your customers and breaking through the status quo. However, experimentation is often siloed, not accessible across the organization, and left out of the overall strategy of connecting with customers.

Optimizely and its customers aim to overcome this hurdle and create a culture of experimentation. A culture of experimentation gets everyone involved—sales, product, merchandise, creative, and design—and empowers your employees to unearth new ideas under a democratic environment. Once everyone is onboard and motivated, you can use experimentation to anticipate and deliver products or content based on what the customers needs, not just what you want them to see.

State Farm started its digital transformation two and a half years ago. After only two years, Forester recognized State Farm as one of the leaders in digital transformation. Additionally, after completing a broad evaluation of digital experiences in the insurance industry, Gartner placed State Farm at the top. State Farm’s VP Digital, Fawad Ahmad shared his perspective on the company’s digital transformation journey.

Insurance companies are unique in the fact that when their customers need them the most, they typically are not sitting in front of a computer. As a result, State Farm focused on creating a great mobile app, one that can serve customers in any situation.

But, the app wouldn’t have become a success without a shift in the organization’s culture.

They started with a vision of creating seamless experiences of personalization across all channels. Once they had the vision, they set out to achieve this transformation in the next four to five years. There were five key areas State Farm focused on to make this transformation happen:

  1. Experience Customization
  2. Data Analytics
  3. Technology
  4. Process
  5. Culture

By shifting the culture towards experimentation, State Farm is able to “better understand what the customer needs rather than just putting things out there which we like,” according to Ahmad.

State Farm is creeping on its 100 years (the organization is currently 96 years old). That said, Ahmad says that experimentation is not new to State Farm, but it was never a core piece of the culture. With the digital transformation, everyone can now have a view of experimentation—from CEO to staff, from design to IT. Initially when they started off, they worked with project and marketing managers to create the experimentation pipeline. During this time, Ahmad says that it was more of asking “Why can’t we test this?” and pushing the team to get to the next level of experimentation. Since then, it’s evolved into more of a pull, fostering learning and curiosity among the organization. Continually pushing will only get the team so far if they are not willing to grasp the culture shift.

As far as scale, State Farm has increased their number of experiments three to four fold in the last to years. The next focus is on the velocity of experiments and increasing the size of the impact they have. They are not quite there, but they continue to scale to achieve this.

All experiment results are shared across the entire organization. For the CEO, they share the results are part of the contribution to the overall goals of the digital transformation. Everyone in the organization has access to the depository of experiments and their results to learn and see how it all relates back to the digital transformation.

When asked what advice he has for companies about to embark on a digital transformation, Ahmad answers, “If you want to swim, you have to get in the water.” It’s important to determine what you want to change within your organization and how you’re going to make the change. Don’t underestimate the resources needed. ”It’s a massive transformation effort, but don’t hesitate to take that leap,” states Ahmad.

You also need to be supportive of your team trying new optimization tools, while at the same time holding them accountable for what the organization is trying to achieve holistically. If you’re apprehensive at first, Ahmad suggests, “go to your comfort zone and start out small, then scale.”

After Fawad Ahmad from State Farm, Jon Noronha, Director of Product for Optimizely, walked the audience through the digital journey of Sky, Europe’s leading entertainment company and “one of the original digital disruptors.” They were on the of the first media groups to provide satellite television. Since then, they have been dedicated to the user experience and changing the way we watch television.

In the last ten years, television has evolved—every device has essentially become a television. It used to be we all gathered in the living room, crowded around the TV; now, we can watch it anywhere. Tap, touch, voice are all technologies that have been inserted into television. We’re in this state of constant digital transformation, but how do you stay ahead of it?

Like State Farm, Sky built an experimentation culture a few years ago. The running joke is that on Optimizely’s scale of experimentation maturity of one through five, Sky aims to be the first six. From the start, Sky always recognized that creating a culture of experimentation is creating a culture that is more agile. They don’t want to just run A/B test; they want to leap pasts the Amazon’s and Netflix’s of the world be one of the top experimenters in the world, according to Noronha. Their goal is to reach every audience at every touchpoint.

Sky has critical personas, but personas adapt and change over time. How do you reach them? This is where many organizations get stuck. Noronha refers to this as the “Personalization Trap”—personas are rigid, experiences lack a human element, and it becomes difficult to scale. As a result, many organizations are using a new type of approach: one-on-one personalization. This approach provides a more holistic and honed personalization strategy.

Sky tracks which content the users are reading on their site and learns from those signals. Using Optimizely’s natural language processing to build a model of all the content on the site. All the articles, blog posts, videos, etc. and reads through all the content to make sense of how each one relates. Noronha provides this example: If someone comes to your site and listens to a podcast about the PGA golf championship, they can learn all the related concepts related to that content. Noronha says, “We pull out keywords like PGA, championship, Tiger Woods and learn how other concepts are related to things like golf, sports, and use that build a bigger picture of who someone is. As users browse around the site, we can use this to build a model of their overall intentions.” The models can adapt and change over time as the user continues to browse the site and read other content.

You can then segment similar users into clusters or adaptive audiences. You would provide a few keywords to describe these clusters to match other users based on interest. They are not rigid rules but adapt over time.

Adaptive audiences is only half of the story of reaching every audience at every touchpoint. In the last ten or fifteen years, there has been an explosion of devices and channels—tablets, mobile phones, wearables, social media, chatbots—that go beyond browsing on the site. How do you create experiments for every single one of these technologies? Luckily, underneath the hood (as Noronha refers it), these devices are all the same, it’s all code. The value will come from experimenting on the underlying code that powers different channels. This is where Optimizely’s Full Stack Experimentation comes into play. You run experiments using service testing and each product and feature launch can be an experiment.

Sky, Blue Apron, and StubHub all use Optimizely’s Full Stack Experimentation. Members of each organization state that the tool helps them experiment and learn faster, take out the guesswork, and open a full range of experimentation.

Sky decided to use a full stack experimentation on their call center. Every month, thousands of users call the customer service center. The Sky team created a way for the call centers to deflect the users and direct them towards the site to quickly answer their questions. Since starting the experiment two years ago, it has seen great success. Based on recent analysis, experiences subject to experimentations resulted in 16 percent fewer calls and customer satisfaction improved 8 percent.

Sky uses full stack to release experiences as experiments—everything that is released is an experiment. As a result, it’s fair to say the organization is customer led. “We let customers decide what is the right thing for them,” says Abdul Mullick, Head of Digital Transformation at Sky.

Mullick compares their philosophy of experimentation to a restaurant. “If you go to a top restaurant, you find it unusual for the kitchen team not to test what they are delivering to you as a product.” He adds, “It’s counterintuitive not to experiment first.” After each experiment, the data feeds back into the process, so Mullick and team are constantly improving and enhancing the customer experience.

If you are going through a digital transformation or about to, keep in mind that a culture of experimentation is key. It may start in the digital team, but it needs to broaden to the whole organization. Once people grasp the culture shift, you will likely discover new, creative ways to experiment and enhance the overall customer journey.

Shannon Kenneally

Content Writer

After graduating from Clemson, Shannon started her career in marketing, focusing on content creation and engagement. As Blue Acorn’s Content Writer, she keeps clients and customers up-to-date on the latest trends and news in the ecommerce world. In her free time, she enjoys running with her dog and checking out the local breweries in Charleston.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.