Jun 17 2016
magento optimization

eCommerce: How to Measure Minor Changes for Maximum Impact

As Magento’s 2015 North American eCommerce Partner of the Year, Blue Acorn has more than a few award-winning site launches under our belt. These include Magento’s 2016 Best Web Design for Le Creuset, which was also a finalist for an Internet Retailer Excellence Award. You might say that we’re in a pretty good position to tell you that overhauling your site is a smart move. But you may be surprised to learn that we also often advise clients that making small, measurable changes can have a really big impact as well. When you consider that the key word here is measurable, small changes make a whole lot of sense.

Blue Acorn has always been driven by data, using metrics and experience rather than gut feel to guide our actions. In this way, we ensure that the changes we make on behalf of our clients will have a positive impact on their customers’ shopping experience, driving engagement and ROI.

Most people agree that it’s smart to use data to analyze decisions, but we rarely see it in practice. (Crunching numbers is like eating kale: You know you should do it, but it’s also kind of distasteful for many of us.) We want to make it easy. Here are two examples of how we measured incremental changes for maximum impact on recent projects. Adopt this philosophy yourself, and you’ll see how much easier it will be to reach your goals when you know what works and why.

Situation #1

One of our clients, a popular global lifestyle brand, has legions of fans who love its eponymous designer’s playful, edgy style. The company sells a wide range of apparel, handbags, accessories and footwear at hundreds of retailers worldwide and on its eCommerce store. With thousands of products to choose from, though, browsing and shopping can get a bit overwhelming. Together our teams wondered: How could we create a better online experience?

The Small Tweak

Wishlists, which allow shoppers to select and keep track of items they like, are a great way for brands and retailers to simplify the online shopping experience for their customers. (Think of wishlists as the online equivalent of starting a fitting room.) But the wishlist feature that comes out-of-the-box with Magento is only available for users who log in, which introduces a bit of friction to the process. We determined, by looking at data, that the number of people using the wishlist on this client’s site was low—just a small percentage of site traffic.

Blue Acorn and our client theorized that building a better wishlist would drive engagement and revenue. So we built a tool, which we call the “Loved Items” feature, that allows shoppers to click on a heart to “love” an item on either a category or a search page. This saves the item to the shopper’s “Loved Items” page, which they can easily access at any time in their top navigation bar. We also designed the tool to display the total number of “loves” for a product, so shoppers could gauge the popularity of an item. Then, we made it easy for shoppers to add single-SKU “loved” items to their cart, or to see details in a quick view to order items that require configuration. Finally, to reduce friction and encourage use, we made the feature available to both guests and registered users.

The Results

Typically, we’d recommend A/B testing a new feature against an old, so that the data could determine a winner. In this case, A/B testing would have been cost-prohibitive, so we turned to Google Analytics for verification instead.

When we looked at the numbers, we saw a large increase in engagement and revenue generated from the “loved items” feature compared to the wishlist (so our theory proved correct), and we saw even larger growth in the percentage of overall revenue generated from users who interacted with the loved items feature vs. the wishlist.

While we’d theorized that the tool would boost engagement by removing friction, increase confidence by offering social proof, and boost revenue by making shopping easy, after looking at the analytics, we knew for sure.

It took our client just nine days to break even on its development investment of less than $15,000. And, of course, annual ROI is projected to be much greater. Pretty impressive for a small tweak! If we’d overhauled more than just the wishlist in a new site rollout, it would have been difficult for us to determine which features were driving revenue. But by making a small tweak, and then measuring it, we know for sure. Can see why we’re such big fans of small changes?

Situation #2

Another one of our clients, a clothing and lifestyle store that sells an eclectic mix of carefully curated items worldwide, was looking to drive conversions and revenue per visitor. The retailer thought that offering product reviews on its product pages, along with recommended items, would help it reach its goal.

The Incremental Change

Our client redesigned its product page to include reviews, as well as tabs presenting recommended items along with “styled with” products. It also increased the size of its
add-to-cart button and created a smaller, less noticeable live chat button. The company theorized that these small changes would drive conversions and revenue-per-visitor. Rather than charge ahead with the change, though, we recommended split-testing the new design against the old to make sure the theory would prove out.

The Results

Once again, relying on data proved to be a smart move. Our analysis showed that the new layout, rather than driving positive change, was actually negatively impacting conversions by 5.6 percent! In fact, the old layout held steady as the winning variation for the duration of the test.

Through detailed analysis, we learned that the larger add-to-cart button increased the rate at which people added items to their cart (good), but had a negative effect on the rate at which people completed their purchases (bad). What’s more, the newer, smaller live chat button performed substantially worse at engaging visitors than the old live chat button (also bad).

Thanks to split testing these small changes, we were able to prove that the tweaks we hoped would drive conversions and revenue didn’t pan out. Far from being a negative, this test proved to be a positive experience by saving our customer from replacing a design that was working with one that would perform worse. Today, the retailer is using those findings to make additional small tweaks, such as a larger add-to-cart button and a larger live-chat button, to continue to improve the shopping experience for its customers and drive revenue for its business.

Bottom Line

Next time you’re thinking of overhauling your site, consider whether dialing your plan back might be a better option. (Sometimes an overhaul is the right way to go, but many times it’s not.) Just like our clients do, consider making small changes one at a time and then measuring them. In this way, you’ll gain a solid understanding of what works and what doesn’t, so that you will be sure to be moving in the right direction going forward, no guesswork required.

Blue Acorn

For all your ecommerce needs, Blue Acorn provides an integrated team of designers, developers and strategists who work together to deliver engaging shopping experiences for your customers and efficient processes for you. Whether you’re looking for a fundamental or incremental change, the team at Blue Acorn will adopt your ecommerce KPIs as our own and deliver the results that matter most to your organization. For brands like The Real Real, Peter Thomas Roth, and Too Faced Cosmetics, we provide services such as ecommerce design, systems integration, A/B testing, new feature development, and personalization.


Jun 19 2016

“We also designed the tool to display the total number of “loves” for a product, so shoppers could gauge the popularity of an item. ” – Love this idea, would be nice to see it implemented by more e-commerce websites.

Did adding the reviews in product pages have any impact on time spent on page / conversions for Case #2?

Matt Rickerby
Aug 31 2016

We did also test showing/hiding reviews and saw that hiding reviews resulted in a RPV lift. However, we didn’t look at time on page as a primary KPI.

Jul 15 2016

This is great, I am starting to focus on e-com a lot more and this is perfect =)

Aug 01 2017

As part of a team working on the Actonlytics eCommerce tool, I do find that it is very important to implement small changes and tracking their performance.

Very good article!

Keep it up.

Parth Desai
May 23 2018

The post is brilliant. Often while looking at the big picture we forget the small things which make a huge difference in the building of a successful e-commerce store.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.