Aug 08 2017
UX versus design

Testing: Before, During, and After Launching a New Design

Launching a new version of an ecommerce website is no easy feat, especially if it’s an entirely new design. It takes time, as well as a great deal of strategy and planning. When it all comes together, the best you can hope for is that your customers and prospective customers will follow the journeys you laid out for them. Unfortunately, we also know that consumers are nothing but unpredictable. Without testing, your data will show that they are finding their own way, in most cases the path of least resistance.

Creating Predictability and Reducing Risk

All change comes with risk. Testing throughout a transition or cutover mitigates risk, and introduces more predictability.

The following approaches and opportunities come from the brains of our internal optimization and UX strategists Meredith Eads and Erin McElwee.

Testing and Benchmarking Before a New Design

Whether you’re considering a full design cutover or iterative changes, you’ll want to start the process by identifying major areas of change. With this in mind, the strategy should consider prototyping, user testing, and optimization (like A/B testing).

Identifying Major Areas of Change

Change can come in the form of simple fixes like designing a more user-friendly navigation (why are you still using hamburger menus?), cutting steps out of your current customer journey based on where they fall off, or adding automated upsells and cross sells to your homepage. Of course, more revolutionary changes are often required. If you’re migrating to a new ecommerce platform, you can save an incredible amount of time and money by basing the design on the platform’s native functionality.


There are a lot of tools available to prototype what these changes could or would look like. Here at Blue Acorn we use Invision, Balsamiq, Sketch, and a few others, but the options are seemingly endless. Regardless of the tool, prototyping allows you to test the usability of a concept or experience without engaging in development. Without putting forth too much effort, you’ll get a truer picture before steaming ahead.

User Testing

If you have an existing web property, getting quantitative data is like shooting fish in a barrel, but qualitative data isn’t as easy to gather. User testing can go a long way for capturing usability feedback. To be effective, you’ll need a working production deployment of the feature you want to test. If you want to get crafty, you could test on a competitors site.


At Blue Acorn we refer to optimization as the incremental approach, or where you take what is currently in place and find ways to squeeze more out of it. On the other hand optimization should also be used to test larger, riskier changes on your production site in order to determine if a feature should move on to a redesign as well. This allows you to test against any hunches that you may have for why customers may be falling off their purchase journey, and avoid bringing it to a new version of the site.

Testing During a Redesign

Before a site launch testing makes perfect sense, as it does after you cutover, but what about during? Absolutely. Now we know what you’re thinking. Testing in the middle of your project will lead to concerns about your current direction, potentially affecting your budget, and throwing your deadlines off. That may occur if you’re testing a core component of the new site, but while progress is being made you can focus on smaller, feature-level changes or business decisions to validate.

One test example may be focused on the presentation and content of a promotional banner, or even the decision to display on-model photography rather than the standard flat products on a category page.

It’s similar to the prototyping design decision, but depending on the scope, you can usually leverage the current production site to evaluate changes like promo messaging, product photography, sizing guides, product page content, low stock messaging, exit intent, data-driven, or event curated upsell/cross-sell opportunities.

According to Erin and Meredith, there is one caveat on this. If the functionality and data doesn’t yet exist to test against on a web property, a test will cost you more time and effort. With that in mind, it may be better suited to test after a cutover takes place.

Testing After a Redesign

Now that you have insights from tests conducted before and during the cutover, it’s time bask in your results! Ok, not so fast, there is still some more work to be done here. Just like a marathon, it’s all about the journey to get to the end, and just when you think you’re there, you’ve actually got one last mile to knock out. If you’re on a platform that doesn’t allow much wiggle room for customizations, you’ll be hard pressed to add many new features post-launch, but you can still test against some of these concepts. For starters though, you’ll want to ensure you have a optimization or personalization platform/tool in mind such as Optimizely.

Maintenance and Feature Development

Post cutover, the next step is to build out a plan for ongoing feature development and improvements. Technology in our space moves surprisingly slow, but that doesn’t mean the platform you’re on won’t roll out something on a quarterly basis that is worth implementing. Take for example an abandon cart test we conducted for cosmetics brand. On their desktop mini-cart and mobile experience we created a custom designed full-width banner. The result? More than a 5 percent increase in cart pageviews and also a small increased to their conversion rate as well. For this particular client we were also able to show that their mobile and desktop users reacted differently to the concepts test, which allowed them to make a strategic choice to move forward with adding the banner on mobile.

Business Initiatives

During the holidays most brands should see an influx of traffic, but not all are willing to risk changes to their site. Leading up to the major holidays, you can conduct a smaller test run, but with a similar promotion or feature.

Take for example what we did for Victoria Emerson and a shipping message test. By adding a message that simply said “Get it in time for the Holidays” to their product detail page (PDP) they saw an increase to their revenue per user by as much as 20 percent and 4.6 percent increase to add to cart. We also lobbed off the zeros from their price listings ($20 versus $20.00) and got some interesting results. From this, Victoria Emerson found that messages that indicated urgency on the PDP resulted in more add to cart actions and drove revenue by 43.5%


If AI isn’t the biggest buzzword you’ve heard this year, personalization is likely second on that list. We’ve heard it for years, but ecommerce platforms and marketing platforms are getting smarter at using the behaviors of customers to create a more personal experience. Less retargeted ads of items you just bought versus upsells, sidesells, and subscription reups later down the road. For more on personalization, our Q&A with Monetate is a good starting point.

Can’t Stop, Won’t Stop

Of the test and processes suggested here, this really only scratches the surface. Each site and brand will have their own set of priorities, and their consumer data will have a different story to tell. Not sure where to start? If you’d like to pick the brains of some of Blue Acorn’s experts, feel free to reach out here.

Elliot Volkman

Digital Marketing Manager

Elliot is Blue Acorn's Digital Marketing Manager. He hold a master's degree in communication from Gonzaga, and has several awards for journalism and digital marketing. In his spare time he is a long-distance runner and triathlete.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.