Nov 13 2013

The End of Lazy Reporting

Making changes to your eCommerce site without measuring the impact is like getting dressed in the dark. The first step in getting smarter about these enhancements and understanding how moving needles relies on a commitment to measuring their effect. To get the most accurate measurement, we recommend testing every change you make side by side with the original. In the event that doing so isn’t possible, a before and after report is your next best choice.

The two most widely used types of before and after reports are “year-over-year” (YOY) and “period-over-period” (POP) techniques. While both give you a glimpse of improvement, they both leave room for error. Year-over-year doesn’t take into account trends you’ve already been seeing this year. You may have already been doing well or poorly. Meanwhile, period-over-period doesn’t take seasonality into account.

There is a third type of report. It’s not as simple as the two I just mentioned, and therefore not as popular. That said, it can give you a more accurate reading.

The Third Option

It’s called the trending delta, a statistical analysis that allows you to compare the trending data with observable data. Trending data is the result of several calculations, which take into account seasonality and the overall trend of your KPI. By comparing it to the observable data, we can calculate our improvement with increased confidence.

For example, the chart below displays the revenue of an eCommerce site over the past year. We’ll assume that a change took place between B and C.

Trending Delta Graph

We can see that comparing A to C (a YOY comparison) makes it look like the change made an improvement. The same goes for comparing B to C (a POP comparison). But when we compare D to C (a trending delta comparison), it’s pretty obvious that our change didn’t make an improvement at all. In fact, our change hurt revenue.

Trending Delta Real example

a real example of trending data for a business whose conversion rate was slowly decreasing

In the graphs above, trending data is shown as a very simple, linear progression but the truth is never so straight and simple. A trending analysis should take prior seasonality into account. For most retailers, this will primarily be expressed as high spikes in traffic and revenue at the end of the year. The more years you take into account, the more accurate your analysis becomes.

Trending delta real example of seasonality

a real example taking seasonality into account by comparing each month to the month before

Another important consideration when looking at trending data is how many changes you’ve made over the past year. The more changes you’ve made, the more erratic and less reliable trending data becomes. It’s like any experiment- introduce too many variables, and drawing a conclusion becomes fuzzy if not impossible.

Calculating Trending Data

There are a number of ways to calculate trending data. Your best bet is to export data from analytics into Excel and use a function to make your calculations. The first go will take some patience, but once you have it, the process shouldn’t take long at all.

Want to start calculating trending data? Check out this article from microsoft on how to use the FORECAST function in Excel to calculate trending data.

In Conclusion

If you’re going to test, measure. If you’re going to measure, be accurate. And if you aren’t going to use trending data, make sure you understand how POP and YOY data may be skewed. Lastly, remember that it’s natural to favor information that confirms your beliefs or hypotheses, but doing so will lead to making bad decisions.

Kevin Eichelberger

Founder & CEO

Founded in 2008, Blue Acorn is the product of Kevin’s great passion and knowledge of all things eCommerce. Kevin’s data-driven approach has culminated in a strong, growing business that’s success is closely tied to the success of its clients. When he’s not immersing himself in eCommerce, Kevin works toward expanding Charleston’s tech community by serving as a board member for the Charleston Digital Corridor Foundation, and is also a mentor and advisor to several startups. A business-savvy technologist, you can find Kevin evangelizing about data, optimization and eCommerce.


David Weightman
Feb 07 2014

Finally! A big cure to me less productive days. Considering that there’s analysis and calculations involved in interpreting trending data in ecommerce, this will definitely give me a longer yet worth time to spend on. Thank you the share Kevin! This is so bookmark worthy! =)

Leah Welsch
May 06 2014

Great post Kevin- informative and well written.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.