How to test changes to your listings properly

Most sellers tweak their listings all the time: a new main image here, a longer title there, a price nudge after a slow week. Then they look at their sales, feel either pleased or worried, and move on. The trouble is that sales go up and down on their own, so it is easy to keep a change that did nothing, or undo one that was working.

A simple, fair test takes very little extra effort and tells you far more.

Measure conversion, not just sales

Sales depend on how many people see your listing as well as on how good it is. If a change happens to coincide with a busy week, sales rise whether or not the change helped. Conversion rate removes most of that noise:

Conversion rate = sales ÷ views

If 820 people viewed a listing and 14 bought, the conversion rate is 14 ÷ 820, about 1.7%. Most platforms show views and sales for a chosen date range in their shop statistics, so you can work this out for any period.

Five rules for a fair test

1. Change one thing at a time. If you change the main image and the price together, you cannot tell which one made the difference.
2. Compare periods of the same length. For example, the 14 days before the change against the 14 days after.
3. Avoid big seasonal shifts. The run-up to Christmas against January, or a sale week against a normal one, will mislead you whatever you changed.
4. Write it down on the day. Note the date, what it was before and what it is now. Memory is surprisingly unreliable a month later.
5. Wait for enough data. A handful of views or sales can swing wildly by chance.

A worked example

A seller replaces a flat lay of a planner cover with a close-up of a filled-in page and a few words of text. In the 14 days before, the listing had 820 views and 14 sales. In the 14 days after, it had 905 views and 23 sales.

Views only rose by about 10%, but conversion rose by about half. That is a change worth keeping, and a clue about what buyers want to see.

When the numbers are too small

Now take a listing with 95 views and 2 sales before a change, and 88 views and 2 sales after. The conversion rate moves from 2.1% to 2.3%, which looks like an improvement. In reality, one extra or one fewer sale would flip the result. With numbers this small, the honest answer is that you cannot tell yet.

As a rough rule of thumb, wait until you have at least a couple of hundred views and five or more sales across the two periods before drawing a conclusion, and treat changes of less than about 20% as no clear difference. These are not strict statistics, but they stop you reacting to chance.

Price tests are different

A higher price usually means fewer people buy. That is not necessarily bad, because each sale earns more. When you test a price, compare the profit for each period as well as the conversion rate.

Say a workbook goes from £8.00 to £9.50. Before, 6 sales from 260 views; after, 2 sales from 240 views. Even though each sale now earns more, total profit for the period fell by more than half, so that price rise did not pay off. If it had still made 5 sales, profit would have been almost the same with fewer buyers to support, and the higher price might have been worth keeping. The Profit Calculator works out profit per sale at each price.

What to test first

• The main image. It decides whether people click at all, so it often has the biggest effect.
• The first few words of the title. They are often all that shows in search results.
• The first two preview images after the main one. These answer the question: what exactly do I get?
• Price, once your images and title are doing their job.

Keep a log

One test tells you a little. Ten tests tell you what works for your shop and your buyers. The free Listing Experiment Log records each change, works out the conversion rates and gives a verdict, then shows which kinds of change tend to help across all your tests.

More articles