CRO tips from Dave Powell: A/B testing, growth strategies, and price perception hack

Are you a Shopify store owner, marketer, or designer looking to boost your online sales? Conversion Rate Optimization (CRO) might just be the missing link in your strategy. In today’s blog, we’re sharing valuable insights from a conversation with Dave Powell, CRO researcher at Loop Earplugs, on Instant’s Click To Buy podcast hosted by Rebecca Anderson.

Dave shares his expertise on everything from A/B testing to understanding customer behavior, as well as how you can use these strategies to transform your online store and maximize conversions. Read on for an abridged version of the interview transcript!

Click To Buy, Episode 23: How do you create a CRO strategy?

To get the full episode, listen in via Spotify or watch the video version on YouTube:

Meet Dave Powell: CRO expert and experimentation advocate

Becca (00:01.248):  

Welcome back to another episode of Instant's "Click To Buy" podcast. Today, I’m joined by Dave Powell, CRO Researcher at Loop Earplugs. I think you might've heard of them. Dave's journey in the field of conversion rate optimization, also known as CRO, has spanned nearly two decades, including previous work at Booking.com, among others. His expertise in qualitative and quantitative methods has been instrumental in driving hypothesis-driven product improvements. At Loop Earplugs, he focuses on optimizing user experience and boosting conversions through strategic testing and analysis. But Dave is not only a father of two, a devoted husband, and a dog lover, but he's also a passionate advocate for embedding experimentation into strategic decision-making. 

Today, Dave is going to guide us through all the intricacies of A/B testing, experimentation, and what other topics we dive into, offering invaluable insights into how you can optimize your online store for maximum conversions. So, if you're eager to leverage CRO to refine your company strategy and stay ahead of the competition, you're in the right space today. Well, all I can say now is thank you so much, Dave, for being my guest here today on Click To Buy!

Dave Powell (01:22.816):  

My absolute pleasure! What an introduction—I believe I'm slightly overwhelmed; thank you so much. That's really quite something. I don't think many of us take stock for a moment to look at our accomplishments, so thank you very much for encapsulating that in such a lovely introduction.

The foundations of a good A/B testing strategy

Becca (03:02.642):  

To kick things off, I know that you've said you are really into building strategies. Can you share some of what you believe are the principles and foundations of setting up a good A/B testing strategy for an e-commerce company?

Dave Powell (03:35.338):  

Absolutely. What better question than to start at the beginning? I've often found that CRO is bolted onto certain departments here and there. It’s difficult—it depends on the size of the company, obviously. I've been around in this industry for a while. For it to work as well as possible, you need standard procedures. It needs to be a closed environment for running A/B tests; otherwise, you pollute your data.

But starting right at the top, if you really want to embed CRO in terms of your strategy, look at something that's been referred to as a reverse funnel tree. Every company has a top-line target: we need to make this money, we need this revenue. How about taking a look at your current website and splitting it out into its respective parts? Gather your sessions, purchases, revenue, conversion rate, and average order values. Then, you can plot it out to say, if we make conservative estimates—a 0.1, 0.5, or 1% increase—what will that mean towards reaching our revenue goals?

You’ll be able to cluster that information and see where you should be optimizing on your website with CRO first. It gives you the highest volumes of traffic, the lower conversion rates, and you see what it's generating. Realistically, you can plug that into your strategy, deciding, "This is what we should work on first in terms of CRO," because then it feeds the goal. That’s something I would really emphasize.

Becca (05:16.929):  

Absolutely; thank you so much for sharing such a strong foundation. I think a lot of what you said is perfect for brands of all sizes. These are things to think about or revisit if you're struggling to figure out where to go next with CRO. Sometimes, going back to those basics and really understanding that customer journey might unearth some of those nuggets, like you said. I love what you said about following that user journey: which browser are they on, and what device? It's crucial to understand the nuances that can make all the difference.

Balancing qualitative and quantitative research

Becca (11:39.264):  

I know that you are an expert in both quantitative and qualitative research. For brands that might not have enough traffic to do certain types of testing significantly, qualitative data can be crucial. How do you balance qualitative and quantitative insights in forming your strategy?

Dave Powell (11:51.252):  

Great question! The key is understanding that what people say and what they do are often two very different things. For instance, qualitative feedback might suggest a strong preference for a product, but A/B testing could reveal otherwise when it comes to actual behavior. One effective strategy is to use qualitative insights to form hypotheses, which you then validate through quantitative testing.

To give you another story about the differences between the two, let's talk about Sony Walkman. This is quite an old example, but it’s very telling. Back in the day, Sony had two Walkman colors to choose from—black and yellow. They conducted a usability test and invited people in to experience the new sporty product, which was bright yellow. Participants gave overwhelmingly positive feedback about the yellow color, saying things like "This would match my pop socks and my sweatbands perfectly!"

Sony then told the participants they could take a Walkman home with them, choosing between the black ones on the left and the yellow ones on the right. And guess what? Nobody took a yellow one. Everyone chose black. Despite all the positive things they said about the yellow Walkman, when it came down to actually making a choice, their behavior was completely different. What people say and what they do can be two entirely different things, which is why A/B testing is so critical—it allows you to validate assumptions at scale.

A surprising experiment on price perception

Becca (18:38.462):  

Can you share a surprising experiment or an interesting test that you've run recently that worked really well—or maybe didn't work as expected?

Dave Powell (19:00.256):  

Absolutely! One of the most surprising experiments we've run involved testing how we display pricing. The concept revolves around what’s known as "the pain of paying." People generally dislike the feeling of spending money, so we wanted to see if subtle changes in how we display prices could reduce this psychological pain.

Here's what we did: we tested how we display the currency on our website. We experimented with minimizing the size of the currency symbol or even removing it altogether and making the cents much smaller than the dollar amount. The idea was that if you make the cents smaller in comparison to the dollars, it perceptually makes the price feel smaller.

We ran this test for two weeks, ensuring we had a minimum of two weekends to account for different shopping behaviors, and achieved statistically significant results at 98% confidence. The result? A 7% increase in add-to-cart rates! Just by playing around with the size and visibility of the currency symbols and cents, we were able to make a noticeable difference in user behavior.

Common Pitfalls in CRO

Becca (24:23.38):  

Are there any pitfalls or common challenges you've seen companies face when implementing CRO strategies?

Dave Powell (24:25.866):  

Absolutely. One of the biggest pitfalls is viewing CRO as a quick fix rather than an ongoing strategic initiative. CRO is not a silver bullet, nor is it something that one person can handle alone. Another issue is when companies operate in silos, with CRO existing as a separate department rather than being integrated into the company's overall strategy. There needs to be a growth plan for CRO within the organization, and it should involve cross-functional collaboration.

Dave’s Click-worthy tip:

Becca (31:04.373):  

Dave, I could ask you a million more questions, but we're reaching the end of this episode. What's one last piece of advice for our listeners who want to improve their customer journey?

Dave Powell (32:40.92):  

I've got three bullet points for you: Where is it going wrong? Why is it going wrong? And what should we do to fix it? Keep these questions in mind to continuously optimize your customer journey.

Conclusion

Dave Powell’s insights offer valuable lessons for anyone looking to improve their CRO strategy. From the importance of embedding CRO in your company’s overall strategy to balancing qualitative and quantitative research, and even understanding the psychology of price perception—these tips can help you unlock new growth opportunities for your e-commerce business!