Include click-through rate in split test's response results

  • 4
  • Idea
  • Updated 1 year ago
  • Not Planned
  • (Edited)
Idea: besides the usual open rate, the Split Test results should also include the CTR for each subject line.

It's unlikely, but I could see a scenario where one subject line performs poorly in open rate but does well with CTR. Plus, it's good feedback and helps us craft better overall subject lines.

Thanks!
Photo of Rocketseed USA

Rocketseed USA

  • 3 Posts
  • 1 Reply Like

Posted 5 years ago

  • 4
Photo of Katie Lewis

Katie Lewis, Official Rep

  • 3414 Posts
  • 309 Reply Likes
Good morning! The goal with subject line split testing is to determine the most successful subject line, but it sounds like you’re looking for the most successful mailing overall — as though each subject line contains a different mailing with separate response stats. Is that right? I want to make sure I’m understanding your goal before taking it to our product teams. :)
Photo of Rocketseed USA

Rocketseed USA

  • 3 Posts
  • 1 Reply Like
Hi Katie,

That's kind of it. If each subject line contained its own mailing, we could test other important marketing elements like the call-to-action or the content layout. The best performing elements could then be combined into a single, optimal mailing. This would be neat! But it also sounds like a BIG update to the current Emma editor, so I'm not sure how feasible it is for your product team.

What I'm proposing is simpler: expand the Split Test response section to include Click Through statistics. My clients want to know if a higher open rate is indicative of a higher click through rate. Return Path has a good post about this on their blog:

http://blog.returnpath.com/blog/casey...

Let me know if this helps?

Thanks, and Happy Holidays to the Emma team!
Photo of Katie Lewis

Katie Lewis, Official Rep

  • 3414 Posts
  • 309 Reply Likes
Thank you for clarifying! I’d like to see what our product teams think. I’m sending this their way, but due to our holiday hours, they likely won’t be able to dig in until January. I’ll keep you posted, though, and let you know any other questions they might have!
Photo of David Wright

David Wright, VP of Product Management

  • 110 Posts
  • 27 Reply Likes
Good morning! Thank you for the feedback on how to improve the split testing experience for our customers. Our current implementation covers the greatest use case: optimize how a message will appear (unopened) within the inbox to drive greater opens. Increasing opens provides a greater reach and higher probability that the click-through-rate will also be higher. So, in general, running a split test to select the best subject line should drive more clicks.

With that said, in the examples provides by Return Path, if the subject line is disjointed from the call-to-action, increasing opens won't necessarily ensure clicks or conversions.

Part of our vision for 2014 includes adding capabilities that facilitate personalized, timely communication to increase conversions. While it hasn't been specifically identified to be worked on in the near term, I could see value in adding click-through-rate as data to bring back and/or as a variable to select in choosing the winner. So, while this particular enhancement is not planned, it certainly fits into our larger theme of work.

Regarding the mailing A/B testing you've described, I love the idea of providing an easy way to build variations of the same mailing, from subject line down to content, and run those side by side to see which mailing converts better. Again, it's not on the immediate roadmap, but we'll add it for consideration.

Thanks again for the feedback. We definitely want to make sure we're surfacing the right data points that set our customers up for success with their marketing. If you have additional ideas, please keep them coming!
Photo of Laszlo Ujj

Laszlo Ujj

  • 14 Posts
  • 0 Reply Likes
Thanks for the quick response, David. I had my fingers crossed that this could be one of those quick updates, but I realize now that the 'winning variation' component of the subject line test feature complicates things.

As I'm sure you are aware, the 'test, test and test again' mantra gets a lot of airtime in Email Marketing. So anything that helps further that capability is a welcome addition!

Thanks again, and best of luck with your roadmap.
Photo of Flyte World Dining & Wine

Flyte World Dining & Wine

  • 23 Posts
  • 3 Reply Likes
I've got a real-world example making the case for this feature. We're running a split test for a mailing to drive signups to our new rewards program. All the subject lines tie in to the rewards program in different ways. The test so far says all the subjects are doing about the same... so now I'd like to know who's actually clicking through to signup. I can see signups happening, but I can't tell which version of the subject they clicked.... if I knew that it would help me choose the best subject line. Without it, I'll be guessing and hoping for the best.
Photo of UC Davis Stores

UC Davis Stores

  • 2 Posts
  • 0 Reply Likes
Has there been any progress on this feature since it was requested over 3 years ago? I have used other email systems that have this feature and often the actual click-thru for a specific test SL (when making a very close call) is extremely helpful. It seems like it would be a very simple metric to include in the AB testing functionality and reporting. 
Photo of Grey Stepp

Grey Stepp, Support Team Manager

  • 1699 Posts
  • 135 Reply Likes
Hi there UC Davis!  

It looks like what you're looking to accomplish could actually be done using our new A/B Content testing.  

Your account currently doesn't have this feature, as it requires being on the new Emma platform.  

Using the Content A/B testing, you can choose what metrics you're looking for to trigger the final send, and we track opens and clicks as well if you prefer to do a manual send to the winner.  

Check out our documentation on that here:  https://support.e2ma.net/Resource_Center/Account_how-to/Content_split_testing  

And if you'd like, check with the admin on your account and reach out to talk about what it would look like to move your account over to the new Emma Plus platform.  

Let me know if you have any questions!  
Photo of UC Davis Stores

UC Davis Stores

  • 2 Posts
  • 0 Reply Likes
Thank you for the information. We will definitely look into moving over to the new platform.