Possible to send split test without completely saved subject

  • 4
  • Problem
  • Updated 4 years ago
  • Solved
  • (Edited)
Archived and Closed

This conversation is no longer open for comments or replies and is no longer visible to community members. The community moderator provided the following reason for archiving: Resolved!

When sending a split test, I entered all three subject lines then went through to send. My email address (as part of an internal distribution group) received the email but the subject line was reduced to just "h."

It would appear that one of the three variations that went out to the split test pool was indeed just "h." Any idea what would cause this? Do I need to not use split test in case this happens again?

On Windows 8.1, Chrome 40.
Photo of Blood Assurance

Blood Assurance

  • 16 Posts
  • 7 Reply Likes
  • less than thrilled.

Posted 5 years ago

  • 4
Photo of Emily Hollingsworth

Emily Hollingsworth, Alum

  • 2203 Posts
  • 158 Reply Likes
Hi there -- thanks for letting us know and for including those details! This definitely isn't expected behavior for the split test feature. In order to dig into what may have happened here, I'd love to learn a little bit more about your workflow. 

I can see that you send split tests quite often. Is it possible when you're building and sending those split tests, that multiple people are working in the account at the same time? Or, do you often work with the account open in multiple tabs? Do you remember when you last cleared the cache in your Chrome browser?

I'm just trying to rule out some things here. Thanks!
Photo of Blood Assurance

Blood Assurance

  • 16 Posts
  • 7 Reply Likes
Greetings. It actually happened again today. I'm the only person in the account and occasionally I do keep open multiple tabs, but it's almost always Audience in one tab and Campaign in another. I clear my cache about once a week.

I sent one today and what it appears to have done is, the third subject line was one I changed a couple of times. One time it started with the word "there's" and I deleted that and wrote a different subject line. I hit send and saw the three options as listed below.

Photo of Berckemeyer Consulting Group

Berckemeyer Consulting Group

  • 1 Post
  • 0 Reply Likes
The same thing just happened to me today.  I did have multiple windows opened in Chrome.  One window was viewing the document I was intending to send, the other was setting up the actual email.  So is this what messed mine up as well?  This is my first time trying the split test.  Now I'm afraid to do it again.  Bill
Photo of Emily Hollingsworth

Emily Hollingsworth, Alum

  • 2203 Posts
  • 158 Reply Likes
Hey guys -- Just checking in here. I wanted to let you know that I'm looping in our tier two support team to really dig in here. I'll post an update here as soon as I have more information! Thanks for your patience. :)
Photo of Emily Hollingsworth

Emily Hollingsworth, Alum

  • 2203 Posts
  • 158 Reply Likes
Hey there -- The latest on this issue is that our team has flagged this as a bug. In our extensive testing, we haven't been able to replicate the issue or pinpoint the exact browser conditions (if any) that might be causing it. 

We'll continue to dig in and hopefully discover more about this bug. If you run into this again, please take note of these things for us:
  • browser, version 
  • mailing id 
  • how the subject was added (typed, pasted, etc. if pasted, how) 
  • if the mailing was scheduled or "send now" 
  • if the mailing was used in a previous split test? 
  • how you navigated to the send page (for example, did you ever use the back button, use multiple tabs or windows? move from preview to review and send, etc?)
As always, you'll want to avoid having Emma open in multiple tabs or windows and steer clear of using the back button at all when sending. 

Thanks for your patience. I'll reach out here to let you know when this bug is resolved!
(Edited)
Photo of Kati Hyer

Kati Hyer

  • 47 Posts
  • 11 Reply Likes
This reply was created from a merged topic originally titled Is it possible to send a campaign with a blank subject line, even after split tes....

Is it possible to do a split test with a subject line and a blank subject line? I'm asking because I believe a colleague did a split test, and the email came in with a blank email subject line. (Yark!) Was this an error on their part or a glitch? Thanks! 
Photo of Emily Hollingsworth

Emily Hollingsworth, Alum

  • 2203 Posts
  • 158 Reply Likes
Hi Kati -- We moved your question over here because it sounds like you might be running into this pesky bug that we've been discussing here. It's a bit unpredictable, and we haven't been able to nail down exactly why it's happening.

You can click to follow this conversation for updates!
Photo of Gresham Hill Management, Inc.

Gresham Hill Management, Inc.

  • 6 Posts
  • 0 Reply Likes
So this is the 2nd time that I created a split test and only 1 of the 3 subject lines I created was saved.  Both times that this messed up, only one letter was saved in the second subject line.  The first time it was "g" and the 2nd time it was "a".  So thousands of my followers got an email with the subject line "a" today.  That's pretty frustrating.  The good news is that these one letter subject lines keep producing the biggest open rates.  But they also produce the biggest opt out rates.  I'm very frustrated and would like to know when this can be fixed.
Photo of Emily Hollingsworth

Emily Hollingsworth, Alum

  • 2203 Posts
  • 158 Reply Likes
Hi Gresham Hill -- I hear your frustration here and hope you know that we take this issue very seriously. Our team is still working hard to pinpoint exactly why this bug is coming into play and how to prevent it. I will reach out here *as soon* as I have any updates or a resolution for you!

Since this bug is pretty unpredictable, I might avoid using the subject split testing feature for now. I know that's not an ideal workaround, but at least you will avoid sending out cut off subject lines while we work to resolve the issue.
Photo of Gresham Hill Management, Inc.

Gresham Hill Management, Inc.

  • 6 Posts
  • 0 Reply Likes
If this helps... I know that I started off with one subject and may have toggled between the split vs. the one line.  But at the end, I definitely entered 3 subjects and then filled out the rest and his sent.  It must be when I was indecisive in the beginning stages.
Photo of Emily Hollingsworth

Emily Hollingsworth, Alum

  • 2203 Posts
  • 158 Reply Likes
Any additional information here is helpful, so thank you for passing that along! I'll let our team know to see if they can replicate it!
Photo of Blood Assurance

Blood Assurance

  • 16 Posts
  • 7 Reply Likes
Just updating you on my end. It happened again today. This time before sending, I clicked save again and again and again to make sure it didn't replicate. After clicking save, waiting for it to say "saved" and doing that a couple more times, I sent. Only to end with this.
Photo of Emily Hollingsworth

Emily Hollingsworth, Alum

  • 2203 Posts
  • 158 Reply Likes
Hi there -- Thank you for including so much detail here. Every tid bit is helpful in getting us closer to isolating and resolving this issue!

You mentioned in an earlier reply here that the subject line affected was one you had changed a few times before saving. I'm just curious if that was the case here as well or if it's unrelated. I haven't heard that specifically from any other users, but it would be great to factor that in to our testing!
Photo of Gresham Hill Management, Inc.

Gresham Hill Management, Inc.

  • 6 Posts
  • 0 Reply Likes
Yes.  It's possible.  However, the email I sent today also had subject issues on the split test.  This one somehow saved the full first Subject and 1/2 of the 2nd one and none of the 3rd.  Even weirder was that I had gone back and edited the 3 subject lines before I finally sent it but it somehow did not get those edits and sent 2 subject lines that I really didn't want it to send.  All in all, I think this is damaging our company's reputation so we're going to stop using the Split Test until you guys can figure this out.
Photo of Gresham Hill Management, Inc.

Gresham Hill Management, Inc.

  • 6 Posts
  • 0 Reply Likes
Yep.  I'm with you.  This is what mine does!
Photo of Emily Hollingsworth

Emily Hollingsworth, Alum

  • 2203 Posts
  • 158 Reply Likes
Hi Gresham -- I hear you. I'm sorry that this bug has affected you so strongly. We are working hard to isolate what might be causing the issue, and your feedback here is super helpful!

Hopefully we will have this handy feature back up and running soon. 
Photo of Blood Assurance

Blood Assurance

  • 16 Posts
  • 7 Reply Likes
Hey Emily, still having the same issue. We sent an email out last week with the subject line of "P" - any chance your tech team can disable this feature until it is working properly for all clients? No sense in having a broken option on the board. Thanks!
Photo of Emily Hollingsworth

Emily Hollingsworth, Alum

  • 2203 Posts
  • 158 Reply Likes
The latest update on this issue....

Our development team has this issue in their active sprint, which kicked off yesterday, and we're working to get a fix in place for it as quickly as possible. We've found this glitch to be especially random, so you may want to shy away from using split testing until I circle back to let you know we have it resolved. If you're set on trying split testing in the mean time, you may want to set that up to manually to pick the winning subject line, rather than Emma automatically sending the one with the best open rate at the end of four hours. That way if it happens on another of your split tests, you can be sure to send the full subject line to the remaining 80%.

I've got you all added to my list of customers who have experienced this particularly frustrating issue, and I'll post an update here as soon as we have it resolved!
Photo of DC Building

DC Building

  • 1 Post
  • 0 Reply Likes
We just had this exact same thing happen about ten minutes ago. I set up three different subject lines to test, but the only one that went out is the subject line 'T.' This mailing was the release of a digital catalog that was the result of thousands of dollars of development. Can someone at least intervene to ensure the remaining 80% receive a coherent email?
Photo of Emily Hollingsworth

Emily Hollingsworth, Alum

  • 2203 Posts
  • 158 Reply Likes
Great news! Our development team has been able to zero in on the exact steps process-wise that lead to this trouble. It's related to the specific timing on when autosave runs during the split test setup on the review and send screen. Now that we've been able to replicate the exact issue, we're optimistic that we'll have a fix in place very soon. 
Photo of Emily Hollingsworth

Emily Hollingsworth, Alum

  • 2203 Posts
  • 158 Reply Likes
Official Response
Hey there -- I have great news. Our development team just released a fix for this bug! We're doing a little celebration dance here in the office! Thank you again for your patience and understanding as we worked to resolve this issue.
Photo of Blood Assurance

Blood Assurance

  • 16 Posts
  • 7 Reply Likes
Alright, gonna give it a try, but if it doesn't work, you're going to owe me a banana split. (Get it? Split? Hilarious.)