scraping server

  • 1
  • Problem
  • Updated 5 years ago
  • Not a Problem
Archived and Closed

This conversation is no longer open for comments or replies and is no longer visible to community members. The community moderator provided the following reason for archiving: Resolved

Our server was taken down today and we identified the issue was with our daily email newsletter that goes out from Emma. With the launch of our new website, old links from the header/footer were removed that are longer present. These links where content scraping the server looking for the content. This initiated a spike in the server.

Link EXAMPLES that hit the server

 32 "http://t.e2ma.net/webview/3ndjj/c69c722b5b7ea940524a749669a88889"
     38 "http://t.e2ma.net/webview/fx0kj/27edf21f7a466b1905256ceada966938"
     24 "http://t.e2ma.net/webview/ryhkj/b6e2df2b938cb0c51d8050805c6900d6"
     24 "http://t.e2ma.net/webview/vlmmj/7bce0b5fed406bd8c15890810165cbe2"

Any idea why this is happening? I can create redirects for the images that were previously included in the template, but it seems strange that this would be causing such a spike considering they are months old. Please let us know what you can. Thank you!

Photo of MJ Freeway

MJ Freeway

  • 6 Posts
  • 0 Reply Likes

Posted 5 years ago

  • 1
Photo of Paul Zimmerman-Clayton

Paul Zimmerman-Clayton, Official Rep

  • 400 Posts
  • 34 Reply Likes
Hey there! You've got me stumped on this one, but I'm going to check in with our systems team to see if anyone around the house knows a little bit more. I'll check back in soon!
Photo of Paul Zimmerman-Clayton

Paul Zimmerman-Clayton, Official Rep

  • 400 Posts
  • 34 Reply Likes
Hey again! I've chatted with a few people around the house, and we have a little more information that might be helpful. 

Our best guess as to the cause of the spikes is the use of an image path in one of your templates that resides on your site. Normally, we require images to be uploaded to the image library for use in templates, but somehow it looks like an path to an image on your site was placed in a hardcoded, editable text block in the template called "Daily Dose Aug 2014." The online version of that mailing, and the presence of the template in your template gallery might be part of the cause of those spikes, but from what I've gathered, it might not be the entire story. 

The solution may be redirects, or perhaps adding that image back in on the page. Additionally, you might try deleting that template from your account. 

I hope this helps! Please let me know if you have any more questions!
Photo of Marc Powell

Marc Powell, Official Rep

  • 6 Posts
  • 0 Reply Likes
Additionally, since those images have been used in a sent campaign, you'll continue to see requests for them as long as people still have them in their mail clients or view the public or private online versions of them. It's also possible that they've been indexed by the normal search engines if someone has posted the campaigns in a public location such as Twitter, Facebook, their own web pages or linked to the online web versions from any of those types of places. The search engines will regularly re-index the content on your site based on those links.