Monitoring and Exception Reporting
Boom! Your infrastructure is set up and emails should flow smoothly to the ISPs. But how do you know? It’s important to have good visibility into your deliverability. Here are some products and tools which can help:
One critical monitor is the use of seed lists, which include email addresses set up at the major ISPs and used solely for the purpose of seeing whether emails land in the inbox, bulk, or disappear into the ether. If sending from one dedicated IP, simply place a seed list into your subscriber list. If using several IPs, randomly place the seed lists onto email and watch inbox placement in some automated fashion. Don’t move, open, or touch emails when they arrive, if they arrive, because ISPs use predictive algorithms to deliver future emails based on past engagement. Remember not to use the seed lists too often, as they can have a negative effect on list performance because those emails are not opened or responded to.
Also set up monitoring to determine how many emails bounce, how many recipients open and click the email, how many subscribers unsubscribe, and how much FBL activity occurs per IP. ISPs and email administrators keep a close eye on this information, and if you see high abuse complaints, unsubscribes, or bounces then the ISPs know something is wrong with your list-collection techniques. If experiencing low opens and clicks, it could be due to poor inbox placement, unengaging content, or lack of proper segmentation.
Scraping MTA Logs
Set up some form of monitoring on your MTA logs. Most MTAs will have ways of handling exceptions, but they won’t be able to handle all exceptions. For instance, some ISPs will report back if your IP is on a blacklist. That’s something you want to be aware of in real time. You should set up some scripts to actively monitor your IPs and domains to check the major and minor blacklists. But you also want to actively mine this data in your MTA logs. Some ISPs and blacklists don’t have a way to look up your IP’s status, so by scanning logs you can catch the exceptions which might occur. There are other scenarios where you want to scrape MTA logs for spam trap addresses, fatal errors, etc. The MTAs are full of useful information, and being able to search the logs will prove handy when troubleshooting problems,
Static/Dynamic Error Handling
We touched briefly on using your MTA to handle certain errors and scraping your MTA logs to find out when errors occur—but there are some errors which require different or additional actions.
The first type is a static error. An ISP, such as Comcast, might throw a static error which looks something like this:
Comcast block for spam. Please see http://help.comcast.net/content/faq/BL000000
If you weren’t scraping logs or using your MTA’s error handling, you’d never know this error took place. So we recommend using both the MTA error handling and log scraping to look for errors and send alerts. In this instance, the MTA should back off sending to allow enough time to unblock the IP with Comcast. You can even switch all Comcast traffic to another MTA, which the MTAs should be able to handle.
The next step would be to find out what’s causing the issue. Having searchable logs is key here. Find out the who, what, where, and when of the incident. Fix the issue. If it's a specific sender, stop their sending. If it's a specific recipient, block them from being sent any emails. Then manually fill out the Comcast unblock form. After you receive an unblock notification, turn the traffic back on if the problem will not persist. Failure to fix the issue will cause emails to bounce continually until the underlying problem is corrected.
Other errors are dynamic errors. Some ISPs will throw a dynamic error which requires slowing down or completely stoping sends for a few hours. Configure your MTA and log scraping to send an alert so your infrastructure can respond properly. If dynamic errors continue for an IP, investigate the cause and remediate. That might involve speaking with the ISP to find out the issue, but investigate thoroughly first. There are tons of codes and resolutions, and ISPs add new ones each year. Work with your MTA vendor and development team to build an application and infrastructure that’s fully aware of these errors and can handle them cleanly.
There are several monitors and tools which will show if you’re blacklisted. Unfortunately, no, they don’t include all of the blacklists, and most monitors will not be “real-time” enough for your needs.
Here’s a list of some major and minor blacklists:
AHBL, ANT, Backscatter.org, BARRACUDA, BURNT-TECH, CASA-CBL, CASA-CBL+, CASA-CDL, CBL, CYBERLOGIC, DEADBEEF, DNSBLINFO, DULRU, EMAILBASURA, FABELSOURCES, FIVETEN, GIRL, GRIP, HIL, HIL, HILLI, ICMFORBIDDEN, IMP-SPAM, IMP-WORM, INTERSIL, ivmSIP, ivmSIP/24, KEMPTBL, KUNDENSERVER, LASHBACK, LNSGBLOCK, LNSGBULK, LNSGDUL, LNSGMULTI, LNSGOR, LNSGSRC, MSRBL-Combined, MSRBL-Images, MSRBL-Phising, MSRBL-Spam, MSRBL-Viruses, NERD, NETHERRELAYS, NETHERUNSURE, NIXSPAM, NJABL, NJABLDUL, NJABLFORMMAIL, NJABLMULTI, NJABLPROXIES, NJABLSOURCES, NLKUNBLACKLIST, NLKUNWHITELIST, NOFALSEPOSITIVE, NOMOREFUNN, ORID, OSPAM, PDL, PSBL, RANGERSBL, RATS-Dyna, RATS-NoPtr, RATS-Spam, REDHAWK, RRBL, SCHULTE, SDERB, SENDERBASE, SERVICESNET, SOLID, SORBS-BLOCK, SORBS-DUHL, SORBS-HTTP, SORBS-MISC, SORBS-SMTP, SORBS-SOCKS, SORBS-SPAM, SORBS-WEB, SORBS-ZOMBIE, SPAMCANNIBAL, SPAMCOP, Spamhaus-ZEN, SPAMSOURCES, SPEWS1, SPEWS2, SWINOG, TECHNOVISION, TRIUMF, UCEPROTECTL1, UCEPROTECTL2, UCEPROTECTL3, VIRBL, WPBL, WSFF, ZONEEDIT, CSMA, DUINV, ORVEDB, RSBL, SPAMRBL
And that’s not even all of them!
People can run blacklists out of their mom’s basement, and corporations can have their own blacklist. Also keep in mind your IPs and domains can be blacklisted, redlisted, or graylisted. Red listed senders are related to senders currently in the blacklist, and gray listed senders are generally associated with email marketing. It's usually not possible to delist IPs and domains from red and gray lists, but most ISPs won't block senders on red or gray lists unless there are other extenuating circumstances, like content blocks.
It’s pretty easy to set up monitoring with most blacklists, and there are many scripts publicly available on the web to help. Additionally, Spamhaus DBL is a realtime queriable database of domains found in spam messages. Some blacklists require registering IPs in order to use their lookups, and failure to do so could get the IP you’re checking from listed as well.
Spam Filter Monitoring
Spam filters are used by most major and minor ISPs to locate spam based on email content. If they're not using Spam Assassin it’s probably a commercial-grade spam filter. These filters will catch most, if not all, spam, but sometimes they can be aggressive. Check your content, and test as much as possible.
Return Path and Litmus offer scanning through a few major spam filters, but you can also do some scanning internally prior to sending campaigns. It’s easy to install Spam Assassin on a test box and run content through it. Run the content through other spam filters and test accounts with the major ISPs as well, just to cover your bases. The difficult part is weeding out false positives and ensuring good content isn’t flagged as spam. Try tweaking the Spam Assassin rules to suit your needs. Just keep in mind some people run default installs of Spam Assassin while others run custom rules.
If you can’t get your hands on the filters used by major ISPs, it’s at least good to know the products used. Return Path provides some information about spam filters used by major ISPs: Yahoo uses a proprietary spam filter called SpamGuard; Hotmail, MSN, and Live use BrightMail; Gmail uses Postini. Other commonly used filters at the enterprise level are MessageLabs, Barracuda, and Forefront. These filters would affect B2B email communication, so it’s important to keep them in mind.
Content fingerprinting is becoming widely used by ISPs and enterprise/corporate email administrators, as well. Cloudmark offers a high-quality product for detecting spam through fingerprinting.
Familiarize yourself with the spam filters used, and, if possible, get these filters into your infrastructure to ensure you’re not sending out content which will get blocked.
Return Path offers several products and tools which give insight into how good your delivery is (or isn’t.) They offer tools for reputation monitoring, seed list monitoring, blacklist monitoring, and previewing emails in over 30 email clients with spam filtering analysis.
Seed list and campaign monitoring allow you to attach a list of monitored addresses to emails. Return Path then collects the delivery stats from the major ISPs, determining whether the message ended up in the inbox, bulked or went missing.
Reputation monitor provides helpful data for determining IP issues, but it should just be one data point as not all major ISPs contribute.
Mail Monitor uses seed lists to provide deliverability monitoring and spam filter testing. You can see which of the major ISPs delivered, bulked, or blocked the emails. Additional information about how well emails fare against spam filters, such as Cloudmark, SpamAssassin, Barracuda, and Symantec.
The MTA can be set to automatically append seed lists to each email campaign, campaigns for specific users, or random campaigns. One advantage to seeding random campaigns is a good overview of how the entire infrastructure is delivering and not just specific users or IP addresses.
IBM Enterprise Marketing Management
IBM Enterprise Marketing Management recently purchased Unica and Pivotal Veracity, delivery monitoring tools with a good reputation. We can’t speak directly to them, but we know a few ESPs use them heavily.
The SDNS tool should be used as a secondary tool to troubleshoot issues and gather information. Register your IPs and SNDS provides information such as number of spam traps hit, abuse complaint ratio, and volume per IP. It's a great tool to check when a client is having delivery issues to Hotmail, MSN, Live.com, or Outlook.com.
It’s not me, it’s them.
Let’s say a big ISP is blocking your email. The first thing you do is email them about it, right? Nope!
Don’t email your buddies at AOL or talk to someone who knows someone. That’s not what deliverability is about. First, start with your infrastructure, your logs, your lists, your content, etc. Not them—you. Generally, most issues are on the sending side, and the issue stems from poor engagement, stale lists, or spammy content. If, and only if, you have checked and fixed everything on your end should you engage with an ISP.
Generally, getting blacklisted boils down to bad list etiquette. You can’t send commercial email because the recipients verbally told you it’s okay. You can’t send commercial email because you bought a list, rented a list, or a vendor said it was cool. There are all sorts of blacklists and different ways to land on them, but all the ways we know of start with bad list etiquette.
Chances are, you know you’re doing something wrong, or your client knows they’re doing it. Just do the right thing, unless you enjoy filling out forms and emailing people about how great your list is.
When you get blacklisted, find out as much about the incident as you can from the listing company. Usually, they’ll at least provide a date and subject line. If you’re scraping your MTA logs, you likely caught the incident when it occurred or just after it occurred and can work backwards—similar to working an ISP block. Figure out what caused the issue, fix it, and then, and only then, do you contact the listing company.
Corporate domains, small ISPs, and international ISPs can employ similar technology to the major ISPs. Corporate domains are sometimes more stringent on rate limiting and spam policies because their servers can't handle large volumes or they want to reduce workers' time deleting junk emails. If you’re getting blocked at Bigco’s domain or a small ISP, treat it just like a major ISP. Do some research, and read up on any public information prior to contacting them.
If marketing to government or large non-profit organizations, they'll likely be using a Barracuda device. We recommend getting one as well to check your content. In some cases, these organizations might reach out for different issues or policy violations. Generally, they’ll ask you to comply with their standards, and it’s important you comply as quickly as possible.
Some corporations and small businesses have such strict policies they might decide to blacklist you if anything looks suspicious. Good thing you set up all those email addresses so companies can alert you to potential issues and possible ways of resolving those issues.