If you send any kind of bulk email—newsletters, receipts, onboarding, whatever—you know how quickly your deliverability can tank. One week, everything lands in the inbox. The next, Gmail’s playing goalie and nobody sees your stuff. If you’re tired of guessing, it’s time to start automatic email testing. This guide breaks down how to set up ongoing deliverability monitoring using Mail-tester, a dead-simple tool that tells you what’s getting flagged and why.
If you want to catch issues before your boss or your customers do, read on. I’ll show you how to automate the whole thing so you don’t have to remember to check every week.
Why bother with automatic email testing?
Email deliverability is a moving target. Warming up a new domain, changing your sending volume, a random DNS hiccup, or a single spam complaint—any of these can flip a switch and send your emails to the spam folder. You don’t want to be the last to know.
Manual testing works, but who has time to check every day? Automating the process means you’ll spot problems early, and you’ll have a track record to show what changed.
What Mail-tester is (and isn’t)
Mail-tester gives you a throwaway address. You send an email, and it spits out a score along with a breakdown: spammy content, missing DNS records, blocklist issues, and so on. It’s not a full-service deliverability suite (think Postmark, GlockApps, or SendForensics), but if you want a no-frills, accurate heads-up about what might be wrong, it does the job.
Mail-tester’s big downside: it’s meant for manual use. But with a little ingenuity and some scripting, you can automate it.
Step 1: Understand what you’re actually automating
Before you set anything up, let’s clarify what’s possible with Mail-tester:
- You get a unique email address each time you open the Mail-tester page. This address is valid for a limited period (about 24 hours).
- You send your test email to that address. Mail-tester analyzes it and gives you a report at a unique URL.
- Mail-tester doesn’t offer an official API. So, true “automation” is a bit of a hack.
If you want to completely automate things, you have two options:
- Manual initial setup, then recurring sending: Get a permanent Mail-tester address (paid feature), or generate a new one daily and update your system to send emails there.
- Semi-automated monitoring: Use a script or email automation tool to send your test message to your Mail-tester address on a schedule, then check the report yourself.
Pro tip: If you’re serious about automation and need API access, consider alternatives. But for most use cases, Mail-tester plus a little scripting is good enough.
Step 2: Get a permanent Mail-tester address (if possible)
If you’re using Mail-tester’s free version, you’ll need to refresh the test address frequently. That’s a pain to automate. Mail-tester offers a paid plan that gives you a permanent address. If you can swing the small fee, do it. Here’s how:
- Sign up for a Mail-tester account (they call it “Mail-tester pro”).
- Generate your permanent test address. It’ll look something like
yourname@yourdomain.mail-tester.com. - Send your test emails to this address any time—no need to update addresses every day.
If you can’t or don’t want to pay, skip to the next section for a workaround.
Step 3: Set up your automated test email
Whether you’re using a permanent or temporary address, you need your system to send a real copy of your email to Mail-tester on a schedule. How you do this depends on your setup:
For ESPs (like Mailchimp, SendGrid, etc.)
- Add your Mail-tester address as a subscriber (use a unique segment or list so you don’t mess up your stats).
- Set up a recurring campaign that sends your template to this address. Most ESPs let you automate this or schedule repeated sends.
- Exclude the address from normal sends so you’re not double-counting opens, clicks, or bounces.
For transactional email (your own app, SMTP, etc.)
- Add a nightly/weekly script that sends your standard email (or a sample) to your Mail-tester address.
- Use a cron job, scheduled Lambda, or whatever fits your stack.
- Make sure it uses the same headers, DKIM signature, and sending domain as your normal traffic.
What to send?
- Use your actual templates and sending infrastructure. Don’t just send a plain text “hello”—you want to catch real-world issues.
- If you A/B test or have multiple templates, rotate through them or test the riskiest ones.
Step 4: Automate report retrieval (if you want to get fancy)
This is where things get a bit hacky. Mail-tester doesn’t have an official API, so you can’t easily pull scores and reports programmatically. Here are your options:
Option 1: Manual check (simple, reliable)
- Each time your scheduled test sends, visit your Mail-tester report URL.
- Bookmark it, or set a calendar reminder.
- Look for big score drops, red flags, or new issues.
Option 2: Scrape the results (advanced, brittle)
- Use a script (Python, Node, etc.) to fetch the Mail-tester report page.
- Parse the HTML for your score and the major sections.
- Alert yourself (email, Slack, etc.) if the score drops or new issues appear.
Gotchas: - Mail-tester’s layout and URLs can change at any time. Your script might break. - Be kind—don’t hammer their servers with bots.
Option 3: Email-based alerts
- Some paid ESPs and monitoring tools can forward Mail-tester’s results to you automatically.
- Not all setups support this, but if yours does, it’s the least maintenance.
Step 5: Set up alerts and monitoring
Deliverability issues are only a problem if you miss them. Here’s how to stay on top:
- Set up an alert for yourself (email, SMS, whatever you’ll actually read) when your Mail-tester score drops below your comfort zone (e.g., below 8/10).
- If you’re scraping, have your script do this automatically.
- For manual checks, block five minutes on your calendar every Monday to review the report.
Pro tip: Don’t panic over every little point lost. Focus on trends—if your score tanks suddenly, investigate.
Step 6: Troubleshoot and act on what you find
Mail-tester’s report is pretty readable, but here’s what actually matters:
- SpamAssassin score: The heart of the report. High numbers (above 5) are bad news.
- DNS checks (SPF, DKIM, DMARC): These should always pass. If they’re failing, fix your records ASAP.
- Blacklist status: If you’re on a blocklist, address it. Sometimes it’s not a big deal (some lists are junk), but major blocklists are worth worrying about.
- Broken links, images, or HTML: Clean these up. Ugly code can trigger filters.
- Spammy content: Mail-tester will flag phrases like “FREE!!!” and “Act now.” Don’t obsess, but if you see big warnings, rewrite.
What to ignore:
- Minor “marketing language” warnings if your main score is high.
- Tiny HTML/CSS nitpicks—unless you see a huge score drop.
- Green checkmarks you’ve already fixed (don’t chase ghosts).
Pro tips for reliable deliverability monitoring
- Test from your real environment. Don’t run tests from localhost or staging—spam filters care about where you’re really sending from.
- Keep a log of scores over time. A simple spreadsheet is enough. If you see trends, dig deeper.
- Rotate test content occasionally. Filters can react differently to new links, images, or wording.
- Don’t overcomplicate it. If you’re spending hours scripting around Mail-tester, consider if a paid monitoring tool is worth your time.
When to look elsewhere
Mail-tester is great for basic, ongoing checks. But it’s not magic, and it’s not a full replacement for comprehensive inbox placement testing. If you’re running at massive scale, or you need to test across dozens of mailbox providers and geographies, look at tools like:
- GlockApps
- Postmark’s monitoring
- SendForensics
- 250ok (now Validity)
These cost more, but they’re built for bigger, more complex setups.
Wrapping up
Automatic deliverability testing doesn’t have to be a giant project. Mail-tester, plus a little scripting and discipline, is enough for most small teams. Set it up, check your scores, fix issues when they pop up, and move on. Don’t chase perfection—just catch the big stuff before it turns into a support headache.
Keep it simple, iterate as you go, and don’t let deliverability drama sneak up on you.