(This post was originally written as a reply on the mailop mailing-list, but a friend asked me to turn it into a blog post. I’ve edited it, mostly adding more links to elsewhere, but there are some additions here.)

Context: someone with a mail-server hosted in a German facility with a poor reputation for handling abuse reports was asking for help on sending email to their Gmail-using friends; they had SPF and didn’t see the point of DKIM; they had TLS setup for their mail-server, using a certificate from CACert.

There’s a PDF from Google from 2006 which is still worth reading: https://research.google.com/pubs/archive/45.pdf, entitled “Sender Reputation in a Large Webmail Service” by Bradley Taylor. Anyone running a mail-server today needs to read that document.

If you don’t send much email, then the only IP-based reputation which Google can assess you on is the reputation of your address-block, so being in a “troublesome” hosting provider will score heavily against you. At that point, if not moving away, you need to try to balance out that negative score with enough positives that any of the large providers using reputation scoring will accept the mail.

Working forward-and-reverse paired DNS is even more important for IPv6 than for IPv4; for better or worse, some of the large providers have decided that exemptions in old standards for old behavior should not apply when folks deploy standards which are far newer. So you absolutely need an MX record, you must not just rely upon address-records (A and AAAA).

With a poor IP-based reputation, you need to see if you can score a better domain-based reputation. This is where DKIM comes into play: once you can provably link a message to really be from a given domain, then even if you don’t send much mail you can benefit from stuff like “not on day-old-bread domain-lists”. But having DKIM and then a DMARC record does help (and I’m no fan of DMARC).

For the mail-server’s TLS: for that to count in your favor instead of being a wash, I strongly suspect that it needs to be a certificate which senders can verify. For those people scoring up for “better TLS”, those senders using DANE will be happy with a TLSA record in DNSSEC for your CACert anchor. But the large webmail providers are Resistant to having to deploy DNSSEC verification, so instead have pushed out an alternative called MTA-STS. With MTA-STS, you’re tied into “whichever subset of CAs all the large senders you care about will trust”, and then using that CA for the certificates both for the MTA-STS web-server and for your mail-server. Note that you don’t need to implement the client logic for MTA-STS (and I think it’s antithetical to an open federated platform) but do need to just publish the static information for those senders who do use it. At that point, CACert is not going to cut it. You’d need to try Let’s Encrypt instead.

The ongoing natural tendency from larger providers is to favor supporting what the majority of their users want the majority of the time. With so many people using larger providers, they naturally tilt towards stuff which works with the larger senders, and requiring more hoops. Those additional hoops create more work for smaller providers and self-hosters doing thing manually.

We need better automation tools around all of this. The below will make it clearer why.

So, here is my current understanding of the best current practices here, in reality not IETF idealism. This includes making mandatory stuff which some folks insist must be optional, because realistically to send to some large providers it’s not optional. This list includes features to make you compatible with ongoing trends in the EU (particularly Germany) to strongly disfavor allowing clear-text SMTP.

This is broad best practices, targeting more than just Gmail. Eg, DANE/DNSSEC will be ignored by the large webmail providers, but will matter with some other providers. [clarification added later]

This assumes that you are not a large sender who should also be setting up feedback loops, learning how to “warm” IPs, considering BIMI, postmaster tooling domain verification, etc.

Deliverability fixes

1. reverse DNS with matching forward DNS; the name used should not pattern-match anything generic and ideally would include a DNS label of mail or mx or the like in it.
2. MX record, always.
3. Accurate SPF;
   • ideally not too broad; pay attention to SPF’s query limits.
   • avoid -all at the end because, with the sole exception of “this domain never sends email” records, the larger operators have metrics to show that using -all tends to be a sign of over-enthusiasm rather than reality, so it will slightly count against you;
   • remember to have an SPF record for your HELO hostname, because when you send a “bounce” rejection, this is the thing which will be looked up (since there’s no domain in <>).
4. DKIM set up, with thought towards the selector namespace.
   • RSA2048 key is effectively a hard-requirement
     • DNS TXT records consist of one or more DNS strings, each of which is limited to 255 ASCII characters. For a key of this size, you will end up needing two DNS strings in the zonefile.
   • Ed25519 keys are not yet widely supported, but by now are not likely to actively break and make things worse for you, if you dual-sign. This needs to be a different selector.
   • Note that for various good reasons you should design this to be something you routinely rotate.
     • Some folks use yearly, some monthly
     • I rotate every three months.
5. DMARC record; see RFC 7489
   • But for domains which humans send from don’t use p=quarantine or p=reject;
   • Do consider setting up a receiver for reports, just so that you can see how much of a privacy breach DMARC reporting is when you send to mailing-lists which don’t re-sign. :-/
6. TLS certificate from a CA in the main trust anchor bundles;
   • Just use Let’s Encrypt.
7. MTA-STS web-server with HTTPS certificate from the same CA, and the relevant MTA-STS txt file in place; add the DNS record when it’s up and happy. See RFC 8461.
   • See https://esmtp.email/tools/mta-sts/ for a testing validation tool (with thanks to Luis Muñoz for the pointer).
8. For the independent mail providers using the stuff broadly supported in open source MTAs, you should look at DNSSEC, because the patterns here are less susceptible to rent-seeking pressures:
   • DNSSEC-signed zone for your own domain
     • Use whichever signing algorithm CloudFlare are currently using: this should be both current for cryptography and widely enough supported that if it’s not supported by someone’s resolver, then they have bigger problems than just your domain.
   • DNSSEC validating resolver for you to look up records of others (consider Unbound or Knot Resolver)
   • DANE records for your own domain (TLSA records in DNS)
     • See RFC 7672 for the SMTP details.
     • See RFC 6698 for the base spec with RFC 7218 for some common acronyms which make talking about it easier.
     • See RFC 7671 for the updates and operational guidance.
     • There are other RFCs, for SRV records and for OpenPGP, etc.
   • Tell your mail-server to obey DANE stuff, so that if there’s a TLSA record in DNSSEC-verified DNS then the mail-server can disable fallback to cleartext for delivery to MX (and ideally also then verify the TLS connection has a cert chain which is anchored in one of the TLSA records)
   • https://dnsviz.net is your friend
9. _smtp._tls record so you can get reports of TLS failures sending to you
10. Seeing if you can get your IP onto one of the open DNS-based allow-lists (also called “whitelists” but some folks are moving away from that term), such as https://www.dnswl.org/ or Spamhaus’s SWL.
11. Periodically check if you appear in any DNS-based deny-lists.
12. Make sure you’re not sending from “ISP residential address-space”; if need be route your mail outbound via a host in better address-space (and update SPF etc to match)
13. Don’t do sender call-out verification to SMTP servers which aren’t yours.
14. For your own sanity, do make sure you set up fail2ban, or something like it, scanning your mail-server logs, because SMTP AUTH online cracking is widespread. If they ever get in, your deliverability will be negatively impacted by their spam campaign through your mail-server.

Convenience Stuff

Outside of “Phil’s BCP” above, additional non-deliverability but convenience options include:

1. DNS SRV records for submission(s)/imap(s)/pop3(s)/sieve, even if just to say with «0 0 0 .» that it’s not supported.
2. If your communications base includes people using OpenPGP with email, then set up WKD to publish OpenPGP keys for your domain too.
   • This is just a fixed schema for laying out keys for HTTPS retrieval.
   • See the WKD draft for details.
   • I wrote https://github.com/PennockTech/openpgpkey-control as a management framework for an organization; the other/standalone-update-website script is designed to be embeddable into an existing site-building workflow without anything else from the repository.
   • The GnuPG project has tooling available which manages the WKD layout as an email-integrated workflow, for people to update their own keys.
3. If your communications base includes people using S/MIME then set up SMIMEA records in your DNSSEC-signed DNS.
   • They look a lot like TLSA records; both are trust anchors in DNS.
   • See RFC 8162 for details.
4. The moment you start specifying “must be TLS-secured” it’s worth adding CAA records into DNS, so that Certificate Authorities which are broadly trusted will refuse to issue for your domain unless you list them.
   • See RFC 8659 for details
   • The values checked by each Certificate Authority as indicating they have permission are required to be listed in their Certification Practice Statement, as part of the CA/Browser forum’s Baseline Requirements. If it’s missing, then browsers are not supposed to be trusting that CA.
   • For domain-validation CAs such as Let’s Encrypt, consider adding account information to those records to tie it to your specific account. See RFC 8657 for details.
   • Beware that at time of writing, Let’s Encrypt only honors the accounturi restriction in their staging environment, not their production setup; this will likely change.
   • Remember that DNS zonefiles support comments. You’ll want them here!
5. https://wiki.libravatar.org/api/ for Gravatar-style images in some MUAs (with caching for privacy reasons); this is a fixed schema HTTPS layout and a DNS record. I think almost nobody uses this but I don’t have usage figures. If you have a Claws user, perhaps?

A lot of the DNS items above can be found in a sample DNS zonefile contributed to the DNSControl project for their regression test suite, after my production zone broke their converters. DNS hosting platforms which try to over-simplify will make some of the above hard.

I’ve written this as a reply so it’s the stuff I can remember / spot now, so I might be missing something big or “too obvious to me”. Other tech exists.

Addenda

1. A second reason to run a local DNS resolver is that if you are checking any DNS-based access lists then you should not do so via public resolvers. All mail-server clusters or stand-alone systems really should have a dedicated DNS resolver for their use. Since you need the resolver to exist anyway, you might as well make it a validating resolver and get DNSSEC verification too, per the above. [2020-08-19]