SPF

• SPF (Sender Policy Framework) is a mechanism that identifies email servers that are authorized to send email from your domain.
  • Uses a TXT record^[4] on your DNS host/nameserver
  • Recipients perform a DNS lookup to confirm the sender.
  • If there is no DMARC record present (or the receiving server is not configured to check DMARC), then mail will be authorized or denied based on the SPF record policy.
• Receivers perform an SPF Lookup by comparing the incoming mail's sending server to the list of authenticated senders in the SPF record.^[5]
  • If the email comes from an authorized sender, the email will pass the SPF lookup.
    • DMARC would check for SPF Alignment by comparing the SMTP MAILFROM and the email's "From:" fields to make sure they match.
  • If the sender is not authenticated, then the email either fail or softfail the SPF lookup.
    • This means the email will rely on DKIM to pass DMARC and could result in undelivered mail.
      • SPF alignment often fails in cases where email is automatically forwarded (e.g., from a professional account to a personal account, or if the email is sent to a distribution list), so it is best to have both SPF and DKIM enabled where possible.
  • If the SPF record is missing, most receivers will treat is as a none or neutral result.
    • This means that the receiver's policies will play a bigger role in whether the email is delivered or not.
• Each sending mail server/domain must be identified
  • This can be directly via IP or through a domain/DNS lookup
    • SPF is limited to 10 DNS lookups, and is evaluated all at once.
      • Going over 10 causes a PermError.
    • TempError is often caused by a transient (e.g. temporary) DNS lookup error.
      • It can be caused by recipient policies or a DNS lookup timeout; there may not be much you can do, but you should still check your record to make sure it's well below 10 look ups.^[6]
    • Once the SPF record is validated, the receiving email server matches the sending email server's address against the SPF record, stopping at the first match and corresponding action (pass, softfail, etc.)
• Use subdomains for third-party services where possible.
  • Configuring a unique subdomain (like newsletter.example.com) for email marketing services can help you stay below the 10 lookup limit with a separate SPF record isolates reputation harm from from misconfigured vendors.
  • It is not recommended to add marketing services, like Mailchimp or Sendgrid, to your main SPF record
    • Marketers use tons of servers to send mail to get around spam filters, and because of this, SPF DNS lookups often reach their limit before getting to the root IP and fail authentication.
• SPF Authorizes a list of approved senders and provides weak^[7] Authentication by comparing the sending IP against the list of approved senders.

DKIM

• DKIM (Domain Keys Identified Mail) is a mechanism that authenticates all messages sent from a domain using asymmetric key cryptography.
  • DKIM uses two keys; a private key used for signing, and a public key for verification
  • The administrator publishes the public key to the DNS record
  • The sending server signs all out-bound messages by using the private key to generate a hash from the message headers and body.
  • Recipients verify the signed email against the public key
• The DKIM key is shared via a TXT record
  • DKIM records are differentiated with key selectors, and each email server requires its own public/private key pair.
    • You may have multiple DKIM records with different key selectors.
    • Because authentication occurs by key pair, tools like MxToolBox often require the selector to test and validate the correct key.
  • Can also be configured as a CNAME record^[8] which points to the actual DKIM TXT record.
    • CNAMEs are often used by service providers to allow them to manage and regularly cycle keys without bothering clients.
  • When rotating keys, you’d publish selector1._domainkey and selector2._domainkey simultaneously, switch signing to the new key, then remove the old record after clients have refreshed.
• Only servers with access to the private key are able to generate a valid signature, and proves the message was sent from an authorized sender.^[9]

DMARC

• DMARC (Domain-based Message Authentication, Reporting, and Conformance) is a mechanism that tells a receiving email server what to do based on the SPF and DKIM authentication checks.
  • DMARC checks whether the domain passes the SPF and DKIM authentication checks and that the email's "From" field is aligned with the SPF and DKIM authenticated domains.
    • DMARC only checks for alignment if the authentication passes first.
  • If an email's "From" domain is aligned with either the SPF or DKIM authenticated domains, then it can be delivered.
• DMARC Alignment requirements can be "Relaxed" or "Strict"
  • Relaxed means email from a matching root-level (or organization level) domain will align
    • e.g., marketing.example.com will align with example.com
  • Strict means that the domain in the email must exactly match the authenticated domain
    • e.g., marketing.example.com would fail to align with example.com
• Delivery aggregate and failure reports can be sent to designated email addresses for review
  • These reports are critical for troubleshooting and setup.
    • failure/forensic reports are not generated by many mailbox providers for GDPR/privacy regulation compliance, so you may be stuck with aggregate reports
  • DMARC can also be configured in purely an audit mode without SPF and DKIM; no action is taken, but you get reports on who is sending emails on your domain's behalf and whether they succeed for fail authentication.
  • Reports can be analyzed using DMARC Report Analyzer, which can ingest and process manually downloaded reports or connect automatically to an email account to download reports.
    • If you do not have an easy way to process these reports or create an account to receive them, services like DMARC Report^[10] offer free report monitoring for a single domain and up to 10,000 email reports per month.
    • Additionally, Cloudflare offers free DMARC report management for domains configured with their service.
• There will most likely only be one DMARC TXT record on your DNS host.
  • Like SPF, it applies to all emails sent from your domain, and not to specific hosts like DKIM
  • The sp tag can be used to apply a different policy action on subdomains
    • However, if you want more granularity (like different aggregate/failure report addresses), you can add another record for that subdomain.
• DMARC verifies authentication by requiring alignment with either SPF or DKIM, specifies a policy instructing receivers how to handle unauthorized senders, and generates XML reports sent to the domain owner for Accounting.

BIMI

• BIMI (Brand Indicators for Message Identification) allows you to visually brand email sent by authenticated servers by adding your logo to the profile icon in recipient email clients.
• Most major email providers (e.g., Gmail) require a trademarked image, verified with a PEM certificate.

MTA-STS and TLS-RPT

• MTA-STS (Mail Transfer Agent Strict Transport Security) is a mechanism by which all email sent to and from your email server is sent over TLS connections.
• TLS-RPT (TLS Reporting) is a standard that allows reporting of any encryption issues, and is helpful in triaging MTA-STS issues.

SPF Implementation

The syntax guide on Open-SPF is phenomenal, but for simplicity, here's an example of a sub-optimal SPF record.

1. Name: @
2. Type: TXT
3. TTL: 3600
4. Value: v=spf1 +a mx ip4:123.45.67.89 a:contoso.com include:spf.example.com ~include:olddomain.com -all

The SPF record is first checked as a whole to ensure it is a valid record and falls below the 10 maximum DNS lookups. Then the record is processed in order, and once the email is matched to a qualifier, it stops getting processed. Let's break it down.

1. Name: @
   1. @
      1. Designates the domain to which the SPF record applies; @ is the current top-level domain (e.g., example.com)
      2. Can also use a subdomain, like mail or mail.example.com, with the exact formatting dependent on your DNS name server.
2. Type: TXT
   1. This is a text record (as opposed to A, CNAME, etc.)
3. TTL: 3600
   1. The time to live is 1 hour (3600 seconds)
   2. This is an expiration date for the SPF record, and helps DNS servers maintain up-to-date records.
4. Value: v=spf1 +a mx ip4:123.45.67.89 a:contoso.com include:spf.example.com -all
   1. v=spf1
      1. SPF version 1
      2. There is currently only one version.
   2. +a
      1. The + qualifier passes emails which match this sender
         1. Because the + is the default qualifier, is not normally explicitly stated, and will not be stated in further records.
      2. The a tells the recipient to check the current domain's DNS for A records (e.g., IPv4 addresses) and mark them as designated senders
      3. Note: dmarcian recommends avoiding a entries as they are often unnecessary and increase the number of lookups.^[17]
   3. mx
      1. Check the current domain's DNS for MX records and mark them as designated senders
         1. Note: Since the default mechanism is +, it does not need to be explicitly written out
      2. The mx mechanism resolve to the A/AAAA records, which can trigger additional lookups.
         1. Note: dmarcian recommends avoiding mx entries as they are often unnecessary and increase the number of lookups.^[18]
   4. ip4:123.45.67.89
      1. Designates the IP address 123.45.67.89 as an authorized sender
      2. Could also be used to designate an IP range (e.g., ip4:123.45.67.0/24)
      3. Note: Use ip6: to designate an IPv6 address or range.
   5. a:contoso.com
      1. Checks contoso.com for A and AAAA records^[19] and designates them as authentic senders
   6. include:spf.example.com
      1. Checks spf.example.com for its SPF record and includes that in the current SPF record lookup
      2. Be careful as this can bring you much closer to the 10 DNS lookup limit and cause a PermError
   7. ~include:olddomain.com
      1. This marks emails from olddomain.com with a "softfail" qualifier, indicating that emails from that domain shouldn't be rejected outright, but possibly tagged as suspicious.
   8. -all
      1. Fail all originating hosts that have not been matched to this point in the record
         1. Because this is the last entry, any email that is authenticated by one of the earlier entries will get through, and everything else will fail authentication
         2. This is similar to Firewall configurations where the last rule is often deny any any
      2. Because this is a Fail qualifier, it must to be manually written out as -
         1. ~all is also frequently seen in default configurations, and is used when transitioning between services or when using email marketing services (Mailchimp, Sendgrid, etc.) which do not get added to the SPF record.
      3. The last entry is always an all mechanism, which sets the qualifier for any email that doesn't match any earlier mechanism.

Honorable mention: redirect

If you manage multiple domains and subdomains that all point reference the same record and information, you can configure them to all point to the same SPF record with a redirect. This replaces the entire SPF record and does not count towards the DNS lookup limit, and can reduce the headache of managing multiple identical SPF records.

A subdomain that redirects to an "spf-primary" record would look like this: mailer.example.com txt "v=spf1 redirect=spf-primary.example.com"

Managing DNS Lookups and DNS Flattening

Going above 10 DNS lookups will cause a permanent error for your SPF record, preventing it from being used for authentication. Because most SPF records will point to external mail services whose structure could change without notice, it's important to stay as far below 10 as possible so you don't accidentally begin erroring out.

There are a few tools you can use to check the number of lookups you have:

• SPF Surveyor - dmarcian
  • Well organized and easy to read.
• EasyDMARC SPF Checker
  • Similar to dmarcian, though a little less clean.
• SPF Record Lookup - DMARCLY
  • Pretty spartan, but gets the job done.
  • Recommends flattening, which is bad practice.

What can you do to reduce lookups?

1. Remove unnecessary and duplicate records
   1. a and mx records are often unnecessary:
      1. a records identify domain-name lookup addresses, like store.example.com, and modernly the same IPs are not often reused for mail (especially with third-party email providers).
      2. mx records instruct sending domains where you receive email, and while intuitively it's the same place as where you send email from, this is often not the case (especially with third-party email providers).
   2. Remove any old or unused vendors or mechanisms
2. Consolidate where possible
   1. If there are overlapping SPF records for different services you use, you may be able to eliminate one of them. The tools listed above can help you find all sub-includes and IP ranges to determine eligibility.
3. Remove records from mass-marketing vendors
   1. Mass-marketing vendors (like MailChimp or SendGrid) do not work well with SPF, and will dramatically inflate the number of lookups you will perform. It's better to just configure DKIM for them instead.
4. Use subdomains to split up certain senders
   1. If certain groups or departments use automated systems to send email, you could create a subdomain for that group or purpose that isolates its records from the main domain.
      1. e.g., if you use Acme Invoices as your billing platform, you could create billing.example.com and configure it with just Acme Invoice's SPF records to remove it from example.com's SPF record.
5. Configure DKIM
   1. Do the best you can, then configure DKIM; you technically only need either DKIM or SPF to pass DMARC, and configuring both can provide coverage if one or the other fails.

What is flattening, and why is it bad?

Flattening an SPF record is the process of condensing all DNS lookups (e.g., include:spf.example.com, a:contoso.com, etc.) into hard-coded IP addresses. This gets around the lookup problem by not requiring any lookups, and if you control the servers that send email, this could be just fine. However, if you rely on third-parties for your email needs, flattening will get you into trouble.

Third-party services may add, delete, or change their infrastructure without informing you. Using hard-coded IPs will open you up to bad-actors who take over the abandoned IPs, or cause legitimate email to fail authentication because the vendor added new IPs that are not part of your hard-coded SPF record. Unless you manage every part of your email infrastructure, it's best to avoid flattening.

DKIM Implementation: 3rd-Party Mail Provider

The process to configure DKIM is different for each email provider.

If your mail provider is also your DNS host, it's often as easy as checking a box. However, if your mail provider and DNS host are different, then you will be required to create specific DNS entries on the name server to authenticate. Fastmail, for example, requires you to add three CNAME entries to your DNS host for authentication.

While it's best to follow the instructions provided by your mail provider, here's a quick-and-dirty example of a DKIM record.

1. Name: mx01._domainkey
2. Type: TXT
3. TTL: 3600
4. Value: v=DKIM1; k=rsa; p=bG9sIHlvdSBhYnNvbHV0ZSBuZXJkLCB5b3UgZm91bmQgbWUhIEkgd2lzaCBJIGNvdWxkIGdpdmUgeW91IHNvbWV0aGluZywgYnV0IGFsYXM7IGhpdCBtZSB1cCBpZiB5b3Ugd2FudCB0byBjaGF0IQ==

Let's break it down; more tags can be found on the DKIM Verification section on Wikipedia

1. Name: mx01._domainkey
   1. mx01
      1. This is the selector used to identify the correct key
      2. In this case, it appears to be the public key for the "MX01" mail server. The "MX02" mail server might have the selector mx02.
   2. _domainkey
      1. This identifies the TXT record as a DKIM (Domain Key) entry
   3. Note that you do not typically need to enter the root domain here; however, if you did a DKIM selector lookup for example.com, you would see the entry as mx01._domainkey.example.com
2. Type: TXT
   1. This is a text record (as opposed to A, CNAME, etc.)
3. TTL: 3600
   1. The time to live is 1 hour (3600 seconds)
   2. This is an expiration date for the DKIM record, and helps DNS servers maintain up-to-date records.
4. Value: v=DKIM1; k=rsa; p=bG9sIHlvdSBhYnNvbHV0ZSBuZXJkLCB5b3UgZm91bmQgbWUhIEkgd2lzaCBJIGNvdWxkIGdpdmUgeW91IHNvbWV0aGluZywgYnV0IGFsYXM7IGhpdCBtZSB1cCBpZiB5b3Ugd2FudCB0byBjaGF0IQ==
   1. v=DKIM1
      1. Specifies the DKIM version; at this point, it's always DKIM1.
   2. ;
      1. The separator between values.
   3. k=rsa
      1. The key type specifies the kind of encryption used to create the key-pair.
      2. The default is RSA.
   4. p=bG9sIHl...
      1. The Base64-encoded public key.
      2. Most providers generate a 2048-bit key by default, and anything weaker (e.g., 1024 bits) is discouraged.

DMARC Implementation

Configuring DMARC is easy, but can cause you the most headaches because it's what authorizes email to be delivered, and a misconfiguration can stop your email in its tracks. Therefore, it's highly recommended that you first configure your DMARC policy to none to take no action on emails for the first couple of weeks, using the reports generated to make sure everything is getting delivered as expected, and then to add a quarantine or reject policy and maybe ramp up implementation through the pct tag.

Below is an example of a DMARC TXT record:

1. Name: _dmarc
2. Type: TXT
3. TTL: 3600
4. Value: v=DMARC1; p=quarantine; sp=reject; pct=100; aspf=r; adkim=r; rua=mailto:dmarc-reports@example.com; ruf=mailto:dmarc-failures@example.com; fo=1; ri=43200

Let's break it down.

1. Name: _dmarc.example.com
   1. _dmarc
      1. Signifies this a DMARC TXT entry
   2. Note that you do not typically need to enter the root domain here; however, if you did a DMARC lookup for example.com, you would see the entry as _dmarc.example.com
      1. Only one DMARC record needs to exist for the apex domain, but you can add more records for different subdomains to take different actions; for example, _dmarc.mailer.example.com
2. Type: TXT
   1. This is a text record (as opposed to A, CNAME, etc.)
3. TTL: 3600
   1. The time to live is 1 hour (3600 seconds)
   2. This is an expiration date for the DMARC record, and helps DNS servers maintain up-to-date records.
4. Value: v=DMARC1; p=quarantine; sp=reject; pct=100; aspf=r; adkim=r; rua=mailto:dmarc-reports@example.com; ruf=mailto:dmarc-failures@example.com; fo=1;
   1. v=DMARC1
      1. DMARC version 1; at present, there is only one version.
   2. ;
      1. The separator between tags.
      2. Spaces and tabs can make entries more readable, but are completely optional.
   3. p=quarantine
      1. The Policy applied to emails which fail their SPF and DKIM authentication checks
      2. There are three options:
         1. none: No action is taken
            1. Typically used while configuring email security and collecting reports.
         2. quarantine: Emails which fail authentication should be treated with suspicion and sent to the spam/junk folder
            1. This allows the recipient server to still receive and process unauthenticated mail, just treats them with suspicion
         3. reject: Instructs the receiving server to outright reject any mail that fails authentication.
            1. This is the most secure, but can also be the most problematic if a configuration changes or something goes wrong.
   4. sp=reject
      1. The policy for subdomains; if sp is not stated in the record, then the policy described by p is inherited by its subdomains.
         1. Setting sp may be helpful to set a stricter policy if there are no subdomains, or a looser policy if configuring a subdomain for mail.
         2. Some receiving servers don't check the root domain for a DMARC policy to inherit; playing it safe and adding a DMARC record for your subdomains is not a bad idea.
      2. Creating a distinct DMARC policy for a subdomain (e.g., _dmarc.mailer.example.com) takes precedence over the sp policy designation.
         1. To reiterate, sp only applies to a subdomain if there isn't a more specific DMARC policy created for it.
   5. pct=100
      1. The percent of unauthenticated emails to apply the policy to.
         1. e.g., pct=20 would only apply the p=reject policy to 20% of emails which fail authentication
      2. This is helpful during a slow rollout to make sure not all email flow stops.
      3. Default is 100, and does not need to be explicitly written.
   6. aspf=r
      1. SPF alignment requirements
         1. r is relaxed, and only the root/organizational domain must match
            1. Relaxed is the default value for both aspf and adkim, and does not need to be explicitly stated
         2. s is strict, and domains must match exactly
   7. adkim=r
      1. DKIM alignment requirements; see aspf for details.
   8. rua=mailto:dmarc-reports@example.com
      1. Identifies the email address to which recipient servers should send delivery aggregate reports
      2. Each address must begin with mailto:, and multiple addresses can be specified if separated by a comma.
         1. e.g., rua=mailto:address1@example.com,mailto:address2@contoso.com
      3. Aggregate reports contain basic information and include successful and failed delivery information
   9. ruf=mailto:dmarc-failures@example.com
      1. Identifies the email address to which recipient servers should send individual delivery forensic failure reports
      2. Forensic failure reports contain detailed information about failed deliveries to assist with triage and troubleshooting.
         1. However, most major providers highly redact or suppress ruf records for privacy and GDPR compliance.
   10. fo=1
       1. The failure reporting option specifies what generates a forensic report .
          1. 0 is default, and only requests forensic reports on DMARC failure.
          2. 1 requests a report for any SPF or DKIM failure, which is helpful for triage and troubleshooting during initial setup.
          3. d generates a report only when DKIM authentication (not alignment)
          4. s generates a report when SPF fails.
       2. You can select multiple options with a colon (e.g., fo=0:d)
       3. Reminder: many mailbox providers don't send forensic/failure reports for GDPR compliance.
   11. ri=43200
       1. The "report interval" is the time in seconds you request receivers to generate reports.
          1. The default is 24 hours (86400 seconds), and 12 hours as configured here.
       2. Most providers ignore this, and send reports every 24 hours or more based on their own infrastructure.
          1. "DMARC implementations MUST be able to provide daily reports and SHOULD be able to provide hourly reports when requested. However, anything other than a daily report is understood to be accommodated on a best-effort basis."^[20]

Sending Reports to a Different Domain

If you are sending DMARC reports to another domain for analysis, you will need to create a TXT record on that domain's name server to identify each sending domain.^[21]

For example:

1. Type: TXT
2. Name: sendingdomain.com._report._dmarc.receivingdomain.com
   1. For example, if the MSP AcmeIT.com was configured to receive and manage DMARC reports for example.com, the record would be example.com._report._dmarc.acmeit.com
3. Value: "v=DMARC1"
   1. Just indicates the version of DMARC

The TXT record name identifies the domain generating the report (sendingdomain.com), followed by ._report._dmarc and the domain receiving the reports (.receivingdomain.com).
The TXT record value just identifies the DMARC version ("v=DMARC1")

Using dig to verify SPF, DKIM, and DMARC

When using dig to verify records and look up changes, only SPF can be searched generally; DKIM and DMARC searches must include the full name of the record.

In the examples, I rotated the DNS server I was querying to demonstrate that any DNS server could be used. Records for subdomains or named records above root (e.g., mail.example.com or _dmarc.example.com) will require their own dig queries.

1. SPF
   1. dig @8.8.8.8 example.com TXT +noall +answer
      1. This will return all root TXT records, so you may get several irrelevant responses in addition to the SPF
2. DKIM
   1. dig @1.1.1.1 mx01._domainkey.example.com TXT +noall +answer
      1. Returns the final DKIM record value, whether stored on your name server or redirected via a CNAME
   2. dig @1.0.0.1 protonmail._domainkey.example.com CNAME +noall +answer
      1. Only returns the CNAME record on your server, if applicable.
3. DMARC
   1. dig @8.8.4.4 _dmarc.example.com TXT +noall +answer
      1. This will return the DMARC record for example.com
   2. dig @1.1.1.1 _dmarc.mail.example.com txt +noall +answer
      1. Returns the DMARC record for the subdomain mail.example.com

If the record exists and you entered the command correctly, you should get responses with all the information in the record being queried.

DMARC Report Analyzer

• DMARC Report Analyzer is a tool written in Python that allows you to analyze all DMARC reports in a folder or inbox and generate a spreadsheet with the results.
• I've forked a version and added^[22] various features, including more progress bars, an offline cache for the Spamhaus list of IPs, and an output CSV which includes more information that can be helpful when troubleshooting email authentication issues.

To install and use DMARC Report Analyzer, there is a helpful script in the Installation section to clone the repository to your local machine, create the local environment, install the requirements, and then run main.py. First, however, I recommend creating a scripts (or similar) folder somewhere to keep everything organized and then opening your Terminal from that folder.

The script expects you to have Python version 3 installed on your system and is currently written for Linux, but there is an option for Windows (which I haven't really tested).

If working on a Linux system, you will need to use a Python Virtual Environment to download and update the correct packages. This is easily managed with a shell script that performs the run/setup process.

#!/usr/bin/env bash
cd ~/scripts/DMARC-Report-Analyzer
# Optional
python3 -m venv env
source env/bin/activate
# Mandatory
pip install -r requirements.txt
python3 main.py

The final result is a spreadsheet with a summary of all of the reports it collected. You can use whois and grep to lookup the IP addresses to to find out where emails are coming from with the command whois [IP address] | grep "NetRange:\|CIDR:\|Organization:", and your output should look something like this: