| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| security:email_authentication [2026/09/09 11:32] – Date the ARC-to-Historic draft (rev -00, 2026-04-22) instead of hedging; scope the BIMI/VMC 'nobody' to the search that was run. Authored by Claude karel.kubicek.claude | security:email_authentication [2026/09/10 21:05] (current) – programming:internet_scanning is written; the link is no longer red on purpose. Authored by Claude. karel.kubicek.claude |
|---|
| This page is about **measuring what mail servers and mail clients have actually deployed** — SPF, DKIM, DMARC, ARC, DANE, MTA-STS, STARTTLS, S/MIME and OpenPGP — and about the delivery path a message really takes. It is a methods page: which instrument answers which question, what the denominator has to be, what the field has already built, and which of its methods are current in 2026. | This page is about **measuring what mail servers and mail clients have actually deployed** — SPF, DKIM, DMARC, ARC, DANE, MTA-STS, STARTTLS, S/MIME and OpenPGP — and about the delivery path a message really takes. It is a methods page: which instrument answers which question, what the denominator has to be, what the field has already built, and which of its methods are current in 2026. |
| |
| It is not an explanation of the protocols. RFC 7208 (SPF), RFC 6376 (DKIM), RFC 9989 (DMARC), RFC 7672 (DANE for SMTP) and RFC 8461 (MTA-STS) define them, and a fresh student can read those faster than any wiki paraphrase. What the RFCs do not tell you is that "60.9% of domains have SPF" and "56.5% of domains have SPF" are both true, in the same year, of the same protocol — because they are different populations, and the difference is bigger than most of the effects people publish. | It is not an explanation of the protocols. RFC 7208 (SPF), RFC 6376 (DKIM), RFC 9989 (DMARC), RFC 7672 (DANE for SMTP) and RFC 8461 (MTA-STS) define them, and a fresh student can read those faster than any wiki paraphrase. What the RFCs do not tell you is that "60.9% of domains have SPF" and "56.5% of domains have SPF" are both true, in the same year, of the same protocol — because they are different populations, and the gap between them is wider than many of the differences papers headline. |
| |
| The evidence base is **31 papers** from the [[literature:corpus|publication corpus]] — CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026, 5,859 extracted papers. Those seven venues are where this subfield publishes, but they are not all of it: EuroS&P, ACSAC, RAID, AsiaCCS, TMA and the CCR/ANRW line are absent, so every count below is a count over those seven. How the 31 were selected is in [[#Use in Publications]] and the full query log is on [[provenance:security:email_authentication]]. | The evidence base is **31 papers** from the [[literature:corpus|publication corpus]] — CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026, 5,859 extracted papers. Those seven venues are where this subfield publishes, but they are not all of it: EuroS&P, ACSAC, RAID, AsiaCCS, TMA and the CCR/ANRW line are absent, so every count below is a count over those seven. How the 31 were selected is in [[#Use in Publications]] and the full query log is on [[provenance:security:email_authentication]]. |
| |
| * {[durumeric2015_neither]} — //Neither Snow Nor Rain Nor MITM//, IMC 2015. The founding measurement: Alexa Top Million MX scan, an IPv4-wide SMTP scan, and a year of Gmail's own SMTP handshake logs, in one paper. Read it for the three-instrument design, and for the STARTTLS-stripping result that no purely passive study could have found. | * {[durumeric2015_neither]} — //Neither Snow Nor Rain Nor MITM//, IMC 2015. The founding measurement: Alexa Top Million MX scan, an IPv4-wide SMTP scan, and a year of Gmail's own SMTP handshake logs, in one paper. Read it for the three-instrument design, and for the STARTTLS-stripping result that no purely passive study could have found. |
| * {[hu2018_measurements]} — //End-to-End Measurements of Email Spoofing Attacks//, USENIX Security 2018. The paper that moved the question from "who publishes a record" to "what arrives in the inbox". Registers accounts at 35 providers and sends forged mail to itself. Every account-based delivery paper since is a variant of this. | * {[hu2018_measurements]} — //End-to-End Measurements of Email Spoofing Attacks//, USENIX Security 2018. The paper that moved the question from "who publishes a record" to "what arrives in the inbox". Registers accounts at 35 providers and sends forged mail to itself. Most account-based delivery papers since are variants of it, though {[foster2015_security]} ran the same instrument at 22 providers three years earlier. |
| * {[ashiq2024_beyond]} — //SPF Beyond the Standard//, USENIX Security 2024. Seventeen months of weekly zone-file scans over 176 million domains, plus an inbound probe that asks whether receiving MTAs //validate// SPF at all, plus an operator survey. The best current example of the three-sided design this topic now expects. | * {[ashiq2024_beyond]} — //SPF Beyond the Standard//, USENIX Security 2024. Seventeen months of weekly zone-file scans over 176 million domains, plus an inbound probe that asks whether receiving MTAs //validate// SPF at all, plus an operator survey. The best current example of the three-sided design that the Ashiq–Chung line has made the expectation in this area. |
| * {[wang2024_breakspf]} — //BreakSPF//, NDSS 2024. The 2024–2026 turn: the interesting object is no longer the record but the //shared infrastructure the record authorises//. 23,916 Tranco-top-million domains are spoofable because their SPF includes a cloud, CDN or CI/CD range an attacker can rent. | * {[wang2024_breakspf]} — //BreakSPF//, NDSS 2024. The 2024–2026 turn: the interesting object is no longer the record but the //shared infrastructure the record authorises//. 23,916 Tranco-top-million domains are spoofable because their SPF includes a cloud, CDN or CI/CD range an attacker can rent. |
| |
| If your object is end-to-end encryption rather than transport, start at {[poddebniak2018_efail]} (Efail) and {[oendaroe2025_mine]} (S/MINE: 41.7 million X.509 certificates crawled from public LDAP address books, of which 38 million are S/MIME-capable). | If your object is end-to-end encryption rather than transport, start at {[poddebniak2018_efail]} (Efail) and {[oendaroe2025_mine]} (S/MINE: 41.7 million X.509 certificates crawled from public LDAP address books, of which 38 million are S/MIME-capable). |
| |
| ===== The Six Instruments ===== | ===== The Instruments: Six That Carry the Field, and Six More ===== |
| |
| The 31 papers use twelve identifiable instruments. The **median paper runs three**, the mean is 2.74, and the range is one to five; four papers run only one instrument and one runs five. Pick before you write the scanner, because the instrument decides which of the four different questions you are answering. | The 31 papers use twelve identifiable instruments. The **median paper runs three**, the mean is 2.74, and the range is one to five; four papers run only one and one runs five. Pick before you write the scanner, because the instrument decides which question you are answering — and there are at least six different questions here, which is why the papers that matter run three instruments rather than one. The period split for all twelve is in [[#Instruments, folded]]. |
| |
| ^ Instrument ^ The question it answers ^ What it cannot see ^ Papers (of 31) ^ | ^ Instrument ^ The question it answers ^ What it cannot see ^ Papers (of 31) ^ |
| | **DNS record scan** — resolve TXT/MX/TLSA/``_mta-sts`` over a domain list | //Does this domain publish a policy?// | Whether anyone honours it; whether the domain sends mail at all; DKIM, unless you already know the selector | 18 | | | **DNS record scan** — resolve TXT/MX/TLSA/''_mta-sts'' over a domain list | //Does this domain publish a policy?// | Whether anyone honours it; whether the domain sends mail at all; DKIM, unless you already know the selector | 18 | |
| | **Account-based delivery test** — register at real providers, send crafted mail, look at the inbox | //What does a receiver actually do with a message that fails?// | Anything about the long tail: you can afford 20–50 providers, not a million | 14 | | | **Account-based delivery test** — register at real providers, send crafted mail, look at the inbox | //What does a receiver actually do with a message that fails?// | Anything about the long tail: you can afford 20–50 providers, not a million | 14 | |
| | **SMTP probe** — connect to MTAs and drive the protocol | //Does this server offer STARTTLS, present a valid certificate, query SPF before DATA?// | Intent; policy; anything above the connection | 13 | | | **SMTP probe** — connect to MTAs and drive the protocol | //Does this server offer STARTTLS, present a valid certificate, query SPF before DATA?// | Intent; policy; anything above the connection | 13 | |
| | **Provider logs** — headers, delivery logs or mailboxes from a real operator | //What happens to mail at scale, including what never arrives?// | Reproducibility. You cannot get the dataset, and neither can your reviewer | 6 | | | **Provider logs** — headers, delivery logs or mailboxes from a real operator | //What happens to mail at scale, including what never arrives?// | Reproducibility. You cannot get the dataset, and neither can your reviewer | 6 | |
| | **Operator survey** — ask the administrators | //Why is it configured that way?// | Everything a self-selected sample of list-subscribing operators is not | 5 | | | **Operator survey** — ask the administrators | //Why is it configured that way?// | Everything a self-selected sample of list-subscribing operators is not | 5 | |
| | |
| | **The delivery test has an outcome variable and it is not binary.** Fourteen of the 31 papers run this instrument and the ones worth copying code at least four outcomes — rejected at the SMTP layer, accepted and spam-foldered, delivered to the inbox, delivered to the inbox **with a visible warning** — and {[hu2018_measurements]} adds a fifth dimension by inspecting the interface itself: 9 of 35 providers showed any security indicator, and 25 of 35 displayed a //misleading// element (a sender photo, a name card, a message history) on a forged message. {[wang2022_revisiting]} separates the same four outcomes again for ARC. A study that codes "arrived or not" throws away the finding. |
| |
| The remaining six, thinner but not marginal: **software testbed** (7 papers — run Postfix, Exim, libspf2 in a lab and see which is wrong), **code analysis** (3), **notification experiment** (3 — tell the operators and rescan, see [[practices:notifying_websites]]), **passive DNS** (2), **user study** (2), **honey domain** (1). | The remaining six, thinner but not marginal: **software testbed** (7 papers — run Postfix, Exim, libspf2 in a lab and see which is wrong), **code analysis** (3), **notification experiment** (3 — tell the operators and rescan, see [[practices:notifying_websites]]), **passive DNS** (2), **user study** (2), **honey domain** (1). |
| |
| **Two instruments are new.** Neither the operator survey nor the software testbed appears at all before 2020 in this population; between them they are run by **eight of the sixteen 2023–2026 papers** and none of the six from 2015–2019. If you are designing a study now and your only instrument is a DNS scan, you are designing a 2015 paper. | **Two instruments are new.** Neither the operator survey nor the software testbed appears at all before 2020 in this population; between them they are run by **eight of the sixteen 2023–2026 papers** and none of the six from 2015–2019. If you are designing a study now and your only instrument is a DNS scan, you are designing one panel of a 2015 paper. |
| | |
| | ==== The end-to-end encryption slice is a different topic wearing the same clothes ==== |
| | |
| | Six of the 31 papers measure S/MIME and OpenPGP rather than transport or authentication: {[poddebniak2018_efail]}, {[mueller2019_johnny]}, {[stransky2022_years]}, {[ising2023_content]}, {[oendaroe2025_mine]}, and part of {[poddebniak2021_better]}. They are in this page's population because the object is still deployed mail machinery, but almost nothing else transfers: |
| | |
| | * **The instrument is the client matrix**, not the DNS scan. Four of the six drive a matrix of 19 to 48 mail clients by hand ({[poddebniak2018_efail]} 48, {[poddebniak2021_better]} 28, {[mueller2019_johnny]} 25, {[ising2023_content]} 19); {[oendaroe2025_mine]} adds a five-client check to a certificate scan, and {[stransky2022_years]} runs no client tests at all. |
| | * **The denominator problem inverts.** For SPF the hard question is which million domains; here it is that 49 clients is not a population and there is no list to sample from. These papers are vulnerability censuses over a purposive set, and none of them claims a prevalence. |
| | * **The two exceptions prove it.** {[stransky2022_years]} got a prevalence — 5.46% of users ever used S/MIME or PGP, 0.06% of emails encrypted — only by getting 27 years of one university's mail server. {[oendaroe2025_mine]} got one by scanning public LDAP address books, which is a population nobody had thought to enumerate. |
| | |
| | If your object is end-to-end encryption, read those six and treat the rest of this page as background. |
| |
| ===== Denominators: the Thing That Will Sink Your Paper ===== | ===== Denominators: the Thing That Will Sink Your Paper ===== |
| |
| This is the whole reason the page exists. Below are adoption figures for the same five mechanisms, from papers in the same corpus, each correct in its own paper and none comparable with any other. | This is the whole reason the page exists. Below are adoption figures for six mechanisms, from papers in the same corpus, each correct in its own paper and almost none comparable with any other. Two of the rows are the same scan on the same day and differ by 18.5 points. |
| |
| ^ Mechanism ^ Figure ^ **Population it is a share of** ^ Source ^ | ^ Mechanism ^ Figure ^ **Population it is a share of** ^ Source ^ |
| | DMARC | 13.6% // 22.6% | 12 million domains // the top 1 million of those 12 million, 2023 | {[czybik2023_lazy]} | | | DMARC | 13.6% // 22.6% | 12 million domains // the top 1 million of those 12 million, 2023 | {[czybik2023_lazy]} | |
| | SPF | 60.9% published, 55.9% **valid** | Tranco top million, 2023–24 | {[wang2024_breakspf]} | | | SPF | 60.9% published, 55.9% **valid** | Tranco top million, 2023–24 | {[wang2024_breakspf]} | |
| | | SPF, **same scan, same day** | 79.4% published, 72.7% valid | the 738,310 of that same top million that have an MX record //or// answer with an SMTP banner on port 25 | {[wang2024_breakspf]} | |
| | MTA-STS | 68,030 domains, 0.07%–0.13% by TLD | domains **with MX records** in the ''.com''/''.net''/''.org''/''.se'' zone files, 29 September 2024 | {[ashiq2025_unraveling]} | | | MTA-STS | 68,030 domains, 0.07%–0.13% by TLD | domains **with MX records** in the ''.com''/''.net''/''.org''/''.se'' zone files, 29 September 2024 | {[ashiq2025_unraveling]} | |
| |
| Four traps are visible in that table, and all four have been published wrong somewhere: | Five traps are visible in that table, or visible by their absence from it: |
| |
| - **"Domains" almost never means domains.** Five of the twelve rows restrict to domains that have an MX record — and one of those five says it as "mail-enabled domains" instead, so you cannot even grep for the filter. That filter removes most registered names, and it is the right filter — a domain with no MX receives no mail — but a figure computed with it is not comparable to one computed without it. Say which you did, in the sentence that carries the number. | - **"Domains" almost never means domains.** Five of the twelve rows restrict to domains that have an MX record — and one of those five says it as "mail-enabled domains" instead, so you cannot even grep for the filter. That filter removes most registered names. For a //receiving//-side mechanism — STARTTLS, DANE, MTA-STS — it is clearly the right filter, because a domain with no MX receives no mail. For a //sending//-side mechanism — SPF, DMARC — it is a proxy and a lossy one: a domain that only sends mail has no reason to publish an MX, and the two BreakSPF rows above are the same scan on the same day, differing by **18.5 points** purely on the filter — which is why that paper reports both rather than substituting one. Either way, a figure computed with the filter is not comparable to one computed without it. Say which you did, in the sentence that carries the number. |
| - **A top list is not a sample of the web, and the list changed.** Alexa is the frame in 7 of the 31 papers; Tranco in 9; TLD zone files in 3. The **last Alexa use in this population is 2023 and no paper from 2024 onwards uses it** — Alexa's site shut down on 1 May 2022 and its APIs on 15 December 2022.((Amazon retired alexa.com on 1 May 2022 and the supporting APIs on 15 December 2022, per Amazon's own support pages. Tranco's methodology page does **not** record those dates — it records **1 August 2023**, the date Tranco stopped ingesting Alexa as a source; its current composition is Cisco Umbrella, Majestic, Farsight, the Chrome UX Report and Cloudflare Radar. Both fetched 2026-09-09.)) A 2026 paper using an Alexa list is using a four-year-old snapshot of a dead ranking, and reviewers now say so. Use Tranco with a pinned list ID {[LePochat2019_tranco]}, or a zone file, and see [[design:website_selection]]. | - **A top list is not a sample of the web, and the list changed.** Alexa is the frame in 7 of the 31 papers; Tranco in 9; TLD zone files in 3. The **last Alexa use in this population is 2023 and no paper from 2024 onwards uses it** — Alexa's site shut down on 1 May 2022 and its APIs on 15 December 2022.((Amazon retired alexa.com on 1 May 2022 and the supporting APIs on 15 December 2022, per Amazon's own support pages. Tranco's methodology page does **not** record those dates — it records **1 August 2023**, the date Tranco stopped ingesting Alexa as a source; its current composition is Cisco Umbrella, Majestic, Farsight, the Chrome UX Report and Cloudflare Radar. Both fetched 2026-09-09.)) A 2026 paper using an Alexa list is using a four-year-old snapshot of a dead ranking. Use Tranco with a pinned list ID {[LePochat2019_tranco]}, or a zone file, and see [[design:website_selection]]. |
| - **"Top 1M" and "top 10M" and "all of a zone" give different answers to the same question, in the same year.** SPF is 60.2% of the top million and 56.5% of twelve million in one 2023 paper; 41.7% of a top-ten-million list in another 2023 paper. The gradient is real — popular domains deploy more — so a percentage without its rank window says nothing. | - **"Top 1M" and "top 10M" and "all of a zone" give different answers to the same question, in the same year.** SPF is 60.2% of the top million and 56.5% of twelve million in one 2023 paper; 41.7% of a top-ten-million list in another 2023 paper. The gradient is real — popular domains deploy more — so a percentage without its rank window says nothing. |
| | - **A published record is not a policy, and this is the trap a fresh student walks into first.** Not one row of the table above splits by policy strength, because not one of those headline figures does. But ''p=none'' asks the receiver to do nothing, ''?all'' authorises the whole internet, and a domain publishing either is counted as "deploying DMARC" or "deploying SPF" in every adoption rate on this page. The distribution is where the content is: {[czybik2023_lazy]} — filed above as "SPF configuration" — is largely //about// the qualifier distribution, and found **5.9% (427,767)** of its domains publishing SPF with no restrictive ''all'' at all; the Dutch government census cited below separates //DMARC at quarantine or reject// (86%) from DMARC-at-all, and //strict SPF// (69%) from SPF-at-all, because the two differ by tens of points. Report the distribution, not the presence rate. A presence rate over a population where most records say ''p=none'' is a measurement of who has read a blog post. |
| - **DKIM has no denominator you can enumerate.** SPF, DMARC and MTA-STS live at fixed DNS names; DKIM lives at ''<selector>._domainkey.<domain>'' and the selector is arbitrary. {[wang2022_longitudinal]} scanned the Alexa top million with **40 selectors harvested from passive DNS** and reports 28.1% as an explicit lower bound. Any DKIM deployment number is a lower bound; say what your selector list was and where it came from, or use passive DNS and say the archive's coverage is your denominator. | - **DKIM has no denominator you can enumerate.** SPF, DMARC and MTA-STS live at fixed DNS names; DKIM lives at ''<selector>._domainkey.<domain>'' and the selector is arbitrary. {[wang2022_longitudinal]} scanned the Alexa top million with **40 selectors harvested from passive DNS** and reports 28.1% as an explicit lower bound. Any DKIM deployment number is a lower bound; say what your selector list was and where it came from, or use passive DNS and say the archive's coverage is your denominator. |
| |
| | Sender domains that validate MTA-STS when sending | 19.6% (469) | 2,394 sender domains in the authors' deliverability tests | {[ashiq2025_unraveling]} | | | Sender domains that validate MTA-STS when sending | 19.6% (469) | 2,394 sender domains in the authors' deliverability tests | {[ashiq2025_unraveling]} | |
| |
| That 6.8% is the number to remember. It is a **2024** measurement of whether receiving mail servers so much as look SPF up before accepting the message body — and it is the only figure in this whole population that measures inbound enforcement at internet scale rather than at 20–50 hand-picked providers. Deployment-rate papers count the left column of a two-column problem. | That 6.8% is the number to remember. It is a **2024** measurement of whether receiving mail servers so much as look SPF up before accepting the message body — and within these 31 papers it is the only figure that measures inbound enforcement at internet scale rather than at 20–50 hand-picked providers. Deployment-rate papers count the left column of a two-column problem. |
| | |
| | ==== Deployed is not the same as correct ==== |
| | |
| | Every one of these papers has a second half, and it is always this: a large share of what //is// deployed is broken. If your study stops at the adoption rate you are reporting the good news only. |
| | |
| | ^ What is wrong ^ Share ^ Population ^ Source ^ |
| | | DKIM keys of at most 1024 bits | **84%** | 3,627,871 domains with DKIM keys in passive DNS | {[wang2022_longitudinal]} | |
| | | DKIM public keys shared across domains | 66.9% (2,427,682) | the DKIM-publishing domains in that passive-DNS set | {[wang2022_longitudinal]} | |
| | | SPF authorising more than 100,000 IPv4 addresses | 34.7% | the SPF-publishing domains of 12 million scanned | {[czybik2023_lazy]} | |
| | | SPF with no restrictive ''all'' qualifier at all | 5.9% (427,767) | the same SPF-publishing domains | {[czybik2023_lazy]} | |
| | | SPF needing more than 10 DNS lookups to evaluate | 6.5% | 55 million domains with SPF records | {[ashiq2024_beyond]} | |
| | | STARTTLS certificates failing validation | **30.0%** | 2,112,682 connectable MXs, 2023 | {[blechschmidt2023_hello]} | |
| | | TLSA records invalid | over 22% | SMTP servers serving ''.com'' domains, hourly snapshots | {[lee2022_under]} | |
| | | MTA-STS records misconfigured | 29.6% (20,144) | the 68,030 domains publishing one, Sep 2024 | {[ashiq2025_unraveling]} | |
| | | External DMARC ''rua'' with no authorisation record at the receiving domain | 26% (520K) | 2 million DMARC records naming an external reporting domain | {[ashiq2023_report]} | |
| | | Auto-configuration deployed at all | 7.52% (79,212) | 1,053,469 domains scanned for Autodiscover / Autoconfig / SRV | {[wen2025_automatic]} | |
| | |
| | Two of these are exploitable rather than merely untidy. {[czybik2023_lazy]} rented about €30 of shared web hosting and could send mail passing SPF for **26,095** domains. {[li2026_coordmail]} found that **77.89%** of reflections off 10,079 exploitable bounce servers pass SPF or DKIM — the authentication is working exactly as specified and authenticating the attacker. |
| | |
| | ==== What it looks like from inside an operator ==== |
| | |
| | Six of the 31 papers got logs, headers or mailboxes from a real mail operator, and they are the only source on this page for base rates — what fraction of mail fails, why, and how concentrated the delivery path is. You almost certainly cannot reproduce these; read them so you know what your scan is a proxy for. |
| | |
| | ^ Figure ^ Population ^ Source ^ |
| | | 94.40% of inbound messages authenticated by SPF, DKIM or both | Gmail's own SMTP handshake logs, April 2015 | {[durumeric2015_neither]} | |
| | | 8.11% hard-bounced, 4.82% soft-bounced | 298 million messages at one large Chinese ESP | {[li2024_bounce]} | |
| | | 2.19% (701,347) of bounces caused by **sender authentication failure** | the **32 million bounced** messages that paper's Table 1 classifies — not the 298 million total, against which the same count is 0.24% | {[li2024_bounce]} | |
| | | 31.10% (9,975,329) of bounces caused by the sender **hitting a spam blocklist** | the same 32 million bounced messages — **fourteen times** the authentication row, like for like | {[li2024_bounce]} | |
| | | 82.7% of delivery paths rely entirely on third-party relaying | 105 million messages whose intermediate path could be reconstructed from ''Received'' headers | {[li2025_characterizing]} | |
| | | HHI of 40% for the relay market | middle nodes across those reconstructed paths | {[li2025_characterizing]} | |
| | | 5.46% of users ever used S/MIME or PGP; 0.06% of messages encrypted | 81.6 million messages from 37,089 accounts at one university over 27 years | {[stransky2022_years]} | |
| | |
| | The two ''li'' rows are the same 32 million bounces and are directly comparable; **the authentication row is not comparable to the 298 million total**, and this page published it against the wrong denominator for several hours before catching it. That is the trap in its natural habitat: an operator paper reports some figures against the whole corpus of mail and others against the failures, in adjacent sentences. |
| |
| ===== Methods, and Which Ones Are Current ===== | ===== Methods, and Which Ones Are Current ===== |
| * **Shared infrastructure as the unit of analysis.** {[wang2024_breakspf]} (2024), {[wang2025_email]} (SMTP smuggling, 2025) and {[li2026_coordmail]} (2026) all ask what an attacker can do by //renting// a piece of infrastructure that thousands of domains have already authorised. This is the newest coherent line in the population and it is three papers old. | * **Shared infrastructure as the unit of analysis.** {[wang2024_breakspf]} (2024), {[wang2025_email]} (SMTP smuggling, 2025) and {[li2026_coordmail]} (2026) all ask what an attacker can do by //renting// a piece of infrastructure that thousands of domains have already authorised. This is the newest coherent line in the population and it is three papers old. |
| * **Operator surveys alongside the scan.** None before 2020; five by 2025, with 39, 74, 16, 95 and 117 respondents ({[lee2022_under]}, {[ashiq2023_report]}, {[blechschmidt2023_hello]}, {[ashiq2024_beyond]}, {[ashiq2025_unraveling]}). Recruitment is from MailOP, NANOG, DENOG, NLNOG and MESSEU in every case where it is stated — a specific, self-selecting, unusually competent population. Report it as such; it is not a sample of operators. | * **Operator surveys alongside the scan.** None before 2020; five by 2025, with 39, 74, 16, 95 and 117 respondents ({[lee2022_under]}, {[ashiq2023_report]}, {[blechschmidt2023_hello]}, {[ashiq2024_beyond]}, {[ashiq2025_unraveling]}). Recruitment is from MailOP, NANOG, DENOG, NLNOG and MESSEU in every case where it is stated — a specific, self-selecting, unusually competent population. Report it as such; it is not a sample of operators. |
| * **Notification-and-rescan as a built-in second phase.** {[bennett2022_spfail]}, {[czybik2023_lazy]} and {[blechschmidt2023_hello]} all notify and remeasure. Only {[blechschmidt2023_hello]} has a comparison arm: domains whose notification **bounced** remediated at 20.7% against 39.7% for domains that were reached (Fisher exact //p// = 7.8 × 10<sup>-43</sup>). That arm is what makes the result more than a trend — but it is not a randomised control, and the authors say so themselves: //"we cannot conclude a causal relation since the lack of deliverable emails may be a hidden variable which influences fix rates"//. Undeliverable postmaster addresses are plausibly a proxy for an unattended domain. If you run a notification experiment, randomise. | * **Notification-and-rescan as a built-in second phase.** {[bennett2022_spfail]}, {[czybik2023_lazy]} and {[blechschmidt2023_hello]} all notify and remeasure. The effect sizes are small and worth knowing before you budget for one: {[czybik2023_lazy]} saw **6,931** SPF errors fixed two weeks after notification, a 3.28% fall in the total, and {[bennett2022_spfail]} found **just over 80%** of the domains it could infer results for were still vulnerable four months on. Only {[blechschmidt2023_hello]} has a comparison arm: domains whose notification **bounced** remediated at 20.7% against 39.7% for domains that were reached (Fisher exact //p// = 7.8 × 10<sup>-43</sup>). That arm is what makes the result more than a trend — but it is not a randomised control, and the authors say so themselves: //"we cannot conclude a causal relation since the lack of deliverable emails may be a hidden variable which influences fix rates"//. Undeliverable postmaster addresses are plausibly a proxy for an unattended domain. If you run a notification experiment, randomise. |
| * **Client matrices.** Still current — {[tang2025_multifaceted]} (49 clients, 2025) and {[wen2025_automatic]} (29 clients, 2025) are recent — but they are a vulnerability census, not a prevalence measurement, and should not be written as one. | * **Client matrices.** Still current — {[tang2025_multifaceted]} (49 clients, 2025) and {[wen2025_automatic]} (29 clients, 2025) are recent — but they are a vulnerability census, not a prevalence measurement, and should not be written as one. |
| |
| The corpus cannot know any of this: the newest paper in it went to press before most of it happened. Every item here was fetched on **2026-09-09**, not recalled. | The corpus cannot know any of this: the newest paper in it went to press before most of it happened. Every item here was fetched on **2026-09-09**, not recalled. |
| |
| * **DMARC is no longer RFC 7489.** It was republished on the **standards track** in **May 2026** as **RFC 9989** (protocol), **RFC 9990** (aggregate reporting) and **RFC 9991** (failure reporting). RFC 9989 states //"Obsoletes: 7489, 9091"// and //"Category: Standards Track"// on its title page.((RFC 9989, //Domain-Based Message Authentication, Reporting, and Conformance (DMARC)//, May 2026, fetched from https://www.rfc-editor.org/rfc/rfc9989.txt on 2026-09-09. Obsoletes RFC 7489 and RFC 9091.)) Every paper in this population cites the Informational RFC 7489. If you are writing now, cite 9989. | * **DMARC is no longer RFC 7489.** It was republished on the **standards track** in **May 2026** as **RFC 9989** (protocol), **RFC 9990** (aggregate reporting) and **RFC 9991** (failure reporting). RFC 9989 states //"Obsoletes: 7489, 9091"// and //"Category: Standards Track"// on its title page.((RFC 9989, //Domain-Based Message Authentication, Reporting, and Conformance (DMARC)//, May 2026, fetched from https://www.rfc-editor.org/rfc/rfc9989.txt on 2026-09-09. Obsoletes RFC 7489 and RFC 9091.)) **18** of the 31 papers here cite RFC 7489 and none could have cited 9989; the other 13 either name DMARC without the RFC or, in 7 cases, never mention DMARC at all. If you are writing now, cite 9989. |
| * **The ''pct='' tag is gone.** RFC 9989 §A.6, "Removal of the 'pct' Tag": //"Operational experience showed that the 'pct' tag was usually not accurately applied … This version of the DMARC mechanism, therefore, introduces the 't' tag as shorthand for 'testing'"//. The DMARC tag registry now marks ''pct'' historic. A 2026 parser that treats ''pct=0'' as a valid partial-rollout signal is parsing a dead tag; a 2026 measurement that reports ''pct'' distributions is measuring a legacy artefact, which is a legitimate thing to do provided you say so. ''p=none'' is unchanged and is still where the RFC tells domain owners to start. | * **The ''pct='' tag is gone.** RFC 9989 §A.6, "Removal of the 'pct' Tag": //"Operational experience showed that the 'pct' tag was usually not accurately applied … This version of the DMARC mechanism, therefore, introduces the 't' tag as shorthand for 'testing'"//. The DMARC tag registry now marks ''pct'' historic. A 2026 parser that treats ''pct=0'' as a valid partial-rollout signal is parsing a dead tag; a 2026 measurement that reports ''pct'' distributions is measuring a legacy artefact, which is a legitimate thing to do provided you say so. |
| * **ARC (RFC 8617) is still Experimental**, and an IETF DMARC working-group document is moving it to Historic.((''draft-ietf-dmarc-arc-to-historic'', IETF DMARC working group: an Active Internet-Draft at revision **-00**, last updated **2026-04-22**, fetched from datatracker.ietf.org on 2026-09-09. A -00 revision is the start of a process, not the end of one — check the datatracker before writing that ARC is Historic.)) {[wang2022_revisiting]} is the only ARC measurement in this population. | * **ARC (RFC 8617) is still Experimental**, and an IETF DMARC working-group document is moving it to Historic.((''draft-ietf-dmarc-arc-to-historic'', IETF DMARC working group: an Active Internet-Draft at revision **-00**, last updated **2026-04-22**, fetched from datatracker.ietf.org on 2026-09-09. A -00 revision is the start of a process, not the end of one — check the datatracker before writing that ARC is Historic.)) {[wang2022_revisiting]} is the only ARC measurement in this population. |
| * **A DKIM revision is in progress.** ''draft-ietf-dkim-dkim2-spec'' is an Active Internet-Draft in the rechartered IETF ''dkim'' working group, last revised **2026-08-28**. The driver is DKIM replay — the weakness {[chen2020_composition]} demonstrated in 2020 with the ''l='' tag and message re-signing. | * **A DKIM revision is in progress.** ''draft-ietf-dkim-dkim2-spec'' is an Active Internet-Draft in the rechartered IETF ''dkim'' working group, last revised **2026-08-28**. The driver is DKIM replay — the weakness {[chen2020_composition]} demonstrated in 2020 with the ''l='' tag and message re-signing. |
| * **BIMI is still not an RFC.** ''draft-brand-indicators-for-message-identification'' is an Active Internet-Draft with no working-group ownership, last updated **2026-05-01**. Anything describing BIMI as a standard is wrong. | * **BIMI is still not an RFC.** ''draft-brand-indicators-for-message-identification'' is an Active Internet-Draft with no working-group ownership, last updated **2026-05-01**. Anything describing BIMI as a standard is wrong. |
| * **The three largest consumer mailbox providers now mandate authentication, on different terms.** Google has required SPF, DKIM //and// DMARC of senders above 5,000 messages a day since 1 February 2024, and its page now adds that //"Starting November 2025, Gmail is ramping up its enforcement on non-compliant traffic. Messages that fail to meet the email sender requirements will experience disruptions, including temporary and permanent rejections."// Yahoo mirrors the same requirements and the same February 2024 date but **never states a numeric volume threshold anywhere on its senders page**. Microsoft's Outlook.com requirement took effect **5 May 2025** above 5,000 messages a day, and as of today Microsoft still describes outright rejection as pending a future date rather than in force.((Google: https://support.google.com/a/answer/14229414 and https://support.google.com/mail/answer/81126; Yahoo: https://senders.yahooinc.com/best-practices/; Microsoft: the Defender for Office 365 blog announcement 4399730 and https://sendersupport.olc.protection.outlook.com/pm/policies.aspx. All fetched 2026-09-09. An earlier run of this wiki recorded the Microsoft announcement as blocked to automated fetch; it resolved cleanly today.)) **This matters to your sampling**: since 2024 the incentive to publish a DMARC record depends on how much mail a domain sends to Gmail, which is not a variable any of the 31 papers controls for. A post-2024 adoption series that does not separate bulk senders from everyone else is measuring a policy change, not a security trend. | * **The three largest consumer mailbox providers now mandate authentication, on different terms.** Google has required SPF, DKIM //and// DMARC of senders above 5,000 messages a day since 1 February 2024, and its page now adds that //"Starting November 2025, Gmail is ramping up its enforcement on non-compliant traffic. Messages that fail to meet the email sender requirements will experience disruptions, including temporary and permanent rejections."// Yahoo mirrors the same requirements and the same February 2024 date but **never states a numeric volume threshold anywhere on its senders page**. Microsoft's Outlook.com requirement took effect **5 May 2025** above 5,000 messages a day, and as of today Microsoft still describes outright rejection as pending a future date rather than in force.((Google: https://support.google.com/a/answer/14229414 and https://support.google.com/mail/answer/81126; Yahoo: https://senders.yahooinc.com/best-practices/; Microsoft: the Defender for Office 365 blog announcement 4399730 and https://sendersupport.olc.protection.outlook.com/pm/policies.aspx. All fetched 2026-09-09.)) **This matters to your sampling**: since 2024 the incentive to publish a DMARC record depends on how much mail a domain sends to Gmail, which is not a variable any of the 31 papers controls for. A post-2024 adoption series that does not separate bulk senders from everyone else is confounded by it. |
| |
| ===== Tools, and Whether They Are Alive ===== | ===== Tools, and Whether They Are Alive ===== |
| | ''parsedmarc'' | parse DMARC aggregate (''rua'') XML reports | **alive** — 11.0.1, released 2026-09-03 | | | ''parsedmarc'' | parse DMARC aggregate (''rua'') XML reports | **alive** — 11.0.1, released 2026-09-03 | |
| | ZMap / ZGrab2 / ZDNS | the internet-wide scanning stack; ZMap is how {[durumeric2015_neither]} found STARTTLS stripping | **alive** — ZDNS v2.1.1 (2026-05-28); ZGrab2 last tagged v1.0.0 (Dec 2025) but committing through 2026 | | | ZMap / ZGrab2 / ZDNS | the internet-wide scanning stack; ZMap is how {[durumeric2015_neither]} found STARTTLS stripping | **alive** — ZDNS v2.1.1 (2026-05-28); ZGrab2 last tagged v1.0.0 (Dec 2025) but committing through 2026 | |
| | ''libspf2'' | the C SPF validator embedded in several MTAs; the subject of {[bennett2022_spfail]} | **stalled and this is a finding.** No tag in the repository at all; the last version bump was 1.2.11 on 2021-06-09, and a commit titled //"Fix integer underflow"// landed 2023-10-04 and **has never been released**. Anyone measuring SPF-validator versions in the wild is measuring 2021 code. | | | ''libspf2'' | the C SPF validator embedded in several MTAs; the subject of {[bennett2022_spfail]} | **stalled and this is a finding.** No tag in the repository at all; the last version bump was 1.2.11 on 2021-06-09, and a commit titled //"Fix integer underflow"// landed 2023-10-04 and **has never been released**. Upstream has shipped no release since 2021, so the fix reaches deployments only through whatever distributions have backported it — which is itself an unmeasured question. | |
| | OpenDKIM / OpenDMARC | the reference milters | OpenDKIM dormant (last commit 2018-02-25); OpenDMARC last tagged 1.4.2 (2021) with bugfix commits into 2026 | | | OpenDKIM / OpenDMARC | the reference milters | OpenDKIM dormant (last commit 2018-02-25); OpenDMARC last tagged 1.4.2 (2021) with bugfix commits into 2026 | |
| | OpenARC | ARC milter. {[wang2022_revisiting]} wrote in 2022 that //"The OpenARC code has not been actively maintained since 2018."// | **Broadly true of the repository they meant, but a year off, and the sentence is now the wrong one to cite.** ''trusteddomainproject/OpenARC'': ''master'' last committed **2018-09-21**, ''develop'' last committed **2020-10-16**, newest tag ''rel-openarc-1-0-0-Beta3'' pointing at a commit of **2019-08-08** — still a beta, seven years on. **The maintained code is a fork under a different owner**: ''flowerysong/OpenARC'', **v1.3.0 released 2025-10-29**, seven releases since October 2024. Cite the fork. | | | OpenARC | ARC milter. {[wang2022_revisiting]} wrote in 2022 that //"The OpenARC code has not been actively maintained since 2018."// | **Broadly true of the repository they meant, but a year off, and the sentence is now the wrong one to cite.** ''trusteddomainproject/OpenARC'': ''master'' last committed **2018-09-21**, ''develop'' last committed **2020-10-16**, newest tag ''rel-openarc-1-0-0-Beta3'' pointing at a commit of **2019-08-08** — still a beta, seven years on. **The maintained code is a fork under a different owner**: ''flowerysong/OpenARC'', **v1.3.0 released 2025-10-29**, seven releases since October 2024. Cite the fork. | |
| This topic has a harder ethics profile than most web measurement, because half its instruments touch third-party production infrastructure and one of them delivers a forged message to a real mailbox. What the 31 papers report about themselves: | This topic has a harder ethics profile than most web measurement, because half its instruments touch third-party production infrastructure and one of them delivers a forged message to a real mailbox. What the 31 papers report about themselves: |
| |
| * **12 of 31 mention no ethics review at all**; 6 explicitly discuss the question and state that no review was sought or required; only 5 report an approval. That is 38.7% silent against **64.2%** corpus-wide (of the 4,965 papers carrying an ethics object) — so this population is //better// than the corpus, not worse, and 5 report an approval against 20.1% corpus-wide. //n// = 31, so read neither gap as a difference. Better than a low bar is still not good enough for an instrument that delivers forged mail to a live provider. | * **12 of 31 mention no ethics review at all.** Of the other 19: 6 discuss the question and state that no review was sought or required, 5 report an approval, 4 say review was not required, 2 sought one and do not say what came back, and 2 report an exemption. That is 38.7% silent against **64.2%** corpus-wide (of the 4,965 papers carrying an ethics object) — so this population is //better// than the corpus, not worse, and 5 report an approval against 20.1% corpus-wide. //n// = 31, so read neither gap as a difference. Better than a low bar is still not good enough for an instrument that delivers forged mail to a live provider. |
| * **19 of 31 notified affected parties**, 6 partially, 2 not, 3 do not say. Disclosure is the norm in this population — usefully so, since these are protocol and implementation flaws with named vendors. | * **19 of 31 notified affected parties**, 6 partially, 2 not, 3 do not say, and 1 is recorded not-applicable. Disclosure is the norm in this population — usefully so, since these are protocol and implementation flaws with named vendors. |
| * **20 of 31 report a public artifact and 18 give a code URL**, including several long-lived project sites (''dane-study.github.io'', ''spf-measurement.github.io'', ''mta-sts.netsecurelab.org'', ''github.com/chenjj/espoofer''). By the standards of this corpus that is good practice. | * **20 of 31 report a public artifact and 18 give a code URL**, including several long-lived project sites (''dane-study.github.io'', ''spf-measurement.github.io'', ''mta-sts.netsecurelab.org'', ''github.com/chenjj/espoofer''). By the standards of this corpus that is good practice. |
| |
| ===== What to Report ===== | ===== What to Report ===== |
| |
| A reviewer who works in this area will look for these. Most rejected drafts are missing three or more. | A reviewer who works in this area will look for these. |
| * **The frame, versioned.** Which list, which day, which version. A Tranco list ID, a zone-file date, a passive-DNS window. "The Alexa top 1M" is not a frame in 2026. | * **The frame, versioned.** Which list, which day, which version. A Tranco list ID, a zone-file date, a passive-DNS window. "The Alexa top 1M" is not a frame in 2026. |
| * **Whether you filtered to domains with an MX record**, and how many that removed. State the numerator and both candidate denominators. | * **Whether you filtered to domains with an MX record**, and how many that removed. State the numerator and both candidate denominators. |
| * **The resolver and where it sat.** Recursive or authoritative, your own or the provider's, and from what network. DNS answers to SPF ''include'' chains are geo- and resolver-dependent. | * **The resolver and where it sat.** Recursive or authoritative, your own or the provider's, and from what network. An SPF record's meaning is the transitive closure of other people's DNS answers — which is exactly why {[ashiq2024_beyond]} finds 6.5% of SPF-publishing domains blow the 10-lookup limit — so two runs behind different resolvers can disagree about the same record. |
| * **For DKIM: the selector list and its provenance.** And the words "lower bound". | * **For DKIM: the selector list and its provenance.** And the words "lower bound". |
| * **For SPF: whether you recursively expanded ''include'' and ''redirect''**, whether you counted the 10-lookup limit, and what you did with ''PermError''. {[czybik2023_lazy]} found 34.7% of SPF-publishing domains authorise more than 100,000 IPv4 addresses and {[wang2024_breakspf]} found 51.7% cover more than 65,536 — both figures exist only because the record was expanded, not just parsed. | * **For SPF: whether you recursively expanded ''include'' and ''redirect''**, whether you counted the 10-lookup limit, and what you did with ''PermError''. {[czybik2023_lazy]} found 34.7% of SPF-publishing domains authorise more than 100,000 IPv4 addresses and {[wang2024_breakspf]} found 51.7% cover more than 65,536 — both figures exist only because the record was expanded, not just parsed. |
| | * **The policy distribution, not just the presence rate.** ''p=none'' against ''quarantine'' against ''reject''; ''-all'' against ''~all'' against ''?all''. If you report one number for "DMARC deployment", say which policies it counts. |
| | * **For a delivery test: the full outcome ladder.** Rejected at SMTP / spam-foldered / inbox / inbox with a warning, and separately what the interface showed. Not a binary. |
| * **Sender-side or receiver-side.** Say which one your number is about, in the sentence with the number. | * **Sender-side or receiver-side.** Say which one your number is about, in the sentence with the number. |
| * **Snapshot or series, and the cadence.** DANE and MTA-STS misconfiguration is largely a rollover problem — {[lee2022_under]} needed hourly snapshots to see that, of the 2,569 DANE SMTP servers whose certificates were ever valid, more than 87% in each management category botch at least one key rollover — and a single snapshot cannot see it at all. | * **Snapshot or series, and the cadence.** DANE and MTA-STS misconfiguration is largely a rollover problem — {[lee2022_under]} needed hourly snapshots to see that, of the 2,569 DANE SMTP servers whose certificates were ever valid, more than 87% in each management category botch at least one key rollover — and a single snapshot cannot see it at all. |
| **2025–2026 are provisional** — CCS 2026 and IMC 2026 have not been held and two other 2026 venue-years are under-represented by construction (see [[literature:corpus]]), so read the 2026 cell as a floor. 28 of 31 are 2018 or later; 21 are 2022 or later. | **2025–2026 are provisional** — CCS 2026 and IMC 2026 have not been held and two other 2026 venue-years are under-represented by construction (see [[literature:corpus]]), so read the 2026 cell as a floor. 28 of 31 are 2018 or later; 21 are 2022 or later. |
| |
| By venue: USENIX Security 16, IMC 7, NDSS 4, and one each from CCS, IEEE S&P, TheWebConf and PETS. **This is a USENIX-and-IMC topic**, more concentrated than almost anything else on this wiki. | By venue: USENIX Security 16, IMC 7, NDSS 4, and one each from CCS, IEEE S&P, TheWebConf and PETS. **This is a USENIX-and-IMC topic**: 23 of the 31 are in those two venues. |
| |
| The authorship is more concentrated still, and this is the caveat that matters most for reading the tables above as if they described a field. Across 111 distinct authors on the 31 papers: | The authorship is more concentrated still, and this is the caveat that matters most for reading the tables above as if they described a field. Across 111 distinct authors on the 31 papers: |
| ==== Where these papers go quiet ==== | ==== Where these papers go quiet ==== |
| |
| * **29 of 31 report descriptive statistics only.** Two run a hypothesis test ({[hu2018_measurements]}, {[blechschmidt2023_hello]}), one a regression ({[szurdi2017_email]}). For a literature whose central claims are comparisons — notified against control, this provider against that one, this year against last — that is thin. If your result is a difference, test it; see [[statistics:hypothesis_testing]]. | * **29 of 31 report descriptive statistics only.** Two run a hypothesis test ({[hu2018_measurements]}, {[blechschmidt2023_hello]}). Exactly one paper runs anything else, and it runs three things: {[szurdi2017_email]} fits a regression (R² = 0.74), reports a correlation, and cross-validates (leave-one-out, R² drops to 0.63). One paper in eleven years accounts for every regression, every correlation and every resampling result in this population. For a literature whose central claims are comparisons — notified against control, this provider against that one, this year against last — that is thin. If your result is a difference, test it; see [[statistics:hypothesis_testing]]. |
| * **19 of 31 coded something by hand and not one reports an inter-rater agreement metric.** Zero of nineteen. Vulnerability labelling in a client matrix is a coding task, and "we manually analysed the identified behaviours" is not a method. See [[statistics:interrater_agreement]]. | * **19 of 31 coded something by hand and the extraction records an inter-rater agreement metric for none of them.** Zero of nineteen — but read that as an extraction result: the 19 papers were not re-read by hand to confirm the absence is theirs and not the extractor's, and it is the least-verified number on this page. Even discounted, the direction is not in doubt: vulnerability labelling in a client matrix is a coding task, and "we manually analysed the identified behaviours" is not a method. See [[statistics:interrater_agreement]]. |
| * **12 of 31 mention no ethics review**, against instruments that send forged mail and probe strangers' servers. | * **12 of 31 mention no ethics review at all** and a further 4 record it as not required, against instruments that send forged mail and probe strangers' servers. |
| * Eight of the 31 recruited people; the five operator surveys carry //n// of 39, 74, 16, 95 and 117. The extraction records a recruitment channel for only three of the five — but that undercounts by one: {[ashiq2023_report]} names MailOP, NANOG and MESSEU in its own text and the extractor recorded ''not-stated''. Four of five, and the fifth is {[blechschmidt2023_hello]}, whose 16 respondents are operators who replied to a notification email rather than a recruited sample at all. | * Eight of the 31 recruited people; the five operator surveys carry //n// of 39, 74, 16, 95 and 117. The extraction records a recruitment channel for only three of the five — but that undercounts by one: {[ashiq2023_report]} names MailOP, NANOG and MESSEU in its own text and the extractor recorded ''not-stated''. Four of five, and the fifth is {[blechschmidt2023_hello]}, whose 16 respondents are operators who replied to a notification email rather than a recruited sample at all. |
| |
| ==== Adjacent, and deliberately not in the 31 ==== | ==== Adjacent, and deliberately not in the 31 ==== |
| |
| These are near-misses the recall probe surfaced. Each carries a different verdict in ''msg_fold.mjs'', and where I think the boundary is genuinely arguable, it says so. | These are near-misses the recall probe surfaced. Each carries a different verdict in ''msg_fold.mjs'', and where the boundary is genuinely arguable the table says so. |
| |
| ^ Paper ^ Verdict it carries ^ Why it is not here ^ | ^ Paper ^ Verdict it carries ^ Why it is not here ^ |
| | {[rao2024_unfiltered]} — //Unfiltered: Measuring Cloud-based Email Filtering Bypasses//, TheWebConf 2024 | spam / filtering | **Arguable.** The measured object is an MX misconfiguration in the delivery path — 80% of organisations using a cloud filter can be bypassed by delivering straight to the hosting provider — and the instrument is SMTP probing. It is filed with filtering because filtering is what the bypass defeats. If you are working on delivery-path integrity, read it as if it were in this population. | | | {[rao2024_unfiltered]} — //Unfiltered: Measuring Cloud-based Email Filtering Bypasses//, TheWebConf 2024 | spam / filtering | **Arguable.** The measured object is an MX misconfiguration in the delivery path — 80% of organisations using a cloud filter can be bypassed by delivering straight to the hosting provider — and the instrument is SMTP probing. It is filed with filtering because filtering is what the bypass defeats. If you are working on delivery-path integrity, read it as if it were in this population. | |
| | {[li2025_hades]} — //HADES Attack//, NDSS 2025 | spam | Object is email blocklist manipulation, not an authentication mechanism. | | | {[li2025_hades]} — //HADES Attack//, NDSS 2025 | spam | **Also arguable.** Its object is DNS-based blocklist (DNSBL) //adoption and operation//, measured end to end from fifteen months of non-delivery reports — infrastructure, by the same argument as //Unfiltered//. It sits with spam because a blocklist is a spam control rather than an authentication or transport mechanism. If you are measuring what actually decides whether mail is delivered, read it: {[li2024_bounce]}, which //is// in the 31, finds 31.10% of bounces are blocklist hits, more than fourteen times the share caused by authentication failure. | |
| | {[wu2026_email]} — //One Email, Many Faces//, NDSS 2026 | the address as identifier | Alias and identity confusion, which is [[privacy:email_tracking]]'s subject. | | | {[wu2026_email]} — //One Email, Many Faces//, NDSS 2026 | the address as identifier | Alias and identity confusion, which is [[privacy:email_tracking]]'s subject. | |
| | {[zhang2024_inbox]} — //Inbox Invasion//, 2024 | spam / filtering | MIME ambiguity against attachment scanners: a content-inspection problem. | | | {[zhang2024_inbox]} — //Inbox Invasion//, 2024 | spam / filtering | MIME ambiguity against attachment scanners: a content-inspection problem. | |
| ==== Methodology and limitations of these figures ==== | ==== Methodology and limitations of these figures ==== |
| |
| Every count above is a count of **papers**, never of extraction tuples, with its denominator named in the same sentence or table cell. Free-text fields were folded before aggregating and the unmapped residue is published rather than dropped. The report script that produces every figure on this page is ''scripts/report_email_authentication.mjs''; it hand-keys the mechanism map, the instrument map and all 49 published figures, and it fails if any of those maps stops covering all 31 papers. | Every count above is a count of **papers**, never of extraction tuples, with its denominator named in the same sentence or table cell. Free-text fields were folded before aggregating and the unmapped residue is published rather than dropped. The report script that produces every figure on this page is ''scripts/report_email_authentication.mjs''; it hand-keys the mechanism map, the instrument map and all 55 published figures, and it fails if any of those maps stops covering all 31 papers. |
| |
| Two limitations specific to this page: | Two limitations specific to this page: |
| |
| * **Quote verification hits the two-column problem.** All 49 figures above were checked against ''paper.cols.txt''; **53 quote fragments** were needed to do it. Four figures carry a second fragment, and **three** of those four are genuine splices — a sentence interrupted by a table row, a figure caption or a column break, and unlocatable as one string in any of the three available renderings. The fourth second fragment is an independent corroborating quote, not a split sentence. At least one figure stated in the prose rather than in a table (the DANE rollover figure) sits on a spliced sentence too, so three is a floor for this page, not a total. Across the 31 papers' 206 extracted evidence quotes, 130 locate contiguously and 76 do not; a hand-read sample of the 76 found real sentences broken by page furniture, plus one case where the extractor's quote merges two adjacent sentences. Details and the sample on the provenance page. | * **Quote verification hits the two-column problem.** All 55 figures above were checked against ''paper.cols.txt''; **60 quote fragments** were needed to do it. Five figures carry a second fragment, and **three** of those five are genuine splices — a sentence interrupted by a table row, a figure caption or a column break, and unlocatable as one string in any of the three available renderings. The other two second fragments are independent corroborating quotations, not split sentences. At least one figure stated in the prose rather than in a table (the DANE rollover figure) sits on a spliced sentence too, so three is a floor for this page, not a total. Across the 31 papers' 206 extracted evidence quotes, 130 locate contiguously and 76 do not; a hand-read sample of the 76 found real sentences broken by page furniture, plus one case where the extractor's quote merges two adjacent sentences. Details and the sample on the provenance page. |
| * **The mechanism and instrument maps are mine, not the extraction's.** The schema has no field for "which deployed mail mechanism is this paper's object", so both tables are hand maps over 31 papers, made by reading each results section. They are reproducible in the sense that they are written down in the script; they are not independent of my judgement. | * **The mechanism and instrument maps are hand maps, not extraction fields.** The schema has no field for "which deployed mail mechanism is this paper's object", so both tables were made by reading each of the 31 results sections. They are reproducible in the sense that they are written down in the script and asserted complete against the population; they are not independent of the judgement that made them, and [[provenance:security:email_authentication]] names the four calls inside them that could reasonably have gone the other way. |
| |
| The full query log, the folds, the residue, the quote spot-checks, the external sources that were rejected and the reviewer findings are on **[[provenance:security:email_authentication]]**. Corpus-level caveats — how the 5,859 were selected, why 2025–2026 are provisional — are on [[literature:corpus]]. | The full query log, the folds, the residue, the quote spot-checks, the external sources that were rejected and the reviewer findings are on **[[provenance:security:email_authentication]]**. Corpus-level caveats — how the 5,859 were selected, why 2025–2026 are provisional — are on [[literature:corpus]]. |
| * [[security:phishing]] — mail as an attack vector, and the feeds that label it. | * [[security:phishing]] — mail as an attack vector, and the feeds that label it. |
| * [[security:tls_certificates]] — the web PKI, CT logs and the scanning instruments this page borrows. | * [[security:tls_certificates]] — the web PKI, CT logs and the scanning instruments this page borrows. |
| * [[programming:internet_scanning]] — ZMap, ZGrab and the ethics and rate limits of internet-wide probing. | * [[programming:internet_scanning]] — ZMap, ZGrab and the rate limits of internet-wide probing. //Written 2026-09-10.// |
| * [[design:website_selection]] — Tranco, zone files, and why a top list is not a sample. | * [[design:website_selection]] — Tranco, zone files, and why a top list is not a sample. |
| * [[design:longitudinal]] — snapshot cadence, which is what DANE and MTA-STS rollover measurement lives or dies by. | * [[design:longitudinal]] — snapshot cadence, which is what DANE and MTA-STS rollover measurement lives or dies by. |
