Measure The Web

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Notifying Websites ======

You crawled 100k sites and found that 8,000 of them leak something. Before you write the paper you have to decide what to do about those 8,000 operators, and the major venues now expect the paper to say what you decided.((See //Disclosure timelines and what venues expect// below: IMC, USENIX Security, IEEE S&P and CCS all require it in some form as of their 2026 calls; PoPETs sets out ethical principles but no disclosure requirement, and NDSS and TheWebConf are not addressed there.)) This page is about the mechanics of doing that at scale: **how you get a contact for a party you have never met, what response rate to plan for, how long to wait before publishing, and what to report so a reviewer accepts the result.**

It is not about one-off coordinated disclosure to a named vendor. Emailing Google's security team is a solved problem with a queue and an SLA; emailing 8,000 strangers is a measurement in its own right, with a delivery funnel, a control group and a p-value. Treat it as one.

<WRAP important>
The single most consequential design decision on this page is the **control group**. Sites get fixed for reasons that have nothing to do with you — automatic updates, unrelated maintenance, going offline. Every notification study without a control arm has attributed some of that background remediation to its own notifications — and **both randomised, control-arm experiments on network operators in this corpus found no significant effect from any treatment**, including treatments that observational studies had reported as effective {[lone2022_sav,qin2024_rov]}. Randomise, hold back an arm, and report both. Note what that does and does not mean: {[maass2021_effective]} is also a randomised controlled experiment — a full factorial design with a control group — and it measured 56.6% against 9.2% on German website owners. The lesson is not that notification never works. It is that a study without a control arm cannot tell you which of those two cases it is in.
</WRAP>

===== How much of the field does this, measured =====

From the publication corpus behind this site: 5,859 papers, seven venues (CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P), 2010–2026. Of the 5,118 empirical papers, **4,472 carry an ethics record** — the extraction emits one only when the paper says something about ethics at all, so that is the denominator below, and the 646 papers that say nothing whatsoever are excluded rather than silently counted as "did not notify".

^ ''ethics.notifiedAffectedParties'' ^ Papers ^ Share of 4,472 ^
| yes | 1,636 | 36.6% |
| partial | 524 | 11.7% |
| no | 194 | 4.3% |
| not-applicable | 1,001 | 22.4% |
| **not-stated** | **1,117** | **25.0%** |

So 48.3% notified somebody (''yes'' or ''partial''), 4.3% explicitly say they did not, and a quarter of the field simply does not address the question. Broken out by the populations this page is about:

^ Population (∧ has an ethics record) ^ N ^ yes ^ partial ^ no ^ not-applicable ^ not-stated ^
| empirical | 4,472 | 36.6% | 11.7% | 4.3% | 22.4% | 25.0% |
| measured the web platform | 1,362 | 33.2% | 11.7% | 7.0% | 19.5% | 28.6% |
| ran an automated web crawl | 992 | 29.8% | 13.7% | 9.4% | 15.9% | 31.1% |
| ran a network scan or probe | 858 | 37.6% | 23.4% | 4.2% | 10.7% | 24.0% |
| assessed compliance with a law | 376 | 42.0% | 14.6% | 9.8% | 21.5% | 12.0% |
| **crawled ∧ assessed a law** | **123** | 35.0% | 14.6% | **14.6%** | 15.4% | 20.3% |

Two things to take from this table. **Network measurement is ahead of web measurement**: scanning papers notify at 61.0% (''yes''+''partial'') against 43.5% for crawling papers, and are less than half as likely to say outright that they did not (4.2% vs 9.4%), as well as meaningfully less likely to leave it unanswered (24.0% vs 31.1%). The scanning community built the norm first, largely because Internet-wide scanning provoked complaints that forced the question. **Legal-compliance papers state a position most often** (only 12.0% ''not-stated''), and the bottom row is the population closest to whoever is reading this: 123 papers that both crawled the web and assessed a law, i.e. found a compliance violation across many sites. **14.6% of them say outright that they did not notify** — more than three times the corpus-wide 4.3%. That is usually a decision rather than negligence. A GDPR-violation finding across thousands of sites is the case where authors most often decide, and say, that individual notification is the wrong instrument, and route the finding to a regulator or to publication instead. If that is your situation, //deciding not to notify is a defensible position that you have to argue in the paper//, not an omission you can leave to the reader.

The direction of travel is unambiguous:

^ Indicator ^ Denominator ^ 2010–2013 ^ 2014–2017 ^ 2018–2021 ^ 2022–2024 ^ 2025–2026* ^
| Notified (''yes'' or ''partial'') | empirical ∧ ethics | 21.3% | 37.7% | 45.9% | 52.8% | 60.3% |
| Said nothing (''not-stated'') | empirical ∧ ethics | 61.5% | 40.2% | 26.5% | 16.9% | 13.1% |
| Notified, crawlers only | crawled ∧ ethics | 14.6% | 34.7% | 41.4% | 50.6% | 54.3% |
| Gave any disclosure detail | empirical ∧ ethics | 31.7% | 48.4% | 58.9% | 70.8% | 81.6% |
| Named a harm-mitigation step | empirical ∧ ethics | 46.4% | 61.1% | 65.0% | 76.8% | 85.0% |
| Contacted a regulator or CERT | empirical ∧ ethics | 0.3% | 3.0% | 4.7% | 3.4% | 2.8% |

<WRAP todo>* 2025–2026 is provisional: CCS 2026 and IMC 2026 have not been held, and IEEE S&P 2026 and TheWebConf 2026 abstracts are not in OpenAlex, so those venue-years are under-represented by construction. See [[Literature:Corpus]].</WRAP>

The one line that is not rising is the interesting one. **Regulator and CERT contact has not grown and remains rare — 3.3% of the 4,472 say yes, and only 7.2% of the 376 papers that assessed a law.** Given that a CERT is often the only party that can reach a whole constituency, and that EU law now obliges every member state to run one for exactly this purpose (below), this is the largest gap between what is available and what is used.

And when the intermediary //is// used it is used **as well as**, not instead of: of the 147 papers that contacted a regulator or CERT, **135 (91.8%) also notified the affected party directly**. Nobody in this corpus treats "we told the CERT" as discharging the obligation. Plan the intermediary as an extra arm, not as the cheap way out of finding contacts.

==== Saying you notified is not saying how ====

2,870 of the 4,472 (64.2%) give some free-text detail about their disclosure, and **all 2,160 that said they notified are among them**. Scoping to those 2,160 — a channel is only a fair question of a paper that says it notified somebody((Of the 710 papers that gave a detail without saying they notified, only 273 (38.5%) are participant-consent or debriefing notes; the rest are data-handling and harm-limitation notes, and a handful even name a channel. So the exclusion is about the question being ill-posed for that group, not about its text being all one kind.)) — and folding the text for any mention of a channel (the rules and their residue are on the provenance page):

**1,633 of the 2,160 (75.6%) do not say through what channel.** Where a channel does appear it is usually a single large platform named outright (14.7% mention Google, Apple, Meta, Microsoft, Amazon or Mozilla); a generic email to the operator or developer is 4.6%, a CERT 1.8%, a bug-bounty programme 1.6%, a hosting provider 1.6%, WHOIS 0.7%, a data protection authority 0.6%. And 2,155 of the 2,870 (75.1%) name no outcome either.

Worse for anyone trying to plan a campaign: **only 7 of the 2,870 report a countable response ratio in that text** — figures like "Contacted 318 of 559 sensitive organizations; received 11 responses by submission" or "Reported 110 malicious extensions to Google; 62.7% were removed". Every rate in the next section had to be read out of the papers' own results sections, not out of their ethics sections. That is the reporting gap this page exists to close: the modal paper in this corpus says //we disclosed responsibly// and stops.

===== Getting a contact: what still works in 2026 =====

This is where notification studies actually fail. Both 2016 campaigns concluded that reachability, not operator willingness, was the binding constraint {[stock2016_hey,li2016_youve]}, and the 2026 interview study of hosting providers opens by summarising the decade since as "the response and remediation rates for notified web vulnerabilities still average between 20 to 30%, and no effective alternatives have yet been identified (e.g., still no better channel than WHOIS)" {[stivala2026_behind]}. That sentence is a related-work summary carrying citations to four earlier papers, not a figure that study measured — but it is the field's own 2026 assessment of where it stands, and it is worth knowing that a 2026 paper still writes it.

==== The channel that carried this literature is gone ====

<WRAP important>
**Domain WHOIS is no longer a contact channel.** Both 2016 campaigns bought machine-readable WHOIS for the Alexa top million and used the registrant or technical address; it was the best channel in {[stock2016_hey]} — "the communication channel with the highest fix ratio (+15.4%)" relative to control, and far fewer bounces than the generic aliases, "since a valid email address is typically necessary to register a domain". Two things then removed it:

  * **GDPR, from May 2018.** ICANN's Temporary Specification told registrars to redact registrant, admin and tech contact fields. Measured over 1.2 billion WHOIS records, "over 85% [of] surveyed large WHOIS providers [were] redacting EEA records at scale" and "over 60% [of] large WHOIS data providers also redact non-EEA records" {[lu2021_whowas]}. The same paper surveyed 51 security papers using WHOIS and found 69% of them needed a field that is now redacted.
  * **The protocol itself, from 28 January 2025.** ICANN: "As of 28 January 2025, the Registration Data Access Protocol (RDAP) will be the definitive source for delivering generic top-level domain name (gTLD) registration information in place of sunsetted WHOIS services."((ICANN, //ICANN Update: Launching RDAP; Sunsetting WHOIS//, 27 January 2025, [[https://www.icann.org/en/announcements/details/icann-update-launching-rdap-sunsetting-whois-27-01-2025-en|icann.org]]. Fetched 2026-08-13. ICANN's replacement Registration Data Policy is separately announced as "Now In Effect for Contracted Parties"; the exact commencement date could not be pinned to a primary source and is therefore not stated here.)) Port-43 WHOIS for gTLDs is not a thing you should be building a 2026 pipeline on.

RDAP (RFC 9082/9083, discovery per RFC 9224) is the successor, but it is a **protocol** replacement, not a **data** replacement: it returns the same redacted fields, plus a documented mechanism for differentiated access. If you need the non-public registration data, ICANN's Registration Data Request Service (RDRS) is the front door, and it names "cybersecurity professionals" among those with a legitimate interest.((Same ICANN announcement. RDRS covers participating registrars only; for the rest you contact the sponsoring registrar directly.)) That is a per-request human process, so it is a channel for tens of domains, not for thousands.

**Any paper that cites the 2016 WHOIS result as a reason to use WHOIS is citing a finding about a system that no longer exists.**
</WRAP>

==== Channels that do work, with what each actually reaches ====

^ Channel ^ How you get it ^ What it reaches ^ Status and evidence ^
| **''security.txt''** (RFC 9116) | ''GET https://<domain>/.well-known/security.txt'' | the security team, if there is one | RFC 9116, Informational, April 2022 — current, not obsoleted.((Verified against [[https://www.rfc-editor.org/rfc/rfc9116.txt|rfc-editor.org/rfc/rfc9116.txt]] on 2026-08-13: "Category: Informational … April 2022". Three errata exist, none touching the field list.)) Adoption is the problem: 11–16% of the Alexa top 100, 8–10% of the top 1K, "3–4% for the top 10K sites, and only a percent for the top 100K" {[poteat2021_securitytxt]}. **That measurement is from 2021 and its ranking frame no longer exists** — Alexa was discontinued in 2022 (see [[Design:Website selection]]), the paper predates RFC 9116, and **nothing in this corpus re-measures security.txt adoption since**. Assume it has risen; you have no citeable current figure |
| **RIR abuse contact** | RIPE ''abuse-c:'' / ARIN Abuse POC, resolved from the IP; RIPEstat's abuse-contact-finder covers all five RIRs over HTTPS | the **hosting provider or CDN**, not the site owner | The one source that scales — free and with no meaningful rate limit. ARIN verifies its Abuse POC annually;((ARIN NRPM §3.6: "Each of the following Points of Contact are to be verified annually … Admin, Tech, NOC, Abuse", [[https://www.arin.net/participate/policy/nrpm/|arin.net]], fetched 2026-08-13. RIPE states it works to keep abuse contacts valid but no validation cadence could be found on a RIPE primary source on 2026-08-13 — do not assume annual.)) RIPE states it keeps abuse contacts valid but publishes no cadence that could be verified from a RIPE source. Still the workhorse: "WHOIS was the most frequently mentioned channel (seven HPOs)" among 24 interviewed providers in 2026, who "confirmed that their contact is available via WHOIS, for example through ARIN or RIPE databases" {[stivala2026_behind]}. Note this is the //network// registry, which GDPR redaction did not touch — not the //domain// registrant record, which it did |
| **PeeringDB technical contact** | PeeringDB API, per AS | the network operator | Preferred over WHOIS by {[lone2022_sav]}: "We preferred peeringDB because it has been used in previous studies and they found the database up-to-date" — i.e. it is relaying prior work's assessment, not measuring it |
| **The site's own imprint / privacy-policy contact** | read it off the page | the legally responsible party — often the actual decision-maker | The highest-yield channel measured. Addresses parsed from privacy-policy and contact pages achieved **87.8% delivery against 33.8% for RFC 2142 aliases** {[utz2023_comparing]}; 9 of 11 organisations reached via a privacy-policy address resolved the issue {[elyadmani2025_keys]}. {[maass2021_effective]} collected German ''Impressum'' addresses by hand, three researchers per site |
| **RFC 2142 role aliases** (''security@'', ''abuse@'', ''webmaster@'', ''info@'') | construct from the domain | whoever reads that mailbox, if anyone | RFC 2142 still current, still unrevised since 1997. Cheap and weak: 33.8% delivery {[utz2023_comparing]}, and for half the WordPress domains in {[stock2016_hey]} //every// alias bounced |
| **''postmaster@''** | construct from the domain | the mail administrator | Required by the SMTP specification, which is why {[bennett2022_spfail]} chose it for mail-server issues — still 31.6% undelivered |
| **National CSIRT as CVD coordinator** | FIRST's team directory; in the EU, your member state's designated coordinator | a whole constituency at once, indirectly | **NIS2 makes this a legal obligation.** Article 12(1): "Each Member State shall designate one of its CSIRTs as a coordinator for the purposes of coordinated vulnerability disclosure", whose tasks "shall include: (a) identifying and contacting the entities concerned; (b) assisting the natural or legal persons reporting a vulnerability; and (c) negotiating disclosure timelines and managing vulnerabilities that affect multiple entities".((Directive (EU) 2022/2555, Article 12, verified against [[https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32022L2555|EUR-Lex]] on 2026-08-13. Member states may also let you report anonymously.)) That is a description of the service a large-scale notifier needs, written into law |
| **Established notification operators** | Shadowserver Foundation, CERT-BUND and equivalents | operators who already trust the sender | Shadowserver still sends free daily reports to 201 national CSIRTs across 175 countries.((Fetched from [[https://www.shadowserver.org/what-we-do/network-reporting/|shadowserver.org]] on 2026-08-13.)) {[munteanu2025_catch22]} ran its whole campaign this way and thanks Shadowserver and CERT-BUND for it — the current worked example |
| **Platform consoles** | the platform's own channel to its registered operators | only operators who pre-registered | The strongest single effect in the literature, and the least available to you: a Search Console message lifted remediation to 82.4% / 76.8% against 54.6% / 43.4% otherwise {[li2016_remedying]}. Only 22–32% of affected sites had a registered webmaster, and you are not Google |
| **HackerOne Disclosure Assistance** | ''hackerone.com/disclosure-assistance'' | organisations with no disclosure policy at all | Newer than the literature: HackerOne "will work with friendly hackers on a **best-effort basis**" to verify a bug, find someone at the affected organisation and relay it.((Fetched from [[https://docs.hackerone.com/en/articles/8466632-disclosure-assistance|docs.hackerone.com]] on 2026-08-13; page dated 11 June 2024.)) It updates {[stock2016_hey]}, which discarded reward programmes because they "usually only accept and forward reports for their customers" — that premise no longer holds for HackerOne. But it is a per-report human service gated on having exhausted other options, so it is **not** a bulk channel |

Scraping ''mailto:'' links off pages **does not scale** in a way the literature is explicit about: unreliable, defeated by CAPTCHAs and obfuscation, and it collects addresses nobody published for this purpose. {[stock2016_hey]} discarded it, along with web contact forms and telephone, as "not a viable option in a large-scale scenario".

<WRAP info>
**"Does not scale" is a function of your N, and the highest rates in this literature come from channels that do not scale.** {[sasaki2022_ics]} notified 160 ICS operators **by manual phone call** — with a dedicated team, over three and a half months — and got the person actually responsible for 212 devices, 50% stating they would mitigate and a 58% confirmed reduction in exposed devices against 13% for un-notified ones. {[maass2021_effective]} read 4,594 ''Impressum'' addresses by hand, three researchers per site, and posted paper letters. Both are far above the 5–20% band that automated email campaigns report.

If your finding affects hundreds of parties rather than tens of thousands, **the manual channel is the method**, and the tooling in this section is the wrong tool. The trade is a real one: {[sasaki2022_ics]} publishes its cost per notification, and {[maass2021_effective]} spent about €5,000 on postage. Decide which you are doing before you build a pipeline.
</WRAP>

==== A tested contact-discovery cascade ====

Two of those channels are machine-queryable per domain, and RDAP is worth querying even though it is mostly redacted, because what it returns tells you //which kind// of redaction you are up against. (PeeringDB is machine-queryable too, but per AS — use it when your unit is a network operator rather than a website.) This runs the three in cost order and prints per-source coverage, because a pooled "we found contacts for N% of domains" is the figure a reviewer sends back. Failures are printed rather than swallowed: a silent zero and a missing contact look identical, and the difference understates your own coverage.

<file python find_contacts.py>
#!/usr/bin/env python3
"""Enumerate the contact points for a website that are still machine-queryable.

Three sources, cheapest first, each reported separately so you can publish a
per-source coverage rate instead of one pooled "we found contacts for N% of
domains" -- the pooled figure is the one reviewers send back.

  python3 find_contacts.py example.com example.org

Deliberately NOT included, and why:
  * domain WHOIS on port 43 -- sunset for gTLDs on 2025-01-28; RDAP is now the
    definitive source (ICANN, "Update: Launching RDAP; Sunsetting WHOIS").
  * the registrant email inside RDAP -- redacted for EEA registrants since the
    2018 ICANN Temporary Specification and, in practice, for most others. What
    survives is a registrar forwarding address or a web form. Ask for the
    non-public record through ICANN's RDRS rather than scraping around it.
  * scraping mailto: links off the page -- highest yield, worst measured
    delivery, and it collects addresses nobody published for this purpose.

Why RIPEstat rather than the Abusix Abuse Contact DB for the RIR abuse contact:
Abusix is a DNS TXT lookup, free and unmetered, and is what the 2016 USENIX
campaign used, so it is the better choice at real scale. RIPEstat is used here
because it needs nothing but the standard library over HTTPS, which keeps this
script runnable -- and therefore testable -- anywhere. Swap in Abusix once you have
a DNS client and thousands of domains.

Failures are printed, never swallowed: a silent zero looks identical to a
missing contact and would understate your own coverage. The one exception is the
IANA bootstrap fetch, which raises: it is not a per-domain zero, it makes RDAP
coverage unmeasurable for the whole batch.
"""

import json
import re
import socket
import sys
import urllib.error
import urllib.request

# Put a real address here. Operators who look up who is probing them and find
# nothing are the ones who complain to your institution instead of to you.
UA = "measuretheweb-contact-probe (research; contact: you@example.edu)"
TIMEOUT = 15
BOOTSTRAP = "https://data.iana.org/rdap/dns.json"  # RFC 9224


def _get(url, accept="*/*"):
    req = urllib.request.Request(url, headers={"User-Agent": UA, "Accept": accept})
    with urllib.request.urlopen(req, timeout=TIMEOUT) as r:
        return r.read().decode("utf-8", "replace")


def security_txt(domain):
    """RFC 9116. /.well-known/security.txt is the correct location; /security.txt
    is the legacy fallback the RFC keeps for compatibility."""
    for path in ("/.well-known/security.txt", "/security.txt"):
        try:
            body = _get(f"https://{domain}{path}", "text/plain")
        except (urllib.error.URLError, socket.timeout, OSError) as e:
            yield ("security.txt", None, f"{path}: {type(e).__name__}")
            continue
        # Servers that return their 404 page as 200 text/html are common enough
        # that a Contact: line is the only reliable signal the file is real.
        contacts = re.findall(r"^Contact:\s*(\S+)", body, re.M | re.I)
        expires = re.findall(r"^Expires:\s*(\S+)", body, re.M | re.I)
        if not contacts:
            yield ("security.txt", None, f"{path}: 200 but no Contact: field")
            continue
        note = f"expires={expires[0] if expires else 'MISSING (required by RFC 9116)'}"
        for c in contacts:
            yield ("security.txt", c, note)
        return


_BOOTSTRAP_CACHE = {}


def _rdap_base(tld):
    if not _BOOTSTRAP_CACHE:
        # Fetched once per run. A failure here is NOT a per-domain coverage zero
        # -- it means RDAP coverage is unmeasurable for the whole batch -- so it
        # raises with the URL rather than degrading into a row of dashes that
        # would read as "no contact found" for every domain in the sample.
        try:
            services = json.loads(_get(BOOTSTRAP))["services"]
        except (urllib.error.URLError, socket.timeout, OSError, json.JSONDecodeError, KeyError) as e:
            raise RuntimeError(
                f"RDAP bootstrap registry unreachable ({BOOTSTRAP}): {type(e).__name__}: {e}. "
                "Refusing to continue: every RDAP row would print as 'not found' and "
                "understate coverage. Retry, or run with RDAP excluded and say so."
            ) from e
        for tlds, urls in services:
            for t in tlds:
                _BOOTSTRAP_CACHE[t] = urls[0].rstrip("/")
    return _BOOTSTRAP_CACHE.get(tld)


def rdap_domain(domain):
    """RFC 9082/9083, discovered through the IANA bootstrap registry (RFC 9224).
    No RDAP base for the TLD means the ccTLD registry runs none -- that is a
    coverage fact about your sample, so it is reported rather than skipped."""
    tld = domain.rsplit(".", 1)[-1]
    base = _rdap_base(tld)
    if base is None:
        yield ("RDAP", None, f".{tld} has no RDAP service in the IANA bootstrap")
        return
    try:
        data = json.loads(_get(f"{base}/domain/{domain}", "application/rdap+json"))
    except (urllib.error.URLError, socket.timeout, OSError, json.JSONDecodeError) as e:
        yield ("RDAP", None, f"{base}: {type(e).__name__}")
        return
    emails = 0
    for ent in data.get("entities", []):
        roles = ",".join(ent.get("roles", [])) or "no-role"
        for item in ent.get("vcardArray", [[], []])[1]:
            if item[0] == "email":
                emails += 1
                yield ("RDAP", item[3], f"role={roles}")
    if emails == 0:
        yield ("RDAP", None, "response carried no email -- redacted, or role-only")


def abuse_contact(domain):
    """The RIR abuse contact for the IP the domain resolves to (RIPE abuse-c:,
    ARIN Abuse POC). RIPEstat resolves it across all five RIRs over HTTPS, so
    it needs no DNS TXT client and is not rate-limited into uselessness the way
    port-43 WHOIS was. This is the one contact source that scales, and it
    reaches the hosting provider -- not the site owner."""
    try:
        ip = socket.gethostbyname(domain)
    except OSError as e:
        yield ("RIR abuse", None, f"DNS: {type(e).__name__}")
        return
    try:
        d = json.loads(
            _get(f"https://stat.ripe.net/data/abuse-contact-finder/data.json?resource={ip}")
        )["data"]
    except (urllib.error.URLError, socket.timeout, OSError, json.JSONDecodeError, KeyError) as e:
        yield ("RIR abuse", None, f"RIPEstat: {type(e).__name__}")
        return
    contacts = d.get("abuse_contacts") or []
    if not contacts:
        yield ("RIR abuse", None, f"ip={ip} rir={d.get('authoritative_rir')} no abuse-c")
    for c in contacts:
        yield ("RIR abuse", c, f"ip={ip} rir={d.get('authoritative_rir')}")


SOURCES = ("security.txt", "RDAP", "RIR abuse")


def main(domains):
    hit = {s: set() for s in SOURCES}
    print(f"{'domain':24} {'source':13} {'contact':40} note")
    for d in domains:
        for src, contact, note in (*security_txt(d), *rdap_domain(d), *abuse_contact(d)):
            if contact:
                hit[src].add(d)
            print(f"{d:24} {src:13} {(contact or '--')[:40]:40} {note}")
    print(f"\ncoverage, domains with >=1 usable contact, of {len(domains)}:")
    for s in SOURCES:
        print(f"  {s:13} {len(hit[s]):3}  {100 * len(hit[s]) / len(domains):5.1f}%")
    union = set().union(*hit.values())
    print(f"  {'any source':13} {len(union):3}  {100 * len(union) / len(domains):5.1f}%")


if __name__ == "__main__":
    if len(sys.argv) < 2:
        sys.exit(__doc__)
    main(sys.argv[1:])
</file>

Real output, run 2026-08-13 on five hand-picked domains — **not a sample, and not representative**: these are large, well-resourced organisations, so ''security.txt'' coverage here is an order of magnitude above the 3–4% {[poteat2021_securitytxt]} measured at the top 10K.

<code>
domain                   source        contact                                  note
google.com               security.txt  https://g.co/vulnz                       expires=2030-04-01T00:00:00z
google.com               security.txt  mailto:security@google.com               expires=2030-04-01T00:00:00z
google.com               RDAP          —                                        response carried no email — redacted, or role-only
google.com               RIR abuse     network-abuse@google.com                 ip=172.217.208.139 rir=arin
ethz.ch                  security.txt  mailto:security@ethz.ch                  expires=2028-01-31T07:00:00.000Z
ethz.ch                  RDAP          —                                        .ch has no RDAP service in the IANA bootstrap
ethz.ch                  RIR abuse     abuse@ethz.ch                            ip=129.132.19.216 rir=ripe
cispa.de                 security.txt  —                                        /.well-known/security.txt: HTTPError
cispa.de                 security.txt  —                                        /security.txt: HTTPError
cispa.de                 RDAP          —                                        .de has no RDAP service in the IANA bootstrap
cispa.de                 RIR abuse     abuse@she.net                            ip=212.21.165.114 rir=ripe
bbc.co.uk                security.txt  mailto:security@bbc.co.uk                expires=2038-01-19T03:14:07Z
bbc.co.uk                RDAP          redacted@nominet.uk                      role=registrant
bbc.co.uk                RIR abuse     abuse@fastly.com                         ip=151.101.128.81 rir=arin
wikipedia.org            security.txt  mailto:security@wikimedia.org            expires=2029-03-31T09:00:00.000Z
wikipedia.org            RDAP          —                                        response carried no email — redacted, or role-only
wikipedia.org            RIR abuse     abuse@wikimedia.org                      ip=185.15.58.224 rir=ripe

coverage, domains with >=1 usable contact, of 5:
  security.txt    4   80.0%
  RDAP            1   20.0%
  RIR abuse       5  100.0%
  any source      5  100.0%
</code>

Four failure modes worth reading off that output before you build the pipeline:

  * **RDAP gives you nothing useful.** ''google.com'' and ''wikipedia.org'' return no email at all; ''bbc.co.uk'' returns the literal placeholder ''redacted@nominet.uk''. Post-2018 redaction, in the response.
  * **ccTLDs often have no RDAP.** ''.ch'' and ''.de'' have no service in the IANA bootstrap registry. If your sample is a national top list, RDAP coverage may be zero and it will not be your bug.
  * **The RIR abuse contact reaches the infrastructure, not the site.** ''cispa.de'' resolves to its hosting provider's ''abuse@she.net''; ''bbc.co.uk'' sits behind Fastly and yields ''abuse@fastly.com''. For any CDN-fronted site the abuse contact is the CDN — and CDN-fronting is now the common case, which is a structural reason this channel has decayed since 2016. Report what fraction of your abuse contacts belong to a CDN or shared host rather than to the party you meant to reach.
  * **A transient failure looks exactly like an absent contact.** An earlier run of exactly this script, on exactly these five domains, printed ''RIPEstat: TimeoutError'' for ''google.com'' and so reported 80% rather than 100% RIR-abuse coverage. Nothing about the sample changed. Distinguish a network failure from a missing contact and retry it, or your denominator is wrong — and note that the script **refuses to run at all** if the IANA bootstrap registry is unreachable, because degrading that into a column of dashes would understate RDAP coverage for every domain at once.

===== Response and remediation rates to plan for =====

{[sasaki2022_ics]} collects the comparison itself, and it is the shortest statement of the range: "its remediation rate was approximately 18% … Our remediation rate is higher than most previous notification experiments: it was approximately 40% for cross-site scripting and a WordPress vulnerability, 33%–42% for different WordPress vulnerability, and less than 20% for DNS zone poisoning. The only campaigns that reported similar remediation rates were on publicly accessible Git repositories (78%–81%) and Heartbleed (approximately 40%–90%)."

These are the campaigns in the corpus that reported a quotable outcome, each with **its own denominator** — they are not comparable to each other, because "response", "remediation" and "fix" are defined differently in each and the populations are wildly different. Plan against the range, not the mean. The hand pass classified 22 campaigns in total; 16 are below, and the six not shown reported a volume sent but no response or remediation figure. One row below — {[munteanu2025_catch22]} — is here for its channel rather than a rate, because delegating a campaign to an established notification operator is the pattern, not the number. They are listed on [[provenance:practices:notifying_websites]], along with the one row below — {[stock2018_didnt]} — whose full text is missing from the corpus and was read from the publisher's PDF instead.

^ Study ^ Notified ^ Channel ^ Outcome, in the paper's own terms ^ Control ^
| {[canali2013_hosting]} TheWebConf 2013 | global and regional hosting providers | abuse notification to the provider | "50% of both the global and regional web hosting providers never replied to any of the real abuse notifications we sent" | — |
| {[durumeric2014_heartbleed]} IMC 2014 | "150,000 hosts", staged in two groups | WHOIS abuse contact for the host IP | "a nearly 50% increase in patching by notified hosts"; 20.6% patching within 24 h and **39.5% after eight days**, Fisher's exact p ≈ 0 | 10.8% / **26.8%** (Group B, not yet notified) |
| {[stock2016_hey]} USENIX Sec 2016 | 44,790 vulnerable sites, five equal arms | RFC 2142 aliases / purchased domain WHOIS / provider abuse contacts / CERTs | of 35,832 reports sent, **2,064 (5.8%) actually received**; 74.5% still exploitable after a month; but ~40% fix //if the report is read// | 23.3% (WordPress), 2.2% (client-side XSS) |
| {[li2016_youve]} USENIX Sec 2016 | 2,563 ICS + 3,536 IPv6 + 5,960 amplifier contacts | WHOIS abuse contact, national CERTs, US-CERT | "at most 18% of the population remediating" under the best regimen; direct verbose 9.8% vs national CERT 3.1% vs US-CERT 1.4% (IPv6, two days) | US-CERT arm statistically indistinguishable from control |
| {[li2016_remedying]} TheWebConf 2016 | 760,935 hijacking incidents | browser interstitial, search warning, Search Console message, WHOIS admin email | 59.5% of incidents resolved over 11 months; **82.4% / 76.8% where a Search Console alert reached a pre-registered webmaster** vs 54.6% / 43.4% otherwise | — (channel comparison) |
| {[stock2018_didnt]} NDSS 2018 | >24,000 domains, seven arms of ~4,000 | six email variants (plain, HTML, tracking, mailbot, S/MIME, friendly tone) | notified 24% (Git) and 17% (WordPress) fixed; **74.4% / 33.3% among those who //viewed// the report** | 13% (Git), 14% (WordPress) |
| {[cetin2019_cleaning]} NDSS 2019 | ISP customers with Mirai infections | ISP walled garden (quarantine + notification) vs email-only vs nothing | walled garden "remediates 92% of the infections within 14 days"; **"Email-only notifications have no observable impact compared to a control group"** | **"The control group achieved the lowest cleanup rate (74%)"** with no notification at all — the cautionary figure on this whole page |
| {[maass2021_effective]} USENIX Sec 2021 | 4,594 German site owners, 18 arms + control | postal letter and email, addresses read by hand from each ''Impressum'' | **"56.6 % of all notified operators remediating within two months"**; 76.3% for a legal-research-group letter citing fines, 33.9% for a computer-science email citing privacy | **9.2%** |
| {[nguyen2021_sharefirst]} USENIX Sec 2021 | 11,914 app developers | contact address from the Play Store listing | **448 responses (3.8%)** | — |
| {[squarcina2021_subdomain]} USENIX Sec 2021 | sites with dangling subdomain records | direct contact vs the authors' national CERT | direct 31% / 22% fixed vs national CERT 10% / 14% | — |
| {[lone2022_sav]} IEEE S&P 2022 | 2,320 network operators, RCT | 8 treatments: direct email (PeeringDB→WHOIS→abuse), national CERT/NIC.br, NOG mailing lists, × nudges | **"none of the notification treatments significantly improved SAV deployment compared to the control group"** | remediation observed in control too |
| {[sasaki2022_ics]} IEEE S&P 2022 | 160 operators of 317 exposed ICS devices | **manual telephone calls**, email only for scheduling — "the first study to directly contact the organization operating the device" | reached the person in charge for 212 devices; **"50% of the persons in charge … stated that they mitigated or will mitigate"**, and follow-up scans confirmed devices "reduced by 58% when we were able to contact the persons in charge" | 13% decrease among un-notified devices, χ² p<0.0001 |
| {[bennett2022_spfail]} IMC 2022 | 6,488 mail-server notifications | ''postmaster@'' per the SMTP spec | 31.6% undelivered; of 4,434 delivered, 512 (12%) opened, 177 (4%) patched, **9 (<1%) between private and public disclosure** | — |
| {[utz2023_comparing]} PoPETs 2023 | 159,035 domains, 4 privacy issues + 1 security issue | parsed privacy-policy/contact addresses vs RFC 2142 aliases | **87.8% vs 33.8% delivery**; remediation effects significant but small — the paper puts them at "0–1" to "1–2 percentage points" over control (Fisher's exact, Holm–Bonferroni), with a few later-date and generic-alias cells around 3 pp | yes, per issue |
| {[qin2024_rov]} NDSS 2024 | 1,012 non-deploying ASes randomised into 6 treatment arms + control; **859 emails sent** | operator email from PeeringDB, falling back to WHOIS; nudge variants (baseline, social norms, authority, reminder, elicitation) and native language | 824 of 859 delivered, **4.07% bounce rate** against the ">50%" it cites from prior work — and **"none of the notification treatments has a significant effect"** (survival analysis; relative risk 0.46–1.35, every confidence interval spanning 1) | 11 of 138 remediated |
| {[elyadmani2025_keys]} IEEE S&P 2025 | 160 organisations leaking cloud-bucket secrets | leaked-file contents, OSINT, disclosure programmes, privacy-policy addresses — routed through a CSIRT partner | **95/160 (59.4%) acted**; only 20 organisations replied at all | — |
| {[munteanu2025_catch22]} USENIX Sec 2025 | operators of compromised hosts | Shadowserver Foundation and CERT-BUND ran the campaign | the current worked example of delegating delivery | — |

==== The loss is in delivery, not in willingness ====

The delivery funnel is the story, and it is why headline remediation rates look so bad. {[stock2016_hey]}: **5.8% of reports received**, but ~40% remediation among those actually read. {[bennett2022_spfail]}: 31.6% undelivered → 12% of the delivered opened → 4% of the openers patched. {[stock2018_didnt]}: 74.4% fixed among Git operators who //viewed// the report, against a 13% control. Operators who read your report largely act on it. Most never read it.

So the highest-leverage thing you can do is not writing a better email. It is:

  - **Use a channel with measured delivery.** Parsed privacy-policy/contact addresses at 87.8% against RFC 2142 aliases at 33.8% {[utz2023_comparing]} is the largest single lever in this literature. A hand-read ''Impressum'' is better still {[maass2021_effective]}.
  - **Instrument the funnel.** Report sent → bounced → delivered → opened → report viewed → remediated, separately. {[stock2016_hey]} classified every domain as reached / bounced / unreachable / unknown, and found "more than half of all domains are marked as unknown" — silently bounced, delivered to an unmonitored mailbox, or ignored. That //unknown// bucket is the honest reporting of your own blind spot; publish it.
  - **Expect spam filtering to fight you.** {[stock2016_hey]} received bounces for 562 domains stating the mail was classified as spam, despite a clean mail server. Warm up the sender, sign the mail, and measure the spam-rejection rate as a separate category. {[stock2018_didnt]} tested S/MIME signing directly and it did not beat the alternatives, so sign for legitimacy, not for delivery.
  - **Put the whole report in the message.** Both 2016 campaigns used a link to a per-domain report so they could measure views; {[stock2018_didnt]} concluded the external link "played a minor role", and Git operators who opened the mail but never viewed the report still fixed at 24.7% against a 13.0% control. Tracking your funnel and maximising remediation pull in opposite directions — say which you chose.

==== What moves the rate, what does not, and where the literature disagrees ====

^ Factor ^ Finding ^ Source ^
| Sender identity | Legal research group beats computer-science group: 59.7% vs 54% remediation (p<0.05) | {[maass2021_effective]} |
| Framing | Legal-compliance-plus-fine beats plain GDPR beats privacy-harm; survival 50.1% / 56.6% / 69.6%, all differences significant. //Survival// throughout this table is the share **still non-compliant**, so lower is better | {[maass2021_effective]} |
| Framing | **Contradicted.** A fine warning had **no** significant effect once the notification arrived; only //receiving// it mattered | {[utz2023_comparing]} |
| Medium | A **postal letter** beats email: survival 55.6% vs 66.3% (p<0.0001), "increasing the remediation rate by between 3.9 and 17.9 percentage points (mean: 11.1)" — for "around 5000 € on domestic postage" across 2,660 letters | {[maass2021_effective]} |
| Message variant | Plain vs HTML vs tracking-pixel vs mailbot vs S/MIME vs friendly tone: for Git all beat control, for WordPress only one did, and **"no group performed significantly better than all others"** | {[stock2018_didnt]} |
| Verbosity | Verbose beat terse by 55–57% in effect size, but **not significantly** after Bonferroni correction | {[li2016_youve]} |
| Localisation | **English beat translation.** Translated messages did worse; recipients "initially suspected our notifications were phishing messages or spam" | {[li2016_youve]} |
| Localisation | Consistent: remediation varied only 7–10 pp across recipient languages | {[li2016_remedying]} |
| Reminders | Repeat notifications to **network operators** had no effect at all | {[li2016_youve]} |
| Reminders | For **webmasters**, the first reminder added ~7 pp at two weeks and ~11 pp at four; the second added nothing. The effective window is longer for webmasters than for network operators | {[stock2018_didnt]} |
| Reminders | A reminder took remediation from 41.2% to 56.6% — a third of the total effect | {[maass2021_effective]} |
| Offering support | A self-service checking tool was used by 46.9% of notified owners, and by 67.6% of those who became compliant | {[maass2021_effective]} |
| Intermediaries | Direct beats CERT beats regional coordinator, consistently and by large margins | {[li2016_youve,squarcina2021_subdomain]} |
| Intermediaries | **Contradicted.** Under an RCT, the trusted national coordinator arm showed no effect over control either | {[lone2022_sav]} |
| Decay | "If recipients did not act within the first couple days, they were unlikely to ever do so" | {[li2016_youve]} |
| Persistence | 7 months later, 3.5% of remediated sites had regressed — "long-term effectiveness of approximately 95%" | {[maass2021_effective]} |
| Public disclosure | Private notification "made little difference"; the **public CVE 60 days later** correlated with a much larger drop in vulnerable servers | {[bennett2022_spfail]} |

Read the two contradictions as the state of the art rather than as noise. **Message-content tuning is a dead end**: three studies varied wording, format, signing, tone and translation, and the effects are small, inconsistent, or vanish under multiple-comparison correction. **Sender identity, medium and reachability are the real variables** — and even those may not survive a control group, which is what {[lone2022_sav]} demonstrated on the one population where somebody randomised properly.

<WRAP important>
Security issues and privacy issues are not the same notification problem. {[utz2023_comparing]} ran both in one design: recipients were aware of the issue beforehand 81.4% of the time for a public Git repository against 56.8% for a privacy violation, planned to fix it 90.7% against 64.9%, thanked the researchers 74.9% against 56.0%, and were more than twice as likely to express negative sentiment about the privacy notification. If your finding is a compliance finding, budget for a lower rate, a slower fix, and more hostility.
</WRAP>

===== Disclosure timelines and what venues expect =====

There is no single norm, so state which one you followed and why. The reference points, all verified as current on 2026-08-13:

  * **CERT/CC: 45 days** from the initial report — "regardless of the existence or availability of patches or workarounds from affected vendors".((CERT/CC Vulnerability Disclosure Policy, [[https://certcc.github.io/certcc_disclosure_policy/|certcc.github.io]].))
  * **Google Project Zero: 90 days to patch plus 30 days before technical detail is published**, with a 14-day grace period on request; **7 days** for bugs under active exploitation. Since a trial that began 29 July 2025, Project Zero also publishes within about a week that a report exists — vendor, product, date, deadline — without the details.(([[https://projectzero.google/vulnerability-disclosure-policy.html|projectzero.google]] and its Reporting Transparency page. Note the ''googleprojectzero.blogspot.com'' URL now redirects here; do not cite the old one.))
  * **ISO/IEC 29147** (disclosure) and **ISO/IEC 30111** (handling) are the process standards a vendor's own policy is likely written against. They define the process, not a day count.
  * **EU law now supplies a coordinator, not a deadline.** NIS2 Article 12 tasks each member state's designated CSIRT with "negotiating disclosure timelines and managing vulnerabilities that affect multiple entities" — which is exactly the multi-party coordination problem a large-scale campaign creates. Using it also gives you a defensible answer to "who decided this timeline".

What campaigns in the corpus actually did, as a range: {[stock2016_hey]} notified in January and published at a security conference months later, monitoring for one month with reminders at two and four weeks; {[bennett2022_spfail]} sent private notifications on 15 November 2021 and published the CVE on 19 January 2022, **60 days later**; {[elyadmani2025_keys]} escalated to the cloud provider after **30 days** of no response and monitored for five months; {[maass2021_effective]} re-measured after **7 months** to test persistence. A month of monitoring is the floor; if you want to say anything about persistence or regression you need half a year.

Venue requirements as of the 2026 calls:

  * **IMC**: "Any submission that discovers security vulnerabilities must discuss their approach to responsible disclosure in their paper's Ethics section."(([[https://conferences.sigcomm.org/imc/2026/submission-instructions/|conferences.sigcomm.org/imc/2026]].))
  * **IEEE S&P**: "Authors are required to disclose vulnerabilities no later than the rebuttal deadline. If this is not possible, the authors should notify the PC chairs by email as soon as possible."(([[https://sp2026.ieee-security.org/cfpapers.html|sp2026.ieee-security.org]].))
  * **USENIX Security**: submissions that fail to disclose before submission, without a convincing ethical argument for delay, may be rejected; a separate Ethical Considerations appendix is required in the final paper.((Read from [[https://www.usenix.org/conference/usenixsecurity26/call-for-papers|usenix.org/conference/usenixsecurity26/call-for-papers]] on 2026-08-13, verbatim: "Submissions that fail to disclose prior to submission and that do not present convincing ethical arguments for delaying disclosure may be rejected" and "All papers MUST have a discussion of research ethics. This MUST be in a separate appendix called 'Ethical Considerations'". WebFetch gets 403 from usenix.org; curl with a browser User-Agent does not.))
  * **ACM CCS**: papers involving "real-world vulnerability analysis" must carry a dedicated Ethical Considerations section, with responsible disclosure named as an example of harm minimisation.(([[https://www.sigsac.org/ccs/CCS2026/call-for/call-for-papers.html|sigsac.org/ccs/CCS2026]].))
  * **PoPETs** states Menlo-Report principles and explicitly names "system vulnerabilities (e.g. cryptographic weaknesses, software exploits, and privacy attacks)" as an area needing ethical consideration, but has **no disclosure or notification requirement** and no timeline.((Read from [[https://petsymposium.org/authors-2026.php|petsymposium.org/authors-2026.php]] on 2026-08-13. An Ethical Principles subsection is "encouraged" and "may be required if deemed necessary during the review process".))

The practical consequence: **the notification has to happen before you submit, so it belongs in your timeline from the start.** A campaign that needs a month of monitoring plus a reminder schedule cannot be bolted on in the week before a deadline, and "we will disclose after acceptance" is now grounds for rejection at two of these venues.

===== What goes wrong, and what it costs you =====

The recurring surprise in this literature is that operators are mostly not hostile. {[li2016_youve]} received 685 email responses, of which 530 (77%) were automated, 62 (9%) bounces and 93 (14%) human — and **"96% of human-sent responses were positive or neutral"**, with only four negative mails and none threatening. {[maass2021_effective]} got thanks and gifts from 260 recipients (34% of those who wrote back). But the tail is real and you should budget for it:

  * **Legal threats and institutional escalation.** {[stock2016_hey]} received a hosting provider's threat to sue their university on day two of the campaign. {[maass2021_effective]}: 19 recipients (2.5% of those who made contact) complained, some threatening legal action, one contacting a university chancellor — though "no legal action was filed against the involved researchers or universities". Have your institution's legal contact briefed **before** you send, not after the first threat. The wider background is in {[gamerogarrido2017_legalrisks]}.
  * **Being taken for a phishing campaign.** You are sending unsolicited mail about a security problem, with a link, to strangers — the exact shape of a phish. {[utz2023_comparing]} coded 12.1% of privacy conversations as the recipient being unsure whether the mail was benign, and 11.8% of survey respondents thought it was a scam. Mitigations that were actually used: S/MIME signing, a persistent institutional study website with a per-recipient report, a named institution in the ''From'' address, and an opt-out link. Anticipate this in the message rather than in the reply.
  * **Collateral damage from your notification.** {[stock2016_hey]} received messages from site owners whose hosting providers had threatened to take their sites down unless they fixed the flaw quickly, and had to contact providers to clear up the misunderstanding. {[stivala2026_behind]} explains the mechanism from the provider side: on unmanaged tiers "the legal obligation typically extends only to notifying the client about the issue … Once the report is delivered, responsibility shifts to the client", and one provider's summary of a compromised customer site was "it's basically out of scope for us to take care of that […]. Then you've got a shell there. Have fun." Notifying the provider can get the customer suspended rather than helped. Say which party you chose to notify and why.
  * **Opt-out has to exist and has to be honoured.** {[stock2016_hey]} received five requests covering 187 domains and excluded all of them — including 38 from the control group, which damages the design and must still be done. {[utz2023_comparing]} logged 497 opt-out requests (466 through a web interface, 31 by email) and excluded 44 domains as false positives. Build the interface before you send, and report the exclusions as part of the funnel.
  * **False positives reach real people.** Twelve IPv6 contacts in {[li2016_youve]} rebutted the vulnerability claim. Your detector's precision is now somebody else's inbox; measure it before, not after.
  * **Ethics review is not automatic and not uniform.** {[stock2016_hey]} recorded that its institutions "neither mandate nor provide an IRB approval before conducting such experiments". {[maass2021_effective]} got approval from the ethics committees of two of three institutions and a dean's approval from the third. See [[Practices:Ethics]], and {[hantke2024_redlines]} for what server operators themselves consider acceptable.

==== Covert notification ====

A notification study has an observer problem: if you say "this is a research experiment", you have changed the thing you are measuring. {[maass2021_effective]} sent the first two messages without disclosing the study, then debriefed every owner with an opt-out, and had that design approved in advance; the secondary analysis {[stoever2023_owners]} obtained the original ethics applications to confirm they covered re-use. {[utz2023_comparing]} took the opposite decision — fully disclosed as a study from the first email, IRB-approved without changes — and still measured effects.

Both are defensible and both were reviewed. What is not defensible is deciding covertly without review, or debriefing nobody. If you go covert: get it approved beforehand, debrief everyone at the end, and offer an opt-out that is honoured retroactively.

===== A reporting checklist =====

What a reviewer in this field will look for, drawn from what the studies above report and what the weaker ones omit:

  - **The population and the unit.** How many //distinct parties//, not how many findings. One contact often covers many domains — {[stock2016_hey]} found different domains sharing a contact address, i.e. one owner, across both notified and control arms.
  - **How you obtained contacts, per source, with coverage per source.** "We used WHOIS" is no longer a method. State the source, the date, whether it was purchased, and what fraction of your population yielded nothing.
  - **What the contact actually reaches.** Site owner, hosting provider, CDN, registrar, or CERT. Give the share of abuse contacts that resolve to a CDN or shared host.
  - **The full funnel:** sent → bounced → spam-rejected → delivered → opened → report viewed → remediated, plus an explicit //unknown// bucket. Do not report only the last number.
  - **A control group**, its size, and the background remediation rate in it. If you cannot randomise, say so and say what that costs your claim.
  - **The statistical test**, with a multiple-comparison correction if you compare arms. Fisher's exact with Holm–Bonferroni {[stock2018_didnt,utz2023_comparing]} and permutation tests with Bonferroni {[li2016_youve]} are the precedents. Several effects in this literature disappear under correction — see [[Statistics:Pvalue corrections]].
  - **The monitoring window and the reminder schedule**, with dates. A rate without a window is not a rate.
  - **Your detector's precision**, because false positives went to real inboxes.
  - **The disclosure timeline you followed** and which convention it came from.
  - **The message itself**, in an appendix or [[Artifacts|artefact]]. Every study above that reports a framing effect published its text; you cannot replicate a framing result without it.
  - **The ethics decision**: review body and outcome, whether the study was covert, how you debriefed, the opt-out mechanism, and how many opted out.
  - **Everything hostile that happened.** Legal threats, complaints to your institution, suspended customers. This is the part later researchers most need and the part most often missing.

===== Papers to read first =====

If you read four: {[stock2016_hey]} for the channel survey and the delivery funnel, {[li2016_youve]} for the arm-by-arm comparison and the decay curve, {[maass2021_effective]} for the only large factorial design on framing and medium, and {[lone2022_sav]} for the control group that undermines the rest. Then {[utz2023_comparing]} for privacy versus security and the delivery figures that supersede the 2016 channel advice, {[stock2018_didnt]} for why message tuning is a dead end, {[li2016_remedying]} for what a platform can do that you cannot, and {[stivala2026_behind]} for the receiving end.

===== Related pages =====

  * [[Practices:Ethics]] — harm from crawling and scanning, robots.txt, ethics approval.
  * [[Practices:Legal enforcement]] — when the right recipient is a regulator rather than an operator.
  * [[Practices:Public relations]] — the other post-publication channel.
  * [[Statistics:Hypothesis testing]] and [[Statistics:Pvalue corrections]] — for the arm comparisons.
  * [[Artifacts]] — publishing the notification text and the contact-discovery code.
  * [[Design:Website selection]] — your notified population is your sample, with the same biases.

===== Methodology and limitations of these figures =====

Every corpus figure above comes from ''scripts/report_notifying_websites.mjs'' over ''data/extract/run1'' (5,859 papers, 2010–2026). Three limits carry directly:

  * **Seven venues only.** EuroS&P, ACSAC, RAID, AsiaCCS, CHI and SOUPS are absent, and CHI/SOUPS are where a good deal of the operator-facing usable-security work appears. NDSS 2016 and NDSS 2018 full text was not retrieved at all, which is why {[stock2018_didnt]} — the canonical follow-up study — was read from the publisher's PDF rather than from the corpus.
  * ''ethics.notifiedAffectedParties'' **agreed with an independent re-extraction on 67% of papers**, so treat every percentage above as accurate to a few points, not to the decimal.((That 67% was measured on 100 papers of the previous, 4,322-paper extraction run and has not been re-measured on the current corpus. Treat it as the right order of magnitude.)) ''ethics.disclosureDetail'' is free text capped at 20 words, so it is reported only as folded families and only as a ranking; the fold rules and the unmapped residue are on the provenance page.
  * **The 22 campaigns are hand-classified, not a census.** A regex over 5,869 full-text files (ten more than the 5,859 extraction records — a few papers have text but no record) produced 179 candidates; 22 were classified as campaigns, 13 rejected with a stated reason, and **144 were never read**. Three further rejections came from a wider first-pass scan and fall outside the committed regex, which is why the reasoned rejections total 16. The table above is a curated reading list, and the unreviewed residue is published in full.

The complete query log, the report script with its unedited output, the folding rules with their unmapped residue, the spot-checked quotes, and every external source that was rejected are on **[[provenance:practices:notifying_websites]]**. Corpus-wide caveats are on [[Literature:Corpus]].

====== References ======
<bibtex bibliography></bibtex>


~~DISCUSSION~~