You crawled 100k sites and found that 8,000 of them leak something. Before you write the paper you have to tell those 8,000 operators, and every venue in this field now expects you to say so in the paper.¹⁾ This page is about the mechanics of doing that at scale: how you get a contact for a party you have never met, what response rate to plan for, how long to wait before publishing, and what to report so a reviewer accepts the result.

It is not about one-off coordinated disclosure to a named vendor. Emailing Google's security team is a solved problem with a queue and an SLA; emailing 8,000 strangers is a measurement in its own right, with a delivery funnel, a control group and a p-value. Treat it as one.

From the publication corpus behind this site: 5,859 papers, seven venues (CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P), 2010–2026. Of the 5,118 empirical papers, 4,472 carry an ethics record — the extraction emits one only when the paper says something about ethics at all, so that is the denominator below, and the 646 papers that say nothing whatsoever are excluded rather than silently counted as “did not notify”.

So 48.3% notified somebody (yes or partial), 4.3% explicitly say they did not, and a quarter of the field simply does not address the question. Broken out by the populations this page is about:

Two things to take from this table. Network measurement is ahead of web measurement: scanning papers notify at 61.0% (yes+partial) against 43.5% for crawling papers, and are less than half as likely to leave the question unanswered. The scanning community built the norm first, largely because Internet-wide scanning provoked complaints that forced the question. Legal-compliance papers state a position most often (only 12.0% not-stated), and the bottom row is the population closest to whoever is reading this: 123 papers that both crawled the web and assessed a law, i.e. found a compliance violation across many sites. 14.6% of them say outright that they did not notify — more than three times the corpus-wide 4.3%. That is not negligence. A GDPR-violation finding across thousands of sites is the case where authors most often decide, and say, that individual notification is the wrong instrument, and route the finding to a regulator or to publication instead. If that is your situation, deciding not to notify is a defensible position that you have to argue in the paper, not an omission you can leave to the reader.

The direction of travel is unambiguous:

* 2025–2026 is provisional: CCS 2026 and IMC 2026 have not been held, and IEEE S&P 2026 and TheWebConf 2026 abstracts are not in OpenAlex, so those venue-years are under-represented by construction. See Corpus.

The one flat line is the interesting one. Regulator and CERT contact has not grown and remains rare — 3.3% of the 4,472 say yes, and only 7.2% of the 376 papers that assessed a law. Given that a CERT is often the only party that can reach a whole constituency, and that EU law now obliges every member state to run one for exactly this purpose (below), this is the largest gap between what is available and what is used.

And when the intermediary is used it is used as well as, not instead of: of the 147 papers that contacted a regulator or CERT, 135 (91.8%) also notified the affected party directly. Nobody in this corpus treats “we told the CERT” as discharging the obligation. Plan the intermediary as an extra arm, not as the cheap way out of finding contacts.

2,870 of the 4,472 (64.2%) give some free-text detail about their disclosure, and all 2,160 that said they notified are among them. Scoping to those 2,160 — because the rest of that field is participant-debriefing notes, where “which channel?” is not a question — and folding the text for any mention of a channel (the rules and their residue are on the provenance page):

1,633 of the 2,160 (75.6%) do not say through what channel. Where a channel does appear it is usually a single large platform named outright (14.7% mention Google, Apple, Meta, Microsoft, Amazon or Mozilla); a generic email to the operator or developer is 4.6%, a CERT 1.8%, a bug-bounty programme 1.6%, a hosting provider 1.6%, WHOIS 0.7%, a data protection authority 0.6%. And 2,155 of the 2,870 (75.1%) name no outcome either.

Worse for anyone trying to plan a campaign: only 7 of the 2,870 report a countable response ratio in that text — figures like “Contacted 318 of 559 sensitive organizations; received 11 responses by submission” or “Reported 110 malicious extensions to Google; 62.7% were removed”. Every rate in the next section had to be read out of the papers' own results sections, not out of their ethics sections. That is the reporting gap this page exists to close: the modal paper in this corpus says we disclosed responsibly and stops.

This is where notification studies actually fail. Both 2016 campaigns concluded that reachability, not operator willingness, was the binding constraint [4Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link), 5Li, Frank; Durumeric, Zakir; Czyz, Jakub; Karami, Mohammad; Bailey, Michael; McCoy, Damon; Savage, Stefan; Paxson, Vern (2016): "You've Got Vulnerability: Exploring Effective Vulnerability Notifications", in: Proceedings of the USENIX Security Symposium. (Link)], and the 2026 interview study of hosting providers opens by summarising the decade since as “the response and remediation rates for notified web vulnerabilities still average between 20 to 30%, and no effective alternatives have yet been identified (e.g., still no better channel than WHOIS)” [6Stivala, Giada; Mrowczynski, Rafael; Hellenthal, Maria; Pellegrino, Giancarlo (2026): "Behind the Curtain: How Shared Hosting Providers Respond to Vulnerability Notifications", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)]. That sentence is a related-work summary carrying citations to four earlier papers, not a figure that study measured — but it is the field's own 2026 assessment of where it stands, and it is worth knowing that a 2026 paper still writes it.

Scraping mailto: links off pages does not scale in a way the literature is explicit about: unreliable, defeated by CAPTCHAs and obfuscation, and it collects addresses nobody published for this purpose. [4Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)] discarded it, along with web contact forms and telephone, as “not a viable option in a large-scale scenario”.

Three of those channels are machine-queryable. This runs them in cost order and prints per-source coverage, because a pooled “we found contacts for N% of domains” is the figure a reviewer sends back. Failures are printed rather than swallowed: a silent zero and a missing contact look identical, and the difference understates your own coverage.

find_contacts.py

#!/usr/bin/env python3
"""Enumerate the contact points for a website that are still machine-queryable.
 
Three sources, cheapest first, each reported separately so you can publish a
per-source coverage rate instead of one pooled "we found contacts for N% of
domains" -- the pooled figure is the one reviewers send back.
 
  python3 find_contacts.py example.com example.org
 
Deliberately NOT included, and why:
  * domain WHOIS on port 43 -- sunset for gTLDs on 2025-01-28; RDAP is now the
    definitive source (ICANN, "Update: Launching RDAP; Sunsetting WHOIS").
  * the registrant email inside RDAP -- redacted for EEA registrants since the
    2018 ICANN Temporary Specification and, in practice, for most others. What
    survives is a registrar forwarding address or a web form. Ask for the
    non-public record through ICANN's RDRS rather than scraping around it.
  * scraping mailto: links off the page -- highest yield, worst measured
    delivery, and it collects addresses nobody published for this purpose.
 
Failures are printed, never swallowed: a silent zero looks identical to a
missing contact and would understate your own coverage.
"""
 
import json
import re
import socket
import sys
import urllib.error
import urllib.request
 
# Put a real address here. Operators who look up who is probing them and find
# nothing are the ones who complain to your institution instead of to you.
UA = "measuretheweb-contact-probe (research; contact: you@example.edu)"
TIMEOUT = 15
BOOTSTRAP = "https://data.iana.org/rdap/dns.json"  # RFC 9224
 
 
def _get(url, accept="*/*"):
    req = urllib.request.Request(url, headers={"User-Agent": UA, "Accept": accept})
    with urllib.request.urlopen(req, timeout=TIMEOUT) as r:
        return r.read().decode("utf-8", "replace")
 
 
def security_txt(domain):
    """RFC 9116. /.well-known/security.txt is the correct location; /security.txt
    is the legacy fallback the RFC keeps for compatibility."""
    for path in ("/.well-known/security.txt", "/security.txt"):
        try:
            body = _get(f"https://{domain}{path}", "text/plain")
        except (urllib.error.URLError, socket.timeout, OSError) as e:
            yield ("security.txt", None, f"{path}: {type(e).__name__}")
            continue
        # Servers that return their 404 page as 200 text/html are common enough
        # that a Contact: line is the only reliable signal the file is real.
        contacts = re.findall(r"^Contact:\s*(\S+)", body, re.M | re.I)
        expires = re.findall(r"^Expires:\s*(\S+)", body, re.M | re.I)
        if not contacts:
            yield ("security.txt", None, f"{path}: 200 but no Contact: field")
            continue
        note = f"expires={expires[0] if expires else 'MISSING (required by RFC 9116)'}"
        for c in contacts:
            yield ("security.txt", c, note)
        return
 
 
_BOOTSTRAP_CACHE = {}
 
 
def _rdap_base(tld):
    if not _BOOTSTRAP_CACHE:
        # Fetched once per run. A failure here is NOT a per-domain coverage zero
        # -- it means RDAP coverage is unmeasurable for the whole batch -- so it
        # raises with the URL rather than degrading into a row of dashes that
        # would read as "no contact found" for every domain in the sample.
        try:
            services = json.loads(_get(BOOTSTRAP))["services"]
        except (urllib.error.URLError, socket.timeout, OSError, json.JSONDecodeError, KeyError) as e:
            raise RuntimeError(
                f"RDAP bootstrap registry unreachable ({BOOTSTRAP}): {type(e).__name__}: {e}. "
                "Refusing to continue: every RDAP row would print as 'not found' and "
                "understate coverage. Retry, or run with RDAP excluded and say so."
            ) from e
        for tlds, urls in services:
            for t in tlds:
                _BOOTSTRAP_CACHE[t] = urls[0].rstrip("/")
    return _BOOTSTRAP_CACHE.get(tld)
 
 
def rdap_domain(domain):
    """RFC 9082/9083, discovered through the IANA bootstrap registry (RFC 9224).
    No RDAP base for the TLD means the ccTLD registry runs none -- that is a
    coverage fact about your sample, so it is reported rather than skipped."""
    tld = domain.rsplit(".", 1)[-1]
    base = _rdap_base(tld)
    if base is None:
        yield ("RDAP", None, f".{tld} has no RDAP service in the IANA bootstrap")
        return
    try:
        data = json.loads(_get(f"{base}/domain/{domain}", "application/rdap+json"))
    except (urllib.error.URLError, socket.timeout, OSError, json.JSONDecodeError) as e:
        yield ("RDAP", None, f"{base}: {type(e).__name__}")
        return
    emails = 0
    for ent in data.get("entities", []):
        roles = ",".join(ent.get("roles", [])) or "no-role"
        for item in ent.get("vcardArray", [[], []])[1]:
            if item[0] == "email":
                emails += 1
                yield ("RDAP", item[3], f"role={roles}")
    if emails == 0:
        yield ("RDAP", None, "response carried no email -- redacted, or role-only")
 
 
def abuse_contact(domain):
    """The RIR abuse contact for the IP the domain resolves to (RIPE abuse-c:,
    ARIN Abuse POC). RIPEstat resolves it across all five RIRs over HTTPS, so
    it needs no DNS TXT client and is not rate-limited into uselessness the way
    port-43 WHOIS was. This is the one contact source that scales, and it
    reaches the hosting provider -- not the site owner."""
    try:
        ip = socket.gethostbyname(domain)
    except OSError as e:
        yield ("RIR abuse", None, f"DNS: {type(e).__name__}")
        return
    try:
        d = json.loads(
            _get(f"https://stat.ripe.net/data/abuse-contact-finder/data.json?resource={ip}")
        )["data"]
    except (urllib.error.URLError, socket.timeout, OSError, json.JSONDecodeError, KeyError) as e:
        yield ("RIR abuse", None, f"RIPEstat: {type(e).__name__}")
        return
    contacts = d.get("abuse_contacts") or []
    if not contacts:
        yield ("RIR abuse", None, f"ip={ip} rir={d.get('authoritative_rir')} no abuse-c")
    for c in contacts:
        yield ("RIR abuse", c, f"ip={ip} rir={d.get('authoritative_rir')}")
 
 
SOURCES = ("security.txt", "RDAP", "RIR abuse")
 
 
def main(domains):
    hit = {s: set() for s in SOURCES}
    print(f"{'domain':24} {'source':13} {'contact':40} note")
    for d in domains:
        for src, contact, note in (*security_txt(d), *rdap_domain(d), *abuse_contact(d)):
            if contact:
                hit[src].add(d)
            print(f"{d:24} {src:13} {(contact or '--')[:40]:40} {note}")
    print(f"\ncoverage, domains with >=1 usable contact, of {len(domains)}:")
    for s in SOURCES:
        print(f"  {s:13} {len(hit[s]):3}  {100 * len(hit[s]) / len(domains):5.1f}%")
    union = set().union(*hit.values())
    print(f"  {'any source':13} {len(union):3}  {100 * len(union) / len(domains):5.1f}%")
 
 
if __name__ == "__main__":
    if len(sys.argv) < 2:
        sys.exit(__doc__)
    main(sys.argv[1:])

Real output, run 2026-08-13 on five hand-picked domains — not a sample, and not representative: these are large, well-resourced organisations, so security.txt coverage here is an order of magnitude above the 3–4% [8Poteat, Tara; Li, Frank (2021): "Who You Gonna Call? An Empirical Evaluation of Website security.txt Deployment", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] measured at the top 10K.

domain                   source        contact                                  note
google.com               security.txt  https://g.co/vulnz                       expires=2030-04-01T00:00:00z
google.com               security.txt  mailto:security@google.com               expires=2030-04-01T00:00:00z
google.com               RDAP          —                                        response carried no email — redacted, or role-only
google.com               RIR abuse     network-abuse@google.com                 ip=74.125.29.113 rir=arin
ethz.ch                  security.txt  mailto:security@ethz.ch                  expires=2028-01-31T07:00:00.000Z
ethz.ch                  RDAP          —                                        .ch has no RDAP service in the IANA bootstrap
ethz.ch                  RIR abuse     abuse@ethz.ch                            ip=129.132.19.216 rir=ripe
cispa.de                 security.txt  —                                        /.well-known/security.txt: HTTPError
cispa.de                 security.txt  —                                        /security.txt: HTTPError
cispa.de                 RDAP          —                                        .de has no RDAP service in the IANA bootstrap
cispa.de                 RIR abuse     abuse@she.net                            ip=212.21.165.114 rir=ripe
bbc.co.uk                security.txt  mailto:security@bbc.co.uk                expires=2038-01-19T03:14:07Z
bbc.co.uk                RDAP          redacted@nominet.uk                      role=registrant
bbc.co.uk                RIR abuse     abuse@fastly.com                         ip=151.101.192.81 rir=arin
wikipedia.org            security.txt  mailto:security@wikimedia.org            expires=2029-03-31T09:00:00.000Z
wikipedia.org            RDAP          —                                        response carried no email — redacted, or role-only
wikipedia.org            RIR abuse     abuse@wikimedia.org                      ip=185.15.58.224 rir=ripe

coverage, domains with >=1 usable contact, of 5:
  security.txt    4   80.0%
  RDAP            1   20.0%
  RIR abuse       5  100.0%
  any source      5  100.0%

Four failure modes worth reading off that output before you build the pipeline:

• RDAP gives you nothing useful. google.com and wikipedia.org return no email at all; bbc.co.uk returns the literal placeholder redacted@nominet.uk. Post-2018 redaction, in the response.
• ccTLDs often have no RDAP. .ch and .de have no service in the IANA bootstrap registry. If your sample is a national top list, RDAP coverage may be zero and it will not be your bug.
• The RIR abuse contact reaches the infrastructure, not the site. cispa.de resolves to its hosting provider's abuse@she.net; bbc.co.uk sits behind Fastly and yields abuse@fastly.com. For any CDN-fronted site the abuse contact is the CDN — and CDN-fronting is now the common case, which is a structural reason this channel has decayed since 2016. Report what fraction of your abuse contacts belong to a CDN or shared host rather than to the party you meant to reach.
• A transient failure looks exactly like an absent contact. An earlier run of exactly this script, on exactly these five domains, printed RIPEstat: TimeoutError for google.com and so reported 80% rather than 100% RIR-abuse coverage. Nothing about the sample changed. Distinguish a network failure from a missing contact and retry it, or your denominator is wrong — and note that the script refuses to run at all if the IANA bootstrap registry is unreachable, because degrading that into a column of dashes would understate RDAP coverage for every domain at once.

[14Sasaki, Takayuki; Fujita, Akira; Gañán, Carlos Hernandez; van Eeten, Michel; Yoshioka, Katsunari; Matsumoto, Tsutomu (2022): "Exposed Infrastructures: Discovery, Attacks and Remediation of Insecure ICS Remote Management Devices", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] collects the comparison itself, and it is the shortest statement of the range: “its remediation rate was approximately 18% … Our remediation rate is higher than most previous notification experiments: it was approximately 40% for cross-site scripting and a WordPress vulnerability, 33%–42% for different WordPress vulnerability, and less than 20% for DNS zone poisoning. The only campaigns that reported similar remediation rates were on publicly accessible Git repositories (78%–81%) and Heartbleed (approximately 40%–90%).”

These are the campaigns in the corpus that reported a quotable outcome, each with its own denominator — they are not comparable to each other, because “response”, “remediation” and “fix” are defined differently in each and the populations are wildly different. Plan against the range, not the mean. The hand pass classified 22 campaigns in total; 16 are below, and the six not shown reported a volume sent but no response or remediation figure. They are listed on notifying_websites, along with the one row below — [15Stock, Ben; Pellegrino, Giancarlo; Li, Frank; Backes, Michael; Rossow, Christian (2018): "Didn't You Hear Me? - Towards More Successful Web Vulnerability Notifications", in: Proceedings of the Network and Distributed System Security Symposium. (DOI)] — whose full text is missing from the corpus and was read from the publisher's PDF instead.

The delivery funnel is the story, and it is why headline remediation rates look so bad. [4Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)]: 5.8% of reports received, but ~40% remediation among those actually read. [11Bennett, Nathaniel; Sowards, Rebekah; Deccio, Casey T. (2022): "SPFail: Discovering, Measuring, and Remediating Vulnerabilities in Email Sender Validation", in: Proceedings of the ACM Internet Measurement Conference. (DOI)]: 31.6% undelivered → 12% of the delivered opened → 4% of the openers patched. [15Stock, Ben; Pellegrino, Giancarlo; Li, Frank; Backes, Michael; Rossow, Christian (2018): "Didn't You Hear Me? - Towards More Successful Web Vulnerability Notifications", in: Proceedings of the Network and Distributed System Security Symposium. (DOI)]: 74.4% fixed among Git operators who opened the report, against a 13% control. Operators who read your report largely act on it. Most never read it.

So the highest-leverage thing you can do is not writing a better email. It is:

1. Use a channel with measured delivery. Parsed privacy-policy/contact addresses at 87.8% against RFC 2142 aliases at 33.8% [9Utz, Christine; Michels, Matthias; Degeling, Martin; Marnau, Ninja; Stock, Ben (2023): "Comparing Large-Scale Privacy and Security Notifications", in: Proceedings on Privacy Enhancing Technologies. (DOI)] is the largest single lever in this literature. A hand-read Impressum is better still [3Maass, Max; Stöver, Alina; Pridöhl, Henning; Bretthauer, Sebastian; Herrmann, Dominik; Hollick, Matthias; Spiecker, Indra (2021): "Effective Notification Campaigns on the Web: A Matter of Trust, Framing, and Support", in: Proceedings of the USENIX Security Symposium. (Link)].
2. Instrument the funnel. Report sent → bounced → delivered → opened → report viewed → remediated, separately. [4Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)] classified every domain as reached / bounced / unreachable / unknown, and found “more than half of all domains are marked as unknown” — silently bounced, delivered to an unmonitored mailbox, or ignored. That unknown bucket is the honest reporting of your own blind spot; publish it.
3. Expect spam filtering to fight you. [4Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)] received bounces for 562 domains stating the mail was classified as spam, despite a clean mail server. Warm up the sender, sign the mail, and measure the spam-rejection rate as a separate category. [15Stock, Ben; Pellegrino, Giancarlo; Li, Frank; Backes, Michael; Rossow, Christian (2018): "Didn't You Hear Me? - Towards More Successful Web Vulnerability Notifications", in: Proceedings of the Network and Distributed System Security Symposium. (DOI)] tested S/MIME signing directly and it did not beat the alternatives, so sign for legitimacy, not for delivery.
4. Put the whole report in the message. Both 2016 campaigns used a link to a per-domain report so they could measure views; [15Stock, Ben; Pellegrino, Giancarlo; Li, Frank; Backes, Michael; Rossow, Christian (2018): "Didn't You Hear Me? - Towards More Successful Web Vulnerability Notifications", in: Proceedings of the Network and Distributed System Security Symposium. (DOI)] concluded the external link “played a minor role”, and Git operators who opened the mail but never viewed the report still fixed at 24.7% against a 13.0% control. Tracking your funnel and maximising remediation pull in opposite directions — say which you chose.

Read the two contradictions as the state of the art rather than as noise. Message-content tuning is a dead end: three studies varied wording, format, signing, tone and translation, and the effects are small, inconsistent, or vanish under multiple-comparison correction. Sender identity, medium and reachability are the real variables — and even those may not survive a control group, which is what [1Lone, Qasim; Frik, Alisa; Luckie, Matthew; Korczyński, Maciej; van Eeten, Michel; Gañán, Carlos (2022): "Deployment of Source Address Validation by Network Operators: A Randomized Control Trial", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] demonstrated on the one population where somebody randomised properly.

There is no single norm, so state which one you followed and why. The reference points, all verified as current on 2026-08-13:

• CERT/CC: 45 days from the initial report — “regardless of the existence or availability of patches or workarounds from affected vendors”.⁹⁾
• Google Project Zero: 90 days to patch plus 30 days before technical detail is published, with a 14-day grace period on request; 7 days for bugs under active exploitation. Since a trial that began 29 July 2025, Project Zero also publishes within about a week that a report exists — vendor, product, date, deadline — without the details.¹⁰⁾
• ISO/IEC 29147 (disclosure) and ISO/IEC 30111 (handling) are the process standards a vendor's own policy is likely written against. They define the process, not a day count.
• EU law now supplies a coordinator, not a deadline. NIS2 Article 12 tasks each member state's designated CSIRT with “negotiating disclosure timelines and managing vulnerabilities that affect multiple entities” — which is exactly the multi-party coordination problem a large-scale campaign creates. Using it also gives you a defensible answer to “who decided this timeline”.

What campaigns in the corpus actually did, as a range: [4Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)] notified in January and published at a security conference months later, monitoring for one month with reminders at two and four weeks; [11Bennett, Nathaniel; Sowards, Rebekah; Deccio, Casey T. (2022): "SPFail: Discovering, Measuring, and Remediating Vulnerabilities in Email Sender Validation", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] sent private notifications on 15 November 2021 and published the CVE on 19 January 2022, 60 days later; [10El Yadmani, Soufian; Gadyatskaya, Olga; Zhauniarovich, Yury (2025): "The File That Contained the Keys Has Been Removed: An Empirical Analysis of Secret Leaks in Cloud Buckets and Responsible Disclosure Outcomes", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] escalated to the cloud provider after 30 days of no response and monitored for five months; [3Maass, Max; Stöver, Alina; Pridöhl, Henning; Bretthauer, Sebastian; Herrmann, Dominik; Hollick, Matthias; Spiecker, Indra (2021): "Effective Notification Campaigns on the Web: A Matter of Trust, Framing, and Support", in: Proceedings of the USENIX Security Symposium. (Link)] re-measured after 7 months to test persistence. A month of monitoring is the floor; if you want to say anything about persistence or regression you need half a year.

Venue requirements as of the 2026 calls:

• IMC: “Any submission that discovers security vulnerabilities must discuss their approach to responsible disclosure in their paper's Ethics section.”¹¹⁾
• IEEE S&P: “Authors are required to disclose vulnerabilities no later than the rebuttal deadline. If this is not possible, the authors should notify the PC chairs by email as soon as possible.”¹²⁾
• USENIX Security: submissions that fail to disclose before submission, without a convincing ethical argument for delay, may be rejected; a separate Ethical Considerations appendix is required in the final paper.¹³⁾
• ACM CCS: papers involving “real-world vulnerability analysis” must carry a dedicated Ethical Considerations section, with responsible disclosure named as an example of harm minimisation.¹⁴⁾
• PoPETs states Menlo-Report principles and explicitly names “system vulnerabilities (e.g. cryptographic weaknesses, software exploits, and privacy attacks)” as an area needing ethical consideration, but has no disclosure or notification requirement and no timeline.¹⁵⁾

The practical consequence: the notification has to happen before you submit, so it belongs in your timeline from the start. A campaign that needs a month of monitoring plus a reminder schedule cannot be bolted on in the week before a deadline, and “we will disclose after acceptance” is now grounds for rejection at two of these venues.

The recurring surprise in this literature is that operators are mostly not hostile. [5Li, Frank; Durumeric, Zakir; Czyz, Jakub; Karami, Mohammad; Bailey, Michael; McCoy, Damon; Savage, Stefan; Paxson, Vern (2016): "You've Got Vulnerability: Exploring Effective Vulnerability Notifications", in: Proceedings of the USENIX Security Symposium. (Link)] received 685 email responses, of which 530 (77%) were automated, 62 (9%) bounces and 93 (14%) human — and “96% of human-sent responses were positive or neutral”, with only four negative mails and none threatening. [3Maass, Max; Stöver, Alina; Pridöhl, Henning; Bretthauer, Sebastian; Herrmann, Dominik; Hollick, Matthias; Spiecker, Indra (2021): "Effective Notification Campaigns on the Web: A Matter of Trust, Framing, and Support", in: Proceedings of the USENIX Security Symposium. (Link)] got thanks and gifts from 260 recipients (34% of those who wrote back). But the tail is real and you should budget for it:

• Legal threats and institutional escalation. [4Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)] received a hosting provider's threat to sue their university on day two of the campaign. [3Maass, Max; Stöver, Alina; Pridöhl, Henning; Bretthauer, Sebastian; Herrmann, Dominik; Hollick, Matthias; Spiecker, Indra (2021): "Effective Notification Campaigns on the Web: A Matter of Trust, Framing, and Support", in: Proceedings of the USENIX Security Symposium. (Link)]: 19 recipients (2.5% of those who made contact) complained, some threatening legal action, one contacting a university chancellor — though “no legal action was filed against the involved researchers or universities”. Have your institution's legal contact briefed before you send, not after the first threat. The wider background is in [21Gamero-Garrido, Alexander; Savage, Stefan; Levchenko, Kirill; Snoeren, Alex C. (2017): "Quantifying the Pressure of Legal Risks on Third-party Vulnerability Research", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)].
• Being taken for a phishing campaign. You are sending unsolicited mail about a security problem, with a link, to strangers — the exact shape of a phish. [9Utz, Christine; Michels, Matthias; Degeling, Martin; Marnau, Ninja; Stock, Ben (2023): "Comparing Large-Scale Privacy and Security Notifications", in: Proceedings on Privacy Enhancing Technologies. (DOI)] coded 12.1% of privacy conversations as the recipient being unsure whether the mail was benign, and 11.8% of survey respondents thought it was a scam. Mitigations that were actually used: S/MIME signing, a persistent institutional study website with a per-recipient report, a named institution in the From address, and an opt-out link. Anticipate this in the message rather than in the reply.
• Collateral damage from your notification. [4Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)] received messages from site owners whose hosting providers had threatened to take their sites down unless they fixed the flaw quickly, and had to contact providers to clear up the misunderstanding. [6Stivala, Giada; Mrowczynski, Rafael; Hellenthal, Maria; Pellegrino, Giancarlo (2026): "Behind the Curtain: How Shared Hosting Providers Respond to Vulnerability Notifications", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] explains the mechanism from the provider side: on unmanaged tiers “the legal obligation typically extends only to notifying the client about the issue … Once the report is delivered, responsibility shifts to the client”, and one provider's summary of a compromised customer site was “it's basically out of scope for us to take care of that […]. Then you've got a shell there. Have fun.” Notifying the provider can get the customer suspended rather than helped. Say which party you chose to notify and why.
• Opt-out has to exist and has to be honoured. [4Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)] received five requests covering 187 domains and excluded all of them — including 38 from the control group, which damages the design and must still be done. [9Utz, Christine; Michels, Matthias; Degeling, Martin; Marnau, Ninja; Stock, Ben (2023): "Comparing Large-Scale Privacy and Security Notifications", in: Proceedings on Privacy Enhancing Technologies. (DOI)] logged 497 opt-out requests (466 through a web interface, 31 by email) and excluded 44 domains as false positives. Build the interface before you send, and report the exclusions as part of the funnel.
• False positives reach real people. Twelve IPv6 contacts in [5Li, Frank; Durumeric, Zakir; Czyz, Jakub; Karami, Mohammad; Bailey, Michael; McCoy, Damon; Savage, Stefan; Paxson, Vern (2016): "You've Got Vulnerability: Exploring Effective Vulnerability Notifications", in: Proceedings of the USENIX Security Symposium. (Link)] rebutted the vulnerability claim. Your detector's precision is now somebody else's inbox; measure it before, not after.
• Ethics review is not automatic and not uniform. [4Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)] recorded that its institutions “neither mandate nor provide an IRB approval before conducting such experiments”. [3Maass, Max; Stöver, Alina; Pridöhl, Henning; Bretthauer, Sebastian; Herrmann, Dominik; Hollick, Matthias; Spiecker, Indra (2021): "Effective Notification Campaigns on the Web: A Matter of Trust, Framing, and Support", in: Proceedings of the USENIX Security Symposium. (Link)] got approval from the ethics committees of two of three institutions and a dean's approval from the third. See Ethics, and [22Hantke, Florian; Roth, Sebastian; Mrowczynski, Rafael; Utz, Christine; Stock, Ben (2024): "Where Are the Red Lines? Towards Ethical Server-Side Scans in Security and Privacy Research", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] for what server operators themselves consider acceptable.

A notification study has an observer problem: if you say “this is a research experiment”, you have changed the thing you are measuring. [3Maass, Max; Stöver, Alina; Pridöhl, Henning; Bretthauer, Sebastian; Herrmann, Dominik; Hollick, Matthias; Spiecker, Indra (2021): "Effective Notification Campaigns on the Web: A Matter of Trust, Framing, and Support", in: Proceedings of the USENIX Security Symposium. (Link)] sent the first two messages without disclosing the study, then debriefed every owner with an opt-out, and had that design approved in advance; the secondary analysis [23Stöver, Alina; Gerber, Nina; Pridöhl, Henning; Maass, Max; Bretthauer, Sebastian; Spiecker gen. Döhmann, Indra; Hollick, Matthias; Herrmann, Dominik (2023): "How Website Owners Face Privacy Issues: Thematic Analysis of Responses from a Covert Notification Study Reveals Diverse Circumstances and Challenges", in: Proceedings on Privacy Enhancing Technologies. (DOI)] obtained the original ethics applications to confirm they covered re-use. [9Utz, Christine; Michels, Matthias; Degeling, Martin; Marnau, Ninja; Stock, Ben (2023): "Comparing Large-Scale Privacy and Security Notifications", in: Proceedings on Privacy Enhancing Technologies. (DOI)] took the opposite decision — fully disclosed as a study from the first email, IRB-approved without changes — and still measured effects.

Both are defensible and both were reviewed. What is not defensible is deciding covertly without review, or debriefing nobody. If you go covert: get it approved beforehand, debrief everyone at the end, and offer an opt-out that is honoured retroactively.

What a reviewer in this field will look for, drawn from what the studies above report and what the weaker ones omit:

1. The population and the unit. How many distinct parties, not how many findings. One contact often covers many domains — [4Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)] found different domains sharing a contact address, i.e. one owner, across both notified and control arms.
2. How you obtained contacts, per source, with coverage per source. “We used WHOIS” is no longer a method. State the source, the date, whether it was purchased, and what fraction of your population yielded nothing.
3. What the contact actually reaches. Site owner, hosting provider, CDN, registrar, or CERT. Give the share of abuse contacts that resolve to a CDN or shared host.
4. The full funnel: sent → bounced → spam-rejected → delivered → opened → report viewed → remediated, plus an explicit unknown bucket. Do not report only the last number.
5. A control group, its size, and the background remediation rate in it. If you cannot randomise, say so and say what that costs your claim.
6. The statistical test, with a multiple-comparison correction if you compare arms. Fisher's exact with Holm–Bonferroni [15Stock, Ben; Pellegrino, Giancarlo; Li, Frank; Backes, Michael; Rossow, Christian (2018): "Didn't You Hear Me? - Towards More Successful Web Vulnerability Notifications", in: Proceedings of the Network and Distributed System Security Symposium. (DOI), 9Utz, Christine; Michels, Matthias; Degeling, Martin; Marnau, Ninja; Stock, Ben (2023): "Comparing Large-Scale Privacy and Security Notifications", in: Proceedings on Privacy Enhancing Technologies. (DOI)] and permutation tests with Bonferroni [5Li, Frank; Durumeric, Zakir; Czyz, Jakub; Karami, Mohammad; Bailey, Michael; McCoy, Damon; Savage, Stefan; Paxson, Vern (2016): "You've Got Vulnerability: Exploring Effective Vulnerability Notifications", in: Proceedings of the USENIX Security Symposium. (Link)] are the precedents. Several effects in this literature disappear under correction — see Pvalue corrections.
7. The monitoring window and the reminder schedule, with dates. A rate without a window is not a rate.
8. Your detector's precision, because false positives went to real inboxes.
9. The disclosure timeline you followed and which convention it came from.
10. The message itself, in an appendix or artefact. Every study above that reports a framing effect published its text; you cannot replicate a framing result without it.
11. The ethics decision: review body and outcome, whether the study was covert, how you debriefed, the opt-out mechanism, and how many opted out.
12. Everything hostile that happened. Legal threats, complaints to your institution, suspended customers. This is the part later researchers most need and the part most often missing.

If you read four: [4Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)] for the channel survey and the delivery funnel, [5Li, Frank; Durumeric, Zakir; Czyz, Jakub; Karami, Mohammad; Bailey, Michael; McCoy, Damon; Savage, Stefan; Paxson, Vern (2016): "You've Got Vulnerability: Exploring Effective Vulnerability Notifications", in: Proceedings of the USENIX Security Symposium. (Link)] for the arm-by-arm comparison and the decay curve, [3Maass, Max; Stöver, Alina; Pridöhl, Henning; Bretthauer, Sebastian; Herrmann, Dominik; Hollick, Matthias; Spiecker, Indra (2021): "Effective Notification Campaigns on the Web: A Matter of Trust, Framing, and Support", in: Proceedings of the USENIX Security Symposium. (Link)] for the only large factorial design on framing and medium, and [1Lone, Qasim; Frik, Alisa; Luckie, Matthew; Korczyński, Maciej; van Eeten, Michel; Gañán, Carlos (2022): "Deployment of Source Address Validation by Network Operators: A Randomized Control Trial", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] for the control group that undermines the rest. Then [9Utz, Christine; Michels, Matthias; Degeling, Martin; Marnau, Ninja; Stock, Ben (2023): "Comparing Large-Scale Privacy and Security Notifications", in: Proceedings on Privacy Enhancing Technologies. (DOI)] for privacy versus security and the delivery figures that supersede the 2016 channel advice, [15Stock, Ben; Pellegrino, Giancarlo; Li, Frank; Backes, Michael; Rossow, Christian (2018): "Didn't You Hear Me? - Towards More Successful Web Vulnerability Notifications", in: Proceedings of the Network and Distributed System Security Symposium. (DOI)] for why message tuning is a dead end, [13Li, Frank; Ho, Grant; Kuan, Eric; Niu, Yuan; Ballard, Lucas; Thomas, Kurt; Bursztein, Elie; Paxson, Vern (2016): "Remedying Web Hijacking: Notification Effectiveness and Webmaster Comprehension", in: Proceedings of the ACM Web Conference. (DOI)] for what a platform can do that you cannot, and [6Stivala, Giada; Mrowczynski, Rafael; Hellenthal, Maria; Pellegrino, Giancarlo (2026): "Behind the Curtain: How Shared Hosting Providers Respond to Vulnerability Notifications", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] for the receiving end.

• Ethics — harm from crawling and scanning, robots.txt, ethics approval.
• Legal enforcement — when the right recipient is a regulator rather than an operator.
• Public relations — the other post-publication channel.
• Hypothesis testing and Pvalue corrections — for the arm comparisons.
• Artifacts — publishing the notification text and the contact-discovery code.
• Website selection — your notified population is your sample, with the same biases.

Every corpus figure above comes from scripts/report_notifying_websites.mjs over data/extract/run1 (5,859 papers, 2010–2026). Three limits carry directly:

• Seven venues only. EuroS&P, ACSAC, RAID, AsiaCCS, CHI and SOUPS are absent, and CHI/SOUPS are where a good deal of the operator-facing usable-security work appears. NDSS 2016 and NDSS 2018 full text was not retrieved at all, which is why [15Stock, Ben; Pellegrino, Giancarlo; Li, Frank; Backes, Michael; Rossow, Christian (2018): "Didn't You Hear Me? - Towards More Successful Web Vulnerability Notifications", in: Proceedings of the Network and Distributed System Security Symposium. (DOI)] — the canonical follow-up study — was read from the publisher's PDF rather than from the corpus.
• ethics.notifiedAffectedParties agreed with an independent re-extraction on 67% of papers, so treat every percentage above as accurate to a few points, not to the decimal.¹⁶⁾ ethics.disclosureDetail is free text capped at 20 words, so it is reported only as folded families and only as a ranking; the fold rules and the unmapped residue are on the provenance page.
• The 22 campaigns are hand-classified, not a census. A regex over 5,869 full texts produced 179 candidates; 22 were classified as campaigns, 16 rejected with a stated reason, and 144 were never read. The table above is a curated reading list, and the unreviewed residue is published in full.

The complete query log, the report script with its unedited output, the folding rules with their unmapped residue, the spot-checked quotes, and every external source that was rejected are on notifying_websites. Corpus-wide caveats are on Corpus.

[1]

Lone, Qasim; Frik, Alisa; Luckie, Matthew; Korczyński, Maciej; van Eeten, Michel; Gañán, Carlos (2022): "Deployment of Source Address Validation by Network Operators: A Randomized Control Trial", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)

[2]

Qin, Lancheng; Chen, Li; Li, Dan; Ye, Honglin; Wang, Yutian (2024): "Understanding Route Origin Validation (ROV) Deployment in the Real World and Why MANRS Action 1 Is Not Followed", in: Proceedings of the Network and Distributed System Security Symposium. (Link)

[3]

Maass, Max; Stöver, Alina; Pridöhl, Henning; Bretthauer, Sebastian; Herrmann, Dominik; Hollick, Matthias; Spiecker, Indra (2021): "Effective Notification Campaigns on the Web: A Matter of Trust, Framing, and Support", in: Proceedings of the USENIX Security Symposium. (Link)

[4]

Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)

[5]

Li, Frank; Durumeric, Zakir; Czyz, Jakub; Karami, Mohammad; Bailey, Michael; McCoy, Damon; Savage, Stefan; Paxson, Vern (2016): "You've Got Vulnerability: Exploring Effective Vulnerability Notifications", in: Proceedings of the USENIX Security Symposium. (Link)

[6]

Stivala, Giada; Mrowczynski, Rafael; Hellenthal, Maria; Pellegrino, Giancarlo (2026): "Behind the Curtain: How Shared Hosting Providers Respond to Vulnerability Notifications", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)

[7]

Lu, Chaoyi; Liu, Baojun; Zhang, Yiming; Li, Zhou; Zhang, Fenglu; Duan, Haixin; Liu, Ying; Chen, Joann Qiongna; Liang, Jinjin; Zhang, Zaifeng; Hao, Shuang; Yang, Min (2021): "From WHOIS to WHOWAS: A Large-Scale Measurement Study of Domain Registration Privacy under the GDPR", in: Proceedings of the Network and Distributed System Security Symposium. (Link)

[8]

Poteat, Tara; Li, Frank (2021): "Who You Gonna Call? An Empirical Evaluation of Website security.txt Deployment", in: Proceedings of the ACM Internet Measurement Conference. (DOI)

[9]

Utz, Christine; Michels, Matthias; Degeling, Martin; Marnau, Ninja; Stock, Ben (2023): "Comparing Large-Scale Privacy and Security Notifications", in: Proceedings on Privacy Enhancing Technologies. (DOI)

[10]

El Yadmani, Soufian; Gadyatskaya, Olga; Zhauniarovich, Yury (2025): "The File That Contained the Keys Has Been Removed: An Empirical Analysis of Secret Leaks in Cloud Buckets and Responsible Disclosure Outcomes", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)

[11]

Bennett, Nathaniel; Sowards, Rebekah; Deccio, Casey T. (2022): "SPFail: Discovering, Measuring, and Remediating Vulnerabilities in Email Sender Validation", in: Proceedings of the ACM Internet Measurement Conference. (DOI)

[12]

Munteanu, Cristian; Smaragdakis, Georgios; Feldmann, Anja; Fiebig, Tobias (2025): "Catch-22: Uncovering Compromised Hosts using SSH Public Keys", in: Proceedings of the USENIX Security Symposium. (Link)

[13]

Li, Frank; Ho, Grant; Kuan, Eric; Niu, Yuan; Ballard, Lucas; Thomas, Kurt; Bursztein, Elie; Paxson, Vern (2016): "Remedying Web Hijacking: Notification Effectiveness and Webmaster Comprehension", in: Proceedings of the ACM Web Conference. (DOI)

[14]

Sasaki, Takayuki; Fujita, Akira; Gañán, Carlos Hernandez; van Eeten, Michel; Yoshioka, Katsunari; Matsumoto, Tsutomu (2022): "Exposed Infrastructures: Discovery, Attacks and Remediation of Insecure ICS Remote Management Devices", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)

[15]

Stock, Ben; Pellegrino, Giancarlo; Li, Frank; Backes, Michael; Rossow, Christian (2018): "Didn't You Hear Me? - Towards More Successful Web Vulnerability Notifications", in: Proceedings of the Network and Distributed System Security Symposium. (DOI)

[16]

Canali, Davide; Balzarotti, Davide; Francillon, Aurélien (2013): "The Role of Web Hosting Providers in Detecting Compromised Websites", in: Proceedings of the ACM Web Conference. (DOI)

[17]

Durumeric, Zakir; Kasten, James; Adrian, David; Halderman, J. Alex; Bailey, Michael D.; Li, Frank; Weaver, Nicholas; Amann, Johanna; Beekman, Jethro; Payer, Mathias; Paxson, Vern (2014): "The Matter of Heartbleed", in: Proceedings of the ACM Internet Measurement Conference. (DOI)

[18]

Cetin, Orcun; Gañán, Carlos; Altena, Lisette; Kasama, Takahiro; Inoue, Daisuke; Tamiya, Kazuki; Tie, Ying; Yoshioka, Katsunari; van Eeten, Michel (2019): "Cleaning Up the Internet of Evil Things: Real-World Evidence on ISP and Consumer Efforts to Remove Mirai", in: Proceedings of the Network and Distributed System Security Symposium. (Link)

[19]

Nguyen, Trung Tin; Backes, Michael; Marnau, Ninja; Stock, Ben (2021): "Share First, Ask Later (or Never?) Studying Violations of GDPR's Explicit Consent in Android Apps", in: Proceedings of the USENIX Security Symposium. (Link)

[20]

Squarcina, Marco; Tempesta, Mauro; Veronese, Lorenzo; Calzavara, Stefano; Maffei, Matteo (2021): "Can I Take Your Subdomain? Exploring Same-Site Attacks in the Modern Web", in: Proceedings of the USENIX Security Symposium. (Link)

[21]

Gamero-Garrido, Alexander; Savage, Stefan; Levchenko, Kirill; Snoeren, Alex C. (2017): "Quantifying the Pressure of Legal Risks on Third-party Vulnerability Research", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)

[22]

Hantke, Florian; Roth, Sebastian; Mrowczynski, Rafael; Utz, Christine; Stock, Ben (2024): "Where Are the Red Lines? Towards Ethical Server-Side Scans in Security and Privacy Research", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)

[23]

Stöver, Alina; Gerber, Nina; Pridöhl, Henning; Maass, Max; Bretthauer, Sebastian; Spiecker gen. Döhmann, Indra; Hollick, Matthias; Herrmann, Dominik (2023): "How Website Owners Face Privacy Issues: Thematic Analysis of Responses from a Covert Notification Study Reveals Diverse Circumstances and Challenges", in: Proceedings on Privacy Enhancing Technologies. (DOI)