| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| practices:notifying_websites [2026/08/13 20:03] – Fix a broken table (invalid \| pipe escape shifted every figure one column); add the crawled-and-assessed-a-law cross-tab, the population closest to this page's reader. Authored by Claude karel.kubicek.claude | practices:notifying_websites [2026/08/21 14:50] (current) – Boxes: <wrap> renders a span, use uppercase <WRAP>; one box per list, not per bullet karel.kubicek.claude |
|---|
| ====== Notifying Websites ====== | ====== Notifying Websites ====== |
| |
| You crawled 100k sites and found that 8,000 of them leak something. Before you write the paper you have to tell those 8,000 operators, and every venue in this field now expects you to say so in the paper.((See //Disclosure timelines and what venues expect// below: IMC, USENIX Security, IEEE S&P and CCS all require it in some form as of their 2026 calls.)) This page is about the mechanics of doing that at scale: **how you get a contact for a party you have never met, what response rate to plan for, how long to wait before publishing, and what to report so a reviewer accepts the result.** | You crawled 100k sites and found that 8,000 of them leak something. Before you write the paper you have to decide what to do about those 8,000 operators, and the major venues now expect the paper to say what you decided.((See //Disclosure timelines and what venues expect// below: IMC, USENIX Security, IEEE S&P and CCS all require it in some form as of their 2026 calls; PoPETs sets out ethical principles but no disclosure requirement, and NDSS and TheWebConf are not addressed there.)) This page is about the mechanics of doing that at scale: **how you get a contact for a party you have never met, what response rate to plan for, how long to wait before publishing, and what to report so a reviewer accepts the result.** |
| |
| It is not about one-off coordinated disclosure to a named vendor. Emailing Google's security team is a solved problem with a queue and an SLA; emailing 8,000 strangers is a measurement in its own right, with a delivery funnel, a control group and a p-value. Treat it as one. | It is not about one-off coordinated disclosure to a named vendor. Emailing Google's security team is a solved problem with a queue and an SLA; emailing 8,000 strangers is a measurement in its own right, with a delivery funnel, a control group and a p-value. Treat it as one. |
| |
| <WRAP important> | <WRAP important> |
| The single most consequential design decision on this page is the **control group**. Sites get fixed for reasons that have nothing to do with you — automatic updates, unrelated maintenance, going offline. Every notification study without a control arm has attributed some of that background remediation to its own notifications — and **both of the properly randomised, control-arm experiments in this corpus found no significant effect from any treatment**, including treatments that observational studies had reported as effective {[lone2022_sav,qin2024_rov]}. Randomise, hold back an arm, and report both. Note what this does and does not mean: those two results are about network operators making an expensive configuration change, and {[maass2021_effective]}, which also had a control group, measured 56.6% against 9.2%. The lesson is not that notification never works — it is that a study without a control arm cannot tell you which case it is in. | The single most consequential design decision on this page is the **control group**. Sites get fixed for reasons that have nothing to do with you — automatic updates, unrelated maintenance, going offline. Every notification study without a control arm has attributed some of that background remediation to its own notifications — and **both randomised, control-arm experiments on network operators in this corpus found no significant effect from any treatment**, including treatments that observational studies had reported as effective {[lone2022_sav,qin2024_rov]}. Randomise, hold back an arm, and report both. Note what that does and does not mean: {[maass2021_effective]} is also a randomised controlled experiment — a full factorial design with a control group — and it measured 56.6% against 9.2% on German website owners. The lesson is not that notification never works. It is that a study without a control arm cannot tell you which of those two cases it is in. |
| </WRAP> | </WRAP> |
| |
| | **crawled ∧ assessed a law** | **123** | 35.0% | 14.6% | **14.6%** | 15.4% | 20.3% | | | **crawled ∧ assessed a law** | **123** | 35.0% | 14.6% | **14.6%** | 15.4% | 20.3% | |
| |
| Two things to take from this table. **Network measurement is ahead of web measurement**: scanning papers notify at 61.0% (''yes''+''partial'') against 43.5% for crawling papers, and are less than half as likely to leave the question unanswered. The scanning community built the norm first, largely because Internet-wide scanning provoked complaints that forced the question. **Legal-compliance papers state a position most often** (only 12.0% ''not-stated''), and the bottom row is the population closest to whoever is reading this: 123 papers that both crawled the web and assessed a law, i.e. found a compliance violation across many sites. **14.6% of them say outright that they did not notify** — more than three times the corpus-wide 4.3%. That is not negligence. A GDPR-violation finding across thousands of sites is the case where authors most often decide, and say, that individual notification is the wrong instrument, and route the finding to a regulator or to publication instead. If that is your situation, //deciding not to notify is a defensible position that you have to argue in the paper//, not an omission you can leave to the reader. | Two things to take from this table. **Network measurement is ahead of web measurement**: scanning papers notify at 61.0% (''yes''+''partial'') against 43.5% for crawling papers, and are less than half as likely to say outright that they did not (4.2% vs 9.4%), as well as meaningfully less likely to leave it unanswered (24.0% vs 31.1%). The scanning community built the norm first, largely because Internet-wide scanning provoked complaints that forced the question. **Legal-compliance papers state a position most often** (only 12.0% ''not-stated''), and the bottom row is the population closest to whoever is reading this: 123 papers that both crawled the web and assessed a law, i.e. found a compliance violation across many sites. **14.6% of them say outright that they did not notify** — more than three times the corpus-wide 4.3%. That is usually a decision rather than negligence. A GDPR-violation finding across thousands of sites is the case where authors most often decide, and say, that individual notification is the wrong instrument, and route the finding to a regulator or to publication instead. If that is your situation, //deciding not to notify is a defensible position that you have to argue in the paper//, not an omission you can leave to the reader. |
| |
| The direction of travel is unambiguous: | The direction of travel is unambiguous: |
| | Contacted a regulator or CERT | empirical ∧ ethics | 0.3% | 3.0% | 4.7% | 3.4% | 2.8% | | | Contacted a regulator or CERT | empirical ∧ ethics | 0.3% | 3.0% | 4.7% | 3.4% | 2.8% | |
| |
| <wrap todo>* 2025–2026 is provisional: CCS 2026 and IMC 2026 have not been held, and IEEE S&P 2026 and TheWebConf 2026 abstracts are not in OpenAlex, so those venue-years are under-represented by construction. See [[Literature:Corpus]].</wrap> | <WRAP todo>* 2025–2026 is provisional: CCS 2026 and IMC 2026 have not been held, and IEEE S&P 2026 and TheWebConf 2026 abstracts are not in OpenAlex, so those venue-years are under-represented by construction. See [[Literature:Corpus]].</WRAP> |
| |
| The one flat line is the interesting one. **Regulator and CERT contact has not grown and remains rare — 3.3% of the 4,472 say yes, and only 7.2% of the 376 papers that assessed a law.** Given that a CERT is often the only party that can reach a whole constituency, and that EU law now obliges every member state to run one for exactly this purpose (below), this is the largest gap between what is available and what is used. | The one line that is not rising is the interesting one. **Regulator and CERT contact has not grown and remains rare — 3.3% of the 4,472 say yes, and only 7.2% of the 376 papers that assessed a law.** Given that a CERT is often the only party that can reach a whole constituency, and that EU law now obliges every member state to run one for exactly this purpose (below), this is the largest gap between what is available and what is used. |
| | |
| | And when the intermediary //is// used it is used **as well as**, not instead of: of the 147 papers that contacted a regulator or CERT, **135 (91.8%) also notified the affected party directly**. Nobody in this corpus treats "we told the CERT" as discharging the obligation. Plan the intermediary as an extra arm, not as the cheap way out of finding contacts. |
| |
| ==== Saying you notified is not saying how ==== | ==== Saying you notified is not saying how ==== |
| |
| 2,870 of the 4,472 (64.2%) give some free-text detail about their disclosure. Folding that text for any mention of a channel — the fold rules and their residue are on the provenance page — **2,303 of those 2,870 (80.2%) name no channel at all**, and 2,155 (75.1%) name no outcome. Where a channel is named it is usually a single large platform (11.7% mention Google, Apple, Meta, Microsoft, Amazon or Mozilla by name); a generic email to the operator or developer is 3.9%, a CERT 1.4%, a hosting provider 1.3%, a bug-bounty programme 1.3%, WHOIS 0.6%, a data protection authority 0.6%. | 2,870 of the 4,472 (64.2%) give some free-text detail about their disclosure, and **all 2,160 that said they notified are among them**. Scoping to those 2,160 — a channel is only a fair question of a paper that says it notified somebody((Of the 710 papers that gave a detail without saying they notified, only 273 (38.5%) are participant-consent or debriefing notes; the rest are data-handling and harm-limitation notes, and a handful even name a channel. So the exclusion is about the question being ill-posed for that group, not about its text being all one kind.)) — and folding the text for any mention of a channel (the rules and their residue are on the provenance page): |
| | |
| | **1,633 of the 2,160 (75.6%) do not say through what channel.** Where a channel does appear it is usually a single large platform named outright (14.7% mention Google, Apple, Meta, Microsoft, Amazon or Mozilla); a generic email to the operator or developer is 4.6%, a CERT 1.8%, a bug-bounty programme 1.6%, a hosting provider 1.6%, WHOIS 0.7%, a data protection authority 0.6%. And 2,155 of the 2,870 (75.1%) name no outcome either. |
| |
| And **only 7 of the 2,870 report a countable response ratio in that text** — figures like "Contacted 318 of 559 sensitive organizations; received 11 responses by submission" or "Reported 110 malicious extensions to Google; 62.7% were removed". Every rate in the next section had to be read out of the papers' own results sections, not out of their ethics sections. That is the reporting gap this page exists to close: the modal paper in this corpus says //we disclosed responsibly// and stops. | Worse for anyone trying to plan a campaign: **only 7 of the 2,870 report a countable response ratio in that text** — figures like "Contacted 318 of 559 sensitive organizations; received 11 responses by submission" or "Reported 110 malicious extensions to Google; 62.7% were removed". Every rate in the next section had to be read out of the papers' own results sections, not out of their ethics sections. That is the reporting gap this page exists to close: the modal paper in this corpus says //we disclosed responsibly// and stops. |
| |
| ===== Getting a contact: what still works in 2026 ===== | ===== Getting a contact: what still works in 2026 ===== |
| ^ Channel ^ How you get it ^ What it reaches ^ Status and evidence ^ | ^ Channel ^ How you get it ^ What it reaches ^ Status and evidence ^ |
| | **''security.txt''** (RFC 9116) | ''GET https://<domain>/.well-known/security.txt'' | the security team, if there is one | RFC 9116, Informational, April 2022 — current, not obsoleted.((Verified against [[https://www.rfc-editor.org/rfc/rfc9116.txt|rfc-editor.org/rfc/rfc9116.txt]] on 2026-08-13: "Category: Informational … April 2022". Three errata exist, none touching the field list.)) Adoption is the problem: 11–16% of the Alexa top 100, 8–10% of the top 1K, "3–4% for the top 10K sites, and only a percent for the top 100K" {[poteat2021_securitytxt]}. **That measurement is from 2021 and its ranking frame no longer exists** — Alexa was discontinued in 2022 (see [[Design:Website selection]]), the paper predates RFC 9116, and **nothing in this corpus re-measures security.txt adoption since**. Assume it has risen; you have no citeable current figure | | | **''security.txt''** (RFC 9116) | ''GET https://<domain>/.well-known/security.txt'' | the security team, if there is one | RFC 9116, Informational, April 2022 — current, not obsoleted.((Verified against [[https://www.rfc-editor.org/rfc/rfc9116.txt|rfc-editor.org/rfc/rfc9116.txt]] on 2026-08-13: "Category: Informational … April 2022". Three errata exist, none touching the field list.)) Adoption is the problem: 11–16% of the Alexa top 100, 8–10% of the top 1K, "3–4% for the top 10K sites, and only a percent for the top 100K" {[poteat2021_securitytxt]}. **That measurement is from 2021 and its ranking frame no longer exists** — Alexa was discontinued in 2022 (see [[Design:Website selection]]), the paper predates RFC 9116, and **nothing in this corpus re-measures security.txt adoption since**. Assume it has risen; you have no citeable current figure | |
| | **RIR abuse contact** | RIPE ''abuse-c:'' / ARIN Abuse POC, resolved from the IP; RIPEstat's abuse-contact-finder covers all five RIRs over HTTPS | the **hosting provider or CDN**, not the site owner | The one source that scales — free and with no meaningful rate limit. ARIN verifies its Abuse POC annually;((ARIN NRPM §3.6: "Each of the following Points of Contact are to be verified annually … Admin, Tech, NOC, Abuse", [[https://www.arin.net/participate/policy/nrpm/|arin.net]], fetched 2026-08-13. RIPE states it works to keep abuse contacts valid but no validation cadence could be found on a RIPE primary source on 2026-08-13 — do not assume annual.)) RIPE states it keeps abuse contacts valid but publishes no cadence I could verify. Still the workhorse: "WHOIS was the most frequently mentioned channel (seven HPOs)" among 24 interviewed providers in 2026, who "confirmed that their contact is available via WHOIS, for example through ARIN or RIPE databases" {[stivala2026_behind]}. Note this is the //network// registry, which GDPR redaction did not touch — not the //domain// registrant record, which it did | | | **RIR abuse contact** | RIPE ''abuse-c:'' / ARIN Abuse POC, resolved from the IP; RIPEstat's abuse-contact-finder covers all five RIRs over HTTPS | the **hosting provider or CDN**, not the site owner | The one source that scales — free and with no meaningful rate limit. ARIN verifies its Abuse POC annually;((ARIN NRPM §3.6: "Each of the following Points of Contact are to be verified annually … Admin, Tech, NOC, Abuse", [[https://www.arin.net/participate/policy/nrpm/|arin.net]], fetched 2026-08-13. RIPE states it works to keep abuse contacts valid but no validation cadence could be found on a RIPE primary source on 2026-08-13 — do not assume annual.)) RIPE states it keeps abuse contacts valid but publishes no cadence that could be verified from a RIPE source. Still the workhorse: "WHOIS was the most frequently mentioned channel (seven HPOs)" among 24 interviewed providers in 2026, who "confirmed that their contact is available via WHOIS, for example through ARIN or RIPE databases" {[stivala2026_behind]}. Note this is the //network// registry, which GDPR redaction did not touch — not the //domain// registrant record, which it did | |
| | **PeeringDB technical contact** | PeeringDB API, per AS | the network operator | Preferred over WHOIS by {[lone2022_sav]}: "We preferred peeringDB because it has been used in previous studies and they found the database up-to-date" — i.e. it is relaying prior work's assessment, not measuring it | | | **PeeringDB technical contact** | PeeringDB API, per AS | the network operator | Preferred over WHOIS by {[lone2022_sav]}: "We preferred peeringDB because it has been used in previous studies and they found the database up-to-date" — i.e. it is relaying prior work's assessment, not measuring it | |
| | **The site's own imprint / privacy-policy contact** | read it off the page | the legally responsible party — often the actual decision-maker | The highest-yield channel measured. Addresses parsed from privacy-policy and contact pages achieved **87.8% delivery against 33.8% for RFC 2142 aliases** {[utz2023_comparing]}; 9 of 11 organisations reached via a privacy-policy address resolved the issue {[elyadmani2025_keys]}. {[maass2021_effective]} collected German ''Impressum'' addresses by hand, three researchers per site | | | **The site's own imprint / privacy-policy contact** | read it off the page | the legally responsible party — often the actual decision-maker | The highest-yield channel measured. Addresses parsed from privacy-policy and contact pages achieved **87.8% delivery against 33.8% for RFC 2142 aliases** {[utz2023_comparing]}; 9 of 11 organisations reached via a privacy-policy address resolved the issue {[elyadmani2025_keys]}. {[maass2021_effective]} collected German ''Impressum'' addresses by hand, three researchers per site | |
| ==== A tested contact-discovery cascade ==== | ==== A tested contact-discovery cascade ==== |
| |
| Three of those channels are machine-queryable. This runs them in cost order and prints per-source coverage, because a pooled "we found contacts for N% of domains" is the figure a reviewer sends back. Failures are printed rather than swallowed: a silent zero and a missing contact look identical, and the difference understates your own coverage. | Two of those channels are machine-queryable per domain, and RDAP is worth querying even though it is mostly redacted, because what it returns tells you //which kind// of redaction you are up against. (PeeringDB is machine-queryable too, but per AS — use it when your unit is a network operator rather than a website.) This runs the three in cost order and prints per-source coverage, because a pooled "we found contacts for N% of domains" is the figure a reviewer sends back. Failures are printed rather than swallowed: a silent zero and a missing contact look identical, and the difference understates your own coverage. |
| |
| <file python find_contacts.py> | <file python find_contacts.py> |
| * scraping mailto: links off the page -- highest yield, worst measured | * scraping mailto: links off the page -- highest yield, worst measured |
| delivery, and it collects addresses nobody published for this purpose. | delivery, and it collects addresses nobody published for this purpose. |
| | |
| | Why RIPEstat rather than the Abusix Abuse Contact DB for the RIR abuse contact: |
| | Abusix is a DNS TXT lookup, free and unmetered, and is what the 2016 USENIX |
| | campaign used, so it is the better choice at real scale. RIPEstat is used here |
| | because it needs nothing but the standard library over HTTPS, which keeps this |
| | script runnable -- and therefore testable -- anywhere. Swap in Abusix once you have |
| | a DNS client and thousands of domains. |
| |
| Failures are printed, never swallowed: a silent zero looks identical to a | Failures are printed, never swallowed: a silent zero looks identical to a |
| missing contact and would understate your own coverage. | missing contact and would understate your own coverage. The one exception is the |
| | IANA bootstrap fetch, which raises: it is not a per-domain zero, it makes RDAP |
| | coverage unmeasurable for the whole batch. |
| """ | """ |
| |
| google.com security.txt mailto:security@google.com expires=2030-04-01T00:00:00z | google.com security.txt mailto:security@google.com expires=2030-04-01T00:00:00z |
| google.com RDAP — response carried no email — redacted, or role-only | google.com RDAP — response carried no email — redacted, or role-only |
| google.com RIR abuse network-abuse@google.com ip=74.125.29.113 rir=arin | google.com RIR abuse network-abuse@google.com ip=172.217.208.139 rir=arin |
| ethz.ch security.txt mailto:security@ethz.ch expires=2028-01-31T07:00:00.000Z | ethz.ch security.txt mailto:security@ethz.ch expires=2028-01-31T07:00:00.000Z |
| ethz.ch RDAP — .ch has no RDAP service in the IANA bootstrap | ethz.ch RDAP — .ch has no RDAP service in the IANA bootstrap |
| bbc.co.uk security.txt mailto:security@bbc.co.uk expires=2038-01-19T03:14:07Z | bbc.co.uk security.txt mailto:security@bbc.co.uk expires=2038-01-19T03:14:07Z |
| bbc.co.uk RDAP redacted@nominet.uk role=registrant | bbc.co.uk RDAP redacted@nominet.uk role=registrant |
| bbc.co.uk RIR abuse abuse@fastly.com ip=151.101.192.81 rir=arin | bbc.co.uk RIR abuse abuse@fastly.com ip=151.101.128.81 rir=arin |
| wikipedia.org security.txt mailto:security@wikimedia.org expires=2029-03-31T09:00:00.000Z | wikipedia.org security.txt mailto:security@wikimedia.org expires=2029-03-31T09:00:00.000Z |
| wikipedia.org RDAP — response carried no email — redacted, or role-only | wikipedia.org RDAP — response carried no email — redacted, or role-only |
| {[sasaki2022_ics]} collects the comparison itself, and it is the shortest statement of the range: "its remediation rate was approximately 18% … Our remediation rate is higher than most previous notification experiments: it was approximately 40% for cross-site scripting and a WordPress vulnerability, 33%–42% for different WordPress vulnerability, and less than 20% for DNS zone poisoning. The only campaigns that reported similar remediation rates were on publicly accessible Git repositories (78%–81%) and Heartbleed (approximately 40%–90%)." | {[sasaki2022_ics]} collects the comparison itself, and it is the shortest statement of the range: "its remediation rate was approximately 18% … Our remediation rate is higher than most previous notification experiments: it was approximately 40% for cross-site scripting and a WordPress vulnerability, 33%–42% for different WordPress vulnerability, and less than 20% for DNS zone poisoning. The only campaigns that reported similar remediation rates were on publicly accessible Git repositories (78%–81%) and Heartbleed (approximately 40%–90%)." |
| |
| These are the campaigns in the corpus that reported a quotable outcome, each with **its own denominator** — they are not comparable to each other, because "response", "remediation" and "fix" are defined differently in each and the populations are wildly different. Plan against the range, not the mean. The hand pass classified 22 campaigns in total; 16 are below, and the six not shown reported a volume sent but no response or remediation figure. They are listed on [[provenance:practices:notifying_websites]], along with the one row below — {[stock2018_didnt]} — whose full text is missing from the corpus and was read from the publisher's PDF instead. | These are the campaigns in the corpus that reported a quotable outcome, each with **its own denominator** — they are not comparable to each other, because "response", "remediation" and "fix" are defined differently in each and the populations are wildly different. Plan against the range, not the mean. The hand pass classified 22 campaigns in total; 16 are below, and the six not shown reported a volume sent but no response or remediation figure. One row below — {[munteanu2025_catch22]} — is here for its channel rather than a rate, because delegating a campaign to an established notification operator is the pattern, not the number. They are listed on [[provenance:practices:notifying_websites]], along with the one row below — {[stock2018_didnt]} — whose full text is missing from the corpus and was read from the publisher's PDF instead. |
| |
| ^ Study ^ Notified ^ Channel ^ Outcome, in the paper's own terms ^ Control ^ | ^ Study ^ Notified ^ Channel ^ Outcome, in the paper's own terms ^ Control ^ |
| | {[li2016_youve]} USENIX Sec 2016 | 2,563 ICS + 3,536 IPv6 + 5,960 amplifier contacts | WHOIS abuse contact, national CERTs, US-CERT | "at most 18% of the population remediating" under the best regimen; direct verbose 9.8% vs national CERT 3.1% vs US-CERT 1.4% (IPv6, two days) | US-CERT arm statistically indistinguishable from control | | | {[li2016_youve]} USENIX Sec 2016 | 2,563 ICS + 3,536 IPv6 + 5,960 amplifier contacts | WHOIS abuse contact, national CERTs, US-CERT | "at most 18% of the population remediating" under the best regimen; direct verbose 9.8% vs national CERT 3.1% vs US-CERT 1.4% (IPv6, two days) | US-CERT arm statistically indistinguishable from control | |
| | {[li2016_remedying]} TheWebConf 2016 | 760,935 hijacking incidents | browser interstitial, search warning, Search Console message, WHOIS admin email | 59.5% of incidents resolved over 11 months; **82.4% / 76.8% where a Search Console alert reached a pre-registered webmaster** vs 54.6% / 43.4% otherwise | — (channel comparison) | | | {[li2016_remedying]} TheWebConf 2016 | 760,935 hijacking incidents | browser interstitial, search warning, Search Console message, WHOIS admin email | 59.5% of incidents resolved over 11 months; **82.4% / 76.8% where a Search Console alert reached a pre-registered webmaster** vs 54.6% / 43.4% otherwise | — (channel comparison) | |
| | {[stock2018_didnt]} NDSS 2018 | >24,000 domains, seven arms of ~4,000 | six email variants (plain, HTML, tracking, mailbot, S/MIME, friendly tone) | notified 24% (Git) and 17% (WordPress) fixed; **74.4% / 33.3% among those who opened the report** | 13% (Git), 14% (WordPress) | | | {[stock2018_didnt]} NDSS 2018 | >24,000 domains, seven arms of ~4,000 | six email variants (plain, HTML, tracking, mailbot, S/MIME, friendly tone) | notified 24% (Git) and 17% (WordPress) fixed; **74.4% / 33.3% among those who //viewed// the report** | 13% (Git), 14% (WordPress) | |
| | {[cetin2019_cleaning]} NDSS 2019 | ISP customers with Mirai infections | ISP walled garden (quarantine + notification) vs email-only vs nothing | walled garden "remediates 92% of the infections within 14 days"; **"Email-only notifications have no observable impact compared to a control group"** | **"The control group achieved the lowest cleanup rate (74%)"** with no notification at all — the cautionary figure on this whole page | | | {[cetin2019_cleaning]} NDSS 2019 | ISP customers with Mirai infections | ISP walled garden (quarantine + notification) vs email-only vs nothing | walled garden "remediates 92% of the infections within 14 days"; **"Email-only notifications have no observable impact compared to a control group"** | **"The control group achieved the lowest cleanup rate (74%)"** with no notification at all — the cautionary figure on this whole page | |
| | {[maass2021_effective]} USENIX Sec 2021 | 4,594 German site owners, 18 arms + control | postal letter and email, addresses read by hand from each ''Impressum'' | **"56.6 % of all notified operators remediating within two months"**; 76.3% for a legal-research-group letter citing fines, 33.9% for a computer-science email citing privacy | **9.2%** | | | {[maass2021_effective]} USENIX Sec 2021 | 4,594 German site owners, 18 arms + control | postal letter and email, addresses read by hand from each ''Impressum'' | **"56.6 % of all notified operators remediating within two months"**; 76.3% for a legal-research-group letter citing fines, 33.9% for a computer-science email citing privacy | **9.2%** | |
| ==== The loss is in delivery, not in willingness ==== | ==== The loss is in delivery, not in willingness ==== |
| |
| The delivery funnel is the story, and it is why headline remediation rates look so bad. {[stock2016_hey]}: **5.8% of reports received**, but ~40% remediation among those actually read. {[bennett2022_spfail]}: 31.6% undelivered → 12% of the delivered opened → 4% of the openers patched. {[stock2018_didnt]}: 74.4% fixed among Git operators who opened the report, against a 13% control. Operators who read your report largely act on it. Most never read it. | The delivery funnel is the story, and it is why headline remediation rates look so bad. {[stock2016_hey]}: **5.8% of reports received**, but ~40% remediation among those actually read. {[bennett2022_spfail]}: 31.6% undelivered → 12% of the delivered opened → 4% of the openers patched. {[stock2018_didnt]}: 74.4% fixed among Git operators who //viewed// the report, against a 13% control. Operators who read your report largely act on it. Most never read it. |
| |
| So the highest-leverage thing you can do is not writing a better email. It is: | So the highest-leverage thing you can do is not writing a better email. It is: |
| ^ Factor ^ Finding ^ Source ^ | ^ Factor ^ Finding ^ Source ^ |
| | Sender identity | Legal research group beats computer-science group: 59.7% vs 54% remediation (p<0.05) | {[maass2021_effective]} | | | Sender identity | Legal research group beats computer-science group: 59.7% vs 54% remediation (p<0.05) | {[maass2021_effective]} | |
| | Framing | Legal-compliance-plus-fine beats plain GDPR beats privacy-harm; survival 50.1% / 56.6% / 69.6%, all differences significant | {[maass2021_effective]} | | | Framing | Legal-compliance-plus-fine beats plain GDPR beats privacy-harm; survival 50.1% / 56.6% / 69.6%, all differences significant. //Survival// throughout this table is the share **still non-compliant**, so lower is better | {[maass2021_effective]} | |
| | Framing | **Contradicted.** A fine warning had **no** significant effect once the notification arrived; only //receiving// it mattered | {[utz2023_comparing]} | | | Framing | **Contradicted.** A fine warning had **no** significant effect once the notification arrived; only //receiving// it mattered | {[utz2023_comparing]} | |
| | Medium | A **postal letter** beats email: survival 55.6% vs 66.3% (p<0.0001), "increasing the remediation rate by between 3.9 and 17.9 percentage points (mean: 11.1)" — for "around 5000 € on domestic postage" across 2,660 letters | {[maass2021_effective]} | | | Medium | A **postal letter** beats email: survival 55.6% vs 66.3% (p<0.0001), "increasing the remediation rate by between 3.9 and 17.9 percentage points (mean: 11.1)" — for "around 5000 € on domestic postage" across 2,660 letters | {[maass2021_effective]} | |
| |
| * **Seven venues only.** EuroS&P, ACSAC, RAID, AsiaCCS, CHI and SOUPS are absent, and CHI/SOUPS are where a good deal of the operator-facing usable-security work appears. NDSS 2016 and NDSS 2018 full text was not retrieved at all, which is why {[stock2018_didnt]} — the canonical follow-up study — was read from the publisher's PDF rather than from the corpus. | * **Seven venues only.** EuroS&P, ACSAC, RAID, AsiaCCS, CHI and SOUPS are absent, and CHI/SOUPS are where a good deal of the operator-facing usable-security work appears. NDSS 2016 and NDSS 2018 full text was not retrieved at all, which is why {[stock2018_didnt]} — the canonical follow-up study — was read from the publisher's PDF rather than from the corpus. |
| * ''ethics.notifiedAffectedParties'' **agreed with an independent re-extraction on 67% of papers**, so treat the percentages as accurate to a few points, not to the decimal. ''ethics.disclosureDetail'' is free text capped at 20 words and is reported below only as folded families with the residue printed. | * ''ethics.notifiedAffectedParties'' **agreed with an independent re-extraction on 67% of papers**, so treat every percentage above as accurate to a few points, not to the decimal.((That 67% was measured on 100 papers of the previous, 4,322-paper extraction run and has not been re-measured on the current corpus. Treat it as the right order of magnitude.)) ''ethics.disclosureDetail'' is free text capped at 20 words, so it is reported only as folded families and only as a ranking; the fold rules and the unmapped residue are on the provenance page. |
| * **The 22 campaigns are hand-classified, not a census.** A regex over 5,869 full texts produced 179 candidates; 22 were classified as campaigns, 16 rejected with a stated reason, and **144 were never read**. The table above is a curated reading list, and the unreviewed residue is published in full. | * **The 22 campaigns are hand-classified, not a census.** A regex over 5,869 full-text files (ten more than the 5,859 extraction records — a few papers have text but no record) produced 179 candidates; 22 were classified as campaigns, 13 rejected with a stated reason, and **144 were never read**. Three further rejections came from a wider first-pass scan and fall outside the committed regex, which is why the reasoned rejections total 16. The table above is a curated reading list, and the unreviewed residue is published in full. |
| |
| The complete query log, the report script with its unedited output, the folding rules with their unmapped residue, the spot-checked quotes, and every external source that was rejected are on **[[provenance:practices:notifying_websites]]**. Corpus-wide caveats are on [[Literature:Corpus]]. | The complete query log, the report script with its unedited output, the folding rules with their unmapped residue, the spot-checked quotes, and every external source that was rejected are on **[[provenance:practices:notifying_websites]]**. Corpus-wide caveats are on [[Literature:Corpus]]. |
