| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| privacy:consent [2025/02/24 14:52] – [Granting Consent to Websites] karelkubicek | privacy:consent [2026/08/26 21:37] (current) – Link to the new privacy:tcf_consent_strings; correct the consensu.org shared-cookie paragraph (global scope deprecated 2021-06-22, invalid since 2021-09-01); note that the probe's getTCData command was deprecated in TCF v2.2. Authored by Claude karel.kubicek.claude |
|---|
| ====== Granting Consent to Websites ====== | ====== Granting Consent to Websites ====== |
| |
| <wrap todo>WiP: Brainstorming of topics to cover.</wrap> | Somewhere near the top of every European crawl there is a box asking permission. What your crawler does with it — click //Accept//, click //Reject//, dismiss it, or drive straight past — is not a configuration detail. It is the **treatment** in your experiment, and it decides which web you measured. A crawl that never touched the banner measures the pre-consent web; a crawl that accepted everything measures the maximally-tracked web; the gap between them is where most of the interesting results are. |
| |
| EU perspective | This page is about that decision and everything downstream of it: how to make the browser express a consent choice, how to check the site actually recorded it, how to read what it recorded, and what the alternative channel — an automated signal such as [[#Global Privacy Control|GPC]] instead of a click — can and cannot do. **Finding** the notice and **labelling** its buttons is a DOM-classification problem and lives on [[Privacy:Requests#Cookie Notices and Their Interactive Elements|Classifying Web Requests]]; the crawler-side mechanics of clicking anything at all are on [[Programming:Interaction|Interaction]]. What the resulting cookies mean is on [[Privacy:Cookies|Classifying Cookies]]. |
| * Detection of cookie notices: EasyCookie list, z-index, NLP, heuristics | |
| * Interaction with notices: | |
| * Consent-O-Matic and forks: Autoconsent by Mozilla, DuckDuckGo, and Brave | |
| * Provides possibility to set concrete consent mode (accept specific processing purposes) | |
| * But it is limited to only set of CMPs | |
| * I (still) don't care about cookies | |
| * Some notices are consented (typically accept all), some are just removed | |
| * ML classification of consent choices: Khandewal et al. (2023) and Bouhoula et al. (2024) | |
| |
| US/global perspective | <WRAP important> |
| * Global Privacy Control (GPC) signal | **The finding that should shape your methods section.** Of the **1,120** papers in [[literature:corpus|this corpus]] that ran an automated web crawl, **349 (31.2%)** say what they did about consent notices at all. Of those 349, **313 say they did not interact with the notice**. That leaves 36 that appear to have interacted with one — and reading all 36 against their own full text leaves **29 papers in the whole 2010–2026 corpus, across seven venues, that verifiably interacted with a consent notice.** That is **2.6% of all crawling papers**.((The other 7 are extraction false positives, hand-adjudicated on 2026-08-19 and listed one by one on [[provenance:privacy:consent|the provenance page]]. They are the reason the page reports 29 rather than 36; see [[#Almost nobody says what they did about consent]].)) |
| * Do-Not-Track (DNT) | |
| * Classifying opt-out links: https://dl.acm.org/doi/abs/10.1145/3366423.3380262 | |
| |
| | Two consequences. First, the reviewer question "what did you do about the banner?" has no established answer to point at, so **state yours explicitly** — you will be in a minority that does. Second, when you compare your prevalence number against a published one, check what that paper did with the notice before you conclude anything from the difference. Excluding the 236 papers for which the question genuinely does not arise, **about three in five say nothing**, and the difference may be entirely the treatment. |
| | </WRAP> |
| |
| | ===== What to Read First ===== |
| |
| /* | * **Do Cookie Banners Respect my Choice?** {[matte2020_cookie]}, IEEE S&P 2020 — the paper that turned "is this banner legal?" into something a crawler can decide, by reading what the CMP //stored// rather than what the banner //said//. Read it for the four-violation taxonomy, which almost every later compliance paper reuses. |
| This is a comment not visible on the page. It outlines the syntax (for more, go to https://measuretheweb.org/wiki/syntax), especially that related to bibliography. Remove it once you created the page. If you use any citations (documented at bottom), keep the References section. | * **Automated Large-Scale Analysis of Cookie Notice Compliance** {[bouhoula2024automated]}, USENIX Security 2024 — the current reference pipeline: find the notice, classify its buttons with a small language model, drive it, and check the cookies against the site's own declarations. Read it for the numbers you will be asked to beat. |
| | * **(Un)informed Consent** {[utz2019_informed]}, CCS 2019 and **Dark Patterns after the GDPR** {[nouwens2020_dark]}, CHI 2020 — the two studies that established that banner //design// moves consent rates, so a banner is a manipulation and not a question. Nouwens et al. is outside this corpus's seven venues; it is still the one everybody cites. |
| | * **A Large-Scale Study of Cookie Banner Interaction Tools** {[demir2024_bannertools]}, PETS 2024 — read before you decide to use an off-the-shelf extension, because it measures how often they actually work. |
| | * **Intractable Cookie Crumbs** {[rasaii2025_crumbs]}, PETS 2025 — read for the single most under-appreciated mechanic: rejection often does nothing until the page is reloaded. |
| | * If your study is US-facing rather than EU-facing, start instead at **Usability and Enforceability of Global Privacy Control** {[zimmeck2023_gpc]}, PETS 2023 and **Websites' Global Privacy Control Compliance at Scale and over Time** {[hausladen2025_websites]}, USENIX Security 2025. |
| |
| ===== Header Level 2 ===== | ===== The Decision: What Your Crawler Does With the Notice ===== |
| ==== Header Level 3 ==== | |
| |
| === Links === | There are six things a crawl can do, and the corpus records which one each paper chose. They are not interchangeable and they are not on a scale — each answers a different question. |
| |
| External links are recognized automatically: www.google.com, but if you want a link text: [[http://www.google.com|This Link points to google]]. | ^ Action ^ What it measures ^ What it costs you ^ Papers stating it ^ Verified ^ |
| | | **no interaction** — load and leave the banner alone | The **pre-consent** web: what a site does before it has any legal basis. This is the right treatment for a violation study, because tracking before consent is the violation | Not what a user experiences. Any "how much tracking is there" number from this treatment is a floor, not a level | **313** | not audited | |
| | | **accept all** | The **upper bound**: everything the site is prepared to do with permission. The right treatment for enumerating vendors, purposes and the full cookie set | Says nothing about compliance, and over-states what a typical user is exposed to | 15 | **11** | |
| | | **reject all** | Whether refusal is honoured. Only meaningful when paired with another arm | On its own it is uninterpretable — you cannot tell "respects rejection" from "has no tracking anyway" | 2 | **2** | |
| | | **accept and reject** (two arms, same sites) | The **difference**, which is the quantity almost every consent result is actually about | Doubles the crawl, and the two arms must be otherwise identical — same vantage, same time window, same profile handling | 14 | **14** | |
| | | **CMP-specific choices** — set individual purposes | Purpose-level compliance: did rejecting //advertising// stop advertising cookies? | Only works on CMPs whose interface you can drive; the population becomes "sites using CMPs your tool supports", which is not the web | 2 | **2** | |
| | | **dismiss or remove** — close the banner, or delete it from the DOM | Nothing about consent. Useful only to unblock a crawl whose real subject is something else | Removing the banner from the DOM is **not** a consent choice: no consent string is written, and the site may behave as it does pre-consent. Say "we removed the overlay", never "we declined" | 3 | **0** | |
| |
| Internal links are created by using square brackets. You can either just give a [[pagename]] or use an additional [[pagename|link text]]. | The //Verified// column comes from reading all 36 candidate papers' own full text (see [[#Almost nobody says what they did about consent|below]] and [[provenance:privacy:consent|the provenance page]]). The pattern in it is worth carrying away even if you never touch this corpus: **the enum is trustworthy exactly where the paper had to describe two arms**, and unreliable where a single ambiguous word in a methods paragraph could be misread. |
| |
| === Lists === | <WRAP important> |
| | **Two arms or no claim.** If your result is "sites track users who rejected", you need the rejection arm //and// a baseline. If your result is "sites track before consent", you need the no-interaction arm. A single accept-all crawl supports neither. Only **14 papers in this corpus** run both an accept and a reject arm — and that is the one value the audit above confirmed at 14 out of 14. It is why so many consent findings are hard to compare. |
| | </WRAP> |
| |
| Lists and their levels are decided by indentation (2 spaces = 1 level) | ===== Verifying That the Consent Actually Registered ===== |
| |
| - this is 1. item | This is the step that most often goes wrong, and it goes wrong silently: the click misses, the crawl carries on, and pre-consent behaviour is reported as post-consent behaviour. **A click is not a measurement. The stored consent is.** Check all four of these: |
| - 2. item | |
| - nested a. item | |
| * bullet-point item | |
| |
| === Code === | - **Did the element exist and did the click land?** Log the element you clicked — its text, its selector, its bounding box — not just "clicked". An element that scrolls out of the viewport, sits under an overlay, or lives in a shadow DOM or a cross-origin ''<iframe>'' will accept a synthetic click and do nothing. |
| | - **Did the banner go away?** Re-run your detector after the click. The banner disappearing is weak evidence but a banner that is still there is strong evidence of failure. |
| | - **Did the site store a decision?** This is the real check. Read the CMP's own record — the TC string for a TCF banner (below), or the vendor's consent cookie (''OptanonConsent'' for OneTrust, ''CookieConsent'' for Cookiebot, ''euconsent-v2'' for TCF, ''cookieyes-consent'', and so on) — and assert that its contents match the choice you made. Matte et al. {[matte2020_cookie]} built their whole method on this: they compare what the banner offered against what the CMP stored, which is why they can detect a positive consent stored **before** the user chose at all (**141 of 1,426 TCF sites, 9.9%**) and one stored **after an explicit refusal** (**27 of 508, 5.3%**). |
| | - **Did you reload?** Rasaii et al. {[rasaii2025_crumbs]} found that rejecting a banner produces **no immediate change** in the cookies being sent, and that on average **25% of "intractable" cookies stop being sent only after the page is reloaded** with the rejection in place. A crawl that rejects and then immediately measures will conclude that rejection does nothing. Reload, in a stateful session, and measure again. |
| |
| For a short inline monospace, use ''double quote''. For proper code (but in a separate paragraph), use ''<code LANG>'': | <WRAP important> |
| | Whether the reload can carry the decision at all is a [[Programming:Stateful Stateless|stateful/stateless]] question. A stateless crawl that clears the profile between visits cannot observe post-consent behaviour on a **later** visit, because the consent cookie went with the profile. In this corpus only **115 crawling papers state both a consent action and a statefulness**, and **93 of those are the no-interaction case**. If you interact with banners, statefulness is not an independent choice — say which you used and why. |
| | </WRAP> |
| |
| <code python> | <WRAP important> |
| string = "World" | **A fifth check, because on many sites the choice does not switch the tag off at all.** Any site running a Google tag (GA4, Google Ads, Tag Manager) can pass the user's consent state to Google rather than simply loading or not loading the tag, and Google requires advertisers serving the EEA to do so in order to keep ad personalisation and measurement.((Google, "Updates to consent mode for traffic in European Economic Area (EEA)", ''support.google.com/tagmanager/answer/13695607'', fetched 2026-08-19: "we are strengthening the enforcement of our EU user consent policy (EU UCP)… you must collect consent for use of personal data from end users based in the EEA and share consent signals with Google". The page does not itself carry the widely-quoted March 2024 enforcement date, so this page does not assert one.)) The consequence for a crawl is concrete: on a consent-mode site a rejection does **not** stop the Google tag from firing — it fires with ''ad_storage'' and ''analytics_storage'' denied and sends cookieless pings instead. A measurement that counts requests, or counts tags loaded, will conclude that rejection did nothing. A measurement that counts cookies set will conclude it worked. **Say which you counted.** |
| print(f'Hello {string}') | </WRAP> |
| </code> | |
| | ===== Reading What the Site Stored: the TCF ===== |
| | |
| | Most consent banners on European sites are operated by a **Consent Management Platform (CMP)**, and a large share of those implement IAB Europe's **Transparency and Consent Framework (TCF)**, which standardises both the API and the storage format. That standardisation is what makes consent machine-readable at scale, and it is why so many measurement papers are TCF papers. |
| | |
| | This section covers what you need to know about the TCF //to make and verify a consent choice//. The string itself — its segments and bit layout, the seven places it can be found, decoding it reproducibly against a pinned Global Vendor List, Google's separate Additional Consent string, and what a decoded string does and does not prove about a site's behaviour — is on [[Privacy:TCF Consent Strings|Decoding TCF Consent Strings]]. |
| | |
| | TCF sites are a **minority, and you must report the denominator as such.** Matte et al. {[matte2020_cookie]} found a TCF banner on **1,426 of 22,949** reachable European sites (**6.2%**) in 2019; Hils et al. {[hils2021_privacy]} tracked adoption longitudinally and detected TCF implementations on the order of thousands of sites in the top 100k. In Android apps the share is comparable: Morel et al. {[morel2026_tcf]} found TCF in **576 of 4,482 apps (12.85%)**. "Consent on the web" and "TCF consent" are not the same population. |
| |
| For large code, use ''<file LANG filename>'', it will make code downloadable. For instance: | The detection primitive is the API function itself. TCF v1.x exposes ''%%__cmp()%%''; TCF v2.x exposes ''%%__tcfapi()%%''. Probing for the function is precise but narrow — it finds TCF banners and nothing else. |
| |
| <file php example.php> | <file javascript tcf_probe.js> |
| <?php echo "hello world!"; ?> | // Run in the page context. Resolves to null when the site has no TCF CMP, |
| | // otherwise to the decoded-by-the-CMP consent state plus the raw TC string. |
| | // Deliberately has no try/catch: a page that defines __tcfapi and then throws |
| | // is a finding, not something to swallow. |
| | function readTcf(timeoutMs = 3000) { |
| | return new Promise((resolve) => { |
| | if (typeof window.__tcfapi !== 'function') return resolve(null); |
| | const timer = setTimeout(() => resolve({ error: 'no callback within timeout' }), timeoutMs); |
| | window.__tcfapi('getTCData', 2, (tcData, success) => { |
| | clearTimeout(timer); |
| | resolve({ |
| | success, |
| | cmpId: tcData.cmpId, // join to the IAB CMP list |
| | cmpVersion: tcData.cmpVersion, |
| | tcfPolicyVersion: tcData.tcfPolicyVersion, |
| | eventStatus: tcData.eventStatus, // 'cmpuishown' | 'useractioncomplete' | 'tcloaded' |
| | gdprApplies: tcData.gdprApplies, // false here means the CMP decided you are not in scope |
| | purposeConsents: tcData.purpose.consents, |
| | purposeLegitimateInterests: tcData.purpose.legitimateInterests, |
| | vendorConsents: tcData.vendor.consents, |
| | tcString: tcData.tcString, // the base64 blob also stored in euconsent-v2 |
| | }); |
| | }); |
| | }); |
| | } |
| </file> | </file> |
| |
| === Figures === | Run against four European news sites on 2026-08-19 from a non-EU vantage, before touching the banner, it returns this — the shape you should expect from a site whose banner is up and unanswered: |
| |
| To use floats, you have to use the ''<WRAP>'' tag. For instance, the following will create figure on the right side 50% large: | <code> |
| <WRAP right 50% box> | www.spiegel.de => {"success":true,"cmpId":6,"cmpVersion":1,"tcfPolicyVersion":5, |
| {{PATH_TO_FILE|ALT_TEXT}} | "eventStatus":"cmpuishown","gdprApplies":true, |
| <div>CAPTION</div> | "purposeConsents":{}, |
| | "purposeLegitimateInterests":{"1":false,"2":false,"3":false,"4":false, |
| | "5":false,"6":false,"7":true,"8":false,"9":true,"10":true}, |
| | "vendorConsentCount":0, |
| | "tcStringPrefix":"CQpMXQAQpMXQAAGABCENCsFgAAAAAALAAAQ4AAAM"} |
| | www.heise.de => {"success":true,"cmpId":6,...,"eventStatus":"cmpuishown","gdprApplies":true, |
| | "purposeConsents":{},"purposeLegitimateInterests":{},"vendorConsentCount":0} |
| | www.repubblica.it=> {"success":true,"cmpId":123,"cmpVersion":387,...,"tcStringPrefix":""} |
| | www.lemonde.fr => {"error":"no callback within timeout"} |
| | </code> |
| | |
| | Four things that output already teaches you. **''gdprApplies'' was ''true'' from a non-EU IP** on all three that answered — the CMP's own scope decision does not always track your vantage point, so measure it rather than assuming it. **''purposeConsents'' is empty and ''vendorConsentCount'' is 0** while the banner is still up: that is the correct pre-consent state and it is what your no-interaction arm should look like. **Spiegel already asserts legitimate interest for purposes 7, 9 and 10 before any click** — legitimate interest is not consent and is not recorded in ''purposeConsents'', so a study that only reads ''purposeConsents'' will under-count what the site is already doing. And **''lemonde.fr'' defines ''%%__tcfapi%%'' but never invokes the callback within four seconds**, which is why the timeout branch exists: without it your crawler hangs, and without recording the timeout you would score the site as having no TCF. |
| | |
| | Three things to record every time, because reviewers ask and papers routinely omit them: |
| | |
| | * **''cmpId'' and ''cmpVersion''.** ''cmpId'' joins to IAB Europe's registered-CMP list, which is how you attribute a behaviour to a vendor rather than to a publisher. Toth et al. {[toth2022_darkpatterns]} showed why that attribution matters: **six of the ten** CMP consent pop-ups they studied made //Accept// and //Reject// visually unequal **by default**, so the dark pattern was shipped by the CMP, not chosen by the site. |
| | * **''gdprApplies''.** The CMP decides this, usually from your IP. ''gdprApplies: false'' means you are measuring the non-EU variant of the site — see [[#Jurisdiction: Your Vantage Point Chooses Your Banner|below]]. |
| | * **''eventStatus''.** ''useractioncomplete'' is the only value that means a choice was recorded. ''cmpuishown'' with a consent already present is exactly Matte et al.'s //consent stored before choice// violation. |
| | |
| | <WRAP important> |
| | **A consent string set on one site used to be readable on another — this mechanism is retired, and the correction matters.** Under TCF v1.x, and under v2.0's optional //global scope//, a CMP could store the string in a shared cookie on the ''consensu.org'' domain — ''euconsent'' and then ''euconsent-v2'' — readable by any other CMP. Matte et al. {[matte2020_cookie]} tested this in 2019: they planted a consent string in the shared cookie and then asked each site's CMP for its state without touching the banner, and **62 sites (4.3%) returned the same consent string** — their CMP had adopted a consent created by a different CMP entirely. IAB Europe **announced the deprecation of global scope, out-of-band consent and the ''euconsent-v2'' cookie on 22 June 2021, and TC strings established with global scope have been invalid since 1 September 2021.**((IAB Tech Lab, //Transparency and Consent String with Global Vendor & CMP List Formats//, section "What happened to Global Scope and Out of Band?" and the version-history rows for July and September 2021. Fetched from the specification repository on 2026-08-26.)) So the cross-site leakage Matte et al. found is a historical finding, not a hazard of a crawl run today. What survives is the same-site version: a consent string persists across page loads within a site, so a stateful crawl still carries your earlier choice forward. Clear consent storage between visits, or crawl statelessly and reload — and say which you did. |
| </WRAP> | </WRAP> |
| |
| */ | **The ''getTCData'' command in the probe above is deprecated.** IAB Tech Lab deprecated it in the CMP API specification with TCF v2.2 (May 2023), in favour of registering an ''addEventListener'' callback; the three required commands are now ''ping'', ''addEventListener'' and ''removeEventListener''. In a four-site spot-check on 2026-08-26 all four CMPs that answered at all still returned a TC string for ''getTCData'', although one set the callback's ''success'' flag to ''false'' while doing so. Published crawlers using the command are therefore not broken. But a CMP is within spec to drop it, and the failure mode is a silently growing "no TCF" bucket. New code should use the listener; [[Privacy:TCF Consent Strings|Decoding TCF Consent Strings]] publishes a tested ''addEventListener'' capture snippet and covers the other six channels the string travels in. |
| |
| ====== References ====== | **Do not decode the TC string ad hoc.** It is a versioned bit-packed base64 format; use a maintained decoder and pin its version — and check //which// package you installed, because the reference implementation moved npm scope and left a three-year-old copy behind under the old name. [[Privacy:TCF Consent Strings|Decoding TCF Consent Strings]] has the details, an audited minimal decoder, and the bit layout it reads. Two papers in this corpus report writing their own decoding script, and both had to pin a **Global Vendor List** version to interpret the vendor bitfield — the GVL changes weekly, so a decoded vendor set is only meaningful together with the GVL version you decoded it against. The list is served from ''vendor-list.consensu.org/v3/vendor-list.json'' with numbered archives under ''/v3/archives/'', which is what makes a retrospective decode reproducible at all: **fetch and archive the GVL alongside your crawl**, do not resolve vendor IDs months later. |
| |
| /* | **The other reason to read the stored string: it lets you check compliance without trusting the interface.** Smith et al. {[smith2024_gdpr]} decoded TC strings across repeated crawls and found recorded-consent violations in **2.2% of domains and 1.3% of crawls** — a much lower rate than the "72% of sites violate something" headlines elsewhere on this page, precisely because it is a narrow, mechanically checkable question ("does the stored string match the choice made?") rather than a broad legal one. When you report a violation rate, say which of those two kinds of question you asked. |
| To insert citations, follow these steps: | |
| |
| - Verify the BibTeX entry exists in https://measuretheweb.org/literature/bibliography. If not, add it there. | <WRAP important> |
| - Use {[CitationKey]} where needed in the text; it will render as a numbered reference. | **Version matters, and the version moved recently.** TCF **v2.3** was released on **19 June 2025** and the transition period ended on **28 February 2026**; its substantive change is that the previously optional ''disclosedVendors'' segment of the TC string became **mandatory**. TC strings created before that date without the segment remain valid; ones created after it without the segment are invalid.((IAB Europe, "All You Need to Know About the Transition to TCF v2.3", page dated 15 December 2025, fetched 2026-08-19. It states the 19 June 2025 release, the 28 February 2026 deadline, and the validity rule quoted here.)) A paper crawling across that boundary is crawling two different formats — say which side of it your data is on, and do not describe "the TCF" without a version number. |
| - Keep this section unchanged to display the bibliography. | |
| |
| If any step fails, a purple warning will appear on the preview page. | **And the TCF is contested law, not settled infrastructure.** On **2 February 2022** the Belgian Data Protection Authority found the TCF in breach of the GDPR, fined IAB Europe €250,000 and ordered an action plan.((Belgian DPA press release, "The BE DPA to restore order to the online advertising industry: IAB Europe held responsible for a mechanism that infringes the GDPR", 2 February 2022, fetched 2026-08-19.)) IAB Europe appealed to the Market Court (part of the Brussels Court of Appeal), which referred two questions to the Court of Justice of the EU; in **C-604/22, judgment of 7 March 2024**, the Court held that a TC string can be personal data and that a sectoral organisation such as IAB Europe can be a **joint controller** for users' preferences within the TCF.((//IAB Europe v Gegevensbeschermingsautoriteit//, Case C-604/22, Judgment of the Court (Fourth Chamber) of 7 March 2024, on a reference from the Hof van beroep te Brussel. Verified on EUR-Lex, CELEX 62022CJ0604, 2026-08-19.)) The Market Court then ruled on **14 May 2025**: it annulled the 2022 decision on procedural grounds but endorsed the DPA's reasoning and **confirmed the €250,000 fine**, while rejecting the finding that IAB Europe is a joint controller for processing that happens entirely inside OpenRTB.((Belgian DPA press release, "The Market Court rules in the IAB Europe case", 14 May 2025, fetched 2026-08-19.)) Date anything you write about TCF compliance, and say which framework version and which stage of that litigation it refers to. See [[Practices:Legal enforcement|Legal enforcement]] for what to do with a violation you find. |
| */ | </WRAP> |
| | |
| | ===== Consent Signals: One Setting Instead of a Thousand Clicks ===== |
| | |
| | The alternative to clicking every banner is for the browser to say //no// once, in a way every site is obliged to read. Three attempts exist; only one is live. |
| | |
| | ==== Do Not Track: dead, and useful only as history ==== |
| | |
| | DNT sent a ''DNT: 1'' request header and exposed ''navigator.doNotTrack''. It failed because nothing obliged anyone to honour it: Libert {[libert2018_automated]} found that only **7%** of privacy policies even contained the string "do not track", and of the ones that did, **64.80% explicitly said they did not honour it** against **19.46%** that committed to honouring it. Among 25 third-party data collectors, nine mentioned DNT and **none offered unqualified support**. |
| | |
| | It is formally dead. The W3C Tracking Protection Working Group concluded its work and republished both specifications as **W3C Working Group Notes on 17 January 2019**, saying in the status section that "there has not been sufficient deployment of these extensions (as defined) to justify further advancement".((''w3.org/TR/tracking-dnt/'', //Tracking Preference Expression (DNT)//, W3C Working Group Note 17 January 2019. Fetched 2026-08-19.)) Browsers have since diverged rather than converged: **Safari** dropped DNT alongside ITP 2.1 in 2019, and **Firefox removed the checkbox in version 135 (4 February 2025)**, whose release notes point users at "Tell websites not to sell or share my data" — which is GPC.((Mozilla, //Firefox 135.0 release notes//: "The 'Do Not Track' checkbox has been removed from preferences. If you wish to ask websites to respect your privacy, you can use the 'Tell websites not to sell or share my data' setting instead. This option is built on top of the Global Privacy Control (GPC)." Fetched 2026-08-19.)) **Chrome still exposes a DNT toggle.** If your crawl runs a default Chrome profile you may be sending ''DNT'' without meaning to; if it runs a current Firefox, the setting you find in the UI is GPC, not DNT. Check what your browser actually sends rather than what you assume. |
| | |
| | Do not build a measurement on DNT today, and do not report "sites ignore DNT" as a finding — it has been the known state for a decade. It is worth knowing for one reason: it is the control condition that explains why GPC was designed the way it was, with a legal hook first and a technical mechanism second. |
| | |
| | ==== Global Privacy Control ==== |
| | |
| | GPC is the live signal, and it has moved up: it is a **W3C Working Draft on the Recommendation track, dated 11 June 2026**, published by the W3C Privacy Working Group.((''w3.org/TR/GPC/''. The status section reads: "This document was published by the Privacy Working Group as a Working Draft using the Recommendation track." Fetched with a real browser on 2026-08-19; ''w3.org'' serves an interstitial to plain HTTP clients.)) If a paper or a page describes GPC as a Community Group report, that is stale. |
| | |
| | It is deliberately narrow — it expresses a **do-not-sell-or-share** request, not a general "do not track" — and the specification defines three carriers, which you should check **separately**, because sites implement them inconsistently: |
| | |
| | * the ''Sec-GPC: 1'' request header (the spec fixes the value at exactly ''1''; a server must ignore any other value), |
| | * the ''navigator.globalPrivacyControl'' DOM property, available on both ''Navigator'' and ''WorkerNavigator'', |
| | * an optional ''/.well-known/gpc.json'' resource, by which an **origin** declares awareness of and support for GPC. Read the spec's own caveat before you use it as a compliance measure: it "is not intended to convey whether the origin abides by GPC requests from the user agent accessing the resource". |
| | |
| | **There is a standard way to turn it on in an automated browser, and you should use it.** §8 of the specification defines two WebDriver extension commands — ''POST /session/{session id}/privacy'' to set the preference and ''GET /session/{session id}/privacy'' to read it back. Driving the preference through WebDriver rather than through a profile pref or an extension means your setup is reproducible and the "did the browser actually send it?" question has an answer. |
| | |
| | <WRAP important> |
| | **Do not enable GPC by injecting the header. It sets one carrier and not the other, and a site that reads the other sees nothing.** Tested on 2026-08-19 with Playwright/Chromium and ''extraHTTPHeaders: { 'Sec-GPC': '1' }'': |
| | |
| | <code> |
| | Sec-GPC on the navigation request = "1" |
| | navigator.globalPrivacyControl = undefined |
| | </code> |
| | |
| | The header goes out; the DOM property does not exist. Any site whose detection reads ''navigator.globalPrivacyControl'' will treat that browser as not signalling, and your compliance rate will be wrong in a direction you cannot see. How the two carriers are split across real sites is exactly the thing this page told you to measure separately, so do not quietly guarantee that only one of them works. **Set the browser's own preference instead.** In Firefox the relevant preferences in ''StaticPrefList.yaml'' are ''privacy.globalprivacycontrol.enabled'', ''privacy.globalprivacycontrol.functionality.enabled'' and ''privacy.globalprivacycontrol.pbmode.enabled'' (the private-browsing one, overridable by the first).((Verified on 2026-08-19 against ''searchfox.org/mozilla-central'', ''modules/libpref/init/StaticPrefList.yaml''.)) Chromium ships no built-in GPC, so a Chrome-based crawl needs an extension or the WebDriver command above — and either way, **assert both carriers on your own endpoint before the crawl**, not just the header. |
| | </WRAP> |
| | |
| | On the legal side the specification is explicit that its force is jurisdictional: "At least four states have specifically identified GPC as a valid means to exercise legal opt-out rights." That is the spec's own conservative wording; the practical number of US states whose comprehensive privacy laws require honouring a universal opt-out mechanism is now roughly a dozen, and it grows every legislative session — **check the current list for your crawl's date rather than reusing one from a paper**. Colorado is the sharpest case — the state Attorney General's office states that GPC has been mandatory there since **1 July 2024** and is currently the **only** universal opt-out mechanism the Department recognises.((Colorado Attorney General, //Universal Opt-Out Mechanisms//, ''coag.gov/opt-out/'': "Beginning July 1, 2024 … Currently, the only UOOM considered valid by The Department is GPC". Fetched 2026-08-19.)) If your paper claims a site is legally obliged to honour GPC, name the state and cite the regulator, not a vendor blog. |
| | |
| | The published compliance numbers are low and are the ones to cite. Zimmeck et al. {[zimmeck2023_gpc]} found that of the sites implementing a US Privacy String, only **54 of 464 (12%)** actually opted the user out when GPC was sent. Hausladen et al. {[hausladen2025_websites]} re-ran it at scale over time and found **44% (1,411/3,226)** in December 2023, **43% (1,473/3,402)** in February 2024 and **45% (1,620/3,566)** in April 2024 — a different, narrower denominator (sites with //all// implemented privacy strings set to opt-out), which is exactly the kind of denominator difference that makes two GPC papers look contradictory when they are not. Charatan and Birrell {[charatan2024_steps]} found **2,429 of the top 25,000 sites (9.9%)** offering a manual opt-out-of-sale link in November 2022, **814** honouring GPC by November 2023, and **340** for which GPC was the //only// opt-out mechanism offered. |
| | |
| | GPC also interacts with banners rather than replacing them. Rasaii et al. {[rasaii2025_crumbs]} measured both together: enabling GPC reduced "intractable" cookies by about **30%**, and rejecting the banner on top of that removed a further **32%**. |
| | |
| | ==== The IAB CCPA framework, and its opt-out string ==== |
| | |
| | Alongside GPC, US opt-outs are carried in a string. The original **US Privacy String** was built for the CCPA specifically, has not been updated since 2020, and **IAB Tech Lab deprecated it on 31 January 2024** in favour of the **Global Privacy Platform (GPP)**, which can represent the Virginia, Colorado, Connecticut and Utah signals that USP cannot.((IAB Tech Lab, "US Privacy Signal Deprecation Deadline Extended To January 31, 2024", fetched 2026-08-19. It states the original 30 September 2023 deadline, the extension to 31 January 2024, and that "the GPP is the only privacy signaling mechanism available to signal consumer privacy choices for all US states with privacy laws".)) **A 2026 crawl that only reads ''usprivacy'' is reading a deprecated field**; read the GPP string too, and report which you read. Every published USP measurement below therefore describes a mechanism that has since been retired — which does not make the findings wrong, but does date them. |
| | |
| | Aziz and Wilson {[aziz2024_johnny]} tested whether the plumbing works: with GPC enabled, **380 of 825 publishers (46.1%)** implementing the USP API set the string to opt-out — and the number of tracking pixels showed **no statistically significant reduction** either way. The signal propagating is not the same thing as the tracking stopping, and a paper that measures only the string has measured the plumbing. |
| | |
| | ==== Opt-out links, which are a text-classification problem ==== |
| | |
| | The US route to refusal is often not a banner at all but a link — //Do Not Sell or Share My Personal Information// — or a paragraph buried in a privacy policy. That makes finding it a **text** problem rather than a DOM problem, and it has its own small literature. Kumar et al. {[kumar2020_finding]} built the reference extractor for opt-out statements in policy text, reaching **precision 0.93, recall 0.90, F1 0.91**, and found that opt-outs are heavily concentrated at the top of the ranking: an average of **3.43** per site for US Alexa ranks 1–200 against **1.22** above rank 1,000. If your sample is a long tail, expect to find almost nothing, and do not read that as compliance. |
| | |
| | Then there is the question of whether the link does anything. Bui et al. {[bui2022_outs]} followed opt-out cookies and the policies that describe them and found **11 trackers** whose behaviour after opt-out contradicted their own stated policy. Liu et al. {[liu2024_opted]} went further downstream and watched the ad auction: personas that had opted out under both GDPR and CCPA still attracted **higher** bids than the control. **An opt-out that is offered, findable and technically registered can still be an opt-out that does nothing**, and each of those is a separate measurement. |
| | |
| | ===== Tools That Interact With Banners For You ===== |
| | |
| | <WRAP important> |
| | **Measure the tool on your own sample before you trust it.** Demir et al. {[demir2024_bannertools]} evaluated five banner-interaction extensions on a hand-checked sample and found each one interacts with **65% of the banners it was shown**, on average — the paper writes it as "on average, with 12 (65%) (SD: 21%; max: 95% min: 48%) of all banners", where the 12 is the mean count on their sample rather than a rate. A third of banners are missed, and which third is not random — it correlates with language, with CMP, and with how the banner is injected. If a tool is your instrument, its success rate on //your// crawl is a number your paper owes the reader. |
| | </WRAP> |
| | |
| | All the checks in this table were run on **2026-08-19** against the project's own repository or store listing. Re-run them: a tool table is the fastest-rotting part of any page.((Commit and release dates from the GitHub REST API (''/repos/{owner}/{repo}'', ''/commits?sha={default_branch}&per_page=1'', ''/releases/latest''); rule-file counts from ''/repos/{owner}/{repo}/contents/{path}''. Store listings were read with a real browser because both stores serve a consent interstitial to plain HTTP clients — which is its own small joke.)) |
| | |
| | ^ Tool ^ What it actually does ^ State on 2026-08-19 ^ Use it when ^ |
| | | **Consent-O-Matic** ([[https://github.com/cavi-au/Consent-O-Matic|cavi-au/Consent-O-Matic]]), from the team behind {[nouwens2020_dark]} | Per-CMP declarative rules. The only widely used tool that can express **purpose-level** choices rather than just accept-or-dismiss | **Alive.** ''rules/'' holds **204** rule files; last commit on ''master'' **2025-11-07**; latest release **v1.1.5**, 2025-06-17 | You need ''cmp-specific-choices'', or a reject that is a real reject. Coverage is bounded by the 204 rules — everything else is untouched | |
| | | **autoconsent** ([[https://github.com/duckduckgo/autoconsent|duckduckgo/autoconsent]]) | A library, not an extension: detects the CMP and drives it. Ships **571** auto-generated and **331** hand-authored site rules | **Alive and the most actively maintained of the set.** Release **v16.23.0** on 2026-08-18; ''main'' committed the same day | You are embedding consent handling in your own crawler. It is a library with a stable API, which is what you want. Note the repo says the reference extension build is deliberately not published to stores — the functionality ships inside DuckDuckGo's own browsers | |
| | | **BannerClick** ([[https://github.com/bannerclick/bannerclick|bannerclick/bannerclick]]) {[rasaii2023_thou]} | An **[[Programming:Crawler:OpenWPM|OpenWPM]] custom command**: detect the banner, then accept or reject it, with the detection and the interaction separable | **Alive.** Default branch ''bannerclick_v0.26.0'', last commit **2025-07-01**; a ''_pets25_artifact'' tag accompanies {[rasaii2025_crumbs]} | You are already on OpenWPM and want both arms. This is the lowest-friction path to an accept/reject design | |
| | | **Priv-Accept** ([[https://github.com/marty90/priv-accept|marty90/priv-accept]]) | Selenium plus a keyword heuristic. **Accept only** — there is no reject arm | **Stale.** Last commit on ''main'' **2022-04-13**. Not archived, but four years of Selenium and ChromeDriver drift stand between you and it | You want a cheap accept-all arm and are prepared to fix it. Two papers in this corpus still use it | |
| | | **CookieBlock** ([[https://github.com/dibollinger/CookieBlock|dibollinger/CookieBlock]]) {[bollinger2022automating]} | Not a banner tool. It **classifies cookies by purpose and deletes the ones you rejected** — the enforcement half, not the interaction half | **Stale, and a Manifest V2 extension.** Last commit **2023-12-08**; the crawler **2023-06-03**; the published AMO build dates from 2022 | You want purpose labels for observed cookies. See [[Privacy:Cookies|Classifying Cookies]]. Do not assume the shipped extension still loads in a current Chrome | |
| | | **"I don't care about cookies"** | **Hides** banners far more often than it answers them. Acquired by Avast | **Original stale** (published build last updated 2023-11). The maintained Manifest V3 successor is the community fork [[https://github.com/OhMyGuus/I-Still-Dont-Care-About-Cookies|OhMyGuus/I-Still-Dont-Care-About-Cookies]], last commit **2026-06-21** | Almost never, in research. Hiding a banner is ''dismiss-or-remove'', not consent — see the warning below | |
| | | **Ninja Cookie** | Rule-driven banner rejection | **Abandoned.** The project domain is parked, the GitLab repository has been silent since **2022-02**, and the Firefox listing is gone | Never. It appears in {[demir2024_bannertools]}, which is why it is here — do not carry it forward from that paper into a 2026 crawl | |
| | | **Super Agent** | Commercial, closed-source consent automation | Live commercial product | Not as a research instrument: you cannot pin its version or read its rules | |
| | | **CookieEnforcer** {[khandelwal2023automated]} and **ConsentChk** {[tang2025_navigating]} | The two strongest published pipelines | **No public artefact found** for either as of 2026-08-19 | You will have to reimplement from the paper. Say so in your methods section | |
| | | **EasyList Cookie List** | Not a tool — the CSS-selector source most notice detectors start from. Fetched from ''secure.fanboy.co.nz/fanboy-cookiemonster.txt'' | Header on 2026-08-19: ''! Last modified: 2026-08-18 13:00 UTC'', ''! Expires: 9 days''. **No ''! Version:'' line** — cite the modification date, not a version | You are building a detector. Its own README says it does **not** cover age gates, region checks or paywall notices, so it is a partial ground truth by design((Rule counts depend entirely on what you call a rule. Counting non-comment lines in the combined file and treating any line containing ''##'', ''#@#'' or ''#?#'' as cosmetic gives **25,111** rules, **23,012** cosmetic and **2,099** network on 2026-08-19. Narrower patterns give smaller numbers — this is why [[Privacy:Requests#Cookie Notices and Their Interactive Elements|the request page]] reports different figures for the same list. Publish your counting rule alongside the count.)) | |
| | |
| | <WRAP important> |
| | **Hiding a banner is not answering it.** An extension that removes the overlay from the DOM leaves the site in its **pre-consent** state: no consent string is written, no CMP callback fires, and any cookie the site sets without consent it will still set. That is a legitimate crawl configuration — it is the ''dismiss-or-remove'' row of the table above, and it is useful when the banner is merely in your way — but it is **not** an accept arm and it is **not** a reject arm. Several published crawls use a banner-hiding extension and describe the result as a user's experience. Do not be one of them. |
| | </WRAP> |
| | |
| | **Which ones the field actually uses.** Counting papers in this corpus that name a tool as ''used'' or ''produced'' — 17 distinct names over 39 papers, folded with ''consent_fold.mjs'': Consent-O-Matic **9**, BannerClick **4**, CookieBlock **4**, a custom GPC extension or crawler **4**, autoconsent **3**, EasyList Cookie List **2**, Priv-Accept **2**, a TC-string decoder **2**, and one paper each for ConsentChk, CookieCheck, CookieEnforcer, GDPR-Consent, "I don't care about cookies", Ninja Cookie, Opt-Out Easy, OptOutCheck and Super Agent. The long tail is the finding: **almost every consent paper builds its own instrument**, which is the main reason consent results are hard to compare across papers. |
| | |
| | ===== Jurisdiction: Your Vantage Point Chooses Your Banner ===== |
| | |
| | The banner you see is chosen by the site from your IP address. Crawl a European site from a US datacentre and you will often get no banner at all, or a different one — and the TCF's own ''gdprApplies'' flag will be ''false''. This makes [[Design:Crawling location|the vantage point]] a co-determinant of every consent result, not an independent choice. |
| | |
| | The corpus says the field mostly does not handle this. Of the **349** crawling papers that state a consent action, **68 (19.5%)** state an EU/EEA vantage point, **71 (20.3%)** state a non-EEA vantage only, and **206 (59.0%)** carry a vantage tuple whose location is //not stated//. The remaining four are 3 papers with no vantage tuple at all and 1 that names a place the geo fold cannot resolve. The picture is much better among the papers that actually clicked: of the **29** whose interaction is verified, **21 (72.4%)** did so from an EU/EEA vantage, 3 from a stated non-EEA vantage only, and 5 never said where they were. |
| | |
| | Ogut et al. {[ogut2024_dissecting]} is the paper to read on the language side of the same problem: button text is the classifier's input, and //Accept all// is //Aceptar todo// and //Alle akzeptieren// elsewhere. They found a consent notice on **37% (1511)** of successfully loaded sites worldwide. Tang et al. {[tang2025_navigating]} did the regional comparison for violations, finding at least one consent violation on **96.18% (EU)** to **97.72% (US)** of sites, with only **3.82%** enforcing preferences correctly. |
| | |
| | ===== Use in Publications ===== |
| | |
| | Everything in this section comes from a structured extraction over **5,859 full-text papers** from CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, **2010–2026**, one record per paper with a verbatim evidence quote per claim. The 2025 and 2026 venue-years are **provisional** — CCS and IMC 2026 have not been held, and IEEE S&P and WWW 2026 are incompletely selected — so any per-year row reaching them is under-represented by construction. Methodology and limits are at the end of this section. |
| | |
| | ==== Almost nobody says what they did about consent ==== |
| | |
| | Population: the **1,120** papers that ran an automated web crawl. |
| | |
| | ^ ''crawlConfig.consentAction'' ^ Papers ^ Share of 1,120 ^ |
| | | no interaction | 313 | 27.9% | |
| | | accept all | 15 | 1.3% | |
| | | accept and reject | 14 | 1.3% | |
| | | dismiss or remove | 3 | 0.3% | |
| | | reject all | 2 | 0.2% | |
| | | CMP-specific choices | 2 | 0.2% | |
| | | //not stated// | 495 | 44.2% | |
| | | //not applicable// | 236 | 21.1% | |
| | | //no crawl configuration extracted// | 40 | 3.6% | |
| | |
| | **And the six stated rows do not all survive contact with the papers.** Every one of the 36 papers in the five interacting rows was read against its own full text; **7 (19.4%) are extraction false positives** — the value fired on push-notification permission prompts, on "banner ads", and three times on IRB participant consent. The audited count of papers that verifiably interacted with a consent notice is **29**, or **2.6%** of crawling papers. The script is ''consent_action_audit.mjs'' and every verdict is on [[provenance:privacy:consent|the provenance page]]. ''no-interaction'' was not audited — 313 papers is beyond a hand pass — so treat it as an upper bound in the same way. |
| | |
| | The three italic rows are **sentinels, not answers**. ''not-applicable'' is a legitimate value — a crawl of an API, a mobile app store or a set of non-European sites may have no banner to handle — but ''not-stated'' at 44.2% is the finding: nearly half of all crawling papers leave the reader unable to tell which web they measured. |
| | |
| | ==== The reporting gap is not closing, but interaction is spreading ==== |
| | |
| | ^ Bucket ^ Crawling papers ^ State an action ^ Share ^ Interaction claimed ^ Verified ^ Share verified ^ |
| | | 2010–2013 | 102 | 25 | 24.5% | 0 | 0 | 0.0% | |
| | | 2014–2017 | 167 | 50 | 29.9% | 0 | 0 | 0.0% | |
| | | 2018–2021 | 308 | 99 | 32.1% | 6 | 3 | 1.0% | |
| | | 2022–2024 | 345 | 116 | 33.6% | 17 | 14 | 4.1% | |
| | | 2025–2026* | 198 | 59 | 29.8% | 13 | 12 | 6.1% | |
| | |
| | //* provisional venue-years.// Two separate trends. **Whether a paper says anything** moved from **24.5%** in 2010–2013 to a peak of **33.6%** in 2022–2024 and back to **29.8%** in the provisional 2025–2026 bucket — about five points net over sixteen years, and no better in the latest bucket than it was before the GDPR. This is not a solved reporting problem, and it is not obviously improving. **Whether a paper interacts** starts at exactly zero before the GDPR and rises steadily afterwards: the treatment arrived with the law, as you would expect, and is now in roughly one crawling paper in sixteen. Note that the false positives cluster in the early buckets — the extraction is most likely to mistake something else for a consent action in a paper that has nothing to do with consent, which is exactly the pre-2022 population. |
| | |
| | ==== The consent literature itself ==== |
| | |
| | No field in the extraction means "is about consent", so the population below (**S2**) is a hand-audited proxy: a vocabulary sweep over tool names, classification resources, detection phenomena and assessed-law strings, plus the ''classification.target = consent-notice'' enum, minus 28 hand-checked false positives. The sweep, the exclusions and their reasons are listed in full on the [[provenance:privacy:consent|provenance page]]. **S2 = 72 papers.** |
| | |
| | ^ Venue ^ Papers in S2 ^ Papers in corpus ^ Share of that venue ^ |
| | | PETS | 30 | 510 | **5.9%** | |
| | | USENIX Security | 14 | 1,410 | 1.0% | |
| | | TheWebConf | 8 | 843 | 0.9% | |
| | | IMC | 7 | 638 | 1.1% | |
| | | CCS | 7 | 990 | 0.7% | |
| | | NDSS | 3 | 701 | 0.4% | |
| | | IEEE S&P | 3 | 767 | 0.4% | |
| | |
| | **PETS is where this work lives**, by more than a factor of five over the next venue as a share of its own programme. If you are surveying consent measurement and you skip PETS, you have skipped the field. (Note also that this corpus does not contain CHI, SOUPS, EuroS&P, ACSAC or WPES, all of which publish consent work — see the limitations below.) |
| | |
| | Per year: 1 (2018), 4 (2019), 3 (2020), 5 (2021), 9 (2022), 9 (2023), **20 (2024)**, 15 (2025*), 6 (2026*). The pre-2018 count is essentially zero, which is what you would expect of a literature created by a statute. |
| | |
| | In **60 of the 72 (83.3%)** the consent mechanism is the object of study; in the other **12 (16.7%)** it appears only as an instrument — a paper measuring something else that had to get past the banner. **57** measure the web and **15** measure mobile apps — the field is multi-valued and a paper can do both, and 7 also measure some other online service — so these are not two halves of a partition. Consent-dialog measurement in apps is a real and growing subfield {[nguyen2022_freely,koch2023_enough,morel2026_tcf,zimmeck2026_exercising]}, and its methods do not transfer from the web unchanged. |
| | |
| | ==== How consent notices get classified ==== |
| | |
| | Of the S2 papers carrying a classification tuple whose target is a consent notice (**24 papers**): |
| | |
| | ^ Method ^ Papers ^ Share ^ |
| | | heuristic rules | 11 | 45.8% | |
| | | manual labelling | 9 | 37.5% | |
| | | supervised ML | 3 | 12.5% | |
| | | regex or signature | 2 | 8.3% | |
| | | blocklist (EasyList Cookie List) | 2 | 8.3% | |
| | | curated database (IAB CMP list) | 1 | 4.2% | |
| | | third-party service | 1 | 4.2% | |
| | | LLM | 1 | 4.2% | |
| | |
| | Shares do not sum to 100% because a paper usually stacks several. The shape is the point: **this is still a hand-built-heuristic field.** Supervised ML enters only with CookieEnforcer {[khandelwal2023automated]} and Bouhoula et al. {[bouhoula2024automated]}, both using BERT-class models on button text, and **exactly one paper in the corpus classifies a consent artefact with an LLM** — WhisperTest (CCS 2025), which uses Qwen2.5-7B for iOS UI automation rather than for consent semantics. |
| | |
| | Validation is better here than on most pages: **19 of 24 (79.2%)** report manual validation and **19 of 24** name a ground-truth source; 4 report no validation at all. Since manual validation of a notice detector is cheap — a few hundred screenshots — there is no excuse for being in the last group. |
| | |
| | ==== Which law, and it is not only the GDPR ==== |
| | |
| | Of the **53** S2 papers that assess a law (folded with ''law_fold.mjs''; a paper can assess several): |
| | |
| | ^ Law family ^ Papers ^ Share of 53 ^ |
| | | EU data protection (GDPR and predecessors) | 42 | 79.2% | |
| | | ePrivacy / cookie law | 20 | 37.7% | |
| | | US state privacy law (CCPA, CPRA, …) | 20 | 37.7% | |
| | | US children / education / health / finance (COPPA, …) | 7 | 13.2% | |
| | | EU platform / digital-market regulation (DSA, …) | 2 | 3.8% | |
| | |
| | Three further families carry one paper each and are omitted above: non-EU/US data protection law, a national-rules catch-all, and one string the fold could not place. The unfolded residue is on the provenance page. |
| | |
| | **The GDPR is not the cookie law.** Consent for storing or reading information on a device comes from Article 5(3) of the **ePrivacy Directive**; the GDPR supplies the definition of valid consent that Article 5(3) points at. Only 20 of 53 papers name ePrivacy at all, and a compliance claim that cites the GDPR alone for a cookie finding is citing the wrong instrument. That also changes **who enforces it** — see [[Practices:Legal enforcement|Legal enforcement]]. |
| | |
| | If you are turning observations into violation claims, three EDPB documents are the cheapest way to align your rules with what regulators actually said, and all three are worth reading before you write your own taxonomy:((All three fetched from ''edpb.europa.eu'' on 2026-08-19; the dates are the ones the EDPB's own document pages carry. Opinion 08/2024 has a published corrigendum — check you have the corrected version.)) |
| | |
| | * **Report of the work undertaken by the Cookie Banner Taskforce** (18 January 2023) — the closest thing to an agreed list of what a non-compliant banner looks like across the EU. |
| | * **Guidelines 2/2023 on the Technical Scope of Art. 5(3) of the ePrivacy Directive**, final version **16 October 2024** — the one that decides **what your measurement is even allowed to be about**. It reads Article 5(3) as covering far more than cookies: tracking pixels, URL and link decoration, local processing followed by transmission, and IP-only tracking. If your study measures "cookie consent" and your denominator is cookies, this document says you have scoped the legal question too narrowly. |
| | * **Opinion 08/2024 on Valid Consent in the Context of Consent or Pay Models Implemented by Large Online Platforms** (17 April 2024) — the reference for cookiewall studies of the kind Rasaii et al. {[rasaii2023_thou]} pioneered. |
| | |
| | |
| | ==== What the papers found: figures with their own denominators ==== |
| | |
| | These are each paper's own numbers, checked against the paper's own full text (see limitations). Note how different the denominators are — this is the main reason "how many sites have banners" has no single answer. |
| | |
| | ^ Finding ^ Figure ^ Denominator ^ Source ^ |
| | | Sites showing a cookie notice, EU-wide, May 2018 | 62.1%, up from 46.1% in January | 6,357 EU sites | {[degeling2019_value]} | |
| | | Sites showing a cookie notice, worldwide, 2024 | 37% (1,511) | successfully loaded sites | {[ogut2024_dissecting]} | |
| | | Sites where a detector found a notice, 2023 | 52.7% | 85,473 sites | {[khandelwal2023automated]} | |
| | | Sites with a TCF banner, EU, 2019 | 6.2% (1,426) | 22,949 reachable EU sites | {[matte2020_cookie]} | |
| | | Android apps implementing TCF, 2026 | 12.85% (576) | 4,482 downloaded apps | {[morel2026_tcf]} | |
| | | Cookiewalls ("accept or pay") | 0.6% (280) | ~45k sites queried | {[rasaii2023_thou]} | |
| | | Notices offering a **one-click opt-out** | 21.5% | sites with a notice | {[khandelwal2023automated]} | |
| | | Notices with **more than one view** | 35.4% | sites with a notice | {[khandelwal2023automated]} | |
| | | Sites with **no way to refuse** | 6.8% (38) | 560 hand-checked TCF sites | {[matte2020_cookie]} | |
| | | Sites with **pre-selected** purposes | 46.5% (236) | 508 TCF sites | {[matte2020_cookie]} | |
| | | Sites storing consent **before any choice** | 9.9% (141) | 1,426 TCF sites | {[matte2020_cookie]} | |
| | | Sites storing consent **after explicit refusal** | 5.3% (27) | 508 TCF sites | {[matte2020_cookie]} | |
| | | Sites with **at least one** suspected violation | 54.29% (304) | 560 hand-checked TCF sites | {[matte2020_cookie]} | |
| | | Sites setting non-necessary cookies **without any interaction** | 69.7% | sites examined | {[bollinger2022automating]} | |
| | | Sites with at least one cookie-notice violation, 2024 | 72.2% | successfully crawled sites | {[bouhoula2024automated]} | |
| | | Sites with at least one consent violation, 2025 | 96.18% (EU) – 97.72% (US) | sites per region | {[tang2025_navigating]} | |
| | | Sites enforcing preferences correctly | 3.82% | same | {[tang2025_navigating]} | |
| | | Banners using nudging in their design | 57.4% | 1,000 hand-inspected notices | {[utz2019_informed]} | |
| | | Sites offering revocation via a substantially different interface | 19.87% (32) | 158 reachable sites | {[kancherla2025_johnny]} | |
| | | Android apps sending personal data before consent | 16.67% (2,181) | 13,082 apps with a consent notice | {[nguyen2022_freely]} | |
| | | Android apps with **no** consent notice at all | 93.26% (30,160) | 32,341 apps sending data to ad domains | {[nguyen2022_freely]} | |
| | |
| | <WRAP important> |
| | **Read that table as a warning about your own abstract.** "72.2% of websites violate cookie law" and "96.18% of EU websites violate cookie law" are two years and two rule-sets apart, over different site samples, with different definitions of //violation//. Neither is wrong. A number of this shape means nothing without its rule-set, its sample and its date, and yours will be quoted without them unless you make that impossible. |
| | </WRAP> |
| | |
| | ==== Methodology and limitations of these figures ==== |
| | |
| | * **How they were produced.** One structured record per paper, extracted from full text, each tuple carrying a verbatim evidence quote and its section. The script that produces every number in this section, with its denominators, is ''report_consent.mjs''; the folding rules are ''consent_fold.mjs'' (tools and CMPs), ''law_fold.mjs'' (laws) and ''geo.mjs'' (vantage locations). Every query, the scripts' unedited output and the full residue of each fold are on [[provenance:privacy:consent|the provenance page for this one]]; corpus-level caveats are on [[literature:corpus]]. |
| | * **The S2 population is a judgement, not an enum.** It is a vocabulary sweep plus one enum, minus a hand-audited exclusion list of 28 papers. The ''classification.target = consent-notice'' enum alone fires on 39 papers, of which the audit keeps 24 — it misfires on cryptomining opt-ins, misinformation labels and login forms. Both lists are on the provenance page so you can disagree with them. |
| | * **A paper counts once**, never once per tuple, and shares do not sum to 100% because the fields are multi-valued. |
| | * **Sentinels are counted as what they are.** ''not-stated'' and ''not-applicable'' are never folded into a stated value; where they are the largest row, that is the finding. |
| | * **Free-text names were folded before counting, and the residue is printed.** The consent-tool fold leaves 5 distinct unmapped names over 5 papers, all of them IAB artefacts that are not banner-interaction tools (''IAB ads.txt crawler'', ''IAB anti-ad-block script'', the IAB content taxonomy). The law fold leaves 2 (''Digital Economy Act 2017'', ''Act against Unfair Competition (UWG)''). The vantage fold leaves 5 strings that name no place (''different continents'', ''various geographic regions''). |
| | * **Every per-paper figure on this page was checked against the paper's own text**, not against the extraction's summary of it. The check covers 61 literals — a superset of what is published, since ten were checked and then cut — and found 60 in both the column-repaired and the plain rendering, 1 in the column-repaired rendering only, and **0 not found**. That pass caught two errors in an earlier draft of this page — a figure attributed to Bouhoula et al. that the paper writes without a thousands separator, and a Matte et al. percentage this page had rounded to the wrong decimal. |
| | * **''consentAction'' was audited paper by paper, and it needed to be.** The schema's stability comparison puts it in the reliable band — an independent extraction run over the same text agrees with it on 93% of papers — but that measures whether two runs agree, not whether either is right, and it was measured on the earlier 4,322-paper corpus. Reading all 36 interacting papers found **7 false positives (19.4%)**, concentrated in ''accept-all'' and ''dismiss-or-remove''. **Note also that the field's own evidence quote cannot catch this**: ''crawlConfig'' carries one quote for the whole configuration object, so the quote behind a ''consentAction'' value usually evidences statefulness or crawl depth instead. Spot-checking quotes, which is the standard check on this site, is structurally blind here. |
| | * **Venue coverage.** Seven venues only. **CHI, SOUPS, EuroS&P, ACSAC, RAID, AsiaCCS and WPES are absent**, and that bites harder on this page than on most: the usable-privacy half of the consent literature (Nouwens et al., Habib et al., Utz et al.'s follow-ups) is largely CHI and SOUPS work. Every count here is a lower bound. |
| | |
| | ===== What to Report ===== |
| | |
| | - **Your consent action, in the enum above**, in one sentence in the methods section, for every arm of the crawl. "We did not interact with consent notices" is a complete and respectable answer; silence is not. |
| | - **How you detected the notice, and the detector's recall** on a hand-labelled sample of your own. Precision is the easy number; recall is the one that conditions every downstream rate. State the denominator as "of sites where we detected a notice". |
| | - **Whether the choice was verified as stored**, and how — TC string, CMP cookie, or nothing. If nothing, say so. |
| | - **Whether you reloaded** after interacting, and whether the crawl was stateful across that reload {[rasaii2025_crumbs]}. |
| | - **Your vantage point and the ''gdprApplies'' value you observed.** Not "we crawled from Europe" — the country, and preferably the observed flag. |
| | - **If you used an off-the-shelf tool, its success rate on your sample**, not its authors' {[demir2024_bannertools]}. Also its version and, for a rule-driven tool, the rule-set commit. |
| | - **Which law you are assessing, by article.** ePrivacy Art. 5(3) for the storage/access question, GDPR Art. 4(11) and 7 for the validity-of-consent question; both, usually. |
| | - **The banner language and the site's locale**, if your button classifier is language-dependent {[ogut2024_dissecting]}. |
| | - **Whether you cleared shared-domain consent cookies between sites**, if the crawl is stateful. A consent string set on one site is readable on another {[matte2020_cookie]}. |
| | - **What you counted as the effect of the choice** — requests, tags loaded, or cookies set. On a Google consent-mode site these disagree, because the tag fires either way. |
| | - **Your violation definition, verbatim**, and preferably as machine-checkable rules. This is what makes your 72% and someone else's 96% comparable. |
| | |
| | ===== Open Questions ===== |
| | |
| | <WRAP todo> |
| | * **Nobody has replicated the banner-interaction-tool evaluation since 2024.** Demir et al.'s 65% is one measurement, on one sample, of tools that have since changed. It is load-bearing for a lot of this page and for a lot of published crawls, and re-running it is a well-scoped, publishable study. |
| | * **LLM-driven banner interaction is unmeasured as an //instrument//, and alarming as a //subject//.** A language model that reads any banner in any language is the obvious successor to CSS-selector rule sets, and exactly one paper in this corpus puts an LLM anywhere near a consent artefact. Nobody has published cost, latency, determinism or accuracy against a hand-labelled set — and determinism is the hard one for a measurement instrument. Meanwhile the reverse question is opening up: work outside this corpus reports that browser-automation agents accept consent banners even when instructed to refuse everything.((Reported in a CHI 2026 Extended Abstracts paper on browser automation by LLM agents and consent, which measures agents defaulting to accept under explicit deny-all instructions. Not in this corpus (CHI is absent) and not read in full by this page's author — treat as a pointer to look up, not as a verified figure.)) Whether an agentic browser can be trusted to express a consent choice is now both a methods question and a research topic. |
| | * **Consent measurement is about to be measuring a different law, and the primary documents exist.** The **Digital Omnibus** proposal, **COM(2025) 837 of 19 November 2025**, would insert a new **Article 88a into the GDPR** carrying "the consent requirement for the storing or accessing of personal data on the terminal equipment of natural persons" — that is, it moves the cookie rule out of the ePrivacy Directive and into the GDPR.((EUR-Lex, CELEX ''52025PC0837'', fetched 2026-08-19. The quoted phrase is from the Explanatory Memorandum's description of paragraph 12.)) The EDPB and EDPS jointly opined on it on **11 February 2026** (Joint Opinion 2/2026), supporting simplification but warning that splitting the cookie rules across instruments risks legal uncertainty.((''edpb.europa.eu'', "EDPB-EDPS Joint opinion 2/2026 on the Proposal for a Regulation as regards the simplification of the digital legislative framework (Digital Omnibus)", legislative opinion dated 11 February 2026. Fetched 2026-08-19. Its editorial note corrects the proposal reference to COM(2025) 837.)) **It is a proposal, not law**, and it is not settled. But a longitudinal consent study spanning 2025–2028 will straddle a change in the legal object it measures, and should say which regime each wave sits under. |
| | * **The UK has already moved, and by more than the EU has.** The Data (Use and Access) Act 2025 amended PECR to exempt several categories — including aggregate-statistics analytics — from prior consent, with effect from **5 February 2026**, and raised the maximum PECR fine from £500,000 to £17.5m or 4% of global turnover. The ICO's own storage-and-access guidance is being consulted on rather than settled.((The commencement date and the fine cap are corroborated across independent legal trackers; the ICO's guidance page confirms only that a revised version is out for consultation. **Not** confirmed here against the Act's own commencement regulations — check before citing. A UK-vantage crawl in 2026 is measuring a different rule from an EU-vantage one.)) |
| | * **The reload effect has been measured once** {[rasaii2025_crumbs]}. If 25% of post-rejection change only appears after a reload, an unknown share of the published consent literature under-reports the effect of rejection. Which share is an answerable question. |
| | * **Consent revocation is nearly unstudied.** One paper {[kancherla2025_johnny]}, 158 sites. Withdrawal is as legally required as consent and is far harder to automate. |
| | * **No shared benchmark exists.** There is no public, versioned set of annotated consent notices that a new detector can report against, which is why every paper reports precision and recall on its own hand-labelled sample and none of them are comparable. Building one would be a bigger contribution than most new detectors. |
| | * **The reporting gap itself.** 44.2% of crawling papers say nothing about consent, and the share that says //something// is no higher in 2025–2026 than it was in 2014–2017. A one-line methods sentence would fix it; the question is why sixteen years of the field have not produced one. |
| | * **How many of the 313 ''no-interaction'' papers really did not interact** is unknown. The 36 papers claiming an interaction were read one by one and 7 turned out not to have interacted; nobody has done the same in the other direction, and 313 is too many for a hand pass. If the error is symmetric, the true count is somewhere either side of 29 — which is a reason to treat every figure in this section as an order of magnitude and to read the papers you actually compare yourself against. |
| | </WRAP> |
| | |
| | ===== Related Pages ===== |
| | |
| | * [[Privacy:Requests#Cookie Notices and Their Interactive Elements|Classifying Web Requests]] — detecting the notice and labelling its buttons, with the comparison table of detectors. |
| | * [[Privacy:Cookies|Classifying Cookies]] — what the cookies you observe before and after the click actually are, and the CookieBlock/Cookiepedia label sources. |
| | * [[Privacy:TCF Consent Strings|Decoding TCF Consent Strings]] — the TC string and Google's Additional Consent string as artefacts: bit layout, where they hide, decoding them reproducibly, and what they do and do not prove. |
| | * [[Programming:Interaction|Interaction with websites]] and [[Programming:Stateful Stateless|Stateful and stateless crawling]] — the crawler-side mechanics this page assumes. |
| | * [[Programming:Crawler|Comparison of crawling libraries]] — which crawlers ship a consent-interaction step. |
| | * [[Design:Crawling location|Crawling location]] — why the vantage point changes which banner you get. |
| | * [[Practices:Legal enforcement|Legal enforcement]] — what to do with a violation once you have found one, and which authority is competent. |
| | * [[Practices:Ethics|Ethics]] — clicking //Accept// at scale on behalf of nobody is a decision with an ethical dimension; it is discussed there. |
| | * [[provenance:privacy:consent|Provenance of this page's figures]] — every query, the report scripts and their unedited output, the folds and their residue. |
| | |
| | ====== References ====== |
| |
| <bibtex bibliography></bibtex> | <bibtex bibliography></bibtex> |
| /* This enables discussion under this article. */ | /* This enables discussion under this article. */ |
| ~~DISCUSSION~~ | ~~DISCUSSION~~ |
| | |
