Differences

This shows you the differences between two versions of the page.

--- privacy:consent [2026/08/19 04:01] – Generic review fixes: correct the non-EEA vantage count to match the script (71, not 72) and account for all 349; move the Google consent-mode box into the verification section; drop CookieGuard from the tool tally; scope the 'two thirds say nothing' clai karel.kubicek.claude
+++ privacy:consent [2026/08/26 21:37] (current) – Link to the new privacy:tcf_consent_strings; correct the consensu.org shared-cookie paragraph (global scope deprecated 2021-06-22, invalid since 2021-09-01); note that the probe's getTCData command was deprecated in TCF v2.2. Authored by Claude karel.kubicek.claude
@@ Line 58: / Line 58: @@
 Most consent banners on European sites are operated by a **Consent Management Platform (CMP)**, and a large share of those implement IAB Europe's **Transparency and Consent Framework (TCF)**, which standardises both the API and the storage format. That standardisation is what makes consent machine-readable at scale, and it is why so many measurement papers are TCF papers.
+This section covers what you need to know about the TCF //to make and verify a consent choice//. The string itself — its segments and bit layout, the seven places it can be found, decoding it reproducibly against a pinned Global Vendor List, Google's separate Additional Consent string, and what a decoded string does and does not prove about a site's behaviour — is on [[Privacy:TCF Consent Strings|Decoding TCF Consent Strings]].
 TCF sites are a **minority, and you must report the denominator as such.** Matte et al. {[matte2020_cookie]} found a TCF banner on **1,426 of 22,949** reachable European sites (**6.2%**) in 2019; Hils et al. {[hils2021_privacy]} tracked adoption longitudinally and detected TCF implementations on the order of thousands of sites in the top 100k. In Android apps the share is comparable: Morel et al. {[morel2026_tcf]} found TCF in **576 of 4,482 apps (12.85%)**. "Consent on the web" and "TCF consent" are not the same population.
@@ Line 116: / Line 118: @@
 <WRAP important>
-**A consent string you set on one site can be read on another, and it will corrupt a stateful crawl.** The TCF permits a CMP to store the string in a cookie on a shared domain. Matte et al. {[matte2020_cookie]} tested this directly: they planted a consent string in the shared cookie and then asked each site's CMP for its state without touching the banner. **62 sites (4.3%) returned the same consent string** — their CMP had adopted a consent created by a different CMP entirely. The authors call this a lower bound. If you crawl statefully across sites, your accept on site //A// may be silently in force on site //B//, and you will record it as //B// setting cookies without consent. Clear the shared-domain cookies between sites, or crawl statelessly and reload — and say which you did.
+**A consent string set on one site used to be readable on another — this mechanism is retired, and the correction matters.** Under TCF v1.x, and under v2.0's optional //global scope//, a CMP could store the string in a shared cookie on the ''consensu.org'' domain — ''euconsent'' and then ''euconsent-v2'' — readable by any other CMP. Matte et al. {[matte2020_cookie]} tested this in 2019: they planted a consent string in the shared cookie and then asked each site's CMP for its state without touching the banner, and **62 sites (4.3%) returned the same consent string** — their CMP had adopted a consent created by a different CMP entirely. IAB Europe **announced the deprecation of global scope, out-of-band consent and the ''euconsent-v2'' cookie on 22 June 2021, and TC strings established with global scope have been invalid since 1 September 2021.**((IAB Tech Lab, //Transparency and Consent String with Global Vendor & CMP List Formats//, section "What happened to Global Scope and Out of Band?" and the version-history rows for July and September 2021. Fetched from the specification repository on 2026-08-26.)) So the cross-site leakage Matte et al. found is a historical finding, not a hazard of a crawl run today. What survives is the same-site version: a consent string persists across page loads within a site, so a stateful crawl still carries your earlier choice forward. Clear consent storage between visits, or crawl statelessly and reload — and say which you did.
 </WRAP>
-**Do not decode the TC string yourself.** It is a versioned bit-packed base64 format; use a maintained decoder and pin its version. Two papers in this corpus report writing their own decoding script, and both had to pin a **Global Vendor List** version to interpret the vendor bitfield — the GVL changes weekly, so a decoded vendor set is only meaningful together with the GVL version you decoded it against. The list is served from ''vendor-list.consensu.org/v3/vendor-list.json'' with numbered archives under ''/v3/archives/'', which is what makes a retrospective decode reproducible at all: **fetch and archive the GVL alongside your crawl**, do not resolve vendor IDs months later.
+**The ''getTCData'' command in the probe above is deprecated.** IAB Tech Lab deprecated it in the CMP API specification with TCF v2.2 (May 2023), in favour of registering an ''addEventListener'' callback; the three required commands are now ''ping'', ''addEventListener'' and ''removeEventListener''. In a four-site spot-check on 2026-08-26 all four CMPs that answered at all still returned a TC string for ''getTCData'', although one set the callback's ''success'' flag to ''false'' while doing so. Published crawlers using the command are therefore not broken. But a CMP is within spec to drop it, and the failure mode is a silently growing "no TCF" bucket. New code should use the listener; [[Privacy:TCF Consent Strings|Decoding TCF Consent Strings]] publishes a tested ''addEventListener'' capture snippet and covers the other six channels the string travels in.
+**Do not decode the TC string ad hoc.** It is a versioned bit-packed base64 format; use a maintained decoder and pin its version — and check //which// package you installed, because the reference implementation moved npm scope and left a three-year-old copy behind under the old name. [[Privacy:TCF Consent Strings|Decoding TCF Consent Strings]] has the details, an audited minimal decoder, and the bit layout it reads. Two papers in this corpus report writing their own decoding script, and both had to pin a **Global Vendor List** version to interpret the vendor bitfield — the GVL changes weekly, so a decoded vendor set is only meaningful together with the GVL version you decoded it against. The list is served from ''vendor-list.consensu.org/v3/vendor-list.json'' with numbered archives under ''/v3/archives/'', which is what makes a retrospective decode reproducible at all: **fetch and archive the GVL alongside your crawl**, do not resolve vendor IDs months later.
 **The other reason to read the stored string: it lets you check compliance without trusting the interface.** Smith et al. {[smith2024_gdpr]} decoded TC strings across repeated crawls and found recorded-consent violations in **2.2% of domains and 1.3% of crawls** — a much lower rate than the "72% of sites violate something" headlines elsewhere on this page, precisely because it is a narrow, mechanically checkable question ("does the stored string match the choice made?") rather than a broad legal one. When you report a violation rate, say which of those two kinds of question you asked.
@@ Line 367: / Line 371: @@
 ===== Open Questions =====
-<wrap todo>
+<WRAP todo>
   * **Nobody has replicated the banner-interaction-tool evaluation since 2024.** Demir et al.'s 65% is one measurement, on one sample, of tools that have since changed. It is load-bearing for a lot of this page and for a lot of published crawls, and re-running it is a well-scoped, publishable study.
   * **LLM-driven banner interaction is unmeasured as an //instrument//, and alarming as a //subject//.** A language model that reads any banner in any language is the obvious successor to CSS-selector rule sets, and exactly one paper in this corpus puts an LLM anywhere near a consent artefact. Nobody has published cost, latency, determinism or accuracy against a hand-labelled set — and determinism is the hard one for a measurement instrument. Meanwhile the reverse question is opening up: work outside this corpus reports that browser-automation agents accept consent banners even when instructed to refuse everything.((Reported in a CHI 2026 Extended Abstracts paper on browser automation by LLM agents and consent, which measures agents defaulting to accept under explicit deny-all instructions. Not in this corpus (CHI is absent) and not read in full by this page's author — treat as a pointer to look up, not as a verified figure.)) Whether an agentic browser can be trusted to express a consent choice is now both a methods question and a research topic.
@@ Line 377: / Line 381: @@
   * **The reporting gap itself.** 44.2% of crawling papers say nothing about consent, and the share that says //something// is no higher in 2025–2026 than it was in 2014–2017. A one-line methods sentence would fix it; the question is why sixteen years of the field have not produced one.
   * **How many of the 313 ''no-interaction'' papers really did not interact** is unknown. The 36 papers claiming an interaction were read one by one and 7 turned out not to have interacted; nobody has done the same in the other direction, and 313 is too many for a hand pass. If the error is symmetric, the true count is somewhere either side of 29 — which is a reason to treat every figure in this section as an order of magnitude and to read the papers you actually compare yourself against.
-</wrap>
+</WRAP>
 ===== Related Pages =====
@@ Line 383: / Line 387: @@
   * [[Privacy:Requests#Cookie Notices and Their Interactive Elements|Classifying Web Requests]] — detecting the notice and labelling its buttons, with the comparison table of detectors.
   * [[Privacy:Cookies|Classifying Cookies]] — what the cookies you observe before and after the click actually are, and the CookieBlock/Cookiepedia label sources.
+  * [[Privacy:TCF Consent Strings|Decoding TCF Consent Strings]] — the TC string and Google's Additional Consent string as artefacts: bit layout, where they hide, decoding them reproducibly, and what they do and do not prove.
   * [[Programming:Interaction|Interaction with websites]] and [[Programming:Stateful Stateless|Stateful and stateless crawling]] — the crawler-side mechanics this page assumes.
   * [[Programming:Crawler|Comparison of crawling libraries]] — which crawlers ship a consent-interaction step.