Differences

This shows you the differences between two versions of the page.

--- privacy:requests [2026/08/26 18:53] – Link the new privacy:cookie_syncing page from Related Pages. Authored by Claude karel.kubicek.claude
+++ privacy:requests [2026/08/29 13:36] (current) – Replace the last literal corpus figure inherited from the carve-out with a pointer to the section that maintains it. Authored by Claude karel.kubicek.claude
@@ Line 69: / Line 69: @@
 ===== Block Lists =====
-==== Crowd-Sourced and Outdated ====
+Matching a request against an Adblock-syntax filter list is what **57.0%** of the papers below do, and it is still the default. What it means to do that well — which lists exist and which are dead, how to record the exact version and commit you matched against, which engine to use, what a rule needs from your crawl in order to be evaluated at all, what the list misses and where, and why "on the list" is a definition rather than a finding — is one topic and it now has its own page:
-The main principle of block lists is their crowd-sourced nature. For instance, the [[https://github.com/easylist/easylist|EasyList repository]] has **324 contributors, roughly 287,000 commits and 8,901 closed issues** as of 2026-08-12.((Measured against the GitHub REST API on 2026-08-12: ''api.github.com/repos/easylist/easylist'' plus ''Link''-header pagination on ''/contributors?per_page=1'' and ''/commits?per_page=1'', and ''search/issues?q=repo:easylist/easylist+type:issue+state:closed''. The count is 386 if anonymous and deleted accounts are included. The commit count is given loosely on purpose: it read 286,987 at 16:26 UTC and 286,990 at 17:00 UTC on the same day. These figures were "almost 300 / over 200k / more than 7k" in an earlier version of this page; they grow, so re-measure rather than cite this sentence.)) This has several implications:
+  * **[[Programming:Filter Lists|Filter lists]]** — the list as instrument and as ground truth.
-  * Widespread advertisers and trackers have well-defined and up-to-date rules, while the long tail of tracking companies might not be covered well.
+The three things from it you cannot skip while reading this page:
-  * Adding rules is much more common than deleting them. Snyder et al. {[snyder2020_who]} applied EasyList to a sample of 10,000 websites and found that **90.16% of its resource-blocking rules provide no benefit to users in common browsing scenarios**; they measured EasyList growing from a few hundred rules to over 60,000 across its first nine years. Böttger et al. {[bottger2025_regional]} reproduced the shape of that result on regional lists five years later: across their crawl profiles **93.3% of distinct rules identified no tracking request at all** — only **8,163 of 122,548 (6.6%)** ever matched one.((The 143,654-rule figure that also appears in that paper belongs to a different experiment — the runtime and memory benchmark over all lists combined — and is not the denominator for this result. Getting that pairing wrong is easy and it is how this page first published it.))
-  * There is a cat-and-mouse game between list maintainers and the advertising industry. Some rules (e.g., [[https://www.engadget.com/inside-the-arms-race-between-youtube-and-ad-blockers-140031824.html|YouTube's war on ad blocking]]) are short-lived, making it necessary to use up-to-date lists.
-<WRAP important>
+  * **A rule is evaluated against a request //in context//.** ''$third-party'', ''$domain='', the resource-type options and ''@@'' exception rules all need the initiator URL, the resource type and the redirect chain. A crawl that logged only request URLs cannot be post-processed with a list, and it fails silently rather than erroring.
-**"We used EasyList" is not a reproducible method, and the list itself tells you why.** Running the provenance script below three times within half an hour on 2026-08-12 returned three different lists: ''Version: 202608121614'' with 52,556 network rules, ''202608121621'' with 52,557, then ''202608121642'' with 52,560. Each carries a ''! Commit:'' field naming the exact git commit that produced it. There is no excuse for not recording it, and no way to reproduce a result that does not.
+  * **The list is a moving target.** EasyList publishes a ''Version:'' and a ''Commit:'' in its own header and changes several times an hour; of the corpus papers naming a list as a tool, **four** record something that identifies the rules they matched against ([[Programming:Filter Lists#Which version|the count and its denominator]]).
-</WRAP>
+  * **A filter-list hit is a noisy label**, estimated at **16%–19% false positives** {[calzavara2026_syntactic]}, with the false negatives measured repeatedly and separately — 25.22% {[fouad2020_missed]}, 34.5% {[lee2023_track]}, and much worse off the desktop web.
-==== Blocking Specific Resources ====
-Rules can prevent actions from happening (useful for protecting user privacy), either by blocking entire domains or specific requests based on their paths. Alternatively, rules can be applied after loading a resource (described by a CSS selector) to prevent its rendering, which is more useful for blocking advertisements or annoying elements.
-**The split matters for your measurement, and it is bigger than people expect.** Measured on the live lists on 2026-08-12: EasyList is **52,557 network rules and 24,422 cosmetic rules**, so nearly a third of it cannot answer "would this request have been blocked". EasyPrivacy is **54,850 network rules and 34 cosmetic ones** — effectively all network. EasyList Cookie List inverts this completely at **1,668 network and 22,979 cosmetic** rules, because hiding a banner is a rendering problem. If your pipeline reports "rules matched", say which kind.
-==== Existing Lists ====
-Status column checked **2026-08-12** against the primary source in each row. Filter lists die quietly, and a dead list in a 2026 crawl is a silent measurement error.
-^ List ^ What it is for ^ Status, 2026-08-12 ^
-| [[https://easylist.to/index.html|EasyList]] | Advertisements, except first-party self-promotion. Full [[https://easylist.to/pages/policy.html|policy]] | **Active**, pushed the day this was checked; ''Version:'' and ''Commit:'' in the header |
-| **EasyPrivacy** | Tracking and privacy. Four documented categories: generic URL/tracking patterns used by first or third parties; self-hosted and CNAME first-party trackers; third-party-hosted tracking scripts; and tracking servers blocked at URL level((Quoted from [[https://easylist.to/pages/policy.html|easylist.to/pages/policy.html]], checked 2026-08-12. The page carries no revision date, so how long that text has stood cannot be established from it.)) | **Active** |
-| [[https://github.com/disconnectme/disconnect-tracking-protection|Disconnect tracking protection]] | Tracker domains grouped by company and category | **Active** (last push 2026-08-07). CC BY-NC-SA 4.0 — **not** a permissive licence, check it before redistributing. Firefox does not consume this repo directly; it ships the Safe-Browsing-format build from [[https://github.com/mozilla-services/shavar-prod-lists|shavar-prod-lists]] |
-| [[https://spreadprivacy.com/duckduckgo-tracker-radar/|DuckDuckGo Tracker Radar]] | Automatically constructed tracker domains with company attribution, prevalence, fingerprinting and cookie signals. [[https://github.com/duckduckgo/tracker-radar/blob/main/docs/DATA_MODEL.md|Data model]] | **Active** (pushed 2026-08-12). The blocklist derived from it is at ''duckduckgo/tracker-blocklists'' — note the **plural**; the singular repository name 404s |
-| [[https://www.ghostery.com/whotracksme/|Ghostery WhoTracks.me]] | Tracker prevalence dataset over the top 10,000 sites, with company attribution | **Active** (repo pushed 2026-08-04). Both URLs moved: ''whotracks.me'' now redirects to ''ghostery.com/whotracksme/'', and the repository to [[https://github.com/whotracksme/whotracks.me|whotracksme/whotracks.me]] |
-| [[https://github.com/AdguardTeam/AdguardFilters|AdGuard filters]] | Ads, tracking, annoyances, and a separate URL-tracking filter | **Active.** GitHub Releases is empty — the canonical download is ''filters.adtidy.org/<platform>/filters/<N>.txt'', whose header carries ''Version'', ''TimeUpdated'' and ''Expires''. The registry mapping IDs to lists is [[https://github.com/AdguardTeam/FiltersRegistry|FiltersRegistry]] |
-| [[https://pgl.yoyo.org/adservers/|Peter Lowe's list]] | Ad and tracking servers, domain-level, deliberately small | **Active**: 3,525 entries, last updated the day this was checked. A useful high-precision, low-recall comparison point |
-| **hpHosts / MalwareBytes ATS** | Historic hosts-file ad/tracking list | **Dead.** ''hosts-file.net'' did not resolve on 2026-08-12; Malwarebytes stopped updating it around 2020. It appears in older papers — do not reuse it |
-| [[https://justdomains.github.io/blocklists/|justdomains]] | Not a list: a domain-only re-encoding of EasyList, EasyPrivacy, AdGuard DNS and others | **Active as a conversion layer.** Cite the upstream list, not this |
-| [[https://easylist.to/pages/other-supplementary-filter-lists-and-easylist-variants.html|Country-specific lists]] | Mostly advertising, region-specific | **Active.** Böttger et al. {[bottger2025_regional]} is the only systematic study of what they add |
-| **EasyList Cookie List** | CSS selectors for cookie notices. Now inside the main easylist repo at ''easylist_cookie/'' | **Active** (last modified 2026-08-11). No ''Version:'' field, only ''Last modified:'' |
-| [[https://github.com/OhMyGuus/I-Still-Dont-Care-About-Cookies|I Still Don't Care About Cookies]] | Removes notices //and// clicks through them. Useful when the specific consent action does not matter | **Active** (pushed 2026-08-12), GPL-3.0, but 14,232 open issues. Click rules are still at ''src/data/js/5_clickHandler.js'' — the numeric prefixes have gaps, so do not assume contiguous ordering |
-| [[https://github.com/duckduckgo/autoconsent|DuckDuckGo autoconsent]] | CMP-specific rules that can express //which// purposes to accept or reject | **Active and the most maintained of the three** (pushed 2026-08-12). This is the one to build on |
-| [[https://github.com/cavi-au/Consent-O-Matic|Consent-O-Matic]] | The original CMP rule set, with per-purpose consent | **Alive but slower** (last push 2026-02-07, 154 open issues) |
-| **Mozilla ''cookie-banner-rules-list''** | Firefox's cookie-banner handling rules | **Archived** on GitHub, last push 2025-01-07. Do not start here |
-| [[https://github.com/AdguardTeam/cname-trackers|AdGuard CNAME disguised trackers]] | The one list for the problem that defeats hostname matching: trackers reached through a first-party CNAME | **Active** — ''data/combined_disguised_trackers.txt'', 175,827 lines, ''TimeUpdated: 2026-08-03''. **You cannot use it from request logs alone**: you need the CNAME chain, which means recording DNS resolution at crawl time. Most crawlers do not, by default |
-| DNS-blocking lists | Domain-level blocking, no path granularity | See [[design:website_classification|Website classification]] |
-==== What a Filter List Misses, Measured ====
-You will be asked this in review, so have the numbers.
-^ Study ^ Method used as the reference ^ What the lists missed ^
-| Fouad et al. {[fouad2020_missed]}, PETS 2020 | Behavioural detection seeded by invisible pixels (35.66% of all collected images were invisible) | EasyList+EasyPrivacy missed **25.22%** and Disconnect **30.34%** of the trackers they detected; every browser extension tested missed at least **24.38%** |
-| Chen et al. {[chen2021_detecting]}, IEEE S&P 2021 | Event-loop-turn JavaScript behaviour signatures built from blocked scripts | **3,589 scripts** hosting behaviour known to be harmful, on **12.48%** of sites measured, missed by the lists that had already blocked the same behaviour elsewhere |
-| Lee et al. {[lee2023_track]}, TheWebConf 2023 | Packet-metadata classifier, then manual inspection of its "errors" | **34.5%** of the classifier's apparent false positives were real trackers the lists had not yet discovered |
-| Lee et al. {[lee2024_adflush]}, TheWebConf 2024 | A deployable ML classifier, reported upstream | **642 ads and trackers across 108 domains** confirmed and reported to EasyList, EasyPrivacy and uBlock |
-| Shuang et al. {[shuang2025_duumviri]}, NDSS 2025 | Differential breakage testing, no labels | **22 previously unreported trackers** (175 occurrences) in one evaluation set |
-| Calzavara et al. {[calzavara2026_syntactic]}, PoPETs 2026 | Dynamic taint tracking, head to head | **7,021 requests (≈17% of the dataset)** found only by taint tracking; and 16%–19% of the list's own hits were likely false positives |
-The reverse direction has a number too, and it is a lesson in how to read one. AdFlush {[lee2024_adflush]} and the filter lists it was compared against flagged almost exactly the same //volume// of traffic — **39.27% versus 39.92% of all requests** — and the paper reads that as evidence AdFlush "can effectively serve as a substitute for filter lists". The same paper also reports **642 ads and trackers on 108 domains that the lists had missed**, which it reported upstream. Both are true, and they are the point: **equal marginals are not the same as equal sets.** If you compare a classifier against a list, report the confusion matrix, not the two totals.
-==== Programming: Using Lists ====
-Block lists use regular expressions with custom syntax to decide which resources to block. The advantage of applying them in post-processing is that you can run a crawl allowing all resources and classify afterwards — and re-classify with a different list later, which you cannot do if you blocked at crawl time.
-Maintenance status checked **2026-08-12**; this is the section of the page that rots fastest.
-^ Library ^ Language ^ Status, 2026-08-12 ^ Use it? ^
-| [[https://github.com/brave/adblock-rust|brave/adblock-rust]] | Rust | **Active** (pushed 2026-08-12). The crate is named ''adblock'', **not** ''adblock-rust'' — v0.13.2, published 2026-07-19 | **Yes.** Brave ships it, so it tracks uBlock-era syntax including ''$removeparam'' |
-| ''adblock-rs'' | Node | **Active**, v0.13.2 published 2026-07-19, tracks the Rust crate 1:1 | **Yes**, if you are in Node |
-| [[https://github.com/ghostery/adblocker|@ghostery/adblocker]] | TypeScript/Node | **Active**, v2.18.2 published 2026-08-05. Renamed: ''@cliqz/adblocker'' is **deprecated** on npm and last published 2024-10-16 | **Yes.** Ghostery's own engine |
-| ''adblock'' (PyPI, ''ArniDagur/python-adblock'') | Python | **Stale**: v0.6.0, uploaded 2022-07-17. Wraps an old ''adblock-rust'' | Only with a pinned, stated version |
-| ''braveblock'' (PyPI) | Python | **Stale**: v0.5.1, uploaded 2024-02-14 | Only with a pinned, stated version |
-| [[https://github.com/englehardt/abp-blocklist-parser|abp-blocklist-parser]] | Python | **Abandoned**: last push 2020-11-02, 10 commits total, open issues include "Python 3 support" | **No.** It predates ''$removeparam'' and most modern rule options |
-| [[https://github.com/scrapinghub/adblockparser|adblockparser]] | Python | **Archived** on GitHub 2019; PyPI v0.7 from 2016-10-17 | **No** |
-<WRAP important>
-**There is no actively maintained Python binding to a modern filter-list engine as of 2026-08-12.** Both PyPI wrappers around Brave's engine are years behind the Rust crate, and the two pure-Python parsers are abandoned. If your pipeline is Python and your claim depends on rule semantics, either shell out to the Rust or Node engine, or state the parser version and accept that unsupported rule options are silently not matched. This is a real threat to validity that papers do not usually mention.
-</WRAP>
-<WRAP important>
-**Before you crawl at all: someone may already have.** [[https://httparchive.org/|HTTP Archive]] runs a monthly request-level crawl of millions of pages and publishes it as a public BigQuery dataset, and the Web Almanac's **Privacy** chapter is this measurement joined against WhoTracks.me and AdGuard's CNAME-tracker list. Note that its //Third Parties// chapter answers a different question and categorises with Patrick Hulce's ''third-party-web'' dataset, not with filter lists — check which chapter's method you are actually reusing.((Both chapters read at ''almanac.httparchive.org/en/2024/'', 2026-08-12. An earlier draft of this box attributed the filter-list join to a non-existent "advertising" chapter; there is none.)) Its limits are real and you must state them if you use it — it loads the landing page, does not interact, and does not carry a consent state — but if your question is a prevalence or a longitudinal trend rather than a behaviour under interaction, running your own crawl may be the expensive way to get a worse answer.
-</WRAP>
-<WRAP important>
-**Post-hoc matching needs more than a list of URLs, and a crawl that logged only URLs fails silently.** An Adblock-syntax rule is evaluated against a //request in context//: ''$third-party'', ''$domain='', the resource-type options (''$script'', ''$image'', ''$xmlhttprequest'', …) and ''@@'' exception rules all depend on where the request came from and what kind of thing it is. Brave's engine says so in its own signature — ''Request::new(url, source_url, request_type, method)''.((''src/request.rs'' in [[https://github.com/brave/adblock-rust|brave/adblock-rust]], checked 2026-08-12: the constructor takes the request URL, the //source// URL, the request type and the method, and derives ''third_party'' by comparing the two domains.)) So record, per request:
-  * the **request URL**;
-  * the **initiator or document URL** of the frame that issued it — without this, every ''$third-party'' and ''$domain='' modifier silently evaluates against nothing;
-  * the **resource type** (''script'', ''image'', ''xmlhttprequest'', ''sub_frame'', …), which CDP and Playwright both give you (''resourceType()'');
-  * the **redirect chain**, because a rule may match a hop you did not keep;
-  * and evaluate **exception rules**. EasyList carries 758 of them and EasyPrivacy 836 (2026-08-12); ignoring them inflates your block count with no error message.
-[[Programming:Crawler|The crawler page]] compares which libraries expose each of these.
-</WRAP>
-Cosmetic filters are a different matter: **do not try to evaluate them with your automation library's CSS selector engine.** Modern cosmetic filters are a superset of CSS — procedural filters such as ''`:has-text()`'', and HTML filtering (''##^'') — that a ''find_element'' call cannot express, and finding an element is not the same as hiding it. If you need cosmetic filtering, use the cosmetic API of the Ghostery or Brave engine to resolve the selectors for a hostname first, then query with those.
-==== Manifest V3 Caps How Many Rules an Extension Can Load ====
-If your measurement design is "install a blocker extension and see what it blocks", this is now a hard constraint rather than an engineering detail. Chrome's ''declarativeNetRequest'' limits, from the API documentation on **2026-08-12**:(([[https://developer.chrome.com/docs/extensions/reference/api/declarativeNetRequest|developer.chrome.com — chrome.declarativeNetRequest]], "Rule limits", fetched and read 2026-08-12. The constant names are the documentation's own.))
-Two of them bite, and both are documented constants you can read at runtime rather than hard-code:
-  * ''GUARANTEED_MINIMUM_STATIC_RULES'' = **30,000** static rules across at most 50 enabled rulesets;
-  * ''MAX_NUMBER_OF_REGEX_RULES'' = **1,000** regular-expression rules of each type, each under 2 KB compiled.
-(The dynamic and session limits — 5,000 unsafe dynamic, 30,000 safe dynamic since Chrome 121, 5,000 session — are in the same reference and matter less for a blocker shipping a list.)
-Compare those against the lists: **EasyList alone carries 52,557 network rules and EasyPrivacy 54,850** (measured 2026-08-12). Neither fits in the 30,000-rule static guarantee, let alone both, and the 1,000-rule regex cap bites well before that. An MV3 extension therefore ships a //subset// of the list, chosen by its authors — which is what uBlock Origin Lite is and why it is not feature-equivalent to uBlock Origin.((''gorhill/uBlock'' remains actively maintained (pushed 2026-08-11) and works fully on Firefox, which still supports MV2. Chrome disabled MV2 extensions in Chrome 138 and its published date for removing them from the Chrome Web Store is 2026-08-31. The MV3 replacement is [[https://github.com/uBlockOrigin/uBOL-home|uBlock Origin Lite]] (pushed 2026-08-12). Repository dates measured against the GitHub API on 2026-08-12; the Chrome Web Store listing itself could not be fetched — it redirects to a Google consent wall — so the store-side dates come from Google's own [[https://developer.chrome.com/docs/extensions/develop/migrate/mv2-deprecation-timeline|Manifest V2 deprecation timeline]], which states verbatim (checked 2026-08-12) that "with Chrome 138 all users on all channels of Chrome have now Manifest V2 extensions disabled" and that on "Aug 31st 2026: All remaining Manifest V2 extensions removed from the Chrome Web Store".))
-**Firefox has its own caps, and they are not Chrome's.** Firefox implements ''declarativeNetRequest'' too and exposes the same constants — ''GUARANTEED_MINIMUM_STATIC_RULES'', ''MAX_NUMBER_OF_ENABLED_STATIC_RULESETS'', ''MAX_NUMBER_OF_REGEX_RULES'' — but MDN records the dynamic and session limits changing at //Chrome 120 and Firefox 128// independently, so the numbers are version-specific per browser rather than shared.((MDN, [[https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/declarativeNetRequest|''declarativeNetRequest'']], checked 2026-08-12. Read the constants at runtime rather than hard-coding them. We did not find a primary Apple source for Safari's content-blocker rule cap and so do not state one.)) **If your measurement compares blockers across browsers, read the constants at runtime and report them** — the rule budget is a confound, not a constant.
-Lukić and Papadopoulos {[lukic2026_mv3]} measured whether this actually degrades blocking and found **no significant reduction in ad blocking and a significant //increase// in anti-tracking** between MV2 and MV3 builds of four popular blockers — MV3 blocked about 1.8 more trackers per site — but they did find MV3 leaving cosmetic placeholders in 21% of cases where MV2 left none. **The blocking survived; whether an MV3 extension is a sound //measurement instrument// is a different question, and it is not settled.** If your method depends on the extension seeing every request, prefer post-processing a full crawl with the Rust or Node engine above, where no rule cap applies.
-==== Recording Which List You Used ====
-Run this at the start of a crawl and archive the JSON with your data. It reads the metadata the lists publish about themselves, counts rules by kind, and hashes the bytes.
-<file python filterlist_provenance.py>
-#!/usr/bin/env python3
-"""Record exactly which filter list you used, so your crawl is reproducible.
-Filter lists change several times a day. "We used EasyList" is not a method;
-"EasyList, Version 202608121614, commit cff91d2b, 52,556 network rules" is. Every
-Adblock-Plus-syntax list carries that metadata in its own header comments --
-this script extracts it, counts the rules by kind, and writes a JSON record you
-can archive next to your crawl data and paste into your methodology section.
-Run it at the START of a crawl, not when you write the paper.
-    python3 filterlist_provenance.py > filterlists.json
-    python3 filterlist_provenance.py --save lists/   # also archive the raw .txt
-"""
-import argparse
-import hashlib
-import json
-import pathlib
-import re
-import sys
-import urllib.request
-from datetime import datetime, timezone
-# The lists a tracking measurement usually wants. Add your regional list here.
-LISTS = {
-    "easylist": "https://easylist.to/easylist/easylist.txt",
-    "easyprivacy": "https://easylist.to/easylist/easyprivacy.txt",
-    "easylist-cookie": "https://secure.fanboy.co.nz/fanboy-cookiemonster.txt",
-    "adguard-url-tracking": "https://filters.adtidy.org/windows/filters/17.txt",
-    "peter-lowe": "https://pgl.yoyo.org/adservers/serverlist.php"
-    "?hostformat=adblockplus&showintro=0&mimetype=plaintext",
-}
-# Header fields Adblock-syntax lists publish about themselves. Not every list
-# sets every field; a missing field is reported as None rather than guessed.
-HEADER_FIELDS = ("Version", "Last modified", "TimeUpdated", "Expires", "Commit", "Title")
-# A cosmetic rule hides an element that has already loaded; a network rule stops
-# the request. Only network rules answer "would this request have been blocked".
-# Note the order in classify(): a cosmetic *exception* (`site.com#@#.ad`) is
-# counted as cosmetic, not as an exception, because the question this count
-# answers is "how much of this list is about requests at all".
-COSMETIC = re.compile(r"#[@?$]?#|#\$#")
-def classify(line):
-    if not line.strip() or line.startswith("!") or line.startswith("["):
-        return "comment"
-    if COSMETIC.search(line):
-        return "cosmetic"
-    if line.startswith("@@"):
-        return "exception"
-    return "network"
-def describe(name, url):
-    req = urllib.request.Request(url, headers={"User-Agent": "filterlist-provenance/1.0"})
-    with urllib.request.urlopen(req, timeout=60) as resp:
-        raw = resp.read()
-    text = raw.decode("utf-8", errors="replace")
-    lines = text.split("\n")
-    header = {}
-    for field in HEADER_FIELDS:
-        m = re.search(rf"^!\s*{re.escape(field)}:\s*(.+)$", text[:8000], re.M)
-        header[field] = m.group(1).strip() if m else None
-    counts = {"comment": 0, "cosmetic": 0, "exception": 0, "network": 0}
-    for line in lines:
-        counts[classify(line)] += 1
-    return {
-        "name": name,
-        "url": url,
-        "fetched_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
-        "sha256": hashlib.sha256(raw).hexdigest(),
-        "bytes": len(raw),
-        "header": header,
-        "rules": counts,
-    }, raw
-def main():
-    ap = argparse.ArgumentParser()
-    ap.add_argument("--save", metavar="DIR", help="also write the raw list files here")
-    args = ap.parse_args()
-    out = []
-    for name, url in LISTS.items():
-        try:
-            record, raw = describe(name, url)
-        except Exception as exc:
-            # Fail loud: a list you could not fetch is a hole in your method,
-            # not something to paper over with a stale local copy.
-            print(f"FAILED {name} <{url}>: {exc}", file=sys.stderr)
-            raise
-        out.append(record)
-        if args.save:
-            d = pathlib.Path(args.save)
-            d.mkdir(parents=True, exist_ok=True)
-            (d / f"{name}.txt").write_bytes(raw)
-        # Not every list sets `Version:`; EasyList Cookie List and Peter Lowe's
-        # list only set `Last modified:`. Report whichever exists, and say so.
-        stamp = (
-            record["header"]["Version"]
-            or record["header"]["TimeUpdated"]
-            or record["header"]["Last modified"]
-            or "(no version or date in header)"
-        )
-        print(
-            f"{name:22} {stamp:24}"
-            f" network={record['rules']['network']:7} cosmetic={record['rules']['cosmetic']:7}"
-            f" exception={record['rules']['exception']:6} sha256={record['sha256'][:12]}",
-            file=sys.stderr,
-        )
-    json.dump(out, sys.stdout, indent=2)
-    print(file=sys.stdout)
-if __name__ == "__main__":
-    main()
-</file>
-Real output, 2026-08-12:
-<code>
-easylist               202608121621             network=  52557 cosmetic=  24422 exception=   758 sha256=652ca564670f
-easyprivacy            202608121630             network=  54850 cosmetic=     34 exception=   836 sha256=809f80964d06
-easylist-cookie        2026-08-11 23:25 UTC     network=   1668 cosmetic=  22979 exception=   427 sha256=4de43038a16a
-adguard-url-tracking   2.0.13.86                network=   2492 cosmetic=      2 exception=    91 sha256=c468efefcdfe
-peter-lowe             Wed, 12 Aug 2026 15:15:56 GMT network=   3525 cosmetic=      0 exception=     0 sha256=25093c8b037f
-</code>
 ===== ML Classification =====
@@ Line 393: / Line 156: @@
 **The pipeline everyone converges on has three stages, and each has a measured cost.**
-  - **Find the notice.** Start with the **EasyList Cookie List** CSS selectors (1,668 network and 22,979 cosmetic rules on 2026-08-12) and add DOM/text heuristics: high ''z-index'', ''position: fixed'', a privacy-related keyword pool, a container that overlaps the viewport bottom or centre.
+  - **Find the notice.** Start with the **EasyList Cookie List** CSS selectors ([[Programming:Filter Lists#The lists|current rule counts here]] — it is overwhelmingly cosmetic, which is what you want for finding a banner) and add DOM/text heuristics: high ''z-index'', ''position: fixed'', a privacy-related keyword pool, a container that overlaps the viewport bottom or centre.
   - **Label the interactive elements.** Accept / reject / close / save / settings / other. Button text is short, multilingual and adversarially designed, which is why this is the stage that moved from keyword lists to learned models.
   - **Decide what to click, and verify it happened.** A click that silently fails is worse than no click, because the crawl continues and reports pre-consent behaviour as post-consent. The consent-interaction crawlers the field shares for this — BannerClick, Priv-Accept and the ''autoconsent'' integration built into [[Programming:Crawler:Tracker Radar Collector|Tracker Radar Collector]] — are compared on [[Programming:Crawler#Specialised Measurement Crawlers|the crawler page]].
@@ Line 458: / Line 221: @@
 ==== Which lists the field actually uses ====
-Of the **197** papers that used or produced an advertising-or-tracking filter list. A paper naming several lists is counted under each, so the shares do not sum to 100%. Names were folded into families, because they are free text: the //Spellings folded// column is how many distinct strings the corpus uses for each.
+Of the **197** papers that used or produced an advertising-or-tracking filter list. A paper naming several lists is counted under each, so the shares do not sum to 100%. Names were folded into families, because they are free text: the //Spellings folded// column is how many distinct strings the corpus uses for each. [[Programming:Filter Lists#Which lists|The filter-lists page]] computes the same table over a slightly wider population (198 papers, because it does not intersect with this page's request-classification task fold) and is the canonical version; the two differ by one or two papers per row, which is itself a useful illustration of how much a population definition moves a count.
 ^ Filter list ^ Papers ^ Share of 197 ^ Spellings folded ^
@@ Line 562: / Line 325: @@
 ===== What to Report =====
-  - **The list, its version and its commit.** ''EasyList Version 202608121614, commit cff91d2b'' — not "EasyList". Archive the ''.txt'' with your artefact. 70% of papers do not do this.
+  - **The list, its version and its commit**, and the archived ''.txt'' — not "EasyList". 70% of papers give no version at all and 98% give nothing that identifies the rules. [[Programming:Filter Lists#Recording Which List You Used|How, and a script that does it]].
-  - **The engine and its version**, separately from the list. ''adblock'' 0.13.2 and ''adblockparser'' 0.7 do not match the same rules, and the second one silently ignores options it does not know.
+  - **The engine and its version**, separately from the list, because two parsers of the same list do not match the same rules and the older ones silently ignore options they do not know. [[Programming:Filter Lists#Post-processing beats blocking at crawl time|Which engines are maintained]].
   - **Which rule kinds you evaluated.** Network only, or cosmetic too? Nearly a third of EasyList is cosmetic and answers a different question.
   - **The unit, and how you decided "party".** Domain, eTLD+1, URL, chain or parameter — and whether party is by public suffix list (name which one) or by **entity map** (name which one). The two give different third-party rates for the same crawl.
@@ Line 580: / Line 343: @@
   * **Python has no maintained filter-list engine.** Someone should either revive ''python-adblock'' against ''adblock'' 0.13.x or state loudly that Python pipelines must shell out.
   * **Nothing in this corpus escapes the filter list.** All 8 learned request classifiers train on filter-list labels, Duumviri included. The two directions that come closest — Duumviri's differential features and breakage detector {[shuang2025_duumviri]}, and taint tracking as an independent detector {[calzavara2026_syntactic]} — have each been done once. **A request classifier whose labels come from something other than a list is an open problem, and it is the one this page would most like solved.**
-  * **What Manifest V3 did to extension-based measurement.** Lukić and Papadopoulos {[lukic2026_mv3]} found no significant loss of blocking effectiveness under ''declarativeNetRequest'', but the 30,000-rule static cap is well under EasyList's 52,557 network rules and nobody has published which rules the shipped MV3 blockers drop, or what that omits from a measurement.
+  * **What Manifest V3 did to extension-based measurement.** Lukić and Papadopoulos {[lukic2026_mv3]} found no significant loss of blocking effectiveness under ''declarativeNetRequest'', but the 30,000-rule static cap is well under EasyList's network-rule count ([[Programming:Filter Lists#If your instrument is a browser extension: Manifest V3 caps the rules|current figure here]]) and nobody has published which rules the shipped MV3 blockers drop, or what that omits from a measurement.
   * **Venue coverage is itself an open problem for this topic.** AdVersa {[lim2026_adversa]} at TheWebConf 2026 and Intumwayase et al. {[intumwayase2025_continents]} at WPES 2025 are both squarely in scope and both invisible to the figures above. A reading list built only from the seven venues in this corpus will be incomplete for exactly the most recent work.
 </WRAP>