This is an old revision of the document!
Table of Contents
Provenance: Roadmap
Working log for roadmap. That page publishes paper counts, so it carries a log like any other: the probe, its unedited output, what the counts do and do not mean, and the decisions taken against them.
1. Where the counts come from
scripts/gap_probe_roadmap.mjs, committed with its output as scripts/gap_probe_roadmap-output.txt. It runs regular expressions over title + summary of every paper in the extraction and prints, per family, the number of matching papers, how many carry the web platform, and the 2020–2023 and 2024–2026 splits. Re-run it with a family name to print the matching titles, which is how every count on roadmap can be audited:
node scripts/gap_probe_roadmap.mjs # the table node scripts/gap_probe_roadmap.mjs llm_annotator
Output as of 2026-09-07, unedited:
family papers web 2020-2023 2024-2026 age_assurance 11 3 2 2 dsar_rights 9 7 1 6 ctv_streaming 16 1 5 5 llm_annotator 30 10 3 27 reproducibility 38 7 16 19 ai_crawlers 4 3 0 3 cdn_edge 48 23 15 8 browser_permissions 8 2 3 2 anti_adblock 3 3 1 1 payments_checkout 48 15 21 11 health_sensitive 16 4 4 4 webrtc_leak 6 4 3 3 sok_survey 44 6 14 21 crowdsourced_vantage 57 7 20 11
2. What these numbers are not
- They are candidate sets, not populations. A title-and-summary probe cannot see a paper that measured smart TVs without saying so in either field, so every count is a floor. A page written against one of these rows derives its own population from the structured fields and publishes that instead.
- They are not comparable across families.
cdn_edge(48) andpayments_checkout(48) are wide regular expressions that match incidental mentions;anti_adblock(3) is narrow and specific. A larger number is not a stronger case, which is why every disposition on roadmap names the reason rather than the rank. - The year split is not a trend. 2025 is thin and 2026 is provisional — see corpus. The split is here because a family that is entirely pre-2020 is a different proposition from one that is entirely post-2024, not because the slope means anything.
- Reading the titles changed two dispositions, and this is the reason the probe prints them.
age_assurancelooks like 11 papers on age gates and is mostly COPPA and children's-privacy work;ctv_streaminglooks like 16 papers with only one on the web platform and contains a coherent six-paper smart-TV tracking spine. Both rows on roadmap say so.
3. Decisions taken on 2026-09-07
- Ten queued pages, with ids fixed so that later work lands in the right place. Four ids were open questions in the backlog and were decided here:
privacy:policiesrather thanprivacy:privacy_policies(which sorts next to this site's own privacy_policy page);programming:internet_scanningrather thandesign:scanning(an instrument, matching Crawler);statistics:annotationrather thanstatistics:classifier_validationordesign:validation, because the page has to cover LLM annotation as well as validation;privacy:data_subject_rightsto avoid colliding with Requests, which is web requests. statistics:annotationabsorbs two backlog items — hoisting the label-validation section out of Website classification, and the LLM-as-annotator gap. They are one page: 30 candidate papers use an LLM to annotate, 27 of them in 2024–2026, and the corpus-wide 42.4% report-no-validation figure cannot live on a topic page without misstating that page's denominator.- Connected TV and Age assurance are queued despite being thin, as an explicit editorial decision: a page that states a method and a small, dated literature is more useful to a student than silence, and the thinness is itself reportable. The line was drawn at WebRTC leaks (6 candidates) — below it sit WebAssembly and every other thinly-published web technology, and admitting one admits all.
- Six proposals were assessed and not written, with counts, on roadmap. Three become sections on existing pages rather than pages.
3b. Decisions taken on 2026-09-08
- Ad archives was written, and its Queued row removed — required, not optional:
scripts/sitemap.mjsreports a written-but-still-queued row as a failure. A disposition row was added to Assessed instead, so the decision survives the removal. - The count that justified queuing it was wrong, and writing the page corrected it. The row read “69 papers”. That figure came from a full-text probe for /Ad Library|Ads? Archive API/i on Platforms, and the probe cannot separate Meta's Ad Library from an Android advertising SDK — the ordinary meaning of “ad library” in mobile-security writing. 44 of the 69 carry only the lowercase SDK sense, and the pattern also matches inside “uplo[ad librar]ies”, so one of the 69 is a paper about Node.js file uploads. The hand-audited figure is 11; precision of the old probe was 14.5%. Platforms has been corrected in the same sitting and the audit is on ad_archives.
- This is a warning about every count in the Queued table. roadmap already says the counts are candidate sets and floors, not populations. The ad-archives row shows the other failure direction: a candidate set can be a large over-count, by a factor of six here, when the probe term is a homonym. A queued row's count justifies looking, and nothing more.
- The queued page still deserved writing on the corrected number. Eleven papers is thin, but they are coherent, five of them measure the archives' own error rates, and the regulatory driver is live — which is what the row claimed.
4. The red-link gate
Red links are a defect on this wiki, and one drain item exists purely to clean up namespace-relative ones. roadmap is the single exception, and it is enforced rather than asserted: scripts/sitemap.mjs now checks that every promised-but-missing page on the whole wiki appears as the first link of a row in the Queued table, and that no Queued row's page already exists. It exits non-zero on either, after printing the full report.
Two consequences worth knowing before editing:
- The declaration is the first link in a row, deliberately. The second column links pages that do exist, and the Assessed table names pages that will never be written; counting every link in the table would license a red link anywhere on the page.
?do=sitemapis cached, so a page created minutes ago is absent from that listing. roadmap is therefore fetched directly and added before any page is scanned — without that, its declared red links are never seen and the gate passes vacuously. That is the same shape of silent pass this script produced on 2026-09-05, when an origin change made it report “Distinct internal link targets: 0” with exit 0.
5. Limitations
The queued list is what the corpus supports as of 2026-09-07 and it is not a schedule. Two rows are conditional in substance even though they read as commitments: statistics:annotation depends on the LLM-annotation slice surviving a hand audit of its 30 candidates, and Connected TV will need its population derived from platform fields rather than the probe, because the probe's web-platform column is exactly the wrong filter for it. The Assessed table's SSO/WebAuthn row is gated on an audit that has not been run: 46 candidates, unaudited, and the probe catches attacks on OAuth as well as deployment measurement.
