User Tools

Site Tools


provenance:roadmap

This is an old revision of the document!


Provenance: Roadmap

Working log for roadmap. That page publishes paper counts, so it carries a log like any other: the probe, its unedited output, what the counts do and do not mean, and the decisions taken against them.

1. Where the counts come from

scripts/gap_probe_roadmap.mjs, committed with its output as scripts/gap_probe_roadmap-output.txt. It runs regular expressions over title + summary of every paper in the extraction and prints, per family, the number of matching papers, how many carry the web platform, and the 2020–2023 and 2024–2026 splits. Re-run it with a family name to print the matching titles, which is how every count on roadmap can be audited:

node scripts/gap_probe_roadmap.mjs             # the table
node scripts/gap_probe_roadmap.mjs llm_annotator

Output as of 2026-09-07, unedited:

family	papers	web	2020-2023	2024-2026
age_assurance	11	3	2	2
dsar_rights	9	7	1	6
ctv_streaming	16	1	5	5
llm_annotator	30	10	3	27
reproducibility	38	7	16	19
ai_crawlers	4	3	0	3
cdn_edge	48	23	15	8
browser_permissions	8	2	3	2
anti_adblock	3	3	1	1
payments_checkout	48	15	21	11
health_sensitive	16	4	4	4
webrtc_leak	6	4	3	3
sok_survey	44	6	14	21
crowdsourced_vantage	57	7	20	11

2. What these numbers are not

  • They are candidate sets, not populations. A title-and-summary probe cannot see a paper that measured smart TVs without saying so in either field, so every count is a floor. A page written against one of these rows derives its own population from the structured fields and publishes that instead.
  • They are not comparable across families. cdn_edge (48) and payments_checkout (48) are wide regular expressions that match incidental mentions; anti_adblock (3) is narrow and specific. A larger number is not a stronger case, which is why every disposition on roadmap names the reason rather than the rank.
  • The year split is not a trend. 2025 is thin and 2026 is provisional — see corpus. The split is here because a family that is entirely pre-2020 is a different proposition from one that is entirely post-2024, not because the slope means anything.
  • Reading the titles changed two dispositions, and this is the reason the probe prints them. age_assurance looks like 11 papers on age gates and is mostly COPPA and children's-privacy work; ctv_streaming looks like 16 papers with only one on the web platform and contains a coherent six-paper smart-TV tracking spine. Both rows on roadmap say so.

3. Decisions taken on 2026-09-07

  • Ten queued pages, with ids fixed so that later work lands in the right place. Four ids were open questions in the backlog and were decided here: privacy:policies rather than privacy:privacy_policies (which sorts next to this site's own privacy_policy page); programming:internet_scanning rather than design:scanning (an instrument, matching Crawler); statistics:annotation rather than statistics:classifier_validation or design:validation, because the page has to cover LLM annotation as well as validation; privacy:data_subject_rights to avoid colliding with Requests, which is web requests.
  • statistics:annotation absorbs two backlog items — hoisting the label-validation section out of Website classification, and the LLM-as-annotator gap. They are one page: 30 candidate papers use an LLM to annotate, 27 of them in 2024–2026, and the corpus-wide 42.4% report-no-validation figure cannot live on a topic page without misstating that page's denominator.
  • Connected TV and Age assurance are queued despite being thin, as an explicit editorial decision: a page that states a method and a small, dated literature is more useful to a student than silence, and the thinness is itself reportable. The line was drawn at WebRTC leaks (6 candidates) — below it sit WebAssembly and every other thinly-published web technology, and admitting one admits all.
  • Six proposals were assessed and not written, with counts, on roadmap. Three become sections on existing pages rather than pages.

3b. Decisions taken on 2026-09-08

  • Ad archives was written, and its Queued row removed — required, not optional: scripts/sitemap.mjs reports a written-but-still-queued row as a failure. A disposition row was added to Assessed instead, so the decision survives the removal.
  • The count that justified queuing it was wrong, and writing the page corrected it. The row read “69 papers”. That figure came from a full-text probe for /Ad Library|Ads? Archive API/i on Platforms, and the probe cannot separate Meta's Ad Library from an Android advertising SDK — the ordinary meaning of “ad library” in mobile-security writing. 44 of the 69 carry only the lowercase SDK sense, and the pattern also matches inside “uplo[ad librar]ies”, so one of the 69 is a paper about Node.js file uploads. The hand-audited figure is 11; precision of the old probe was 14.5%. Platforms has been corrected in the same sitting and the audit is on ad_archives.
  • This is a warning about every count in the Queued table. roadmap already says the counts are candidate sets and floors, not populations. The ad-archives row shows the other failure direction: a candidate set can be a large over-count, by a factor of six here, when the probe term is a homonym. A queued row's count justifies looking, and nothing more.
  • The queued page still deserved writing on the corrected number. Eleven papers is thin, but they are coherent, five of them measure the archives' own error rates, and the regulatory driver is live — which is what the row claimed.

Red links are a defect on this wiki, and one drain item exists purely to clean up namespace-relative ones. roadmap is the single exception, and it is enforced rather than asserted: scripts/sitemap.mjs now checks that every promised-but-missing page on the whole wiki appears as the first link of a row in the Queued table, and that no Queued row's page already exists. It exits non-zero on either, after printing the full report.

Two consequences worth knowing before editing:

  • The declaration is the first link in a row, deliberately. The second column links pages that do exist, and the Assessed table names pages that will never be written; counting every link in the table would license a red link anywhere on the page.
  • ?do=sitemap is cached, so a page created minutes ago is absent from that listing. roadmap is therefore fetched directly and added before any page is scanned — without that, its declared red links are never seen and the gate passes vacuously. That is the same shape of silent pass this script produced on 2026-09-05, when an origin change made it report “Distinct internal link targets: 0” with exit 0.

5. Limitations

The queued list is what the corpus supports as of 2026-09-07 and it is not a schedule. Two rows are conditional in substance even though they read as commitments: statistics:annotation depends on the LLM-annotation slice surviving a hand audit of its 30 candidates, and Connected TV will need its population derived from platform fields rather than the probe, because the probe's web-platform column is exactly the wrong filter for it. The Assessed table's SSO/WebAuthn row is gated on an audit that has not been run: 46 candidates, unaudited, and the probe catches attacks on OAuth as well as deployment measurement.

You could leave a comment if you were logged in.
provenance/roadmap.1788867354.txt.gz · Last modified: by karel.kubicek.claude