Differences

This shows you the differences between two versions of the page.

--- privacy:javascript [2026/08/12 09:33] – Refresh all corpus figures for the extended 2010-2026 extraction (4,322 -> 5,859 papers; JS-analysis population 160 -> 206). Report bucket bug fixed (2022+ was open-ended) and a 2025-2026 provisional row added; hardcoded corpus-wide legal rate replaced. N karel.kubicek.claude
+++ privacy:javascript [2026/10/08 20:49] (current) – Finish PII leakage collateral: back-links and provenance; Authored by Claude karel.kubicek.claude
@@ Line 109: / Line 109: @@
 detection tuples did not fold into any family and are printed by the report script rather than dropped — a slightly larger share of a larger population than the 45 of the earlier corpus, so this fold has aged well where others have not.
-The shape to notice: **the privacy reader's own family is the largest but is barely a quarter of the field.** If you search these venues for "JavaScript detection" you will mostly get client-side vulnerability and malware papers, which use the same parsers and the same instrumented browsers on a different question. They are worth reading for method and misleading as related work.
+The shape to notice: **the privacy reader's own two families — tracking/advertising classification and fingerprinting-script detection — lead the ranking, and together they are barely a quarter of the field (53 of 206).** If you search these venues for "JavaScript detection" you will mostly get client-side vulnerability and malware papers, which use the same parsers and the same instrumented browsers on a different question. They are worth reading for method and misleading as related work.
 ==== Anchor papers to read first ====
@@ Line 147: / Line 147: @@
 ==== What the corpus cannot tell you, and what we found instead ====
-**LLM-based classification of web scripts is, as of 2026-08-06, essentially absent from the peer-reviewed literature.** A targeted search across PETS 2025/2026, USENIX Security 2025, NDSS 2025/2026, IMC 2025, TheWebConf 2025/2026, CCS 2025 and arXiv found no paper that classifies web scripts as trackers with a language model, or that uses one to summarise a script's privacy-relevant behaviour. The nearest work is adjacent rather than on-point: LLM-aided **deobfuscation** feeding a graph classifier for JavaScript //malware//,((//Breaking Obfuscation: Cluster-Aware Graph with LLM-Aided Recovery for Malicious JavaScript Detection//, [[https://arxiv.org/abs/2507.22447|arXiv:2507.22447]], 2025.)) LLM screening of malicious npm packages, and ''humanify'', which uses a model only to //suggest identifier names// during de-minification.(([[https://github.com/jehna/humanify|github.com/jehna/humanify]], v3.1.1, checked 2026-08-06. The AST rewrite is done by ''oxc''; the model only proposes names.))
+**LLM-based classification of web scripts is, as of 2026-09-03, still absent from the peer-reviewed literature in these seven venues — and this is now a measured zero rather than the result of a keyword search.** ''classification.method == "llm"'' fires on **177 of the 5,859 corpus papers**, but on **0 of the 44 papers that classify a ''javascript'' target and 0 of the 31 that classify a ''fingerprinting-script'' target**. A targeted search outside the corpus — PETS 2025/2026, USENIX Security 2025/2026, NDSS 2025/2026, IMC 2025, TheWebConf 2025/2026, CCS 2025 and arXiv — found no paper that classifies web scripts as trackers with a language model, or that uses one to summarise a script's privacy-relevant behaviour.
-<wrap todo>Treat this as an opportunity, not a settled answer. If you are planning an LLM-based script classifier, you are not late — but you also have no baseline to cite, so budget for building one, and for the reviewer question about cost, reproducibility and prompt/version drift that this page cannot yet answer for you.</wrap>
+**Read that against the corpus-wide curve before concluding the field is not interested.** LLM classification went from 2 papers in 2023 to 71 in the partial 2026 (17.1% of that year), at all seven venues, so the zero above is specific to this target rather than a statement about the method. The per-target table on [[design:website_classification#Where LLMs actually appear]] gives the full ranking; ''javascript'' and ''fingerprinting-script'' are **zero rows** in it, alongside ''malware'', ''sdk-or-library'' and ''website-popularity''. The lowest //non-zero// rows are ''ip-address'' (1 of 295) and ''web-request'' (1 of 258).
+**The nearest peer-reviewed work is one layer down, at the request.** TGNN {[xiong2026_tgnn]} (TheWebConf 2026) uses Qwen3 to label HTTP request/response quadruples as tracking or not, and reports an annotation F1 of **98.17%** against expert labels where filter lists reach **55.14%** on the same ground truth.((The paper reports this figure twice as **98.17%** — in §4.1.3 //LLM-based Labeling// ("𝑀𝜆 performs well (𝐹1-score of 98.17%)") and again beside Figure 5 — and once as **98.19%**, in the contributions paragraph of its Introduction. Its abstract states no F1 for the annotation component at all. The discrepancy is the paper's, not ours; the body figure is quoted here and on [[privacy:requests]], so the two pages agree. The 55.14% filter-list comparison appears only in the Introduction. Located in ''paper.cols.txt'' on 2026-09-03; note that this file contains NUL bytes, so shell ''grep'' reports nothing without ''-a''.)) It is not a script classifier — it explicitly contrasts its approach with prior methods that do "single-domain analysis-such as string matching against domain lists or scrutinizing JavaScript execution within a page" — but it is the closest thing to a citable precedent for using a model to //manufacture tracker labels//, which is this page's weakest link. [[privacy:requests]] treats it in full. Beyond it the adjacent work is not peer-reviewed and not on-point: LLM-aided **deobfuscation** feeding a graph classifier for JavaScript //malware//,((//Breaking Obfuscation: Cluster-Aware Graph with LLM-Aided Recovery for Malicious JavaScript Detection//, [[https://arxiv.org/abs/2507.22447|arXiv:2507.22447]], 2025.)) LLM screening of malicious npm packages, and ''humanify'', which uses a model only to //suggest identifier names// during de-minification.(([[https://github.com/jehna/humanify|github.com/jehna/humanify]], v3.1.1, checked 2026-08-06. The AST rewrite is done by ''oxc''; the model only proposes names.))
+<WRAP todo>Treat this as an opportunity, not a settled answer. If you are planning an LLM-based script classifier, you are not late — but you also have no baseline to cite, so budget for building one, and for the reviewer question about cost, reproducibility and prompt/version drift that this page cannot yet answer for you. Two things you can borrow rather than invent: TGNN's annotation-versus-filter-list comparison {[xiong2026_tgnn]} is the experimental design a reviewer will expect, and the model-reporting figures on [[design:website_classification#And almost nobody names a model you could resolve]] show that only **19.4%** of the 175 corpus papers that actually use an LLM name a model resolvable to an actual artefact — so naming yours to the checkpoint is cheap novelty.</WRAP>
 ==== Two 2025 results that change how you design a crawl ====
@@ Line 175: / Line 179: @@
 | **llm** | **2** | **1.0%** |
-The ''llm'' row is new: on the 4,322-paper corpus this enum never fired for a JavaScript-classification task at all. Two papers is not a trend, and it does not contradict the finding below that no peer-reviewed paper yet classifies web scripts //as trackers// with a language model.
+**The ''llm'' row does not mean what it looks like it means, and it is worth being precise because two other pages depend on the same field.** The enum is per //paper//, not per JavaScript task: these are two papers in this page's population that used an LLM for //some// classification, and neither classified a script. {[chen2025_semantics]} (TheWebConf 2025) fine-tunes GPT-3.5 to label **cookie** purposes, and PhishLang (NDSS 2026) queried GPT-4 once to pick which **HTML tags** matter for phishing detection. Restricted to script targets the count is **zero** — 0 of the 44 papers classifying ''javascript'' and 0 of the 31 classifying ''fingerprinting-script'' — which is the figure quoted above and the one to cite. On the 4,322-paper corpus this row was empty for either reading.
 And what they treat as truth. These 198 papers produce **351 distinct free-text ground-truth strings**, folded here into families; 80 tuples did not fold and are printed by the report script.
@@ Line 231: / Line 235: @@
 | Esprima | 21 | 10.2% |
 | OpenWPM | 19 | 9.2% |
+| Project Foxhound (taint tracking)((**9, not the 8 published here until 2026-09-21.** The column counts ''tools[]'' tuples marked ''used'' or ''produced'', and Khodayari et al.'s NDSS 2025 open-redirection paper was extracted as ''compared'' — it ran Foxhound as one of six baseline detectors over 42,288 pages, which is running it. The per-paper role audit on [[Programming:Crawler:Foxhound]] reads all 13 full-text matches in the corpus and that verdict now feeds this row; it is the only hand verdict in this table, and it cannot add a paper to the 206, which this one was already in on its detection and classification signals.)) | 9 | 4.4% |
 | PageGraph | 8 | 3.9% |
-| Project Foxhound (taint tracking) | 8 | 3.9% |
 | VisibleV8 | 8 | 3.9% |
 | js-beautify | 7 | 3.4% |
@@ Line 242: / Line 246: @@
 | Acorn, AdGraph, Emscripten, JSgraph, JSNice, Khaleesi, Rhino, SpiderMonkey, TAJS, UglifyJS, unnamed deobfuscator | 1 each | 0.5% |
-Two rows moved enough to matter. **Esprima has overtaken OpenWPM** as the most-named tool, and **Project Foxhound went from 2 papers to 8 and PageGraph from 4 to 8** — the taint-tracking and page-graph instruments the //Methods// table above calls current are the ones the 2025–2026 papers actually picked up. That is the rare case where the corpus confirms a currency judgement instead of only dating it.
+Two rows moved enough to matter. **Esprima has overtaken OpenWPM** as the most-named tool, and **Project Foxhound went from 2 papers to 9 and PageGraph from 4 to 8** — the taint-tracking and page-graph instruments the //Methods// table above calls current are the ones the 2025–2026 papers actually picked up. That is the rare case where the corpus confirms a currency judgement instead of only dating it.
-**120 of 206 papers (58.3%) name no JavaScript-analysis tool at all** — they wrote their own parser, regexes or instrumentation and did not name it. That is the field's reproducibility problem in one number.
+**119 of 206 papers (57.8%) name no JavaScript-analysis tool at all** — they wrote their own parser, regexes or instrumentation and did not name it. That is the field's reproducibility problem in one number.
 ==== Maintenance status, because half of these are frozen ====
@@ Line 339: / Line 343: @@
 // JavaScript a page runs". Neither of the obvious schema handles answers it:
 //
-//   * `tools[].category == "program-analysis"` fires on 959 papers (used or
+//   * `tools[].category == "program-analysis"` fires on 1,385 papers (used or
 //     produced), but that category is dominated by binary, Android and
-//     smart-contract analysis (LLVM 66, Soot 61, IDA Pro 36, FlowDroid 34,
+//     smart-contract analysis (LLVM 101, Soot 81, IDA Pro 62, FlowDroid 48,
-//     angr 28). Esprima, the highest-ranked JavaScript parser, is 15.
+//     angr 39). Esprima, the highest-ranked JavaScript parser, is 23.
-//   * `studyTypes` includes `code-or-binary-analysis` on 1,063 papers, same
+//   * `studyTypes` includes `code-or-binary-analysis` on 1,484 papers, same
 //     problem, and it is the least reproducible field in the schema (57%).
 //
@@ Line 463: / Line 467: @@
 // The population itself, with the reasons recorded.
 // ---------------------------------------------------------------------------
+// `usedOrMentioned == "compared"` is not always "did not run it". A paper that
+// benchmarks its own detector against a prior tool normally RUNS that tool, and
+// the extraction files that as `compared`, which drops it out of a "papers
+// using this instrument" count. Deciding that in general needs a per-paper read,
+// so it is NOT done in general: this map holds only the papers where a
+// published audit has read the sentence and recorded the verdict, and it names
+// the audit. Currently one entry, from the Foxhound role audit behind
+// [[programming:crawler:foxhound]] (scripts/fh_fold.mjs ROLE, 2026-08-17), whose
+// `baseline` role means "ran it as a comparison point".
+//
+// It adds to `tools` ONLY — not to `strictTools`, and so not to `reasons` and
+// not to the page's population rule. A hand entry must not be able to pull a
+// paper into the 206; this paper is already in it on its detection and
+// classification signals.
+export const TOOL_ROLE_OVERRIDE = new Map([
+  [
+    'NDSS/2025/do-not-follow-the-white-rabbit-challenging-the-myth-of-harmless-open-redirection',
+    {
+      tool: 'Foxhound',
+      audit: 'programming:crawler:foxhound — fh_fold.mjs ROLE, role "baseline"',
+      quote:
+        'We considered the following state-of-the-art detectors as baselines: dynamic taint-tracking [24, 36, 45] using Foxhound [44, 45]',
+      what: 'ran Foxhound as one of six baseline detectors over 42,288 pages',
+    },
+  ],
+]);
 export function jsSignals(p) {
   const reasons = [];
@@ Line 475: / Line 506: @@
     const strict = foldJsToolStrict(t.name);
     if (strict) strictTools.add(strict);
+  }
+  const override = TOOL_ROLE_OVERRIDE.get(paperKey(p));
+  if (override) {
+    // Fail loudly if the tuple the override is about has disappeared or has
+    // been re-extracted as used/produced: a hand entry that fires on nothing is
+    // a hand entry that hides a change in the data.
+    const t = p.tools.find((x) => foldJsTool(x.name) === foldJsTool(override.tool));
+    if (!t)
+      throw new Error(
+        `TOOL_ROLE_OVERRIDE for ${paperKey(p)} names ${override.tool}, which this paper's tools[] no longer contains`
+      );
+    if (t.usedOrMentioned === 'used' || t.usedOrMentioned === 'produced')
+      throw new Error(
+        `TOOL_ROLE_OVERRIDE for ${paperKey(p)} is now redundant: ${override.tool} is extracted as "${t.usedOrMentioned}". Remove the entry.`
+      );
+    tools.add(foldJsTool(override.tool));
   }
   if (strictTools.size) reasons.push('tool');
@@ Line 599: / Line 646: @@
 ==== Methodology and limitations of these figures ====
-  * **Seven venues only.** CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P, 2010–2026, with 2025 and 2026 provisional. **EuroS&P, ACSAC, RAID, AsiaCCS, CHI and SOUPS are absent entirely** — for this topic ACSAC and EuroS&P are a real hole, since a good deal of web-script security work lands there. Every claim here is a claim about those seven venues.
+  * **Seven venues only**, 2010–2026, with 2025 and 2026 provisional. Which venues, which years, what each stage of the selection funnel costs and which venue-years are empty are on [[literature:corpus]] and are not restated here. For //this// topic ACSAC and EuroS&P are a real hole, since a good deal of web-script security work lands there. Every claim here is a claim about those seven venues.
   * **206 is a floor, and it has a false-positive tail.** Papers whose extraction never names a script as the object of detection are missing; conversely a handful of papers in the population (an e-voting client audit, a router-attack paper, a PHP injection-sink study) analyse JavaScript incidentally. The report script prints the full list so you can judge.
   * **Not every field can carry a percentage.** ''crawlConfig.*'', ''legal.law'' and ''platforms'' reproduce to within a few points on a repeat extraction and carry the figures here. ''classification.method'' agrees on only **58%** of papers between two runs of the same schema over the same text, so its table above is a **rough share, not a precise figure** — a repeat extraction moves those rows. ''detection.phenomenon'', ''classification.resourceName'' and ''groundTruthSource'' agree on roughly 20% of exact strings, which is what the folding is for and why the family and ground-truth tables print their residue.
   * **Silence is not absence.** "Does not state whether it ran headless" means the paper did not say. These are reporting figures, not practice figures.
   * **Every quoted figure was checked against the paper's own text.** The prevalence values in the extraction are model summaries, so each number reproduced on this page was re-located in ''paper.cols.txt'' after whitespace normalisation. The dataset's own "0.9% of quotes cannot be located" figure was measured on the earlier 4,322-paper run and has not been re-measured.
+  * **Every query behind this section, the report script and its unedited output** are on [[provenance:privacy:javascript]]; corpus-level caveats are on [[literature:corpus]].
 ===== Open Questions =====
-  * <wrap todo>**No public, hand-labelled corpus of tracking scripts exists.** Every current method builds its own labels from filter lists plus manual inspection, which is why cross-paper comparison is impossible. A shared benchmark would do for this field what EasyList did for request blocking.</wrap>
+<WRAP todo>
-  * <wrap todo>**LLM-based script classification is unmeasured.** No peer-reviewed paper found as of 2026-08-06. The obvious study — LLM against WebGraph, AdFlush and NoT.js on a fixed script corpus, reporting cost and version drift as well as F1 — has no baseline yet.</wrap>
+  * **No public, hand-labelled corpus of tracking scripts exists.** Every current method builds its own labels from filter lists plus manual inspection, which is why cross-paper comparison is impossible. A shared benchmark would do for this field what EasyList did for request blocking.
-  * <wrap todo>**Nobody has measured how much a headless or containerised crawler under-counts //script// classification specifically.** {[jueckstock2021_realistic]} and {[annamalai2024_fpfed]} show the gap exists for API traces and fingerprinting scripts; its size for tracking-script prevalence at scale is unknown.</wrap>
+  * **LLM-based script classification is unmeasured.** Zero of the 44 corpus papers that classify a ''javascript'' target and zero of the 31 that classify a ''fingerprinting-script'' target use one, re-derived 2026-09-03, and no peer-reviewed paper outside the corpus was found either. The obvious study — LLM against WebGraph, AdFlush and NoT.js on a fixed script corpus, reporting cost and version drift as well as F1 — still has no baseline. The nearest template is TGNN's request-level annotation experiment {[xiong2026_tgnn]}, which beat filter lists 98.17% to 55.14% F1 on expert-labelled ground truth; the same comparison at script granularity has not been run.
-  * <wrap todo>**Function-granularity blocking has no successor paper.** NoT.js {[amjad2024_notjs]} and ByteDefender {[bahrami2025_bytedefender]} both stop at detection plus surrogate generation; nobody has measured what happens when either is deployed to real users at scale, or whether trackers adapt.</wrap>
+  * **Nobody has measured how much a headless or containerised crawler under-counts //script// classification specifically.** {[jueckstock2021_realistic]} and {[annamalai2024_fpfed]} show the gap exists for API traces and fingerprinting scripts; its size for tracking-script prevalence at scale is unknown.
-  * <wrap todo>**Cross-platform divergence is a confound in every older result.** If 20.6% of scripts execute differently by platform {[zafar2025_samescript]}, every desktop-only prevalence figure in this page's tables is a measurement of the desktop path only. Re-running any of them on mobile is a well-defined study.</wrap>
+  * **Function-granularity blocking has no successor paper.** NoT.js {[amjad2024_notjs]} and ByteDefender {[bahrami2025_bytedefender]} both stop at detection plus surrogate generation; nobody has measured what happens when either is deployed to real users at scale, or whether trackers adapt.
+  * **Cross-platform divergence is a confound in every older result.** If 20.6% of scripts execute differently by platform {[zafar2025_samescript]}, every desktop-only prevalence figure in this page's tables is a measurement of the desktop path only. Re-running any of them on mobile is a well-defined study.
+</WRAP>
 ===== Related Pages =====
   * [[Privacy:Requests]] — classifying at the request/URL layer, where filter lists live, and why they cap this page's ground truth.
+  * [[Privacy:PII leakage]] — what scripts that read form fields and the DOM are found sending: session replay, pixels' advanced matching, configuration versus transmission.
   * [[Privacy:Cookies]] — what the scripts write; the provenance argument (a cookie set by a blocked resource) is the same idea one layer down.
-  * [[Privacy:Fingerprinting]] — 42.4% of browser-fingerprinting papers are really detecting //scripts//, so that page and this one share a method.
+  * [[Privacy:Fingerprinting]] — 39.8% of browser-fingerprinting papers are really detecting //scripts//, so that page and this one share a method.
-  * [[Programming:Crawler]] — the instrumentation this page assumes you already have, compared in detail. Its per-tool pages ([[Programming:Crawler:OpenWPM]], [[Programming:Crawler:PageGraph]], [[Programming:Crawler:Foxhound]]) are promised but not yet written.
+  * [[Programming:Crawler]] — the instrumentation this page assumes you already have, compared in detail. Its per-tool pages [[Programming:Crawler:OpenWPM]] and [[Programming:Crawler:PageGraph]] are written; [[Programming:Crawler:Foxhound]] is still promised.
-  * [[Programming:Stateful stateless]] — only 28.6% of these papers state it, and a stateless crawl sees first-visit script behaviour only.
+  * [[Programming:Stateful stateless]] — only 27.6% of these papers state it, and a stateless crawl sees first-visit script behaviour only.
   * [[Design:Website classification]] — where script classification sits in the wider taxonomy.
   * [[Design:Crawling location]] — the vantage-point half of the "the site served you different code" problem.
+  * [[Security:Malicious web content]] — when the script is malicious: miners, exploit pages, skimmers and compromised third-party scripts, and why lists miss them.
   * [[Practices:Ethics]] — forced execution, deobfuscation and stealth crawling all override a site operator's expressed preferences.