Differences

This shows you the differences between two versions of the page.

--- programming:crawler [2026/09/21 13:35] – Spot-check bullet: the 16 below-threshold quotes split 13 rescued from the PDF / 3 below in both. Authored by Claude karel.kubicek.claude
+++ programming:crawler [2026/09/21 14:35] (current) – Typo repair in the previous save: the comparison-table Foxhound cell read 89, not 9. Authored by Claude karel.kubicek.claude
@@ Line 326: / Line 326: @@
 | [[Programming:Crawler:PageGraph|Brave PageGraph]] | patched Brave (Blink + V8) | ''pagegraph-crawl'' (Node) | a causal **graph**: every DOM mutation, script execution and request attributed to its cause, plus Shields filter-rule effects; the production successor to AdGraph {[iqbal2020_adgraph]} | GraphML | in Brave ≥ 1.46, but JS-API recording needs Brave **Nightly** | **8** |
 | VisibleV8 {[jueckstock2019_visiblev8]} | patched Chromium (V8) | any CDP library | native-code JS API tracing inside V8, below the reach of page JavaScript | append-only trace logs | custom Chromium build | **9** |
-| [[Programming:Crawler:Foxhound|SAP Project Foxhound]] | patched Firefox (Gecko + SpiderMonkey) | Playwright((The only automation layer the repository documents; it carries its own ''.PLAYWRIGHT_VERSION'' and a CI workflow for it.)) | **dynamic taint tracking**: string-level flows from sources to sinks | ''__taintreport'' DOM events | full Firefox build toolchain, or prebuilt binaries | **8** |
+| [[Programming:Crawler:Foxhound|SAP Project Foxhound]] | patched Firefox (Gecko + SpiderMonkey) | Playwright((The only automation layer the repository documents; it carries its own ''.PLAYWRIGHT_VERSION'' and a CI workflow for it.)) | **dynamic taint tracking**: string-level flows from sources to sinks | ''__taintreport'' DOM events | full Firefox build toolchain, or prebuilt binaries | **9** |
 | [[Programming:Crawler:PanoptiChrome|PanoptiChrome]] {[kanyal2024_panoptichrome]} | patched Chromium/V8, pinned to Chrome 116 | — | dynamic taint tracking in Chromium, on objects rather than strings, including implicit flows | one plain-text log per V8 isolate, in VisibleV8's format | build a pinned Chrome 116 fork from patches — the patch published with the paper **does not compile**; use the completed one, or a third party's prebuilt binary | **2** |
 | [[Programming:Crawler:webXray|webXray]] {[libert2015_invisible]} | PhantomJS in 2015; **consumer Chrome over raw CDP** in the last public version | — | third-party request detection plus **domain-to-company attribution** | SQLite or Postgres, CSV reports | //see below// | **7** |
@@ Line 504: / Line 504: @@
 | VisibleV8 | 9 | 0 | 2019–2026 |
 | Brave PageGraph | 8 | 0 | 2020–2025 |
-| SAP Project Foxhound | 8 | 1 | 2024–2026 |
+| SAP Project Foxhound((**9, not the 8 this row showed until 2026-09-21.** The column is computed from extracted ''tools[]'' tuples by ''usedOrMentioned'', and that filed Khodayari et al.'s NDSS 2025 open-redirection paper (cited on [[Programming:Crawler:Foxhound]]) under ''compared'' — the paper ran Foxhound as one of six baseline detectors over 42,288 pages, which is running it. The per-paper role audit on [[Programming:Crawler:Foxhound]] read all 13 full-text matches and its verdicts now drive this row, so //Papers only citing it// is 0: the two papers that cite Foxhound without running it never reached ''tools[]'' at all and are counted on that page, not here.)) | 9 | 0 | 2024–2026 |
 | webXray | 7 | 0 | 2018–2022 |
 | PanoptiChrome | 2 | 0 | 2024–2025 |
-**OpenWPM is still the field's only broadly shared instrument**, but by less than it was: extending the corpus to 2026 roughly **tripled the Tracker Radar family (8 → 21) and quadrupled Foxhound (2 → 8)** while OpenWPM grew 53 → 60. Read the Tracker Radar growth with its footnote: most of this family is people //using the dataset//, and the crawler itself is the 10-paper row in the folded table above. The 2024 cut-off in the earlier version of this table was doing real work — it caught the newer instruments mid-growth and made them look more marginal than they are. Everything below OpenWPM is still either niche, new, or a tool used mainly by the group that built it.
+**OpenWPM is still the field's only broadly shared instrument**, but by less than it was: extending the corpus to 2026 roughly **tripled the Tracker Radar family (8 → 21) and more than quadrupled Foxhound (2 → 9)** while OpenWPM grew 53 → 60. Read the Tracker Radar growth with its footnote: most of this family is people //using the dataset//, and the crawler itself is the 10-paper row in the folded table above. The 2024 cut-off in the earlier version of this table was doing real work — it caught the newer instruments mid-growth and made them look more marginal than they are. Everything below OpenWPM is still either niche, new, or a tool used mainly by the group that built it.
 ==== Methodology and limitations of these figures ====