| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| programming:crawler:openwpm [2026/08/14 10:33] – [Open Questions] formatting admin | programming:crawler:openwpm [2026/08/21 08:32] (current) – [Open Questions] karelkubicek |
|---|
| OpenWPM {[englehardt2016online]} is the closest thing web privacy measurement has to a standard instrument: a Python platform that drives an **unbranded Firefox** through **Selenium**, and records what the browser did through a **privileged WebExtension** rather than through the automation protocol. It is the most widely shared specialised crawler in our corpus: **60 papers used or extended it**, against 21 each for the next two, on the mention-matching count over all 5,859 papers in [[Programming:Crawler#Which specialised crawlers actually get used|the comparison page]]. 59 of those 60 are inside the 1,120 papers that ran a crawl; the exception re-analysed someone else's OpenWPM data. | OpenWPM {[englehardt2016online]} is the closest thing web privacy measurement has to a standard instrument: a Python platform that drives an **unbranded Firefox** through **Selenium**, and records what the browser did through a **privileged WebExtension** rather than through the automation protocol. It is the most widely shared specialised crawler in our corpus: **60 papers used or extended it**, against 21 each for the next two, on the mention-matching count over all 5,859 papers in [[Programming:Crawler#Which specialised crawlers actually get used|the comparison page]]. 59 of those 60 are inside the 1,120 papers that ran a crawl; the exception re-analysed someone else's OpenWPM data. |
| |
| This page is about running it and reading papers that ran it: which instrumentation surfaces exist and which are silent by default, what stateful and stateless mean //in OpenWPM's implementation//, what the version number commits you to, and what its maintenance looks like today. For the choice between OpenWPM and Playwright, Tracker Radar Collector or a patched browser, see [[Programming:Crawler]]. For the design question of whether to keep browser state at all, see [[Programming:Stateful stateless]] — currently a notes stub, so treat it as a reading list rather than an answer. | This page is about running it and reading papers that ran it: which instrumentation surfaces exist and which are silent by default, what stateful and stateless mean //in OpenWPM's implementation//, what the version number commits you to, and what its maintenance looks like today. For the choice between OpenWPM and Playwright, Tracker Radar Collector or a patched browser, see [[Programming:Crawler]]. For the design question of whether to keep browser state at all, see [[Programming:Stateful stateless]], which also covers what a "reset" actually resets and why the browser's own partitioning defaults now decide part of the answer. |
| |
| <WRAP important> | <WRAP important> |
| ===== Open Questions ===== | ===== Open Questions ===== |
| |
| | <WRAP todo> |
| * Nobody has published what OpenWPM's default ''collection_fingerprinting'' misses on a modern fingerprinting corpus. The list has not changed since 2020 and the surface has; measuring the gap is a small paper. | * Nobody has published what OpenWPM's default ''collection_fingerprinting'' misses on a modern fingerprinting corpus. The list has not changed since 2020 and the surface has; measuring the gap is a small paper. |
| * How much the unbranded Firefox build differs from a stock Firefox of the same version, in fingerprint and in behaviour, is unmeasured — the same question [[Programming:Crawler]] raises for Playwright's patched builds. | * How much the unbranded Firefox build differs from a stock Firefox of the same version, in fingerprint and in behaviour, is unmeasured — the same question [[Programming:Crawler]] raises for Playwright's patched builds. |
| * Whether the per-browser cookie-jar partition in a stateful multi-browser crawl changes published results, and by how much. Every stateful OpenWPM crawl with ''num_browsers > 1'' has this property; no paper we found reports ''num_browsers'' alongside a stateful claim. | * Whether the per-browser cookie-jar partition in a stateful multi-browser crawl changes published results, and by how much. Every stateful OpenWPM crawl with ''num_browsers > 1'' has this property; no paper we found reports ''num_browsers'' alongside a stateful claim. |
| * Where ARM support would come from: Mozilla's ''add-on-devel'' build matrix, or an OpenWPM-side path that does not need an unbranded build. | * Where ARM support would come from: Mozilla's ''add-on-devel'' build matrix, or an OpenWPM-side path that does not need an unbranded build. |
| | </WRAP> |
| |
| ====== References ====== | ====== References ====== |
