programming
This is an old revision of the document!
Table of Contents
Programming
This namespace is for the instrument — which browser renders the page, which protocol drives it, what the crawler does after page.goto, whether the profile survives, what file you write, and how you pin the image. It is not a library tutorial, and it is not the design choice of whether to crawl at all (that is Design, and Automated measurements). The publication corpus behind the corpus-backed pages here is seven venues (CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P, 2010–2026, 5,859 extracted papers). Only 1,120 of them ran a crawl. Each child names its own population.
A namespace page outlines the pages inside it rather than carrying its own content. 1) All 17 children below are written. 3 of them (CrUX, Similarweb, Multilingual support) are short instrument notes that have not been re-derived against the 2010–2026 extraction — treat them as API notes, not as evidence-backed siblings of Crawler.
Driving the browser
| Page | What a student needs it for |
|---|---|
| Crawler | Browser vs control channel. Selenium / Puppeteer / Playwright / CDP, then the specialised crawlers. |
| OpenWPM | Unbranded Firefox + privileged WebExtension. Which version, and which instruments were on. |
| Tracker Radar Collector | DuckDuckGo's Chromium/CDP collector, vs the Tracker Radar dataset / detector / entity map. |
| PageGraph | Brave page-execution recorder: which actor caused this request. |
| webXray | Domain-to-company ownership lists, and what remains of the tool. |
| Foxhound | Firefox fork that taints strings. When a taint report is a flow and when it is a vulnerability. |
| PanoptiChrome | V8 object taint in Chromium. When it is the instrument and when it is not. |
After the page loads
| Page | What a student needs it for |
|---|---|
| Interaction | What the crawler does after page.goto: depth, scroll, click. |
| Registration | Login, account creation, newsletter subscribe. |
| Stateful stateless | Whether the browser profile survives between visits, and what a reset actually resets. |
What you record, and how you pin it
| Page | What a student needs it for |
|---|---|
| Traffic files | HAR vs pcap vs proxy vs CDP — choosing where to tap. |
| Docker | Pinning a crawler image (digest, shared memory). |
Sampling-frame instruments
These are the APIs Website selection points at. They are not a second copy of “which list”.
| Page | What a student needs it for | On the wiki (2026-08-27) |
|---|---|---|
| Tranco | Tranco API and construction; permanent list ids. | written, corpus-backed |
| Cloudflare Radar | Radar Domain Rankings API; ordered head vs unordered buckets. | written, corpus-backed |
| CrUX | Chrome UX Report access (BigQuery). | written, stub (8,394 bytes) |
| Similarweb | Similarweb extension/API note. | written, stub (4,514 bytes) |
Other
| Page | What a student needs it for | On the wiki (2026-08-27) |
|---|---|---|
| Multilingual support | Language detection and translation in a crawl. | written, stub (8,012 bytes) |
Where this namespace stops
- Website selection / Sampling / Crawling location / Longitudinal — which list, how you draw, where from, what you pin across waves.
- Archives — reading someone else's recording; Traffic files is writing your own.
- Consent — the banner is a treatment. Interaction is how you click; consent is which click and what it means.
- Requests — classifying the requests the instrument captured.
- Artifacts — archiving the pin. Docker is how you pin; artifacts is where a 2026 Available badge will accept it.
Methodology and limitations of these figures
The 5,859 / 1,120 are paper counts from the 5,859-paper extraction (seven venues, 2010–2026). 2025–2026 venue-years are provisional — see corpus. The 17 / 3 and the stub byte counts are wiki-page counts as of 2026-08-27. Queries: programming. Joint sitting: design.
1)
contributing, “Namespace and page structure”.
You could leave a comment if you were logged in.
programming.1787840553.txt.gz · Last modified: by karel.kubicek.claude