| Both sides previous revisionPrevious revision | |
| programming:stateful_stateless [2026/08/21 08:34] – [Open Questions] karelkubicek | programming:stateful_stateless [2026/08/31 19:48] (current) – Correct the storageState() sentence: measured on Playwright 1.62.1, it does not carry IndexedDB unless {indexedDB:true} is passed, and never carries sessionStorage, Cache Storage or service-worker registrations. Link Privacy:Browser storage for the stores karel.kubicek.claude |
|---|
| > once some tracking has happened, it is hard to start from a truly clean profile | > once some tracking has happened, it is hard to start from a truly clean profile |
| |
| The complete list of things you should be able to say you reset, or say you did not: cookies (including partitioned ones), ''localStorage'' and ''sessionStorage'', IndexedDB, Cache Storage and service workers, the HTTP disk cache, the favicon cache, HSTS and TLS session state, DNS cache, permission grants, and the extension state of anything you installed. | The complete list of things you should be able to say you reset, or say you did not: cookies (including partitioned ones), ''localStorage'' and ''sessionStorage'', IndexedDB, Cache Storage and service workers, the HTTP disk cache, the favicon cache, HSTS and TLS session state, DNS cache, permission grants, and the extension state of anything you installed. What each of those stores is as a //measurement target// — whether your instrument can read it at all, whether it is partitioned, and what a per-origin reset looks like — is [[Privacy:Browser storage]]. |
| |
| ===== What each design can and cannot measure ===== | ===== What each design can and cannot measure ===== |
| ==== Stateful ==== | ==== Stateful ==== |
| |
| * **Playwright:** ''chromium.launchPersistentContext(userDataDir)'' and reuse ''userDataDir''. Everything persists, including the cache. To carry a profile //deliberately and legibly// instead, use ''storageState()'' — it serialises cookies, ''localStorage'' and IndexedDB to JSON you can commit as an artefact, which makes the seed reproducible in a way a binary profile directory is not. It does not carry the HTTP cache. | * **Playwright:** ''chromium.launchPersistentContext(userDataDir)'' and reuse ''userDataDir''. Everything persists, including the cache. To carry a profile //deliberately and legibly// instead, use ''storageState()'' — it serialises cookies and ''localStorage'' to JSON you can commit as an artefact, which makes the seed reproducible in a way a binary profile directory is not. **IndexedDB is behind an option that is off by default** (''storageState({ indexedDB: true })''), and ''sessionStorage'', Cache Storage and service-worker registrations are not in it at all; measured, on Playwright 1.62.1, in [[Privacy:Browser storage#What each capture method actually returns]]. It does not carry the HTTP cache. |
| * **Puppeteer:** ''puppeteer.launch({ userDataDir })'' and reuse the directory — the same mechanism as Playwright's persistent context, and worth knowing because Puppeteer is the more used of the two in this corpus (76 crawling papers against Playwright's 34). Puppeteer has no ''storageState'' equivalent, so a legible seed means either shipping the profile directory or writing your own cookie/storage dump. | * **Puppeteer:** ''puppeteer.launch({ userDataDir })'' and reuse the directory — the same mechanism as Playwright's persistent context, and worth knowing because Puppeteer is the more used of the two in this corpus (76 crawling papers against Playwright's 34). Puppeteer has no ''storageState'' equivalent, so a legible seed means either shipping the profile directory or writing your own cookie/storage dump. |
| * **OpenWPM:** stateful is the **default**, and stateless is per-command-sequence: ''CommandSequence(url, reset=True)'', documented as "True if browser should clear state and restart after sequence".((OpenWPM ''openwpm/command_sequence.py'' at ''master'' commit ''b9dd4c3a'', checked 2026-08-19.)) There is no global switch, which is why papers describe this in prose and reviewers cannot check it. Watch ''num_browsers'': with ''N'' browsers your "stateful crawl" is ''N'' cookie jars. Details on [[Programming:Crawler:OpenWPM#Stateful and stateless in OpenWPM]]. | * **OpenWPM:** stateful is the **default**, and stateless is per-command-sequence: ''CommandSequence(url, reset=True)'', documented as "True if browser should clear state and restart after sequence".((OpenWPM ''openwpm/command_sequence.py'' at ''master'' commit ''b9dd4c3a'', checked 2026-08-19.)) There is no global switch, which is why papers describe this in prose and reviewers cannot check it. Watch ''num_browsers'': with ''N'' browsers your "stateful crawl" is ''N'' cookie jars. Details on [[Programming:Crawler:OpenWPM#Stateful and stateless in OpenWPM]]. |
