| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| programming:stateful_stateless [2026/08/19 11:46] – Review round 3 (generic): fix the false claim that a stateless crawl cannot see cookie syncing (it sees first contact and over-triggers it), replace an uncomputed 'only flat field' superlative with a computed per-field trend table, rename the published co karel.kubicek.claude | programming:stateful_stateless [2026/08/31 19:48] (current) – Correct the storageState() sentence: measured on Playwright 1.62.1, it does not carry IndexedDB unless {indexedDB:true} is passed, and never carries sessionStorage, Cache Storage or service-worker registrations. Link Privacy:Browser storage for the stores karel.kubicek.claude |
|---|
| Playwright 1.62.1, Chromium 151.0.7922.34, Linux. Reproduce with [[#The code|the script below]]. | Playwright 1.62.1, Chromium 151.0.7922.34, Linux. Reproduce with [[#The code|the script below]]. |
| |
| Read off the three rows in bold. **Clearing cookies clears cookies.** It does not clear ''localStorage'', and it does not clear the HTTP cache — in the ''clearCookies()'' rows the cached subresource was never re-requested from the origin, so a request-counting measurement silently loses it on every visit after the first. Playwright's own documentation is accurate and easy to misread: ''clearCookies'' "removes cookies from context",((Playwright API reference, ''BrowserContext.clearCookies'' and ''BrowserContext.storageState'', checked 2026-08-19.)) and ''storageState'' returns "current cookies, local storage snapshot, IndexedDB snapshot and virtual WebAuthn credentials" — note what is missing from that list, and note that the ''storageState'' row above is the only one where cookies came back while the cache did not. | Read off the three rows in bold. **Clearing cookies clears cookies.** It does not clear ''localStorage'', and it does not clear the HTTP cache — in the ''clearCookies()'' rows the cached subresource was never re-requested from the origin, so a request-counting measurement silently loses it on every visit after the first. Playwright's own documentation is accurate and easy to misread: ''clearCookies'' "Removes cookies from context",((Playwright API reference, ''BrowserContext.clearCookies'' and ''BrowserContext.storageState'', checked 2026-08-19.)) and ''storageState'' returns "current cookies, local storage snapshot, IndexedDB snapshot and virtual WebAuthn credentials" — note what is missing from that list, and note that the ''storageState'' row above is the only one where cookies came back while the cache did not. |
| |
| The cache matters because it is a tracking channel in its own right, not just a performance detail. Solomos et al. showed that Chrome's **favicon cache** is a separate store that browser "clear browsing data" controls do not touch and that persists into incognito {[solomos2021_tales]}; ETag- and cache-based identifiers have the same property. A crawl whose "stateless" guarantee is ''clearCookies()'' is stateful in exactly the channels that were designed to survive a cookie clear. Acar et al. put the general version of this more starkly {[acar2014_never]}: | The cache matters because it is a tracking channel in its own right, not just a performance detail. Solomos et al. showed that Chrome's **favicon cache** is a separate store that browser "clear browsing data" controls do not touch and that persists into incognito {[solomos2021_tales]}; ETag- and cache-based identifiers have the same property. A crawl whose "stateless" guarantee is ''clearCookies()'' is stateful in exactly the channels that were designed to survive a cookie clear. Acar et al. put the general version of this more starkly {[acar2014_never]}: |
| > once some tracking has happened, it is hard to start from a truly clean profile | > once some tracking has happened, it is hard to start from a truly clean profile |
| |
| The complete list of things you should be able to say you reset, or say you did not: cookies (including partitioned ones), ''localStorage'' and ''sessionStorage'', IndexedDB, Cache Storage and service workers, the HTTP disk cache, the favicon cache, HSTS and TLS session state, DNS cache, permission grants, and the extension state of anything you installed. | The complete list of things you should be able to say you reset, or say you did not: cookies (including partitioned ones), ''localStorage'' and ''sessionStorage'', IndexedDB, Cache Storage and service workers, the HTTP disk cache, the favicon cache, HSTS and TLS session state, DNS cache, permission grants, and the extension state of anything you installed. What each of those stores is as a //measurement target// — whether your instrument can read it at all, whether it is partitioned, and what a per-origin reset looks like — is [[Privacy:Browser storage]]. |
| |
| ===== What each design can and cannot measure ===== | ===== What each design can and cannot measure ===== |
| | Logged-in versus anonymous web | **stateful** (a session) | the session cookie //is// the state | {[kaizer2016_characterizing]}, {[rautenstrauch2024_auth]}, {[rautenstrauch2023_leaky]} | | | Logged-in versus anonymous web | **stateful** (a session) | the session cookie //is// the state | {[kaizer2016_characterizing]}, {[rautenstrauch2024_auth]}, {[rautenstrauch2023_leaky]} | |
| | First-party-cookie abuse for cross-site tracking | **stateful** | the abuse is the reuse of a first-party value elsewhere | {[chen2021_cookieswap]} | | | First-party-cookie abuse for cross-site tracking | **stateful** | the abuse is the reuse of a first-party value elsewhere | {[chen2021_cookieswap]} | |
| | Cache-based attacks and leaks | **stateful cache**, explicitly primed | an empty cache is the negative control, not the experiment | {[mirheidari2022_cache]}, {[solomos2021_tales]} | | | Cache-based attacks and leaks | **the cache is the state**, and its contents have to be controlled per URL | a cache hit and a cache miss are the two outcomes you are distinguishing, so "we cleared state" without saying whether the cache was cleared makes the result unreadable | {[mirheidari2022_cache]}, {[solomos2021_tales]} | |
| | Effect of a blocker or a setting | **either**, but the //same// for both arms | a blocker's effectiveness depends on how much history it has learned from | {[matthews2018_addons]}, {[jueckstock2022_privacy]} | | | Effect of a blocker or a setting | **either**, but the //same// for both arms | a blocker's effectiveness depends on how much history it has learned from | {[matthews2018_addons]}, {[jueckstock2022_privacy]} | |
| | Anything you want to parallelise over a million sites | **stateless** | see the next section | {[englehardt2016online]} | | | Anything you want to parallelise over a million sites | **stateless** | see the next section | {[englehardt2016online]} | |
| > our current capabilities of saving/restoring the storage are not exactly compatible with partitioning … Without CDP support, it does not seem practical to replicate all the intricate details of storage partitioning outside of the browser, so disabling the feature is the only way to make things work for now | > our current capabilities of saving/restoring the storage are not exactly compatible with partitioning … Without CDP support, it does not seem practical to replicate all the intricate details of storage partitioning outside of the browser, so disabling the feature is the only way to make things work for now |
| |
| ((Playwright maintainer, ''github.com/microsoft/playwright/issues/38455'' ("Enable storage partitioning and consider expanding storage state API to support storage keys", opened 2025-12-05), comment of 2025-12-09. The issue was closed on 2025-12-22 after the corresponding Chromium request, ''crbug.com/468317746'', was closed as "infeasible — too far outside of the product scope". Issue 32230 was closed 2024-09-27, fixed by PR 32701, "fix(chromium): disable ThirdPartyStoragePartitioning", merged 2024-09-19. All checked 2026-08-19.)) | ((Playwright maintainer, ''github.com/microsoft/playwright/issues/38455'' ("Enable storage partitioning and consider expanding storage state API to support storage keys", opened 2025-12-05), comment of 2025-12-09. The issue was closed on 2025-12-22 after the corresponding Chromium request, ''crbug.com/468317746'', was closed as "infeasible - too far outside of the product scope". Issue 32230 was closed 2024-09-27, fixed by PR 32701, "fix(chromium): disable ThirdPartyStoragePartitioning", merged 2024-09-19. All checked 2026-08-19.)) |
| |
| So the consequence is that two crawlers driving the same engine version accumulate different third-party state, on the one axis this page is about, and the divergence exists because one of them has a profile-serialisation API that cannot express partitioned storage. Neither documents this where you would look. | So the consequence is that two crawlers driving the same engine version accumulate different third-party state, on the one axis this page is about, and the divergence exists because one of them has a profile-serialisation API that cannot express partitioned storage. Neither documents this where you would look. |
| ==== Stateful ==== | ==== Stateful ==== |
| |
| * **Playwright:** ''chromium.launchPersistentContext(userDataDir)'' and reuse ''userDataDir''. Everything persists, including the cache. To carry a profile //deliberately and legibly// instead, use ''storageState()'' — it serialises cookies, ''localStorage'' and IndexedDB to JSON you can commit as an artefact, which makes the seed reproducible in a way a binary profile directory is not. It does not carry the HTTP cache. | * **Playwright:** ''chromium.launchPersistentContext(userDataDir)'' and reuse ''userDataDir''. Everything persists, including the cache. To carry a profile //deliberately and legibly// instead, use ''storageState()'' — it serialises cookies and ''localStorage'' to JSON you can commit as an artefact, which makes the seed reproducible in a way a binary profile directory is not. **IndexedDB is behind an option that is off by default** (''storageState({ indexedDB: true })''), and ''sessionStorage'', Cache Storage and service-worker registrations are not in it at all; measured, on Playwright 1.62.1, in [[Privacy:Browser storage#What each capture method actually returns]]. It does not carry the HTTP cache. |
| * **Puppeteer:** ''puppeteer.launch({ userDataDir })'' and reuse the directory — the same mechanism as Playwright's persistent context, and worth knowing because Puppeteer is the more used of the two in this corpus (76 crawling papers against Playwright's 34). Puppeteer has no ''storageState'' equivalent, so a legible seed means either shipping the profile directory or writing your own cookie/storage dump. | * **Puppeteer:** ''puppeteer.launch({ userDataDir })'' and reuse the directory — the same mechanism as Playwright's persistent context, and worth knowing because Puppeteer is the more used of the two in this corpus (76 crawling papers against Playwright's 34). Puppeteer has no ''storageState'' equivalent, so a legible seed means either shipping the profile directory or writing your own cookie/storage dump. |
| * **OpenWPM:** stateful is the **default**, and stateless is per-command-sequence: ''CommandSequence(url, reset=True)'', documented as "True if browser should clear state and restart after sequence".((OpenWPM ''openwpm/command_sequence.py'' at ''master'' commit ''b9dd4c3a'', checked 2026-08-19.)) There is no global switch, which is why papers describe this in prose and reviewers cannot check it. Watch ''num_browsers'': with ''N'' browsers your "stateful crawl" is ''N'' cookie jars. Details on [[Programming:Crawler:OpenWPM#Stateful and stateless in OpenWPM]]. | * **OpenWPM:** stateful is the **default**, and stateless is per-command-sequence: ''CommandSequence(url, reset=True)'', documented as "True if browser should clear state and restart after sequence".((OpenWPM ''openwpm/command_sequence.py'' at ''master'' commit ''b9dd4c3a'', checked 2026-08-19.)) There is no global switch, which is why papers describe this in prose and reviewers cannot check it. Watch ''num_browsers'': with ''N'' browsers your "stateful crawl" is ''N'' cookie jars. Details on [[Programming:Crawler:OpenWPM#Stateful and stateless in OpenWPM]]. |
| | ''headless'' | 1.0% | 15.6% | 13.3% | 14.5% | 11.1% | 14.6 pp | +10.1 pp | | | ''headless'' | 1.0% | 15.6% | 13.3% | 14.5% | 11.1% | 14.6 pp | +10.1 pp | |
| |
| Denominators are the crawling papers in each bucket, from the table above (102 / 167 / 308 / 345 / 198). Naming the browser gained 18 points and headless mode gained 10 from a near-zero base; statefulness gained 2.5 and has the narrowest range of any of them. Only interaction depth moved less in one sense, and it moved //down//. It is not that the field decided the axis does not matter — Demir et al. made it a named criterion in 2022, and Zeber et al. and Jueckstock et al. had made it a measured concern in 2020 and 2021. It is that nothing turned the concern into a reporting norm: no venue asks for it on a checklist, and no widely used tool writes it into a config file that ends up in an artifact. | Denominators are the crawling papers in each bucket, from the table above (102 / 167 / 308 / 345 / 198). Naming the browser gained 18 points and headless mode gained 10 from a near-zero base; statefulness gained 2.5 and has the narrowest **range** of the six (5.3 pp). Interaction depth is the only field whose range is nearly as narrow (6.2 pp), and it got that way by //declining// from 78.4% to 72.2% rather than by standing still. It is not that the field decided the axis does not matter — Demir et al. made it a named criterion in 2022, and Zeber et al. and Jueckstock et al. had made it a measured concern in 2020 and 2021. It is that nothing turned the concern into a reporting norm: no venue asks for it on a checklist, and no widely used tool writes it into a config file that ends up in an artifact. |
| |
| What //did// move is the answer among those who give one: **the stateless share of stated values rose from 43.8% into the 50s** (50.8%, 59.2%, 52.8% over the last three buckets). Read this as the field's default hardening rather than as a swing in practice — the whole cell is small (16 papers in the first bucket), and the modern tooling defaults to stateless. | What //did// move is the answer among those who give one: **the stateless share of stated values rose from 43.8% into the 50s** (50.8%, 59.2%, 52.8% over the last three buckets). Read this as the field's default hardening rather than as a swing in practice — the whole cell is small (16 papers in the first bucket), and the modern tooling defaults to stateless. |
| | {[chen2021_cookieswap]} | repeat visits retaining state alongside fresh-profile visits | | | {[chen2021_cookieswap]} | repeat visits retaining state alongside fresh-profile visits | |
| | {[mehrnezhad2022_protect]} | consent accepted on visit two and opted out of on visit three, plus a private-mode arm | | | {[mehrnezhad2022_protect]} | consent accepted on visit two and opted out of on visit three, plus a private-mode arm | |
| | {[mirheidari2022_cache]} | primed cache against unprimed | | | {[mirheidari2022_cache]} | per-URL cache hit against cache miss, verified for each candidate | |
| | {[rautenstrauch2023_leaky]} | logged-in state against anonymous, "a fresh browser context that we reset between" | | | {[rautenstrauch2023_leaky]} | logged-in state against anonymous, "a fresh browser context that we reset between" | |
| | {[liu2024_opted]} | personas accumulating over nine iterated visits, against control personas | | | {[liu2024_opted]} | personas accumulating over nine iterated visits, against control personas | |
| ===== Open Questions ===== | ===== Open Questions ===== |
| |
| | <WRAP todo> |
| * **Nobody has run the clean experiment.** Demir et al. announce one and do not deliver it: §2.2 says "Since the effects of C5 and C11 are not yet adequately discussed by previous work, we analyze them in Section 4", and §4 then runs "four exemplarily case studies focusing on C4, C5, C10, and C12" — repetition, crawler technology, interaction and geolocation. C11, the crawling strategy, is the one criterion they flagged and did not vary; their own runs are described in Appendix C as "stateless coordinated crawls" {[demir2022_reproducibility]}. Sixteen papers in this corpus run both arms, every one of them incidentally to another question. A same-sites, same-time, same-vantage crawl differing //only// in statefulness, reporting the effect on third-party counts, tracker counts and filter-list hit rates, would be a short and highly citable paper. | * **Nobody has run the clean experiment.** Demir et al. announce one and do not deliver it: §2.2 says "Since the effects of C5 and C11 are not yet adequately discussed by previous work, we analyze them in Section 4", and §4 then runs "four exemplarily case studies focusing on C4, C5, C10, and C12" — repetition, crawler technology, interaction and geolocation. C11, the crawling strategy, is the one criterion they flagged and did not vary; their own runs are described in Appendix C as "stateless coordinated crawls" {[demir2022_reproducibility]}. Sixteen papers in this corpus run both arms, every one of them incidentally to another question. A same-sites, same-time, same-vantage crawl differing //only// in statefulness, reporting the effect on third-party counts, tracker counts and filter-list hit rates, would be a short and highly citable paper. |
| * **How much does visit order actually change a stateful result?** The confound is universally acknowledged and never quantified. | * **How much does visit order actually change a stateful result?** The confound is universally acknowledged and never quantified. |
| * **What does statefulness mean under partitioning?** If a stateful crawl's cross-site accumulation is the thing being measured, and every default browser now partitions storage while the research tooling either disables the partitioning (Playwright, OpenWPM — both verified above) or predates it entirely, then the stateful/stateless dichotomy needs a third dimension. No paper in the corpus addresses this, and the ground is still moving: Playwright's opt-out is expected to become unavailable when Chromium removes the flag, at which point every Playwright-based crawl changes behaviour without any change to the paper's own code. | * **What does statefulness mean under partitioning?** If a stateful crawl's cross-site accumulation is the thing being measured, and every default browser now partitions storage while the research tooling either disables the partitioning (Playwright, OpenWPM — both verified above) or predates it entirely, then the stateful/stateless dichotomy needs a third dimension. No paper in the corpus addresses this, and the ground is still moving: Playwright's opt-out is expected to become unavailable when Chromium removes the flag, at which point every Playwright-based crawl changes behaviour without any change to the paper's own code. |
| * **Does the per-browser cookie-jar partition change published results?** Carried over from [[Programming:Crawler:OpenWPM]] because it is the same question: no paper we found reports ''num_browsers'' alongside a stateful claim. | * **Does the per-browser cookie-jar partition change published results?** Carried over from [[Programming:Crawler:OpenWPM]] because it is the same question: no paper we found reports ''num_browsers'' alongside a stateful claim. |
| | </WRAP> |
| ====== References ====== | ====== References ====== |
| |
