| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| programming:stateful_stateless [2026/08/19 11:29] – Review round 2 (citations): switch to the pre-existing kancherla2025_johnny key (my duplicate entry is deleted from the bibliography), correct Demir et al. to 2016-2021 and note they cover the same seven venues, mark 56% as our arithmetic on their Table 2 karel.kubicek.claude | programming:stateful_stateless [2026/08/31 19:48] (current) – Correct the storageState() sentence: measured on Playwright 1.62.1, it does not carry IndexedDB unless {indexedDB:true} is passed, and never carries sessionStorage, Cache Storage or service-worker registrations. Link Privacy:Browser storage for the stores karel.kubicek.claude |
|---|
| ====== Stateful and Stateless Crawling ====== | ====== Stateful and Stateless Crawling ====== |
| |
| A crawl is **stateless** when the browser starts each visit from an empty profile, and **stateful** when it carries the profile — cookies, ''localStorage'', IndexedDB, the HTTP cache — from one visit to the next. That single switch decides what your crawl is able to observe at all: a crawl that starts every visit from an empty profile and visits each target once cannot see cookie syncing, retargeting, or the effect of a consent choice on the //next// site, because none of those exist without accumulated state. (Respawning is the exception that proves the rule — it can be caught from a stateless harness, but only by deliberately staging a clear between two visits to the same site; see [[#What a stateful design buys you, quantified]].) It also decides how fast you can go, whether your results depend on the order you visited the list in, and how far your numbers can be pushed towards a claim about real users. | A crawl is **stateless** when the browser starts each visit from an empty profile, and **stateful** when it carries the profile — cookies, ''localStorage'', IndexedDB, the HTTP cache — from one visit to the next. That single switch decides what your crawl is able to observe at all: a crawl that starts every visit from an empty profile and visits each target once cannot see retargeting, cookie respawning, or the effect of a consent choice on the //next// site, because none of those exist without accumulated state. It sees cookie //syncing// — a fresh profile is in fact an unusually attractive target for it, see [[#A crawler sees a different web from a user, and statefulness is part of why|below]] — but only first contact, never the sync graph a real user has accumulated. It also decides how fast you can go, whether your results depend on the order you visited the list in, and how far your numbers can be pushed towards a claim about real users. |
| |
| This page is about that design choice. For how one particular tool implements it, see [[Programming:Crawler:OpenWPM#Stateful and stateless in OpenWPM]]; for the tools themselves, [[Programming:Crawler]]; for where to crawl from, [[Design:Crawling location]]; for what to do on the page once you are there, [[Programming:Interaction]] and [[Privacy:Consent]]. | This page is about that design choice. For how one particular tool implements it, see [[Programming:Crawler:OpenWPM#Stateful and stateless in OpenWPM]]; for the tools themselves, [[Programming:Crawler]]; for where to crawl from, [[Design:Crawling location]]; for what to do on the page once you are there, [[Programming:Interaction]] and [[Privacy:Consent]]. |
| Three things a fresh measurement gets wrong most often. | Three things a fresh measurement gets wrong most often. |
| |
| - **Almost nobody reports it.** Of the **1,120 papers in our corpus that ran an automated web crawl, 219 (19.6%) say whether the crawl was stateful** — the second-least-reported crawl-configuration field after headless mode, and the only crawl-configuration reporting rate in this corpus with no trend at all across sixteen years (see [[#Use in Publications]]). | - **Almost nobody reports it.** Of the **1,120 papers in our corpus that ran an automated web crawl, 219 (19.6%) say whether the crawl was stateful** — the second-least-reported crawl-configuration field after headless mode, and the **flattest**: its reporting rate moved 2.5 percentage points between the first and last four-year bucket, against +18.2 for naming a browser and +10.1 for headless (see [[#Reporting has not improved in sixteen years]]). |
| - **"We cleared cookies" is not a reset.** Measured here on Playwright 1.62.1 / Chromium 151: ''context.clearCookies()'' leaves ''localStorage'' intact //and// leaves the HTTP cache warm, so the next visit serves subresources from disk and never touches the origin. See [[#What a reset actually resets]]. | - **"We cleared cookies" is not a reset.** Measured here on Playwright 1.62.1 / Chromium 151: ''context.clearCookies()'' leaves ''localStorage'' intact //and// leaves the HTTP cache warm, so the next visit serves subresources from disk and never touches the origin. See [[#What a reset actually resets]]. |
| - **Since 2022 the browser decides how stateful your stateful crawl is, not you.** Firefox partitions third-party cookies per top-level site by default; Chrome partitions third-party //storage// for every user from Chrome 115 on. Playwright launches Chromium with ''--disable-features=ThirdPartyStoragePartitioning'', Puppeteer does not. Same engine, two different webs. See [[#Since 2022 the engine decides, not you]]. | - **Since 2022 the browser decides how stateful your stateful crawl is, not you.** Firefox partitions third-party cookies per top-level site by default; Chrome partitions third-party //storage// for every user from Chrome 115 on. Playwright launches Chromium with ''--disable-features=ThirdPartyStoragePartitioning'', Puppeteer does not. Same engine, two different webs. See [[#Since 2022 the engine decides, not you]]. |
| Playwright 1.62.1, Chromium 151.0.7922.34, Linux. Reproduce with [[#The code|the script below]]. | Playwright 1.62.1, Chromium 151.0.7922.34, Linux. Reproduce with [[#The code|the script below]]. |
| |
| Read off the three rows in bold. **Clearing cookies clears cookies.** It does not clear ''localStorage'', and it does not clear the HTTP cache — in the ''clearCookies()'' rows the cached subresource was never re-requested from the origin, so a request-counting measurement silently loses it on every visit after the first. Playwright's own documentation is accurate and easy to misread: ''clearCookies'' "removes cookies from context",((Playwright API reference, ''BrowserContext.clearCookies'' and ''BrowserContext.storageState'', checked 2026-08-19.)) and ''storageState'' returns "current cookies, local storage snapshot, IndexedDB snapshot and virtual WebAuthn credentials" — note what is missing from that list, and note that the ''storageState'' row above is the only one where cookies came back while the cache did not. | Read off the three rows in bold. **Clearing cookies clears cookies.** It does not clear ''localStorage'', and it does not clear the HTTP cache — in the ''clearCookies()'' rows the cached subresource was never re-requested from the origin, so a request-counting measurement silently loses it on every visit after the first. Playwright's own documentation is accurate and easy to misread: ''clearCookies'' "Removes cookies from context",((Playwright API reference, ''BrowserContext.clearCookies'' and ''BrowserContext.storageState'', checked 2026-08-19.)) and ''storageState'' returns "current cookies, local storage snapshot, IndexedDB snapshot and virtual WebAuthn credentials" — note what is missing from that list, and note that the ''storageState'' row above is the only one where cookies came back while the cache did not. |
| |
| The cache matters because it is a tracking channel in its own right, not just a performance detail. Solomos et al. showed that Chrome's **favicon cache** is a separate store that browser "clear browsing data" controls do not touch and that persists into incognito {[solomos2021_tales]}; ETag- and cache-based identifiers have the same property. A crawl whose "stateless" guarantee is ''clearCookies()'' is stateful in exactly the channels that were designed to survive a cookie clear. Acar et al. put the general version of this more starkly {[acar2014_never]}: | The cache matters because it is a tracking channel in its own right, not just a performance detail. Solomos et al. showed that Chrome's **favicon cache** is a separate store that browser "clear browsing data" controls do not touch and that persists into incognito {[solomos2021_tales]}; ETag- and cache-based identifiers have the same property. A crawl whose "stateless" guarantee is ''clearCookies()'' is stateful in exactly the channels that were designed to survive a cookie clear. Acar et al. put the general version of this more starkly {[acar2014_never]}: |
| > once some tracking has happened, it is hard to start from a truly clean profile | > once some tracking has happened, it is hard to start from a truly clean profile |
| |
| The complete list of things you should be able to say you reset, or say you did not: cookies (including partitioned ones), ''localStorage'' and ''sessionStorage'', IndexedDB, Cache Storage and service workers, the HTTP disk cache, the favicon cache, HSTS and TLS session state, DNS cache, permission grants, and the extension state of anything you installed. | The complete list of things you should be able to say you reset, or say you did not: cookies (including partitioned ones), ''localStorage'' and ''sessionStorage'', IndexedDB, Cache Storage and service workers, the HTTP disk cache, the favicon cache, HSTS and TLS session state, DNS cache, permission grants, and the extension state of anything you installed. What each of those stores is as a //measurement target// — whether your instrument can read it at all, whether it is partitioned, and what a per-origin reset looks like — is [[Privacy:Browser storage]]. |
| |
| ===== What each design can and cannot measure ===== | ===== What each design can and cannot measure ===== |
| ^ Phenomenon ^ Needs ^ Why ^ Example ^ | ^ Phenomenon ^ Needs ^ Why ^ Example ^ |
| | Third-party presence, request counts, filter-list hit rates | either, but say which | a fresh profile draws //more// third-party traffic than an aged one, so the two are not interchangeable | {[jueckstock2021_realistic]}, {[zeber2020representativeness]} | | | Third-party presence, request counts, filter-list hit rates | either, but say which | a fresh profile draws //more// third-party traffic than an aged one, so the two are not interchangeable | {[jueckstock2021_realistic]}, {[zeber2020representativeness]} | |
| | Cookie syncing / ID sharing | **stateful** | there is no ID to sync until one has been set on a previous site | {[englehardt2016online]}, {[agarwal2020_stop]} | | | Cookie syncing / ID sharing | **either, but they measure different things** | a fresh profile sees //first-contact// syncing and over-triggers it {[zeber2020representativeness]}; reconstructing the sync graph of an aged identity, or how much history a partner can merge, needs accumulation | {[englehardt2016online]} runs its sync analysis on the //stateful// 100k crawl; {[agarwal2020_stop]}, {[acar2014_never]} | |
| | Cookie respawning, evercookies | **stateful** across a clear | the phenomenon //is// state surviving a reset | {[acar2014_never]}; detectable from a stateless harness by comparing paired visits {[fouad2022my]} | | | Cookie respawning, evercookies | **stateful** across a clear | the phenomenon //is// state surviving a reset | {[acar2014_never]}; detectable from a stateless harness by comparing paired visits {[fouad2022my]} | |
| | Ad retargeting, personalisation, differential pricing | **stateful** training, then usually seeded-stateless measurement | the profile is the independent variable | {[bashir2016tracing]}, {[agarwal2020_stop]}, {[liu2024_opted]}, {[meng2014_pollution]}, {[robertson2018_auditing]} | | | Ad retargeting, personalisation, differential pricing | **stateful** training, then usually seeded-stateless measurement | the profile is the independent variable | {[bashir2016tracing]}, {[agarwal2020_stop]}, {[liu2024_opted]}, {[meng2014_pollution]}, {[robertson2018_auditing]} | |
| | Logged-in versus anonymous web | **stateful** (a session) | the session cookie //is// the state | {[kaizer2016_characterizing]}, {[rautenstrauch2024_auth]}, {[rautenstrauch2023_leaky]} | | | Logged-in versus anonymous web | **stateful** (a session) | the session cookie //is// the state | {[kaizer2016_characterizing]}, {[rautenstrauch2024_auth]}, {[rautenstrauch2023_leaky]} | |
| | First-party-cookie abuse for cross-site tracking | **stateful** | the abuse is the reuse of a first-party value elsewhere | {[chen2021_cookieswap]} | | | First-party-cookie abuse for cross-site tracking | **stateful** | the abuse is the reuse of a first-party value elsewhere | {[chen2021_cookieswap]} | |
| | Cache-based attacks and leaks | **stateful cache**, explicitly primed | an empty cache is the negative control, not the experiment | {[mirheidari2022_cache]}, {[solomos2021_tales]} | | | Cache-based attacks and leaks | **the cache is the state**, and its contents have to be controlled per URL | a cache hit and a cache miss are the two outcomes you are distinguishing, so "we cleared state" without saying whether the cache was cleared makes the result unreadable | {[mirheidari2022_cache]}, {[solomos2021_tales]} | |
| | Effect of a blocker or a setting | **either**, but the //same// for both arms | a blocker's effectiveness depends on how much history it has learned from | {[matthews2018_addons]}, {[jueckstock2022_privacy]} | | | Effect of a blocker or a setting | **either**, but the //same// for both arms | a blocker's effectiveness depends on how much history it has learned from | {[matthews2018_addons]}, {[jueckstock2022_privacy]} | |
| | Anything you want to parallelise over a million sites | **stateless** | see the next section | {[englehardt2016online]} | | | Anything you want to parallelise over a million sites | **stateless** | see the next section | {[englehardt2016online]} | |
| |
| * **Consent given on one site follows you to the next.** Rasaii et al. accepted banners across the first half of Tranco's top 20,000 and then measured the second half with that profile loaded — the denominator for the headline figure is the second-half domains where a banner was successfully rejected, not all 20,000. "Our findings reveal that around 50% of websites send at least one intractable cookie" — a tracking cookie transmitted before any consent on the site sending it. Sites with a CMP banner sent **6.91×** more of them than sites with a native banner; enabling Global Privacy Control cut them by about **30%**, with a further **32%** on later visits after rejecting; and about **25%** stop being sent only after the page is reloaded {[rasaii2025_crumbs]}. Partitioning does not yet blunt this: "only 1.3% of all unique tracking cookies are partitioned, with more than half accompanied by nonpartitioned cookies from the same tracker domain". | * **Consent given on one site follows you to the next.** Rasaii et al. accepted banners across the first half of Tranco's top 20,000 and then measured the second half with that profile loaded — the denominator for the headline figure is the second-half domains where a banner was successfully rejected, not all 20,000. "Our findings reveal that around 50% of websites send at least one intractable cookie" — a tracking cookie transmitted before any consent on the site sending it. Sites with a CMP banner sent **6.91×** more of them than sites with a native banner; enabling Global Privacy Control cut them by about **30%**, with a further **32%** on later visits after rejecting; and about **25%** stop being sent only after the page is reloaded {[rasaii2025_crumbs]}. Partitioning does not yet blunt this: "only 1.3% of all unique tracking cookies are partitioned, with more than half accompanied by nonpartitioned cookies from the same tracker domain". |
| * **Respawning plus syncing survives a state clear.** Acar et al. found "33 different Flash cookies from 30 different domains respawned a total of 355 cookies on 107 first party domains", and concluded that through one ad exchange present on ~11% of first parties, "This scenario enables at least 11% of a user's history to be tracked over time" {[acar2014_never]}. | * **Respawning plus syncing survives a state clear.** The 2014 mechanism is historical — Flash reached end of life in December 2020 — but the finding is the reason a state clear cannot be assumed to work, and the technique moved to fingerprint-keyed respawning rather than disappearing ({[fouad2022my]} in 2022, server-side in {[fouad2024_devil]} in 2024). Acar et al. found "33 different Flash cookies from 30 different domains respawned a total of 355 cookies on 107 first party domains", and concluded that through one ad exchange present on ~11% of first parties, "This scenario enables at least 11% of a user's history to be tracked over time" {[acar2014_never]}. |
| * **A trained profile is treated differently.** Agarwal et al.: "having an established persona from a particular demographic … results in up to 15% more cookies stored than for a baseline with no set persona" {[agarwal2020_stop]}. | * **A trained profile is treated differently.** Agarwal et al.: "having an established persona from a particular demographic … results in up to 15% more cookies stored than for a baseline with no set persona" {[agarwal2020_stop]}. |
| * **State accumulates within a site, not only across sites.** Urban et al., under a seeded-stateless design with state kept across a site's own subpages: "subsites set considerably more (36 %) cookies than the respective landing pages. On average, 55 cookies were set when loading a landing page while 78 were set when a subsite was accessed" {[urban2020beyond]}. See [[Programming:Interaction]]. | * **State accumulates within a site, not only across sites.** Urban et al., under a seeded-stateless design with state kept across a site's own subpages: "subsites set considerably more (36 %) cookies than the respective landing pages. On average, 55 cookies were set when loading a landing page while 78 were set when a subsite was accessed" {[urban2020beyond]}. See [[Programming:Interaction]]. |
| |
| And one that cuts the other way: **cookie respawning with browser fingerprinting** was measured on 30,000 Alexa sites with a stateless harness, by comparing paired visits rather than by accumulating a profile — "1, 150 (3.83%) of the Alexa top 30, 000 websites use cookie respawning with browser fingerprinting" {[fouad2022my]}. Note the qualifier: that figure counts respawning //combined with// fingerprinting, not respawning in general. A stateful //phenomenon// does not always require a stateful //crawl//; sometimes it requires two controlled visits. | And one that cuts the other way: **cookie respawning with browser fingerprinting** was measured on 30,000 Alexa sites with a stateless harness, by comparing paired visits rather than by accumulating a profile — "1, 150 (3.83%) of the Alexa top 30, 000 websites use cookie respawning with browser fingerprinting" {[fouad2022my]}. Note the qualifier: that figure counts respawning //combined with// fingerprinting, not respawning in general. A stateful //phenomenon// does not always require a stateful //crawl//; sometimes it requires two controlled visits. The same is true of cookie syncing at first contact. What accumulation buys is the //aged// identity, not the mechanism. |
| |
| ===== Since 2022 the engine decides, not you ===== | ===== Since 2022 the engine decides, not you ===== |
| > our current capabilities of saving/restoring the storage are not exactly compatible with partitioning … Without CDP support, it does not seem practical to replicate all the intricate details of storage partitioning outside of the browser, so disabling the feature is the only way to make things work for now | > our current capabilities of saving/restoring the storage are not exactly compatible with partitioning … Without CDP support, it does not seem practical to replicate all the intricate details of storage partitioning outside of the browser, so disabling the feature is the only way to make things work for now |
| |
| ((Playwright maintainer, ''github.com/microsoft/playwright/issues/38455'' ("Enable storage partitioning and consider expanding storage state API to support storage keys", opened 2025-12-05), comment of 2025-12-09. The issue was closed on 2025-12-22 after the corresponding Chromium request, ''crbug.com/468317746'', was closed as "infeasible — too far outside of the product scope". Issue 32230 was closed 2024-09-27, fixed by PR 32701, "fix(chromium): disable ThirdPartyStoragePartitioning", merged 2024-09-19. All checked 2026-08-19.)) | ((Playwright maintainer, ''github.com/microsoft/playwright/issues/38455'' ("Enable storage partitioning and consider expanding storage state API to support storage keys", opened 2025-12-05), comment of 2025-12-09. The issue was closed on 2025-12-22 after the corresponding Chromium request, ''crbug.com/468317746'', was closed as "infeasible - too far outside of the product scope". Issue 32230 was closed 2024-09-27, fixed by PR 32701, "fix(chromium): disable ThirdPartyStoragePartitioning", merged 2024-09-19. All checked 2026-08-19.)) |
| |
| So the consequence is that two crawlers driving the same engine version accumulate different third-party state, on the one axis this page is about, and the divergence exists because one of them has a profile-serialisation API that cannot express partitioned storage. Neither documents this where you would look. | So the consequence is that two crawlers driving the same engine version accumulate different third-party state, on the one axis this page is about, and the divergence exists because one of them has a profile-serialisation API that cannot express partitioned storage. Neither documents this where you would look. |
| |
| **And it is a moving target.** In the same thread, the person who filed the request notes that "when the ThirdPartyStoragePartitioning flag is removed, bug #32230 will start reoccurring" — that is, Playwright's opt-out is expected to stop being available, and as of January 2026 the maintainers had no replacement design. Whenever that lands, a Playwright crawl starts accumulating //partitioned// storage with no change to your code, at whatever version boundary it happens on. Pin and report the Playwright version alongside the statefulness claim. | **And it is a moving target.** In the same thread, the person who filed the request notes that "when the ThirdPartyStoragePartitioning flag is removed, bug #32230 will start reoccurring" — that is, Playwright's opt-out is expected to stop being available, and as late as January 2026 the maintainers were still asking the reporter for a design that would keep ''storageState'' working without the flag. Whenever that lands, a Playwright crawl starts accumulating //partitioned// storage with no change to your code, at whatever version boundary it happens on. Pin and report the Playwright version alongside the statefulness claim. |
| |
| * **OpenWPM also opts out, by default.** ''BrowserParams.tp_cookies'' defaults to ''"always"'', which sets ''network.cookie.cookieBehavior = 0'' — all third-party cookies allowed, unpartitioned. Firefox tracking protection cannot be switched on at all: the code raises ''RuntimeError("Firefox Tracking Protection is not currently supported")''.((OpenWPM ''openwpm/config.py'' and ''openwpm/deploy_browsers/configure_firefox.py'', read at ''master'' commit ''b9dd4c3a'' (2026-07-02); latest release ''v0.35.0'' (2026-06-17). Checked 2026-08-19.)) | * **OpenWPM also opts out, by default.** ''BrowserParams.tp_cookies'' defaults to ''"always"'', which sets ''network.cookie.cookieBehavior = 0'' — all third-party cookies allowed, unpartitioned. Firefox tracking protection cannot be switched on at all: the code raises ''RuntimeError("Firefox Tracking Protection is not currently supported")''.((OpenWPM ''openwpm/config.py'' and ''openwpm/deploy_browsers/configure_firefox.py'', read at ''master'' commit ''b9dd4c3a'' (2026-07-02); latest release ''v0.35.0'' (2026-06-17). Checked 2026-08-19.)) |
| |
| So the plain fact is that a stateful research crawl in 2026 accumulates an **unpartitioned** cross-site profile that resembles a default Chrome user's and does //not// resemble a default Firefox or Safari user's — and it does so whichever engine you drive, because the research tooling disables the partitioning. Whether that is the right choice depends on your question; it is never the right thing to leave unsaid. Measured, on the default Playwright Chromium context: | So the plain fact is that a stateful research crawl in 2026 accumulates an **unpartitioned** cross-site profile. That resembles a default Chrome user's //cookie jar// — but not that user's storage, which Chrome has partitioned since 115 — and it does not resemble a default Firefox or Safari user in either respect. And it happens whichever engine you drive, because the research tooling disables the partitioning. Whether that is the right choice depends on your question; it is never the right thing to leave unsaid. Measured, on the default Playwright Chromium context: |
| |
| <code> | <code> |
| ==== Stateful ==== | ==== Stateful ==== |
| |
| * **Playwright:** ''chromium.launchPersistentContext(userDataDir)'' and reuse ''userDataDir''. Everything persists, including the cache. To carry a profile //deliberately and legibly// instead, use ''storageState()'' — it serialises cookies, ''localStorage'' and IndexedDB to JSON you can commit as an artefact, which makes the seed reproducible in a way a binary profile directory is not. It does not carry the HTTP cache. | * **Playwright:** ''chromium.launchPersistentContext(userDataDir)'' and reuse ''userDataDir''. Everything persists, including the cache. To carry a profile //deliberately and legibly// instead, use ''storageState()'' — it serialises cookies and ''localStorage'' to JSON you can commit as an artefact, which makes the seed reproducible in a way a binary profile directory is not. **IndexedDB is behind an option that is off by default** (''storageState({ indexedDB: true })''), and ''sessionStorage'', Cache Storage and service-worker registrations are not in it at all; measured, on Playwright 1.62.1, in [[Privacy:Browser storage#What each capture method actually returns]]. It does not carry the HTTP cache. |
| | * **Puppeteer:** ''puppeteer.launch({ userDataDir })'' and reuse the directory — the same mechanism as Playwright's persistent context, and worth knowing because Puppeteer is the more used of the two in this corpus (76 crawling papers against Playwright's 34). Puppeteer has no ''storageState'' equivalent, so a legible seed means either shipping the profile directory or writing your own cookie/storage dump. |
| * **OpenWPM:** stateful is the **default**, and stateless is per-command-sequence: ''CommandSequence(url, reset=True)'', documented as "True if browser should clear state and restart after sequence".((OpenWPM ''openwpm/command_sequence.py'' at ''master'' commit ''b9dd4c3a'', checked 2026-08-19.)) There is no global switch, which is why papers describe this in prose and reviewers cannot check it. Watch ''num_browsers'': with ''N'' browsers your "stateful crawl" is ''N'' cookie jars. Details on [[Programming:Crawler:OpenWPM#Stateful and stateless in OpenWPM]]. | * **OpenWPM:** stateful is the **default**, and stateless is per-command-sequence: ''CommandSequence(url, reset=True)'', documented as "True if browser should clear state and restart after sequence".((OpenWPM ''openwpm/command_sequence.py'' at ''master'' commit ''b9dd4c3a'', checked 2026-08-19.)) There is no global switch, which is why papers describe this in prose and reviewers cannot check it. Watch ''num_browsers'': with ''N'' browsers your "stateful crawl" is ''N'' cookie jars. Details on [[Programming:Crawler:OpenWPM#Stateful and stateless in OpenWPM]]. |
| * **Seeding:** build the seed in a separate, documented run; store it (''storageState'' JSON, or OpenWPM's ''seed_tar''); record when it was built and what it visited. A seed profile ages — 2016's top 10,000 sites are not 2026's, and a seed built before a crawl that ran for three weeks is not the same instrument at the end as at the start. | * **Seeding:** build the seed in a separate, documented run; store it (''storageState'' JSON, or OpenWPM's ''seed_tar''); record when it was built and what it visited. A seed profile ages — 2016's top 10,000 sites are not 2026's, and a seed built before a crawl that ran for three weeks is not the same instrument at the end as at the start. |
| ==== The code ==== | ==== The code ==== |
| |
| This is the **complete** script behind the [[#What a reset actually resets|reset table]] — all nine reset strategies, so every row is reproducible. It needs nothing but Playwright and a free port. | This is the **complete** script behind the [[#What a reset actually resets|reset table]] — all nine reset strategies, so every row is reproducible. Save the two files side by side as ''server.mjs'' and ''probe.mjs''; the second imports the first. It needs nothing but Playwright and a free port. |
| |
| <file javascript state_probe_server.mjs> | <file javascript server.mjs> |
| // Minimal instrumented origin for the state-channel probe. Counts every | // Minimal instrumented origin for the state-channel probe. Counts every |
| // request it receives, per path, and reports what the client sent back. | // request it receives, per path, and reports what the client sent back. |
| </file> | </file> |
| |
| <file javascript state_probe.mjs> | <file javascript probe.mjs> |
| // What each "reset" actually resets. Drives Playwright's own Chromium against a | // What each "reset" actually resets. Drives Playwright's own Chromium against a |
| // local instrumented origin and reports, for each reset strategy, whether the | // local instrumented origin and reports, for each reset strategy, whether the |
| ===== Use in Publications ===== | ===== Use in Publications ===== |
| |
| All figures below are over the **1,120 papers in the corpus that ran an automated web crawl**, out of 5,859 extracted papers from CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026. They are reporting rates: "does not state" means the paper did not say, not that the crawl had no state. Every number here is produced by ''scripts/report_stateful_stateless.mjs''; the full query log is on [[provenance:programming:stateful_stateless]]. | All figures below are over the **1,120 papers in the corpus that ran an automated web crawl**, out of 5,859 extracted papers from CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026. They are reporting rates: "does not state" means the paper did not say, not that the crawl had no state. The figures come from three scripts — ''report_stateful_stateless.mjs'' for the tables, ''statefulness_audit.mjs'' for the 29-paper adjudication, ''statefulness_probe.mjs'' for the text-corroboration counts — and the full query log, with each script's unedited output, is on [[provenance:programming:stateful_stateless]]. |
| |
| ==== Almost nobody says ==== | ==== Almost nobody says ==== |
| | 2025–2026* | 36/198 = **18.2%** | 907/1185 = 76.5% | 467/1019 = 45.8% | | | 2025–2026* | 36/198 = **18.2%** | 907/1185 = 76.5% | 467/1019 = 45.8% | |
| |
| Artifact release more than tripled and ethics-review reporting more than quadrupled. Statefulness has sat between 16% and 21% throughout, with no trend. It is not that the field decided the axis does not matter — Demir et al. made it a named criterion in 2022, and Zeber et al. and Jueckstock et al. had made it a measured concern in 2020 and 2021. It is that nothing turned the concern into a reporting norm: no venue asks for it on a checklist, and no widely used tool writes it into a config file that ends up in an artifact. | Artifact release more than tripled and ethics-review reporting more than quadrupled. Statefulness has sat between 16% and 21% throughout, with no trend. It is also the flattest of the crawl-configuration fields, which is the sharper version of the claim because those fields compete for the same paragraph of the same methods section: |
| |
| What //did// move is the answer among those who give one: **the stateless share of stated values rose from 43.8% to about 55–59%**. Read this as the field's default hardening rather than as a swing in practice — the whole cell is small (16 papers in the first bucket), and the modern tooling defaults to stateless. | ^ Field ^ 2010–2013 ^ 2014–2017 ^ 2018–2021 ^ 2022–2024 ^ 2025–2026* ^ max−min ^ last − first ^ |
| | | ''interactionDepth'' | 78.4% | 75.4% | 76.9% | 73.9% | 72.2% | 6.2 pp | **−6.2 pp** | |
| | | ''authentication'' | 59.8% | 70.1% | 69.2% | 72.8% | 69.2% | 12.9 pp | +9.4 pp | |
| | | ''browsers'' (≥1 named) | 32.4% | 48.5% | 49.7% | 47.0% | 50.5% | 18.2 pp | **+18.2 pp** | |
| | | ''consentAction'' | 24.5% | 29.9% | 32.1% | 33.6% | 29.8% | 9.1 pp | +5.3 pp | |
| | | **''statefulness''** | 15.7% | 21.0% | 19.8% | 20.6% | 18.2% | **5.3 pp** | **+2.5 pp** | |
| | | ''headless'' | 1.0% | 15.6% | 13.3% | 14.5% | 11.1% | 14.6 pp | +10.1 pp | |
| | |
| | Denominators are the crawling papers in each bucket, from the table above (102 / 167 / 308 / 345 / 198). Naming the browser gained 18 points and headless mode gained 10 from a near-zero base; statefulness gained 2.5 and has the narrowest **range** of the six (5.3 pp). Interaction depth is the only field whose range is nearly as narrow (6.2 pp), and it got that way by //declining// from 78.4% to 72.2% rather than by standing still. It is not that the field decided the axis does not matter — Demir et al. made it a named criterion in 2022, and Zeber et al. and Jueckstock et al. had made it a measured concern in 2020 and 2021. It is that nothing turned the concern into a reporting norm: no venue asks for it on a checklist, and no widely used tool writes it into a config file that ends up in an artifact. |
| | |
| | What //did// move is the answer among those who give one: **the stateless share of stated values rose from 43.8% into the 50s** (50.8%, 59.2%, 52.8% over the last three buckets). Read this as the field's default hardening rather than as a swing in practice — the whole cell is small (16 papers in the first bucket), and the modern tooling defaults to stateless. |
| |
| ==== By venue ==== | ==== By venue ==== |
| | IEEE-SP | 110 | 23 | 20.9% | 11 | 9 | 3 | | | IEEE-SP | 110 | 23 | 20.9% | 11 | 9 | 3 | |
| |
| PETS states it at 34.1% — about 60% more often than the next venue (IMC, 21.2%) and two and a half times as often as USENIX Security (13.6%) — and holds 9 of the 29 both-arms papers on just over half of USENIX's crawling volume. If you are writing for PETS, the reviewers are used to seeing it. | PETS states it at 34.1% — about 60% more often than the next venue (IMC, 21.2%) and two and a half times as often as USENIX Security (13.6%) — and holds 9 of the 29 both-arms papers on just over half of USENIX's crawling volume. PETS is where this reporting norm is strongest. |
| |
| ==== The instrument decides whether you say it ==== | ==== The instrument decides whether you say it ==== |
| | //all crawling papers// | 1,120 | 219 | 19.6% | 113 | 77 | 29 | | | //all crawling papers// | 1,120 | 219 | 19.6% | 113 | 77 | 29 | |
| |
| The consent row needs a caveat that [[privacy:consent]] supplies: the same 36 papers were hand-audited there and **7 (19.4%) turned out to be extraction false positives** — they never touched a banner. The second row recomputes the rate on the 28 that survived that audit, and it barely moves, so the finding is robust to the error. | The consent row needs a caveat that [[privacy:consent]] supplies: the same 36 papers were hand-audited there and **7 (19.4%) turned out to be extraction false positives** — they never touched a banner. The second row recomputes the rate on the 28 the audit **fully** supported (a 29th is supported but with an overstated enum value), and it barely moves, so the finding is robust to the error. |
| |
| The good news first: where the design makes the axis unavoidable, reporting roughly triples. A login is state, and 39 of the 54 login crawls that say anything say stateful. The bad news is the repeat-visit row: **199 papers visit the same target two or more times and 114 of them (57.3%) never say whether state carried between the visits** — which is the one thing that determines whether the repeat visit is a replication or a second step in a sequence. If you take one reporting rule from this page, take that one. | The good news first: where the design makes the axis unavoidable, reporting roughly triples. A login is state, and 39 of the 54 login crawls that say anything say stateful. The bad news is the repeat-visit row: **199 papers visit the same target two or more times and 114 of them (57.3%) never say whether state carried between the visits** — which is the one thing that determines whether the repeat visit is a replication or a second step in a sequence. If you take one reporting rule from this page, take that one. |
| | {[chen2021_cookieswap]} | repeat visits retaining state alongside fresh-profile visits | | | {[chen2021_cookieswap]} | repeat visits retaining state alongside fresh-profile visits | |
| | {[mehrnezhad2022_protect]} | consent accepted on visit two and opted out of on visit three, plus a private-mode arm | | | {[mehrnezhad2022_protect]} | consent accepted on visit two and opted out of on visit three, plus a private-mode arm | |
| | {[mirheidari2022_cache]} | primed cache against unprimed | | | {[mirheidari2022_cache]} | per-URL cache hit against cache miss, verified for each candidate | |
| | {[rautenstrauch2023_leaky]} | logged-in state against anonymous, "a fresh browser context that we reset between" | | | {[rautenstrauch2023_leaky]} | logged-in state against anonymous, "a fresh browser context that we reset between" | |
| | {[liu2024_opted]} | personas accumulating over nine iterated visits, against control personas | | | {[liu2024_opted]} | personas accumulating over nine iterated visits, against control personas | |
| ===== Open Questions ===== | ===== Open Questions ===== |
| |
| | <WRAP todo> |
| * **Nobody has run the clean experiment.** Demir et al. announce one and do not deliver it: §2.2 says "Since the effects of C5 and C11 are not yet adequately discussed by previous work, we analyze them in Section 4", and §4 then runs "four exemplarily case studies focusing on C4, C5, C10, and C12" — repetition, crawler technology, interaction and geolocation. C11, the crawling strategy, is the one criterion they flagged and did not vary; their own runs are described in Appendix C as "stateless coordinated crawls" {[demir2022_reproducibility]}. Sixteen papers in this corpus run both arms, every one of them incidentally to another question. A same-sites, same-time, same-vantage crawl differing //only// in statefulness, reporting the effect on third-party counts, tracker counts and filter-list hit rates, would be a short and highly citable paper. | * **Nobody has run the clean experiment.** Demir et al. announce one and do not deliver it: §2.2 says "Since the effects of C5 and C11 are not yet adequately discussed by previous work, we analyze them in Section 4", and §4 then runs "four exemplarily case studies focusing on C4, C5, C10, and C12" — repetition, crawler technology, interaction and geolocation. C11, the crawling strategy, is the one criterion they flagged and did not vary; their own runs are described in Appendix C as "stateless coordinated crawls" {[demir2022_reproducibility]}. Sixteen papers in this corpus run both arms, every one of them incidentally to another question. A same-sites, same-time, same-vantage crawl differing //only// in statefulness, reporting the effect on third-party counts, tracker counts and filter-list hit rates, would be a short and highly citable paper. |
| * **How much does visit order actually change a stateful result?** The confound is universally acknowledged and never quantified. | * **How much does visit order actually change a stateful result?** The confound is universally acknowledged and never quantified. |
| * **What does statefulness mean under partitioning?** If a stateful crawl's cross-site accumulation is the thing being measured, and every default browser now partitions storage while the research tooling either disables the partitioning (Playwright, OpenWPM — both verified above) or predates it entirely, then the stateful/stateless dichotomy needs a third dimension. No paper in the corpus addresses this, and the ground is still moving: Playwright's opt-out is expected to become unavailable when Chromium removes the flag, at which point every Playwright-based crawl changes behaviour without any change to the paper's own code. | * **What does statefulness mean under partitioning?** If a stateful crawl's cross-site accumulation is the thing being measured, and every default browser now partitions storage while the research tooling either disables the partitioning (Playwright, OpenWPM — both verified above) or predates it entirely, then the stateful/stateless dichotomy needs a third dimension. No paper in the corpus addresses this, and the ground is still moving: Playwright's opt-out is expected to become unavailable when Chromium removes the flag, at which point every Playwright-based crawl changes behaviour without any change to the paper's own code. |
| * **Does the per-browser cookie-jar partition change published results?** Carried over from [[Programming:Crawler:OpenWPM]] because it is the same question: no paper we found reports ''num_browsers'' alongside a stateful claim. | * **Does the per-browser cookie-jar partition change published results?** Carried over from [[Programming:Crawler:OpenWPM]] because it is the same question: no paper we found reports ''num_browsers'' alongside a stateful claim. |
| | </WRAP> |
| ====== References ====== | ====== References ====== |
| |
