| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| programming:stateful_stateless [2026/08/19 11:08] – Tighten two quoted paper claims to the papers' exact wording and name the real denominator of the Rasaii et al. 50% figure (second-half domains with a successfully rejected banner). Authored by Claude karel.kubicek.claude | programming:stateful_stateless [2026/08/31 19:48] (current) – Correct the storageState() sentence: measured on Playwright 1.62.1, it does not carry IndexedDB unless {indexedDB:true} is passed, and never carries sessionStorage, Cache Storage or service-worker registrations. Link Privacy:Browser storage for the stores karel.kubicek.claude |
|---|
| ====== Stateful and Stateless Crawling ====== | ====== Stateful and Stateless Crawling ====== |
| |
| A crawl is **stateless** when the browser starts each visit from an empty profile, and **stateful** when it carries the profile — cookies, ''localStorage'', IndexedDB, the HTTP cache — from one visit to the next. That single switch decides what your crawl is able to observe at all: a stateless crawl cannot see cookie syncing, respawning, retargeting, or the effect of a consent choice on the //next// site, because none of those exist without accumulated state. It also decides how fast you can go, whether your results depend on the order you visited the list in, and how far your numbers can be pushed towards a claim about real users. | A crawl is **stateless** when the browser starts each visit from an empty profile, and **stateful** when it carries the profile — cookies, ''localStorage'', IndexedDB, the HTTP cache — from one visit to the next. That single switch decides what your crawl is able to observe at all: a crawl that starts every visit from an empty profile and visits each target once cannot see retargeting, cookie respawning, or the effect of a consent choice on the //next// site, because none of those exist without accumulated state. It sees cookie //syncing// — a fresh profile is in fact an unusually attractive target for it, see [[#A crawler sees a different web from a user, and statefulness is part of why|below]] — but only first contact, never the sync graph a real user has accumulated. It also decides how fast you can go, whether your results depend on the order you visited the list in, and how far your numbers can be pushed towards a claim about real users. |
| |
| This page is about that design choice. For how one particular tool implements it, see [[Programming:Crawler:OpenWPM#Stateful and stateless in OpenWPM]]; for the tools themselves, [[Programming:Crawler]]; for what to do on the page once you are there, [[Programming:Interaction]] and [[Privacy:Consent]]. | This page is about that design choice. For how one particular tool implements it, see [[Programming:Crawler:OpenWPM#Stateful and stateless in OpenWPM]]; for the tools themselves, [[Programming:Crawler]]; for where to crawl from, [[Design:Crawling location]]; for what to do on the page once you are there, [[Programming:Interaction]] and [[Privacy:Consent]]. |
| |
| <WRAP important> | <WRAP important> |
| Three things a fresh measurement gets wrong most often. | Three things a fresh measurement gets wrong most often. |
| |
| - **Almost nobody reports it.** Of the **1,120 papers in our corpus that ran an automated web crawl, 219 (19.6%) say whether the crawl was stateful** — the second-least-reported crawl-configuration field after headless mode, and the only crawl-configuration reporting rate in this corpus with no trend at all across sixteen years (see [[#Use in Publications]]). | - **Almost nobody reports it.** Of the **1,120 papers in our corpus that ran an automated web crawl, 219 (19.6%) say whether the crawl was stateful** — the second-least-reported crawl-configuration field after headless mode, and the **flattest**: its reporting rate moved 2.5 percentage points between the first and last four-year bucket, against +18.2 for naming a browser and +10.1 for headless (see [[#Reporting has not improved in sixteen years]]). |
| - **"We cleared cookies" is not a reset.** Measured here on Playwright 1.62.1 / Chromium 151: ''context.clearCookies()'' leaves ''localStorage'' intact //and// leaves the HTTP cache warm, so the next visit serves subresources from disk and never touches the origin. See [[#What a reset actually resets]]. | - **"We cleared cookies" is not a reset.** Measured here on Playwright 1.62.1 / Chromium 151: ''context.clearCookies()'' leaves ''localStorage'' intact //and// leaves the HTTP cache warm, so the next visit serves subresources from disk and never touches the origin. See [[#What a reset actually resets]]. |
| - **Since 2022 the browser decides how stateful your stateful crawl is, not you.** Firefox partitions third-party cookies per top-level site by default; Chrome partitions third-party //storage// for every user from Chrome 115 on. Playwright launches Chromium with ''--disable-features=ThirdPartyStoragePartitioning'', Puppeteer does not. Same engine, two different webs. See [[#Since 2022 the engine decides, not you]]. | - **Since 2022 the browser decides how stateful your stateful crawl is, not you.** Firefox partitions third-party cookies per top-level site by default; Chrome partitions third-party //storage// for every user from Chrome 115 on. Playwright launches Chromium with ''--disable-features=ThirdPartyStoragePartitioning'', Puppeteer does not. Same engine, two different webs. See [[#Since 2022 the engine decides, not you]]. |
| Playwright 1.62.1, Chromium 151.0.7922.34, Linux. Reproduce with [[#The code|the script below]]. | Playwright 1.62.1, Chromium 151.0.7922.34, Linux. Reproduce with [[#The code|the script below]]. |
| |
| Read off the three rows in bold. **Clearing cookies clears cookies.** It does not clear ''localStorage'', and it does not clear the HTTP cache — in the ''clearCookies()'' rows the cached subresource was never re-requested from the origin, so a request-counting measurement silently loses it on every visit after the first. Playwright's own documentation is accurate and easy to misread: ''clearCookies'' "removes cookies from context",((Playwright API reference, ''BrowserContext.clearCookies'' and ''BrowserContext.storageState'', checked 2026-08-19.)) and ''storageState'' returns "current cookies, local storage snapshot, IndexedDB snapshot and virtual WebAuthn credentials" — note what is missing from that list, and note that the ''storageState'' row above is the only one where cookies came back while the cache did not. | Read off the three rows in bold. **Clearing cookies clears cookies.** It does not clear ''localStorage'', and it does not clear the HTTP cache — in the ''clearCookies()'' rows the cached subresource was never re-requested from the origin, so a request-counting measurement silently loses it on every visit after the first. Playwright's own documentation is accurate and easy to misread: ''clearCookies'' "Removes cookies from context",((Playwright API reference, ''BrowserContext.clearCookies'' and ''BrowserContext.storageState'', checked 2026-08-19.)) and ''storageState'' returns "current cookies, local storage snapshot, IndexedDB snapshot and virtual WebAuthn credentials" — note what is missing from that list, and note that the ''storageState'' row above is the only one where cookies came back while the cache did not. |
| |
| The cache matters because it is a tracking channel in its own right, not just a performance detail. Solomos et al. showed that Chrome's **favicon cache** is a separate store that browser "clear browsing data" controls do not touch and that persists into incognito {[solomos2021_tales]}; ETag- and cache-based identifiers have the same property. A crawl whose "stateless" guarantee is ''clearCookies()'' is stateful in exactly the channels that were designed to survive a cookie clear. Acar et al. put the general version of this more starkly {[acar2014_never]}: | The cache matters because it is a tracking channel in its own right, not just a performance detail. Solomos et al. showed that Chrome's **favicon cache** is a separate store that browser "clear browsing data" controls do not touch and that persists into incognito {[solomos2021_tales]}; ETag- and cache-based identifiers have the same property. A crawl whose "stateless" guarantee is ''clearCookies()'' is stateful in exactly the channels that were designed to survive a cookie clear. Acar et al. put the general version of this more starkly {[acar2014_never]}: |
| > once some tracking has happened, it is hard to start from a truly clean profile | > once some tracking has happened, it is hard to start from a truly clean profile |
| |
| The complete list of things you should be able to say you reset, or say you did not: cookies (including partitioned ones), ''localStorage'' and ''sessionStorage'', IndexedDB, Cache Storage and service workers, the HTTP disk cache, the favicon cache, HSTS and TLS session state, DNS cache, permission grants, and the extension state of anything you installed. | The complete list of things you should be able to say you reset, or say you did not: cookies (including partitioned ones), ''localStorage'' and ''sessionStorage'', IndexedDB, Cache Storage and service workers, the HTTP disk cache, the favicon cache, HSTS and TLS session state, DNS cache, permission grants, and the extension state of anything you installed. What each of those stores is as a //measurement target// — whether your instrument can read it at all, whether it is partitioned, and what a per-origin reset looks like — is [[Privacy:Browser storage]]. |
| |
| ===== What each design can and cannot measure ===== | ===== What each design can and cannot measure ===== |
| ^ Phenomenon ^ Needs ^ Why ^ Example ^ | ^ Phenomenon ^ Needs ^ Why ^ Example ^ |
| | Third-party presence, request counts, filter-list hit rates | either, but say which | a fresh profile draws //more// third-party traffic than an aged one, so the two are not interchangeable | {[jueckstock2021_realistic]}, {[zeber2020representativeness]} | | | Third-party presence, request counts, filter-list hit rates | either, but say which | a fresh profile draws //more// third-party traffic than an aged one, so the two are not interchangeable | {[jueckstock2021_realistic]}, {[zeber2020representativeness]} | |
| | Cookie syncing / ID sharing | **stateful** | there is no ID to sync until one has been set on a previous site | {[englehardt2016online]}, {[agarwal2020_stop]} | | | Cookie syncing / ID sharing | **either, but they measure different things** | a fresh profile sees //first-contact// syncing and over-triggers it {[zeber2020representativeness]}; reconstructing the sync graph of an aged identity, or how much history a partner can merge, needs accumulation | {[englehardt2016online]} runs its sync analysis on the //stateful// 100k crawl; {[agarwal2020_stop]}, {[acar2014_never]} | |
| | Cookie respawning, evercookies | **stateful** across a clear | the phenomenon //is// state surviving a reset | {[acar2014_never]}; detectable from a stateless harness by comparing paired visits {[fouad2022my]} | | | Cookie respawning, evercookies | **stateful** across a clear | the phenomenon //is// state surviving a reset | {[acar2014_never]}; detectable from a stateless harness by comparing paired visits {[fouad2022my]} | |
| | Ad retargeting, personalisation, differential pricing | **stateful** training, then usually seeded-stateless measurement | the profile is the independent variable | {[bashir2016tracing]}, {[agarwal2020_stop]}, {[liu2024_opted]}, {[meng2014_pollution]}, {[robertson2018_auditing]} | | | Ad retargeting, personalisation, differential pricing | **stateful** training, then usually seeded-stateless measurement | the profile is the independent variable | {[bashir2016tracing]}, {[agarwal2020_stop]}, {[liu2024_opted]}, {[meng2014_pollution]}, {[robertson2018_auditing]} | |
| | Effect of a consent choice on //other// sites | **stateful** | the consent decision only travels via stored state | {[rasaii2025_crumbs]} | | | Effect of a consent choice on //other// sites | **stateful** | the consent decision only travels via stored state | {[rasaii2025_crumbs]} | |
| | Consent revocation, opt-out persistence | **stateful** within a session at minimum | you must be in the consented state before you can revoke it | {[kancherla2025_revoke]}, {[liu2024_opted]} | | | Consent revocation, opt-out persistence | **stateful** within a session at minimum | you must be in the consented state before you can revoke it | {[kancherla2025_johnny]}, {[liu2024_opted]} | |
| | Logged-in versus anonymous web | **stateful** (a session) | the session cookie //is// the state | {[kaizer2016_characterizing]}, {[rautenstrauch2024_auth]}, {[rautenstrauch2023_leaky]} | | | Logged-in versus anonymous web | **stateful** (a session) | the session cookie //is// the state | {[kaizer2016_characterizing]}, {[rautenstrauch2024_auth]}, {[rautenstrauch2023_leaky]} | |
| | First-party-cookie abuse for cross-site tracking | **stateful** | the abuse is the reuse of a first-party value elsewhere | {[chen2021_cookieswap]} | | | First-party-cookie abuse for cross-site tracking | **stateful** | the abuse is the reuse of a first-party value elsewhere | {[chen2021_cookieswap]} | |
| | Cache-based attacks and leaks | **stateful cache**, explicitly primed | an empty cache is the negative control, not the experiment | {[mirheidari2022_cache]}, {[solomos2021_tales]} | | | Cache-based attacks and leaks | **the cache is the state**, and its contents have to be controlled per URL | a cache hit and a cache miss are the two outcomes you are distinguishing, so "we cleared state" without saying whether the cache was cleared makes the result unreadable | {[mirheidari2022_cache]}, {[solomos2021_tales]} | |
| | Effect of a blocker or a setting | **either**, but the //same// for both arms | a blocker's effectiveness depends on how much history it has learned from | {[matthews2018_addons]}, {[jueckstock2022_privacy]} | | | Effect of a blocker or a setting | **either**, but the //same// for both arms | a blocker's effectiveness depends on how much history it has learned from | {[matthews2018_addons]}, {[jueckstock2022_privacy]} | |
| | Anything you want to parallelise over a million sites | **stateless** | see the next section | {[englehardt2016online]} | | | Anything you want to parallelise over a million sites | **stateless** | see the next section | {[englehardt2016online]} | |
| ^ Metric, on the same site domains ^ Human users ^ Crawler ^ | ^ Metric, on the same site domains ^ Human users ^ Crawler ^ |
| | median third-party domains per visit | 4.5 | **11.6** | | | median third-party domains per visit | 4.5 | **11.6** | |
| | median third-party domains per visit, popularity-weighted | 2.9 | 11.6 | | | median third-party domains per visit, popularity-weighted | 2.9((The paper gives the human median "dropping 35% to 2.9" under popularity weighting and describes the crawler distribution only as "similar", so no separate weighted crawler median is quoted here.)) | //not restated// | |
| | median tracking domains per visit (Disconnect list) | 1.9 | **6.1** | | | median tracking domains per visit (Disconnect list) | 1.9 | **6.1** | |
| | trackers reached in 99% of visits | up to 8 | up to **26** | | | trackers reached | "up to 8 … in 99% of visits" | "the crawler may reach **26**"((The paper's framing, verbatim, on both sides; it gives a percentile for users and no matching percentile for the crawler, so the two are not strictly comparable.)) | |
| | Jaccard similarity of the third-party //sets// | median **20%** | — | | | Jaccard similarity of the third-party //sets// | median **20%** | — | |
| |
| |
| * **Consent given on one site follows you to the next.** Rasaii et al. accepted banners across the first half of Tranco's top 20,000 and then measured the second half with that profile loaded — the denominator for the headline figure is the second-half domains where a banner was successfully rejected, not all 20,000. "Our findings reveal that around 50% of websites send at least one intractable cookie" — a tracking cookie transmitted before any consent on the site sending it. Sites with a CMP banner sent **6.91×** more of them than sites with a native banner; enabling Global Privacy Control cut them by about **30%**, with a further **32%** on later visits after rejecting; and about **25%** stop being sent only after the page is reloaded {[rasaii2025_crumbs]}. Partitioning does not yet blunt this: "only 1.3% of all unique tracking cookies are partitioned, with more than half accompanied by nonpartitioned cookies from the same tracker domain". | * **Consent given on one site follows you to the next.** Rasaii et al. accepted banners across the first half of Tranco's top 20,000 and then measured the second half with that profile loaded — the denominator for the headline figure is the second-half domains where a banner was successfully rejected, not all 20,000. "Our findings reveal that around 50% of websites send at least one intractable cookie" — a tracking cookie transmitted before any consent on the site sending it. Sites with a CMP banner sent **6.91×** more of them than sites with a native banner; enabling Global Privacy Control cut them by about **30%**, with a further **32%** on later visits after rejecting; and about **25%** stop being sent only after the page is reloaded {[rasaii2025_crumbs]}. Partitioning does not yet blunt this: "only 1.3% of all unique tracking cookies are partitioned, with more than half accompanied by nonpartitioned cookies from the same tracker domain". |
| * **Respawning plus syncing survives a state clear.** Acar et al. found "33 different Flash cookies from 30 different domains respawned a total of 355 cookies on 107 first party domains", and concluded that through one ad exchange present on ~11% of first parties, "This scenario enables at least 11% of a user's history to be tracked over time" {[acar2014_never]}. | * **Respawning plus syncing survives a state clear.** The 2014 mechanism is historical — Flash reached end of life in December 2020 — but the finding is the reason a state clear cannot be assumed to work, and the technique moved to fingerprint-keyed respawning rather than disappearing ({[fouad2022my]} in 2022, server-side in {[fouad2024_devil]} in 2024). Acar et al. found "33 different Flash cookies from 30 different domains respawned a total of 355 cookies on 107 first party domains", and concluded that through one ad exchange present on ~11% of first parties, "This scenario enables at least 11% of a user's history to be tracked over time" {[acar2014_never]}. |
| * **A trained profile is treated differently.** Agarwal et al.: "having an established persona from a particular demographic … results in up to 15% more cookies stored than for a baseline with no set persona" {[agarwal2020_stop]}. | * **A trained profile is treated differently.** Agarwal et al.: "having an established persona from a particular demographic … results in up to 15% more cookies stored than for a baseline with no set persona" {[agarwal2020_stop]}. |
| * **State accumulates within a site, not only across sites.** Urban et al., under a seeded-stateless design with state kept across a site's own subpages: "subsites set considerably more (36 %) cookies than the respective landing pages. On average, 55 cookies were set when loading a landing page while 78 were set when a subsite was accessed" {[urban2020beyond]}. See [[Programming:Interaction]]. | * **State accumulates within a site, not only across sites.** Urban et al., under a seeded-stateless design with state kept across a site's own subpages: "subsites set considerably more (36 %) cookies than the respective landing pages. On average, 55 cookies were set when loading a landing page while 78 were set when a subsite was accessed" {[urban2020beyond]}. See [[Programming:Interaction]]. |
| |
| And one that cuts the other way: cookie respawning was measured on **30,000 Alexa sites with a stateless harness**, by comparing paired visits rather than by accumulating a profile — 1,150 sites (3.83%) respawned {[fouad2022my]}. A stateful //phenomenon// does not always require a stateful //crawl//; sometimes it requires two controlled visits. | And one that cuts the other way: **cookie respawning with browser fingerprinting** was measured on 30,000 Alexa sites with a stateless harness, by comparing paired visits rather than by accumulating a profile — "1, 150 (3.83%) of the Alexa top 30, 000 websites use cookie respawning with browser fingerprinting" {[fouad2022my]}. Note the qualifier: that figure counts respawning //combined with// fingerprinting, not respawning in general. A stateful //phenomenon// does not always require a stateful //crawl//; sometimes it requires two controlled visits. The same is true of cookie syncing at first contact. What accumulation buys is the //aged// identity, not the mechanism. |
| |
| ===== Since 2022 the engine decides, not you ===== | ===== Since 2022 the engine decides, not you ===== |
| * **Your crawler probably turns the partitioning off.** Playwright launches Chromium with ''--disable-features=…,ThirdPartyStoragePartitioning,…''; Puppeteer does not. Measured here: | * **Your crawler probably turns the partitioning off.** Playwright launches Chromium with ''--disable-features=…,ThirdPartyStoragePartitioning,…''; Puppeteer does not. Measured here: |
| |
| ^ Chromium feature ^ Playwright 1.62.1 ^ Puppeteer 25.5.0 ^ | ^ Chromium feature ^ Playwright 1.62.1 ^ Puppeteer 25.5.0((The same list in Puppeteer 25.8.0, the latest release as of 2026-08-19, is byte-identical, so the comparison is not an artefact of the pinned version.)) ^ |
| | ''ThirdPartyStoragePartitioning'' | **disabled** | left on | | | ''ThirdPartyStoragePartitioning'' | **disabled** | left on | |
| | ''HttpsUpgrades'' | **disabled** | left on | | | ''HttpsUpgrades'' | **disabled** | left on | |
| | ''Translate'' | disabled | disabled | | | ''Translate'' | disabled | disabled | |
| |
| Playwright's own source names the reason — a Playwright issue, not a research decision.((''playwright-core'' 1.62.1, the default ''--disable-features'' list, with the inline comment pointing at ''github.com/microsoft/playwright/issues/32230''. Reproduce with ''scripts/state_probe/launch_args.mjs''.)) The consequence is that two crawlers driving the same engine version accumulate different third-party state, on the one axis this page is about, and neither documents it where you would look. | Playwright's own source names the reason, and it is directly about state-carrying: the flag is disabled so that ''storageState'' keeps working. Issue 32230 — "Local storage items set via ''browser.newContext()'' missing for an iframe in Chromium" — was //fixed// by turning partitioning off, and a 2025 request to turn it back on was declined with: |
| | |
| | > our current capabilities of saving/restoring the storage are not exactly compatible with partitioning … Without CDP support, it does not seem practical to replicate all the intricate details of storage partitioning outside of the browser, so disabling the feature is the only way to make things work for now |
| | |
| | ((Playwright maintainer, ''github.com/microsoft/playwright/issues/38455'' ("Enable storage partitioning and consider expanding storage state API to support storage keys", opened 2025-12-05), comment of 2025-12-09. The issue was closed on 2025-12-22 after the corresponding Chromium request, ''crbug.com/468317746'', was closed as "infeasible - too far outside of the product scope". Issue 32230 was closed 2024-09-27, fixed by PR 32701, "fix(chromium): disable ThirdPartyStoragePartitioning", merged 2024-09-19. All checked 2026-08-19.)) |
| | |
| | So the consequence is that two crawlers driving the same engine version accumulate different third-party state, on the one axis this page is about, and the divergence exists because one of them has a profile-serialisation API that cannot express partitioned storage. Neither documents this where you would look. |
| | |
| | **And it is a moving target.** In the same thread, the person who filed the request notes that "when the ThirdPartyStoragePartitioning flag is removed, bug #32230 will start reoccurring" — that is, Playwright's opt-out is expected to stop being available, and as late as January 2026 the maintainers were still asking the reporter for a design that would keep ''storageState'' working without the flag. Whenever that lands, a Playwright crawl starts accumulating //partitioned// storage with no change to your code, at whatever version boundary it happens on. Pin and report the Playwright version alongside the statefulness claim. |
| |
| * **OpenWPM also opts out, by default.** ''BrowserParams.tp_cookies'' defaults to ''"always"'', which sets ''network.cookie.cookieBehavior = 0'' — all third-party cookies allowed, unpartitioned. Firefox tracking protection cannot be switched on at all: the code raises ''RuntimeError("Firefox Tracking Protection is not currently supported")''.((OpenWPM ''openwpm/config.py'' and ''openwpm/deploy_browsers/configure_firefox.py'', read at ''master'' commit ''b9dd4c3a'' (2026-07-02); latest release ''v0.35.0'' (2026-06-17). Checked 2026-08-19.)) | * **OpenWPM also opts out, by default.** ''BrowserParams.tp_cookies'' defaults to ''"always"'', which sets ''network.cookie.cookieBehavior = 0'' — all third-party cookies allowed, unpartitioned. Firefox tracking protection cannot be switched on at all: the code raises ''RuntimeError("Firefox Tracking Protection is not currently supported")''.((OpenWPM ''openwpm/config.py'' and ''openwpm/deploy_browsers/configure_firefox.py'', read at ''master'' commit ''b9dd4c3a'' (2026-07-02); latest release ''v0.35.0'' (2026-06-17). Checked 2026-08-19.)) |
| |
| So the plain fact is that a stateful research crawl in 2026 accumulates an **unpartitioned** cross-site profile that resembles a default Chrome user's and does //not// resemble a default Firefox or Safari user's — and it does so whichever engine you drive, because the research tooling disables the partitioning. Whether that is the right choice depends on your question; it is never the right thing to leave unsaid. Measured, on the default Playwright Chromium context: | So the plain fact is that a stateful research crawl in 2026 accumulates an **unpartitioned** cross-site profile. That resembles a default Chrome user's //cookie jar// — but not that user's storage, which Chrome has partitioned since 115 — and it does not resemble a default Firefox or Safari user in either respect. And it happens whichever engine you drive, because the research tooling disables the partitioning. Whether that is the right choice depends on your question; it is never the right thing to leave unsaid. Measured, on the default Playwright Chromium context: |
| |
| <code> | <code> |
| ==== Stateful ==== | ==== Stateful ==== |
| |
| * **Playwright:** ''chromium.launchPersistentContext(userDataDir)'' and reuse ''userDataDir''. Everything persists, including the cache. To carry a profile //deliberately and legibly// instead, use ''storageState()'' — it serialises cookies, ''localStorage'' and IndexedDB to JSON you can commit as an artefact, which makes the seed reproducible in a way a binary profile directory is not. It does not carry the HTTP cache. | * **Playwright:** ''chromium.launchPersistentContext(userDataDir)'' and reuse ''userDataDir''. Everything persists, including the cache. To carry a profile //deliberately and legibly// instead, use ''storageState()'' — it serialises cookies and ''localStorage'' to JSON you can commit as an artefact, which makes the seed reproducible in a way a binary profile directory is not. **IndexedDB is behind an option that is off by default** (''storageState({ indexedDB: true })''), and ''sessionStorage'', Cache Storage and service-worker registrations are not in it at all; measured, on Playwright 1.62.1, in [[Privacy:Browser storage#What each capture method actually returns]]. It does not carry the HTTP cache. |
| | * **Puppeteer:** ''puppeteer.launch({ userDataDir })'' and reuse the directory — the same mechanism as Playwright's persistent context, and worth knowing because Puppeteer is the more used of the two in this corpus (76 crawling papers against Playwright's 34). Puppeteer has no ''storageState'' equivalent, so a legible seed means either shipping the profile directory or writing your own cookie/storage dump. |
| * **OpenWPM:** stateful is the **default**, and stateless is per-command-sequence: ''CommandSequence(url, reset=True)'', documented as "True if browser should clear state and restart after sequence".((OpenWPM ''openwpm/command_sequence.py'' at ''master'' commit ''b9dd4c3a'', checked 2026-08-19.)) There is no global switch, which is why papers describe this in prose and reviewers cannot check it. Watch ''num_browsers'': with ''N'' browsers your "stateful crawl" is ''N'' cookie jars. Details on [[Programming:Crawler:OpenWPM#Stateful and stateless in OpenWPM]]. | * **OpenWPM:** stateful is the **default**, and stateless is per-command-sequence: ''CommandSequence(url, reset=True)'', documented as "True if browser should clear state and restart after sequence".((OpenWPM ''openwpm/command_sequence.py'' at ''master'' commit ''b9dd4c3a'', checked 2026-08-19.)) There is no global switch, which is why papers describe this in prose and reviewers cannot check it. Watch ''num_browsers'': with ''N'' browsers your "stateful crawl" is ''N'' cookie jars. Details on [[Programming:Crawler:OpenWPM#Stateful and stateless in OpenWPM]]. |
| * **Seeding:** build the seed in a separate, documented run; store it (''storageState'' JSON, or OpenWPM's ''seed_tar''); record when it was built and what it visited. A seed profile ages — 2016's top 10,000 sites are not 2026's, and a seed built before a crawl that ran for three weeks is not the same instrument at the end as at the start. | * **Seeding:** build the seed in a separate, documented run; store it (''storageState'' JSON, or OpenWPM's ''seed_tar''); record when it was built and what it visited. A seed profile ages — 2016's top 10,000 sites are not 2026's, and a seed built before a crawl that ran for three weeks is not the same instrument at the end as at the start. |
| ==== The code ==== | ==== The code ==== |
| |
| This is the script behind the [[#What a reset actually resets|reset table]]. It needs nothing but Playwright and a free port. | This is the **complete** script behind the [[#What a reset actually resets|reset table]] — all nine reset strategies, so every row is reproducible. Save the two files side by side as ''server.mjs'' and ''probe.mjs''; the second imports the first. It needs nothing but Playwright and a free port. |
| |
| <file javascript state_probe_server.mjs> | <file javascript server.mjs> |
| // Minimal instrumented origin: counts every request it receives and reports what | // Minimal instrumented origin for the state-channel probe. Counts every |
| // the client sent back. | // request it receives, per path, and reports what the client sent back. |
| import http from 'node:http'; | import http from 'node:http'; |
| |
| const log = []; | const log = []; |
| const server = http.createServer((req, res) => { | const server = http.createServer((req, res) => { |
| log.push({ path: req.url, cookie: req.headers.cookie ?? null }); | log.push({ path: req.url, cookie: req.headers.cookie ?? null, ims: req.headers['if-none-match'] ?? null }); |
| if (req.url === '/') { | if (req.url === '/') { |
| res.writeHead(200, { | res.writeHead(200, { |
| window.__lsBefore = localStorage.getItem('ls'); | window.__lsBefore = localStorage.getItem('ls'); |
| if (!window.__lsBefore) localStorage.setItem('ls', 'visit-' + Date.now()); | if (!window.__lsBefore) localStorage.setItem('ls', 'visit-' + Date.now()); |
| </script>`); | </script> |
| | <img src="/cached.png">`); |
| return; | return; |
| } | } |
| if (req.url === '/cached.js') { | if (req.url === '/cached.js') { |
| // Aggressively cacheable: a second visit should not hit the network. | // Aggressively cacheable: a second visit should not hit the network. |
| res.writeHead(200, { | res.writeHead(200, { 'content-type': 'application/javascript', 'cache-control': 'public, max-age=31536000, immutable' }); |
| 'content-type': 'application/javascript', | |
| 'cache-control': 'public, max-age=31536000, immutable', | |
| }); | |
| res.end('window.__cached = true;'); | res.end('window.__cached = true;'); |
| | return; |
| | } |
| | if (req.url === '/cached.png') { |
| | res.writeHead(200, { 'content-type': 'image/png', 'cache-control': 'public, max-age=31536000, immutable' }); |
| | res.end(Buffer.from('89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c4890000000a49444154789c6300010000050001', 'hex')); |
| return; | return; |
| } | } |
| </file> | </file> |
| |
| <file javascript state_probe.mjs> | <file javascript probe.mjs> |
| // What each "reset" actually resets. | // What each "reset" actually resets. Drives Playwright's own Chromium against a |
| | // local instrumented origin and reports, for each reset strategy, whether the |
| | // second visit still carried a cookie, still had localStorage, and still served |
| | // the cacheable subresource from cache instead of the network. |
| | // |
| // npm i playwright && npx playwright install chromium | // npm i playwright && npx playwright install chromium |
| // node state_probe.mjs | // node scripts/state_probe/probe.mjs |
| | // |
| | // Read as: a ✓ under "cookie", "localStorage" or "cache hit" means state |
| | // SURVIVED the reset. A stateless crawl needs all three to be ✗. |
| import fs from 'node:fs'; | import fs from 'node:fs'; |
| import os from 'node:os'; | import os from 'node:os'; |
| import path from 'node:path'; | import path from 'node:path'; |
| import { chromium } from 'playwright'; | import { chromium } from 'playwright'; |
| import { startServer } from './state_probe_server.mjs'; | import { startServer } from './server.mjs'; |
| |
| const PORT = 8123; | const PORT = 8123; |
| const TARGET = `http://127.0.0.1:${PORT}/`; // do NOT name this URL: it shadows the global | const TARGET = `http://127.0.0.1:${PORT}/`; |
| const { server, log } = await startServer(PORT); | const { server, log } = await startServer(PORT); |
| | |
| const tmp = () => fs.mkdtempSync(path.join(os.tmpdir(), 'pw-profile-')); | const tmp = () => fs.mkdtempSync(path.join(os.tmpdir(), 'pw-profile-')); |
| |
| | // Visit the page and report what the origin saw and what the page found. |
| async function visit(page) { | async function visit(page) { |
| const before = log.length; | const before = log.length; |
| cookieSentOnDoc: hits.find((h) => h.path === '/')?.cookie ?? null, | cookieSentOnDoc: hits.find((h) => h.path === '/')?.cookie ?? null, |
| cachedJsFromNetwork: hits.some((h) => h.path === '/cached.js'), | cachedJsFromNetwork: hits.some((h) => h.path === '/cached.js'), |
| | cachedPngFromNetwork: hits.some((h) => h.path === '/cached.png'), |
| // __lsBefore is what the page found BEFORE writing its own marker, so it is | // __lsBefore is what the page found BEFORE writing its own marker, so it is |
| // non-null only when localStorage genuinely survived into this visit. | // non-null only when localStorage genuinely survived into this visit. |
| lsCarriedIn: await page.evaluate(() => window.__lsBefore ?? null), | lsCarriedIn: await page.evaluate(() => window.__lsBefore ?? null), |
| | lsNow: await page.evaluate(() => localStorage.getItem('ls')), |
| | jarCookies: (await page.context().cookies()).map((c) => c.name).sort().join(','), |
| }; | }; |
| } | } |
| |
| const mark = (b) => (b ? 'survived' : 'gone'); | const mark = (b) => (b ? '✓' : '✗'); |
| const results = []; | const results = []; |
| const record = (strategy, second) => | const record = (strategy, second) => |
| results.push([strategy, mark(!!second.cookieSentOnDoc), mark(second.lsCarriedIn !== null), | results.push({ |
| mark(!second.cachedJsFromNetwork)]); | strategy, |
| | cookie: mark(!!second.cookieSentOnDoc), |
| | localStorage: mark(second.lsCarriedIn !== null), |
| | cacheHit: mark(!second.cachedJsFromNetwork), |
| | detail: |
| | `visit 2 sent Cookie: ${second.cookieSentOnDoc ?? '(none)'}; ` + |
| | `localStorage carried in: ${second.lsCarriedIn ?? '(none)'}; ` + |
| | `/cached.js re-requested from origin: ${second.cachedJsFromNetwork ? 'yes' : 'no'}`, |
| | }); |
| |
| { // clearCookies() only | // 1. Same page object, second navigation. |
| | { |
| | const b = await chromium.launch(); |
| | const c = await b.newContext(); |
| | const p = await c.newPage(); |
| | await visit(p); |
| | record('nothing — second page.goto() in the same context', await visit(p)); |
| | await b.close(); |
| | } |
| | // 2. New page in the same context. |
| | { |
| | const b = await chromium.launch(); |
| | const c = await b.newContext(); |
| | await visit(await c.newPage()); |
| | record('context.newPage()', await visit(await c.newPage())); |
| | await b.close(); |
| | } |
| | // 3. context.clearCookies() only. |
| | { |
| const b = await chromium.launch(); | const b = await chromium.launch(); |
| const c = await b.newContext(); | const c = await b.newContext(); |
| await b.close(); | await b.close(); |
| } | } |
| { // a new context in the same browser process | // 4. clearCookies() + clearPermissions() (the usual "we cleared cookies" claim). |
| | { |
| const b = await chromium.launch(); | const b = await chromium.launch(); |
| await visit(await (await b.newContext()).newPage()); | const c = await b.newContext(); |
| record('browser.newContext()', await visit(await (await b.newContext()).newPage())); | const p = await c.newPage(); |
| | await visit(p); |
| | await c.clearCookies(); |
| | await c.clearPermissions(); |
| | record('clearCookies() + clearPermissions()', await visit(p)); |
| await b.close(); | await b.close(); |
| } | } |
| { // the same persistent profile, relaunched, cookies cleared | // 5. New browser CONTEXT in the same browser process. |
| | { |
| | const b = await chromium.launch(); |
| | const c1 = await b.newContext(); |
| | await visit(await c1.newPage()); |
| | const c2 = await b.newContext(); |
| | record('browser.newContext() — new context, same browser process', await visit(await c2.newPage())); |
| | await b.close(); |
| | } |
| | // 6. Fresh browser.launch() — Playwright's non-persistent default. |
| | { |
| | const b1 = await chromium.launch(); |
| | await visit(await (await b1.newContext()).newPage()); |
| | await b1.close(); |
| | const b2 = await chromium.launch(); |
| | record('fresh chromium.launch() (Playwright default, no user-data-dir)', await visit(await (await b2.newContext()).newPage())); |
| | await b2.close(); |
| | } |
| | // 7. launchPersistentContext, same user-data-dir, relaunched. |
| | { |
| | const dir = tmp(); |
| | const c1 = await chromium.launchPersistentContext(dir); |
| | await visit(await c1.newPage()); |
| | await c1.close(); |
| | const c2 = await chromium.launchPersistentContext(dir); |
| | record('launchPersistentContext() twice on the SAME user-data-dir', await visit(await c2.newPage())); |
| | await c2.close(); |
| | } |
| | // 8. launchPersistentContext, same dir, clearCookies() in between. |
| | { |
| const dir = tmp(); | const dir = tmp(); |
| const c1 = await chromium.launchPersistentContext(dir); | const c1 = await chromium.launchPersistentContext(dir); |
| const c2 = await chromium.launchPersistentContext(dir); | const c2 = await chromium.launchPersistentContext(dir); |
| await c2.clearCookies(); | await c2.clearCookies(); |
| record('same user-data-dir + clearCookies()', await visit(await c2.newPage())); | record('same user-data-dir + clearCookies() on relaunch', await visit(await c2.newPage())); |
| await c2.close(); | await c2.close(); |
| } | } |
| { // storageState round-trip: carrying a profile on purpose | // 9. storageState round-trip: the documented way to carry a profile on purpose. |
| | { |
| const b = await chromium.launch(); | const b = await chromium.launch(); |
| const c1 = await b.newContext(); | const c1 = await b.newContext(); |
| await c1.close(); | await c1.close(); |
| const c2 = await b.newContext({ storageState: state }); | const c2 = await b.newContext({ storageState: state }); |
| record('storageState() saved and reloaded', await visit(await c2.newPage())); | record('storageState() saved and reloaded into a new context', await visit(await c2.newPage())); |
| await b.close(); | await b.close(); |
| | fs.writeFileSync( |
| | path.join(import.meta.dirname, 'storagestate-sample.json'), |
| | JSON.stringify(state, null, 1) |
| | ); |
| } | } |
| |
| for (const r of results) console.log(r.map((c) => String(c).padEnd(38)).join('')); | const W = Math.max(...results.map((r) => r.strategy.length)); |
| | console.log( |
| | ['Reset between visit 1 and visit 2'.padEnd(W), 'cookie', 'localStorage', 'cache hit'].join(' ') |
| | ); |
| | console.log([('-'.repeat(W)), '------', '------------', '---------'].join(' ')); |
| | for (const r of results) |
| | console.log([r.strategy.padEnd(W), r.cookie.padEnd(6), r.localStorage.padEnd(12), r.cacheHit].join(' ')); |
| | console.log('\n✓ = the state SURVIVED the reset and visit 2 saw it. A stateless crawl needs ✗ in all three columns.'); |
| | console.log('"cache hit" ✓ means the immutable subresource was NOT re-requested from the origin.\n'); |
| | for (const r of results) console.log(` ${r.strategy}\n ${r.detail}`); |
| | { |
| | const pkg = JSON.parse( |
| | fs.readFileSync(new URL('./package.json', import.meta.resolve('playwright')), 'utf8') |
| | ); |
| | const b = await chromium.launch(); |
| | console.log(`\nplaywright ${pkg.version}; chromium ${b.version()}; ${process.platform}`); |
| | await b.close(); |
| | } |
| server.close(); | server.close(); |
| </file> | </file> |
| ===== Use in Publications ===== | ===== Use in Publications ===== |
| |
| All figures below are over the **1,120 papers in the corpus that ran an automated web crawl**, out of 5,859 extracted papers from CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026. They are reporting rates: "does not state" means the paper did not say, not that the crawl had no state. Every number here is produced by ''scripts/report_stateful_stateless.mjs''; the full query log is on [[provenance:programming:stateful_stateless]]. | All figures below are over the **1,120 papers in the corpus that ran an automated web crawl**, out of 5,859 extracted papers from CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026. They are reporting rates: "does not state" means the paper did not say, not that the crawl had no state. The figures come from three scripts — ''report_stateful_stateless.mjs'' for the tables, ''statefulness_audit.mjs'' for the 29-paper adjudication, ''statefulness_probe.mjs'' for the text-corroboration counts — and the full query log, with each script's unedited output, is on [[provenance:programming:stateful_stateless]]. |
| |
| ==== Almost nobody says ==== | ==== Almost nobody says ==== |
| | Headless or headful | 140 | 12.5% | | | Headless or headful | 140 | 12.5% | |
| |
| An external cross-check disagrees, informatively. Demir et al. hand-coded 117 web-measurement papers from 2018–2021 against 18 reproducibility criteria; their criterion C11, "describe crawling strategy", derived from exactly this design question, was **omitted by 41%** and fully satisfied by 44% {[demir2022_reproducibility]}. Their 59% "at least partial" is three times our 19.6%. Both can be right: their corpus is 117 papers hand-picked as web measurements from top venues, ours is 1,120 crawling papers inside a broad security corpus and includes a long tail of papers that crawl incidentally; their "crawling strategy" is also read more broadly than the stateful/stateless enum. Treat 19.6% as the rate across everything that crawls and ~59% as the rate among papers whose main contribution is a web measurement. | An external cross-check disagrees, informatively — and it is a close comparison, because it covers **the same seven venues**. Demir et al. hand-coded 117 web-measurement papers from 2016–2021 against 18 reproducibility criteria; their criterion C11, "describe crawling strategy", derived from exactly this design question, was **omitted by 41%**, partially met by 12% and fully satisfied by 44% {[demir2022_reproducibility]}. Partial plus satisfied is **56%**, which is 2.9 times our 19.6%.((The 56% is our arithmetic on their Table 2, not a figure they state. Their categories are N/A 3%, Omit 41%, Undocumented 12%, Satisfied 44%.)) Both can be right, and the reason is not venue coverage: their 117 papers are hand-picked as //web measurements// from those venues, while our 1,120 are every paper the extraction found to have run a crawl, including a long tail that crawls incidentally to something else. Their "crawling strategy" is also read more broadly than the stateful/stateless enum. Treat 19.6% as the rate across everything that crawls in these venues, and ~56% as the rate among papers whose main contribution is a web measurement. |
| |
| ==== Reporting has not improved in sixteen years ==== | ==== Reporting has not improved in sixteen years ==== |
| | 2025–2026* | 36/198 = **18.2%** | 907/1185 = 76.5% | 467/1019 = 45.8% | | | 2025–2026* | 36/198 = **18.2%** | 907/1185 = 76.5% | 467/1019 = 45.8% | |
| |
| Artifact release more than tripled and ethics-review reporting more than quadrupled. Statefulness has sat between 16% and 21% throughout, with no trend. It is not that the field decided the axis does not matter — Demir et al. made it a named criterion in 2022, and Zeber et al. and Jueckstock et al. had made it a measured concern in 2020 and 2021. It is that nothing turned the concern into a reporting norm: no venue asks for it on a checklist, and no widely used tool writes it into a config file that ends up in an artifact. | Artifact release more than tripled and ethics-review reporting more than quadrupled. Statefulness has sat between 16% and 21% throughout, with no trend. It is also the flattest of the crawl-configuration fields, which is the sharper version of the claim because those fields compete for the same paragraph of the same methods section: |
| |
| What //did// move is the answer among those who give one: **the stateless share of stated values rose from 43.8% to about 55–59%**. Read this as the field's default hardening rather than as a swing in practice — the whole cell is small (16 papers in the first bucket), and the modern tooling defaults to stateless. | ^ Field ^ 2010–2013 ^ 2014–2017 ^ 2018–2021 ^ 2022–2024 ^ 2025–2026* ^ max−min ^ last − first ^ |
| | | ''interactionDepth'' | 78.4% | 75.4% | 76.9% | 73.9% | 72.2% | 6.2 pp | **−6.2 pp** | |
| | | ''authentication'' | 59.8% | 70.1% | 69.2% | 72.8% | 69.2% | 12.9 pp | +9.4 pp | |
| | | ''browsers'' (≥1 named) | 32.4% | 48.5% | 49.7% | 47.0% | 50.5% | 18.2 pp | **+18.2 pp** | |
| | | ''consentAction'' | 24.5% | 29.9% | 32.1% | 33.6% | 29.8% | 9.1 pp | +5.3 pp | |
| | | **''statefulness''** | 15.7% | 21.0% | 19.8% | 20.6% | 18.2% | **5.3 pp** | **+2.5 pp** | |
| | | ''headless'' | 1.0% | 15.6% | 13.3% | 14.5% | 11.1% | 14.6 pp | +10.1 pp | |
| | |
| | Denominators are the crawling papers in each bucket, from the table above (102 / 167 / 308 / 345 / 198). Naming the browser gained 18 points and headless mode gained 10 from a near-zero base; statefulness gained 2.5 and has the narrowest **range** of the six (5.3 pp). Interaction depth is the only field whose range is nearly as narrow (6.2 pp), and it got that way by //declining// from 78.4% to 72.2% rather than by standing still. It is not that the field decided the axis does not matter — Demir et al. made it a named criterion in 2022, and Zeber et al. and Jueckstock et al. had made it a measured concern in 2020 and 2021. It is that nothing turned the concern into a reporting norm: no venue asks for it on a checklist, and no widely used tool writes it into a config file that ends up in an artifact. |
| | |
| | What //did// move is the answer among those who give one: **the stateless share of stated values rose from 43.8% into the 50s** (50.8%, 59.2%, 52.8% over the last three buckets). Read this as the field's default hardening rather than as a swing in practice — the whole cell is small (16 papers in the first bucket), and the modern tooling defaults to stateless. |
| |
| ==== By venue ==== | ==== By venue ==== |
| | IEEE-SP | 110 | 23 | 20.9% | 11 | 9 | 3 | | | IEEE-SP | 110 | 23 | 20.9% | 11 | 9 | 3 | |
| |
| PETS is twice as likely to say as any other venue and holds 9 of the 29 both-arms papers on a third of USENIX's crawling volume. USENIX Security is the outlier the other way: 30 of 221. If you are writing for PETS, the reviewers are used to seeing it. | PETS states it at 34.1% — about 60% more often than the next venue (IMC, 21.2%) and two and a half times as often as USENIX Security (13.6%) — and holds 9 of the 29 both-arms papers on just over half of USENIX's crawling volume. PETS is where this reporting norm is strongest. |
| |
| ==== The instrument decides whether you say it ==== | ==== The instrument decides whether you say it ==== |
| | //no framework named// | 397 | 37 | **9.3%** | 22 | 9 | 6 | | | //no framework named// | 397 | 37 | **9.3%** | 22 | 9 | 6 | |
| |
| OpenWPM papers state it at **six times** the rate of papers that do not name a framework, and twice the rate of Selenium papers. The mechanism is not virtue but interface: OpenWPM's ''CommandSequence'' has a ''reset'' argument and its documentation names the choice, so authors have a word for what they did. Selenium hands you a fresh session and no vocabulary. ([[Programming:Crawler:OpenWPM]] reports 33 of 59 (55.9%) using a slightly wider definition of "an OpenWPM paper" — any ''tools[]'' tuple whose name matches OpenWPM, rather than only those the extractor filed under a crawler category. Both numbers are in the report script so the two pages cannot drift.) | OpenWPM papers state it at **six times** the rate of papers that do not name a framework, and twice the rate of Selenium papers. The mechanism is not virtue but interface: OpenWPM's ''CommandSequence'' has a ''reset'' argument and its documentation names the choice, so authors have a word for what they did. Selenium hands you a fresh session and no vocabulary. ([[Programming:Crawler:OpenWPM]] reports 33 of 60 (55.0%) using a wider definition of "an OpenWPM paper" — any ''tools[]'' tuple whose name matches OpenWPM, regardless of the category the extractor filed it under, and without restricting to the crawling population. Applying that name match inside the crawling population gives 33 of 59 (55.9%). Both definitions are computed by this page's report script, so the two pages cannot drift.) |
| |
| ==== Designs whose result cannot be read without it ==== | ==== Designs whose result cannot be read without it ==== |
| | //all crawling papers// | 1,120 | 219 | 19.6% | 113 | 77 | 29 | | | //all crawling papers// | 1,120 | 219 | 19.6% | 113 | 77 | 29 | |
| |
| The consent row needs a caveat that [[privacy:consent]] supplies: the same 36 papers were hand-audited there and **7 (19.4%) turned out to be extraction false positives** — they never touched a banner. The second row recomputes the rate on the 28 that survived that audit, and it barely moves, so the finding is robust to the error. | The consent row needs a caveat that [[privacy:consent]] supplies: the same 36 papers were hand-audited there and **7 (19.4%) turned out to be extraction false positives** — they never touched a banner. The second row recomputes the rate on the 28 the audit **fully** supported (a 29th is supported but with an overstated enum value), and it barely moves, so the finding is robust to the error. |
| |
| The good news first: where the design makes the axis unavoidable, reporting roughly triples. A login is state, and 39 of the 54 login crawls that say anything say stateful. The bad news is the third row: **199 papers visit the same target two or more times and 114 of them (57.3%) never say whether state carried between the visits** — which is the one thing that determines whether the repeat visit is a replication or a second step in a sequence. If you take one reporting rule from this page, take that one. | The good news first: where the design makes the axis unavoidable, reporting roughly triples. A login is state, and 39 of the 54 login crawls that say anything say stateful. The bad news is the repeat-visit row: **199 papers visit the same target two or more times and 114 of them (57.3%) never say whether state carried between the visits** — which is the one thing that determines whether the repeat visit is a replication or a second step in a sequence. If you take one reporting rule from this page, take that one. |
| |
| ==== One word, two literatures, in the data ==== | ==== One word, two literatures, in the data ==== |
| | neither (unmatched residue) | 45 | 22 | 19 | 4 | | | neither (unmatched residue) | 45 | 22 | 19 | 4 | |
| |
| In the tracking literature stateless outnumbers stateful more than two to one; in the scanning literature it is the reverse, because there "state" means the application's own session and database, and coverage depends on reaching it. The 45-paper residue is largely papers that are not web crawls in either sense — an NTP-pool study, an IPv6 study, a 5G performance study — and it is printed in full in the report output so it does not vanish quietly. | In the tracking literature stateless outnumbers stateful more than two to one; in the scanning literature it is the reverse, because there "state" means the application's own session and database, and coverage depends on reaching it. The 45-paper residue is largely papers that are not web crawls in either sense — an NTP-pool robustness study, a carrier-grade-NAT deployment study, a commercial-VPN ecosystem study, a 5G performance study, several underground-marketplace studies — and it is printed in full in the report output so it does not vanish quietly. |
| |
| ==== The comparison studies, audited ==== | ==== The comparison studies, audited ==== |
| | wrong | 3 | 10.3% | no stateful-versus-stateless contrast in the paper at all | | | wrong | 3 | 10.3% | no stateful-versus-stateless contrast in the paper at all | |
| |
| So the corpus holds **16 genuine comparison studies out of 1,120 crawling papers (1.4%)**, not 29 (2.6%). Use the 16 as a reading list and not the 29. Across all 219 stated values, a mechanical text probe finds a state-management sentence in the paper's own text for 178 (81.3%) and none for 41 (18.7%); the shared configuration quote itself contains a state term for only 72 (32.9%), which is the clearest possible demonstration that it is not evidence for this field. The 18.7% is an upper bound on false positives, not a measurement of them — hand-reading showed some are misses by the probe's regex rather than errors in the extraction. Full verdicts and reasoning: [[provenance:programming:stateful_stateless]]. | So the corpus holds **16 genuine comparison studies out of 1,120 crawling papers (1.4%)**, not 29 (2.6%). Use the 16 as a reading list and not the 29. Here they are, with what the two arms actually were: |
| | |
| | ^ Paper ^ The two arms ^ |
| | | {[acar2014_never]} | one sequential crawl keeping profile state, plus parallel crawls that do not | |
| | | {[meng2014_pollution]} | profiles polluted by a CSRF-style attack, against clean profiles replayed from user traces | |
| | | {[pan2015_summer]} | each site visited "once starting with a clean browser and once more after priming the client-side state" | |
| | | {[englehardt2016online]} | //Default Stateless// over 1M sites beside //Default Stateful// over 100k | |
| | | {[matthews2018_addons]} | blockers measured with no browsing history, then again post-calibration | |
| | | {[englehardt2018_email]} | each email loaded twice: fresh profile, then the same profile again | |
| | | {[robertson2018_auditing]} | a standard window and an incognito window driven side by side | |
| | | {[agarwal2020_stop]} | personas trained statefully, then measured stateless | |
| | | {[chen2021_cookieswap]} | repeat visits retaining state alongside fresh-profile visits | |
| | | {[mehrnezhad2022_protect]} | consent accepted on visit two and opted out of on visit three, plus a private-mode arm | |
| | | {[mirheidari2022_cache]} | per-URL cache hit against cache miss, verified for each candidate | |
| | | {[rautenstrauch2023_leaky]} | logged-in state against anonymous, "a fresh browser context that we reset between" | |
| | | {[liu2024_opted]} | personas accumulating over nine iterated visits, against control personas | |
| | | {[rautenstrauch2024_auth]} | the same site crawled twice in parallel, once with a session | |
| | | {[rasaii2025_crumbs]} | banners accepted statefully on the first half of the list, measured on the second | |
| | | {[ablove2026_censorship]} | persistent browser sessions for most services, fresh sessions for the one with a query limit | |
| | |
| | Ten more are labelled ''both'' but contrast something else — a login, a seeded profile, an extension, a consent step — and three have no statefulness contrast at all. Across all 219 stated values, a mechanical text probe finds a state-management sentence in the paper's own text for 178 (81.3%) and none for 41 (18.7%); the shared configuration quote itself contains a state term for only 72 (32.9%), which is the clearest possible demonstration that it is not evidence for this field. The 18.7% is an upper bound on false positives, not a measurement of them — hand-reading showed some are misses by the probe's regex rather than errors in the extraction. Full verdicts and reasoning: [[provenance:programming:stateful_stateless]]. |
| |
| ==== Methodology and limitations of these figures ==== | ==== Methodology and limitations of these figures ==== |
| - **The engine's partitioning posture**: which browser and version, and whether third-party cookie or storage partitioning was on. In 2026 this is not a detail — see [[#Since 2022 the engine decides, not you]]. | - **The engine's partitioning posture**: which browser and version, and whether third-party cookie or storage partitioning was on. In 2026 this is not a detail — see [[#Since 2022 the engine decides, not you]]. |
| |
| One sentence that does all of it: //"OpenWPM 0.35.0 (Firefox 152), stateful with ''num_browsers=1'' and ''tp_cookies="always"'' (third-party cookies allowed, unpartitioned), visit order randomised with seed 20260819, profile dumped after each 1,000 sites and published."// | One sentence that does all of it: //"OpenWPM 0.35.0 (its pinned unbranded Firefox build), stateful with ''num_browsers=1'' and ''tp_cookies="always"'' (third-party cookies allowed, unpartitioned), visit order randomised with seed 20260819, profile dumped after each 1,000 sites and published."// |
| |
| ===== Recommendations ===== | ===== Recommendations ===== |
| |
| - **Default to stateless** unless your question needs accumulation. It parallelises, it is order-independent, and every visit is an independent observation, which is what the statistics on [[Statistics:Hypothesis testing]] assume. | - **Default to stateless** unless your question needs accumulation. It parallelises, it is order-independent, and every visit is an independent observation, which is what the statistics on [[Statistics:Hypothesis testing]] assume. |
| - **Say so anyway.** Getting it by default is not the same as reporting it, and 75.4% of the corpus did not. | - **Say so anyway.** Getting it by default is not the same as reporting it, and 75.4% of the crawling papers in this corpus did not. |
| - **If you need state, prefer seeded-stateless** to a rolling profile. You keep a non-empty starting state and lose the order confound. Publish the seed. | - **If you need state, prefer seeded-stateless** to a rolling profile. You keep a non-empty starting state and lose the order confound. Publish the seed. |
| - **If you need a rolling profile, run one browser** or report how many you ran and how the site list was partitioned across them. | - **If you need a rolling profile, run one browser** or report how many you ran and how the site list was partitioned across them. |
| - **{[agarwal2020_stop]} — train stateful, measure stateless.** The hybrid pattern most personalisation work now uses. | - **{[agarwal2020_stop]} — train stateful, measure stateless.** The hybrid pattern most personalisation work now uses. |
| - **{[song2026_wfpllm]} — the 2026 version of the realism problem.** Models trained on scripted-crawler traffic score under 10% on real users; LLM-agent personas close most of the gap. | - **{[song2026_wfpllm]} — the 2026 version of the realism problem.** Models trained on scripted-crawler traffic score under 10% on real users; LLM-agent personas close most of the gap. |
| - **{[jueckstock2021_realistic]} — the neighbouring axis.** Vantage point and browser configuration, with every crawl launched from "a clean user profile". Often cited as varying statefulness; it does not. | - **{[jueckstock2021_realistic]} — the neighbouring axis.** Vantage point and browser configuration, with every crawl launched from "a clean user profile". Often cited as varying statefulness; it does not. See [[Design:Crawling location]] for that axis. |
| |
| ===== Open Questions ===== | ===== Open Questions ===== |
| |
| * **Nobody has run the clean experiment.** Demir et al. singled out the crawling strategy (C11) as a criterion whose effects "are not yet adequately discussed by previous work" and then varied browser, region, interaction and repetition instead — their own 24 profiles are all stateless {[demir2022_reproducibility]}. Sixteen papers in the corpus run both arms, all incidentally to another question. A same-sites, same-time, same-vantage crawl differing only in statefulness, reporting the effect on third-party counts, tracker counts and filter-list hit rates, would be a short and highly citable paper. | <WRAP todo> |
| | * **Nobody has run the clean experiment.** Demir et al. announce one and do not deliver it: §2.2 says "Since the effects of C5 and C11 are not yet adequately discussed by previous work, we analyze them in Section 4", and §4 then runs "four exemplarily case studies focusing on C4, C5, C10, and C12" — repetition, crawler technology, interaction and geolocation. C11, the crawling strategy, is the one criterion they flagged and did not vary; their own runs are described in Appendix C as "stateless coordinated crawls" {[demir2022_reproducibility]}. Sixteen papers in this corpus run both arms, every one of them incidentally to another question. A same-sites, same-time, same-vantage crawl differing //only// in statefulness, reporting the effect on third-party counts, tracker counts and filter-list hit rates, would be a short and highly citable paper. |
| * **How much does visit order actually change a stateful result?** The confound is universally acknowledged and never quantified. | * **How much does visit order actually change a stateful result?** The confound is universally acknowledged and never quantified. |
| * **Does the seed profile artefact bite?** Englehardt and Narayanan predicted that cloning one seed into ''N'' browsers inflates cookie-sync counts. Nobody has measured the size of the inflation, and every large stateful crawl since uses the design. | * **Does the seed profile artefact bite?** Englehardt and Narayanan predicted that cloning one seed into ''N'' browsers inflates cookie-sync counts. Nobody has measured the size of the inflation, and it is the design large stateful crawls have used since. |
| * **What does statefulness mean under partitioning?** If a stateful crawl's cross-site accumulation is the thing being measured, and every default browser now partitions storage while every research crawler disables the partitioning, then the stateful/stateless dichotomy needs a third dimension. No paper in the corpus addresses this. | * **What does statefulness mean under partitioning?** If a stateful crawl's cross-site accumulation is the thing being measured, and every default browser now partitions storage while the research tooling either disables the partitioning (Playwright, OpenWPM — both verified above) or predates it entirely, then the stateful/stateless dichotomy needs a third dimension. No paper in the corpus addresses this, and the ground is still moving: Playwright's opt-out is expected to become unavailable when Chromium removes the flag, at which point every Playwright-based crawl changes behaviour without any change to the paper's own code. |
| * **Does the per-browser cookie-jar partition change published results?** Carried over from [[Programming:Crawler:OpenWPM]] because it is the same question: no paper we found reports ''num_browsers'' alongside a stateful claim. | * **Does the per-browser cookie-jar partition change published results?** Carried over from [[Programming:Crawler:OpenWPM]] because it is the same question: no paper we found reports ''num_browsers'' alongside a stateful claim. |
| | </WRAP> |
| ====== References ====== | ====== References ====== |
| |
