programming:stateful_stateless
Differences
This shows you the differences between two versions of the page.
| Next revision | Previous revision | ||
| programming:stateful_stateless [2025/03/18 14:20] – filled with references karelkubicek | programming:stateful_stateless [2026/08/31 19:48] (current) – Correct the storageState() sentence: measured on Playwright 1.62.1, it does not carry IndexedDB unless {indexedDB:true} is passed, and never carries sessionStorage, Cache Storage or service-worker registrations. Link Privacy:Browser storage for the stores karel.kubicek.claude | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== Stateful and Stateless Crawling ====== | ====== Stateful and Stateless Crawling ====== | ||
| - | <wrap todo> | + | A crawl is **stateless** when the browser starts each visit from an empty profile, and **stateful** when it carries the profile — cookies, '' |
| + | This page is about that design choice. For how one particular tool implements it, see [[Programming: | ||
| - | Key message: | + | <WRAP important> |
| - | * Majority of web measurements studies use stateless crawls, as it is easy to associate events with the single browsed visited website. Also, stateless crawls do not depend on crawling order and are easier to parallelize. | + | Three things a fresh measurement gets wrong most often. |
| - | * Stateful crawling is however more representative of real users, that rarely clear their browser state. | + | |
| - | ===== Relevant Literature ===== | + | - **Almost nobody reports it.** Of the **1,120 papers in our corpus that ran an automated web crawl, 219 (19.6%) say whether the crawl was stateful** — the second-least-reported crawl-configuration field after headless mode, and the **flattest**: |
| + | - **"We cleared cookies" | ||
| + | - **Since 2022 the browser decides how stateful your stateful crawl is, not you.** Firefox partitions third-party cookies per top-level site by default; Chrome partitions third-party //storage// for every user from Chrome 115 on. Playwright launches Chromium with '' | ||
| + | </ | ||
| - | Since the majority of publication uses stateless crawling, below we list examples of influential publications doing otherwise. However, not all contribute specifically to the question of difference between stateful and stateless crawling. | + | ===== Two words, two literatures ===== |
| + | Before anything else: **" | ||
| - | * [[https:// | + | * **Crawl statefulness** — this page. A property of //your measurement//: does the browser profile survive between page visits. |
| - | * Comparison | + | * **Tracking statefulness** — a property |
| - | * Stateless crawls surprisingly | + | |
| - | | + | |
| - | * The most impactful publication utilizing | + | |
| - | * [[https://www.usenix.org/system/files/ | + | |
| - | * Created " | + | |
| - | * The web never forgets: Persistent tracking mechanisms in the wild | + | |
| - | * Cookies that give you away: The surveillance implications of web tracking | + | |
| - | ==== Studies | + | This is not a pedantic distinction. It is a live source |
| - | The following studies used stateless crawls, but were interpreting some stateful properties of web: | + | ===== The axis has three positions, not two ===== |
| + | The literature' | ||
| - | | + | ^ Design ^ What happens between visits ^ What it buys ^ What it costs ^ |
| - | * Evaluation of cookie respawning | + | | **Stateless** | profile discarded and recreated | order-independence; |
| + | | **Stateful** | profile carried forward | cookie syncing, respawning, retargeting, | ||
| + | | **Seeded stateless** | a **fixed, pre-built** profile is loaded before every visit and //not// written back | a non-empty starting state that is identical for every site, so order still does not matter | the seed goes stale, and nothing the crawl learns is carried forward | | ||
| - | ==== Shallow vs Deep crawling ==== | + | The seeded-stateless design is the one most often mistaken for stateful. Urban et al. state its logic exactly {[urban2020beyond]}: |
| - | * [[https:// | + | > This profile is loaded before each website visit but is not altered. Hence, each website visit uses the same profile and the order of visited websites does not impact the results. |
| - | * Comparison | + | |
| - | * Visiting sub-pages increases amount of tracking significantly | + | |
| - | * [[https:// | + | |
| - | /* | + | Agarwal et al. combine |
| - | This is a comment not visible on the page. It outlines | + | |
| - | ===== Header Level 2 ===== | + | You also have to say **what the unit of the reset is**, and papers almost never do: |
| - | ==== Header Level 3 ==== | + | |
| - | === Links === | + | * per **page visit** (strictest stateless); |
| + | * per **site**, keeping state across that site's subpages — which is what a subpage crawl usually means in practice; | ||
| + | * per **browser instance**, so a crawl with '' | ||
| + | * per **crawl**, never. | ||
| - | External links are recognized automatically: | + | ===== What a reset actually resets ===== |
| - | Internal links are created by using square brackets. You can either just give a [[pagename]] or use an additional [[pagename|link text]]. | + | "We cleared the browser state between visits" |
| - | === Lists === | + | ^ Reset between visit 1 and visit 2 ^ cookie ^ '' |
| + | | nothing — a second '' | ||
| + | | '' | ||
| + | | '' | ||
| + | | '' | ||
| + | | '' | ||
| + | | a fresh '' | ||
| + | | '' | ||
| + | | same '' | ||
| + | | '' | ||
| - | Lists and their levels are decided by indentation (2 spaces = 1 level) | + | Playwright |
| - | - this is 1. item | + | Read off the three rows in bold. **Clearing cookies clears cookies.** It does not clear '' |
| - | - 2. item | + | |
| - | - nested | + | |
| - | * bullet-point item | + | |
| - | === Code === | + | The cache matters because it is a tracking channel in its own right, not just a performance detail. Solomos et al. showed that Chrome' |
| - | For a short inline monospace, use '' | + | > once some tracking has happened, it is hard to start from a truly clean profile |
| - | <code python> | + | The complete list of things you should be able to say you reset, or say you did not: cookies (including partitioned ones), '' |
| - | string | + | |
| - | print(f'Hello {string}') | + | ===== What each design can and cannot measure ===== |
| + | |||
| + | ^ Phenomenon ^ Needs ^ Why ^ Example ^ | ||
| + | | Third-party presence, request counts, filter-list hit rates | either, but say which | a fresh profile draws //more// third-party traffic than an aged one, so the two are not interchangeable | {[jueckstock2021_realistic]}, | ||
| + | | Cookie syncing / ID sharing | **either, but they measure different things** | a fresh profile sees // | ||
| + | | Cookie respawning, evercookies | **stateful** across a clear | the phenomenon //is// state surviving a reset | {[acar2014_never]}; | ||
| + | | Ad retargeting, | ||
| + | | Effect of a consent choice on //other// sites | **stateful** | the consent decision only travels via stored state | {[rasaii2025_crumbs]} | | ||
| + | | Consent revocation, opt-out persistence | **stateful** within a session at minimum | you must be in the consented state before you can revoke it | {[kancherla2025_johnny]}, | ||
| + | | Logged-in versus anonymous web | **stateful** (a session) | the session cookie //is// the state | {[kaizer2016_characterizing]}, | ||
| + | | First-party-cookie abuse for cross-site tracking | **stateful** | the abuse is the reuse of a first-party value elsewhere | {[chen2021_cookieswap]} | | ||
| + | | Cache-based attacks and leaks | **the cache is the state**, and its contents have to be controlled per URL | a cache hit and a cache miss are the two outcomes you are distinguishing, | ||
| + | | Effect of a blocker or a setting | **either**, but the //same// for both arms | a blocker' | ||
| + | | Anything you want to parallelise over a million sites | **stateless** | see the next section | {[englehardt2016online]} | | ||
| + | |||
| + | ===== The measured consequences ===== | ||
| + | |||
| + | ==== A crawler sees a different web from a user, and statefulness is part of why ==== | ||
| + | |||
| + | Zeber et al. compared an OpenWPM crawl against telemetry from over 50,000 opt-in Firefox users over the same period {[zeber2020representativeness]}. The gap is large and consistently in one direction: | ||
| + | |||
| + | ^ Metric, on the same site domains ^ Human users ^ Crawler ^ | ||
| + | | median third-party domains per visit | 4.5 | **11.6** | | ||
| + | | median third-party domains per visit, popularity-weighted | 2.9((The paper gives the human median " | ||
| + | | median tracking domains per visit (Disconnect list) | 1.9 | **6.1** | | ||
| + | | trackers reached | "up to 8 … in 99% of visits" | ||
| + | | Jaccard similarity of the third-party //sets// | median **20%** | — | | ||
| + | |||
| + | > crawler site visits issued requests to a median of 11.6 third-party domains, whereas for visits by humans, the median was 4.5 third parties | ||
| + | |||
| + | Two cautions. First, this is **crawler versus human**, not stateful versus stateless: automation, vantage point, interaction and statefulness all differ at once, so it is not a clean experiment on this axis. The existing note on this page previously read the paper' | ||
| + | |||
| + | > cookie syncing is not necessary for users who have already had their cookies synced, whereas a stateless crawler browser instance with a fresh profile would be a clear target for cookie syncing | ||
| + | |||
| + | And the direction is not universal. Fingerprinting prevalence agreed between crawl and users to within 1 percentage point in the same study — so " | ||
| + | |||
| + | The 2026 restatement of the same problem is much sharper, and it is about interaction as well as state. Song et al. trained nine website-fingerprinting models on traffic from scripted browser automation and tested them on traffic from 30 real users across 20 sites: **every model scored under 10% accuracy**. Training on LLM-agent-generated, | ||
| + | |||
| + | ==== Statefulness does not scale, and the standard workaround has a known artefact ==== | ||
| + | |||
| + | Englehardt and Narayanan are blunt about it {[englehardt2016online]}: | ||
| + | |||
| + | > Making stateful measurements is fundamentally at odds with parallelism. | ||
| + | |||
| + | Their own numbers, on one 2016 EC2 '' | ||
| + | |||
| + | * **10** stateful browser instances in parallel, against **20** stateless ones — "stateful parallel measurements are memory-limited while stateless parallel measurements are typically CPU-limited"; | ||
| + | * the census itself ran **1,000,000 sites stateless** but only **100,000 stateful**. | ||
| + | |||
| + | The workaround is the seed profile: visit the top 10,000 sites serially, save the profile, clone it into '' | ||
| + | |||
| + | > We find that a seed profile which has visited the top 10,000 sites will have communicated with 76% of all third-party domains present on more than 5 of the top 100,000 sites. | ||
| + | |||
| + | — but it has an artefact you must report, in the authors' | ||
| + | |||
| + | ==== Order dependence, and what to do about it ==== | ||
| + | |||
| + | A stateful crawl of a ranked list confounds rank with visit order: by the time you reach rank 10,000 the profile has seen 9,999 sites, so " | ||
| + | |||
| + | > In stateful experiments, | ||
| + | |||
| + | Three mitigations are in use, in ascending order of cost: | ||
| + | |||
| + | - **Go seeded-stateless** and say so, as Urban et al. do, which removes the dependence by construction {[urban2020beyond]}. | ||
| + | - **Randomise the visit order and publish the seed**, so the confound becomes noise rather than a gradient. Rasaii et al. run their stateful campaign in a randomised order, again in the reverse of that order, and again in a recombined order, precisely so the order effect can be measured rather than assumed away {[rasaii2025_crumbs]}. | ||
| + | - **Repeat the whole crawl with an independently drawn order** and report the between-run variation. Nobody in our corpus does this at scale; see [[#Open Questions]]. | ||
| + | |||
| + | ==== What a stateful design buys you, quantified ==== | ||
| + | |||
| + | These are the results that a stateless crawl could not have produced. Each is quoted with the paper's own denominator. | ||
| + | |||
| + | * **Consent given on one site follows you to the next.** Rasaii et al. accepted banners across the first half of Tranco' | ||
| + | * **Respawning plus syncing survives a state clear.** The 2014 mechanism is historical — Flash reached end of life in December 2020 — but the finding is the reason a state clear cannot be assumed to work, and the technique moved to fingerprint-keyed respawning rather than disappearing ({[fouad2022my]} in 2022, server-side in {[fouad2024_devil]} in 2024). Acar et al. found "33 different Flash cookies from 30 different domains respawned a total of 355 cookies on 107 first party domains", | ||
| + | * **A trained profile is treated differently.** Agarwal et al.: " | ||
| + | * **State accumulates within a site, not only across sites.** Urban et al., under a seeded-stateless design with state kept across a site's own subpages: " | ||
| + | |||
| + | And one that cuts the other way: **cookie respawning with browser fingerprinting** was measured on 30,000 Alexa sites with a stateless harness, by comparing paired visits rather than by accumulating a profile — "1, 150 (3.83%) of the Alexa top 30, 000 websites use cookie respawning with browser fingerprinting" | ||
| + | |||
| + | ===== Since 2022 the engine decides, not you ===== | ||
| + | |||
| + | This is the part of the topic where the literature is out of date and a page written from the corpus alone would mislead. Everything in this section was checked against primary sources on 2026-08-19. | ||
| + | |||
| + | * **Firefox partitions third-party cookies by default.** Total Cookie Protection has been on by default since June 2022, " | ||
| + | * **Chrome partitions third-party //storage// for every user.** "The feature has been enabled for all users on Chrome 115 and later." | ||
| + | * **Third-party cookies were //not// deprecated.** On 22 April 2025 Google announced it would " | ||
| + | * **Your crawler probably turns the partitioning off.** Playwright launches Chromium with '' | ||
| + | |||
| + | ^ Chromium feature ^ Playwright 1.62.1 ^ Puppeteer 25.5.0((The same list in Puppeteer 25.8.0, the latest release as of 2026-08-19, is byte-identical, | ||
| + | | '' | ||
| + | | '' | ||
| + | | '' | ||
| + | | '' | ||
| + | | '' | ||
| + | | '' | ||
| + | |||
| + | Playwright' | ||
| + | |||
| + | > our current capabilities of saving/ | ||
| + | |||
| + | ((Playwright maintainer, '' | ||
| + | |||
| + | So the consequence is that two crawlers driving the same engine version accumulate different third-party state, on the one axis this page is about, and the divergence exists because one of them has a profile-serialisation API that cannot express partitioned storage. Neither documents this where you would look. | ||
| + | |||
| + | **And it is a moving target.** In the same thread, the person who filed the request notes that "when the ThirdPartyStoragePartitioning flag is removed, bug #32230 will start reoccurring" | ||
| + | |||
| + | * **OpenWPM also opts out, by default.** '' | ||
| + | |||
| + | So the plain fact is that a stateful research crawl in 2026 accumulates an **unpartitioned** cross-site profile. That resembles a default Chrome user's //cookie jar// — but not that user's storage, which Chrome has partitioned since 115 — and it does not resemble a default Firefox or Safari user in either respect. And it happens whichever engine you drive, because the research tooling disables the partitioning. Whether that is the right choice depends on your question; it is never the right thing to leave unsaid. Measured, on the default Playwright Chromium context: | ||
| + | |||
| + | < | ||
| + | Engine | ||
| + | ---------------------- | ||
| + | Chromium 151.0.7922.34 | ||
| </ | </ | ||
| - | For large code, use '' | + | We could not run the same probe on Playwright' |
| + | |||
| + | ===== How to do it ===== | ||
| + | |||
| + | ==== Stateless ==== | ||
| + | |||
| + | In Playwright and Puppeteer you get this by accident, which is both convenient and a reporting hazard: a fresh '' | ||
| + | |||
| + | ==== Stateful ==== | ||
| + | |||
| + | * **Playwright: | ||
| + | * **Puppeteer: | ||
| + | * **OpenWPM: | ||
| + | * **Seeding: | ||
| + | |||
| + | ==== The code ==== | ||
| + | |||
| + | This is the **complete** script behind the [[#What a reset actually resets|reset table]] — all nine reset strategies, so every row is reproducible. Save the two files side by side as '' | ||
| + | |||
| + | <file javascript server.mjs> | ||
| + | // Minimal instrumented origin for the state-channel probe. Counts every | ||
| + | // request it receives, per path, and reports what the client sent back. | ||
| + | import http from 'node:http'; | ||
| - | <file php example.php> | + | export function startServer(port = 8123) { |
| - | <?php echo "hello world!"; | + | const log = []; |
| + | const server = http.createServer((req, | ||
| + | | ||
| + | if (req.url === '/' | ||
| + | res.writeHead(200, | ||
| + | ' | ||
| + | ' | ||
| + | ' | ||
| + | }); | ||
| + | res.end(`< | ||
| + | <script src="/ | ||
| + | < | ||
| + | document.cookie = ' | ||
| + | // Stamp a marker ONCE. A later visit that still sees the FIRST visit' | ||
| + | // proves localStorage survived; one that writes its own proves it did not. | ||
| + | window.__lsBefore = localStorage.getItem(' | ||
| + | if (!window.__lsBefore) localStorage.setItem(' | ||
| + | </ | ||
| + | <img src="/ | ||
| + | return; | ||
| + | } | ||
| + | if (req.url === '/ | ||
| + | // Aggressively cacheable: a second visit should not hit the network. | ||
| + | res.writeHead(200, | ||
| + | res.end(' | ||
| + | return; | ||
| + | } | ||
| + | if (req.url === '/ | ||
| + | res.writeHead(200, | ||
| + | res.end(Buffer.from(' | ||
| + | return; | ||
| + | } | ||
| + | res.writeHead(404, | ||
| + | res.end(' | ||
| + | }); | ||
| + | return new Promise((resolve) => server.listen(port, | ||
| + | } | ||
| </ | </ | ||
| - | === Figures === | + | <file javascript probe.mjs> |
| + | // What each " | ||
| + | // local instrumented origin and reports, for each reset strategy, whether the | ||
| + | // second visit still carried a cookie, still had localStorage, | ||
| + | // the cacheable subresource from cache instead of the network. | ||
| + | // | ||
| + | // npm i playwright && npx playwright install chromium | ||
| + | // node scripts/ | ||
| + | // | ||
| + | // Read as: a ✓ under " | ||
| + | // SURVIVED the reset. A stateless crawl needs all three to be ✗. | ||
| + | import fs from ' | ||
| + | import os from ' | ||
| + | import path from ' | ||
| + | import { chromium } from ' | ||
| + | import { startServer } from ' | ||
| - | To use floats, you have to use the '' | + | const PORT = 8123; |
| - | <WRAP right 50% box> | + | const TARGET = `http:// |
| - | {{PATH_TO_FILE|ALT_TEXT}} | + | const { server, log } = await startServer(PORT); |
| - | <div>CAPTION</div> | + | |
| + | const tmp = () => fs.mkdtempSync(path.join(os.tmpdir(), | ||
| + | |||
| + | // Visit the page and report what the origin saw and what the page found. | ||
| + | async function visit(page) { | ||
| + | const before = log.length; | ||
| + | await page.goto(TARGET, | ||
| + | await page.waitForTimeout(400); | ||
| + | const hits = log.slice(before); | ||
| + | return { | ||
| + | cookieSentOnDoc: | ||
| + | cachedJsFromNetwork: | ||
| + | cachedPngFromNetwork: | ||
| + | // __lsBefore is what the page found BEFORE writing its own marker, so it is | ||
| + | // non-null only when localStorage genuinely survived into this visit. | ||
| + | lsCarriedIn: | ||
| + | lsNow: await page.evaluate(() => localStorage.getItem(' | ||
| + | jarCookies: (await page.context().cookies()).map((c) => c.name).sort().join(',' | ||
| + | }; | ||
| + | } | ||
| + | |||
| + | const mark = (b) => (b ? ' | ||
| + | const results = []; | ||
| + | const record = (strategy, second) => | ||
| + | results.push({ | ||
| + | strategy, | ||
| + | cookie: mark(!!second.cookieSentOnDoc), | ||
| + | localStorage: | ||
| + | cacheHit: mark(!second.cachedJsFromNetwork), | ||
| + | detail: | ||
| + | `visit 2 sent Cookie: ${second.cookieSentOnDoc ?? ' | ||
| + | `localStorage carried in: ${second.lsCarriedIn ?? ' | ||
| + | `/cached.js re-requested from origin: ${second.cachedJsFromNetwork ? ' | ||
| + | }); | ||
| + | |||
| + | // 1. Same page object, second navigation. | ||
| + | { | ||
| + | const b = await chromium.launch(); | ||
| + | const c = await b.newContext(); | ||
| + | const p = await c.newPage(); | ||
| + | await visit(p); | ||
| + | record(' | ||
| + | await b.close(); | ||
| + | } | ||
| + | // 2. New page in the same context. | ||
| + | { | ||
| + | const b = await chromium.launch(); | ||
| + | const c = await b.newContext(); | ||
| + | await visit(await c.newPage()); | ||
| + | record(' | ||
| + | await b.close(); | ||
| + | } | ||
| + | // 3. context.clearCookies() only. | ||
| + | { | ||
| + | const b = await chromium.launch(); | ||
| + | const c = await b.newContext(); | ||
| + | const p = await c.newPage(); | ||
| + | await visit(p); | ||
| + | await c.clearCookies(); | ||
| + | record(' | ||
| + | await b.close(); | ||
| + | } | ||
| + | // 4. clearCookies() + clearPermissions() (the usual "we cleared cookies" | ||
| + | { | ||
| + | const b = await chromium.launch(); | ||
| + | const c = await b.newContext(); | ||
| + | const p = await c.newPage(); | ||
| + | await visit(p); | ||
| + | await c.clearCookies(); | ||
| + | await c.clearPermissions(); | ||
| + | record(' | ||
| + | await b.close(); | ||
| + | } | ||
| + | // 5. New browser CONTEXT in the same browser process. | ||
| + | { | ||
| + | const b = await chromium.launch(); | ||
| + | const c1 = await b.newContext(); | ||
| + | await visit(await c1.newPage()); | ||
| + | const c2 = await b.newContext(); | ||
| + | record(' | ||
| + | await b.close(); | ||
| + | } | ||
| + | // 6. Fresh browser.launch() — Playwright' | ||
| + | { | ||
| + | const b1 = await chromium.launch(); | ||
| + | await visit(await (await b1.newContext()).newPage()); | ||
| + | await b1.close(); | ||
| + | const b2 = await chromium.launch(); | ||
| + | record(' | ||
| + | await b2.close(); | ||
| + | } | ||
| + | // 7. launchPersistentContext, | ||
| + | { | ||
| + | const dir = tmp(); | ||
| + | const c1 = await chromium.launchPersistentContext(dir); | ||
| + | await visit(await c1.newPage()); | ||
| + | await c1.close(); | ||
| + | const c2 = await chromium.launchPersistentContext(dir); | ||
| + | record(' | ||
| + | await c2.close(); | ||
| + | } | ||
| + | // 8. launchPersistentContext, | ||
| + | { | ||
| + | const dir = tmp(); | ||
| + | const c1 = await chromium.launchPersistentContext(dir); | ||
| + | await visit(await c1.newPage()); | ||
| + | await c1.close(); | ||
| + | const c2 = await chromium.launchPersistentContext(dir); | ||
| + | await c2.clearCookies(); | ||
| + | record(' | ||
| + | await c2.close(); | ||
| + | } | ||
| + | // 9. storageState round-trip: the documented way to carry a profile on purpose. | ||
| + | { | ||
| + | const b = await chromium.launch(); | ||
| + | const c1 = await b.newContext(); | ||
| + | await visit(await c1.newPage()); | ||
| + | const state = await c1.storageState(); | ||
| + | await c1.close(); | ||
| + | const c2 = await b.newContext({ storageState: | ||
| + | record(' | ||
| + | await b.close(); | ||
| + | fs.writeFileSync( | ||
| + | path.join(import.meta.dirname, | ||
| + | JSON.stringify(state, | ||
| + | ); | ||
| + | } | ||
| + | |||
| + | const W = Math.max(...results.map((r) => r.strategy.length)); | ||
| + | console.log( | ||
| + | [' | ||
| + | ); | ||
| + | console.log([(' | ||
| + | for (const r of results) | ||
| + | console.log([r.strategy.padEnd(W), | ||
| + | console.log(' | ||
| + | console.log('" | ||
| + | for (const r of results) console.log(` | ||
| + | { | ||
| + | const pkg = JSON.parse( | ||
| + | fs.readFileSync(new URL(' | ||
| + | ); | ||
| + | const b = await chromium.launch(); | ||
| + | console.log(`\nplaywright ${pkg.version}; | ||
| + | await b.close(); | ||
| + | } | ||
| + | server.close(); | ||
| + | </file> | ||
| + | |||
| + | ===== Use in Publications ===== | ||
| + | |||
| + | All figures below are over the **1,120 papers in the corpus that ran an automated web crawl**, out of 5,859 extracted papers from CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026. They are reporting rates: "does not state" means the paper did not say, not that the crawl had no state. The figures come from three scripts — '' | ||
| + | |||
| + | ==== Almost nobody says ==== | ||
| + | |||
| + | ^ crawlConfig.statefulness ^ Papers ^ Share of 1,120 crawling papers ^ | ||
| + | | stateless | 113 | 10.1% | | ||
| + | | stateful | 77 | 6.9% | | ||
| + | | both arms | 29 | 2.6% | | ||
| + | | **not stated** | 844 | **75.4%** | | ||
| + | | not applicable | 17 | 1.5% | | ||
| + | | //no crawl-configuration record at all// | 40 | 3.6% | | ||
| + | |||
| + | **219 of 1,120 (19.6%)** state it. Among those 219: stateless 51.6%, stateful 35.2%, both arms 13.2%. Against the other configuration fields the same papers could have reported: | ||
| + | |||
| + | ^ Field ^ Papers stating it ^ Share of 1,120 ^ | ||
| + | | Interaction depth | 841 | 75.1% | | ||
| + | | Authentication | 779 | 69.6% | | ||
| + | | At least one browser named | 529 | 47.2% | | ||
| + | | Consent action | 349 | 31.2% | | ||
| + | | **Stateful or stateless** | **219** | **19.6%** | | ||
| + | | Headless or headful | 140 | 12.5% | | ||
| + | |||
| + | An external cross-check disagrees, informatively — and it is a close comparison, because it covers **the same seven venues**. Demir et al. hand-coded 117 web-measurement papers from 2016–2021 against 18 reproducibility criteria; their criterion C11, " | ||
| + | |||
| + | ==== Reporting has not improved in sixteen years ==== | ||
| + | |||
| + | ^ Bucket ^ Crawling papers ^ State it ^ Share stating ^ stateless ^ stateful ^ both ^ stateless share of stated ^ | ||
| + | | 2010–2013 | 102 | 16 | 15.7% | 7 | 8 | 1 | 43.8% | | ||
| + | | 2014–2017 | 167 | 35 | 21.0% | 14 | 15 | 6 | 40.0% | | ||
| + | | 2018–2021 | 308 | 61 | 19.8% | 31 | 21 | 9 | 50.8% | | ||
| + | | 2022–2024 | 345 | 71 | 20.6% | 42 | 22 | 7 | 59.2% | | ||
| + | | 2025–2026* | 198 | 36 | 18.2% | 19 | 11 | 6 | 52.8% | | ||
| + | |||
| + | <WRAP info> | ||
| + | **2025–2026 is provisional: | ||
| </ | </ | ||
| - | */ | + | This is the finding, and it is easiest to see against reporting norms that //did// move. Same buckets, same corpus: |
| - | ====== | + | ^ Bucket ^ States statefulness //(of crawling papers)// ^ Releases an artifact link //(of all papers)// ^ Mentions an ethics review //(of empirical papers)// ^ |
| + | | 2010–2013 | 16/ | ||
| + | | 2014–2017 | 35/ | ||
| + | | 2018–2021 | 61/ | ||
| + | | 2022–2024 | 71/ | ||
| + | | 2025–2026* | 36/198 = **18.2%** | 907/1185 = 76.5% | 467/1019 = 45.8% | | ||
| - | /* | + | Artifact release more than tripled and ethics-review reporting more than quadrupled. Statefulness has sat between 16% and 21% throughout, with no trend. It is also the flattest of the crawl-configuration fields, which is the sharper version of the claim because those fields compete for the same paragraph of the same methods section: |
| - | To insert citations, follow these steps: | + | |
| - | - Verify the BibTeX entry exists in https:// | + | ^ Field ^ 2010–2013 ^ 2014–2017 ^ 2018–2021 ^ 2022–2024 ^ 2025–2026* ^ max−min ^ last − first ^ |
| - | - Use {[CitationKey]} where needed in the text; it will render as a numbered reference. | + | | '' |
| - | - Keep this section unchanged to display the bibliography. | + | | '' |
| + | | '' | ||
| + | | '' | ||
| + | | **'' | ||
| + | | '' | ||
| - | If any step fails, a purple warning will appear | + | Denominators are the crawling papers in each bucket, from the table above (102 / 167 / 308 / 345 / 198). Naming the browser gained 18 points and headless mode gained 10 from a near-zero base; statefulness gained 2.5 and has the narrowest **range** of the six (5.3 pp). Interaction depth is the only field whose range is nearly as narrow (6.2 pp), and it got that way by // |
| - | */ | + | |
| + | What //did// move is the answer among those who give one: **the stateless share of stated values rose from 43.8% into the 50s** (50.8%, 59.2%, 52.8% over the last three buckets). Read this as the field' | ||
| + | |||
| + | ==== By venue ==== | ||
| + | |||
| + | ^ Venue ^ Crawling papers ^ State it ^ Share stating ^ stateless ^ stateful ^ both ^ | ||
| + | | WWW | 242 | 45 | 18.6% | 20 | 19 | 6 | | ||
| + | | USENIX | 221 | 30 | 13.6% | 17 | 12 | 1 | | ||
| + | | CCS | 163 | 27 | 16.6% | 11 | 10 | 6 | | ||
| + | | IMC | 132 | 28 | 21.2% | 17 | 9 | 2 | | ||
| + | | NDSS | 129 | 24 | 18.6% | 15 | 7 | 2 | | ||
| + | | PETS | 123 | 42 | **34.1%** | 22 | 11 | 9 | | ||
| + | | IEEE-SP | 110 | 23 | 20.9% | 11 | 9 | 3 | | ||
| + | |||
| + | PETS states it at 34.1% — about 60% more often than the next venue (IMC, 21.2%) and two and a half times as often as USENIX Security (13.6%) — and holds 9 of the 29 both-arms papers on just over half of USENIX' | ||
| + | |||
| + | ==== The instrument decides whether you say it ==== | ||
| + | |||
| + | ^ Framework family ^ Crawling papers ^ State it ^ Share stating ^ stateless ^ stateful ^ both ^ | ||
| + | | OpenWPM | 58 | 32 | **55.2%** | 15 | 12 | 5 | | ||
| + | | Vulnerability / state-space crawlers | 29 | 11 | 37.9% | 4 | 7 | 0 | | ||
| + | | Puppeteer | 76 | 27 | 35.5% | 18 | 6 | 3 | | ||
| + | | Playwright | 34 | 12 | 35.3% | 6 | 3 | 3 | | ||
| + | | Tracker Radar Collector | 10 | 3 | 30.0% | 2 | 0 | 1 | | ||
| + | | Selenium | 242 | 67 | 27.7% | 31 | 26 | 10 | | ||
| + | | //any framework named// | 723 | 182 | 25.2% | 91 | 68 | 23 | | ||
| + | | //no framework named// | 397 | 37 | **9.3%** | 22 | 9 | 6 | | ||
| + | |||
| + | OpenWPM papers state it at **six times** the rate of papers that do not name a framework, and twice the rate of Selenium papers. The mechanism is not virtue but interface: OpenWPM' | ||
| + | |||
| + | ==== Designs whose result cannot be read without it ==== | ||
| + | |||
| + | ^ Subset of crawling papers ^ N ^ State statefulness ^ Share stating ^ stateless ^ stateful ^ both ^ | ||
| + | | Crawls that acted on a consent notice | 36 | 22 | **61.1%** | 11 | 4 | 7 | | ||
| + | | …the 28 of those whose interaction was verified by hand | 28 | 16 | **57.1%** | 8 | 3 | 5 | | ||
| + | | Crawls that logged in | 90 | 54 | **60.0%** | 7 | 39 | 8 | | ||
| + | | Repeat-visit designs (2 or more visits per target) | 199 | 85 | 42.7% | 49 | 21 | 15 | | ||
| + | | Crawls beyond the landing page | 303 | 83 | 27.4% | 30 | 40 | 13 | | ||
| + | | Deep crawls | 157 | 40 | 25.5% | 8 | 26 | 6 | | ||
| + | | //all crawling papers// | 1,120 | 219 | 19.6% | 113 | 77 | 29 | | ||
| + | |||
| + | The consent row needs a caveat that [[privacy: | ||
| + | |||
| + | The good news first: where the design makes the axis unavoidable, | ||
| + | |||
| + | ==== One word, two literatures, | ||
| + | |||
| + | Folding the 219 stating papers by subject matter (keyword match over slug, '' | ||
| + | |||
| + | ^ Subject matter ^ Papers stating statefulness ^ stateless ^ stateful ^ both ^ | ||
| + | | tracking / privacy measurement | 133 | 78 | 35 | 20 | | ||
| + | | web-application security scanning | 23 | 7 | 14 | 2 | | ||
| + | | both vocabularies present | 18 | 6 | 9 | 3 | | ||
| + | | neither (unmatched residue) | 45 | 22 | 19 | 4 | | ||
| + | |||
| + | In the tracking literature stateless outnumbers stateful more than two to one; in the scanning literature it is the reverse, because there " | ||
| + | |||
| + | ==== The comparison studies, audited ==== | ||
| + | |||
| + | The 29 papers labelled '' | ||
| + | |||
| + | ^ Verdict ^ Papers ^ Share of 29 ^ What it means ^ | ||
| + | | ok | 16 | 55.2% | a stateful arm and a stateless arm really were both run | | ||
| + | | partial | 10 | 34.5% | two conditions exist, but the contrast is login, seeding or consent, not statefulness | | ||
| + | | wrong | 3 | 10.3% | no stateful-versus-stateless contrast in the paper at all | | ||
| + | |||
| + | So the corpus holds **16 genuine comparison studies out of 1,120 crawling papers (1.4%)**, not 29 (2.6%). Use the 16 as a reading list and not the 29. Here they are, with what the two arms actually were: | ||
| + | |||
| + | ^ Paper ^ The two arms ^ | ||
| + | | {[acar2014_never]} | one sequential crawl keeping profile state, plus parallel crawls that do not | | ||
| + | | {[meng2014_pollution]} | profiles polluted by a CSRF-style attack, against clean profiles replayed from user traces | | ||
| + | | {[pan2015_summer]} | each site visited "once starting with a clean browser and once more after priming the client-side state" | | ||
| + | | {[englehardt2016online]} | //Default Stateless// over 1M sites beside //Default Stateful// over 100k | | ||
| + | | {[matthews2018_addons]} | blockers measured with no browsing history, then again post-calibration | | ||
| + | | {[englehardt2018_email]} | each email loaded twice: fresh profile, then the same profile again | | ||
| + | | {[robertson2018_auditing]} | a standard window and an incognito window driven side by side | | ||
| + | | {[agarwal2020_stop]} | personas trained statefully, then measured stateless | | ||
| + | | {[chen2021_cookieswap]} | repeat visits retaining state alongside fresh-profile visits | | ||
| + | | {[mehrnezhad2022_protect]} | consent accepted | ||
| + | | {[mirheidari2022_cache]} | per-URL cache hit against cache miss, verified for each candidate | | ||
| + | | {[rautenstrauch2023_leaky]} | logged-in state against anonymous, "a fresh browser context that we reset between" | ||
| + | | {[liu2024_opted]} | personas accumulating over nine iterated visits, against control personas | | ||
| + | | {[rautenstrauch2024_auth]} | the same site crawled twice in parallel, once with a session | | ||
| + | | {[rasaii2025_crumbs]} | banners accepted statefully on the first half of the list, measured on the second | | ||
| + | | {[ablove2026_censorship]} | persistent browser sessions for most services, fresh sessions for the one with a query limit | | ||
| + | |||
| + | Ten more are labelled '' | ||
| + | |||
| + | ==== Methodology and limitations of these figures ==== | ||
| + | |||
| + | * **Population.** '' | ||
| + | * **Stability.** '' | ||
| + | * **The evidence quote does not evidence this field.** See [[#The comparison studies, audited]]. This is the single biggest threat to every number above, and it is why the audit exists. | ||
| + | * **Reporting, | ||
| + | * **Seven venues.** EuroS& | ||
| + | * **Full query log, folding rules, residue, quote checks and reviewer findings:** [[provenance: | ||
| + | |||
| + | ===== What to report ===== | ||
| + | |||
| + | Demir et al. set the bar {[demir2022_reproducibility]}: | ||
| + | |||
| + | > authors need to document what part of a browser profile is maintained statefully, what part is reset, and when | ||
| + | |||
| + | Concretely, one short paragraph in your methodology, | ||
| + | |||
| + | - **Stateful, stateless or seeded**, in those words. | ||
| + | - **The unit of the reset** — per page visit, per site, per browser instance, per crawl. | ||
| + | - **What exactly is reset**, given that "we cleared cookies" | ||
| + | - **The seed's provenance**, | ||
| + | - **Number of parallel browsers**, alongside the stateful claim, because '' | ||
| + | - **Visit order** and whether it was randomised, with the seed, if the crawl was stateful. | ||
| + | - **The engine' | ||
| + | |||
| + | One sentence that does all of it: //" | ||
| + | |||
| + | ===== Recommendations ===== | ||
| + | |||
| + | - **Default to stateless** unless your question needs accumulation. It parallelises, | ||
| + | - **Say so anyway.** Getting it by default is not the same as reporting it, and 75.4% of the crawling papers in this corpus did not. | ||
| + | - **If you need state, prefer seeded-stateless** to a rolling profile. You keep a non-empty starting state and lose the order confound. Publish the seed. | ||
| + | - **If you need a rolling profile, run one browser** or report how many you ran and how the site list was partitioned across them. | ||
| + | - **Never claim a reset you did not measure.** Run [[#The code|the probe]] against your own harness once; it takes a minute and it is the cheapest methodological insurance on this page. | ||
| + | - **Do not compare your numbers to a paper on the other side of this axis** without saying so. Third-party counts from a fresh-profile crawl and from an aged profile are different quantities. | ||
| + | - **State the engine' | ||
| + | |||
| + | ===== Papers to read first ===== | ||
| + | |||
| + | - **{[englehardt2016online]} — the reference implementation of both modes.** Read §3.3 for the cost of statefulness and §4 for the seed-profile design and its artefact. Everything later argues with this paper. | ||
| + | - **{[zeber2020representativeness]} — how far a crawl is from a user.** The numbers you will be asked about in review. | ||
| + | - **{[demir2022_reproducibility]} — what to write down.** Criterion C11 and practice P9 are the reporting standard; the paper also measures how badly the field met it. | ||
| + | - **{[rasaii2025_crumbs]} — the modern stateful design, done well.** Split the list, accept on the first half, measure the second half, randomise the order. The clearest recent example of a result that a stateless crawl cannot produce. | ||
| + | - **{[urban2020beyond]} — seeded stateless, and why.** One sentence in §4.3.2 explains the whole third design position. | ||
| + | - **{[acar2014_never]} — why a clean profile is hard.** Respawning and syncing across a deliberate state clear. | ||
| + | - **{[agarwal2020_stop]} — train stateful, measure stateless.** The hybrid pattern most personalisation work now uses. | ||
| + | - **{[song2026_wfpllm]} — the 2026 version of the realism problem.** Models trained on scripted-crawler traffic score under 10% on real users; LLM-agent personas close most of the gap. | ||
| + | - **{[jueckstock2021_realistic]} — the neighbouring axis.** Vantage point and browser configuration, | ||
| + | |||
| + | ===== Open Questions ===== | ||
| + | |||
| + | <WRAP todo> | ||
| + | * **Nobody has run the clean experiment.** Demir et al. announce one and do not deliver it: §2.2 says "Since the effects of C5 and C11 are not yet adequately discussed by previous work, we analyze them in Section 4", and §4 then runs "four exemplarily case studies focusing on C4, C5, C10, and C12" — repetition, crawler technology, interaction and geolocation. C11, the crawling strategy, is the one criterion they flagged and did not vary; their own runs are described in Appendix C as " | ||
| + | * **How much does visit order actually change a stateful result?** The confound is universally acknowledged and never quantified. | ||
| + | * **Does the seed profile artefact bite?** Englehardt and Narayanan predicted that cloning one seed into '' | ||
| + | * **What does statefulness mean under partitioning? | ||
| + | * **Does the per-browser cookie-jar partition change published results?** Carried over from [[Programming: | ||
| + | </ | ||
| + | ====== References ====== | ||
| <bibtex bibliography></ | <bibtex bibliography></ | ||
| Line 101: | Line 629: | ||
| /* This enables discussion under this article. */ | /* This enables discussion under this article. */ | ||
| ~~DISCUSSION~~ | ~~DISCUSSION~~ | ||
| + | |||
programming/stateful_stateless.1742307641.txt.gz · Last modified: by karelkubicek