This is an old revision of the document!
Table of Contents
Literature Review
A related-work section for a web measurement is not a search problem. It is a denominator problem. The instinct is to type a keyword into Scholar, snowball the citations, and treat the resulting pile as “the literature”. That pile is the papers that used the words you searched for. If the question you wanted to answer was “how often does this field do X”, the papers that never said X are exactly the ones the search dropped — and they are the ones that move the rate.
This page is about that trap, measured on the same seven-venue corpus the rest of this site uses, and about what to do instead. It is not a tutorial on Scholar, PRISMA, or how to write a related-work section in general. Those belong in a methods textbook. What belongs here is what those methods do to web measurement literature: a field whose important nouns are homographs, whose reporting gaps are the finding, and whose best surveys are still built by cutting a venue-complete list with a keyword.
The corpus behind every figure is 5,859 extracted papers from CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026. Only 1,120 of them ran a crawl. 2025 and 2026 are provisional — CCS and IMC 2026 have not been held, and two more 2026 venue-years are incompletely selected. Queries, folds and the unedited report are on literature_review. The corpus-wide caveats are on Corpus.
A keyword-derived denominator is biased along the axis being measured.
- Searching this corpus for “fingerprint” hits 280 papers whose
detection.phenomenonnames one. 83 (29.6%) are browser fingerprinting; 105 (37.5%) are website / traffic fingerprinting, an encrypted-traffic-analysis problem that shares the word and almost none of the method. See Fingerprinting. - Searching for
pre-regist*hits 62 papers. 47 (75.8%) mean a pre-registered domain, OAuth redirect URI, FIDO device or test account — not a study preregistration. See Study preregistration. - Searching for the word “SoK” in these seven venues finds IEEE S&P's track. 30 of the 43 extracted SoKs are IEEE S&P (69.8%). CCS, IMC and TheWebConf contribute zero, in the index and in the extraction: they do not run that track.
- Searching crawled papers for “headless” cannot tell you how often crawls are headless. 140 of 1,120 crawled papers (12.5%) state
crawlConfig.headless; the other 87.5% never said, and a keyword search never sees them.
The rest of this page is that last bullet, applied to the way the field reviews itself.
What to Read First
- Stafeev and Pellegrino [1Stafeev, Aleksei; Pellegrino, Giancarlo (2024): "SoK: State of the Krawlers – Evaluating the Effectiveness of Crawling Algorithms for Web Security Measurements", in: Proceedings of the USENIX Security Symposium. (Link)] (USENIX Security 2024), SoK: State of the Krawlers. The closest sibling this site has in the peer-reviewed literature: the same seven venues, 2010–2022, 7,840 papers, then a keyword cut (Tranco, Alexa, and “top” with “site”) down to 1,057, of which 654 (61.9%) did not employ automated crawling, leaving 403. Read it for the crawling-algorithm taxonomy; read it also as a worked example of a keyword-derived population.
- Vekaria et al. [2Vekaria, Yash; Beugin, Yohan; Munir, Shaoor; Acar, Gunes; Bielova, Nataliia; Englehardt, Steven; Iqbal, Umar; Kapravelos, Alexandros; Laperdrix, Pierre; Nikiforakis, Nick; Polakis, Jason; Roesner, Franziska; Shafiq, Zubair; Zimmeck, Sebastian (2025): "SoK: Advances and Open Problems in Web Tracking". arXiv preprint arXiv:2506.14057. (Link)] (arXiv:2506.14057, June 2025). The tracking SoK, by fourteen of the field's authors. Same seven venues, twenty years, “200+” papers, organised thematically. It is still a preprint and is not in this extraction — arXiv is not one of the seven venues. Check for a venue version before you treat it as archival.
- Rieder et al. [3Rieder, Wolf; Raschke, Philip; Cory, Thomas; Sechting, Christian René; Kumar, Aditya; Küpper, Axel (2026): "SoK: After Decades of Web Tracker Detection, What's Next?", in: Proceedings of the IEEE Symposium on Security and Privacy. (Link)] (IEEE S&P 2026). The tracker-detector SoK. It is in this project's IEEE S&P 2026 index and on oaklandsok.github.io. It has no abstract in the index, so it was never screened, so it is not among the 5,859. The most on-topic tracker-detection SoK in these venues is missing from every figure on this site for a mechanical reason. That is the funnel on Corpus, sitting inside this page's own topic.
- Laperdrix, Bielova, Baudry and Avoine [4Laperdrix, Pierre; Bielova, Nataliia; Baudry, Benoit; Avoine, Gildas (2020): "Browser Fingerprinting: A Survey", ACM Transactions on the Web 14(2). (DOI)] (ACM TWEB 2020). The browser-fingerprinting survey. Not an SoK, not in these seven venues as a TWEB paper, and the right first paper for that topic — which is why Fingerprinting sends you there and then measures the homograph this corpus adds.
- Usman and Zappala [5Usman, Warda; Zappala, Daniel (2025): "SoK: A Framework and Guide for Human-Centered Threat Modeling in Security and Privacy Research", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] (IEEE S&P 2025). A human-centered threat-modeling SoK that refused keyword search on purpose: “we found this to be much more effective than keyword searching in a database. The language used to describe human-centered threat modeling is highly varied … so using keywords may have excluded a significant portion of relevant literature.” The quote is the argument of this page, in someone else's methods section.
1. The Keyword-Derived Denominator
You cannot measure a reporting rate by searching for the thing being reported
Suppose you want to know how often web crawls are run headless. You search Scholar for “headless crawler”. Every hit is a paper that used the word. The papers that ran a crawl and never said “headless” — 980 of the 1,120 (87.5%) by the schema — are invisible, so the search cannot produce a rate. It can only produce a list of papers that spoke.
That is not a Scholar limitation. It is what a keyword is. The population has to be defined independently of the word. Here the population is the 1,120 papers that ran a crawl; the word is then a column, not the row filter.
The script below does that cut on this extraction. It is the whole lesson in twelve lines of output.
- keyword_denominator.py
#!/usr/bin/env python3 """Show what a keyword search cannot see among crawls. Prints two counts over the crawled papers in extract/run1: * crawled papers whose paper.cols.txt matches the regex * crawled papers that do not match ← the denominator the keyword never sees A Scholar search for the word returns the first row. It cannot estimate a rate against the second. That is the reporting-rate trap on writing:literature_review. Fails loudly if a crawled paper has no paper.cols.txt. Usage: uv run python pages/keyword_denominator.py 'tranco' uv run python pages/keyword_denominator.py 'headless' uv run python pages/keyword_denominator.py 'openwpm|open-wpm' """ from __future__ import annotations import json import re import sys from pathlib import Path ROOTS = [ Path("/workspace/publications_dataset/data"), Path("/workspace/publications_dataset"), ] def data_root() -> Path: for root in ROOTS: if (root / "extract/run1/extractions.jsonl").is_file(): return root raise FileNotFoundError( "extract/run1/extractions.jsonl not found under " + ", ".join(str(r) for r in ROOTS) ) def is_crawled(p: dict) -> bool: return p["crawlConfig"] is not None or "automated-web-crawl" in p["studyTypes"] def main() -> None: if len(sys.argv) != 2: raise SystemExit( "usage: keyword_denominator.py '<regex>'\n" "example: keyword_denominator.py 'tranco'" ) pattern = re.compile(sys.argv[1], re.I) root = data_root() path = root / "extract/run1/extractions.jsonl" rows = [json.loads(line) for line in path.read_text().splitlines() if line] crawled = [p for p in rows if is_crawled(p)] if not crawled: raise RuntimeError("crawled population is empty") hit = 0 miss = 0 for p in crawled: f = root / "fulltext" / str(p["year"]) / p["venue"] / p["slug"] / "paper.cols.txt" if not f.is_file(): raise FileNotFoundError(f"missing crawled full text: {f}") text = f.read_text(encoding="latin-1") if pattern.search(text): hit += 1 else: miss += 1 if hit + miss != len(crawled): raise RuntimeError(f"hit {hit} + miss {miss} != crawled {len(crawled)}") def pct(n: int, d: int) -> str: return f"{100 * n / d:.1f}%" print(f"regex: {sys.argv[1]}") print(f"crawled papers (denominator): {len(crawled)}") print(f"crawled papers matching the regex: {hit:5d} ({pct(hit, len(crawled))})") print(f"crawled papers the regex cannot see: {miss:5d} ({pct(miss, len(crawled))})") print() print("The second row is the reporting-rate trap: you cannot estimate how often") print("crawls do X by searching for X, because the papers that never said X") print("are exactly the ones the search drops.") if __name__ == "__main__": main()
Real output, 2026-08-27, against extract/run1:
regex: \bheadless\b crawled papers (denominator): 1120 crawled papers matching the regex: 137 (12.2%) crawled papers the regex cannot see: 983 (87.8%) The second row is the reporting-rate trap: you cannot estimate how often crawls do X by searching for X, because the papers that never said X are exactly the ones the search drops.
The schema's 140/1,120 (12.5%) and the word's 137/1,120 (12.2%) differ by three in the totals. Membership is not those three papers: 28 mention the word with a schema sentinel or absent field, 31 have the schema set and no word in .cols (59 papers disagree). Neither number is a search you could have run in Scholar, because both start from the crawl population rather than from the word.
Homographs make the false-positive side as large as the false-negative
The reporting-rate trap is the papers the keyword misses. Homographs are the papers it hits that are about something else.
| Keyword in this corpus | What you were looking for | What the keyword actually is |
|---|---|---|
“fingerprint” in detection.phenomenon | browser fingerprinting | 83 of 280 (29.6%) browser; 105 (37.5%) website / traffic fingerprinting; the rest are biometrics, PRNU, TLS stacks, sandboxes, LLMs |
pre-regist* in full text | study preregistration | 15 of 62 are a study; 47 (75.8%) are a pre-registered domain, redirect URI, device or account |
| “survey” in the title | a literature survey | 11 papers; 7 of 11 are tagged user-study or interview-or-survey. One of the remaining four is Surveylance, a detector of survey scams. |
| “SoK” in the title | a web-measurement systematisation | 4 of 43 (9.3%) are about measuring the deployed web. 26 (60.5%) are TEE, binaries, fuzzing, crypto, hardware, DeFi. |
A related-work search that does not name the homograph will fill the section with the wrong literature. Fingerprinting exists because that search is about 30% precise in this corpus. This page exists because the same shape repeats for every method word the field uses.
2. What "SoK" Names in These Venues
IEEE S&P has solicited Systematization of Knowledge papers since 2010. The 2026 CFP still distinguishes them by the prefix “SoK:” in the title, holds them to the same review standards as research papers, and says survey papers without a new viewpoint “are not appropriate and may be rejected without full review.”1)
That prefix is a venue artifact, not a research-area name.
In the extraction: 43 papers, 30 of them IEEE S&P
Title-start “SoK” / “SOK:” and slug-start “sok-” agree on every extracted record: 43 and 43, intersection 43, residue 0. That is the population on this page.
| Venue | Extracted SoKs | Extracted papers at that venue | SoKs / venue |
|---|---|---|---|
| IEEE S&P | 30 | 767 | 3.9% |
| USENIX Security | 7 | 1,410 | 0.5% |
| NDSS | 4 | 701 | 0.6% |
| PETS | 2 | 510 | 0.4% |
| CCS | 0 | 990 | 0.0% |
| IMC | 0 | 638 | 0.0% |
| TheWebConf | 0 | 843 | 0.0% |
69.8% of extracted SoKs are IEEE S&P. CCS, IMC and TheWebConf are not listed as SoK tracks on oaklandsok.github.io and contribute no SoK-titled paper to the index either — 0, 0, 0 of 16,864 index records. A Scholar search for “SoK” over “web measurement” is, in these venues, a search of Oakland's track plus whatever USENIX, NDSS and PETS have started to copy.
The 43 are not mostly about the web. After a hand fold (every paper, deciding sentence in the report):
| Topic | Papers | Share of 43 |
|---|---|---|
| Web measurement — crawls, live-site audits, phishing-site detectors, privacy-regulation impact on the web | 4 | 9.3% |
| Adjacent online privacy / abuse / traffic analysis | 13 | 30.2% |
| Not this field — TEE, binaries, fuzzing, crypto, hardware, DeFi, … | 26 | 60.5% |
The four web-measurement SoKs, in full:
- Birrell et al. [6Birrell, Eleanor; Rodolitz, Jay; Ding, Angel; Lee, Jenna; McReynolds, Emily; Hutson, Jevan A.; Lerner, Ada (2024): "SoK: Technical Implementation and Human Impact of Internet Privacy Regulations", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)], IEEE S&P 2024 — privacy-regulation impact; seeded from ten named venues, then expanded through backward and forward citation searches.
- Blessing, Hugenroth, Anderson and Beresford [7Blessing, Jenny; Hugenroth, Daniel; Anderson, Ross; Beresford, Alastair (2025): "SoK: Web Authentication and Recovery in the Age of End-to-End Encryption", in: Proceedings on Privacy Enhancing Technologies. (DOI)], PoPETs 2025 — web authentication and recovery; a 245-paper venue-and-citation census plus an Alexa top-300 passkey audit.
- Alam et al. [8Alam, Mahbub; Rahman, Muhammad Lutfor; Paul, Sonjoy Kumar; Hays, Amy W.; Hussain, Aftab; Huq, Md Imanul; Saxena, Nitesh (2026): "SoK: PHILTER: Uncovering Security and Functional Gaps in AI-based Phishing Website Detection Literature via an LLM-based Reasoning Framework", in: Proceedings of the USENIX Security Symposium. (Link)], USENIX Security 2026 — AI phishing-website detectors; 38 papers from DBLP plus 17 from snowballing, 55 in total.
Website-fingerprinting defenses [9Mathews, Nate; Holland, James K.; Oh, Se Eun; Rahman, Mohammad Saidur; Hopper, Nicholas; Wright, Matthew (2023): "SoK: A Critical Evaluation of Efficient Website Fingerprinting Defenses", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] is in the adjacent bucket on purpose: it is a web traffic SoK, and putting it in “web measurement” would re-create the homograph Fingerprinting exists to kill.
In the index: 167 SoKs, most correctly dropped
The 43 are not “the SoKs in these venues”. They are the SoKs this corpus selected.
| Stage | SoKs | Lost here |
|---|---|---|
| Index (title-start or slug-start) | 167 | — |
| …with an abstract | 163 | 4 |
| …labelled (screened) | 163 | 0 |
| …selected by the over-inclusive –any rule | 47 | 116 |
| …extracted | 43 | 4 |
116 of 163 screened SoKs are out of scope for a measurement corpus, and looking at the PETS drop makes the reason obvious: PETS has 43 SoKs in the index and 2 in the extraction. The 41 that screening dropped include local differential privacy, MPC, federated-learning aggregation, e-voting, self-sovereign identity, trajectory generation. They are SoKs. They are not web measurement. The label did its job.
The four selected-but-not-extracted SoKs are a Docker-container attack SoK (IEEE S&P 2024) and three USENIX Security 2026 papers that have been selected but not yet retrieved. The four with no abstract are all IEEE S&P 2026, and one of them is Rieder et al.'s tracker-detection SoK [3Rieder, Wolf; Raschke, Philip; Cory, Thomas; Sechting, Christian René; Kumar, Aditya; Küpper, Axel (2026): "SoK: After Decades of Web Tracker Detection, What's Next?", in: Proceedings of the IEEE Symposium on Security and Privacy. (Link)]. Screening runs on abstracts; a paper with no abstract is never eligible, for a reason that has nothing to do with its topic. Corpus is the page that number lives on; it is repeated here because it deletes the paper you would otherwise have put first.
The first SoK-titled paper in this index is 2013. oaklandsok.github.io dates the track to 2010. Either the 2010–2012 SoKs were not titled “SoK:” in the metadata we have, or they are not in the index. This page does not pretend to have them.
3. How the SoKs Themselves Search
Of the 43 extracted SoKs, 19 (44.2%) are not a paper-census at all — they are a system, an evaluation, or a vulnerability corpus with an SoK prefix. 24 (55.8%) claim a literature sample; 18 of those 24 (75.0%) state a search protocol a reader can name.
Across all 43 (the table is not restricted to the 18):
| How the paper list was built | Papers | Share of 43 |
|---|---|---|
| Venue-complete (or a named venue list) then filter | 7 | 16.3% |
| Digital-library / DBLP keyword query | 3 | 7.0% |
| Google Scholar and/or citation snowball | 8 | 18.6% |
| Literature sample claimed; search protocol not stated | 6 | 14.0% |
| Not a paper-census | 19 | 44.2% |
Venue-complete-then-filter is 7 of 18 stated protocols (38.9%). It is the method this site also uses, and it is not the majority method even among SoKs that describe a search.
Two of those seven are the useful extremes.
Warford et al. [10Warford, Noel; Matthews, Tara; Yang, Kaitlyn; Akgul, Omer; Consolvo, Sunny; Kelley, Patrick Gage; Malkin, Nathan; Mazurek, Michelle L.; Sleeper, Manya; Thomas, Kurt (2022): "SoK: A Framework for Unifying At-Risk User Research", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] (IEEE S&P 2022) gathered every paper from CCS, CHI, CSCW, IEEE S&P, NDSS, PETS, SOUPS and USENIX Security on DBLP — 6,534 papers — and hand-filtered to 127 candidates, plus 12 from other sources to 139, then 95. No keyword. The cost is reading; the gain is that a paper which never says “at-risk user” can still be in.
Stafeev and Pellegrino [1Stafeev, Aleksei; Pellegrino, Giancarlo (2024): "SoK: State of the Krawlers – Evaluating the Effectiveness of Crawling Algorithms for Web Security Measurements", in: Proceedings of the USENIX Security Symposium. (Link)] started from the same seven venues this corpus uses, 2010–2022, 7,840 papers, and then keyword-cut on Tranco, Alexa, and the combination of “top” and “site”, to 1,057. They discarded 654 (61.9%) of those hits as not employing automated crawling, and surveyed the remaining 403. Of those 403, 130 (32.3%) navigate beyond a single page.
That second cut is the keyword-derived denominator, in a paper whose other contribution is excellent. Reproduced against this extraction's crawled papers 2010–2022 (687 papers, a different funnel — we already screened for measurement, they started from every paper in the proceedings):
| Krawlers keyword, on crawled papers 2010–2022 in this extraction | Papers | Share of 687 |
|---|---|---|
| full text matches “tranco” | 56 | 8.2% |
| full text matches “alexa” | 351 | 51.1% |
| tranco or alexa (the two named lists) | 375 | 54.6% |
| neither tranco nor alexa | 312 | 45.4% |
312 of 687 crawled papers in the same years (45.4%) mention neither list. A related-work search that starts from “Tranco or Alexa” cannot see them: custom seed lists, zone files, CrUX (which that SoK explicitly excluded because rankings started in 2022; 7 of our 687 still mention CrUX), store crawls that the schema still scores as crawled. Their 403 is a census of crawls that matched one of their keyword criteria (Tranco, Alexa, or “top” with “site”), not a census of crawls, and not a census of crawls that named a popular list.
Their keyword false-positive is the other half: 654 of 1,057 (61.9%) hits were not crawls. On our already-screened 2010–2022 extraction the analogue is 954 of 1,493 (63.9%) papers matching a generous reading of their three keywords that are not in the crawled population. Screening does not remove the homograph; it only starts you closer.
Usman and Zappala [5Usman, Warda; Zappala, Daniel (2025): "SoK: A Framework and Guide for Human-Centered Threat Modeling in Security and Privacy Research", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] are the methods-section that names this. They read SOUPS, CHI, CSCW, USENIX Security, IEEE S&P and PETS, 2018–2023, paper by paper, “because we found this to be much more effective than keyword searching in a database.” They also looked at NDSS and CCS and found no relevant papers in that window — a negative they can only state because they looked.
4. This Site as a Worked Example
This wiki is what a venue-complete pass plus structured extraction looks like when you keep the keyword out of the population rule.
The funnel is on Corpus: 16,864 index records → 15,800 with abstracts → 6,103 selected → 5,873 PDFs → 5,859 extraction records. Selection is an over-inclusive LLM screen on the abstract, not a keyword. Each page then defines its own population from what the paper did (ran a crawl, classified a request, recruited participants), and the keyword, if any, is a column on that population.
Three consequences that a Scholar search cannot produce:
- Silence is measurable. “Of 1,120 crawled papers, 12.5% state whether the browser was headless” is a number. “We searched for headless crawlers and found N papers” is not a rate.
- Homographs are separable. The fingerprinting page's 29.6% precision, the preregistration page's 47/62 other-sense, this page's 4/43 web-measurement SoKs — each is a fold with a printed residue, not a tighter regex.
- Two independent signals can be near-orthogonal. Requests builds its population as a union of “used a filter list” and “classified requests for an ad/tracking purpose”. They overlap on only 107 of 254 papers. A search for EasyList finds one proxy and misses the other, and the missed papers are not noise: they are the papers that classified requests without naming a list.
The cost is the cost: you have to have the corpus. You do not, for your own related-work section. What transfers is the discipline, not the 5,859 files.
Vekaria et al. [2Vekaria, Yash; Beugin, Yohan; Munir, Shaoor; Acar, Gunes; Bielova, Nataliia; Englehardt, Steven; Iqbal, Umar; Kapravelos, Alexandros; Laperdrix, Pierre; Nikiforakis, Nick; Polakis, Jason; Roesner, Franziska; Shafiq, Zubair; Zimmeck, Sebastian (2025): "SoK: Advances and Open Problems in Web Tracking". arXiv preprint arXiv:2506.14057. (Link)] did the transferable version: the same seven venues, twenty years, “200+” papers, thematic coding by two authors. It is still a preprint (arXiv:2506.14057, v1 16 June 2025, checked 2026-08-27); it is not in this extraction because arXiv is not a venue in the funnel. Use it as orientation, and do not treat “200+” as a denominator you can reproduce from here.
5. What to Do Instead
You are a PhD student writing related work for a crawl, not rebuilding this corpus. The methods that survive contact with the traps above:
- Define the population by what the paper did, then search. “Papers that crawled the web and classified cookies” is a population. “cookie classification” is a query over it. Reverse those and you cannot report a rate.
- Name the homograph before you search. Fingerprint, survey, SoK, HAR, stateful, landing page, preregistration — if the word is ordinary English or a term of art in two communities, write the other meaning down first. Fingerprinting, Traffic files and Stateful stateless each exist partly because a search does not.
- Prefer a venue-complete list plus a hand filter over a keyword, for any question that is a rate (how often, what share, what is missing). Warford et al. [10Warford, Noel; Matthews, Tara; Yang, Kaitlyn; Akgul, Omer; Consolvo, Sunny; Kelley, Patrick Gage; Malkin, Nathan; Mazurek, Michelle L.; Sleeper, Manya; Thomas, Kurt (2022): "SoK: A Framework for Unifying At-Risk User Research", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] and Usman and Zappala [5Usman, Warda; Zappala, Daniel (2025): "SoK: A Framework and Guide for Human-Centered Threat Modeling in Security and Privacy Research", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] are the templates; Stafeev and Pellegrino [1Stafeev, Aleksei; Pellegrino, Giancarlo (2024): "SoK: State of the Krawlers – Evaluating the Effectiveness of Crawling Algorithms for Web Security Measurements", in: Proceedings of the USENIX Security Symposium. (Link)] is the template for “venue-complete, then a keyword”, and this page is the measurement of what that keyword dropped.
- If you must keyword-search — you usually must — write the query, the date, the venues, and the misses you know about. “We searched Scholar for X on DATE, restricted to VENUES, and we know this drops papers that do Y without saying X.” That sentence is the difference between a search and a method.
- Snowballing from a seed SoK inherits the SoK's search. Citation snowballing is biased toward well-cited papers. If the quantity you want is “what the field does”, well-cited is the axis being measured. Birrell et al. [6Birrell, Eleanor; Rodolitz, Jay; Ding, Angel; Lee, Jenna; McReynolds, Emily; Hutson, Jevan A.; Lerner, Ada (2024): "SoK: Technical Implementation and Human Impact of Internet Privacy Regulations", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] seeded from ten named venues and then snowballed; copy the named-venue part, and do not write “inside ten venues” as if the snowball stayed there.
- Read the SoK, then read what its keyword cannot have included. For crawling, that is Stafeev and Pellegrino plus the crawled papers that never said Tranco or Alexa. For tracking, Vekaria et al. plus the pages on this site whose populations were built from extraction fields rather than from “tracking”.
- Do not use “SoK” as a quality filter. In these venues it is an IEEE S&P (and now USENIX / NDSS / PETS) track label. Most SoKs are not about your topic; most papers about your topic are not SoKs. CCS, IMC and TheWebConf will not give you one.
6. What to Report
A related-work or literature-review section that a reviewer in this field can accept says:
- The venues and the years, not “the literature”. “CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P, 2018–2024” is a population. “recent web-measurement papers” is not.
- How the list was built: venue-complete, database keyword (with the query), Scholar (with the date and the result-page cutoff), snowball (from which seeds). Usman and Zappala's sentence about why they did not keyword-search is the right length.
- The homographs you checked, and how many hits they ate. A number here is worth more than a claim of exhaustiveness.
- The denominator of any rate, named before the numerator. “Of N papers that crawled, M state a tool version” — never “we found M papers that state a tool version”.
- What the search cannot have seen. Papers that do the thing without the word; venues that do not use the SoK label; years that are still being indexed. Rieder et al. [3Rieder, Wolf; Raschke, Philip; Cory, Thomas; Sechting, Christian René; Kumar, Aditya; Küpper, Axel (2026): "SoK: After Decades of Web Tracker Detection, What's Next?", in: Proceedings of the IEEE Symposium on Security and Privacy. (Link)] missing from this corpus is the form of that sentence when the miss is mechanical rather than topical.
- Which SoK you started from, and that it is a starting point. Date preprints as preprints.
- The 4 web-measurement SoKs are a hand fold of 43. A different reader might move Birrell et al. (privacy-regulation impact) or Mathews et al. (website-fingerprinting defenses) across the web / adjacent line; the report records the deciding sentence so that disagreement is cheap.
- Vekaria et al. [2Vekaria, Yash; Beugin, Yohan; Munir, Shaoor; Acar, Gunes; Bielova, Nataliia; Englehardt, Steven; Iqbal, Umar; Kapravelos, Alexandros; Laperdrix, Pierre; Nikiforakis, Nick; Polakis, Jason; Roesner, Franziska; Shafiq, Zubair; Zimmeck, Sebastian (2025): "SoK: Advances and Open Problems in Web Tracking". arXiv preprint arXiv:2506.14057. (Link)] will need a venue update when one exists. As of 2026-08-27 it is arXiv v1 only.
- Rieder et al. [3Rieder, Wolf; Raschke, Philip; Cory, Thomas; Sechting, Christian René; Kumar, Aditya; Küpper, Axel (2026): "SoK: After Decades of Web Tracker Detection, What's Next?", in: Proceedings of the IEEE Symposium on Security and Privacy. (Link)] will enter this corpus if an abstract lands in OpenAlex and the screening is re-run. Until then every “tracker-detection SoK” claim on this site has to say it is missing.
- CCS / IMC / TheWebConf are still not listed as SoK tracks on oaklandsok.github.io. If one of them adds one, the venue table on this page is the thing that goes stale, not the argument.
Related Pages
- Corpus — how the 5,859 were selected; the funnel this page's 167 → 43 is a slice of.
- Fingerprinting — the 29.6% precision figure, and the survey to read instead of an SoK.
- Requests — two near-independent membership signals; a keyword for EasyList finds one of them.
- Study preregistration — the 47/62 homograph.
- Biases — denominator bias when the population you built is not the population you write about.
- Crawler — the crawled population this page's reporting-rate trap is computed on.
- Conferences — where to send the paper, once the related work is a method rather than a search.
Methodology and limitations of these figures
Every corpus figure on this page is produced by scripts/report_literature_review.mjs. The population rule is mechanical (title-start “SoK” or slug-start “sok-”); the topic and search-method folds are hand maps with a deciding sentence per paper, and the report throws if either map and the extracted 43 diverge. The Krawlers funnel numbers are quoted from their PDF and checked against paper.cols.txt. The IEEE S&P CFP, oaklandsok.github.io and the Vekaria arXiv page were fetched on 2026-08-27. Full query log, residue, quote checks and the review log: literature_review. Corpus-wide caveats: Corpus.
References
- [1]
- Stafeev, Aleksei; Pellegrino, Giancarlo (2024): "SoK: State of the Krawlers – Evaluating the Effectiveness of Crawling Algorithms for Web Security Measurements", in: Proceedings of the USENIX Security Symposium. (Link)
- [2]
- Vekaria, Yash; Beugin, Yohan; Munir, Shaoor; Acar, Gunes; Bielova, Nataliia; Englehardt, Steven; Iqbal, Umar; Kapravelos, Alexandros; Laperdrix, Pierre; Nikiforakis, Nick; Polakis, Jason; Roesner, Franziska; Shafiq, Zubair; Zimmeck, Sebastian (2025): "SoK: Advances and Open Problems in Web Tracking". arXiv preprint arXiv:2506.14057. (Link)
- [3]
- Rieder, Wolf; Raschke, Philip; Cory, Thomas; Sechting, Christian René; Kumar, Aditya; Küpper, Axel (2026): "SoK: After Decades of Web Tracker Detection, What's Next?", in: Proceedings of the IEEE Symposium on Security and Privacy. (Link)
- [4]
- Laperdrix, Pierre; Bielova, Nataliia; Baudry, Benoit; Avoine, Gildas (2020): "Browser Fingerprinting: A Survey", ACM Transactions on the Web 14(2). (DOI)
- [5]
- Usman, Warda; Zappala, Daniel (2025): "SoK: A Framework and Guide for Human-Centered Threat Modeling in Security and Privacy Research", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)
- [6]
- Birrell, Eleanor; Rodolitz, Jay; Ding, Angel; Lee, Jenna; McReynolds, Emily; Hutson, Jevan A.; Lerner, Ada (2024): "SoK: Technical Implementation and Human Impact of Internet Privacy Regulations", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)
- [7]
- Blessing, Jenny; Hugenroth, Daniel; Anderson, Ross; Beresford, Alastair (2025): "SoK: Web Authentication and Recovery in the Age of End-to-End Encryption", in: Proceedings on Privacy Enhancing Technologies. (DOI)
- [8]
- Alam, Mahbub; Rahman, Muhammad Lutfor; Paul, Sonjoy Kumar; Hays, Amy W.; Hussain, Aftab; Huq, Md Imanul; Saxena, Nitesh (2026): "SoK: PHILTER: Uncovering Security and Functional Gaps in AI-based Phishing Website Detection Literature via an LLM-based Reasoning Framework", in: Proceedings of the USENIX Security Symposium. (Link)
- [9]
- Mathews, Nate; Holland, James K.; Oh, Se Eun; Rahman, Mohammad Saidur; Hopper, Nicholas; Wright, Matthew (2023): "SoK: A Critical Evaluation of Efficient Website Fingerprinting Defenses", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)
- [10]
- Warford, Noel; Matthews, Tara; Yang, Kaitlyn; Akgul, Omer; Consolvo, Sunny; Kelley, Patrick Gage; Malkin, Nathan; Mazurek, Michelle L.; Sleeper, Manya; Thomas, Kurt (2022): "SoK: A Framework for Unifying At-Risk User Research", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)
