You have a list of domains — a Tranco slice, the third parties a crawl touched, the sites that set a cookie before consent — and a reviewer wants to know what kind of sites they are. Are the offenders news sites or shops? Does the effect hold outside adult content? Is the sample dominated by one sector?

This page is about turning a domain into a category you can defend. Its companion Website Selection is about which domains to put on the list in the first place; popularity ranking lives there, category and company data live here.

Before anything else: “classify a website” is not one measurement. It is at least six, they use different data, and they are not equally reliable.

These are different measurements with different failure modes, and they are easy to conflate — the structured extraction behind Use in Publications files them all under one label, because papers describe them all as “classifying websites”. The tracker row is where the conflation costs most, and unlike the others it has been measured: see The trap that has caught the most papers.

If you read one thing before starting, make it [1Vallina, Pelayo; Le Pochat, Victor; Feal, ´Alvaro; Paraschiv, Marius; Gamba, Julien; Burke, Tim; Hohlfeld, Oliver; Tapiador, Juan; Vallina-Rodriguez, Narseo (2020): "Mis-shapes, Mistakes, Misfits: An Analysis of Domain Classification Services", in: Proceedings of the ACM Internet Measurement Conference, pp. 598–618. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] (IMC 2020). It is the only large-scale independent audit of these services that exists, it is six years old, and nobody has redone it — which is itself something you should know before you build a result on top of one.

Note that several of these services are not meant for large-scale data mining, and querying them at research scale may violate their terms. MeasureTheWeb authors take no responsibility for your use of these services. Check with your IRB and legal department if you want to be sure that you can use them.

Vallina et al. [1Vallina, Pelayo; Le Pochat, Victor; Feal, ´Alvaro; Paraschiv, Marius; Gamba, Julien; Burke, Tim; Hohlfeld, Oliver; Tapiador, Juan; Vallina-Rodriguez, Narseo (2020): "Mis-shapes, Mistakes, Misfits: An Analysis of Domain Classification Services", in: Proceedings of the ACM Internet Measurement Conference, pp. 598–618. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] studied 13 domain classification services. DMOZ and Curlie were set aside as discovery directories rather than classifiers, leaving 11 in the label comparison. Nine label sources — OpenDNS, McAfee, FortiGuard, and Alexa, Bitdefender, Forcepoint, Dr.Web, Trend Micro and Websense as re-served through VirusTotal — were queried across the full 4,424,142 domains (September 2019, Alexa rankings aggregated Tranco-style). Three — Symantec, Trend Micro queried directly, and Webshrinker — were rate-limited down to the top-10k. The case studies in §6 cover nine services. Trend Micro appears in both groups because it was queried two ways, and Websense appears only via VirusTotal; counting “how many services” is genuinely ambiguous in this paper, which is why every figure below names its own group. Six years on it is still the reference, so it is worth having its actual numbers rather than the vibe of them.

Coverage varies by two orders of magnitude. FortiGuard and McAfee labelled around 94% of the 4.4M domains. OpenDNS managed 15% — its human submit-and-vote pipeline cannot keep up with domain registration. Alexa managed 0.53%. Combining services buys less than you would hope: the union of FortiGuard and McAfee reaches just over 98%, because their intersection was already large.

Coverage collapses outside the head of the distribution. McAfee and FortiGuard hold at ≥93% from the top-1k down into the long tail past rank 1M. Bitdefender falls from 93% to 27% over the same range, and Forcepoint from 98% to 48%. Symantec, Trend Micro (queried directly) and Webshrinker all exceed 96% on the top-10k — but their rate limits make anything larger infeasible, which is why the paper could only measure them on the top-10k in the first place. If your population is not a top list, the vendor's headline coverage number does not apply to you.

Documented taxonomies do not match observed ones. Of the services whose documentation could be compared against their output, every one except Symantec returned at least one category that is not in its own documentation. McAfee returned six categories that its own docs had marked deprecated since 2010. FortiGuard omits sensitive categories (e.g. homosexuality) from public documentation while still assigning them. Trend Micro's labels via VirusTotal matched its 2011 taxonomy better than its 2019 one, implying VirusTotal was sourcing from an older product.

Labels are multi-valued in ways you will not expect. Six of the label sources they observed — OpenDNS, McAfee, Dr.Web, Forcepoint, Trend Micro and Websense — can return more than one category per domain. In Dr.Web, 67% of labelled domains have several; in Forcepoint and Websense, under 1%. If your pipeline assumes one label per domain, you are silently dropping data from some services and not others.

Human labelling is genuinely hard. In the paper's own small manual experiment, two labellers using the OpenDNS taxonomy disagreed on 35.5% of domains, and reached only 71% accuracy against OpenDNS's own answer. Budget for this: it is the number your inter-annotator agreement is competing against, and it is worse than most people assume before they try it.

Vallina et al.'s case studies are the part of the paper that should change what you do. They took ground-truth sets and asked whether the services could recover them.

Adult content is the case where the services do work well — four of the nine measured recovered it at over 90%, and a fifth (OpenDNS) at 88%. That is a real, usable result, and it is the strongest single reason a topic categoriser earns its place in a measurement pipeline.

Mostly the corporate ownership, and not in your favour.

The one modern data point comes from a 2025/2026 arXiv preprint on LLM-assisted web measurement [2Bozzolan, Simone; Calzavara, Stefano; Cazzaro, Lorenzo (2026): "LLM-Assisted Web Measurements". arXiv:2510.08101, v3, 30 April 2026 (Link)], whose Appendix B explicitly reuses Vallina et al.'s service list. Its findings:

• Of the services they surveyed, three offer a free API at all — Google Topics, Cloudflare Radar and VirusTotal — and Topics is deprecated and scheduled for removal, leaving two. The rate-limited ones are hopeless at scale: Cloudflare's Domain Threat Intelligence API allows 100 requests per month, so 100k websites would take “roughly 30,000 days”; VirusTotal's 500/day cap puts the same job at “around 200 days”. For topic labels specifically that leaves one free API, VirusTotal, at 200 days per 100k domains — Cloudflare's unmetered endpoint is the other one, and it answers a different question (next bullet).
• The Cloudflare Radar API has no explicit rate limit, so they could benchmark it — but on country, not on topic, and the distinction matters because the number below gets quoted as if it were a categorisation result. Against a 7,780-site ground truth for target-audience country, Radar returned a label for 3,925 sites (50.4%) and, of those, agreed with the ground truth in 39.7% of cases. They hand-checked 50 disagreements: their own label was right in 49 (98%), Cloudflare's in 1 (2%). The reason is mechanical: Radar assigns a domain the country that issues the most DNS queries for it against Cloudflare's resolver, which is a measure of where the traffic is, not of who the site is for. No comparable 2020s coverage-or-accuracy number exists for anyone's topic labels.
• Every open-weight LLM they tested reached 100% coverage and at least 76% accuracy on the same data, without visiting the sites.

That is one service on one task, not a re-audit. But it is the only 2020s number available, and it points the same way the 2020 numbers did.

Each service below was checked by direct fetch on 7 August 2026; where something could not be verified from a primary source, a footnote says so. Where a lookup exists but no API does, that is stated plainly — it is the most common case, and it is what makes these services awkward for research.

These sell web filtering to enterprises. Their taxonomies are risk-oriented (Proxy Avoidance, Potentially Unwanted Program) with a general-interest tail bolted on. High coverage, coarse labels, essentially no bulk access.

The database papers cite as “McAfee URL categorization”, “TrustedSource”, “SmartFilter” or the “Customer URL Ticketing System” is one thing, and it no longer belongs to McAfee. McAfee Enterprise split into Trellix and Skyhigh Security; the URL-categorisation service migrated to them on 20 February 2024.¹⁾

• Lookup: trustedsource.org — live, footer reads “Copyright © Musarubra US LLC” (Trellix's legal entity), and the product dropdown offers both Trellix products (Real-Time Database, Endpoint Security Web Control, SmartFilter) and Skyhigh's Secure Web Gateway. One database, two companies.
• API: none. The page states: “this Web-based service is designed for occasional use by individuals. If you have hundreds or thousand of URLs you would like to test, please write to us about your needs… If we see heavy use of this Web-based service by automated programs, we reserve the right to deny access.” Bulk requests go to sites@trellix.com.
• Advantages: joint-highest measured coverage of any service (~94% in 2020, tied with FortiGuard), holding into the long tail; multi-label; automated with manual oversight.
• Disadvantages: coarse labels, unsuitable for marketing or behavioural analysis; documentation carries categories deprecated since 2010 that the service still returns; no programmatic access, so at research scale you are either asking permission or violating the notice above.

• Lookup: fortiguard.com/webfilter (single URL or IP, with rating history). Published taxonomy: fortiguard.com/webfilter/categories — 93 named categories in 7 groups (Adult/Mature Content, Bandwidth Consuming, General Interest – Business, General Interest – Personal, Potentially Liable, Security Risk, Unrated), each with a one-sentence definition. Counted from the live page on 2026-08-07.
• API: no public bulk API. FortiOS exposes /api/v2/cmdb/webfilter/ftgd-local-rating but that requires a licensed FortiGate appliance, not a research credential.
• Advantages: the joint-best coverage in the 2020 audit (~94%, stable into the long tail); the most completely and publicly documented taxonomy of any vendor here, which makes it the best choice when you need to name categories in a paper; real-time classification.
• Disadvantages: security-shaped granularity; documentation omits some sensitive categories the service nonetheless assigns; no bulk access; recovered only 7.7% of a tracking ground truth.

Broadcom acquired Symantec Enterprise in 2019; WebFilter and WebPulse are still operated under the Symantec brand. Both the lookup tool and Broadcom's WebFilter/WebPulse documentation served HTTP 200 on 2026-08-07, which is the strongest statement we can make from outside — Broadcom publishes categorisation-update notices on its status site, but we could not fetch a specific one to date them.

• Lookup: sitereview.symantec.com — alive. It is an Angular single-page app, so a plain curl returns a splash screen with a <noscript> notice; that is a JavaScript UI, not a dead service. Category definitions live in Broadcom TechDocs.
• API: none public. Single-URL lookup plus a re-review request workflow.
• Advantages: high accuracy on top-tier domains; the only service in the 2020 audit that returned no undocumented categories; good for threat assessment and content filtering.
• Disadvantages: less diverse taxonomy than the marketing-oriented services; rate-limited hard enough that the 2020 audit could only measure it on the top-10k; poor long-tail coverage.

• Lookup: Site Safety Center — alive. Taxonomy documented at docs.trendmicro.com: 7 top-level groups, roughly 80–90 sub-categories.
• API: none public; same occasional-individual-use posture as Trellix.
• Advantages: high coverage for popular domains, rapid updates, labels aligned with threat intelligence.
• Disadvantages — the earlier version of this page asked whether there were any, and the audit answers it: coverage is poor everywhere except the very top of the list. Trend Micro assigned an adult-related label to only 11.8% of a curated adult-content set (it returned any label at all for just 12.1% of them) and a tracking-related label to 0.9% of a tracking set. Its labels through VirusTotal also correspond to an older taxonomy than the one it documents, so which route you query by changes the answer.

• Lookup: not fully public — the Site Lookup Tool sits behind a support.forcepoint.com login, and the category list is published inside the Cloud portal help. Forcepoint's product page claims “more than 95 URL categories in more than 50 languages”.
• API: a Management API exists for administering a deployment you already own, not for third-party lookups. An XLabs submission tool accepts up to 10,000 URLs for classification requests, not retrieval.
• Advantages: fast, scalable label updates; the best performer in the whole 2020 audit on tracking domains (15.1% correct) and strong on adult content (94.9%).
• Disadvantages: needing an account to look a single domain up makes it the least accessible service here; coverage drops from 98% to 48% between the top-1k and the long tail; multi-labelling exists but fires on under 1% of domains.

• Lookup: online.drweb.com (“Check URL”) and vms.drweb.com/online. Parental-control categories (Adult content, Violence, Weapons, Gambling, Drugs, Terrorism, Obscene language, plus optional ones) are documented in fragments rather than as one canonical list.²⁾
• API: none.
• Advantages: minimalistic and consistent labels; recovered 92.4% of adult content, which is its whole design goal.
• Disadvantages: very low coverage outside its niche — it returned any label for 9.0% of tracking domains and 0.3% of CDN domains, and a correctly related label for 0.0% of both; 67% of its labelled domains carry several categories, the highest multi-labelling rate measured, which breaks single-label pipelines silently.

Three corpus papers categorise domains with Cyren, so it is worth knowing where it went. As of 2026-08-07 both cyren.com and www.cyren.com 301 to data443.com/cyren-threat-intelligence — the brand survives as a Data443 threat-intelligence product line. If you are reproducing one of those papers, you are not querying the same organisation they did.

• Zvelo (zveloDB / zveloCAT) — a current, actively marketed competitor to the six above. Self-reported ~500 categories over 200+ languages, delivered as API, SDK or raw feed. Sales-quoted only, no public pricing or free tier, and its coverage claims are self-reported and have never been independently audited.
• Palo Alto Networks PAN-DB — free public single-site lookup (“Test A Site”), up to four categories per URL, with per-category benign test URLs published for verification. The total category count varies by PAN-OS version (figures between 57 and 90+ circulate), so quote it against a version rather than in the abstract.

Finer taxonomies aimed at advertising rather than filtering, so the labels are the ones an IAB-shaped research question actually wants — at the cost of coverage, and of money.

• Advantages: high-quality data; industry, region- and origin-based popularity. A PETS 2021 paper reports classifying into “24 possible categories” via the category feature; SimilarWeb does not publish a canonical taxonomy size, so pin the number you observe rather than quoting that one.
• Disadvantages: the official API v5 is paid on a “Data Credits” model with no published price list — you get a sales quote. The unofficial endpoint is a clear terms violation: SimilarWeb's terms prohibit “use any robot, spider, scraper, or other automated means to access or monitor the Platform, Site or Applications for any purpose, or to copy, access, collect, download, or otherwise obtain data, content or other information”.³⁾
• See SimilarWeb for documentation of the unofficial API, code and output example, and note the date it was last confirmed working.

Still alive, and not renamed. DNSFilter acquired Web Shrinker on 17 April 2018;⁴⁾ eight years on, webshrinker.com is live with its own branding and product line (Offline Database, Domain API Feed, Screenshot API Feed) sold through DNSFilter's pricing page. Its homepage claims 380 million websites categorised with a five-day re-scan cycle.

• Advantages: offers both a “Simplified Categories” scheme and an IAB-aligned taxonomy, which is what makes it attractive for advertising research; automated real-time updates; the widest label vocabulary of the audited services (299 observed labels against 401 documented).
• Disadvantages: precision and granularity vary; 103 of its documented categories were never observed in the 2020 audit, so the documentation over-promises; rate limits confined the audit to the top-10k; commercial terms are now DNSFilter's.

Two more appear in the corpus, and their fates illustrate the risk of building on a small vendor.

• Klazify — two corpus papers use it: an IMC 2023 study mapping dead links to categories, and a USENIX Security 2024 brand-impersonation study that took its IAB classification and manually validated the result. The domain resolves and is served behind Cloudflare, but Cloudflare's bot interstitial returned HTTP 403 to both curl and a headless browser on 2026-08-07, so we could not confirm anything about the current product from the site itself.
• Website Categorify — used by a CCS 2020 paper on DNS hijacking, which reported 195 domains left uncategorised. As of 2026-08-07 websitecategorify.com does not resolve at all: the domain is gone, and with it any hope of reproducing that paper's categorisation.

Neither has been independently audited. Treat the accuracy of a small vendor as unknown until you measure it on your own sample — and treat its continued existence as unknown too, which is an argument for publishing your label file rather than the query script.

The living successor is Cisco Umbrella Investigate, which offers domain/IP categorisation through a documented API — but it is an enterprise product with no published price list and no free tier, sold through a sales process. This is a strict downgrade for researchers: the community system was free and transparent, and what replaced it is neither.

For the historical record, OpenDNS's properties in the 2020 audit remain instructive about crowd-curation in general: 15% coverage, human submission and voting as the bottleneck, five security categories closed to community submission, and domains with as many as 58 rejected label proposals.

DMOZ / the Open Directory Project closed in 2017; Curlie is its volunteer-run continuation and is the one free bulk source of human-curated website categories that still exists.

• Bulk dump: curlie.org/directory-dl redirects to the current archive. Tab-separated text, gzipped. Fetched 2026-08-07: 169 MiB (177,289,960 bytes), last modified 2 February 2026, covering ~2.9 million entries with the category hierarchy, URLs, titles and editorial descriptions. The filenames still say “RDF” for historical reasons; the format is TSV.
• Cadence: Curlie's documentation says “we strive to pull a fresh copy from the Curlie database every month”. The copy we fetched was six months old, so check Last-Modified before you cite a snapshot date rather than assuming monthly freshness.
• Taxonomy: 15 top-level categories in the English tree — Arts, Business, Computers, Games, Health, Home, News, Recreation, Reference, Regional, Science, Shopping, Society, Sports, Kids and Teens — deeply hierarchical below that, with 92 further language trees alongside. Counted from curlie.org/en on 2026-08-07. Homepage2Vec predicts into 14 of these, dropping Regional; the LLM benchmark in LLM Classification reduces the same taxonomy to 13. If three sources hand you 15, 14 and 13 Curlie categories, none of them is a typo — check which ones each one dropped before you compare their numbers.
• Advantages: free, redistributable, human-curated, multilingual, and it is the training set behind essentially every open website classifier — including Homepage2Vec.
• Disadvantages: 2.9M URLs is tiny against the modern web and skews to older, established sites; many branches have not been touched in years. The most important caveat is under-labelling: editors assign one or two relevant categories, not all of them. When Homepage2Vec's authors had crowdworkers re-label 807 Curlie sites, Curlie had supplied 836 labels (1.04 per site) and the crowd assigned 2,088 (2.59 per site) — a 2.5× increase [3Lugeon, Sylvain; Piccardi, Tiziano; West, Robert (2022): "Homepage2Vec: Language-Agnostic Website Embedding and Classification", in: Proceedings of the International AAAI Conference on Web and Social Media, pp. 1285-1291. (DOI)]. Treated as ground truth, Curlie will make any multi-label classifier look far less precise than it is.
• Vallina et al. deliberately excluded DMOZ and Curlie from their comparison, on the grounds that a discovery directory and a classification service are answering different questions. That distinction is worth keeping.

• API: GET /api/v3/domains/{domain} returns a categories attribute, “a dictionary mapping that relates categorisation services with the category it assigns the domain to”. Documented at docs.virustotal.com.
• Which vendors: Dr.Web, Forcepoint ThreatSeeker, BitDefender, Sophos, Trend Micro, Websense and (legacy) Alexa, among others — so several of the vendors above reach you through VirusTotal without you querying them.
• Rate limit: the free tier is 500 requests per day and 4 requests per minute.⁵⁾ The same page adds that the Public API “must not be used in commercial products or services”. At 500/day, a 100k-domain study takes 200 days. Vallina et al. had an academic key at 20k/day; we could find no self-serve academic application form in 2026, so budget for a direct conversation with VirusTotal rather than assuming access.
• Advantages: aggregates many providers in a single call; widely used and easy to cite.
• Disadvantages: the integration is lossy. In the 2020 audit, several services returned labels when queried directly but not through VirusTotal, and Trend Micro's VirusTotal labels tracked its 2011 taxonomy rather than its 2019 one. You do not control which product version you are reading, and “we used VirusTotal categories” does not identify the underlying source.

None of these existed when the older parts of this page were written, and all three appear in corpus papers from 2019 onward.

Two different products, easily conflated:

• Radar API — free, no explicit rate limit, data licensed CC BY-NC 4.0 (non-commercial only, which matters for industry co-authors). Radar is a popularity-ranking product, not a categoriser, and this is the single most common misreading of it on this page's topic. It does return a categories array, but only from GET /radar/ranking/domain/{domain}, only for the POPULAR ranking, and only for the most recent ranking — so a domain gets categories only if Radar ranks it, and Radar ranks an ordered top 100 (global and per country) plus unordered buckets up to one million.⁶⁾ There is no Radar endpoint that categorises an arbitrary domain. The 50.4% / 39.7% figures quoted above are Radar's country output, not its categories [2Bozzolan, Simone; Calzavara, Stefano; Cazzaro, Lorenzo (2026): "LLM-Assisted Web Measurements". arXiv:2510.08101, v3, 30 April 2026 (Link)]; nobody has published a coverage or accuracy figure for Radar's topic categories at all.
• Domain Intelligence, now under Security Center Intel — GET /accounts/{account_id}/intel/domain?domain=… returns content_categories. This is the one the corpus papers used: [4Ruth, Kimberly; Kumar, Deepak; Wang, Brandon; Valenta, Luke; Durumeric, Zakir (2022): "Toppling Top Lists: Evaluating the Accuracy of Popular Website Lists", in: Proceedings of the 22nd ACM Internet Measurement Conference, pp. 374–387. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] labelled Cloudflare-managed sites with it, and [5Ruth, Kimberly; Fass, Aurore; Azose, Jonathan; Pearson, Mark; Thomas, Emma; Sadowski, Caitlin; Durumeric, Zakir (2022): "A World Wide View of Browsing the World Wide Web", in: Proceedings of the ACM Internet Measurement Conference, pp. 317-336. (DOI)] built a whole global browsing study on it. It needs an API token with Intel permission scoped to an account; whether it is available on the free plan is not stated in Cloudflare's public docs.⁷⁾

The best worked example of using it well is [5Ruth, Kimberly; Fass, Aurore; Azose, Jonathan; Pearson, Mark; Thomas, Emma; Sadowski, Caitlin; Durumeric, Zakir (2022): "A World Wide View of Browsing the World Wide Web", in: Proceedings of the ACM Internet Measurement Conference, pp. 317-336. (DOI)] (IMC 2022): the API places domains into 26 super-categories and 114 categories; the authors manually validated ten random websites per category and kept only categories that reached 80% accuracy, excluding 19 of them and merging others, ending at 22 super-categories and 61 categories. Notably, Search Engines and Social Networks were among the categories that failed — two of the most important classes on the web. Copy this procedure.

Classifies text, not domains, so you have to fetch and extract the page yourself — but the taxonomy is the advertising-shaped one many privacy questions actually need. Papers in the corpus report it variously as 27 level-1 categories, 35 categories with 400 subcategories, and ~620 AdWords-style buckets, depending on the year and API version; state the version and the date you queried. Priced per unit of text, no free research tier of consequence.

The browser-side interest taxonomy (349 topics, assigned by a model Google trained on a manually curated set of 10,000 domains) is a measurement target rather than a tool — [6Jha, Nikhil; Trevisan, Martino; Leonardi, Emilio; Mellia, Marco (2023): "On the Robustness of Topics API to a Re-Identification Attack", Proceedings on Privacy Enhancing Technologies 2023(4):66-78. (DOI)] attacks its re-identification properties. Treat it as an object of study, not a categoriser you can call; the 2026 survey of classification services reports it as deprecated and scheduled for removal [2Bozzolan, Simone; Calzavara, Stefano; Cazzaro, Lorenzo (2026): "LLM-Assisted Web Measurements". arXiv:2510.08101, v3, 30 April 2026 (Link)].

Amazon retired Alexa.com on 1 May 2022, per its own end-of-service notice (“we will be retiring Alexa.com on May 1, 2022”).⁸⁾ Its ranking service and its category service died together.

• Why it still matters: it appears in 12 of the 330 corpus papers that categorise websites, under eight different spellings, and papers published as late as 2024 still use it because their data collection predates the shutdown. If you are reading such a paper, the labels are from a dead service with a documented 0.53% coverage rate.
• What it was: highly granular (7,557 distinct labels observed through VirusTotal, against 279,716 categories in Alexa's own directory, which was derived from DMOZ), and capped at 500 sites per category — hence the tiny coverage. Leaf labels came back in the language of the branch, so the same concept appeared as Arts, Artes and Kultur.

Ranked for a researcher who needs topic labels for tens of thousands of domains. Coverage figures are from [1Vallina, Pelayo; Le Pochat, Victor; Feal, ´Alvaro; Paraschiv, Marius; Gamba, Julien; Burke, Tim; Hohlfeld, Oliver; Tapiador, Juan; Vallina-Rodriguez, Narseo (2020): "Mis-shapes, Mistakes, Misfits: An Analysis of Domain Classification Services", in: Proceedings of the ACM Internet Measurement Conference, pp. 598–618. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)], measured in 2019 on 4.4M domains, and are the most recent independent numbers that exist.

The service catalogue above is a list of things you cannot get at scale, cannot redistribute, and cannot reproduce. Running your own classifier avoids all three problems, and since 2022 the open options have been good enough that this is the default recommendation for a topic-labelling task.

[3Lugeon, Sylvain; Piccardi, Tiziano; West, Robert (2022): "Homepage2Vec: Language-Agnostic Website Embedding and Classification", in: Proceedings of the International AAAI Conference on Web and Social Media, pp. 1285-1291. (DOI)] (ICWSM 2022, EPFL) is the reference open model. It takes a homepage — HTML text, metadata, optionally a screenshot — and emits both a 100-dimensional embedding and independent multi-label probabilities over Curlie's 14 top-level categories. It is language-agnostic by construction, built on XLM-R sentence embeddings, and trained on more than two million Curlie-labelled sites in 92 languages (April 2021 snapshot).

• Code: github.com/epfl-dlab/homepage2vec, MIT licensed, pip install homepage2vec. Not actively maintained — last commit 22 January 2024, no tagged releases.
• Training data: the curated Curlie snapshot is released separately at figshare under CC BY 3.0. This is the practical way to get an aligned, labelled Curlie set without parsing the raw dump.

Its documented weaknesses: rare classes are much worse than frequent ones (Kids_and_Teens is ~1% of the data, Business ~27%); the visual branch requires a full Selenium and Chrome pipeline for a limited gain the authors themselves call out; and the Curlie under-labelling problem above depresses its apparent precision — crowd re-labelling raised measured macro precision on the affected subset from 0.734 to 0.873.

“Multilingual Taxonomic Web Page Categorization Through Ensemble Knowledge Distillation” [7Ye, Eric; Bai, Xiao; O'Hare, Neil; Asgarieh, Eliyar; Thadani, Kapil; Perez-Sorrosal, Francisco; Adiga, Sujyothi (2024): "Multilingual Taxonomic Web Page Categorization Through Ensemble Knowledge Distillation", IEEE Transactions on Knowledge and Data Engineering 36(11):6614-6627. (DOI)] (IEEE TKDE 36(11):6614–6627) is Yahoo Research's production contextual-ad-targeting classifier, an extended version of a KDD 2022 paper. A unified multilingual Transformer classifies a page from either full content or the URL alone — URL-only matters because ad bid requests arrive as bare URLs — with class-based loss re-weighting for label skew and an ensemble of teacher models distilled into one lightweight student.

Its taxonomy is Yahoo's proprietary 442-category, 5-tier Interest Category scheme over five languages (English, Spanish, French, Portuguese, Traditional Chinese), trained on ~248k editorially annotated pages. It reports “at least 2.6% improvement in mean average precision across 5 languages” over single-teacher distillation and “at least 37% better mAP than the legacy tree-based models” in production.

Neither the model, the data nor the taxonomy is released. Read it for the method — the URL-only-and-content-in-one-model trick, and the distillation setup — not as something you can use. It also does not compare against Homepage2Vec, and could not sensibly: 442 proprietary categories versus 14 public ones is not a shared benchmark.

• WebOrganizer (ICML 2025) — two 24-category taxonomies (Topic and Format) produced by prompting Llama-3.1-405B over Common Crawl and distilling into 140M-parameter classifiers, released on HuggingFace with -NoURL variants. Built for pre-training data curation, so it classifies pages, not sites — you would have to aggregate per-page predictions to a domain yourself.
• SoAC (DocEng 2025) — 195,495 websites collected in 2024, labelled with 10 coarse sectors and 148 fine-grained industry categories, MIT licensed, including extracted site text. Their own classifier reaches 72.6% accuracy on it. This is the closest thing to an open industry-sector ground truth, which the company datasets below otherwise monopolise.
• WebKB (CMU, cs.cmu.edu/~webkb) — still cited, but it is 7 classes of university CS-department pages in English. It is a text-classification benchmark, not website-category ground truth. Do not use it to justify a claim about the web.

The evidence outside the corpus is a 2025/2026 preprint, [2Bozzolan, Simone; Calzavara, Stefano; Cazzaro, Lorenzo (2026): "LLM-Assisted Web Measurements". arXiv:2510.08101, v3, 30 April 2026 (Link)], which by its authors' account is the first systematic study of LLMs for web-measurement classification.

It evaluates five open-weight, self-hosted models (gpt-oss:20b, llama3.3:70b, mistral-small3.2:24b, phi4-mini:3.8b, qwen3:32b, all via Ollama, temperature 0) on three tasks with hand-built benchmarks, in two modes: URL string only, and URL plus live browsing through LangChain and Playwright.

The number that matters for this page: on the same 13-category benchmark, Homepage2Vec scores 0.60 Jaccard and 0.55 macro F1 — the paper's words are that “a general-purpose LLM like gpt-oss can label websites with higher accuracy than a classic machine learning model specifically trained for website categorization.” A stock open-weight instruct model, with no fine-tuning, beat the field's reference specialist classifier.

Two cautions before you repeat that sentence.

1. 0.55 is not a refutation of the 0.90 in Homepage2Vec's abstract, and anyone who puts the two side by side has compared different evaluation regimes. 0.90 is the balanced per-class setup; Homepage2Vec's own authors report 0.634 macro F1 on an unbalanced test set, which is the regime a benchmark like this one is in. See Homepage2Vec.
2. 0.55 is nonetheless below that 0.634, on a task that drops one of the 14 classes, and the preprint never states how it ran the baseline. Homepage2Vec emits independent calibrated probabilities per class, so turning it into a multi-label prediction requires a decision threshold — its own paper uses 0.5 on calibrated output — and the word “threshold” does not occur anywhere in the preprint.¹⁰⁾ A head-to-head in which the winner ran the loser's model, without reporting the one hyperparameter that decides the loser's score, is a result to re-run before you build on it. Both models are free and the benchmark is small.

Three practical consequences:

1. URL-only works for coarse questions and fails for topics. Every model reached at least 0.86 accuracy on the binary government task from the URL string alone, and exceeded 0.75 on both accuracy and macro F1 for the 10-way country task. Topic classification is where live access earns its cost: 0.51 → 0.65 Jaccard for the best model. If all you have is a domain list with no crawl, budget accordingly.
2. Self-hosting is free in dollars and expensive in wall-clock. On one A40 GPU, median under 2 s and mean under 4 s per site: ~3 days for 100k sites URL-only, ~4 days 9 hours with live access, and ~30–44 days for the Tranco top 1M. A hosted small model is cheaper in time — at August 2026 list prices for a nano-tier model and ~300 input / 20 output tokens per domain, URL-only classification of 100k domains costs on the order of a few dollars, roughly halved by a batch API.¹¹⁾ The tradeoff is reproducibility: an open-weight model you pin by digest is reproducible five years from now, and gpt-4o-2024-08-06 is not.
3. Ensembling helps because single models are unstable. A 2025 evaluation of ten LLMs zero-shot against the IAB taxonomy on 8,660 human-annotated samples names “inconsistency, hallucination, category inflation, and misclassification” as the individual-model failure modes, and reports up to 65% F1 improvement from majority-vote ensembling over the strongest single model.¹²⁾

Constrain the output. Every failure mode above gets worse when the model is free to invent a category name. Give it the taxonomy in the prompt, force a structured output (a JSON schema or a constrained-decoding grammar), reject anything outside the enum, and count the rejections — that count is a result.

Three things nobody has measured yet, so do not assert them and consider measuring them yourself:

• Label instability across seeds and temperature for website classification specifically. The one systematic paper set temperature to 0 to sidestep the question rather than quantify it.
• Behaviour on parked, dead and holding-page domains — a large fraction of any long-tail sample, and a case where “visit the site” gives the model a page that is not about anything.
• A formal agreement statistic (Cohen's or Fleiss' kappa) between LLM labels and human annotators for this task. Nobody has published one. What does exist, on those same 150 hand-relabelled sites, is a Jaccard similarity of 0.73 between the human labels and gpt-oss's predictions, against 0.75 between the human labels and the benchmark's own ground truth — the model is as close to careful humans as the benchmark is. Which is also the reason to read the 0.65 Jaccard above as a floor rather than a ceiling: a multi-label benchmark that under-labels punishes a model for being right. (For scale on how much under-labelling there is: the benchmark's own heuristic label-construction matched the human label set exactly on 86 of 150 sites (57%), where Lugeon et al.'s Curlie snapshot matched on 67 (45%). Both figures are about the ground truth, not about any model.) And note what that pair cannot tell you: those 150 sites were relabelled by the preprint's own authors, with no second annotator and no agreement statistic of any kind — no kappa, no Krippendorff's alpha, nothing.¹³⁾ So 0.73 says the model tracks that annotator's labels. It does not say how fuzzy the task is, and a single-annotator relabelling cannot be used to argue that the remaining label noise is small — the disagreement you have not measured is exactly the quantity in question. If you relabel a sample, use two annotators and report the agreement; Validating Your Labels is the whole procedure.

Studies that check compliance against a national law need to know which country a site is for, and this gets treated as a minor preprocessing step. It is not. There is no service that answers it, the available proxies measure different things, and none of them is a target-audience label:

• The TLD. Precise where it exists and absent where it matters — a ccTLD is strong evidence, but generic TLDs carry no country at all, and that is most of the head of any list.
• CrUX country lists, via Tranco. Tranco's list-generation API takes filterCRUX, filterCRUXType (global / country / region / subregion), filterCRUXValue (e.g. a list of country codes) and filterCRUXMonth, so you can generate a country-restricted list reproducibly, with a permalink, from a free account.¹⁴⁾ This is the cleanest free source and it is under-used. But it ranks by page loads from Chrome users in a country, which is popularity, not audience — and the overlap is severe: the union of five country top-10k lists (China, Germany, Italy, Korea, Turkey) is only 18,718 domains rather than 50,000, with 4,017 domains common to all five, and a preliminary labelling built from these lists is incompatible with the site's own TLD in at least 25% of cases for every label [2Bozzolan, Simone; Calzavara, Stefano; Cazzaro, Lorenzo (2026): "LLM-Assisted Web Measurements". arXiv:2510.08101, v3, 30 April 2026 (Link)].
• Site language. A good proxy, and cheap, but it splits badly on English, Spanish, Arabic and Portuguese, which is a large share of the web.
• Host IP or CDN location. Measures where bytes are served from. Behind a CDN — most of the head of any list — it tells you about the CDN. See IP Classification and Crawling Location.
• Cloudflare Radar's country field. DNS-query volume by country against Cloudflare's resolver: 50.4% coverage and 39.7% agreement on the one benchmark that exists (see Has Anything Changed Since 2020?).

So combine them and report the combination. The one systematic evaluation on this task got its ground truth by starting from CrUX and then correcting with site language and host IP, and still hand-inspected the result — and an LLM given the URL alone beat the best single proxy on that benchmark. Whatever you do, say which proxies you combined and how you broke ties: “we classified sites by country” is not a method, and a compliance result is only as good as the jurisdiction assignment underneath it.

Pick the taxonomy before the service, because it is much harder to change later, and because 75% of the corpus papers that name a taxonomy at all name a bespoke one and do not even state how many classes it has (see Use in Publications) — which makes their results incomparable with everyone else's.

• IAB Tech Lab Content Taxonomy — the advertising industry's reference schema, and the right choice if your question is about ad targeting or if you want to be comparable to Webshrinker's output. Machine-readable TSV in the repo under Content Taxonomies/, CC BY 3.0, up to four tiers with a relational ID system. Versions 1.0 through 3.1 sit side by side; 2.x → 3.0 is a breaking change and IAB says the two must not be used together. Category rows counted from the TSVs on 2026-08-07, excluding the two header lines: 2.2 has 1,196 rows, 3.0 has 703, 3.1 has 704 — so 3.1 is marginally larger than 3.0. Tier-1 rows: 37 in 3.1, but only 31 of those carry a numeric Unique ID; the other six (Communication, Entertainment, Events, Genres, Holidays, Sensitive Topics) are alphanumeric extension rows. State which convention you counted with — three independent counts of this one file in preparing this page produced three different numbers. Vendor marketing claiming that 3.x “expands to 1,500+ categories” is wrong in both directions — 3.x is roughly 40% smaller than 2.2, because it consolidated and restructured rather than adding. Count the rows in the file yourself; it takes ten seconds and the summaries are not reliable.
• Curlie's 15 top-level categories — the right choice if you want free ground truth, an open model that already predicts into it, and a hierarchy you can prune to whatever depth your sample supports. Check that your classes are actually in it before you commit. The 15 are Arts, Business, Computers, Games, Health, Home, News, Recreation, Reference, Regional, Science, Shopping, Society, Sports, and Kids and Teens — a directory taxonomy from the early 2000s. There is no top-level class for social media, finance and banking, education, government, or online services, and none at all for adult content; those questions land inside Business, Society, Reference or Computers, or nowhere. If your paper is about the privacy of banking sites, or trackers on education sites, or anything on social platforms, a Curlie-trained classifier will not give you the class you are measuring, and pruning to top level will hide that rather than fix it.
• A vendor's own list — fine, but pin the version, and expect to discover categories that are not in the documentation. Vallina et al. found at least one undocumented category in eight of the nine services whose documentation they could compare against its output.
• A standard industry code (NAICS, SIC, NACE) — the right choice for a question about firms rather than content, and the natural join key to the company datasets below. Exactly one paper in our corpus did this.

Whatever you pick: report the number of classes and the pruning depth. “We categorised sites using IAB” is not reproducible: IAB 3.1 has 31–37 tier-1 categories depending on whether you count the extension rows, and 704 rows in total. Reporting at one depth or the other is two very different measurements.

Compared to the services above, the following datasets are company-oriented instead of website-oriented. Some include the company's website, but this matching might be incomplete and might cause the following issues:

• If a company owns multiple websites:
  • Likely only the main website will be listed.
  • This is especially pronounced with international versions of the website.
• Likewise, the dataset may contain multiple companies for a given website:
  • Because of sister companies in a corporate group.
  • Many small businesses list social media as their website. Sometimes this link does not include the full path, so a single-person company might indicate facebook.com as its domain.

• Advantages:
  1. Freely available from PeopleDataLabs' website, which also includes good documentation. No registration is needed. CC BY 4.0, updated quarterly.
  2. Confirmed 2026-08-07: over 22 million companies, exactly 10 fields — country, founded, id, industry, linkedin_url, locality, name, region, size, website.
• Disadvantages:
  1. Based on LinkedIn profiles that are self-reported — prone to adversarial data.
  2. Only a subset of PeopleDataLabs' full dataset. The “22M of 70M rows” framing is longstanding on this page; the 22M is confirmed, the 70M total could not be re-confirmed in 2026 and PDL's marketing page now cites 23.8M+ without saying which corpus that is.

TODO: cite Machine Learning Compliance Analysis for Email Regulation when it is public.

Crunchbase is proprietary. You can check individual entries (e.g. Google) on their website in visual form, which is useful for a quick content check.

• Advantages:
  1. Very rich data: website, rank, region, industries, financial data.
• Disadvantages:
  1. URLs are extremely noisy — they are not the priority.¹⁵⁾
  2. Focuses mostly on variables useful for investments and market competitiveness.
  3. Academic access is no longer publicly documented; budget for a sales conversation.

TODO: cite Machine Learning Compliance Analysis for Email Regulation when it is public.

Orbis (Bureau van Dijk, acquired by Moody's in 2017 and now marketed as Moody's Orbis) is in many ways similar to Crunchbase. It is proprietary but offers academic licensing, and Moody's claims coverage of more than 635 million companies. It is primarily company-focused but contains URLs, and it covers the longer tail of private companies — which also means the data is noisier.

• Advantages:
  1. Longer tail data than Crunchbase.
  2. Academic access is real and still available — but it is arranged through your institution's library as a site licence or SSO entitlement, not through a public request form. Check your library's database list before contacting Moody's.
• Disadvantages:
  1. Extremely noisy.
  2. Fewer fields than Crunchbase.

There are many scraping services for datasets like LinkedIn, Glassdoor, Yahoo Finance business information, Yelp businesses, Indeed, etc. Their use might violate the terms and conditions of the primary data sources, but the whole industry is built on scraping each other's data, so you might be causing very limited harm. Nevertheless, check with your IRB and potentially your legal department before using these services.

• Coresignal: runs an explicit, application-based pro bono programme for academic and NGO researchers, with enriched company, job and workplace datasets given free to selected applicants; it expects the resulting research to be published within 12–24 months. This is the only concretely documented academic programme among the scraping services listed here. A free developer tier (200 Collect + 400 Search credits) exists for testing.
• BrightData: confirmed free tier of 5,000 credits per month plus a free-samples page. Its academic-licensing offer is informally reported rather than documented — do not plan around it.
• Amazon AWS Marketplace offers various datasets through external services.
• Kaggle: often contains smaller samples of various company datasets.

Multiple lists exist, mostly maintained for child protection in routers and similar. They are simple to access, and you can directly download a large list of URLs. For these specific categories, prefer them to a general classification service. That is what Vallina et al.'s case studies show: taking curated lists as ground truth, no commercial service recovered more than 15% of the tracking domains, and only five of nine recovered the adult-content domains well.

• https://github.com/Bon-Appetit/porn-domains: daily updated list of porn websites.
• https://github.com/hagezi/dns-blocklists: DNS blocklists for adult content, piracy, gambling, security (scam, malware), and privacy (ads, tracking).
• https://github.com/search?q=pihole%20blocklist&type=repositories: search for pi-hole blocklists for specific uses.
• https://oisd.nl/, https://firebog.net/
• https://dsi.ut-capitole.fr/blacklists/index_en.php contains filtering designed for blocking university traffic. Contains many lists, from porn to malicious or misinformation content.
• Citizen Lab test lists: per-country lists of politically and socially sensitive URLs with a documented category scheme, maintained for censorship measurement and used as such in the corpus.

Visit individual privacy-oriented pages for more details regarding classification of Requests, Cookies, Fingerprinting, and JavaScript.

• Martech provides data on 17k companies in the advertising industry. They are easy to download after registering on https://martechmap.com/; search for martech_data_N.json files in the network tab of your browser's dev tools.
• IAB Europe lists advertising and tracking third parties. On https://iabeurope.eu/vendor-list-tcf/, open the source code and find the table <table id=“tablepress-72” class=“tablepress tablepress-id-72”>. Copy that table into an HTML-table-to-CSV converter.

Used by several corpus papers on misinformation and political advertising, and worth knowing exists rather than rebuilding: Media Bias/Fact Check and AllSides both publish per-outlet bias and factuality ratings. Both are human-rated by small teams with their own editorial stance — report which one you used and treat the ratings as one organisation's judgement, not ground truth.

This is the part most papers skip, and it costs an afternoon.

The protocol is the one [5Ruth, Kimberly; Fass, Aurore; Azose, Jonathan; Pearson, Mark; Thomas, Emma; Sadowski, Caitlin; Durumeric, Zakir (2022): "A World Wide View of Browsing the World Wide Web", in: Proceedings of the ACM Internet Measurement Conference, pp. 317-336. (DOI)] used and the one Vallina et al. recommend: draw a stratified random sample, have two people label it independently while blind to the service's answer, report agreement, report per-category precision with a confidence interval, and drop the categories that do not clear a floor.

category_validation.py

#!/usr/bin/env python3
"""Validate a website categoriser's output the way a reviewer will ask you to.
 
Whatever assigned your categories -- a vendor API, Homepage2Vec, an LLM -- you
are claiming that a label means what its name says. This script produces the
evidence for that claim, and it is deliberately boring: stratified sample,
blind double annotation, per-category precision with a confidence interval,
inter-annotator agreement, and a drop rule for categories that do not hold up.
 
Standard library only: no numpy, no scipy, no network.
 
    # 1. draw the sample and write two blind annotation sheets
    python3 category_validation.py sample labels.csv --per-category 25 --seed 20260807
 
    # 2. annotate sheet_a.csv and sheet_b.csv by hand (fill the `verdict` column
    #    with the category you think is right, or `none` if nothing fits)
 
    # 3. score
    python3 category_validation.py score labels.csv sheet_a.csv sheet_b.csv --floor 0.80
 
`labels.csv` is `domain,category` with one row per domain. Use the literal
category `unlabelled` for domains the service returned nothing for -- coverage
is a result, not a preprocessing step, and dropping those rows silently is the
single most common way this measurement goes wrong.
"""
 
import argparse
import csv
import math
import random
import sys
from collections import Counter, defaultdict
 
UNLABELLED = "unlabelled"
 
 
# --------------------------------------------------------------------------
# statistics
# --------------------------------------------------------------------------
def wilson(k, n, z=1.959963984540054):
    """95% Wilson score interval for k successes in n trials.
 
    Wilson rather than the normal approximation because per-category samples
    are small (25-50) and precisions are often near 1.0, where the normal
    interval runs past 100% and stops being an interval.
    """
    if n == 0:
        return (0.0, 0.0, 1.0)
    p = k / n
    d = 1 + z * z / n
    centre = (p + z * z / (2 * n)) / d
    half = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
    return (p, max(0.0, centre - half), min(1.0, centre + half))
 
 
def cohen_kappa(pairs):
    """Cohen's kappa over (annotator_a, annotator_b) label pairs."""
    n = len(pairs)
    if n == 0:
        return float("nan")
    agree = sum(1 for a, b in pairs if a == b)
    po = agree / n
    ca = Counter(a for a, _ in pairs)
    cb = Counter(b for _, b in pairs)
    pe = sum(ca[k] * cb[k] for k in set(ca) | set(cb)) / (n * n)
    if pe == 1.0:
        return float("nan")
    return (po - pe) / (1 - pe)
 
 
def sample_size_for(halfwidth, p=0.9, z=1.959963984540054):
    """How many domains per category for a target CI half-width at precision p."""
    return math.ceil(z * z * p * (1 - p) / (halfwidth * halfwidth))
 
 
# --------------------------------------------------------------------------
# io
# --------------------------------------------------------------------------
def read_labels(path):
    with open(path, newline="", encoding="utf-8") as fh:
        rdr = csv.DictReader(fh)
        if rdr.fieldnames is None or "domain" not in rdr.fieldnames or "category" not in rdr.fieldnames:
            sys.exit(f"{path}: need a header row with columns `domain` and `category`")
        return [(r["domain"].strip(), r["category"].strip()) for r in rdr]
 
 
def read_sheet(path):
    with open(path, newline="", encoding="utf-8") as fh:
        rdr = csv.DictReader(fh)
        out = {}
        for r in rdr:
            verdict = r["verdict"].strip()
            if not verdict:
                sys.exit(f"{path}: domain {r['domain']} has an empty verdict; annotate every row")
            out[r["domain"].strip()] = verdict
        return out
 
 
# --------------------------------------------------------------------------
# commands
# --------------------------------------------------------------------------
def cmd_sample(args):
    rows = read_labels(args.labels)
    by_cat = defaultdict(list)
    n_unlabelled = 0
    for domain, cat in rows:
        # Unlabelled domains are a coverage result, measured over the whole
        # population in `score`. Asking an annotator to check the precision of
        # "no answer" is not a question, so they are held out of the sample.
        if cat.lower() == UNLABELLED:
            n_unlabelled += 1
            continue
        by_cat[cat].append(domain)
 
    rng = random.Random(args.seed)
    picked = []
    print(f"population: {len(rows)} domains, {n_unlabelled} unlabelled, "
          f"{len(rows) - n_unlabelled} in {len(by_cat)} categories")
    print(f"seed: {args.seed}   target per category: {args.per_category}\n")
    print(f"{'category':<32} {'size':>8} {'sampled':>8}")
    print("-" * 50)
    for cat in sorted(by_cat, key=lambda c: -len(by_cat[c])):
        pool = sorted(by_cat[cat])
        take = min(args.per_category, len(pool))
        chosen = rng.sample(pool, take)
        picked.extend((d, cat) for d in chosen)
        flag = "  <- undersized" if take < args.per_category else ""
        print(f"{cat:<32} {len(pool):>8} {take:>8}{flag}")
    print("-" * 50)
    print(f"{'total':<32} {len(rows):>8} {len(picked):>8}\n")
 
    # Blind: the sheet carries no category, and the order is shuffled so an
    # annotator cannot infer it from adjacency.
    order = list(picked)
    rng.shuffle(order)
    for sheet in ("a", "b"):
        path = f"sheet_{sheet}.csv"
        with open(path, "w", newline="", encoding="utf-8") as fh:
            w = csv.writer(fh)
            w.writerow(["domain", "verdict"])
            for domain, _ in order:
                w.writerow([domain, ""])
        print(f"wrote {path}  ({len(order)} rows, category hidden, order shuffled)")
 
    key = "sample_key.csv"
    with open(key, "w", newline="", encoding="utf-8") as fh:
        w = csv.writer(fh)
        w.writerow(["domain", "category"])
        w.writerows(picked)
    print(f"wrote {key}     (the labels under test -- do not show this to annotators)")
 
    # Planning sizes use the normal approximation; the reported intervals use
    # Wilson, which is wider near p = 1. Treat these as a floor, not a promise.
    for hw in (0.20, 0.10, 0.05):
        print(
            f"note: +/-{hw:.0%} half-width at 90% precision needs "
            f"{sample_size_for(hw)} domains per category (normal approximation; "
            f"the Wilson interval reported by `score` is wider near 100%)"
        )
 
 
def cmd_score(args):
    truth_labels = dict(read_labels(args.labels))
    a = read_sheet(args.sheet_a)
    b = read_sheet(args.sheet_b)
 
    common = sorted(set(a) & set(b))
    if not common:
        sys.exit("the two sheets share no domains")
    if set(a) != set(b):
        print(f"warning: sheets differ; scoring the {len(common)} shared domains\n")
 
    pairs = [(a[d], b[d]) for d in common]
    kappa = cohen_kappa(pairs)
    n_agree = sum(1 for x, y in pairs if x == y)
 
    print("=" * 66)
    print("INTER-ANNOTATOR AGREEMENT")
    print("=" * 66)
    print(f"domains double-annotated : {len(common)}")
    print(f"raw agreement            : {n_agree}/{len(common)} = {n_agree / len(common):.1%}")
    print(f"Cohen's kappa            : {kappa:.3f}")
    print(
        "Report both. Kappa alone hides how many categories there were, and raw\n"
        "agreement alone is inflated when one category dominates.\n"
    )
 
    # Precision is scored only where the two annotators agree. Where they do not,
    # the ground truth is genuinely unknown and forcing an adjudication would
    # bury the disagreement that is itself the result.
    scored = [d for d in common if a[d] == b[d]]
    disputed = len(common) - len(scored)
 
    per_cat = defaultdict(lambda: [0, 0])  # category -> [correct, n]
    for d in scored:
        service = truth_labels[d]
        per_cat[service][1] += 1
        if a[d] == service:
            per_cat[service][0] += 1
 
    print("=" * 66)
    print("PER-CATEGORY PRECISION")
    print("=" * 66)
    print(f"scored on the {len(scored)} domains the annotators agreed on; "
          f"{disputed} disputed and excluded\n")
    print(f"{'category':<32} {'ok':>4} {'n':>4} {'prec':>7}  {'95% CI':>16}  verdict")
    print("-" * 82)
    keep, drop = [], []
    for cat in sorted(per_cat, key=lambda c: (per_cat[c][0] / per_cat[c][1] if per_cat[c][1] else 0)):
        ok, n = per_cat[cat]
        p, lo, hi = wilson(ok, n)
        # The rule from Ruth et al., IMC 2022: keep a category only if it clears
        # the floor. Applied to the point estimate, with the CI shown so a reader
        # can see how much the sample size is actually supporting.
        verdict = "keep" if p >= args.floor else "DROP"
        (keep if p >= args.floor else drop).append(cat)
        print(f"{cat:<32} {ok:>4} {n:>4} {p:>6.1%}  [{lo:>5.1%}, {hi:>5.1%}]  {verdict}")
    print("-" * 82)
 
    tot_ok = sum(v[0] for v in per_cat.values())
    tot_n = sum(v[1] for v in per_cat.values())
    p, lo, hi = wilson(tot_ok, tot_n)
    print(f"{'ALL (micro-average)':<32} {tot_ok:>4} {tot_n:>4} {p:>6.1%}  [{lo:>5.1%}, {hi:>5.1%}]\n")
 
    # Scoring only where the annotators agreed biases precision UPWARD, because
    # a domain two people find easy to label is also a domain the service finds
    # easy. Report the pessimistic bound alongside it -- every disputed domain
    # counted as a service error -- and let the gap between the two speak.
    lo_ok = tot_ok
    lo_n = tot_n + disputed
    pp, plo, phi = wilson(lo_ok, lo_n)
    print(
        f"{'ALL (disputed counted wrong)':<32} {lo_ok:>4} {lo_n:>4} {pp:>6.1%}  "
        f"[{plo:>5.1%}, {phi:>5.1%}]"
    )
    print(
        "The true precision is between these two rows. If they are far apart your\n"
        "annotators disagreed too much for the number to mean anything -- fix the\n"
        "codebook and re-annotate rather than picking the flattering row.\n"
    )
 
    # Coverage over the whole population, not the sample.
    total = len(truth_labels)
    unlabelled = sum(1 for c in truth_labels.values() if c.lower() == UNLABELLED)
    cov, clo, chi = wilson(total - unlabelled, total)
    print("=" * 66)
    print("COVERAGE (whole population, not the sample)")
    print("=" * 66)
    print(f"{total - unlabelled}/{total} domains got a label = {cov:.1%} [{clo:.1%}, {chi:.1%}]\n")
 
    print("=" * 66)
    print("PASTE THIS INTO YOUR METHODOLOGY SECTION")
    print("=" * 66)
    print(
        f"We categorised {total} domains, of which {total - unlabelled} ({cov:.1%}) received a "
        f"label. To validate the labels we drew a stratified random sample of "
        f"{len(common)} domains and had two authors independently assign a category "
        f"while blind to the service's answer (Cohen's kappa = {kappa:.2f}, raw agreement "
        f"{n_agree / len(common):.0%}). On the {len(scored)} domains where the annotators "
        f"agreed, the service's micro-averaged precision was {p:.1%} "
        f"(95% Wilson CI [{lo:.1%}, {hi:.1%}]); counting every disputed domain as an error "
        f"gives a lower bound of {pp:.1%}. "
        + (
            f"We discarded the {len(drop)} categor{'y' if len(drop) == 1 else 'ies'} that fell "
            f"below {args.floor:.0%} precision ({', '.join(drop)}) and report results over the "
            f"remaining {len(keep)}."
            if drop
            else f"All {len(keep)} categories cleared our {args.floor:.0%} precision floor."
        )
    )
 
 
def main():
    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    sub = ap.add_subparsers(dest="cmd", required=True)
 
    s = sub.add_parser("sample", help="draw a stratified sample and write blind annotation sheets")
    s.add_argument("labels")
    s.add_argument("--per-category", type=int, default=25)
    s.add_argument("--seed", type=int, required=True, help="record this in the paper")
    s.set_defaults(func=cmd_sample)
 
    c = sub.add_parser("score", help="score two completed annotation sheets")
    c.add_argument("labels")
    c.add_argument("sheet_a")
    c.add_argument("sheet_b")
    c.add_argument("--floor", type=float, default=0.80, help="per-category precision floor")
    c.set_defaults(func=cmd_score)
 
    args = ap.parse_args()
    args.func(args)
 
 
if __name__ == "__main__":
    main()

A worked example on synthetic data, where one category (“Search Engines”) was deliberately built to be only 45% correct — the category [5Ruth, Kimberly; Fass, Aurore; Azose, Jonathan; Pearson, Mark; Thomas, Emma; Sadowski, Caitlin; Durumeric, Zakir (2022): "A World Wide View of Browsing the World Wide Web", in: Proceedings of the ACM Internet Measurement Conference, pp. 317-336. (DOI)] found unreliable in Cloudflare's taxonomy. Real output, reproducible with the demo script below:

population: 1100 domains, 220 unlabelled, 880 in 4 categories
seed: 20260807   target per category: 25

category                             size  sampled
--------------------------------------------------
News                                  400       25
Shopping                              300       25
Adult                                 120       25
Search Engines                         60       25
--------------------------------------------------
total                                1100      100

wrote sheet_a.csv  (100 rows, category hidden, order shuffled)
wrote sheet_b.csv  (100 rows, category hidden, order shuffled)
wrote sample_key.csv     (the labels under test -- do not show this to annotators)
note: +/-20% half-width at 90% precision needs 9 domains per category (normal approximation; the Wilson interval reported by `score` is wider near 100%)
note: +/-10% half-width at 90% precision needs 35 domains per category (normal approximation; the Wilson interval reported by `score` is wider near 100%)
note: +/-5% half-width at 90% precision needs 139 domains per category (normal approximation; the Wilson interval reported by `score` is wider near 100%)
==================================================================
INTER-ANNOTATOR AGREEMENT
==================================================================
domains double-annotated : 100
raw agreement            : 67/100 = 67.0%
Cohen's kappa            : 0.564
Report both. Kappa alone hides how many categories there were, and raw
agreement alone is inflated when one category dominates.

==================================================================
PER-CATEGORY PRECISION
==================================================================
scored on the 67 domains the annotators agreed on; 33 disputed and excluded

category                           ok    n    prec            95% CI  verdict
----------------------------------------------------------------------------------
Search Engines                      5    9  55.6%  [26.7%, 81.1%]  DROP
News                               19   19 100.0%  [83.2%, 100.0%]  keep
Shopping                           16   16 100.0%  [80.6%, 100.0%]  keep
Adult                              23   23 100.0%  [85.7%, 100.0%]  keep
----------------------------------------------------------------------------------
ALL (micro-average)                63   67  94.0%  [85.6%, 97.7%]

ALL (disputed counted wrong)       63  100  63.0%  [53.2%, 71.8%]
The true precision is between these two rows. If they are far apart your
annotators disagreed too much for the number to mean anything -- fix the
codebook and re-annotate rather than picking the flattering row.

==================================================================
COVERAGE (whole population, not the sample)
==================================================================
880/1100 domains got a label = 80.0% [77.5%, 82.3%]

==================================================================
PASTE THIS INTO YOUR METHODOLOGY SECTION
==================================================================
We categorised 1100 domains, of which 880 (80.0%) received a label. To validate the labels we drew a stratified random sample of 100 domains and had two authors independently assign a category while blind to the service's answer (Cohen's kappa = 0.56, raw agreement 67%). On the 67 domains where the annotators agreed, the service's micro-averaged precision was 94.0% (95% Wilson CI [85.6%, 97.7%]); counting every disputed domain as an error gives a lower bound of 63.0%. We discarded the 1 category that fell below 80% precision (Search Engines) and report results over the remaining 3.

Four things to read out of that:

1. It found the bad category. Search Engines came back at 55.6% and got dropped, exactly as intended — and it did so from 9 usable annotations, which is why per-category stratification matters more than total sample size.
2. Kappa 0.56 on a 5-way label set is realistic, not a failure. Vallina et al.'s own labellers disagreed on 35.5% of domains. If yours agree on 95% you have probably let them see each other's sheets.
3. The two “ALL” rows disagree by 31 points, and that gap is the honest measure of how much the annotation is carrying. The synthetic pipeline's true prevalence-weighted precision is about 88%: reporting only the agreed-subset row publishes it as 94%, reporting only the pessimistic row buries it at 63%, and reporting both brackets it. Report both.
4. The micro-average is a diagnostic, not an estimate of your population's precision. The sample is stratified with equal n per category, so a small bad category is overweighted — Search Engines is 6.8% of the population here but 13% of the scored sample. Prevalence-weight before quoting a single number for the whole corpus, or say plainly that the row is per-category diagnostics rolled up.

category_validation_demo.py

#!/usr/bin/env python3
"""Reproduce the worked example shown on design:website_classification.
 
Builds a synthetic 1,100-domain labelling with one deliberately bad category
("Search Engines", 45% correct -- the category Ruth et al. found unreliable in
Cloudflare's taxonomy), runs `category_validation.py sample`, simulates two
annotators, and runs `score`. Stdlib only.
 
    python3 category_validation_demo.py
"""
 
import csv
import pathlib
import random
import subprocess
import sys
import tempfile
 
HERE = pathlib.Path(__file__).resolve().parent
TOOL = HERE / "category_validation.py"
SEED = 20260807
 
# category -> (population size, share of labels that are actually correct)
TRUTH = {
    "News": (400, 0.92),
    "Shopping": (300, 0.88),
    "Adult": (120, 0.96),
    "Search Engines": (60, 0.45),
    "unlabelled": (220, 1.0),
}
VERDICTS = ["News", "Shopping", "Adult", "Search Engines", "none"]
# per-annotator probability of slipping to a different verdict
ANNOTATOR_NOISE = {"sheet_a.csv": 0.02, "sheet_b.csv": 0.05}
 
 
def run(work):
    rng = random.Random(SEED)
 
    rows, i = [], 0
    for cat, (n, _) in TRUTH.items():
        for _ in range(n):
            i += 1
            rows.append((f"site{i:04d}.example", cat))
    with open(work / "labels.csv", "w", newline="", encoding="utf-8") as fh:
        w = csv.writer(fh)
        w.writerow(["domain", "category"])
        w.writerows(rows)
 
    subprocess.run(
        [sys.executable, str(TOOL), "sample", "labels.csv",
         "--per-category", "25", "--seed", str(SEED)],
        cwd=work, check=True,
    )
 
    key = {r["domain"]: r["category"]
           for r in csv.DictReader(open(work / "sample_key.csv", encoding="utf-8"))}
    rng = random.Random(11)
    for sheet, noise in ANNOTATOR_NOISE.items():
        annotated = list(csv.DictReader(open(work / sheet, encoding="utf-8")))
        for r in annotated:
            true = key[r["domain"]]
            v = true if rng.random() < TRUTH[true][1] else rng.choice(
                [c for c in VERDICTS if c != true])
            if rng.random() < noise:
                v = rng.choice([c for c in VERDICTS if c != v])
            r["verdict"] = v
        with open(work / sheet, "w", newline="", encoding="utf-8") as fh:
            w = csv.DictWriter(fh, ["domain", "verdict"])
            w.writeheader()
            w.writerows(annotated)
 
    print()
    subprocess.run(
        [sys.executable, str(TOOL), "score", "labels.csv",
         "sheet_a.csv", "sheet_b.csv", "--floor", "0.80"],
        cwd=work, check=True,
    )
 
 
if __name__ == "__main__":
    with tempfile.TemporaryDirectory() as tmp:
        run(pathlib.Path(tmp))

A checklist, derived from what the 330 corpus papers below leave out. Every line is something a reviewer can reasonably ask for and most papers do not supply.

1. Which service or model, and which version. “McAfee” identified a different company before and after February 2024. “Homepage2Vec” without a commit hash is not reproducible; “GPT-4” without a dated snapshot is not either.
2. The date you queried. These are live services whose labels change without notice.
3. Coverage on your population, as a fraction, before any filtering. How many of your domains got no label — and what you did with them. Dropping them silently changes the denominator of every downstream percentage.
4. The taxonomy and its size. Name it, give the number of classes, and say at which tier you pruned. If it is bespoke, say so and publish the codebook.
5. Whether labels are multi-valued, and how you collapsed them if they are.
6. Validation: sample size, how it was stratified, how many annotators, whether they were blind to the service's label, the agreement statistic, and per-category precision — not one global accuracy number. Publish the annotated sample as an artefact (page not yet written); it is small, and it is the only part of this a reader can check.
7. The categories you dropped, and the rule you dropped them by.
8. If you used more than one service, the disagreement between them. Only 9 of 155 corpus papers that name a service name two, so this is nearly free novelty.

Two sources here. The historical survey is Vallina et al. [1Vallina, Pelayo; Le Pochat, Victor; Feal, ´Alvaro; Paraschiv, Marius; Gamba, Julien; Burke, Tim; Hohlfeld, Oliver; Tapiador, Juan; Vallina-Rodriguez, Narseo (2020): "Mis-shapes, Mistakes, Misfits: An Analysis of Domain Classification Services", in: Proceedings of the ACM Internet Measurement Conference, pp. 598–618. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)], whose figures on service popularity are reproduced below. The rest is a fresh query over a structured extraction of 5,859 papers from CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026 — so every figure is a claim about those seven venues over that window, not about the field. The 2025 and 2026 venue-years are provisional: CCS and IMC 2026 have not been held, and IEEE S&P and WWW 2026 abstracts are not yet in the selection source.

The extraction marks 424 papers as classifying something with the target website-category. That number is not the population this page is about, and the difference is instructive: the same label covers several unrelated tasks.

Website fingerprinting is a traffic-analysis attack that shares no method, taxonomy or ground truth with topic categorisation; it is here because “classify a website” is the same English phrase. Everything below uses the 330, which is 5.6% of the corpus. The other 94 papers are counted, named, and excluded — see Methodology and limitations of these figures.

TheWebConf and PETS each devote about 8% of their programme to it, IMC nearly as much. The overall rate has been flat-to-falling: 5.5% of papers in 2010–2013, peaking at 7.3% in 2014–2017, 4.6% in 2022–2024 and 5.5% in the provisional 2025–2026 window.

Multi-valued, so shares exceed 100%. Reliance on third-party services grew steadily for four windows — 25.0% of categorising papers in 2010–2013, 33.9%, 43.5%, 49.4% in 2022–2024 — while use of curated databases (DMOZ and friends) fell from 21.4% to 9.0% across the middle of that run. The field moved from open, inspectable label sources toward closed, unauditable ones, over exactly the period in which the only audit of those sources found them wanting.

In the provisional 2025–2026 window both of those trends reverse: third-party services drop to 32.3% and curated databases return to 18.5%, with hand-built heuristics up to 23.1% and the first two LLM papers appearing at 3.1%. Two incomplete venue-years cannot establish a turn, and this is exactly the kind of row not to quote as a trend — but it is the first movement against the closed-source drift in fifteen years, and it is worth re-checking when 2026 is complete.

Free-text service names agree run-to-run on only about 20% of exact strings, so these are folded into canonical families and counted by paper, never by mention. The “spellings” column is why this matters.

155 of the 330 papers (47.0%) name at least one third-party service or dataset. McAfee alone appears under eighteen different names — from bare “McAfee” to “McAfee's URL Ticketing System” to “McAfee SmartFilter Internet Database” — and Symantec under ten. Any count of these services built on exact string matching will be wrong by a factor of several, which is the same failure that has bitten geolocation and crawler-tool tallies on other pages of this site.

Everything that is not a named service, ranked: unnamed custom methods (58 papers), manual labelling by the authors or hired coders (41), custom keyword and regex rules (32), random forests (13), clustering (10), SVMs (9), then a long tail. Those first and third rows are one family: 87 papers (26.4%) classify websites with rules they wrote themselves, and 58 of them do not name the rules at all.

The last row is a sentinel and is shown because the page's own rule says sentinels are never folded into an answer. It means the extraction could not find a validation step and did not think one was called for — typically because the paper is reporting a distribution of vendor labels with no claim that depends on them being right. Read it as “no validation reported”, not as “validation was unnecessary”, unless you check the paper.

Of those that do validate: 92 papers by manual validation, 21 by cross-validation, 13 by a held-out test set, 10 by comparison to another method. Note the shape — manual spot-checking is the dominant and correct instrument here, because a held-out test set only tells you about the labels you already had.

The 51.2% figure is worth dwelling on: for website classification, the share naming a ground-truth source is well below the 72.9% the same extraction measures across all 4,439 papers that classify anything. Website categorisation is reported worse than classification in general, not better.

Of the 248 papers that name a taxonomy at all:

Three quarters invent a scheme and do not say how many classes it has. That single omission makes a result impossible to compare against any other paper, and it is free to fix.

177 papers, and the curve is no longer starting — it is the fastest-moving thing in this corpus. From 2 papers in 2023 to one in six in 2026, and both of the last two venue-years are incomplete, so those shares are floors. They point the model at “other” targets (116 papers), user-generated text (32), vulnerabilities (14), privacy policies (12), mobile apps (6), website categories (5), and single-digit counts of domains, network traffic, consent notices, requests, IP addresses, cookies, email and dark patterns.

They establish a norm worth copying: 148 of 177 (83.6%) validate the model's labels against something, usually manual annotation of a sample — the same share as when this was 29 papers, which is the encouraging part.

Five papers now point an LLM at a website category, and two of those are genuine topic classification rather than an abuse verdict:

• [8Tsai, Elisa; Mangaokar, Neal; Zheng, Boyuan; Zheng, Haizhong; Prakash, Atul (2025): "Harmful Terms and Where to Find Them: Measuring and Modeling Unfavorable Financial Terms and Conditions in Shopping Websites at Scale", in: Proceedings of the ACM Web Conference. (DOI)] (TheWebConf 2025) classifies Tranco sites as shopping or non-shopping with GPT-4o prompted with the URL and a screenshot, validates against 500 manually annotated sites, and reports 92% accuracy, “comparable to commercial website classification services”. That sentence is the first thing in this corpus that makes an LLM a real alternative to the vendors catalogued above, and it is worth reading before you buy an API key.
• DarkGram (USENIX Security 2025) has GPT-4 label cybercriminal Telegram channels into 15 predefined categories, which the model itself extended to 19, validated by coders on 1,000 sampled posts. Categories of a channel, not of a website — count it as adjacent.

The other three are abuse verdicts: the USENIX Security 2024 phishing detector using GPT-3.5-turbo-16k for a binary credential-page question, a TheWebConf 2025 crypto-scam study using a Llama3:70b + GPT-4 hybrid, and an NDSS 2026 credential-theft phishing detector using GPT-4o with retrieval-augmented prompts. All three validate against manual annotation.

See LLM Classification for what has happened outside the corpus.

• [5Ruth, Kimberly; Fass, Aurore; Azose, Jonathan; Pearson, Mark; Thomas, Emma; Sadowski, Caitlin; Durumeric, Zakir (2022): "A World Wide View of Browsing the World Wide Web", in: Proceedings of the ACM Internet Measurement Conference, pp. 317-336. (DOI)] (IMC 2022) is the best worked example of using a categorisation API responsibly that the corpus contains: state the taxonomy sizes, validate ten random sites per category, keep only categories above 80% accuracy, report what you dropped and why. It is one paragraph of a large paper and it is the paragraph to imitate.
• [1Vallina, Pelayo; Le Pochat, Victor; Feal, ´Alvaro; Paraschiv, Marius; Gamba, Julien; Burke, Tim; Hohlfeld, Oliver; Tapiador, Juan; Vallina-Rodriguez, Narseo (2020): "Mis-shapes, Mistakes, Misfits: An Analysis of Domain Classification Services", in: Proceedings of the ACM Internet Measurement Conference, pp. 598–618. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] (IMC 2020) is the audit itself. Read §4.2 (coverage), §4.3 (documented versus observed taxonomies) and §6 (the case studies) before you commit to a service.

• Denominators. Every figure names its own. The corpus is 5,859 papers; 4,439 classify something; 424 carry a website-category target; 330 assign a topic, industry or sector label and are this section's population. Never “of 5,859 papers”.
• Papers, not mentions. A paper naming McAfee three times counts once.
• Sentinels are not answers. not-stated and none-mentioned are excluded from every numerator and reported as their own row where they matter — which they do, since “does not validate” is the headline finding.
• Free-text names are folded into canonical families by an ordered regex list before counting, because exact strings agree run-to-run only about 20% of the time. The fold left 33 distinct strings unclassified, each appearing once: among them Taobao main-business categories, SNET pillar portals, Rakuten Popshops, NAICSlite, ORES topic routing, multiple domain classification services and public classification service from the security vendor. The last two are papers that used a service and declined to say which — a small, exact instance of the reporting gap this section measures. The residue was 40 strings before the fold was extended for this corpus; website_classification lists what was added and what was deliberately left out.
• The task fold is imperfect and its exclusions are visible. Eight side-channel and website-fingerprinting papers named their classifier only as “a 1D CNN” or “k-NN”, leaving no signal in any extracted field; they were caught by matching the paper's own slug and are excluded by an explicit named list rather than by a general rule. The 94 excluded papers are counted in the table above rather than dropped.
• Enum fields versus free text. Method and validation are enums, stable enough to publish as rough shares (classification.method agrees 58% run-to-run, so read those as a ranking). Service names and taxonomies are free text and are reported as rankings and folded families only.
• Quotes were checked. Every figure above traces to tuples carrying a verbatim evidence quote; a sample of these was re-located in the source PDFs. Of six quotes checked by hand, two initially “failed” a literal grep and turned out to be intact but split across a two-column break — normalise whitespace before concluding that a quote is not in the paper. The five new LLM website-category tuples were re-checked individually on 2026-08-12; see website_classification.
• Coverage. Seven venues only, with 2025 and 2026 incomplete by construction rather than by relevance, so any row that reaches them is a floor; the scope and the selection funnel are on corpus. Notably, [1Vallina, Pelayo; Le Pochat, Victor; Feal, ´Alvaro; Paraschiv, Marius; Gamba, Julien; Burke, Tim; Hohlfeld, Oliver; Tapiador, Juan; Vallina-Rodriguez, Narseo (2020): "Mis-shapes, Mistakes, Misfits: An Analysis of Domain Classification Services", in: Proceedings of the ACM Internet Measurement Conference, pp. 598–618. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] itself is in the venue index but has no extracted full text — the reference work for this page is not in the population the page measures.
• Every query behind this section, the report script and its unedited output are on website_classification; corpus-level caveats are on corpus.

• Website Selection — which domains to study, and popularity rankings.
• IP Classification — the same problem for addresses instead of names, with the same “which question are you actually asking” structure.
• Crawling Location — where you crawl from, which is a different question from which country a site is aimed at (Target country is not topic).
• Requests — tracker and advertiser classification, where filter lists beat categorisers.
• Cookies, Fingerprinting, JavaScript — classification of the other things a crawl collects.
• SimilarWeb — API documentation, code and output example for the one service on this page that has a page of its own.
• Sampling, Biases (both not yet written) — drawing a sample you can generalise from, and what a categoriser's uneven coverage does to a downstream estimate.
• Tranco, Cloudflare Radar (both not yet written) — API documentation for two services this page references.

[1]

Vallina, Pelayo; Le Pochat, Victor; Feal, ´Alvaro; Paraschiv, Marius; Gamba, Julien; Burke, Tim; Hohlfeld, Oliver; Tapiador, Juan; Vallina-Rodriguez, Narseo (2020): "Mis-shapes, Mistakes, Misfits: An Analysis of Domain Classification Services", in: Proceedings of the ACM Internet Measurement Conference, pp. 598–618. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)

[2]

Bozzolan, Simone; Calzavara, Stefano; Cazzaro, Lorenzo (2026): "LLM-Assisted Web Measurements". arXiv:2510.08101, v3, 30 April 2026 (Link)

[3]

Lugeon, Sylvain; Piccardi, Tiziano; West, Robert (2022): "Homepage2Vec: Language-Agnostic Website Embedding and Classification", in: Proceedings of the International AAAI Conference on Web and Social Media, pp. 1285-1291. (DOI)

[4]

Ruth, Kimberly; Kumar, Deepak; Wang, Brandon; Valenta, Luke; Durumeric, Zakir (2022): "Toppling Top Lists: Evaluating the Accuracy of Popular Website Lists", in: Proceedings of the 22nd ACM Internet Measurement Conference, pp. 374–387. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)

[5]

Ruth, Kimberly; Fass, Aurore; Azose, Jonathan; Pearson, Mark; Thomas, Emma; Sadowski, Caitlin; Durumeric, Zakir (2022): "A World Wide View of Browsing the World Wide Web", in: Proceedings of the ACM Internet Measurement Conference, pp. 317-336. (DOI)

[6]

Jha, Nikhil; Trevisan, Martino; Leonardi, Emilio; Mellia, Marco (2023): "On the Robustness of Topics API to a Re-Identification Attack", Proceedings on Privacy Enhancing Technologies 2023(4):66-78. (DOI)

[7]

Ye, Eric; Bai, Xiao; O'Hare, Neil; Asgarieh, Eliyar; Thadani, Kapil; Perez-Sorrosal, Francisco; Adiga, Sujyothi (2024): "Multilingual Taxonomic Web Page Categorization Through Ensemble Knowledge Distillation", IEEE Transactions on Knowledge and Data Engineering 36(11):6614-6627. (DOI)

[8]

Tsai, Elisa; Mangaokar, Neal; Zheng, Boyuan; Zheng, Haizhong; Prakash, Atul (2025): "Harmful Terms and Where to Find Them: Measuring and Modeling Unfavorable Financial Terms and Conditions in Shopping Websites at Scale", in: Proceedings of the ACM Web Conference. (DOI)