Differences

This shows you the differences between two versions of the page.

--- design:website_selection [2026/08/27 14:14] – Closed remaining TODOs; added corpus Use in Publications from 5859-paper vendor fold; fixed Alexa 2022 vs Tranco 2023 dates. Authored by Claude karel.kubicek.claude
+++ design:website_selection [2026/08/27 14:39] (current) – Rewrite toward IP-classification shape: decision table, live rank comparison, 2026 status table, quotes, open questions. Kept human catalogue. Authored by Claude karel.kubicek.claude
@@ Line 4: / Line 4: @@
 This page overlaps with [[Design:Website classification]] only in that popularity is one label people attach to sites. Classification is about topic, industry and company data. Rankings are the sampling frame.
+The single most useful thing to internalise is that **"a top-sites list" is not one instrument.** The vendors measure different things, they disagree on the head, and a Tranco row is a mixture of those instruments rather than a ground truth:
+^ If your claim is about ^ Use ^ Do not use as if it measured that ^
+| Pages people actually load | **CrUX** (origins, monthly buckets) | Umbrella, Radar, Farsight, Majestic. Tranco is not a substitute: CrUX is one of five inputs |
+| DNS / infrastructure / non-browser clients | Umbrella, Radar, SecRank, Farsight-in-Tranco | CrUX |
+| Link structure | Majestic | any traffic or DNS list |
+| "The web" | None of these. Say which frame you had. | A top-//n// described as a sample of the web |
 The short version, from 1,153 papers in our corpus of seven security and privacy venues that drew a population of websites, domains or web pages (see [[#Use in Publications]]):
 <WRAP important>
-**Tranco is what 2025 papers use; CrUX is what Ruth et al. found most accurate in 2022.** Of 133 papers published in 2025 that sampled the web, **75 (56.4%) named Tranco** and **12 (9.0%) named CrUX**. Ruth et al. measured Alexa, Majestic, Umbrella, Tranco and CrUX against Cloudflare's server-side HTTP request logs in February 2022 and found **CrUX the most accurate across all metrics** {[ruth2022_toppling]}. That comparison has **not** been re-run on the five-provider Tranco list you download today. Alexa.com was retired on **1 May 2022** (APIs 15 December 2022); **66 papers from 2023–2026 still name it**, 34 of them with no version at all. Do not write "the Tranco top 1M" without a list id: the default provider set changed on 1 August 2023. Pin the id with [[Programming:Tranco]]'s ''pin_tranco.py''.
+**Pick the list that matches the claim; then pin it.** Of 133 papers published in 2025 that sampled the web, **75 (56.4%) named Tranco** and **12 (9.0%) named CrUX** — Tranco is current practice, not a page-load ranking. Ruth et al. measured Alexa, Majestic, Umbrella, Tranco and CrUX against Cloudflare HTTP logs in **February 2022** and found **CrUX the most accurate across all metrics** {[ruth2022_toppling]}. That is 2022 evidence, not a re-run on the five-provider Tranco list you download now. Alexa.com was retired on **1 May 2022** (APIs 15 December 2022); **66 papers from 2023–2026 still name it**, 34 of them with no version at all. Do not write "the Tranco top 1M" without a list id: the default provider set changed on 1 August 2023. Pin the id with [[Programming:Tranco]]'s ''pin_tranco.py''.
 </WRAP>
-===== Popular Website Lists =====
+If you read three papers before starting: {[scheitle2018_long]} (IMC 2018) on overlap, churn and head-vs-zone bias; {[LePochat2019_tranco]} (NDSS 2019) on manipulation and why Tranco exists; {[ruth2022_toppling]} (IMC 2022) on which list matched HTTP logs. {[galloway2024_practical]} (IEEE S&P 2024) is the current attack paper, and it is about Radar and Tranco-via-Radar, not about CrUX.
-==== 1. CrUX (Chrome User Experience Report) ====
+===== The lists are different instruments =====
-[[https://developer.chrome.com/docs/crux|CrUX]] provides rank-magnitude buckets (1k, 5k, 10k, …, 5M) of popular **origins** based on user-initiated page loads from eligible Chrome installs — usage-statistic reporting, history sync without a passphrase, and a supported platform (not Chrome on iOS).(([[https://developer.chrome.com/docs/crux/methodology|CrUX methodology]], fetched 2026-08-27: user eligibility; datasets licensed **CC BY 4.0**. Tranco's methodology page still says CC BY-SA 4.0; Google's own page is the primary source.)) Docs live as of 2026-08-27.
-  * **Advantages**: Closest to Cloudflare's HTTP request logs in February 2022, according to {[ruth2022_toppling]}; derived from eligible Chrome page loads; includes country- and device-specific slices; CC BY 4.0; a default Tranco input since 1 August 2023.
+Each list ranks an **observable** that people then treat as popularity. The observable is the whole design:
-  * **Limitations**: Rankings are aggregated into buckets — there is no rank 5,000; opt-in Chrome, not Chrome's full install base; monthly, not daily. Galloway et al. did not evaluate it {[galloway2024_practical]}.
-  * **More details, API**: [[Programming:CrUX]]
-==== 2. Tranco ====
+  * **Page loads** (CrUX): user-initiated navigations from eligible Chrome installs. Closest to "sites people visit".
-[[https://tranco-list.eu/|Tranco]] is a research-oriented, hardened top-sites ranking designed to reduce churn and manipulation. It was introduced in NDSS 2019 {[LePochat2019_tranco]}, when it combined Alexa, Cisco Umbrella, Majestic and Quantcast; **its inputs have changed since**. The daily list of 2026-08-26 (id ''46W9X'') is built from CrUX, Farsight, Majestic, Cloudflare Radar and Cisco Umbrella, aggregated over 30 days with the Dowdall rule — Alexa is no longer among them.((Read from Tranco's own API on 2026-08-27: ''https://tranco-list.eu/api/lists/date/latest'' returns ''list_id: 46W9X'', ''providers: [crux, farsight, majestic, radar, umbrella]'', ''combinationMethod: dowdall'', ''filterPLD: on'', ''listPrefix: 1000000''. Describe Tranco's composition from the live API rather than from the 2019 paper. See [[Design:Sampling]] and [[Programming:Tranco]].))
+  * **DNS queries** (Umbrella, Radar, Farsight, SecRank): names resolvers looked up. Includes software updates, telemetry, CDNs and names no human opened.
+  * **Backlinks** (Majestic): the link graph. Independent of visits; cheap to manipulate relative to a panel {[LePochat2019_tranco]}.
+  * **An aggregate** (Tranco): Dowdall over a month of several of the above. More stable than a single daily source; not a fourth kind of popularity, and not CrUX.
-  * **Advantages**: Permanent list ids; research-oriented generator; Dowdall over a month is more stable than a single daily source; configurable (with a Farsight caveat, below).
+Tranco's daily list of 2026-08-26 (id ''46W9X'') mixes CrUX with three DNS rankings and Majestic. Treating that id as "the CrUX top 1M" is a methods error; treating it as "websites" is the same error one layer down. The script below looks the same names up on that Tranco list, Umbrella and Majestic (fetched 2026-08-27T14:26:07Z):
-  * **Limitations**: An aggregate, not a ground truth — Ruth et al. found it less accurate than CrUX against HTTP logs {[ruth2022_toppling]}; dependent on its inputs, which have changed three times since the 2019 configuration (Quantcast dropped 2020-04, Farsight joined 2022-05-01, Alexa replaced by CrUX and Radar on 2023-08-01); Radar's CC BY-NC 4.0 rides along on the default list; a 2019 id and a 2026 id are different instruments.
-  * **More details, API**: [[Programming:Tranco]]
-==== 3. Cloudflare Radar ====
+^ Name ^ Tranco (PLD, 46W9X) ^ Cisco Umbrella (hostname) ^ Majestic Million (backlinks) ^
-[[https://radar.cloudflare.com/domains|Cloudflare Radar Domain Rankings]] ranks **pay-level domains** from DNS queries to 1.1.1.1, not from HTTP hits on Cloudflare-operated websites. An ordered top 100 (daily, global and per country) plus unordered buckets up to 1M (weekly). Details on [[Programming:Cloudflare Radar]].
+| ''google.com'' | 1 | 1 | 1 |
+| ''facebook.com'' | 4 | 44 | 2 |
+| ''wikipedia.org'' | 31 | 2,219 | 11 |
+| ''baidu.com'' | 81 | 2,115 | 68 |
+| ''nytimes.com'' | 158 | 4,093 | 67 |
+| ''microsoft.com'' | 5 | 4 | 9 |
+| ''gstatic.com'' | 3 | 2 | 1,232 |
+| ''windowsupdate.com'' | 60 | 30 | 23,840 |
+| ''windowsupdate.microsoft.com'' | absent | 79,026 | absent |
+| ''googleusercontent.com'' | 35 | 26 | 178 |
-  * **Advantages**: Free API (token with Radar Read); per-country top 100; includes infrastructure names a page-load ranking drops; a default Tranco input since 1 August 2023.
+<WRAP important>
-  * **Limitations**: Unordered below rank 100 — there is no rank 5,000; DNS-based (see [[#Limitations of DNS-Based Lists]]); CC BY-NC 4.0; no permanent id; manipulable with a cheap VPN {[galloway2024_practical]}.
+**''gstatic.com'' is rank 3 on Tranco list ''46W9X'' (2026-08-26).** It is rank 2 on Umbrella and rank 1,232 on Majestic. A static-content hostname sits in the head of the list the field treats as "top websites" because Tranco's default inputs include DNS rankings. ''windowsupdate.com'' is rank 60 on Tranco and rank 23,840 on Majestic. ''windowsupdate.microsoft.com'' exists only on Umbrella — Tranco and Majestic are pay-level domains. Wikipedia is rank 31 on Tranco and rank 2,219 on Umbrella. These are not permutations of one ordering.
-  * **More details, API**: [[Programming:Cloudflare Radar]]
+</WRAP>
-==== 4. Cisco Umbrella ====
+Umbrella still publishes a **separate** TLD file (''top-1m-TLD.csv.zip'', 10,905 rows on 2026-08-27: ''com'' = 1, ''net'' = 2, ''org'' = 6). Bare TLDs are **not** in the 2026-08-27 top-1m.csv — an earlier version of this page described the popularity list as beginning ''1,com'' / ''2,net''. That is the TLD file, not the list you crawl.
-[[https://umbrella-static.s3-us-west-1.amazonaws.com/index.html|Cisco Umbrella]] ranks names by DNS query traffic to OpenDNS resolvers. The daily top-1M zip was still published at the S3 URL above on 2026-08-27. The provider's own example file begins ''1,com'' / ''2,net'' / ''3,google.com'' — TLDs and infrastructure names sit in the head.
-  * **Advantages**: Captures non-browser traffic; daily snapshot with dated historical URLs; still a Tranco default input.
+==== A script ====
-  * **Limitations**: DNS-based (see [[#Limitations of DNS-Based Lists]]); biased to organisations that point resolvers at OpenDNS; typos and ''%%*.ec2.internal%%'' survive; the published example is not a list of websites.
-==== 5. Majestic Million ====
+Looks the same names up on the three lists you can fetch without an account. Stdlib only. Rerun it on the day you freeze a sample; the table above is the latest Tranco daily list as of the fetch (**dated 2026-08-26, fetched 2026-08-27T14:26:07Z**), plus that day's Umbrella zip and Majestic CSV.
-[[https://majestic.com/reports/majestic-million|Majestic]] ranks domains by backlinks. The Million page still advertised free search and download on 2026-08-27 (''downloads.majestic.com/majestic_million.csv''). CC BY 3.0, per Tranco.
-  * **Advantages**: Independent of user-traffic panels; useful when the question is about link structure; still a Tranco default input.
+<file python compare_ranks.py>
-  * **Limitations**: Does not measure visits. Link-graph rankings are cheap to manipulate relative to a toolbar or panel list {[LePochat2019_tranco]}. That paper did not evaluate CrUX.
+#!/usr/bin/env python3
+"""Look the same names up on today's Tranco daily list, Cisco Umbrella, and
+Majestic Million, and print the ranks side by side.
-==== 6. Other ====
+The point is not the ranks. It is that they disagree, and that Umbrella ranks
-  * ''SimilarWeb'': Commercial panel and crawl estimates; paywalled (some research exceptions exist). See [[Programming:SimilarWeb]]. Named by 12 of 1,153 web-sampling papers.
+TLDs and infrastructure hostnames a page-load list does not contain.
+    python3 compare_ranks.py
+    python3 compare_ranks.py --wiki
+Stdlib only. Caches the bulk CSVs under /tmp/measuretheweb-compare-ranks.
+Tranco is a ranks API (1 query/second). Dated in the output; rerun before you
+cite the table.
+"""
+from __future__ import annotations
+import argparse
+import csv
+import hashlib
+import io
+import json
+import sys
+import time
+import urllib.request
+import zipfile
+from datetime import datetime, timezone
+from pathlib import Path
+UA = "measuretheweb-compare-ranks/1.0"
+CACHE = Path("/tmp/measuretheweb-compare-ranks")
+CACHE.mkdir(parents=True, exist_ok=True)
+TRANCO_RANKS = "https://tranco-list.eu/api/ranks/domain/{domain}"
+TRANCO_LATEST = "https://tranco-list.eu/api/lists/date/latest"
+UMBRELLA_ZIP = "https://s3-us-west-1.amazonaws.com/umbrella-static/top-1m.csv.zip"
+UMBRELLA_TLD_ZIP = "https://s3-us-west-1.amazonaws.com/umbrella-static/top-1m-TLD.csv.zip"
+MAJESTIC_CSV = "https://downloads.majestic.com/majestic_million.csv"
+# Names chosen to hit the disagreements the rest of the page argues: a page
+# people visit, a news site, a Chinese head, DNS/update infrastructure, and a
+# hostname vs its registrable domain. Bare TLDs are a separate Umbrella file
+# (top-1m-TLD.csv.zip); they are not in the top-1m this script loads.
+NAMES = [
+    "google.com",
+    "facebook.com",
+    "wikipedia.org",
+    "baidu.com",
+    "nytimes.com",
+    "microsoft.com",
+    "gstatic.com",
+    "windowsupdate.com",
+    "windowsupdate.microsoft.com",
+    "googleusercontent.com",
+]
+def fetch(url: str) -> bytes:
+    req = urllib.request.Request(url, headers={"User-Agent": UA})
+    with urllib.request.urlopen(req, timeout=180) as resp:
+        return resp.read()
+def cached(url: str, name: str) -> Path:
+    path = CACHE / name
+    if path.exists() and path.stat().st_size > 0:
+        return path
+    path.write_bytes(fetch(url))
+    return path
+def load_rank_csv(path: Path, domain_col: int, rank_col: int, skip_header: bool) -> dict[str, int]:
+    raw = path.read_bytes()
+    if raw[:2] == b"PK":
+        with zipfile.ZipFile(io.BytesIO(raw)) as zf:
+            text = zf.read(zf.namelist()[0]).decode()
+    else:
+        text = raw.decode(errors="replace")
+    out: dict[str, int] = {}
+    reader = csv.reader(io.StringIO(text))
+    if skip_header:
+        next(reader)
+    for row in reader:
+        if len(row) <= max(domain_col, rank_col):
+            raise RuntimeError(f"{path}: short row {row!r}")
+        domain = row[domain_col].strip().lower()
+        rank = int(row[rank_col])
+        if domain in out:
+            continue
+        out[domain] = rank
+    if len(out) < 1000:
+        raise RuntimeError(f"{path}: only {len(out)} rows — file is not a top list")
+    return out
+def tranco_today(domain: str) -> int | None:
+    raw = fetch(TRANCO_RANKS.format(domain=domain))
+    body = json.loads(raw.decode())
+    ranks = body["ranks"]
+    if len(ranks) == 0:
+        return None
+    rank = ranks[0]["rank"]
+    if rank is None:
+        return None
+    return int(rank)
+def fmt(rank: int | None) -> str:
+    if rank is None:
+        return "absent"
+    return f"{rank:,}"
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--wiki", action="store_true", help="DokuWiki table instead of TSV")
+    args = ap.parse_args()
+    meta = json.loads(fetch(TRANCO_LATEST).decode())
+    list_id = meta["list_id"]
+    cfg = meta["configuration"]
+    fetched_at = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
+    umbrella_path = cached(UMBRELLA_ZIP, "umbrella_top1m.csv.zip")
+    majestic_path = cached(MAJESTIC_CSV, "majestic_million.csv")
+    umbrella = load_rank_csv(umbrella_path, domain_col=1, rank_col=0, skip_header=False)
+    majestic = load_rank_csv(majestic_path, domain_col=2, rank_col=0, skip_header=True)
+    rows: list[tuple[str, int | None, int | None, int | None]] = []
+    for i, name in enumerate(NAMES):
+        if i:
+            time.sleep(1.05)
+        t = tranco_today(name)
+        u = umbrella[name] if name in umbrella else None
+        m = majestic[name] if name in majestic else None
+        rows.append((name, t, u, m))
+    if rows[0][0] != "google.com" or rows[0][1] is None:
+        raise RuntimeError("google.com missing from Tranco — the ranks API changed")
+    if "gstatic.com" not in umbrella:
+        raise RuntimeError("gstatic.com missing from Umbrella — the zip changed shape")
+    tld_path = cached(UMBRELLA_TLD_ZIP, "umbrella_top1m_tld.csv.zip")
+    tld = load_rank_csv(tld_path, domain_col=1, rank_col=0, skip_header=False)
+    if "com" not in tld:
+        raise RuntimeError("TLD 'com' missing from Umbrella top-1m-TLD.csv.zip")
+    print(f"fetched_at {fetched_at}")
+    print(f"tranco {list_id} providers {','.join(cfg['providers'])} {cfg['combinationMethod']}")
+    print(f"umbrella sha256 {hashlib.sha256(umbrella_path.read_bytes()).hexdigest()[:12]} n={len(umbrella)}")
+    print(f"majestic sha256 {hashlib.sha256(majestic_path.read_bytes()).hexdigest()[:12]} n={len(majestic)}")
+    print(
+        "umbrella TLD file: "
+        f"com={tld['com']} net={tld['net']} org={tld['org']} n={len(tld)}"
+    )
+    print()
+    if args.wiki:
+        print("^ Name ^ Tranco (PLD, " + list_id + ") ^ Cisco Umbrella (hostname) ^ Majestic Million (backlinks) ^")
+        for name, t, u, m in rows:
+            print(f"| ''{name}'' | {fmt(t)} | {fmt(u)} | {fmt(m)} |")
+    else:
+        print("name\ttranco\tumbrella\tmajestic")
+        for name, t, u, m in rows:
+            print(f"{name}\t{fmt(t)}\t{fmt(u)}\t{fmt(m)}")
+    return 0
+if __name__ == "__main__":
+    sys.exit(main())
+</file>
+To pin a Tranco id rather than compare ranks, use ''python3 pin_tranco.py'' on [[Programming:Tranco]].
+===== Which list in 2026 =====
+Checked against each vendor's own documentation and APIs on **2026-08-27**. Dated deliberately.
+^ Source ^ What it measures ^ Pin with ^ Status 2026-08-27 ^ Papers of 1,153 ^
+| **CrUX** | Page-load buckets of **origins** from eligible Chrome | month + bucket | Current. Docs live. Google: CC BY 4.0. Tranco homepage still labels CrUX CC BY-SA 4.0; methodology page states no CrUX licence | 36 (3.1%) |
+| **Tranco** | Aggregate of CrUX, Farsight, Majestic, Radar, Umbrella | list id (''46W9X'' dated 2026-08-26) | Current. Daily list; Farsight is default-list-only | 262 (22.7%) |
+| **Cloudflare Radar** | DNS to 1.1.1.1, pay-level domains | snapshot date (no permanent id) | Current. Ordered top 100; unordered buckets below. CC BY-NC 4.0 | 4 (0.3%) |
+| **Cisco Umbrella** | DNS to OpenDNS, hostnames | dated zip | Current. top-1m.csv.zip still published | 24 (2.1%) |
+| **Majestic Million** | Backlinks | download date + file hash | Current. CSV advertised | 26 (2.3%) |
+| **SecRank** | Chinese DNS, voting | snapshot date | Current. secrank.cn live | 4 (0.3%) |
+| **SimilarWeb** | Commercial panel + crawl | only if the contract lets you keep the exact export | Current, paywalled. No public dated dump or permanent id | 12 (1.0%) |
+| **Farsight** (as ranking) | DNSDB cache-misses, 1M PLD | a Tranco daily id | Current **only inside Tranco's default list** | **0** |
+| **Quantcast** | Was a Tranco input | — | **Historical as a public ranking.** Measure still exists as publisher analytics | 8 (0.7%) |
+| **Alexa** | Toolbar / panel visits | snapshot, if you still have one | **Dead.** Site 1 May 2022; APIs 15 December 2022 | 463 (40.2%) |
+==== CrUX ====
+[[https://developer.chrome.com/docs/crux|CrUX]] publishes rank-magnitude buckets (1k, 5k, 10k, …, 5M) of popular **origins** based on user-initiated page loads from eligible Chrome installs — usage-statistic reporting, history sync without a passphrase, and a supported platform (not Chrome on iOS).(([[https://developer.chrome.com/docs/crux/methodology|CrUX methodology]], fetched 2026-08-27: user eligibility; datasets licensed **CC BY 4.0**. Tranco's homepage still labels CrUX [[https://creativecommons.org/licenses/by-sa/4.0/|CC BY-SA 4.0]]; the methodology page states no CrUX licence. Google's page is the primary source.)) There is no rank 5,000. Country- and device-specific slices exist; they measure Chrome page loads in that country, not "sites for that audience" — see [[Design:Website classification]]. API detail: [[Programming:CrUX]]. Galloway et al. did not evaluate it {[galloway2024_practical]}.
+==== Tranco ====
+[[https://tranco-list.eu/|Tranco]] is a research-oriented generator with permanent list ids, introduced in NDSS 2019 {[LePochat2019_tranco]} combining Alexa, Umbrella, Majestic and Quantcast. **Its inputs have changed three times** (Quantcast dropped after 2020-04-01, Farsight joined 2022-05-01, Alexa replaced by CrUX and Radar on 2023-08-01). The daily list of 2026-08-26 (id ''46W9X'') is CrUX, Farsight, Majestic, Radar and Umbrella, Dowdall over 30 days, ''filterPLD: on'', prefix 1,000,000.((Read from Tranco's own API on 2026-08-27: ''https://tranco-list.eu/api/lists/date/latest''. Describe composition from the live API rather than from the 2019 paper.)) A 2019 id and a 2026 id are different instruments. Radar's CC BY-NC 4.0 rides along on the default list. API, Farsight trap, ''pin_tranco.py'': [[Programming:Tranco]].
+==== Cloudflare Radar ====
+[[https://radar.cloudflare.com/domains|Cloudflare Radar Domain Rankings]] ranks **pay-level domains** from DNS queries to 1.1.1.1, not from HTTP hits on Cloudflare-operated websites. Ordered top 100 (daily, global and per country) plus unordered buckets up to 1M (weekly). Details: [[Programming:Cloudflare Radar]]. Manipulable with a cheap VPN {[galloway2024_practical]}.
+==== Cisco Umbrella ====
+[[https://umbrella-static.s3-us-west-1.amazonaws.com/index.html|Cisco Umbrella]] ranks names by DNS query traffic to OpenDNS resolvers. The daily top-1m zip was still published on 2026-08-27. Today's file is hostnames: ''1,google.com'' / ''2,gstatic.com'' / ''3,www.google.com''. The vendor page still says the list "includes root domains, subdomains in addition to TLDs"; the TLD ranking is a **different zip** (above). Typos and infrastructure hostnames survive. Still a Tranco default input.
+==== Majestic Million ====
+[[https://majestic.com/reports/majestic-million|Majestic]] ranks domains by backlinks. The Million page still advertised free search and download on 2026-08-27 (''downloads.majestic.com/majestic_million.csv''). CC BY 3.0, per Tranco. Does not measure visits. Link-graph rankings are cheap to manipulate relative to a toolbar or panel list {[LePochat2019_tranco]}. That paper did not evaluate CrUX.
+==== Farsight (DomainTools) ====
+A **1-million pay-level-domain ranking** built from cache-misses in Farsight's DNSDB — "above-resolver" DNS that organisations share, so a name that stays in a recursive cache is invisible. Tranco has used it as a default input since **1 May 2022**, the same day Alexa.com was retired.((Tranco [[https://tranco-list.eu/methodology|methodology]], fetched 2026-08-27: "Farsight Security, part of DomainTools, provides a daily updated list consisting of one million entries. … Only pay-level domains are ranked. This list is available since May 1, 2022." Ranks "are based on passive DNS traffic from its DNSDB dataset" of cache misses. DomainTools' own post [[https://www.domaintools.com/blog/mirror-mirror-on-the-wall-whos-the-fairest-website-of-them-all|Mirror, Mirror]] is the ranking's public description.)) It is **not a public CSV**. You get it only as part of Tranco's daily list: the custom-list API's ''providers'' enum is ''crux / majestic / radar / umbrella / alexa / quantcast'' — Farsight, which **is** in the daily list, is **not** in that enum. See [[Programming:Tranco]]. **Zero papers in this corpus name the Farsight ranking as a ''sourceList''.** The 13 papers that name Farsight or DNSDB used the passive-DNS **dataset**, which is a different instrument.
+==== Alexa, Quantcast, SimilarWeb, SecRank ====
+  * ''[[https://en.wikipedia.org/wiki/Alexa_Internet|Alexa]]'': **Discontinued — alexa.com was retired on 1 May 2022 and the Alexa Top Sites and Web Information Service APIs on 15 December 2022.**((Alexa Support, "We retired Alexa.com on May 1, 2022", archived at [[https://web.archive.org/web/20221126115049/https://support.alexa.com/hc/en-us/articles/4410503838999-We-retired-Alexa-com-on-May-1-2022|web.archive.org]]. The date **1 August 2023** is when Tranco dropped Alexa from the default list and folded in CrUX and Cloudflare Radar — not when Amazon switched Alexa off.)) Rankings came from a browser toolbar / panel {[scheitle2018_long]}. **Do not start a new crawl from an Alexa snapshot** unless the study is a reproduction of a pre-2022 paper, in which case name the snapshot.
   * ''Quantcast'': Was a Tranco default input until 1 April 2020. The Measure product still exists as publisher analytics (''quantcast.com/measure/'' redirects to ''/publisher/measure''); it is not a public top-sites ranking you can pin. Named by 8 papers, none in 2025–2026.
-  * ''[[https://en.wikipedia.org/wiki/Alexa_Internet|Alexa]]'': **Discontinued — alexa.com was retired on 1 May 2022 and the Alexa Top Sites and Web Information Service APIs on 15 December 2022.**((Alexa Support, "We retired Alexa.com on May 1, 2022", archived at [[https://web.archive.org/web/20221126115049/https://support.alexa.com/hc/en-us/articles/4410503838999-We-retired-Alexa-com-on-May-1-2022|web.archive.org]]. The date **1 August 2023** is when Tranco dropped Alexa from the default list and folded in CrUX and Cloudflare Radar — not when Amazon switched Alexa off. Papers published years later still cite it as a live source: see [[Design:Sampling]].)) Rankings came from a browser toolbar / panel, i.e. opted-in visits rather than DNS {[scheitle2018_long]}. **Do not start a new crawl from an Alexa snapshot** unless the study is a reproduction of a pre-2022 paper, in which case name the snapshot.
+  * ''SimilarWeb'': Commercial panel and crawl estimates; paywalled (some research exceptions exist). See [[Programming:SimilarWeb]]. Named by 12 of 1,153 web-sampling papers.
-  * ''Farsight'' (DomainTools): a **1-million pay-level-domain ranking** built from cache-misses in Farsight's DNSDB — "above-resolver" DNS that organisations share, so a name that stays in a recursive cache is invisible. Tranco has used it as a default input since **1 May 2022**, the same day Alexa.com was retired.((Tranco [[https://tranco-list.eu/methodology|methodology]], fetched 2026-08-27: "Farsight Security, part of DomainTools, provides a daily updated list consisting of one million entries. … Only pay-level domains are ranked. This list is available since May 1, 2022." Ranks "are based on passive DNS traffic from its DNSDB dataset" of cache misses. DomainTools' own post [[https://www.domaintools.com/blog/mirror-mirror-on-the-wall-whos-the-fairest-website-of-them-all|Mirror, Mirror]] is the ranking's public description.)) It is **not a public CSV**. You get it only as part of Tranco's daily list: the custom-list API's ''providers'' enum is ''crux | majestic | radar | umbrella | alexa | quantcast'' — Farsight, which **is** in the daily list, is **not** in that enum. See [[Programming:Tranco]]. Advantages: sees cache-miss DNS from participating organisations' resolvers, including names that only infrastructure looks up; Tranco's only exclusive input. Limitations: cache-miss bias (popular names are undercounted if they stay cached); biased to organisations that share with DomainTools; DNS-based; cannot be pinned independently of a Tranco daily id. **Zero papers in this corpus name the Farsight ranking as a ''sourceList''.** The 13 papers that name Farsight or DNSDB used the passive-DNS **dataset**, which is a different instrument.
   * ''[[https://secrank.cn/|SecRank]]'': Voting-based ranking from Chinese DNS, introduced in USENIX Security 2022 {[xie2022_building]}. Site live on 2026-08-27. Named by 4 papers. Use it when the audience is Chinese resolver traffic; it is not a drop-in Alexa replacement for a global crawl.
-===== Best Practices =====
+===== They disagree, and that is the finding =====
+==== Rank-magnitude, not a shuffled head ====
+Ruth et al. compared public lists to Cloudflare HTTP logs in February 2022 {[ruth2022_toppling]}:
+> of the 1,790 domains we measure in the Alexa top 10K, 70% of them are ranked by Cloudflare in a lower rank-magnitude bucket, and 27.2% of them are ranked by Cloudflare in a bucket two or more orders of magnitude less popular.
+On the Alexa top 1K the same paper reports **87.1%** overranked (average of 210 sites) and **56.7%** by two or more orders. CrUX is the exception they found: **47.1%** of 1,410 CrUX top-10K domains overranked against Cloudflare, and **only 1%** by two or more orders. "Inaccurate" is not a small permutation of the head; it is a different web. They did not evaluate the five-provider Tranco list current in 2026.
+Scheitle et al. measured agreement between the lists themselves {[scheitle2018_long]}: for the Top 1M, Alexa and Majestic shared **285k** domains on average in their 2017–2018 joint window — well under a third. Daily churn reached **50%** of domains on some lists. An unused test domain reached rank **22k** on Umbrella.
-  - **Pick the instrument that matches the claim.** Page-load questions want CrUX (or Tranco, knowing CrUX is one input). DNS / infrastructure questions can use Umbrella, Radar or SecRank, and must say so. Backlink questions can use Majestic. "The web" is not a use-case any of these lists enumerate.
+==== The head is a technological outlier ====
-  - **Prefer a month of ranks, or buckets, over a single daily total order** unless you need one. {[LePochat2019_tranco]} is the stability argument for aggregating over 30 days; CrUX publishes magnitudes rather than a total order, which is a different kind of stability and not one this corpus re-measured.
+Scheitle et al. compared list domains against the general ''com''/''net''/''org'' population {[scheitle2018_long]}:
-  - **Cite a list identity, not a list name.** A Tranco id, a CrUX month, an Umbrella dated zip, a hash of the file you archived. 52.0% of papers that named a ranking vendor in this corpus stated no version on that tuple (397 of 763). [[Design:Sampling]] is the page for that gap.
-  - **Do not treat Tranco as CrUX.** Tranco's default list mixes a page-load ranking with three DNS rankings and a backlink ranking. Ruth et al. found CrUX closest to HTTP logs; that comparison has not been re-run on the five-provider list you download today {[ruth2022_toppling]}.
-  - **Publish the file.** A GitHub URL is not an archival artefact — see [[Artifacts]].
-  - **If you can, report on more than one list.** 55 of 763 ranking-users (7.2%) named two or more vendors. Ruth et al. showed CrUX and Tranco are different instruments against HTTP logs {[ruth2022_toppling]}; they did not evaluate reporting both.
-===== Detailed Analysis =====
+> the HTTP/2 adoption of all com/net/org domains is 7.84% on average and thus significantly lower than for domains listed in Top 1M lists, (up to 26.6% for Alexa)
-==== Limitations of DNS-Based Lists ====
-DNS-based lists — **Cisco Umbrella**, **Farsight**, **SecRank**, and partially **Cloudflare Radar** and **Tranco** — are built from resolver observations rather than page loads. Umbrella ranks by query traffic; Radar estimates user-population size from 1.1.1.1 (see [[Programming:Cloudflare Radar]]); Farsight ranks pay-level domains by DNSDB cache-misses. The input is DNS, which introduces several limitations:
-DNS lists capture queries from devices and applications that resolve names without a user looking at a page. That over-represents software updates, network configuration and telemetry:
+Top 1k lists showed adoption around **35%** or more. The same paper's IPv6 and CDN gaps went the same way. On that indicator the head was a technological outlier; it is not a theorem that every later adoption measure will. How you //draw// from the head is [[Design:Sampling]].
-  * ''windowsupdate.microsoft.com'' is queried for updates and is not a typical user-visited website.
-  * Internal or infrastructure names, such as ''%%ec2.internal%%'', appear on Umbrella; Umbrella's own example file ranks the TLDs ''com'' and ''net'' above ''google.com''.
-Galloway et al. generated names, queried them through a **$10/month VPN**, and "consistently achieved a ranking in the top 100,000" on **Radar**; Tranco then placed the same names at rank 1 million within 10 days and inside 500k within 14 days {[galloway2024_practical]}. On today's Radar API that top 100,000 is an **unordered bucket**, not a total order — see [[Programming:Cloudflare Radar]]. That is a finding about Radar (and Tranco via that input), not about every DNS ranking.
+==== DNS lists are not lists of websites ====
+DNS-based lists — **Umbrella**, **Farsight**, **SecRank**, and parts of **Radar** and **Tranco** — count resolver observations. That over-represents names that devices look up without a user opening a page. The table above is the current evidence: ''gstatic.com'' and ''windowsupdate.com'' sit in Tranco's head; ''windowsupdate.microsoft.com'' is an Umbrella hostname that a PLD list drops. An earlier draft of this page used ''ec2.internal'' as the running example; it is **not** in the 2026-08-27 Umbrella top-1m.
-==== Temporal Stability and Manipulations ====
+Galloway et al. generated names, queried them through a **$10/month VPN**, and "consistently achieved a ranking in the top 100,000" on **Radar**; Tranco then placed the same names at rank 1 million within 10 days and inside 500k within 14 days {[galloway2024_practical]}. They also sat consistently under rank 250,000 on Umbrella. On the Radar API as of 2026-08-27 that top 100,000 is an **unordered bucket**, not a total order — see [[Programming:Cloudflare Radar]]. That is a finding about Radar (and Tranco via that input), not about every DNS ranking, and not about CrUX.
+==== Stability and manipulation ====
 **Alexa**, when it was live, ranked from a toolbar panel. Two problems, both measured:
   * **Manipulations**: ranks could be altered with as little as a single HTTP request {[LePochat2019_tranco]}. There was a real-world incentive to do this.
   * **Stability**: a small panel, a large web, and a daily publish produced high churn.
-Tranco was designed to reduce both by aggregating sources over 30 days. CrUX publishes buckets rather than a total order, so a name that moves inside a magnitude stays in the same published bucket; this page does not re-measure that as a stability rate. CrUX is theoretically susceptible to manipulation; it is collected from eligible Chrome installs (not Chrome's full install base). {[LePochat2019_tranco]} costed attacking **Alexa** as cheap; it did not evaluate CrUX. Galloway et al. also did not evaluate CrUX {[galloway2024_practical]}.
+Tranco was designed to reduce both by aggregating sources over 30 days. CrUX publishes buckets rather than a total order, so a name that moves inside a magnitude stays in the same published bucket; this page does not re-measure that as a stability rate. {[LePochat2019_tranco]} costed attacking **Alexa** as cheap; it did not evaluate CrUX. Galloway et al. also did not evaluate CrUX {[galloway2024_practical]}.
 <WRAP right 50% box>
@@ Line 84: / Line 300: @@
 <div>Possible manipulations of lists and their estimated cost according to {[LePochat2019_tranco]}.</div>
 </WRAP>
-==== Representativeness ====
-  * CrUX ranks origins by completed page loads, which is why Ruth et al. found it closest to Cloudflare HTTP logs. Majestic ranks by backlinks. Umbrella, Radar and Farsight rank by DNS. Those are three different populations. A result on one does not transfer to the others, and a Tranco aggregate does not make them the same population.
-  * Of the 1,790 Alexa top-10K domains Ruth et al. could measure against Cloudflare, **70% sat in a lower rank-magnitude bucket** and **27.2% were two or more orders of magnitude lower** {[ruth2022_toppling]}. "Inaccurate" is not a small permutation of the head; it is a different web.
-  * Scheitle et al. compared list domains against the general ''com''/''net''/''org'' population: HTTP/2 adoption **26.6% on the Alexa Top 1M against 7.84% overall** {[scheitle2018_long]}. On that indicator the head was a technological outlier. The same paper's IPv6 and CDN gaps went the same way; it is not a theorem that every later adoption measure will.
 ===== Use in Publications =====
@@ Line 94: / Line 305: @@
 The figures below come from a structured extraction over **5,859 full-text papers** from CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026. The population for this page is the **1,153 papers that drew at least one study population whose unit is websites, domains or web pages** — 19.7% of the corpus, the same population as [[Design:Sampling]]. Only those web-unit tuples are counted. Free-text ''sourceList'' is folded into vendor families by ''scripts/rank_fold.mjs'' before counting (multi-label: a string naming Tranco and Radar counts for both). Sentinel values are silence. 2025 and 2026 are provisional venue-years.
-This replaces the borrowed Scheitle 2018 and Xie 2024 survey figures as a description of **what this corpus did**. Those two papers remain the right citations for the pre-corpus surveys; they are below, dated.
+Of the 1,143 WEB papers that name any source, **764 (66.8%)** name a popularity ranking and **257 (22.5%)** name a custom or ad-hoc seed list — the same frame-kind fold as [[Design:Sampling]]. This page is about the 763 of those ranking papers that name a **vendor**. A custom seed is a different instrument; it does not go in the table below.
-==== Counting exact strings undercounts Alexa by 89% ====
+==== Folded vendors, not exact strings ====
-The work item that commissioned this section pointed at exact-string counts over all 5,712 papers that sampled anything: "custom seed list 377, Google Play 183, Tranco 119, Alexa 117" on the previous corpus. On this run the same method produces:
-^ Exact ''sourceList'' string ^ Papers of 5,712 sampled ^
-| custom seed list | 516 |
-| Tranco | 180 |
-| Google Play | 143 |
-| Prolific | 122 |
-| Alexa | 51 |
-**Google Play is not a website ranking.** It is 281 papers after folding ''/google play/i'' over all sampled papers, and **0** over the 1,153 that sampled the web. Publishing that top-four as "which lists researchers use" would have ranked an app store above Alexa. Restrict to web units and fold spellings:
 ^ Vendor ^ Papers of 1,153 ^ Share ^ Exact name only ^ Spellings folded ^ Exact-string undercount ^
@@ Line 121: / Line 321: @@
 **763 papers (66.2%) name at least one of these vendors.** Family counts do not sum to 763: 55 papers (7.2% of vendor-users) name two or more. Farsight-as-ranking: **0 papers**. Radar as a named source: 4, all in 2025–2026 — it is a Tranco input, not a list people cite. The full fold, including commercial also-rans (Semrush 3, BuiltWith 2, Open PageRank 2, Ahrefs / Moz / Netcraft / Trexa 1 each) and the unnamed-top-N residue (4 strings), is on the [[provenance:design:website_selection|provenance page]].
+Counting the exact string "Alexa" on all 5,712 papers that sampled //anything// finds 51; folding on WEB finds 463. The same un-folded method on ALL ranks "custom seed list" (516), Tranco (180) and Google Play (143) above Alexa (51). Google Play is 281 papers after folding ''/google play/i'' over ALL, and **0** over the 1,153 that sampled the web. Restrict to web units and fold spellings, or you will publish an app store as a website ranking.
 ==== The list changed under the field ====
@@ Line 142: / Line 344: @@
 ==== What the pre-corpus surveys said ====
-Scheitle et al. {[scheitle2018_long]} and Xie et al. {[xie2024_crawling]} surveyed ranking-list usage in web-measurement papers by hand. They are why this page had figures before this corpus existed. Scheitle ends in 2016-era crawls; Xie's survey of crawling papers ends around 2022. Neither includes 2023–2026, which is the window in which Tranco became the default and Alexa became residue. **Alexa was not "discontinued in 2023"** — that sentence was on an earlier version of this page and confused Amazon's 2022 retirement with Tranco's 2023 provider swap.
+Scheitle et al. {[scheitle2018_long]} and Xie et al. {[xie2024_crawling]} surveyed ranking-list usage in web-measurement papers by hand. They are why this page had figures before this corpus existed. Scheitle ends in 2016-era crawls; Xie's survey of crawling papers ends around 2022 (PAM 2024; outside the seven venues). Neither includes 2023–2026, which is the window in which Tranco became the default and Alexa became residue. **Alexa was not "discontinued in 2023"** — that sentence was on an earlier version of this page and confused Amazon's 2022 retirement with Tranco's 2023 provider swap.
 <WRAP center 100% box>
@@ Line 158: / Line 360: @@
   * **Folding.** ''scripts/rank_fold.mjs'' maps a string onto every vendor it names. Alexa undercounts by 89.2% if you count the exact string "Alexa". Farsight/DNSDB strings are a hand map of 12 spellings classified as a passive-DNS **dataset**, not a ranking; the report fails if a new spelling appears. Residue of unnamed "top N websites" strings: 4.
   * **This is not the 5,712-paper ''sampled'' population.** That denominator is 97.5% of a corpus that is mostly not web measurement. Google Play, Prolific and MNIST live in it.
-  * **Sampling's 764 "popularity-ranking" papers** are a frame-kind fold, not a vendor fold; the vendor union here is 763. The one-paper gap is an unnamed top-N string that names no vendor. [[Design:Sampling]] reports the frame kind; this page reports the vendor.
+  * **Sampling's 764 "popularity-ranking" papers** are a frame-kind fold, not a vendor fold; the vendor union here is 763. The one-paper gap is an unnamed top-N string that names no vendor. [[Design:Sampling]] reports the frame kind; this page reports the vendor. Custom-seed 257 is on that page too.
   * **Posters.** 23 of 1,153 WEB papers are posters; 10 of 763 vendor-users. Trimmed shares (posters and ≤4-page records removed, 1,113 papers) move Alexa 40.2% → 41.0% and Tranco 22.7% → 22.8%. The headline is not a silence rate.
-  * **Venue coverage.** Seven venues only. 2025 and 2026 are incomplete by construction.
+  * **Venue coverage.** Seven venues only. 2025 and 2026 are incomplete by construction. Xie 2024 is PAM, cited as a survey, not counted in the tables.
   * **Every query, the report script and its unedited output** are on [[provenance:design:website_selection]]; corpus-level caveats are on [[literature:corpus]].
@@ Line 172: / Line 374: @@
   - **The file** — archived, hashed, and deposited. See [[Artifacts]].
-''python3 pin_tranco.py'' on [[Programming:Tranco]] prints the live configuration and a ''cite'' line. Use it.
+Pin **one dated frame**, not an informal average of several daily lists. Tranco's own daily id already aggregates 30 days (Dowdall); that is a defined procedure with a permalink. CrUX is a monthly bucket snapshot — cite the month. {[LePochat2019_tranco]} is the stability argument for that kind of aggregation, not a licence to mix unpublished daily CSVs. Do not treat Tranco as CrUX. If you can, report on more than one list: 55 of 763 ranking-users (7.2%) named two or more vendors; Ruth et al. showed CrUX and Tranco are different instruments against HTTP logs {[ruth2022_toppling]}, and they did not evaluate reporting both. 52.0% of papers that named a ranking vendor in this corpus stated no version on that tuple (397 of 763).
+===== Open Questions =====
+<WRAP todo>
+  * **Ruth et al. has not been re-run on the five-provider Tranco list.** The 2022 comparison predates Radar-in-Tranco and the Alexa→CrUX swap. Whether the 2026 default list is closer to CrUX or to Umbrella is an unmeasured claim, and the ''gstatic.com'' rank above is a reason to measure it.
+  * **How much does swapping Tranco for CrUX move a published web-measurement figure?** Several papers could be re-crawled on both; we found none that reports the same measurement on both as its contribution. The IP-classification analogue — four geolocation databases on one address set — is the template.
+  * **No public download of the Farsight ranking**, and zero papers name it as a ''sourceList''. Tranco's methodology describes it; DomainTools does not publish a CSV equivalent of Umbrella's. Whether that input is doing work in the daily list is opaque from outside.
+  * **The 34 undated 2023–2026 Alexa papers.** A full-text pass that distinguished cached CSVs, inherited lists, and crawls that started before May 2022 would turn a residual count into a finding.
+</WRAP>
 ===== Related pages =====
-  * [[Design:Sampling]] — how to draw, sample size, unit, versioning. Assumes you have chosen a list.
+  * [[Design:Sampling]] — how to draw, sample size, unit, versioning. Assumes you have chosen a list. Custom-seed lists live there.
   * [[Design:Longitudinal]] — why a 2019 Tranco id and a 2026 Tranco id are not commensurable.
   * [[Programming:Tranco]] — ids, API, the Farsight-is-default-list-only trap, ''pin_tranco.py''.
@@ Line 182: / Line 393: @@
   * [[Programming:CrUX]] — magnitude buckets, BigQuery, country/device slices.
   * [[Programming:SimilarWeb]] — the commercial panel.
-  * [[Design:Website classification]] — topic labels, not popularity ranks. Alexa died as a categoriser on the same day it died as a ranking.
+  * [[Design:Website classification]] — topic labels, not popularity ranks. Alexa died as a categoriser on the same day it died as a ranking. CrUX country lists are popularity, not audience.
+  * [[Design:IP classification]] — the servers a crawl connects to are mostly CDNs; geolocating them is a different question from picking the crawl's seed list.
   * [[Artifacts]] — depositing the pinned list.
   * [[literature:corpus]] — how the 5,859-paper extraction was built.