User Tools

Site Tools


design:ownership_resolution

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
design:ownership_resolution [2026/09/11 17:50] – Create Design:Ownership resolution: domain-to-company attribution as its own page, carved out of Programming:Crawler:webXray where the material was filed under a dead tool's name. Carries the three-list comparison, the coverage census, the 28-row disagree karel.kubicek.claudedesign:ownership_resolution [2026/09/11 22:42] (current) – Site-wide sweep: the methodology bullet's second copy of '29 of the 525' hand-changed verdicts was still stale after round four fixed the first. Authored by Claude karel.kubicek.claude
Line 6: Line 6:
 **The short answer, dated 2026-09-11.** **The short answer, dated 2026-09-11.**
  
-  * **Starting a new measurement:** use **Disconnect ''entities.json''** for the owner name and **DuckDuckGo Tracker Radar** for breadth, and say which file and which commit. On a stratified random sample of each list's own coverage, adjudicated 2026-09-05, Disconnect names today's owner for **94.4%** of the entries that could be settled against Tracker Radar's **73.7%**; Tracker Radar names an owner for **17.2%** of the registrable third-party domains in the frame against Disconnect's **7.0%**. Neither dominates, so most papers should merge them and say in which order.+  * **Starting a new measurement:** use **Disconnect ''entities.json''** for the owner name and **DuckDuckGo Tracker Radar** for breadth, and say which file and which commit. On a stratified random sample of each list's own coverage, drawn 2026-09-05 and re-adjudicated over its residue on 2026-09-11, Disconnect names today's owner for **89.2%** of the entries that could be settled against Tracker Radar's **73.9%** — a gap that is significant if you count //granularity// as acceptable (//p// = 0.004) and is not if you require the exact legal entity (//p// = 0.083, against a Bonferroni threshold of 0.0167, and it never cleared it); Tracker Radar names an owner for **17.2%** of the registrable third-party domains in the frame against Disconnect's **7.0%**. Neither dominates, so most papers should merge them: **Disconnect first for the owner name, Tracker Radar as the fallback when Disconnect has no entry** — and record which list and which key matched each domain, so the mix is visible in your data rather than buried in the pipeline. Those two headline figures are the metrics on which this recommendation looks best, and the body of the page argues that two //other// metrics matter more: weighted by how often a crawl meets the domain, the live lists are near-tied on coverage (84.3% against 80.5%) and the **frozen** list tops accuracy (93.8%). The recommendation survives that because webXray's file has not moved since 2021 and its 93.8% is measured over the fifth of its entries a 2026 crawl still meets — but you should know the comparison is not one-sided before you cite it.
   * **webXray's ''domain_owners.json'' is historical.** It has not moved since **2021-03-04**, covers **2.1%** of the same frame, and the last corpus paper to use it was published in **2022**. It is still the only one of the three with an ownership //tree// and with per-language policy URLs. See [[Programming:Crawler:webXray]] for the file, the tool and its licence history.   * **webXray's ''domain_owners.json'' is historical.** It has not moved since **2021-03-04**, covers **2.1%** of the same frame, and the last corpus paper to use it was published in **2022**. It is still the only one of the three with an ownership //tree// and with per-language policy URLs. See [[Programming:Crawler:webXray]] for the file, the tool and its licence history.
   * **No list removes the manual work**, and the literature is consistent on this: {[steffens2021_blockparty]} spent about eight person-hours hand-vetting same-entity pairs //after// using the lists, and {[sanchezrola2021_journey]} merged three of them and added a WHOIS-and-graph pass on top.   * **No list removes the manual work**, and the literature is consistent on this: {[steffens2021_blockparty]} spent about eight person-hours hand-vetting same-entity pairs //after// using the lists, and {[sanchezrola2021_journey]} merged three of them and added a WHOIS-and-graph pass on top.
 +  * **Three papers to read first**, in this order: {[steffens2021_blockparty]} for what the lists miss and what the manual work actually costs; {[sanchezrola2021_journey]} for how to merge several and state a priority order; {[matte2020_cookie]} for the reproducibility table to copy.
   * **WHOIS is not an ownership database.** Used unfiltered it returns registrar-privacy strings, and one published top-ten table of "organizations" is a third registrar proxies — see the box in [[#Choosing a resolution source now]].   * **WHOIS is not an ownership database.** Used unfiltered it returns registrar-privacy strings, and one published top-ten table of "organizations" is a third registrar proxies — see the box in [[#Choosing a resolution source now]].
 </WRAP> </WRAP>
Line 23: Line 24:
  
   * **Granularity.** Brand (''DoubleClick'', ''Bing'', ''YouTube'') / operating legal entity (''Google LLC'', ''Microsoft Corporation'') / ultimate parent (''Alphabet''). webXray can give you any of the three, because it carries a ''parent_id'' tree; Tracker Radar gives the legal entity; Disconnect gives a compact grouping name. The difference is not small: ''bing.com'', ''linkedin.com'' and ''adnxs.com'' all become Microsoft at the entity level.   * **Granularity.** Brand (''DoubleClick'', ''Bing'', ''YouTube'') / operating legal entity (''Google LLC'', ''Microsoft Corporation'') / ultimate parent (''Alphabet''). webXray can give you any of the three, because it carries a ''parent_id'' tree; Tracker Radar gives the legal entity; Disconnect gives a compact grouping name. The difference is not small: ''bing.com'', ''linkedin.com'' and ''adnxs.com'' all become Microsoft at the entity level.
-  * **Vintage.** Which snapshot, and when ownership was last checked. Ad tech consolidates continuously — the same problem {[selmo2025_borges]} names at the network layer as "an Internet shaped by constant mergers, rebrandings, and regional variation". ''adnxs.com'' is Xandr under AT&T in one list and Microsoft in another, and the difference is a 2022 acquisition, not an error.+  * **Vintage.** Which snapshot, and when ownership was last checked. Ad tech consolidates continuously — the same problem {[selmo2025_borges]} names at the network layer as "an Internet shaped by constant mergers, rebrandings, and regional variation".((That sentence is contiguous in the published PDF — checked with pypdf on 2026-09-11 — but the string does **not** appear in this corpus's column-repaired paper.cols.txt rendering of the paper, which reorders the surrounding text. If you are quote-checking two-column ACM PDFs against a single rendering, this is the failure mode: the two renderings fail on different sentences, and a miss in one is not evidence of a miss in the source.)) ''adnxs.com'' is Xandr under AT&T in one list and Microsoft in another, and the difference is a 2022 acquisition, not an error.
  
 A third axis appears only once you have chosen a list: **what counts as the same company for a first-party check**. Resolving the //request// domain is the easy half; the step almost always missing is resolving the //visited site// too, so that ''google.com'' embedding ''gstatic.com'' does not count as third-party tracking. See [[#Assembling the pipeline]]. A third axis appears only once you have chosen a list: **what counts as the same company for a first-party check**. Resolving the //request// domain is the easy half; the step almost always missing is resolving the //visited site// too, so that ''google.com'' embedding ''gstatic.com'' does not count as third-party tracking. See [[#Assembling the pipeline]].
Line 37: Line 38:
 | Extra per-owner data | purpose, country, trade bodies, per-language policy URLs | ''displayName'', ''aliases''; prevalence, categories, fingerprinting and cookie behaviour in sibling files | ''properties'' vs ''resources'' split; category in ''services.json'' | | Extra per-owner data | purpose, country, trade bodies, per-language policy URLs | ''displayName'', ''aliases''; prevalence, categories, fingerprinting and cookie behaviour in sibling files | ''properties'' vs ''resources'' split; category in ''services.json'' |
 | How ownership is decided | hand curation. Libert's own paper calls the database "the product of years of detective work" {[libert2018_automated]} | "automatically generated" by Tracker Radar Detector; "new development and bug fixes, other than broken sites, are handled internally"((''docs/DATA_MODEL.md'' in ''duckduckgo/tracker-radar'', checked 2026-08-17.)) | Disconnect's own process page describes step 4 as "Connect every tracker to its parent entity through DNS, WHOIS, and behavioral evidence"((''https://disconnect.me/trackerprotection'', checked 2026-08-17.)) | | How ownership is decided | hand curation. Libert's own paper calls the database "the product of years of detective work" {[libert2018_automated]} | "automatically generated" by Tracker Radar Detector; "new development and bug fixes, other than broken sites, are handled internally"((''docs/DATA_MODEL.md'' in ''duckduckgo/tracker-radar'', checked 2026-08-17.)) | Disconnect's own process page describes step 4 as "Connect every tracker to its parent entity through DNS, WHOIS, and behavioral evidence"((''https://disconnect.me/trackerprotection'', checked 2026-08-17.)) |
-| Update cadence | **none** — frozen; last public commit 2021-03-04 | monthly regeneration; last commit on ''main'' 2026-08-28, releases tagged ''2026.08.28'', ''2026.07.27'', … | continuous; last commit 2026-08-28 |+| Update cadence | **none** — frozen; last public commit 2021-03-04 | monthly regeneration; last commit on ''main'' 2026-08-28, releases tagged ''2026.08.28'', ''2026.07.27'', … | the **repository** commits continuously (HEAD 2026-09-05), but ''entities.json'' itself last changed **2026-08-07** — read the file's history, not the repo's |
 | Licence | PolyForm Strict 1.0.0 (no redistribution) on the most-mirrored snapshot; the project ended MIT, the 2018 standalone list copy is GPL-3.0 — see [[Programming:Crawler:webXray]] | **CC BY-NC-SA 4.0** | **CC BY-NC-SA 4.0** (GPL-3.0 until 2020-06-24) | | Licence | PolyForm Strict 1.0.0 (no redistribution) on the most-mirrored snapshot; the project ended MIT, the 2018 standalone list copy is GPL-3.0 — see [[Programming:Crawler:webXray]] | **CC BY-NC-SA 4.0** | **CC BY-NC-SA 4.0** (GPL-3.0 until 2020-06-24) |
 | Documented method | the 2018 paper, plus the list's own ''notes'' field | ''docs/DATA_MODEL.md'', ''docs/FAQ.md'', a vendor blog post; no paper | a six-step process page and inclusion criteria on ''disconnect.me''; ''entities.json'''s own schema is undocumented | | Documented method | the 2018 paper, plus the list's own ''notes'' field | ''docs/DATA_MODEL.md'', ''docs/FAQ.md'', a vendor blog post; no paper | a six-step process page and inclusion criteria on ''disconnect.me''; ''entities.json'''s own schema is undocumented |
Line 165: Line 166:
 ==== Which list is right, when they disagree? ==== ==== Which list is right, when they disagree? ====
  
-30 of the highest-prevalence disagreements were adjudicated one by one against primary sources — company newsrooms, SEC filings, or the domain's own legal documents — on 2026-08-17. Two could not be settled from a primary source and are **excluded from the table**, leaving 28.+30 of the highest-prevalence disagreements were adjudicated one by one against primary sources — company newsrooms, SEC filings, or the domain's own legal documents — on 2026-08-17, **by Sonnet sub-agents working to a published brief, single-rated**, not by a human expert. Read every verdict below with that in mind; the inter-rater figure further down is measured on the 2026-09-05 random sample, not on this table. Two could not be settled from a primary source and are **excluded from the table**, leaving 28.
  
 ^ List ^ Names today's owner ^ Stale (a real former owner) ^ Granularity only ^ Outright error ^ No entry ^ ^ List ^ Names today's owner ^ Stale (a real former owner) ^ Granularity only ^ Outright error ^ No entry ^
Line 182: Line 183:
 ===== How often is each list right? A random sample ===== ===== How often is each list right? A random sample =====
  
-The table above is what you see when you look **where the lists disagree**. It is not an accuracy rate. This section is the measurement that was missing until 2026-09-05: **60 registrable domains drawn at random from each list's own coverage**, stratified by prevalence quartile, each adjudicated against a primary source. 180 draws over **175 distinct domains** (five were drawn for more than one list); each domain was adjudicated **once** and scored against all three lists, giving 525 verdicts. Only a list's own draws enter that list's rate, and of each list's 60 the ones that could be settled //and// make an ownership claim are **49, 47 and 42** — 138 rows carry every accuracy figure below.+The table above is what you see when you look **where the lists disagree**. It is not an accuracy rate. This section is the measurement that was missing until 2026-09-05: **60 registrable domains drawn at random from each list's own coverage**, stratified by prevalence quartile, each adjudicated against a primary source. **The adjudicators were language models** — Sonnet sub-agents working to a published brief, one rater per domain, with a second model re-rating a random fifth so the agreement could be measured rather than assumed (below). No human expert adjudicated these rows. 180 draws over **175 distinct domains** (five were drawn for more than one list); each domain was adjudicated **once** and scored against all three lists, giving 525 verdicts. Only a list's own draws enter that list's rate, and of each list's 60 the ones that could be settled //and// make an ownership claim are **56, 56 and 54** — 166 rows carry every accuracy figure below. Those counts are **after a second pass on 2026-09-11** that re-adjudicated the 40 domains the first pass could not settle, using sourcing the first pass did not attempt: national company registers searched by //company number// rather than by name, the registrar's own port-43 WHOIS rather than RDAP, and archived captures of the domains' own legal pages. It settled **28 of the 40**, cutting the unadjudicable share from 23% of the draws to 7.8% — see [[#What the second pass over the residue changed|the section on it]] for what that did to each rate.
  
 Two numbers per verdict, because there are two different questions: Two numbers per verdict, because there are two different questions:
  
-  * **domain-level** — of the domains this list names an owner for, what share does it name correctly? Every entry weighs the same. +  * **domain-level** — of the domains this list names an owner for, what share does it name correctly? Not a raw proportion: the sample is stratified 24/12/12/12 over prevalence quartiles, so each stratum is weighted back to its share of the list's coverage. **The raw counts are 48 of 54, 42 of 56 and 40 of 56** — divide those and you get 88.9 / 75.0 / 71.4, which is why they do not equal the 89.2 / 73.9 / 69.2 in the table. 
-  * **encounter-weighted** — when a crawl resolves a third-party request through this list, what share of //resolutions// are right? Weighted by Tracker Radar prevalence — its own per-domain figure for the share of the sites it crawls that request the domain, so ''gstatic.com'' at 0.40 is met on 40% of them. This is the number a paper's attribution error depends on. It is dominated by a handful of very common domains — ''gstatic.com'' alone carries 39.4% of webXray's — which is why its intervals are three times wider than the domain-level ones.+  * **encounter-weighted** — when a crawl resolves a third-party request through this list, what share of //resolutions// are right? Weighted by Tracker Radar prevalence — its own per-domain figure for the share of the sites it crawls that request the domain, so ''gstatic.com'' at 0.40 is met on 40% of them. This is the number a paper's attribution error depends on. It is dominated by a handful of very common domains — ''gstatic.com'' alone carries 38.8% of webXray's — which is why its intervals are three times wider than the domain-level ones.
  
 ^ List ^ Entries settled, of 60 drawn ^ Names today's owner (domain-level) ^ 95% CI ^ Encounter-weighted ^ 95% CI ^ ^ List ^ Entries settled, of 60 drawn ^ Names today's owner (domain-level) ^ 95% CI ^ Encounter-weighted ^ 95% CI ^
-| webXray | 49 | 69.9% | 55.5–83.1% | **93.9%** | 80.9–98.8% | +| webXray | 56 | 69.2% | 56.1–81.4% | **93.8%** | 80.9–98.6% | 
-| Tracker Radar | 47 | 73.7% | 60.8–85.8% | 68.6% | 29.3–97.0% | +| Tracker Radar | 56 | 73.9% | 61.6–85.3% | 74.3% | 41.1–97.8% | 
-| Disconnect | 42 | **94.4%** | 86.9–100.0% | 83.1% | 53.7–100.0% |+| Disconnect | 54 | **89.2%** | 80.5–96.8% | 82.8% | 53.8–100.0% |
  
-Counting //granularity// (right company, wrong level of the corporate tree) as acceptable rather than wrong — which is the right choice if your unit of analysis is "which company", not "which legal entity" — the domain-level figures become webXray 80.2% (67.7–90.8), Tracker Radar 76.5% (63.5–88.0), Disconnect 98.8% (96.2–100.0).+<WRAP important> 
 +**30 of the 525 verdicts were changed after the adjudicators returned them, and the direction is overwhelmingly favourable to the lists.** 29 on 2026-09-05: 14 landed on ''current'', 2 on ''granularity'', 1 on ''stale'' and 9 on ''unknown'' (which removes the row from every rate); one more on 2026-09-11, ''i.ua'' from ''error'' to ''current'', which also favours a list. Most of that is retiring a verdict the brief should never have offered, but the net effect is to move rates **up**, and every figure in the table above should be discounted accordingly. The full list with reasons is on the provenance pages. 
 +</WRAP> 
 + 
 +Counting //granularity// (right company, wrong level of the corporate tree) as acceptable rather than wrong — which is the right choice if your unit of analysis is "which company", not "which legal entity" — the domain-level figures become webXray 78.0% (65.8–88.8), Tracker Radar 76.4% (64.6–87.5), Disconnect 96.8% (91.5–100.0).
  
 <WRAP tip> <WRAP tip>
Line 204: Line 209:
 Four readings, each of which needs the random sample and could not come from the disagreement table: Four readings, each of which needs the random sample and could not come from the disagreement table:
  
-  * **//error// — a name that was never the owner — is rare in all three lists.** Zero of webXray's 49 settled entries and zero of Disconnect's 42; the only one in Tracker Radar's 47 is ''elfsight.com'', attributed to "Vladimir Fedotov", who is Elfsight's co-founder and not the owning entity (''elfsight.com/terms-of-service/'' names **Elfsight, SL**). A zero cell's bootstrap interval is [0,0] and asserts nothing; the usable statement is the rule-of-three bound — below **6.1%** for webXray and **7.1%** for Disconnect at 95% confidence. Almost all list error is **staleness**, not invention. +  * **//error// — a name that was never the owner — is uncommon, and it is no longer absent from two of the three lists.** Zero of webXray's 56 settled entries, and the rule-of-three bound on that zero is below **5.4%** at 95% confidence. Tracker Radar has **3 of 56**: ''elfsight.com'' attributed to "Vladimir Fedotov", Elfsight's co-founder rather than the owning entity (''elfsight.com/terms-of-service/'' names **Elfsight, SL**); ''cnevids.com'' attributed to the law firm "Sabin, Bermant & Gould LLP", which appears in the record as the registrant's counsel and never operated the domain (''player.cnevids.com'' is Condé Nast Entertainment's own player); and ''cedscdn.it'' attributed to "The Trustico Group Ltd", a UK certificate reseller, where the ''.it'' registry names **CED DIGITAL & SERVIZI Srl**. Disconnect has **1 of 54**: ''km0trk.com'', whose entry is the bare domain string, where Good On You Pty Ltd's own privacy policy names ''www.km0trk.com'' as its own. That one is a **scoring decision, not an attribution blunder like the other three**, and the rule separating it from ''i.ua'' — where Disconnect's bare-looking "I.UA" was scored //current// — is whether the label is a brand the operator's own documents use for the property. "I.UA" is (the user agreement says "порталу I.UA"); "km0trk.com" is not. Reasonable people would score these the other way, and the entry-shape census below counts the whole class without needing a verdict at all. **Three of the four are new**: ''elfsight.com'' was already there, and ''cnevids.com'', ''cedscdn.it'' and ''km0trk.com'' came out of the 2026-09-11 second pass, because an unsettled row is scored against nobody. Staleness is still the dominant failure, but "error is rare" was partly an artefact of what could not be checked. 
-  * **Tracker Radar is the list whose encounter-weighted accuracy is worst (68.6%), despite being the broadest — and partly because the three rates are not strictly on one axis.** A brand name survives an acquisition and a legal-entity name does not: for ''crwdcntrl.net'', "Lotame" scores current and "Lotame Solutions, Inc." scores stale, on one fact about one company. A list that carries legal names is therefore penalised by this measure for being more precise. Tracker Radar regenerates monthly but carries legal-entity names, and legal names lag renames: ''zoominfo.com'' is still "Zoom Information, Inc.", ''mountain.com'' "Mountain Digital, Inc." for MNTN, ''20min.ch'' "Tamedia AG" where the group renamed to TX Group, ''tvtime.com'' "Whipclip". +  * **Tracker Radar's encounter-weighted accuracy (74.3%) is the lowest of the three, despite its being the broadest — and partly because the three rates are not strictly on one axis.** A brand name survives an acquisition and a legal-entity name does not: for ''crwdcntrl.net'', "Lotame" scores current and "Lotame Solutions, Inc." scores stale, on one fact about one company. A list that carries legal names is therefore penalised by this measure for being more precise. Tracker Radar regenerates monthly but carries legal-entity names, and legal names lag renames: ''zoominfo.com'' is still "Zoom Information, Inc.", ''mountain.com'' "Mountain Digital, Inc." for MNTN, ''20min.ch'' "Tamedia AG" where the group renamed to TX Group, ''tvtime.com'' "Whipclip". 
-  * **The encounter-weighted column is concentrated enough that one domain moves it.** ''gstatic.com'' alone carries **39.4%** of webXray's encounter-weighted estimate, and webXray's top three drawn domains carry 53.1%; had ''gstatic.com'' been stale, webXray's 93.9% would read 54.4%. The equivalent heaviest rows are ''omtrdc.net'' for Tracker Radar (24.9%, and 68.6% would read 43.7%) and ''id5-sync.com'' for Disconnect (16.7%, 83.1% → 66.4%). That is not a defect in the estimator — it is what "weighted by how often you meet the domain" means when prevalence spans five orders of magnitude — but it is why the encounter-weighted column should be read as an ordering rather than as a number. +  * **The encounter-weighted column is concentrated enough that one domain moves it.** ''gstatic.com'' alone carries **38.8%** of webXray's encounter-weighted estimate, and webXray's top three drawn domains carry 52.2%; had ''gstatic.com'' been stale, webXray's 93.8% would read 55.0%. The equivalent heaviest rows are ''omtrdc.net'' for Tracker Radar (20.0%, and 74.3% would read 54.3%) and ''id5-sync.com'' for Disconnect (16.6%, 82.8% → 66.2%). That is not a defect in the estimator — it is what "weighted by how often you meet the domain" means when prevalence spans five orders of magnitude — but it is why the encounter-weighted column supports **neither a number nor an ordering**: all three intervals overlap every other, and no pairwise test is run on them. It is here because it is the quantity a paper's attribution error actually depends on, not because this sample can pin it down. 
-  * **A third of the sample could not be settled at all**, and the share rises toward the tail: 11, 13 and 18 of each 60 drawn rows. Those are domains that serve nothing, have no legal document and no newsroom, and only third-party aggregators to go on. Every rate above is therefore **conditional on the entry being adjudicable**, which is a real limit, not a footnote. And the shares are **not** equal — 18%, 22% and **30%** of each list's draws — so if unadjudicable entries are worse than adjudicable ones, the list flattered most is **Disconnect**, the one this page recommends. The worst case, every unresolved entry being stale, gives 58.3% / 58.3% / 65.0%: it narrows the ordering's margin without reversing it.+  * **What could not be settled has shrunk from a quarter of the sample to a fourteenth, and settling it moved the answers.** After the second pass, 14 of the 180 draws are unresolved (7.8%), over **12 distinct domains of the 175** — down from 42 draws and 40 domains. Per list it is 4, 4 and 6 of each 60, i.e. 6.7% / 6.7% / **10.0%**, where it was 18% / 22% / 30%. Every rate above is still **conditional on the entry being adjudicable**, but that condition now binds on a seventh of what it bound on. The worst case, every remaining unresolved entry being stale, gives 66.7% / 70.0% / **80.0%** where it gave 58.3% / 58.3% / 65.0%. What is left is genuinely hard: dead domains with a privacy-proxied registrant, no archived legal page and, in four cases, nothing but tracking endpoints ever captured.
  
 <WRAP important> <WRAP important>
-**Disconnect's lead over webXray is a difference this sample can establish; its lead over Tracker Radar is not, and neither is Tracker Radar's over webXray.** On the eligible rows Fisher's exact test gives **p = 0.014** two-sided for Disconnect against webXray (39 of 42 against 35 of 49). That is the claim to make, rather than pointing at the gap between the intervals: a percentile bootstrap on 42 rows undercovers at the top, which is why one endpoint reads 100.0. webXray against //Tracker Radar// domain-level, 69.9 against 73.7, is by contrast **not** a difference this sample can establish (p = 0.82).+**Only one of the three pairwise differences survives a correction for having tested all three.** Fisher's exact on the eligible raw counts, two-sided:
  
-Two further limits on the 69.9%, both of which cut against reading it as a rehabilitation of webXray. First, those 49 settled entries come from the **664** webXray domains that are still in a 2026 crawl frame at all — **a fifth of its 3,215 domain→owner pairs**. A domain still requested in 2026 is a survivor, and survivors are where "current" concentrates, so the rate is over the surviving fifth and not over the file. Second, seven domains were adjudicated in both the 2026-08-17 and the 2026-09-05 sittings against a **bit-identical** copy of the file; of the five settled in both, webXray's verdict differs in **four**, every one towards //current//, and the difference is a change of scoring convention (which level of the tree, and whether the acquiring group or the operating subsidiary is "the owner") rather than a change of sample.+^ Pair ^ Counting ''current'' only ^ ^ ^ Counting ''current+granularity'' ^ ^ ^ 
 +^ ^ Eligible rows ^ //p// ^ //p// before ^ Eligible rows ^ //p// ^ //p// before ^ 
 +| Disconnect vs webXray | 48 of 54 against 40 of 56 | 0.031 | **0.014** | 52 of 54 against 44 of 56 | **0.008** | **0.010** | 
 +| Disconnect vs Tracker Radar | 48 of 54 against 42 of 56 | 0.083 | 0.025 | 52 of 54 against 43 of 56 | **0.004** | **0.004** | 
 +| Tracker Radar vs webXray | 42 of 56 against 40 of 56 | 0.83 | 0.82 | 43 of 56 against 44 of 56 | 1.00 | 0.81 | 
 + 
 +Three tests on one sample, so the threshold is 0.0167 rather than 0.05. **Which comparisons survive it depends entirely on whether //granularity// counts as right**, and that is a choice about your unit of analysis, not about the data: 
 + 
 +  * **If you care which //legal entity// owns the domain** (''current'' only), **none of the three survives**. Disconnect is still the highest and by the widest margin, but on 54 and 56 adjudicated rows that ordering is not established at this threshold and should be described rather than asserted. 
 +  * **If you care which //company// owns it** — the unit most papers use, and the one this page recommends two paragraphs above — **Disconnect beats both of the others**, at //p// = 0.008 and //p// = 0.004, and it did before the second pass too. 
 + 
 +Quote the p-values rather than pointing at the gap between the intervals: a percentile bootstrap on 54 rows undercovers at the top, which is why one endpoint reads 100.0. 
 + 
 +**What the second pass actually overturned is narrower than it first looks, and the narrowing matters.** Before 2026-09-11, Disconnect-versus-webXray cleared the corrected threshold on the ''current''-only metric at //p// = 0.014; it is now 0.031 and does not. **On ''current+granularity'' nothing changed**: Disconnect cleared the bar against both lists before (0.010, 0.004) and clears it after (0.008, 0.004). So the second pass cost Disconnect a //legal-entity-level// lead over a list this page calls historical, and cost it nothing at the company level. Settling 28 of the 40 previously unadjudicable rows added **7** eligible rows to webXray, **9** to Tracker Radar and **12** to Disconnect — and on the ''current''-only count, **9 of Disconnect's 12 were //current//**, against **39 of its previous 42**; its point estimate fell from 94.4% to 89.2% there, and from 98.8% to 96.8% on the looser metric. An earlier version of this page asserted that the Disconnect-versus-Tracker-Radar difference could not be established **without having computed it**; on ''current'' only it was 0.025 then and 0.083 now, and the correction is recorded on the provenance page. 
 + 
 +Two further limits on the 69.2%, both of which cut against reading it as a rehabilitation of webXray. First, those 56 settled entries come from the **664** webXray domains that are still in a 2026 crawl frame at all — **a fifth of its 3,215 domain→owner pairs**. A domain still requested in 2026 is a survivor, and survivors are where "current" concentrates, so the rate is over the surviving fifth and not over the file. Second, seven domains were adjudicated in both the 2026-08-17 and the 2026-09-05 sittings against a **bit-identical** copy of the file; of the five settled in both, webXray's verdict differs in **four**, every one towards //current//, and the difference is a change of scoring convention (which level of the tree, and whether the acquiring group or the operating subsidiary is "the owner") rather than a change of sample.
 </WRAP> </WRAP>
  
 <WRAP tip> <WRAP tip>
-**If you cite one figure from this page, cite a coverage figure, not an accuracy figure.** Coverage is a census over the whole frame with no sampling error; accuracy is 42 to 49 adjudicated rows per list and the intervals are ±13 percentage points at best. The strongest defensible claims are: no list names an owner for more than **17.2%** of the registrable third-party domains a Tracker Radar crawl sees; and among entries that exist and can be checked, **staleness rather than error** is what goes wrong.+**If you cite one figure from this page, cite a coverage figure, not an accuracy figure.** Coverage is a census over the whole frame with no sampling error; accuracy is 54 to 56 adjudicated rows per list, with intervals from ±8 (Disconnect) to ±13 (webXray) percentage points. The strongest defensible claims are: no list names an owner for more than **17.2%** of the registrable third-party domains a Tracker Radar crawl sees; and among entries that exist and can be checked, **staleness rather than error** is what goes wrong.
 </WRAP> </WRAP>
 +
 +==== What the second pass over the residue changed ====
 +
 +The 2026-09-05 sample left **40 of its 175 domains unadjudicable**, and every rate it produced was conditional on that. Because the share rose toward the prevalence tail and was worst for the list this page recommends, the obvious worry was that the unsettled rows were systematically worse and all three rates were flattered — which nothing inside that sample could test. On **2026-09-11** those 40 rows, and only those 40, were re-adjudicated — again **by Sonnet sub-agents working to a published brief, one rater per domain, with no inter-rater agreement measured for this pass** — against harder sourcing: **national company registers searched by company number**, the **registrar's own port-43 WHOIS** rather than RDAP (which returns a thin registry record with no registrant for ''.com'' and ''.net''), and **archived captures of each domain's own legal pages**. 28 of the 40 settled.
 +
 +^ ^ Before ^ After ^
 +| Distinct domains unresolved, of 175 | 40 (22.9%) | **12 (6.9%)** |
 +| Draws unresolved, of 180 | 42 (23.3%) | 14 (7.8%) |
 +| Eligible rows carrying the rates | 49 / 47 / 42 | **56 / 56 / 54** |
 +| Disconnect, domain-level //current// | 94.4% | **89.2%** |
 +| Tracker Radar, domain-level //current// | 73.7% | 73.9% |
 +| webXray, domain-level //current// | 69.9% | 69.2% |
 +| Tracker Radar, encounter-weighted //current// | 68.6% | **74.3%** |
 +| ''error'' verdicts found | 1 (Tracker Radar) | **4** (3 Tracker Radar, 1 Disconnect) |
 +| Pairwise comparisons surviving Bonferroni | 1 of 3 | **0 of 3** |
 +
 +The **largest single movement of the pass is not in the domain-level column at all**: Tracker Radar's encounter-weighted rate rose 68.6% → 74.3%, almost all of it one recovered row, ''spot.im'' at prevalence 0.011. That is the column the page calls "the number a paper's attribution error depends on", and it moved five times as far as any domain-level rate.
 +
 +**The formal test does not confirm the worry, and the point estimates do not refute it.** Comparing the recovered rows against the rows already in each rate — ''current''+''granularity'' against ''stale''+''error'', Fisher's exact two-sided — gives //p// = 0.64 for webXray (5 of 7 recovered good, against 39 of 49), //p// = 1.00 for Tracker Radar (7 of 9, against 36 of 47) and //p// = 0.40 for Disconnect (11 of 12, against 41 of 42). None is close to significant. But 7, 9 and 12 rows have almost no power to detect anything, and Disconnect's estimate still fell 5.2 points, which is the direction the worry predicted for the list the worry named. And the answer depends on the metric: counting ''granularity'' as acceptable, Disconnect's recovered rows are 11 of 12 against 41 of 42 and its estimate falls 2.0 points rather than 5.2. **The honest summary is that nothing here establishes that the tail was worse for any list**, that Disconnect's point estimate moved in the direction the worry predicted on the stricter metric and barely at all on the looser one, and that none of the three tests has the power to tell those apart.
 +
 +<WRAP important>
 +**The second pass used a bar the first pass did not, and you should know where it is weaker.** Three source kinds are new: ''domain-register'' (an un-redacted ''Registrant Organization'' in the registry or registrar WHOIS), ''archived-legal-doc'' (a Wayback capture of the domain's own legal page), and ''tls-san'' (another organisation's certificate on this host — corroboration only, never used alone). Of the 28 recovered rows, **9 rest on ''domain-register'' and 2 on ''archived-legal-doc''**; the other 17 clear the original bar.
 +
 +''domain-register'' is genuinely weaker: the field is asserted by the registrant and validated by nobody. This pass produced its own counter-example. ''i.ua'''s registry record names **Digital Ventures LLC**; the portal's own user agreement names a **different** company as its administration, ТОВ «КЕПРЕЙТ ПАРТНЕРС», register code 33500955. **On that domain the registrant of record is not the operator**, and the row was re-sourced to the legal document. Dropping every remaining recovered row that rests on an //uncorroborated// registrant organisation — ''cratecamera.com'' and ''sa-as.com'' — moves nothing: webXray +0.0, Tracker Radar −0.6, Disconnect +0.0 percentage points.
 +
 +**The comparison that actually answers the objection is the same-bar one**, and it is the one to quote if you distrust the new sources: drop **all 11** rows resting on a source kind the first pass would not have accepted, so that what is left is only what the 2026-09-05 bar could itself have reached. The rates barely move — webXray 69.2% (+0.0), Tracker Radar 72.1% (−1.8), Disconnect 89.0% (−0.2) — and the ordering, the significance verdicts and the residue conclusion are all unchanged. **So the second pass's result does not depend on the bar it widened.** Under the //other// reading of ''i.ua'' — that "I.UA" names no company and the row is an ''error'' rather than ''current'', the reading this pass argued against — Disconnect is 88.6% rather than 89.2%. All three runs are in section E3 of the report script's output.
 +</WRAP>
 +
 +**What is left is a different kind of hard.** The 12 domains still unresolved — ''1rx.io'', ''agkn.com'', ''cdnbasket.net'', ''collective-media.net'', ''contentabc.com'', ''hqseek.com'', ''htplayground.com'', ''mapixl.com'', ''marphezis.com'', ''mmstat.com'', ''stat-track.com'', ''stripst.com'' — are not domains where the register was not tried. **Ten of the twelve** are domains with a privacy-proxied registrant, no live site, no archived legal page, and in several cases nothing in the Archive but tracking endpoints, so there is no document to read at any date. Two have a further obstacle worth naming for anyone attempting this: ''mmstat.com'' would be settled by China's ICP registry, which serves a JavaScript challenge no fetch here could pass, and ''agkn.com'' by TransUnion's own pages, which return 403 to everything that is not a browser session.
 +
 +**The other two are limits of this pass rather than of the evidence, and a person could still close them.** ''hqseek.com'' redirects to a live adult site and the adjudicator declined to follow it on content policy — so the row is unresolved because a rater stopped, not because nothing is there; its note also records that Tracker Radar's claim, a personal name, matches nothing the domain's own pages ever surfaced, which would be an ''error'' rather than an ''unknown'' if it were run down. ''stripst.com'''s operator pages returned HTTP 406 to every non-browser fetch — a transport failure, not an absence. Both would be worth a browser session; neither was given one here.
 +
 +**The residue is also not a random slice of the sample, and the way it is skewed matters for the bound below.** Three of Disconnect's six remaining rows are entries where Disconnect's own value carries no company name a register could be searched for — ''cdnbasket.net'' and ''htplayground.com'' are the bare domain string, and ''stat-track.com'''s "StackTrack" is not a findable company. That is the same class as ''km0trk.com'', the one entry Disconnect was scored ''error'' on, so the worst case is more likely to bind on Disconnect than the even split of the residue across the three lists would suggest. **The remaining residue is a lower bound on what harder sourcing can reach, not a claim that these domains have no owner.** Every attempted route per row is printed on [[provenance:design:ownership_resolution]].
  
 ==== Inter-rater agreement, measured rather than assumed ==== ==== Inter-rater agreement, measured rather than assumed ====
  
-A random 20% of the 2026-09-05 sample (35 of 175 domains) was **re-adjudicated on 2026-09-11 by a different model against the identical brief and the identical evidence blocks**. On the rows the brief asks about, the two raters agree on 58.5% of 65 verdicts, Cohen kappa **+0.400** (95% CI +0.241 to +0.551); per list, +0.027 for webXray on 12 rows, +0.478 for Tracker Radar on 30 and +0.460 for Disconnect on 23.+A random 20% of the 2026-09-05 sample (35 of 175 domains) was **re-adjudicated on 2026-09-11 by a different model against the identical brief and the identical evidence blocks**. On the rows the brief asks about, the two raters agree on 58.5% of 65 verdicts, Cohen kappa **+0.400** (95% CI +0.241 to +0.551); per list, **+0.027 for webXray on 12 rows** — chance agreement — +0.478 for Tracker Radar on 30 and +0.460 for Disconnect on 23. So for webXray specifically the two raters agreed no better than chance, and its 69.2% carries rater uncertainty on top of the sampling interval already quoted. That kappa was measured on the 2026-09-05 rows; **no inter-rater agreement was measured for the second pass over the residue**, so the 28 rows it recovered carry no agreement figure at all.
  
 **About two-thirds of the disagreement is one rater finding a primary source where the other did not, rather than two readings of the same evidence** — restricted to rows both settled, the pooled kappa is **+0.680**. If you are planning your own adjudication, that is the transferable result: **specify how hard the adjudicator must look, not just what the labels mean.** Rater 2 was //more// favourable to webXray than rater 1 (70.0% of its adjudicable rows current against 50.0%), so the disagreement does not point at the published rates being too kind. See also [[Statistics:Interrater agreement]] for what a kappa of that size means and what it does not. **About two-thirds of the disagreement is one rater finding a primary source where the other did not, rather than two readings of the same evidence** — restricted to rows both settled, the pooled kappa is **+0.680**. If you are planning your own adjudication, that is the transferable result: **specify how hard the adjudicator must look, not just what the labels mean.** Rater 2 was //more// favourable to webXray than rater 1 (70.0% of its adjudicable rows current against 50.0%), so the disagreement does not point at the published rates being too kind. See also [[Statistics:Interrater agreement]] for what a kappa of that size means and what it does not.
Line 233: Line 286:
 Dated, because a list of what the literature //did// is not advice about what to do: Dated, because a list of what the literature //did// is not advice about what to do:
  
-^ Source ^ Status — repository state re-checked 2026-09-05, accuracy figures measured 2026-09-05 ^ Use it when ^ +^ Source ^ Status — repository state re-checked **2026-09-11**, accuracy figures drawn 2026-09-05 and re-adjudicated over the residue **2026-09-11** ^ Use it when ^ 
-| **Disconnect ''entities.json''** | **current**; last commit 2026-08-28; CC BY-NC-SA 4.0; feeds Firefox's tracking protection via Mozilla's ''shavar-prod-lists''((''mozilla-services/shavar-prod-lists'' README, checked 2026-08-17: "Firefox's Enhanced Tracking Protection features rely on lists of trackers maintained by Disconnect… Mozilla does not maintain these lists", and ''disconnect-blacklist.json'' is "a version controlled copy of Disconnect's list of trackers".)) and, per Disconnect, Microsoft Edge as well | you want the **most current** owner name, and you can live with a sparser list and no hierarchy. Note that ''properties'' (5,843 domains) and ''resources'' (4,148) are different claims and 2,007 domains appear only in the second; the schema is not documented, so state which key you read. Do not quote Disconnect's own headline "14,332 verified domains and entity mappings" as the size of ''entities.json'' — that file holds 7,850 domains; count the file you read.((''https://disconnect.me/trackerprotection'', checked 2026-08-17, states "14,332 Verified domains and entity mappings" and that the lists "power Microsoft Edge, Mozilla Firefox, other partners". The 7,850 figure is measured from ''entities.json'' by ''owner_dbs.py''.)) | +| **Disconnect ''entities.json''** | **current**, but less current than the repository looks: the repo's HEAD is 2026-09-05 and ''entities.json'' has not changed since **2026-08-07**, five weeks earlier — the 2026-09-05 commit touched ''services.json''. The file is byte-identical (412,191 bytes) at 2026-08-07, at 2026-09-05 and on ''master'' today.((''api.github.com/repos/disconnectme/disconnect-tracking-protection/commits?path=entities.json'' → ''ab6ff5a3a8'', 2026-08-07T17:07:16Z, "Latest list updates."; the repo's HEAD ''4b592c288a'', 2026-09-05T20:21:14Z, "Recent list updates.", does not touch it. The three raw files fetched and hashed, 2026-09-11. **Query the path, not the repo** — a repository that commits every few days can carry a file that has not moved in over a month.)) CC BY-NC-SA 4.0; feeds Firefox's tracking protection via Mozilla's ''shavar-prod-lists''((''mozilla-services/shavar-prod-lists'' README, checked 2026-08-17: "Firefox's Enhanced Tracking Protection features rely on lists of trackers maintained by Disconnect… Mozilla does not maintain these lists", and ''disconnect-blacklist.json'' is "a version controlled copy of Disconnect's list of trackers".)) and, per Disconnect, Microsoft Edge as well | you want the **most current** owner name, and you can live with a sparser list and no hierarchy. Note that ''properties'' (5,843 domains) and ''resources'' (4,148) are different claims and 2,007 domains appear only in the second; the schema is not documented, so state which key you read. Do not quote Disconnect's own headline "14,332 verified domains and entity mappings" as the size of ''entities.json'' — that file holds 7,850 domains; count the file you read.((''https://disconnect.me/trackerprotection'', checked 2026-08-17, states "14,332 Verified domains and entity mappings" and that the lists "power Microsoft Edge, Mozilla Firefox, other partners". The 7,850 figure is measured from ''entities.json'' by ''owner_dbs.py''.)) | 
-| **DuckDuckGo Tracker Radar** ''entity_map.json'' / ''domain_map.json'' | **current**; regenerated monthly, last commit on ''main'' 2026-08-28 — the commit the newest release tag ''2026.08.28'' points at. The repository's ''pushed_at'' field reads later than that because unmerged automation branches count towards it, so read the branch, not the field; CC BY-NC-SA 4.0 | you want the **broadest** coverage, prevalence weights, or per-domain categories and fingerprinting scores in the same dataset. Expect legal-entity names, and expect renames to lag. |+| **DuckDuckGo Tracker Radar** ''entity_map.json'' / ''domain_map.json'' | **current**; regenerated monthly, last commit on ''main'' 2026-08-28 — the commit the newest release tag ''2026.08.28'' points at. The repository's ''pushed_at'' field reads later than that (2026-09-02 on 2026-09-11) because unmerged automation branches count towards it, so read the branch, not the field; CC BY-NC-SA 4.0 | you want the **broadest** coverage, prevalence weights, or per-domain categories and fingerprinting scores in the same dataset. Expect legal-entity names, and expect renames to lag. |
 | **Ghostery ''trackerdb''** / WhoTracks.me | **current**; ''trackerdb'' last commit 2026-09-01, **CC BY-NC-SA 4.0** (its ''package.json'' says so explicitly; GitHub reports no SPDX id, so do not trust the API field); WhoTracks.me data repo last commit 2026-09-02 ("August update"); the site now redirects to ''ghostery.com/whotracksme'' | you want an ''organizations'' + ''patterns'' model where one company can carry several independently categorised behaviours (Google Analytics separate from Google Tag Manager), or the {[karaj2018whotracksme]} longitudinal data | | **Ghostery ''trackerdb''** / WhoTracks.me | **current**; ''trackerdb'' last commit 2026-09-01, **CC BY-NC-SA 4.0** (its ''package.json'' says so explicitly; GitHub reports no SPDX id, so do not trust the API field); WhoTracks.me data repo last commit 2026-09-02 ("August update"); the site now redirects to ''ghostery.com/whotracksme'' | you want an ''organizations'' + ''patterns'' model where one company can carry several independently categorised behaviours (Google Analytics separate from Google Tag Manager), or the {[karaj2018whotracksme]} longitudinal data |
-| **webXray ''domain_owners.json''** | **historical**; the file is frozen at 2021-03-04 with no update path, and the tool has had no commit since 2023. Measured accuracy on a random sample of its own coverage: 69.9% current (55.5–83.1), 19.8% stale — but it covers only 2.1% of the frame | reproducing or extending a pre-2022 result that used it, or you specifically need the ''parent_id'' tree or the per-language policy URLs and will re-verify each owner you rely on. Not the list to start a new measurement with: Disconnect is 94.4% current on the same test and Tracker Radar covers eight times as many domains |+| **webXray ''domain_owners.json''** | **historical**; the file is frozen at 2021-03-04 with no update path, and the tool has had no commit since 2023. Measured accuracy on a random sample of its own coverage: 69.2% current (56.1–81.4), 22.0% stale — but it covers only 2.1% of the frame | reproducing or extending a pre-2022 result that used it, or you specifically need the ''parent_id'' tree or the per-language policy URLs and will re-verify each owner you rely on. Not the list to start a new measurement with: Disconnect is 89.2% current on the same test and Tracker Radar covers eight times as many domains |
 | **WHOIS / RDAP** | current, but see the box below | as a **fallback** for domains no list covers, and only with privacy-proxy filtering | | **WHOIS / RDAP** | current, but see the box below | as a **fallback** for domains no list covers, and only with privacy-proxy filtering |
 | **TLS certificates, DNS/SOA, CNAME chains** | current | first-party CDN and sibling-domain detection, which the ownership lists are worst at: {[steffens2021_blockparty]} needed it because those lists "frequently miss connections among two hostnames, e.g., ''twitch.tv'' and ''twitchcdn.net''" | | **TLS certificates, DNS/SOA, CNAME chains** | current | first-party CDN and sibling-domain detection, which the ownership lists are worst at: {[steffens2021_blockparty]} needed it because those lists "frequently miss connections among two hostnames, e.g., ''twitch.tv'' and ''twitchcdn.net''" |
 | **Crunchbase** | live but **paid**: the v4 API returns HTTP 401 without a key | you have institutional access; used this way by {[yang2020_comparative]} and {[cui2023_poligraph]} | | **Crunchbase** | live but **paid**: the v4 API returns HTTP 401 without a key | you have institutional access; used this way by {[yang2020_comparative]} and {[cui2023_poligraph]} |
-| **LLM-based entity resolution** | **emerging, and not yet at the domain layer.** In this corpus the 2025 work is at the network layer: {[selmo2025_borges]} maps AS numbers to organisations with an LLM and releases prompts and code; {[gouda2025_prefix2org]} maps BGP prefixes. No corpus paper applies this to third-party //domain// ownership | you are willing to build and validate it yourself. The problem statement transfers directly; the validation burden does too |+| **LLM-based entity resolution** | **emerging, and only just reaching the domain layer.** In this corpus the 2025 work is at the **network** layer: {[selmo2025_borges]} maps AS numbers to organisations with an LLM and releases prompts and code; {[gouda2025_prefix2org]} maps BGP prefixes. **No corpus paper applies this to third-party //domain// ownership** — but outside the seven venues it has started, and the first result is a warning rather than an endorsement: a 2026-06 preprint evaluates four models on domain-to-brand attribution over 36 heavily-phished brands and finds they enumerate a brand's domains at up to 82% precision from memory alone yet **fail at ownership verification without external tools, macro F1 at most 0.37**, rising by up to 0.65 with WHOIS lookup.((//Can LLMs Reason About Brand Ownership? An Empirical Study of Domain Attribution Intelligence//, Mashood and Nabeel, arXiv:2606.20868v1, submitted 2026-06-18, abstract fetched 2026-09-11. Its task is phishing and squatting defence — is this domain the brand's own? — not third-party tracker attribution, and 36 brands is not a coverage claim. It is cited here as evidence that the domain layer is no longer untouched, not as a method to adopt.)) | you are willing to build and validate it yourself. The problem statement transfers directly; the validation burden does too — and on the one published attempt, an unaugmented model is worse than the frozen list this page calls historical |
  
 <WRAP important> <WRAP important>
Line 293: Line 346:
 | webXray | 15 | 0.3% | 13 | 1.2% | **yes, all 15** | | webXray | 15 | 0.3% | 13 | 1.2% | **yes, all 15** |
  
-The Public Suffix List is **not** in the 136: it answers "what is the registrable domain", not "whose is it", and adding it takes the union to 221 on a resource crawling papers touch for unrelated reasons. Every probe was run at two widths and the wide one printed, because a narrow probe returns a confident wrong number: plain ''%%/disconnect/i%%'' matches **700** papers and almost none of them mean the list — the CSP literature uses the word as a technical term — which is why the published row is the tightened list-sense pattern giving 74. The residues are on [[provenance:design:ownership_resolution|the provenance page]].+The Public Suffix List is **not** in the 136: it answers "what is the registrable domain", not "whose is it", and adding it takes the union to 221 on a resource crawling papers touch for unrelated reasons. The Disconnect row is the one to be careful with — plain ''%%/disconnect/i%%'' matches **700** papers and almost none mean the list, because the CSP literature uses the word as a technical term, so the published 74 comes from a tightened pattern. Every probe was run at two widths and both are printed on [[provenance:design:ownership_resolution|the provenance page]], with what the tightening costs.
  
-**It is not growing.** The share of each year's papers naming an ownership resource rose to about 3.5% around 2020–2022 and has sat between 2.5% and 2.7% since:+**It is not growing.** The share of each year's papers naming an ownership resource has moved between 1% and 4% since 2016 with no trend this corpus can establish — 2022 against 2023 is 19 of 546 against 19 of 719 (Fisher's exact, //p// = 0.41), and pooling 2020–2022 against 2023–2026 gives //p// = 0.11. Read the table as a level, not a curve:
  
 ^ Year ^ Corpus papers ^ Naming an ownership resource ^ Share ^ ^ Year ^ Corpus papers ^ Naming an ownership resource ^ Share ^
Line 323: Line 376:
 | NDSS | 701 | 8 | 1.1% | | NDSS | 701 | 8 | 1.1% |
  
-PETS is where this work lands, at seven times NDSS's rate. Everywhere else the shares are small, which is the honest headline: ownership resolution is a **step inside a paper**, not a paper topic, and that is precisely why most papers that do it do not say how.+PETS is where this work lands, at **more than eight times** NDSS's rate (9.6% of 510 against 1.1% of 701). Everywhere else the shares are small, which is the honest headline: ownership resolution is a **step inside a paper**, not a paper topic, and that is precisely why most papers that do it do not say how.
  
 ==== Which source the literature used, and when ==== ==== Which source the literature used, and when ====
Line 342: Line 395:
 The counts are small enough that the crossover is a signal about **direction**, not a market share, and 2025–2026 are the provisional edge. Combined with the repository and licence findings on [[Programming:Crawler:webXray]], the corpus supports "webXray's ownership list is historical and Tracker Radar's is current practice"; it does not support any claim about which is more accurate, which is what the random sample above is for. The counts are small enough that the crossover is a signal about **direction**, not a market share, and 2025–2026 are the provisional edge. Combined with the repository and licence findings on [[Programming:Crawler:webXray]], the corpus supports "webXray's ownership list is historical and Tracker Radar's is current practice"; it does not support any claim about which is more accurate, which is what the random sample above is for.
  
-**The schema sees a fraction of it.** The extraction's ''%%tools[]%%'', ''%%classification[]%%'' and ''%%otherToolsMentioned[]%%'' fields name one of the five resources in **89** papers against the full text's 136 — 82 in both, 54 full-text-only, and **7 schema-only**, all but one of them a Disconnect mention the sweep's tightened pattern deliberately excludes. That gap is not a defect in either signal: most mentions of these lists are in related work or a reference list rather than in a sentence an extractor reads as a tool. Both signals are reported and neither is called "the" population. The seven-paper residue is listed in full on the provenance page.+**Why these are full-text counts and not schema counts.** The extraction's own tool and classification fields name one of the five resources in **89** papers against the full text's 136, because most mentions sit in related work or a reference list rather than in a sentence an extractor reads as a tool. Both signals are reported; neither is "the" population. The overlap, the 54 the schema misses, and the 7 it finds that the sweep does not are itemised on [[provenance:design:ownership_resolution|the provenance page]].
  
 ==== Three findings that matter more than the counts ==== ==== Three findings that matter more than the counts ====
Line 357: Line 410:
   * Nothing on this wiki or in this corpus measures **Ghostery ''trackerdb''** against the other three. It is live, it is the fourth real option, and it is the one gap in the comparison above.   * Nothing on this wiki or in this corpus measures **Ghostery ''trackerdb''** against the other three. It is live, it is the fourth real option, and it is the one gap in the comparison above.
   * webXray's ''parent_id'' tree is the only hierarchy any of these files carries. Whether a hierarchy //built from a current list// — Tracker Radar's entities plus a corporate-structure source — would beat both flat lookup and webXray's frozen tree is unmeasured.   * webXray's ''parent_id'' tree is the only hierarchy any of these files carries. Whether a hierarchy //built from a current list// — Tracker Radar's entities plus a corporate-structure source — would beat both flat lookup and webXray's frozen tree is unmeasured.
-  * The accuracy rates here are conditional on a domain being adjudicable, and 18–30% of each list's draws were not. What those domains are, and whether they differ systematically between lists, would need a different instrument than a random sample.+  * The accuracy rates here are conditional on a domain being adjudicable. That was 18–30% of each list's draws until 2026-09-11, when a second pass with harder sourcing settled 28 of the 40 and brought it to **6.7% / 6.7% / 10.0%**; it is not zero and the 12 domains still unresolved cannot be bounded from inside the sample. Whether //those// differ systematically between lists still needs a different instrument.
 </WRAP> </WRAP>
  
Line 363: Line 416:
  
   * **Every query, every unedited script output, the fold residues, the sources rejected and the figures deliberately not published are on [[provenance:design:ownership_resolution]].** Corpus-wide selection and extraction caveats are on [[Literature:Corpus]]. The full 175-row adjudication table and the scripts behind the accuracy figures are on [[provenance:programming:crawler:webxray:random_sample]], which was written before this page was split out and keeps its id so the links in the published record still resolve.   * **Every query, every unedited script output, the fold residues, the sources rejected and the figures deliberately not published are on [[provenance:design:ownership_resolution]].** Corpus-wide selection and extraction caveats are on [[Literature:Corpus]]. The full 175-row adjudication table and the scripts behind the accuracy figures are on [[provenance:programming:crawler:webxray:random_sample]], which was written before this page was split out and keeps its id so the links in the published record still resolve.
-  * The corpus figures come from ''scripts/report_ownership_resolution.mjs'', which re-derives them independently of ''scripts/report_webxray.mjs'' and **cross-checks every overlapping row against it**; both scripts and their unedited output are on the provenance page. The live-database comparison is ''scripts/owner_dbs.py''; the 28-row hand adjudication is ''scripts/owner_adjudication.py''; the random sample is ''scripts/owner_sample.py'' (draw) and ''scripts/owner_random_sample.py'' (estimate); the inter-rater re-adjudication is ''scripts/owner_irr_kappa.py''. +  * The corpus figures come from ''scripts/report_ownership_resolution.mjs'', which re-derives them independently of ''scripts/report_webxray.mjs'' and **cross-checks every overlapping row against it**; both scripts and their unedited output are on the provenance page. The live-database comparison is ''scripts/owner_dbs.py''; the 28-row hand adjudication is ''scripts/owner_adjudication.py''; the random sample is ''scripts/owner_sample.py'' (draw) and ''scripts/owner_random_sample.py'' (estimate); the inter-rater re-adjudication is ''scripts/owner_irr_kappa.py''; the second pass over the residue is ''scripts/owner_tail_probe.sh'' (evidence), ''scripts/owner_tail_merge.py'' (gate) and ''scripts/report_tail_pass.py'' (what it changed). 
-  * **The coverage and agreement figures use a single snapshot per list, taken 2026-08-17; the random sample uses a second set taken 2026-09-05**, and two of the six files had moved between them. Two of the three lists change weekly. Re-run the scripts rather than quoting these numbers in 2027.+  * **The coverage and agreement figures use a single snapshot per list, taken 2026-08-17; the random sample uses a second set taken 2026-09-05**, and two of the six files had moved between them. Both live lists move between months — Tracker Radar regenerates monthly, Disconnect commits continuously. Re-run the scripts rather than quoting these numbers in 2027.
   * The coverage denominator is **Tracker Radar's own crawl output**. There is no neutral census of third-party domains, so this favours Tracker Radar; the page says so wherever a coverage figure appears rather than pretending otherwise.   * The coverage denominator is **Tracker Radar's own crawl output**. There is no neutral census of third-party domains, so this favours Tracker Radar; the page says so wherever a coverage figure appears rather than pretending otherwise.
   * The 28 adjudicated rows are the highest-prevalence **disagreements**, deliberately not a random sample, so they characterise the shape of disagreement and not any list's accuracy.   * The 28 adjudicated rows are the highest-prevalence **disagreements**, deliberately not a random sample, so they characterise the shape of disagreement and not any list's accuracy.
-  * The **random sample** is 60 domains per list from that list's own coverage, stratified into prevalence quartiles with allocation 24/12/12/12, seed 20260905. Its limits, in the order they matter: every rate is conditional on the entry being adjudicable, and 11–18 of each 60 were not; the frame is Tracker Radar's ''domain_summary.json'', so no list is scored on domains that crawl never saw; and the encounter-weighted column is a ratio estimator concentrated in a handful of head domains, which is why its intervals reach 68 percentage points wide. **29 of the 525 verdicts were changed after the adjudicators returned them**, and **the direction is overwhelmingly favourable to the lists** — discount accordingly. Every change is listed with its reason on the provenance pages.+  * The **random sample** is 60 domains per list from that list's own coverage, stratified into prevalence quartiles with allocation 24/12/12/12, seed 20260905. Its limits, in the order they matter: every rate is conditional on the entry being adjudicable, and after the 2026-09-11 second pass 4–6 of each 60 still are not (11–18 before it); the frame is Tracker Radar's ''domain_summary.json'', so no list is scored on domains that crawl never saw; and the encounter-weighted column is a ratio estimator concentrated in a handful of head domains, which is why its intervals reach 68 percentage points wide. **30 of the 525 verdicts were changed after the adjudicators returned them**, and **the direction is overwhelmingly favourable to the lists** — discount accordingly. Every change is listed with its reason on the provenance pages.
   * The corpus is seven venues and omits EuroS&P, ACSAC, RAID, AsiaCCS, CHI and SOUPS, so "136 papers" is a statement about those seven. Ownership resolution is also published outside them: webXray's own author placed most of his results in The BMJ, JAMA, //Communications of the ACM// and //New Media and Society//, none of which this corpus can see.   * The corpus is seven venues and omits EuroS&P, ACSAC, RAID, AsiaCCS, CHI and SOUPS, so "136 papers" is a statement about those seven. Ownership resolution is also published outside them: webXray's own author placed most of his results in The BMJ, JAMA, //Communications of the ACM// and //New Media and Society//, none of which this corpus can see.
   * Full-text counts are **paper** counts over ''paper.cols.txt'' with whitespace collapsed, so a term broken across a PDF column boundary still matches. They are mentions, not verified uses, except where the table says hand-verified.   * Full-text counts are **paper** counts over ''paper.cols.txt'' with whitespace collapsed, so a term broken across a PDF column boundary still matches. They are mentions, not verified uses, except where the table says hand-verified.
design/ownership_resolution.1789149009.txt.gz · Last modified: by karel.kubicek.claude