Differences

This shows you the differences between two versions of the page.

--- provenance:privacy:ads_txt [2026/09/02 17:34] – Embed the re-run external checks (DOI line). Authored by Claude karel.kubicek.claude
+++ provenance:privacy:ads_txt [2026/09/22 07:48] (current) – Fifth (generic, fable) review pass logged with 17 rows, each accepted or rejected with a reason. Fixed here: the retracted 'none reports what fraction' universal that survived under What could not be established; header-bidder-expert called maintained bes karel.kubicek.claude
@@ Line 13: / Line 13: @@
 | Models | Page, scripts and this log: Claude (Fable 5.1). Paper reading: two ''sonnet'' sub-agents, which fanned out one child per paper (13 child reports came back); every figure and quote they supplied was re-checked mechanically against ''paper.cols.txt'' (see //Quotes//). Review layer: three ''sonnet'' focused passes on the first revision, then one ''fable'' generic pass on the second; all four reported and were acted on (see //Review//) |
 | Scripts added | ''scripts/report_ads_txt.mjs'' (+ ''-output.txt''), ''scripts/adstxt_fulltext_probe.mjs'' (the first, wider probe; superseded by the report), ''scripts/adstxt_quotecheck.mjs'' (+ ''-output.txt''), ''scripts/adstxt_external_checks.sh'' (+ ''-output.txt''), ''scripts/bib_additions_ads_txt.bib'', ''scripts/build_provenance_ads_txt.py'', ''scripts/provenance_ads_txt_prose.txt'' |
-| Guards run before publication | ''check_wrap.mjs'' (OK), ''check_tables.mjs'' (OK), ''check_attributions.mjs'' table mode (0 matches — the page cites in prose) and ''--prose'' mode (5 checked, 4 SUSPECT, all false positives: the words //Exchange//, //CPM//, //CPM//, //Since// precede a citekey and are not names), ''check_page_numbers.mjs'' windowed on //Use in Publications// against the report output (OK), and whole-page against the concatenation of report + quote-check + external-check outputs (OK after the live ''sellers.json'' figures were moved into the external-check script) |
+| Guards run before publication | ''check_wrap.mjs'' (OK), ''check_tables.mjs'' (OK), ''check_attributions.mjs'' table mode (0 matches — the page cites in prose) and ''%%--prose%%'' mode (5 checked, 4 SUSPECT, all false positives: the words //Exchange//, //CPM//, //CPM//, //Since// precede a citekey and are not names), ''check_page_numbers.mjs'' windowed on //Use in Publications// against the report output (OK), and whole-page against the concatenation of report + quote-check + external-check outputs (OK after the live ''sellers.json'' figures were moved into the external-check script) |
-| Write path | ''node scripts/dw.mjs put'' (JSON-RPC) with ''--if-rev'' on every save |
+| Write path | ''node scripts/dw.mjs put'' (JSON-RPC) with ''%%--if-rev%%'' on every save |
 | Accidental exposure | None. Credentials stayed in ''.env''. Unauthenticated fetches only: IAB Tech Lab, GitHub API, Prebid docs, Google/PubMatic/OpenX ''sellers.json'', arXiv, EasyList, the papers' artifact URLs |
@@ Line 83: / Line 83: @@
 | Q16 | ''vantage.locations'' folded through ''geo.mjs'' | 16 | 11 state a location; United States 6, Europe 3, Germany 3, Spain 2, then singletons; 1 unmapped ("same country") | yes (11 of 16; 6 US) |
 | Q17 | ''detection[].prevalence'' on tuples naming an observable | 16 | printed per paper in the output; the page quotes the papers' own denominators, not this field | indirectly |
-| Q18 | Authors' own artifact links (''belongsToAuthors'') | 16 | 11 papers flag at least one; all URLs re-fetched (see //External//) | yes, tools table |
+| Q18 | Authors' own artifact links (''belongsToAuthors'') | 16 | **10** papers flag at least one, counted from the PASS C artifact-links block below (this cell said 11 until 2026-09-22). Every URL in the external-check list was re-fetched, but {[liu2025_fingerprinting]}'s OneDrive dataset link is not in that list — so //all URLs re-fetched// was true of the list and not of the papers | yes, tools table |
 | Q19 | The S&P 2024 paper: venue index / screening label / extraction / full text | 1 | present / all fields false or empty / absent / absent | yes, twice |
@@ Line 119: / Line 119: @@
   * **Vantage locations** (Q16): ''geo.mjs''. 1 unmapped: "same country" ({[papadopoulos2017_paying]}; the paper says Spain elsewhere and the folded table has Spain 2 from its ''vantage'' tuples that do say so).
   * **Ad-system names are not folded on this page** — the page ranks no ad systems. Where a paper's ranking is quoted (Google 80.8%, AppNexus 52.5%, Index Exchange 49%) it is the paper's own, on the paper's own domain strings.
-  * **Term probe** — the regexes above are the fold. Width decisions: ''RTB'' case-sensitive to avoid ''rtb'' in URLs, but the phrase //real-time bidding// case-insensitive — the first version applied one case-sensitive flag to the whole alternation and undercounted the phrase (64 papers instead of 73; found by the figures reviewer); ''prebid'' catches ''Prebid.js'' and ''prebid-server''; ''schain'' also catches "supply chain object" with any spacing. Nothing was widened after the first run, because the candidate set is hand-labelled and the labels absorbed the false positives.
+  * **Term probe** — the regexes above are the fold. Width decisions: ''RTB'' case-sensitive to avoid ''rtb'' in URLs, but the phrase //real-time bidding// case-insensitive — the first version applied one case-sensitive flag to the whole alternation and undercounted the phrase (64 papers instead of 73; found by the figures reviewer); ''prebid'' catches ''Prebid.js'' and ''prebid-server''; ''schain'' also catches "supply chain object" with any spacing. Nothing was widened after the first run **except the RTB alternation described above** (64 → 73 papers, on the figures reviewer's finding); no CORE term was widened, because the candidate set is hand-labelled and the labels absorbed the false positives.
 ===== Quotes and figures spot-checked against the papers =====
@@ Line 313: / Line 313: @@
 | **SupplyChain object 1.0** | ''github.com/InteractiveAdvertisingBureau/openrtb/blob/master/supplychainobject.md'': version "1.0"; ''asi'', ''sid'', ''hp'' required; ''name''/''domain'' optional and "should NOT be included if it exists in the advertising system's sellers.json file". IAB page dates finalisation 2019-07-31 | **Accepted** |
 | **OpenRTB version** | ''iabtechlab.com/standards/openrtb/'': latest 2.x is 2.6-202309; history on GitHub | Accepted; not load-bearing on the page |
-| **app-ads.txt** | ''iabtechlab.com/standards/app-ads-txt/'' gives only the Nov 2018 beta date; the ads.txt 1.1 changelog dates the app-ads.txt differentiation to March 2019, as does {[bashir2019_adstxt]} | Accepted with both sources; page says "March 2019" via the paper |
+| **app-ads.txt** | ''iabtechlab.com/standards/app-ads-txt/'' gives only the Nov 2018 beta date; the ads.txt 1.1 changelog dates the app-ads.txt differentiation to March 2019, as does {[bashir2019_adstxt]} | Accepted with both sources. The content page does **not** print the date — it names the file and the open question and leaves the dating here. Until 2026-09-22 this cell said the page says "March 2019", which it never did |
 | **Prebid.js: ''getBidResponses'' fields** | ''docs.prebid.org/dev-docs/publisher-api-reference/getBidResponses.html'' | **Accepted** |
 | **Prebid.js: ''globalVarName'' / ''defineGlobal''** | ''raw.githubusercontent.com/prebid/Prebid.js/master/README.md'' build-options table: default ''pbjs''; "If false, do not set a global variable" | **Accepted**; this is what makes {[liu2024_opted]}'s 5,421 a lower bound |
@@ Line 327: / Line 327: @@
 | **Sheaib dataset** | ''doi.org/10.17617/3.STVMDI'' → ''edmond.mpg.de'' 200 | **Accepted** |
 | **Iqbal code page; Aziz repo** | 200; MIT, pushed 2024-07-09 | Accepted (Aziz not on the page) |
-| **HBDetector** | ''github.com/mipach/HBDetector'' 404; GitHub API "Not Found"; web search finds no relocation. Maintained equivalents: ''prebid/header-bidder-expert'' (Apache-2.0, last pushed 2022-09-12) and ''prebid/professor-prebid'' (Apache-2.0, pushed 2026-02-19) | **Dead link recorded on the page** |
+| **HBDetector** | ''github.com/mipach/HBDetector'' 404; GitHub API "Not Found"; web search finds no relocation. Nearest live equivalent: ''prebid/professor-prebid''. ''prebid/header-bidder-expert'' (Apache-2.0) still resolves but was last pushed 2022-09-12, so it is **not** a maintained equivalent — the date in this cell contradicted the word //maintained// until 2026-09-22 (Apache-2.0, pushed 2026-02-19) | **Dead link recorded on the page** |
 | **HARPO code** | ''github.com/bitzj2015/Harpo-NDSS22'' 404 | **Dead link recorded** |
 | **DeepCrawling** | redirects to ''github.com/sajjadium/Crawlium'' (MIT, pushed 2023-10-03) | **Move recorded** |
@@ Line 518: / Line 518: @@
 ===== What could not be established =====
-  * **How much of header bidding is still client-side.** The only split is February 2019 (17.3 / 34.7 / 48). Every later bids paper is conditioned on the sites that expose the auction, and none reports what fraction of its candidates that was. The page names this as the top open question rather than guessing.
+  * **How much of header bidding is still client-side.** The only split is February 2019 (17.3 / 34.7 / 48). Every later bids paper is conditioned on the sites that expose the auction. Two do report what fraction of their candidate list that was — {[liu2024_opted]} (5,421 of 100K) and {[zeng2022_factors]} (703 of 10,000) — so the gap is the client / hybrid / server split //among// HB sites, not the exposure rate. Until 2026-09-22 this bullet still said //none reports// it, the universal the generic pass had already retracted on the content page. The page names this as the top open question rather than guessing.
   * **Adoption of OWNERDOMAIN / MANAGERDOMAIN.** Zero papers; not measured here either (it would need a crawl, not a corpus query). Left as the page's "cheapest open measurement".
   * **Whether the arXiv v3 figures match the S&P camera-ready.** Not checked.
@@ Line 527: / Line 527: @@
 ===== Mistakes caught during the run =====
-  * A child agent's quotation of a Cook et al. footnote could not be found as given; the sentence exists, spliced across columns. The checker's proximity level was added because of this, and 27 of 167 checks rely on it — which is a statement about ''.cols'' rendering, not about the quotes.
+  * A child agent's quotation of a Cook et al. footnote could not be found as given; the sentence exists, spliced across columns. The checker's proximity level was added because of this. On the run that sentence was written, 27 of 167 checks relied on it — which is a statement about ''.cols'' rendering, not about the quotes.
   * **The published "empty 200 body" claim about Google's ''sellers.json'' was wrong.** The run's ''curl'' did not follow redirects; the canonical URL is a 302, and the 0-byte redirect body was read as an empty file — first blamed on the User-Agent, then on the URL, then published. The external-currency reviewer reproduced clean 302s and pointed out the script recorded byte counts but no status codes. Both pages, the script and the run's memory note were corrected. Lesson: record ''%{http_code}'' and ''%{redirect_url}'', never a byte count alone.
   * **A ''WebFetch'' summary was quoted as a primary source.** The Prebid Server sentence on the first revision was the fetch tool's paraphrase, attributed to a page that does not contain it. Replaced with the sentence that exists, on the page where it exists.
@@ Line 539: / Line 539: @@
 ===== Review =====
-Four reviewers, all told the context might not be exhaustive and all handed the page text, the report script, its unedited output, the quote-check and external-check scripts with their outputs, and this log. The three focused ones ran in parallel on the frozen first revision (rev 1788368710); their accepted fixes were applied and republished before the generic pass.
+**Five** reviewers, all told the context might not be exhaustive and all handed the page text, the report script, its unedited output, the quote-check and external-check scripts with their outputs, and this log. The three focused ones ran in parallel on the frozen first revision (rev 1788368710); their accepted fixes were applied and republished before the generic pass.
 ^ Reviewer ^ Finding ^ Action ^
@@ Line 553: / Line 553: @@
 | citations-and-quotes (sonnet) | **Denominator splice.** "Of 188 DIRECT entries, 5 could be confirmed; 169 IDs were absent and 418 had a domain mismatch" welds two counts the paper keeps apart (5 of 188 verified; 169 + 418 = 587 "remaining unique direct" entries) and the arithmetic does not close. Confirmed against a clean ''pypdf'' extraction: the paper's own wording is the source of the confusion. | **Accepted.** The page now gives both totals and says the paper does not reconcile them |
 | citations-and-quotes (sonnet) | The tightened quote checker produced 3 FAILs at review time where this page claimed 0; the reviewer verified all three as genuine quotes broken by hyphenation merges ("under"+"has" → "underhas") in the column-repaired text. | **Accepted.** Those three are now checked as their two verbatim halves; 170 of 170 pass on the final run, 24 by proximity |
-| citations-and-quotes (sonnet) | All 18 keys resolve exactly once; no DOI or title duplicates; every added entry's authors and DOI verified. The three author-list doubts recorded above — HARPO's four authors, Lukić's second author (Papadopoulos), Sheaib's hand-filled list — are all **correct as published** (side-by-side bylines in ''paper.norm.txt'' and the PoPETs landing page). ''check_attributions --prose'' reproduces the four false-positive SUSPECTs. Sampled figures and denominators in the //Pick the Unit// and //Three Observables// tables and all 15 dark-pooling citations check out. | Noted; the "pre-existing hygiene issue" wording about ''lukic2026_mv3'' under //Bibliography// is withdrawn |
+| citations-and-quotes (sonnet) | All 18 keys resolve exactly once; no DOI or title duplicates; every added entry's authors and DOI verified. The three author-list doubts recorded above — HARPO's four authors, Lukić's second author (Papadopoulos), Sheaib's hand-filled list — are all **correct as published** (side-by-side bylines in ''paper.norm.txt'' and the PoPETs landing page). ''check_attributions %%--prose%%'' reproduces the four false-positive SUSPECTs. Sampled figures and denominators in the //Pick the Unit// and //Three Observables// tables and all 15 dark-pooling citations check out. | Noted; the "pre-existing hygiene issue" wording about ''lukic2026_mv3'' under //Bibliography// is withdrawn |
 | generic (fable) | **All 15 findings accepted; the highest-yield pass.** (1) A universal negative the page's own //Pick the Unit// table contradicted: "none of [the bids papers] reports what fraction of its candidate list" exposed Prebid — {[liu2024_opted]} (5,421 of 100K) and {[zeng2022_factors]} (703 of 10,000) do. | **Fixed** in the superseded bullet and the //What to Report// item; the true claim (the client / hybrid / server split among HB sites is unmeasured since 2019) kept |
 | generic (fable) | (2) The three non-crawl measuring papers were described from a **literal string in the report script** ("extension, proxy, industrial logs") — the third is a cites-only paper; the real three are two passive proxies and one extension. Also "fraud detected from an ad network's own bidding logs" misdescribed the CCS 2024 app-analysis paper. | **Fixed** on the page; the script now prints the slugs with ''crawlConfig === null'' instead of a string |
@@ Line 569: / Line 569: @@
 | generic (fable) | (14) Stale numbers on this page after the quote-checker change (27 vs 24; 500 vs 750-character window; 169 vs 170). | **Fixed** |
 | generic (fable) | (15) Minor: "18 do so substantively" (3 are cites); the 2010–2016 zero explained by ''ads.txt'''s birth alone; "every exchange refused" without "they contacted"; ''header-bidder-expert'' called maintained (last push 2022); this page's run table already claimed the generic pass had run. | **All fixed** |
+| generic (fable), 2026-09-22 | **A retracted universal survived on //this// page.** Finding (1) of the 2026-09-02 pass killed //"none of [the bids papers] reports what fraction of its candidate list"// on the content page. Under //What could not be established// this log still said //"none reports what fraction of its candidates that was"// — the same false negative, in the section a reader goes to for the page's own limits | **Accepted, fixed.** The bullet now names {[liu2024_opted]} (5,421 of 100K) and {[zeng2022_factors]} (703 of 10,000) and scopes the gap to the client / hybrid / server split //among// HB sites. **This is the highest-value finding of the pass**: a fix applied to the content page and not to its provenance is worse than no fix, because the provenance is where a careful reader goes to check |
+| generic (fable), 2026-09-22 | **Fix (15) landed on the content page only.** The content page says ''header-bidder-expert'' //"has not been touched since 2022"//; the //External sources// table here still called it a //Maintained equivalent// — with ''last pushed 2022-09-12'' in the same cell | **Accepted, fixed.** The cell now names ''professor-prebid'' as the live equivalent and says in terms that ''header-bidder-expert'' is not maintained |
+| generic (fable), 2026-09-22 | **The content page contradicted itself about the below-threshold papers.** //"the 21 papers with one to four mentions were //not// read"// against, 38 lines later, //"Three papers with 1–4 mentions happened to be read during the run"//. The report and the script's ''BELOW_READ'' set both say 3 read / 18 not | **Accepted, fixed** in favour of the report: 3 read, 18 not, all 21 listed |
+| generic (fable), 2026-09-22 | **''app-ads.txt'' is named by three papers, not two.** The page said it //"appears only as background in two mobile ad-fraud papers"//, while its own reach table and the report both say 3. The third is {[bashir2019_adstxt]} — the paper the page quotes two sections later asking for //"a separate study"// of it | **Accepted, fixed.** Verified independently by counting ''app-ads.txt'' occurrences per paper over the corpus: 10 in the CCS 2024 laundering paper, 3 in Bashir, 3 in the CCS 2021 invalid-traffic paper. The //only// is dropped and all three are named |
+| generic (fable), 2026-09-22 | **Q18 overstated its own report, twice.** //"11 papers flag at least one; all URLs re-fetched"//. Counting the report's PASS C artifact-links block gives **10**; and {[liu2025_fingerprinting]}'s OneDrive dataset link appears nowhere in the external-check output | **Accepted, fixed.** Both corrected, and the cell now says //all URLs re-fetched// was true of the check list and not of the papers — which is the distinction that made the error invisible |
+| generic (fable), 2026-09-22 | **A claim about the content page that the content page does not contain.** The ''app-ads.txt'' external row said //page says "March 2019" via the paper//; ''grep'' returns zero hits for that string | **Accepted, fixed.** The cell now says the page does not print the date and that the dating lives here |
+| generic (fable), 2026-09-22 | **//PETS-centred// contradicted by the row's own paper list.** The //Bids as a dependent variable// row named six papers and called the family PETS-centred; two are at PoPETs, the rest at IMC, NDSS and TheWebConf. This log records the same error being found and fixed in the by-venue prose during the run — and the method-table cell was not fixed with it | **Accepted, fixed**, and recorded as the more instructive half: the run already knew, and the correction did not reach every place the claim was made |
+| generic (fable), 2026-09-22 | **//Shrinking// asserted as fact, then posed as an open question.** //"an observation, but of a shrinking and self-selected slice"// against //Open Questions//: //"Has the client-side share of header bidding fallen since 2019?"// Its only support is the ATOM authors' aside, which the page itself frames as an expectation | **Accepted, fixed.** //Shrinking// is now named as an unmeasured expectation, which is what the Open Question says |
+| generic (fable), 2026-09-22 | **Fix (10) was incomplete.** The client/server-split negative carries //in these venues// in the method table and is absolute in two further places (//nobody has reported since 2019//, //what nobody has re-measured//). The corpus query can only support the qualified form | **Accepted, fixed** in both. Two further instances the reviewer listed were already qualified or were the fixed sentence itself |
+| generic (fable), 2026-09-22 | **An absolute on the Related Pages list.** //"EasyList is how every paper here found the ads"//. By the page's own method table, the ''ads.txt''/''sellers.json'' papers find no ads at all, the RTB-price papers read ''nURL'' macros, the field study used an extension, and the Prebid papers read ''getBidResponses()'' | **Accepted, fixed** — scoped to the ad-attribution crawls, with the reason EasyList matters to the others (it removes the auction) kept |
+| generic (fable), 2026-09-22 | **A judgement about an unread paper, presented as a finding.** //"the venue's single mention is a passing one"// — the IEEE S&P paper carrying that mention is marked //not read// in the report and is not in ''BELOW_READ'' | **Accepted, fixed.** The sentence now says what is known — it is a cookie-banner paper the run did not read — instead of characterising its contents |
+| generic (fable), 2026-09-22 | **A script comment contradicting the script's own output.** ''report_ads_txt.mjs'' said the case-sensitivity bug //"undercounted the phrase by 16 papers (64 vs 80)"//. The report, this log and the reviewer who found the bug all say 64 → **73**; 80 appears nowhere in the output | **Accepted, fixed**, with the old figure recorded in the comment. The report output is byte-identical after the edit, which is the proof it was comment-only |
+| generic (fable), 2026-09-22 | **This log denied a widening it had just described.** The //Term probe// bullet explains the RTB alternation being split and the count going 64 → 73, then ends //"Nothing was widened after the first run"// | **Accepted, fixed** — the sentence now carves out the RTB alternation and keeps the true part, that no CORE term was widened |
+| generic (fable), 2026-09-22 | **Present tense for a superseded run.** //"27 of 167 checks rely on it"// beside the current 24 of 170 | **Accepted, fixed** — past tense, dated to the run it describes |
+| generic (fable), 2026-09-22 | //"The first version of that test allowed any order inside a 750-character window (250 before and 500 after)"// against //500-character window// in three other places | **Rejected as a contradiction, and nothing changed.** 250 + 500 //is// a 750-character span: the two are the same window described at different precision, and the three //500// instances are quotations of what the figures reviewer reported at the time. Recorded because the next reader will trip on it too |
+| generic (fable), 2026-09-22 | Nine further items raised as **SUSPECTED** and explicitly not decidable from the snapshot: the //25,764 winning bids// unit against the quote-check needle's //"ads where a header bidding auction took place"//; //56.4% failed WHOIS/DNS validation// against the needle's //"syntactic errors"//; unguarded figures on the personas line; //none among the papers they cite//; the two unread ''ads.cert'' papers; //both 2025 studies infer ownership from WHOIS//; //the only bid dataset about people//; ''{[poteat2021_securitytxt]}'' as a 19th citekey; and //13 child reports// against 20 labelled candidates | **Recorded, not fixed.** The first two are the most likely to be real and are the most expensive to settle, because each needs the paper read rather than grepped — the needle in both cases describes a //different quantity// from the sentence it is attached to, which is exactly the failure a quote check cannot see. Carried as the ''adstxt-suspected-nine'' work item rather than guessed at |
+| generic (fable), 2026-09-22 | Verified clean: the figures, tables and denominators in //Use in Publications//, the six-family table, the crawl-configuration percentages and the live ''sellers.json'' arithmetic all agree with the report; the embedded report and quote-check blocks are byte-identical to the outputs; the tables and footnotes parse; no dangling references | no change needed. Independently re-run on 2026-09-22: ''report_ads_txt.mjs'' is **byte-identical** to its committed output, ''adstxt_quotecheck.mjs'' is **170 PASS / 0 FAIL**, and ''check_page_numbers.mjs'' whole-page against report + quote-check + external output is **OK** |
 ===== Report output, unedited =====
@@ Line 920: / Line 937: @@
 <bibtex bibliography></bibtex>
+===== Markup sweep, 2026-09-17 =====
+Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.