Differences

This shows you the differences between two versions of the page.

--- provenance:design:ownership_resolution [2026/09/11 22:40] – Section L: untruncated residue notes, post-correction route counts, and the full reviewer log including the generic pass's 15 findings (14 accepted, 1 rejected). Authored by Claude karel.kubicek.claude
+++ provenance:design:ownership_resolution [2026/09/17 07:23] (current) – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude
@@ Line 208: / Line 208: @@
 ^ Script ^ Invocation ^ Result ^
-| ''report_webxray.mjs'' | ''node scripts/report_webxray.mjs --quotes'' | **byte-identical** to the committed ''report_webxray-output.txt'' before the fix in section C; regenerated after it |
+| ''report_webxray.mjs'' | ''node scripts/report_webxray.mjs %%--quotes%%'' | **byte-identical** to the committed ''report_webxray-output.txt'' before the fix in section C; regenerated after it |
-| ''owner_dbs.py'' | ''python3 scripts/owner_dbs.py --cache out/webxray/cache --disagreements 25'' | reproduces the 2026-08-17 frame exactly: 32,369 domains; 669 / 5,581 / 2,268; 58.6% / 84.3% / 80.5%; 15,651 merged rows; 19 non-hostname rows dropped; 6,941 ICANN and 3,290 PRIVATE rules; 782 of 827 owner names untouched by the fold, 16 of the 45 changed losing a legal suffix |
+| ''owner_dbs.py'' | ''python3 scripts/owner_dbs.py %%--cache%% out/webxray/cache %%--disagreements%% 25'' | reproduces the 2026-08-17 frame exactly: 32,369 domains; 669 / 5,581 / 2,268; 58.6% / 84.3% / 80.5%; 15,651 merged rows; 19 non-hostname rows dropped; 6,941 ICANN and 3,290 PRIVATE rules; 782 of 827 owner names untouched by the fold, 16 of the 45 changed losing a legal suffix |
-| ''owner_random_sample.py'' | ''python3 scripts/owner_random_sample.py --sample out/owner_sample.json --rows out/adj_rows_scored.json --table --wiki'' | **byte-identical** to ''out/owner_random_sample-output.txt'' |
+| ''owner_random_sample.py'' | ''python3 scripts/owner_random_sample.py %%--sample%% out/owner_sample.json %%--rows%% out/adj_rows_scored.json %%--table%% %%--wiki%%'' | **byte-identical** to ''out/owner_random_sample-output.txt'' |
 | ''owner_selfname_census.py'' | ''python3 scripts/owner_selfname_census.py'' | reproduces 0.8% / 0.7% / 3.2% (25 of 3,215; 250 of 38,368; 250 of 7,850) |
-| ''owner_adjudication.py'' | ''python3 scripts/owner_adjudication.py --table'' | reproduces the 28-row table exactly: 1/18/4/1/4, 8/12/6/2/0, 26/0/2/0/0, and the three rows scored as error rather than staleness |
+| ''owner_adjudication.py'' | ''python3 scripts/owner_adjudication.py %%--table%%'' | reproduces the 28-row table exactly: 1/18/4/1/4, 8/12/6/2/0, 26/0/2/0/0, and the three rows scored as error rather than staleness |
 | ''owner_irr_kappa.py'' | invocation reconstructed by the figures reviewer, since it was missing from the brief | reproduces pooled kappa +0.400 (+0.241, +0.551), per-list +0.027 / +0.478 / +0.460, settled-only +0.680 |
 | ''report_ownership_resolution.mjs'' | ''node scripts/report_ownership_resolution.mjs'' | new; section K |
@@ Line 500: / Line 500: @@
 ^ Check ^ Result ^
 | All published scripts re-run against pinned inputs | they **reproduce their committed output** — which, as C and I both show, is not evidence the output is //right//: byte-identity blessed a wrong denominator for three and a half weeks and five wrong constants for longer. A reproduction record is not a correctness record. |
-| ''owner_adjudication.py --table'' | reproduces the 28-row table exactly: 1/18/4/1/4, 8/12/6/2/0, 26/0/2/0/0, and the three error rows (''1rx.io'', ''jsdelivr.net'', ''stackadapt.com'') |
+| ''owner_adjudication.py %%--table%%'' | reproduces the 28-row table exactly: 1/18/4/1/4, 8/12/6/2/0, 26/0/2/0/0, and the three error rows (''1rx.io'', ''jsdelivr.net'', ''stackadapt.com'') |
 | ''domain_owners.json'' schema table on [[Programming:Crawler:webXray]], every cell | reproduces: 319/827 parent_id, depths 508/211/80/24/2/2, 761 uses, 803 platforms, 826 country, 104 trade_groups, 591 policy URLs, 132 GDPR, 4 CCPA, 13 opt-out, 34 crunchbase_id, 52 health-segment, 263 notes, 275 aliases, 69 languages, 2,175 of 3,215 |
 | Mutation test of ''report_ownership_resolution.mjs'' | **four mutations, four failures**. Swapping tight/loose on Disconnect → "tight (700) > loose (74)". Making ''loose'' disjoint from ''tight'' at equal size → "23 tight hits are not in the loose set" — the containment guard catches what a size comparison alone would miss. Injecting a fake union member → cross-check throws. Replacing the crawled intersection with the full union → cross-check throws. No guard is vacuous. |
@@ Line 526: / Line 526: @@
 | Mechanical evidence | ''bash scripts/owner_tail_probe.sh out/tail40.json out/tail_probe'' | ''out/tail_probe.txt'' (2,471 lines, 2m 29s, 6-way parallel) |
 | Adjudication | 5 Sonnet sub-agents × 8 domains, round-robin over the prevalence order so every batch spans the range | ''out/tail_adj/result1..5.json'' |
-| Gate | ''python3 scripts/owner_tail_merge.py --sample out/owner_sample.json --base out/adj_rows_scored.json --tail out/tail40.json --dir out/tail_adj --out out/adj_rows_tail.json'' | ''out/adj_rows_tail.json'', **0 problems** |
+| Gate | ''python3 scripts/owner_tail_merge.py %%--sample%% out/owner_sample.json %%--base%% out/adj_rows_scored.json %%--tail%% out/tail40.json %%--dir%% out/tail_adj %%--out%% out/adj_rows_tail.json'' | ''out/adj_rows_tail.json'', **0 problems** |
-| Citation check | ''python3 scripts/owner_verify_sources.py --rows out/tail_rows_only.json --cache out/tail_verify_cache --out out/tail_verify.json'' | 3 defects in the checker itself — see L.5 |
+| Citation check | ''python3 scripts/owner_verify_sources.py %%--rows%% out/tail_rows_only.json %%--cache%% out/tail_verify_cache %%--out%% out/tail_verify.json'' | 3 defects in the checker itself — see L.5 |
-| Hand corrections | ''python3 scripts/owner_tail_corrections.py --rows out/adj_rows_tail.json --out out/adj_rows_tail_final.json'' | 6 rows: resource ×3, keep ×2, rescore ×1 |
+| Hand corrections | ''python3 scripts/owner_tail_corrections.py %%--rows%% out/adj_rows_tail.json %%--out%% out/adj_rows_tail_final.json'' | 6 rows: resource ×3, keep ×2, rescore ×1 |
-| Re-estimate | ''python3 scripts/owner_random_sample.py --sample out/owner_sample.json --rows out/adj_rows_tail_final.json --table --wiki'' | ''out/owner_random_sample-tail-output.txt'' |
+| Re-estimate | ''python3 scripts/owner_random_sample.py %%--sample%% out/owner_sample.json %%--rows%% out/adj_rows_tail_final.json %%--table%% %%--wiki%%'' | ''out/owner_random_sample-tail-output.txt'' |
 | What changed | ''python3 scripts/report_tail_pass.py …'' | ''out/report_tail_pass-output.txt'' |
@@ Line 535: / Line 535: @@
 <WRAP tip>
-**Reproducing the //before// is a trap worth recording.** ''out/adj_rows.json'' and ''out/adj_rows_final.json'' both still carry ''self-named'' verdicts, and ''owner_random_sample.py'' exits fatally on them. The committed 2026-09-05 output is reproduced byte-identically **only** by ''--rows out/adj_rows_scored.json --wiki'' — the ''--wiki'' matters, because sections F and G exist only under it.
+**Reproducing the //before// is a trap worth recording.** ''out/adj_rows.json'' and ''out/adj_rows_final.json'' both still carry ''self-named'' verdicts, and ''owner_random_sample.py'' exits fatally on them. The committed 2026-09-05 output is reproduced byte-identically **only** by ''%%--rows%% out/adj_rows_scored.json %%--wiki%%'' — the ''%%--wiki%%'' matters, because sections F and G exist only under it.
 </WRAP>
@@ Line 630: / Line 630: @@
 The one non-OK row is ''adgrx.com'': ''samsungads.ca'' serves an **expired
 certificate**, so ''curl'' refuses it and the checker never sees a body. Fetched
-by hand with ''--insecure'': HTTP 200, 46,160 bytes, and the quoted row is
+by hand with ''%%--insecure%%'': HTTP 200, 46,160 bytes, and the quoted row is
 byte-verbatim in it. The checker was deliberately **not** given an
-''--insecure'' retry — an ownership citation whose host identity cannot be
+''%%--insecure%%'' retry — an ownership citation whose host identity cannot be
 validated should surface, not be swallowed.
@@ Line 785: / Line 785: @@
 also named the ''%%<WRAP todo>%%'' that still asked for the thing this sitting did;
 that one was **not** yet fixed and has been.
+=== A second sweep, after the generic pass's fixes were published ===
+''sweep_moved_figures.py'' was re-run over a **fresh export of all 182 pages**
+taken after publishing, with five pairs added for figures that carry no ''%'' —
+the class the generic pass showed the sweep was blind to (''42–49''→''54–56'',
+''69.9 vs 73.7''→''69.2 vs 73.9'', ''29 of the 525''→''30 of the 525'',
+''All four were found'', ''9 of 13''→''10 of 12'').
+**69 suspect lines, 68 legitimate and one real.** The content page's
+methodology bullet still said **"29 of the 525 verdicts were changed"**: the
+page carried that figure **twice**, round four fixed the copy in the section it
+was editing, and ''check_tail_figures.py'' could not see the other because it
+asserts a figure is *present* and a correct copy was. The other 68 are
+before/after columns, the published mutation harnesses' own output, the
+-09-05 estimator run this family deliberately preserves as the "before",
+and one coincidental match on an unrelated rater-agreement figure.
+Closed with code rather than with an edit: the guard now derives the total from
+''owner_tail_corrections-output.txt'''s own ''actions: ... rescore=N'' line plus the
+first sitting's 29, asserts both that the right value is present and that the
+superseded one is absent, and the mutation harness reverts **one** copy to prove
+it bites. Control: run against the pre-fix export, it fails on exactly that
+line.
+**This is the same shape as the defect three reviewers found earlier in the
+sitting** — a presence check blessing a page with one stale copy — and it
+recurred inside the fix for it. The harness header now states the limit plainly
+and points at the sweep as the tool that covers it.
+=== A third sweep, and the last two stale figures ===
+Re-run once more over a fresh export after publishing the guard fix. **68 hits,
+all legitimate except two**, both on ''provenance:programming:crawler:webxray''
+and both in the //judgement calls// section written by the 2026-09-05 sitting:
+  * "webXray at 69.9% domain-level and 93.9% encounter-weighted" — stated in the present tense about figures that are now 69.2% and 93.8%;
+  * "93.9% encounter-weighted is a flattering number … and Disconnect is better on the measured axis anyway" — the second clause is now **metric-dependent**, which is the finding the generic pass produced two sections above.
+Both were **dated rather than refreshed**: a record of a judgement made on a
+particular day should keep the figures it was made on. Each now carries the
+-09-11 value beside the 2026-09-05 one, and the "better on the measured
+axis" claim names both metrics and both p-values.
+The remaining 66 were read and are all one of: a //before// column, a
+struck-through open question quoting its own superseded text, a reviewer-log
+entry quoting the old page, the published mutation harnesses' own output, a
+''-'' line in the committed unified diff, the 2026-09-05 estimator run that
+''provenance:programming:crawler:webxray:random_sample'' exists to preserve, and
+one coincidental match of an unrelated inter-rater figure (76.5%).
+**Three sweeps were needed and each found something the one before did not**,
+because each round of edits moved figures the previous pair list did not name.
+The pair list is now 28 entries and is committed at ''out/moved_figures.json'';
+the next sitting should extend it rather than rebuild it.
@@ Line 1341: / Line 1400: @@
 <bibtex bibliography></bibtex>
+===== Markup sweep, 2026-09-17 =====
+Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.