| Both sides previous revisionPrevious revision | |
| provenance:design:algorithm_audits [2026/09/11 11:21] – Generic review log (9 findings, all accepted); the null adjudication and its definition; verdicts for all 58 shortlisted papers; residue re-derived (157/391). Authored by Claude karel.kubicek.claude | provenance:design:algorithm_audits [2026/09/17 07:23] (current) – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude |
|---|
| **A generic review pass found the weak point here.** 26 of the 58 had been read and dropped with no written verdict, and one of them — //Cart-ology// (CCS 2022, shortlist score 7) — satisfies the rule: five browser profiles, four of them unused baselines, //"created and mechanistically measured in the same way, with separation between attacker, victim, and baselines on different machines with different IPs"//, read out as ad distributions. It had been rejected on the strength of ''_aa_adj.mjs'' returning **zero** sentences for it — a 420-character sentence cap meeting a column-spliced PDF. **A probe returning nothing is not a negative result**, and it was treated as one. All 58 now carry a written verdict, and the IMC 2024 filter-bubble poster was promoted at the same time (four bot arms differing only in video-selection strategy). The population moved from 32 to **34**. | **A generic review pass found the weak point here.** 26 of the 58 had been read and dropped with no written verdict, and one of them — //Cart-ology// (CCS 2022, shortlist score 7) — satisfies the rule: five browser profiles, four of them unused baselines, //"created and mechanistically measured in the same way, with separation between attacker, victim, and baselines on different machines with different IPs"//, read out as ad distributions. It had been rejected on the strength of ''_aa_adj.mjs'' returning **zero** sentences for it — a 420-character sentence cap meeting a column-spliced PDF. **A probe returning nothing is not a negative result**, and it was treated as one. All 58 now carry a written verdict, and the IMC 2024 filter-bubble poster was promoted at the same time (four bot arms differing only in video-selection strategy). The population moved from 32 to **34**. |
| |
| **Every verdict, with the sentence that settled it, is in the script output below** (''--list''). The script refuses to run if any entry lacks an adjudication note. | **Every verdict, with the sentence that settled it, is in the script output below** (''%%--list%%''). The script refuses to run if any entry lacks an adjudication note. |
| |
| ===== The screening loss ===== | ===== The screening loss ===== |
| | Khattak et al., NDSS 2016, //Do You See What I See?// | no label record — NDSS 2016 has none | | | Khattak et al., NDSS 2016, //Do You See What I See?// | no label record — NDSS 2016 has none | |
| |
| The full list of 48 is in the ''--list'' output below. **Consequence, stated on the content page:** every audit count on it is a lower bound, biased against fairness-framed work, and the page publishes no estimate of the wider literature's size. | The full list of 48 is in the ''%%--list%%'' output below. **Consequence, stated on the content page:** every audit count on it is a lower bound, biased against fairness-framed work, and the page publishes no estimate of the wider literature's size. |
| |
| ===== Folding ===== | ===== Folding ===== |
| Four reviewers, all told explicitly that the author's context may not be exhaustive, and all handed the page text, the scripts, their outputs and these notes. The three focused passes ran in parallel first; the generic pass ran afterwards, on the corrected text. | Four reviewers, all told explicitly that the author's context may not be exhaustive, and all handed the page text, the scripts, their outputs and these notes. The three focused passes ran in parallel first; the generic pass ran afterwards, on the corrected text. |
| |
| ==== Figures vs script (''sonnet'') ==== | ==== Figures vs script (sonnet) ==== |
| |
| Re-ran all four scripts (outputs byte-identical to the committed ones), re-derived every candidate count from the five probe scripts, and mutation-tested the contracts. | Re-ran all four scripts (outputs byte-identical to the committed ones), re-derived every candidate count from the five probe scripts, and mutation-tested the contracts. |
| | Everything else — every table cell, year bucket, venue row, the 8-term noise table and its 11-paper list, the vocabulary table, the statistics ranking, the hand-folded Bonferroni families, the 11-paper no-test list, and all six probe candidate counts | **no defect** | — | | | Everything else — every table cell, year bucket, venue row, the 8-term noise table and its 11-paper list, the vocabulary table, the statistics ranking, the hand-folded Bonferroni families, the 11-paper no-test list, and all six probe candidate counts | **no defect** | — | |
| |
| ==== Citations and quotes (''sonnet'') ==== | ==== Citations and quotes (sonnet) ==== |
| |
| Checked all 12 new keys against ''data/corpus2/.meta'' or Crossref, ran a DOI-and-title dedup scan over all ~993 bibliography entries, re-grepped the quotes independently of the checker, and verified every prose attribution. | Checked all 12 new keys against ''data/corpus2/.meta'' or Crossref, ran a DOI-and-title dedup scan over all ~993 bibliography entries, re-grepped the quotes independently of the checker, and verified every prose attribution. |
| | **Accidental exposure caught by the reviewer.** Mid-review it observed ''algorithm_audits_set.mjs'' in a transient state with {[guha2010_challenges]} missing and N=31, and flagged that the file //"appears to have flickered during the session"// | **acknowledged.** That was my own mutation test of the new population-size contract, running against the same working copy the reviewer was reading. The set was restored and re-verified. Recorded here because a reviewer seeing a file mutate under it is exactly the kind of thing that should not be quietly dropped — and because it is an argument for mutation-testing on a copy, not in place | | | **Accidental exposure caught by the reviewer.** Mid-review it observed ''algorithm_audits_set.mjs'' in a transient state with {[guha2010_challenges]} missing and N=31, and flagged that the file //"appears to have flickered during the session"// | **acknowledged.** That was my own mutation test of the new population-size contract, running against the same working copy the reviewer was reading. The set was restored and re-verified. Recorded here because a reviewer seeing a file mutate under it is exactly the kind of thing that should not be quietly dropped — and because it is an argument for mutation-testing on a copy, not in place | |
| |
| ==== External currency (''sonnet'') ==== | ==== External currency (sonnet) ==== |
| |
| Fetched rather than recalled, as of 2026-09-11. | Fetched rather than recalled, as of 2026-09-11. |
| | The AI Act contains no researcher-access provision bearing on sock-puppet methodology, so its absence is not a gap | **no defect** | | | The AI Act contains no researcher-access provision bearing on sock-puppet methodology, so its absence is not a gap | **no defect** | |
| |
| ==== Generic (''fable'') ==== | ==== Generic (fable) ==== |
| |
| Handed the corrected text after the three focused passes. No checklist; asked for whatever they were not looking for. It returned nine findings, ranked. **All nine accepted**; the first three changed the page materially. | Handed the corrected text after the three focused passes. No checklist; asked for whatever they were not looking for. It returned nine findings, ranked. **All nine accepted**; the first three changed the page materially. |
| ===== The full verdict list ===== | ===== The full verdict list ===== |
| |
| ''node scripts/report_algorithm_audits.mjs --list'' — the 32 verdicts with their evidence sentence, the 12 rejections with their reason, and all 48 screening losses. | ''node scripts/report_algorithm_audits.mjs %%--list%%'' — the 32 verdicts with their evidence sentence, the 12 rejections with their reason, and all 48 screening losses. |
| |
| <file text report_algorithm_audits-list-output.txt> | <file text report_algorithm_audits-list-output.txt> |
| |
| <bibtex bibliography></bibtex> | <bibtex bibliography></bibtex> |
| | |
| | ===== Markup sweep, 2026-09-17 ===== |
| | |
| | Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. |
| |
