Differences

This shows you the differences between two versions of the page.

--- provenance:security:authentication [2026/09/11 07:14] – New page: working log for Security:Authentication — the inclusion rule, the 184-candidate hand audit, five probes plus a full-text recall pass, folds and residue, 95 quote checks, external sources and rejections, four review passes, and the scripts with t karel.kubicek.claude
+++ provenance:security:authentication [2026/09/22 07:11] (current) – Name the three ethics/artifacts fields and their two denominators in the 2026-09-22 note; record reviewer's rejected item. Authored by Claude karel.kubicek.claude
@@ Line 11: / Line 11: @@
 | Corpus at the time | ''data/extract/run1/extractions.jsonl'', **5,859** extracted papers, 7 venues, 2010–2026 |
 | Page status | **New page.** Nothing on this wiki covered authentication deployment; the closest neighbour, [[Programming:Registration]], covers login as an //instrument// and explicitly hands the "SSO as the subject" question over to a page that did not exist |
-| Scripts written | ''scripts/auth_pop.mjs'' (probes, inclusion rule, hand verdicts, citekeys, assertions), ''scripts/auth_audit.mjs'' (audit report), ''scripts/auth_report.mjs'' (the page's figures), ''scripts/auth_recall.py'' (full-text recall probe), ''scripts/auth_quotecheck.mjs'' (84 quote and figure checks) |
+| Scripts written | ''scripts/auth_pop.mjs'' (probes, inclusion rule, hand verdicts, citekeys, assertions), ''scripts/auth_audit.mjs'' (audit report), ''scripts/auth_report.mjs'' (the page's figures), ''scripts/auth_recall.py'' (full-text recall probe), ''scripts/auth_quotecheck.mjs'' (95 quote and figure checks) |
 | Models | Page and analysis: Claude Opus 5 (the session model). Two research sub-agents on ''sonnet'' (standards currency; tools and adoption data), both instructed to fetch rather than recall and to mark anything unverified. Four review passes: three on ''sonnet'', one on ''fable'' |
 | Bibliography | **27 entries added**, generated by ''scripts/bibgen.mjs'' from the corpus index; no entry written by hand from memory |
@@ Line 57: / Line 57: @@
 node scripts/auth_report.mjs --prevalence --wikitable   # every prevalence tuple, and the 45-paper table
 python3 scripts/auth_recall.py --threshold 8            # the full-text recall pass (~7 min)
-node scripts/auth_quotecheck.mjs       # 84 quote and figure checks
+node scripts/auth_quotecheck.mjs       # 95 quote and figure checks
 </code>
@@ Line 111: / Line 111: @@
 | Intro | 45 papers; 42 of them web-platform (2.6% of 1,622); 25 of them ran a crawl (2.2% of 1,120) | ''auth_report.mjs'' header. The first draft divided 45 by both denominators, which put three operator-telemetry papers over a web-platform base and twenty non-crawling papers over a crawl base; the generic review caught it and the script now prints the subsetted numerators |
 | The denominator is the whole argument | 9.3% / 6.30% / 7.23% / 27% / 57.8% and their denominators | each paper's own text, all five verified by ''auth_quotecheck.mjs''; the 85.9K reachable denominator is derived from the paper's own 14,104-failure figure, not back-calculated from the percentage |
-| Five questions | every prevalence figure quoted | ''auth_report.mjs --prevalence'', which prints all **258** prevalence strings with the extraction's evidence quote and section, for all 45 papers |
+| Five questions | every prevalence figure quoted | ''auth_report.mjs %%--prevalence%%'', which prints all **258** prevalence strings with the extraction's evidence quote and section, for all 45 papers |
 | Methods and which are current | median site-unit n = **872**; 39 of 85 site-unit entries under 500 | ''auth_report.mjs'' §5 |
 | " | ''classification.method == llm'': **0** of 45, **177** corpus-wide (2023:2, 2024:27, 2025:77, 2026:71) | ''auth_report.mjs'' §8 |
@@ Line 118: / Line 118: @@
 | Ethics | 45.5% / 68.2% / 68.3% with web and corpus base rates | ''auth_report.mjs'' §6 |
 | Use in publications | population, window, venue, instrument, frame and quiet-field tables | ''auth_report.mjs'' §§1, 3, 4, 6, 7 |
-| The 45 papers | the whole table | ''auth_report.mjs --wikitable'', generated rather than typed |
+| The 45 papers | the whole table | ''auth_report.mjs %%--wikitable%%'', generated rather than typed |
 Population definitions used, each naming its own denominator:
@@ Line 138: / Line 138: @@
 **Residue: 203 distinct tool strings, 218 mentions, unmapped and printed in full** in the embedded report output. It is not noise — it is the most informative part of the fold, because it is almost entirely **one-off crawlers named after their own paper** (SSOScan, Distinct, SAAT, Web3AuthChecker, QRLChecker, IDB-DETECTOR, FP-Extractor, PASSKEYS-RADAR, PASSKEYS-ATTACKER, Account Framework, Live-Monitor, Communication-Inspector). That is the finding the page states: this subfield builds bespoke instruments and reuses almost nothing, and **OpenWPM appears zero times**.
-**Sampling frames** (''population[].sourceList''), counted by paper. Rules: Tranco; CrUX including "Chrome UX Report" and "Chrome User Experience Report"; Alexa; Majestic; Quantcast; Cisco Umbrella; 2fa.directory; Similarweb; HTTP Archive; Fortune 1000; "custom / not stated" for custom seed lists and ''not-stated''; participant panels. **Residue: 44 distinct strings**, printed in full below — mostly per-paper artefacts (Google login traces, partner RP authentication logs, the FIDO Metadata Service, PASSKEYS-RADAR merged sources, DappRadar, npm and GitHub APIs) rather than spelling variants of a ranking list.
+**Sampling frames** (''population[].sourceList''), counted by paper. Rules: Tranco; CrUX including "Chrome UX Report" and "Chrome User Experience Report"; Alexa; Majestic; Quantcast; Cisco Umbrella; 2fa.directory; Similarweb; HTTP Archive; Fortune 1000; "custom / not stated" for custom seed lists and ''not-stated''; participant panels. **Residue: 47 distinct strings**, printed in full below — mostly per-paper artefacts (Google login traces, partner RP authentication logs, the FIDO Metadata Service, PASSKEYS-RADAR merged sources, DappRadar, npm and GitHub APIs) rather than spelling variants of a ranking list.
 ===== Quotes and Figures Checked Against the Papers =====
@@ Line 195: / Line 195: @@
 **27 entries**, all generated by ''%%node scripts/bibgen.mjs <venue>/<year>/<slug>%%'' from ''data/corpus2/.meta'' — publisher metadata, not recalled strings. **23** are population papers; the other **four** are cited in prose only: {[lassak2024_aren]} (why passkey deployment stalls), {[wang2012_signing]} (the founding web-SSO security study), {[sahin2023_investigating]} (the administrators' side of password policy) and {[brandao2015_mending]} (brokered-identity history). The population's other 22 citekeys were already in the bibliography. Checks run before appending:
-  * ''scripts/bib_dedup_scan.py --bib'' on the **merged** file: 977 entries, 977 distinct citekeys, **no same-DOI and no same-title pair**. The scan's [D] candidate class (same year, same folded surname) printed 85 pairs, all previously judged and none of them mine except surname coincidences (''wu2026_email'' against ''wu2026_tracking'', ''zhang2025_qrcode'' against ''zhang2025_abusability'', ''yan2024_stealing'' against ''yan2024_quality'').
+  * ''scripts/bib_dedup_scan.py %%--bib%%'' on the **merged** file: 977 entries, 977 distinct citekeys, **no same-DOI and no same-title pair**. The scan's [D] candidate class (same year, same folded surname) printed 85 pairs, all previously judged and none of them mine except surname coincidences (''wu2026_email'' against ''wu2026_tracking'', ''zhang2025_qrcode'' against ''zhang2025_abusability'', ''yan2024_stealing'' against ''yan2024_quality'').
   * Every ''%%{[key]}%%'' on both pages resolves against the merged bibliography: **51 distinct keys used across the two pages, 0 missing**, and every one of the 27 new entries is cited. (The generic review caught that this very sentence had been counting itself: an unescaped citekey inside inline code still resolves, because monospace markup does not escape the plugin's syntax.)
   * **Author lists for PETS and USENIX** are absent from the corpus index by construction. ''scripts/fetch_authors.py'' recovered nine from the venue landing pages; **three PETS records failed to parse** and were filled by hand from the paper's own first page — {[corre2017_users]}, {[kepkowski2022_handle]} and {[brandao2015_mending]} — then cross-checked against Crossref, which agreed on names and order in all three. The 2015 brokered-identity paper genuinely has a fourth author listed as "Anonymous", on the PDF and in Crossref alike; the entry keeps it.
@@ Line 225: / Line 225: @@
 | **The ''poster'' paper {[tchokodeu2024_poster]}** | IN, flagged in its verdict reason as a poster whose figures are preliminary | OUT for being two pages. It is the only paper here whose frame is organisations rather than a ranking list, which is worth showing |
 | **Publishing "nobody uses an LLM"** | Published, with the year-by-year corpus counts and the thin-slice caveat in the same paragraph | Withhold it. The claim rests on an enum over a provisional slice, and "nobody" is the kind of word a probe should not be allowed to produce — which is why it is stated as an enum count over the population, not as a probe result |
+===== After Publishing: What the Rendered Page Showed =====
+Source-level checks pass on a page that renders wrong, so both pages were verified against the rendered DOM after saving, not against their wikitext.
+^ Check ^ Result ^
+| **The bibliography was stale, silently.** The content page first rendered **24 reference entries for 51 distinct citekeys**, numbered up to [51] with gaps — every inline marker resolved, so nothing looked broken. bibtex4dw had served a cached parse of ''literature:bibliography'' that predated the 27 new entries | Fixed by requesting ''?purge=true'' on the bibliography and then on each citing page. The content page now renders **51 of 51** with no gaps, and ''security'' renders its four new keys |
+| Content page structure | 11 h2, 21 h3, 12 tables, 131 table rows, both boxes as ''div''s, **0 red links**, no leftover literal markup |
+| Provenance page structure | 16 h2, 22 h3, 15 tables (this one included), 13 ''pre'' blocks — 12 file blocks and one code block — 23 reference entries, **0 red links** after fixing one namespace-relative ''%%[[artifacts]]%%'' |
+| The escaped citekey | ''%%{[key]}%%'' renders as literal text rather than resolving, which is the point of the escape |
+| Site-wide red-link gate | ''node scripts/sitemap.mjs'' — 174 pages, 4 promised-but-missing, all 4 declared on [[:roadmap]], **0 undeclared**, exit 0 |
 ===== Review Passes =====
@@ Line 239: / Line 250: @@
 | 3 | The ''LISTFOLD'' bucket for 2fa.directory was ''/2fa/i'', which also matches "UCB anonymized 2FA log events" and "UIUC 2FA-related support tickets" — so an operator-telemetry paper was counted as sampling from the directory, and the paper the prose named ({[gavazzi2023_multi]}) was not in the bucket at all because it sampled from Tranco | **Accepted and fixed.** Regex tightened to ''/2fa\.?\s?directory/i''; the table row moved from "3 papers, 2023–2025" to "2 papers, 2023"; the prose now names the two that sample from it and reports what the third actually did with it |
 | 4 | "two independent studies **four years apart**" compared a 2022 and a 2023 paper, and the 27.4% being compared is not printed by either paper — it is my arithmetic on the IMC study's two figures | **Accepted and fixed.** The years are now named, and the derived figure is labelled as derived in the sentence that uses it |
-| — | Setup checks | **No finding.** All four scripts exit 0 and reproduce their committed outputs byte-for-byte; the 45-row table on the page matches ''--wikitable'' byte-for-byte |
+| — | Setup checks | **No finding.** All four scripts exit 0 and reproduce their committed outputs byte-for-byte; the 45-row table on the page matches ''%%--wikitable%%'' byte-for-byte |
 | — | Mutation test of the published assertions | **No finding, and this is the result I wanted.** A bogus subgroup, a deleted citekey, a duplicated verdict reason and an ambiguated hand-map key each made ''audit()'' exit non-zero with a message naming the broken paper, and ''auth_report.mjs'' fails identically because it imports the same function. None of the four assertions is vacuous |
@@ Line 262: / Line 273: @@
 | 2 | The CTAP row named 2.3 (Proposed Standard, 2026-02-26) and missed that **CTAP 2.3.1 has been in Working Draft since 2026-05-29**, published to the specs directory in August | **Accepted and fixed.** The row now carries both |
 | 3 | The Microsoft Entra citation was dated 2026-08-26, which is an ''updated_at'' metadata value whose matching commit edits a shared include; the page itself displays "Last updated on 2026-06-23" | **Accepted and fixed.** The footnote now gives the displayed date and explains the discrepancy — which Pass 2 read the other way round, so the footnote records both readings |
-| 4 | Six URLs inside the //reproduced script output// are dead (a 401 anonymous-review link, three 404s, a 502, a 403) | **Accepted as informational, not fixed.** They are artifact links the **papers themselves** declared, reproduced verbatim inside a ''%%<file>%%'' block. Rewriting them would falsify the audit trail; the rot is the finding, and [[artifacts]] is the page about it |
+| 4 | Six URLs inside the //reproduced script output// are dead (a 401 anonymous-review link, three 404s, a 502, a 403) | **Accepted as informational, not fixed.** They are artifact links the **papers themselves** declared, reproduced verbatim inside a ''%%<file>%%'' block. Rewriting them would falsify the audit trail; the rot is the finding, and [[:artifacts]] is the page about it |
 | — | Tally | 31 of 34 external claims verified correct, including every GitHub date via the commits API rather than ''pushed_at'', the FIDO MDS 429 reproduced on three retries, and the Web Almanac's zero WebAuthn mentions |
@@ Line 1575: / Line 1586: @@
 candidates from five probes: 184
 POPULATION (hand audit, inclusion rule in scripts/auth_pop.mjs): 45 papers
-.8% of the 1622 web-platform papers
+  web-platform: 42 of the 45 population papers = 2.6% of the 1622 web-platform papers
-.0% of the 1120 papers that ran a crawl
+  ran a crawl:  25 of the 45 population papers = 2.2% of the 1120 papers that ran a crawl
+  have a crawlConfig object: 23 of 45 (the denominator for every crawlConfig share below)
+or later: 33 of 45
 ===== 1. Population shape =====
@@ Line 2789: / Line 2802: @@
 candidates from five probes: 184
 POPULATION (hand audit, inclusion rule in scripts/auth_pop.mjs): 45 papers
-.8% of the 1622 web-platform papers
+  web-platform: 42 of the 45 population papers = 2.6% of the 1622 web-platform papers
-.0% of the 1120 papers that ran a crawl
+  ran a crawl:  25 of the 45 population papers = 2.2% of the 1120 papers that ran a crawl
+  have a crawlConfig object: 23 of 45 (the denominator for every crawlConfig share below)
+or later: 33 of 45
 ===== 1. Population shape =====
@@ Line 4021: / Line 4036: @@
   * The roadmap row this page closed: [[:roadmap]], with its own log at [[provenance:roadmap]]
   * The neighbour that owns login-as-an-instrument: [[Programming:Registration]] and [[provenance:programming:registration]]
+====== References ======
+The same shared bibliography as the content page; this log adds no entries of its own.
+<bibtex bibliography></bibtex>
+===== Markup sweep, 2026-09-17 =====
+Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.
+===== Quote-check figure reconciled, 2026-09-21 =====
+Two places on this page described ''auth_quotecheck.mjs'' as **84 quote and figure checks** (the //Scripts written// row and the ''%%<code>%%'' block in //Running it//) while the //Quotes and figures checked// section says **95 needles, 95 PASS, 0 FAIL**. The page contradicted itself. Re-run, unedited:
+<code>
+$ node scripts/auth_quotecheck.mjs
+==== 95 PASS, 0 FAIL ====
+</code>
+**95 is right and the two 84s were stale**; the checker gained needles after those two lines were written. Both were repointed to 95. **Nothing else moved.** The check reports 0 FAIL, so the 2026-09-04 PDF-fallback change to ''quote_check.mjs'' and the sibling checkers has nothing to rescue here — this page was swept in that pass and this was the only defect found on it.
+^ Item ^ Value ^
+| Date | 2026-09-21, unsupervised |
+| Command | ''%%node scripts/auth_quotecheck.mjs%%'' |
+| Artifact | ''out/qc0921b/auth.txt'' |
+| Script changes | none |
+| Reviewers | one ''sonnet'' figures-vs-script pass |
+| Pages saved | this page only |
+===== Cached report output refreshed, 2026-09-22 =====
+The two embedded ''%%<file text>%%'' blocks reproducing ''auth_report.mjs'' output were **stale against the script that produces them**. When Pass 4's finding 1 was applied (the headline denominators putting 45 over both bases), the script was changed to print the subsetted numerators and the live [[security:authentication]] page was corrected — but the cached artifacts ''scripts/auth_report-output.txt'' and ''scripts/auth_report_prevalence-output.txt'', and therefore the copies embedded here, were never regenerated. They still carried the pre-fix header.
+**No figure on either page was wrong.** The risk ran the other way: a reviewer who diffs the content page against the embedded output instead of re-running the script reads the page's correct 2.6% / 2.2% against a cached 2.8% / 4.0% and files a false finding. That has happened on this wiki before.
+Re-run, unedited, and diffed against the cached artifacts:
+<code>
+$ node scripts/auth_report.mjs > scripts/auth_report-output.txt
+$ node scripts/auth_report.mjs --prevalence --wikitable > scripts/auth_report_prevalence-output.txt
+$ diff <old> <new>          # identical in both files
+c4
+  POPULATION (hand audit, inclusion rule in scripts/auth_pop.mjs): 45 papers
+- 2.8% of the 1622 web-platform papers
+- 4.0% of the 1120 papers that ran a crawl
++ web-platform: 42 of the 45 population papers = 2.6% of the 1622 web-platform papers
++ ran a crawl:  25 of the 45 population papers = 2.2% of the 1120 papers that ran a crawl
++ have a crawlConfig object: 23 of 45 (the denominator for every crawlConfig share below)
++ 2022 or later: 33 of 45
+</code>
+That four-line header block is the **only** difference in either file: the remaining 257 lines of the 261-line report and 1,226 of the 1,230 lines of the prevalence dump are byte-identical to what was published. Both blocks here were replaced with the fresh output. The old file's ''2.8%'' and ''4.0%'' were 45/1,622 and 45/1,120 — the unsubsetted numerators Pass 4 rejected — and they survived only inside the cache.
+**A second, real defect fell out of the re-read.** //Folds and their residue// gave the sampling-frame residue as **44 distinct strings**; the script prints — and this page has always embedded — ''RESIDUE — 47 distinct source-list strings unmapped'', and the list below it has 47 entries. The 44 was a hand count that never matched its own output. Corrected to 47. The instrument residue in the paragraph above it (**203 distinct strings, 218 mentions**) is right, and so is every other figure in the //Every figure on the page// table that this run re-derived: median site-unit n **872**, **39 of 85** entries under 500, ''classification.method == llm'' **0 of 45** and **177** corpus-wide (2023:2 2024:27 2025:77 2026:71), ''ethics.reviewOutcome'' **45.5%** and ''ethics.notifiedAffectedParties'' **68.2%** over a population of 44, and ''artifacts.availability'' **68.3%** over 41 — three fields over two denominators, which the //Every figure// table groups under the page section named //Ethics//, and **258** prevalence strings across 261 detection tuples.
+The other four cached outputs on this page were re-run as well and reproduce byte-for-byte: ''auth_audit-output.txt'', ''auth_quotecheck-output.txt'', ''auth_absence_probe-output.txt'' and ''auth_recall-output.txt''. All six embedded script blocks are byte-identical to the files under ''scripts/''.
+**Rejected:** nothing. The reviewer noted that the //Every figure// table files ''artifacts.availability'' under its //Ethics// row; that is a page-section label rather than a mis-attributed figure, and the row is left as it is. No finding on the content page — its prose already matched the current script, which is why this was a cache defect and not a figure defect, and [[security:authentication]] was not saved.
+^ Item ^ Value ^
+| Date | 2026-09-22, unsupervised |
+| Command | ''%%node scripts/auth_report.mjs%%'' and ''%%node scripts/auth_report.mjs --prevalence --wikitable%%'' |
+| What was wrong | two cached ''-output.txt'' artifacts predating the Pass 4 denominator fix; a hand-counted residue figure of 44 against a printed 47 |
+| What changed | both embedded output blocks regenerated; residue 44 → 47 |
+| Script changes | none |
+| Reviewers | one ''sonnet'' figures-vs-script pass |
+| Pages saved | this page only |