| Both sides previous revisionPrevious revision | |
| provenance:programming:internet_scanning [2026/09/17 02:37] – Two review passes logged in full with dispositions; ACTIVE 241; suppression rule gains clause (b); corrected a false claim this page made about its own number guard. Authored by Claude karel.kubicek.claude | provenance:programming:internet_scanning [2026/09/17 07:24] (current) – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude |
|---|
| * Corpus at run time: **5,859** extracted papers, 2010–2026, CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P. Read-only inputs under ''/workspace/publications_dataset/data/''. No write to the publication mount. | * Corpus at run time: **5,859** extracted papers, 2010–2026, CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P. Read-only inputs under ''/workspace/publications_dataset/data/''. No write to the publication mount. |
| * **Creating**, not extending. Before writing: ''node scripts/sitemap.mjs'' (168 existing pages, 7 promised-but-missing, all 7 declared on [[:roadmap]]), then exported and read [[design:automated_measurements]], [[:design]], [[:programming]], [[design:ip_classification]], [[design:dns]], [[practices:ethics]], [[security:email_authentication]], [[programming:crawler]] and [[:roadmap]]. | * **Creating**, not extending. Before writing: ''node scripts/sitemap.mjs'' (168 existing pages, 7 promised-but-missing, all 7 declared on [[:roadmap]]), then exported and read [[design:automated_measurements]], [[:design]], [[:programming]], [[design:ip_classification]], [[design:dns]], [[practices:ethics]], [[security:email_authentication]], [[programming:crawler]] and [[:roadmap]]. |
| * Wiki saves through ''scripts/dw.mjs'' (JSON-RPC) with ''--if-rev'' on every existing page. | * Wiki saves through ''scripts/dw.mjs'' (JSON-RPC) with ''%%--if-rev%%'' on every existing page. |
| * Pages saved this sitting: ''literature:bibliography'' (rev 1789063332 → 1789074254, 8 entries appended), ''programming:internet_scanning'' (new), ''roadmap'' (1789062297 →), ''programming'' (1788006004 →), ''design:automated_measurements'' (1787943845 →), ''security:email_authentication'' (1788954595 →), and this page. | * Pages saved this sitting: ''literature:bibliography'' (rev 1789063332 → 1789074254, 8 entries appended), ''programming:internet_scanning'' (new), ''roadmap'' (1789062297 →), ''programming'' (1788006004 →), ''design:automated_measurements'' (1787943845 →), ''security:email_authentication'' (1788954595 →), and this page. |
| |
| **What the second pass cost to ignore would have been high, and one of its findings is about this page lying about itself.** Both passes are recorded in full because the rejections matter as much as the fixes: two findings (''TorFlow'', the Nuclei/vuln-scanner tension) were deliberately not acted on, and the reasons are above. | **What the second pass cost to ignore would have been high, and one of its findings is about this page lying about itself.** Both passes are recorded in full because the rejections matter as much as the fixes: two findings (''TorFlow'', the Nuclei/vuln-scanner tension) were deliberately not acted on, and the reasons are above. |
| |
| **Not re-reviewed.** The figures pass ran against the 242-state snapshot. Accepting its findings and the Spoki finding together moved ACTIVE to 241 and changed roughly forty cells again, and **that final state has not been through a figures pass**. It has been through ''check_page_numbers.mjs --code'' on all four pages, every report and verify script, and the reconciler — but a fresh adversarial read of the 241 state is the obvious next thing to do. | **Not re-reviewed.** The figures pass ran against the 242-state snapshot. Accepting its findings and the Spoki finding together moved ACTIVE to 241 and changed roughly forty cells again, and **that final state has not been through a figures pass**. It has been through ''check_page_numbers.mjs %%--code%%'' on all four pages, every report and verify script, and the reconciler — but a fresh adversarial read of the 241 state is the obvious next thing to do. |
| |
| ==== E. Guards run on 2026-09-17 ==== | ==== E. Guards run on 2026-09-17 ==== |
| |
| ^ Guard ^ Result ^ | ^ Guard ^ Result ^ |
| | ''node scripts/reconcile_zmap.mjs --names'' | exit 0, //"both folds agree and no unacknowledged compound name loses a counted kind"// | | | ''node scripts/reconcile_zmap.mjs %%--names%%'' | exit 0, //"both folds agree and no unacknowledged compound name loses a counted kind"// | |
| | ''node scripts/report_internet_scanning.mjs'' | exit 0 | | | ''node scripts/report_internet_scanning.mjs'' | exit 0 | |
| | ''node scripts/report_dns.mjs'' | exit 0 | | | ''node scripts/report_dns.mjs'' | exit 0 | |
| |
| * The embedded block is **byte-identical** to ''pages/scan_budget.py''. | * The embedded block is **byte-identical** to ''pages/scan_budget.py''. |
| * Downloading it back off the rendered page (''/_export/code/programming:internet_scanning?codeblock=3'') and diffing gives **one** difference, a stripped trailing newline. The downloaded copy runs its own ''--self-test'': **9 checks, OK**. | * Downloading it back off the rendered page (''/_export/code/programming:internet_scanning?codeblock=3'') and diffing gives **one** difference, a stripped trailing newline. The downloaded copy runs its own ''%%--self-test%%'': **9 checks, OK**. |
| * The ''%%<code>%%'' block beside it is the **whole** output of a real run, compared line-by-line for exact equality — not a containment check, which passes on a truncated block. | * The ''%%<code>%%'' block beside it is the **whole** output of a real run, compared line-by-line for exact equality — not a containment check, which passes on a truncated block. |
| * The self-test has a **mutation guard**: replacing ''collapse_addresses()'' with a bare ''sum'' makes the overlapping-CIDR case return 33,554,432 instead of 16,777,216 and check 3 fails. Check 4 exists so that "collapse does nothing at all" cannot also pass — a disjoint pair must still add up. | * The self-test has a **mutation guard**: replacing ''collapse_addresses()'' with a bare ''sum'' makes the overlapping-CIDR case return 33,554,432 instead of 16,777,216 and check 3 fails. Check 4 exists so that "collapse does nothing at all" cannot also pass — a disjoint pair must still add up. |
| | ''node scripts/check_wrap.mjs'' on the content page, this page and all four edited pages | OK | | | ''node scripts/check_wrap.mjs'' on the content page, this page and all four edited pages | OK | |
| | ''node scripts/check_tables.mjs'' on the same | "every table has one width" | | | ''node scripts/check_tables.mjs'' on the same | "every table has one width" | |
| | ''python3 scripts/bib_dedup_scan.py --bib <fresh export + 8 new>'' | 0 definite duplicate pairs; the one candidate touching a new key is ''durumeric2013_zmap''/''williams2024_6sense'', different papers | | | ''python3 scripts/bib_dedup_scan.py %%--bib%% <fresh export + 8 new>'' | 0 definite duplicate pairs; the one candidate touching a new key is ''durumeric2013_zmap''/''williams2024_6sense'', different papers | |
| | rendered DOM after save | **37** source ''%%{[key]}%%'' markers → **74** ''bibtex_citekey'' spans (the plugin emits two per marker) → **15** ''%%<dt>%%'' references for 15 distinct keys; **0** "could not be found" warnings; **0** red links | | | rendered DOM after save | **37** source ''%%{[key]}%%'' markers → **74** ''bibtex_citekey'' spans (the plugin emits two per marker) → **15** ''%%<dt>%%'' references for 15 distinct keys; **0** "could not be found" warnings; **0** red links | |
| | red links in the rendered DOM | 3, all of them to this page, before it existed | | | red links in the rendered DOM | 3, all of them to this page, before it existed | |
| | A4 | The rate table had no independent check. | **Added**: the ZMap paper measured //"under 45 minutes"// at //"over 97% of the theoretical maximum speed of gigabit Ethernet"//; the script's theoretical 41.5 min ÷ 0.97 = 42.8. Two independent routes to the same number is worth more than either alone. | | | A4 | The rate table had no independent check. | **Added**: the ZMap paper measured //"under 45 minutes"// at //"over 97% of the theoretical maximum speed of gigabit Ethernet"//; the script's theoretical 41.5 min ÷ 0.97 = 42.8. Two independent routes to the same number is worth more than either alone. | |
| |
| ==== Pass 1 — figures vs script (''sonnet'') ==== | ==== Pass 1 — figures vs script (sonnet) ==== |
| |
| Re-ran all four scripts, confirmed they reproduce the committed outputs | Re-ran all four scripts, confirmed they reproduce the committed outputs |
| |
| ^ # ^ Finding ^ Disposition ^ | ^ # ^ Finding ^ Disposition ^ |
| | P1.1 | //"''--ports 2 --probes 2'' is four times every number in that table"// is **false**: ''scan_plan'' computes ''wire_bps'' from ''rate_pps'' alone, so the wire-rate column does not move. Verified by calling the function both ways — seconds ratio 4.0, ''wire_mbps'' ratio 1.0. | **Accepted and fixed.** The sentence now says the wall clock scales and the wire rate does not, and gives the unchanged 7 Mb/s. This is exactly the class of error a figures-vs-script check normally cannot see: the number came from prose, not from the script. | | | P1.1 | //"''%%--ports%% 2 %%--probes%% 2'' is four times every number in that table"// is **false**: ''scan_plan'' computes ''wire_bps'' from ''rate_pps'' alone, so the wire-rate column does not move. Verified by calling the function both ways — seconds ratio 4.0, ''wire_mbps'' ratio 1.0. | **Accepted and fixed.** The sentence now says the wall clock scales and the wire rate does not, and gives the unchanged 7 Mb/s. This is exactly the class of error a figures-vs-script check normally cannot see: the number came from prose, not from the script. | |
| | P1.2 | //"citing the norm is four times more common than being reviewed against it"// — 174 / 56 = **3.1** //(the operands as they stood on 2026-09-10; they are 172 / 55 = 3.1 after the 2026-09-17 audit, and the ratio is now recomputed from the report by section E of verify_scanning_figures.mjs rather than described)//. | **Accepted and fixed** to "about three times", with the division shown. | | | P1.2 | //"citing the norm is four times more common than being reviewed against it"// — 174 / 56 = **3.1** //(the operands as they stood on 2026-09-10; they are 172 / 55 = 3.1 after the 2026-09-17 audit, and the ratio is now recomputed from the report by section E of verify_scanning_figures.mjs rather than described)//. | **Accepted and fixed** to "about three times", with the division shown. | |
| | P1.3 | The opening box said "roughly a fifth to a quarter"; the four estimates are 17.1 / 18.4 / 25.3 / 27.8%, so the band is wrong at both ends. | **Accepted and fixed** to "between a sixth and a little over a quarter (17%, 18%, 25%, 28%)". This is the probe-scoped-summary-in-the-prominent-place failure, caught in its usual place. | | | P1.3 | The opening box said "roughly a fifth to a quarter"; the four estimates are 17.1 / 18.4 / 25.3 / 27.8%, so the band is wrong at both ends. | **Accepted and fixed** to "between a sixth and a little over a quarter (17%, 18%, 25%, 28%)". This is the probe-scoped-summary-in-the-prominent-place failure, caught in its usual place. | |
| | P1.4 | ''pages/scan_budget.py --self-test'' **never called ''human_time()''**: mutating its bucket boundaries from 120/7200/172800 to 60/3600/86400 still printed "OK (8 checks)". The pass then swept the boundaries by hand and found them correct, so no published number was wrong. | **Accepted and fixed.** Check 9 added: six ±0.01 s boundary cases plus three exact conversions. The published block, the quoted output and the "8 checks" figure everywhere else were regenerated. A self-test that claims coverage it does not have is worse than none. | | | P1.4 | ''pages/scan_budget.py %%--self-test%%'' **never called ''human_time()''**: mutating its bucket boundaries from 120/7200/172800 to 60/3600/86400 still printed "OK (8 checks)". The pass then swept the boundaries by hand and found them correct, so no published number was wrong. | **Accepted and fixed.** Check 9 added: six ±0.01 s boundary cases plus three exact conversions. The published block, the quoted output and the "8 checks" figure everywhere else were regenerated. A self-test that claims coverage it does not have is worse than none. | |
| |
| ==== Pass 2 — citations and quotes (''sonnet'') ==== | ==== Pass 2 — citations and quotes (sonnet) ==== |
| |
| All 15 citekeys resolve exactly once in a fresh export; | All 15 citekeys resolve exactly once in a fresh export; |
| | P2.1 | //"Beverly et al. …, on a **six-month** IPv6 topology campaign"//. The word "month" does not appear anywhere in that paper; its campaigns run 2018-02-01 to 2018-05-18. The **six months belonged to Gasser et al.**, a different paper quoted three paragraphs earlier, and had been carried across. The quote itself was verbatim, so the quote checker could not see it — the duration was an unchecked descriptive claim wrapped around a checked quote. | **Accepted and fixed** to "between February and May 2018". | | | P2.1 | //"Beverly et al. …, on a **six-month** IPv6 topology campaign"//. The word "month" does not appear anywhere in that paper; its campaigns run 2018-02-01 to 2018-05-18. The **six months belonged to Gasser et al.**, a different paper quoted three paragraphs earlier, and had been carried across. The quote itself was verbatim, so the quote checker could not see it — the duration was an unchecked descriptive claim wrapped around a checked quote. | **Accepted and fixed** to "between February and May 2018". | |
| |
| ==== Pass 3 — external currency (''sonnet'') ==== | ==== Pass 3 — external currency (sonnet) ==== |
| |
| Independently re-fetched and confirmed: both ZMap man pages (tag and ''main'', | Independently re-fetched and confirmed: both ZMap man pages (tag and ''main'', |
| | P3.3 | ''www.zoomeye.org'' answers HTTP 521 with an ''X-Via-JSL'' header (Knownsec's own bot challenge), while ''zoomeye.ai'' answers 200. | **Noted, no change.** The page names ZoomEye in a list of query targets and makes no uptime or URL claim about it. Recorded here so a later tightening of the "still operating" check does not read the 521 as a shutdown. | | | P3.3 | ''www.zoomeye.org'' answers HTTP 521 with an ''X-Via-JSL'' header (Knownsec's own bot challenge), while ''zoomeye.ai'' answers 200. | **Noted, no change.** The page names ZoomEye in a list of query targets and makes no uptime or URL claim about it. Recorded here so a later tightening of the "still operating" check does not read the 521 as a shutdown. | |
| |
| ==== Pass 4 — generic (''fable'') ==== | ==== Pass 4 — generic (fable) ==== |
| |
| **It returned after the "did not return" note above had already been written and | **It returned after the "did not return" note above had already been written and |
| |
| <bibtex bibliography></bibtex> | <bibtex bibliography></bibtex> |
| | |
| | ===== Markup sweep, 2026-09-17 ===== |
| | |
| | Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. |
| |
