Differences

This shows you the differences between two versions of the page.

--- design:website_classification [2026/09/11 03:00] – Point at the new statistics:annotation page, which owns the corpus-wide validation figures this page's section cannot carry. Authored by Claude karel.kubicek.claude
+++ design:website_classification [2026/09/21 13:45] (current) – Quote bullet: the 6 unmatched LLM tuples are all located in a pypdf rendering; none fails in both renderings. Authored by Claude karel.kubicek.claude
@@ Line 1130: / Line 1130: @@
   * **The task fold is imperfect and its exclusions are visible.** Eight side-channel and website-fingerprinting papers named their classifier only as "a 1D CNN" or "k-NN", leaving no signal in any extracted field; they were caught by matching the paper's own slug and are excluded by an explicit named list rather than by a general rule. The 94 excluded papers are counted in the table above rather than dropped.
   * **Enum fields versus free text.** Method and validation are enums, stable enough to publish as rough shares (''classification.method'' agrees 58% run-to-run, so read those as a ranking). Service names and taxonomies are free text and are reported as rankings and folded families only.
-  * **Quotes were checked.** Every figure above traces to tuples carrying a verbatim evidence quote; a sample of these was re-located in the source PDFs. Of six quotes checked by hand, two initially "failed" a literal grep and turned out to be intact but split across a two-column break — normalise whitespace before concluding that a quote is not in the paper. The five LLM website-category tuples were re-checked individually on 2026-08-12 and again on 2026-09-03, this time by a committed script rather than by hand: ''scripts/llm_currency_quotecheck.mjs'' checks all 30 LLM tuples across the targets the three classification pages make claims about, and **6 of the 30 could not be matched even after allowing for elisions, punctuation, the PDF's math-italic glyphs and dropped citation markers**. All six are ''privacy-policy'' or ''consent-notice'' tuples that no page on this site quotes; every tuple behind a figure on //this// page matched. Its unedited output is on [[provenance:design:website_classification]].
+  * **Quotes were checked.** Every figure above traces to tuples carrying a verbatim evidence quote; a sample of these was re-located in the source PDFs. Of six quotes checked by hand, two initially "failed" a literal grep and turned out to be intact but split across a two-column break — normalise whitespace before concluding that a quote is not in the paper. The five LLM website-category tuples were re-checked individually on 2026-08-12 and again on 2026-09-03, this time by a committed script rather than by hand: ''scripts/llm_currency_quotecheck.mjs'' checks all 30 LLM tuples across the targets the three classification pages make claims about. **Six of the 30 could not be matched against the rendering the extractor read, even after allowing for elisions, punctuation, the PDF's math-italic glyphs and dropped citation markers — and all six are located in an independent ''pypdf'' rendering of the same PDF, so none fails in both renderings.** The six are ''privacy-policy'' or ''consent-notice'' tuples that no page on this site quotes; every tuple behind a figure on //this// page matched against the stored rendering already. Its unedited output is on [[provenance:design:website_classification]].
   * **The 2025–2026 reversal was stress-tested, not just re-derived.** The two composition controls in [[#How they classify]] are computed by the same report script and printed with their bucket sizes. The direction survives both; the magnitude is not established. See the box there for what would settle it.
   * **Coverage.** Seven venues only, with 2025 and 2026 incomplete by construction rather than by relevance, so any row that reaches them is a floor; the scope and the selection funnel are on [[literature:corpus]]. Notably, **{[vallina2020_misshapes]} itself is in the venue index but has no extracted full text** — the reference work for this page is not in the population the page measures.