| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| statistics:regression [2026/08/21 08:34] – [Open Questions] karelkubicek | statistics:regression [2026/09/21 13:33] (current) – Quote-check bullet refreshed against quote_check.mjs --statistics regression with the pypdf fallback: 267/171/65/27 of 530. Authored by Claude karel.kubicek.claude |
|---|
| The corpus separates them, and the split is stark. **155 papers name a regression model in ''tools[]'' rather than ''statistics[]'', and 148 of those tuples are categorised ''ml-model-or-algorithm''.** **253 papers name one in ''statistics[]''. The two sets overlap in only 31 papers — so 124 papers use a regression purely as a classifier and never as an estimator.** If you searched the literature for "logistic regression in web measurement" and read the first ten hits, you would probably read ten baselines and learn nothing about inference. | The corpus separates them, and the split is stark. **155 papers name a regression model in ''tools[]'' rather than ''statistics[]'', and 148 of those tuples are categorised ''ml-model-or-algorithm''.** **253 papers name one in ''statistics[]''. The two sets overlap in only 31 papers — so 124 papers use a regression purely as a classifier and never as an estimator.** If you searched the literature for "logistic regression in web measurement" and read the first ten hits, you would probably read ten baselines and learn nothing about inference. |
| |
| <wrap todo> | <WRAP todo> |
| **Say which one you are doing, in the sentence that introduces the model.** "We fit a logistic regression of //banner present// on //rank, category and country// to estimate the association between …" and "we train a logistic-regression classifier as a baseline against our detector" are two different sentences, and a reader cannot tell them apart from a table of coefficients. Shmueli {[shmueli2010_explain]} is the reference if a reviewer pushes back on doing one and reporting the other. | **Say which one you are doing, in the sentence that introduces the model.** "We fit a logistic regression of //banner present// on //rank, category and country// to estimate the association between …" and "we train a logistic-regression classifier as a baseline against our detector" are two different sentences, and a reader cannot tell them apart from a table of coefficients. Shmueli {[shmueli2010_explain]} is the reference if a reviewer pushes back on doing one and reporting the other. |
| </wrap> | </WRAP> |
| |
| The rest of this page is about the first activity. Classifier evaluation belongs with [[Design:Website classification]] and [[Privacy:Javascript]]. | The rest of this page is about the first activity. Classifier evaluation belongs with [[Design:Website classification]] and [[Privacy:Javascript]]. |
| * Vu et al. {[vu2025_assessing]} (USENIX Sec 2025) combine the count model with a design: //"we modelled the weekly attack counts … using negative binomial regression - a well-established statistical technique for interrupted time series analysis"//. Weekly counts with a trend, seasonality and an intervention term is the shape of most "did the deadline change anything" questions in this field. | * Vu et al. {[vu2025_assessing]} (USENIX Sec 2025) combine the count model with a design: //"we modelled the weekly attack counts … using negative binomial regression - a well-established statistical technique for interrupted time series analysis"//. Weekly counts with a trend, seasonality and an intervention term is the shape of most "did the deadline change anything" questions in this field. |
| |
| <wrap todo> | <WRAP todo> |
| **If your count is "how many //X// out of an opportunity that varies", you need an offset, not a ratio.** Regressing //trackers per request// or //predatory ads as a share of ads// on covariates throws away the fact that a site with 300 requests and a site with 12 give you very different amounts of information. The count model's answer is a **log offset**: put the exposure on the right-hand side with its coefficient fixed at 1. Mai et al. {[mai2025_more]} do exactly this — //"To normalize the count of predatory ads, we add an offset term that is the log of the total ad count"// — and then report **rate ratios** rather than raw coefficients, which is the form a reader can act on. One paper in 5,859 does this.((Probe: ''offset'' in the regression context, read by hand in the papers the count-model query returned. A paper could use an offset without writing the word; this is a floor.)) | **If your count is "how many //X// out of an opportunity that varies", you need an offset, not a ratio.** Regressing //trackers per request// or //predatory ads as a share of ads// on covariates throws away the fact that a site with 300 requests and a site with 12 give you very different amounts of information. The count model's answer is a **log offset**: put the exposure on the right-hand side with its coefficient fixed at 1. Mai et al. {[mai2025_more]} do exactly this — //"To normalize the count of predatory ads, we add an offset term that is the log of the total ad count"// — and then report **rate ratios** rather than raw coefficients, which is the form a reader can act on. One paper in 5,859 does this.((Probe: ''offset'' in the regression context, read by hand in the papers the count-model query returned. A paper could use an offset without writing the word; this is a floor.)) |
| </wrap> | </WRAP> |
| |
| ==== Binary per-site outcomes ==== | ==== Binary per-site outcomes ==== |
| **Your sample size is the number of //events//, not the number of sites.** A crawl of 100,000 sites where 400 show the behaviour you are modelling gives a logistic regression 400 events, and the usual guidance is on the order of ten events per predictor — relaxable, but knowingly {[vittinghoff2007_relaxing]}. This is the failure mode of the rare-phenomenon paper: an enormous //n// and an unstable model. | **Your sample size is the number of //events//, not the number of sites.** A crawl of 100,000 sites where 400 show the behaviour you are modelling gives a logistic regression 400 events, and the usual guidance is on the order of ten events per predictor — relaxable, but knowingly {[vittinghoff2007_relaxing]}. This is the failure mode of the rare-phenomenon paper: an enormous //n// and an unstable model. |
| |
| <wrap todo> | <WRAP todo> |
| **A pseudo-R² of 0.07 is not a broken model, and saying so is not a weakness.** Kieserman et al. {[kieserman2025_tracker]} fit logistic regressions on 40,150 websites to explain which ones configure Google and Meta trackers to harvest form data, and write: //"It is important to acknowledge that our models are a relatively weak fit; the former has a pseudo R-squared of 0.0721"// … //"This means that their explanatory power is limited. However, we can still draw some useful insights from them"//. They report odds ratios, //p//-values and confidence intervals per feature. That is the model to copy. An inferential regression is not //primarily// judged on fit — a low pseudo-R² says the outcome has other causes, not that the coefficient is wrong — and pretending otherwise is what leads people to report a classifier's AUC in place of a coefficient. (Fit still matters when a reviewer's objection is an omitted variable; that is an argument about which covariates are in the model, not about R².) | **A pseudo-R² of 0.07 is not a broken model, and saying so is not a weakness.** Kieserman et al. {[kieserman2025_tracker]} fit logistic regressions on 40,150 websites to explain which ones configure Google and Meta trackers to harvest form data, and write: //"It is important to acknowledge that our models are a relatively weak fit; the former has a pseudo R-squared of 0.0721"// … //"This means that their explanatory power is limited. However, we can still draw some useful insights from them"//. They report odds ratios, //p//-values and confidence intervals per feature. That is the model to copy. An inferential regression is not //primarily// judged on fit — a low pseudo-R² says the outcome has other causes, not that the coefficient is wrong — and pretending otherwise is what leads people to report a classifier's AUC in place of a coefficient. (Fit still matters when a reviewer's objection is an omitted variable; that is an argument about which covariates are in the model, not about R².) |
| </wrap> | </WRAP> |
| |
| ==== Proportions, rates and ordinal scales ==== | ==== Proportions, rates and ordinal scales ==== |
| * **A paper counts once**, never once per mention. 530 tuples across 391 papers. | * **A paper counts once**, never once per mention. 530 tuples across 391 papers. |
| * **''statistics.kind'' is a mid-band extraction field.** Two independent runs over identical text agreed on it for about two-thirds of papers, so a repeat run would move these percentages by a few points. That agreement figure was measured on the **previous** corpus run and has not been re-measured on this one; treat it as an order of magnitude. | * **''statistics.kind'' is a mid-band extraction field.** Two independent runs over identical text agreed on it for about two-thirds of papers, so a repeat run would move these percentages by a few points. That agreement figure was measured on the **previous** corpus run and has not been re-measured on this one; treat it as an order of magnitude. |
| * **Quotes were checked in bulk and by hand.** All 530 regression quotes against the text the extractor read: **267 exact (50.4%) after whitespace normalisation, 170 partial (32.1%) at ≥60% of five-word windows, 93 (17.5%) below that**. Below-threshold is **not** "unsupported" — every quote published on this page was located by hand in ''paper.cols.txt'', including several that scored below threshold because the extractor dropped a citation marker or the two-column splice cut the sentence. Details, including which ones, on [[provenance:statistics:regression]]. | * **Quotes were checked in bulk and by hand.** All 530 regression quotes against the text the extractor read, and then — for anything below threshold — against an independent ''pypdf'' rendering of the same ''paper.pdf'': **267 exact (50.4%) after whitespace normalisation, 171 partial (32.3%) at ≥60% of five-word windows, 65 (12.3%) located only in the second rendering, 27 (5.1%) below threshold in both**. The 65 are a defect in the stored text, not in the extraction. Below-threshold is **not** "unsupported" either — every quote published on this page was located by hand in ''paper.cols.txt'', including several that scored below threshold because the extractor dropped a citation marker or the two-column splice cut the sentence. Details, including which ones, on [[provenance:statistics:regression]]. |
| * **''statistics'' has no ''not-stated'' sentinel**, so the reporting-gap figures above are bounds on the silence rather than rates of it. | * **''statistics'' has no ''not-stated'' sentinel**, so the reporting-gap figures above are bounds on the silence rather than rates of it. |
| * **The ''detail'' field is not a measurement of reporting practice.** It is a capped free-text field the extractor filled when a value happened to be in the quoted sentence. Counts of R², CIs and //p//-values inside it establish an ordering and nothing more. | * **The ''detail'' field is not a measurement of reporting practice.** It is a capped free-text field the extractor filled when a value happened to be in the quoted sentence. Counts of R², CIs and //p//-values inside it establish an ordering and nothing more. |
