This is an old revision of the document!
Table of Contents
Security
This namespace is for measuring web security as it is deployed — TLS on public sites, the headers a crawl can see, the label sources people use for “malicious”, phishing sites in the wild, web vulnerabilities classified in papers that also measured the web. It is not a tutorial on attacks, and it is not the rest of computer security. The publication corpus behind these pages is seven broad security, privacy and measurement venues (CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P, 2010–2026, 5,859 extracted papers), so a keyword search for “security” is not a population. Each child page names its own.
A namespace page outlines the pages inside it rather than carrying its own content. 1) The five children below are proposed from this corpus: four have a schema population, and Headers has only a full-text upper bound until a later sitting hand-maps it. Web vulnerabilities, TLS certificates and Phishing are now written; Headers and VirusTotal remain red links until written; this page exists so a reader landing from start is not sent into an empty namespace. Three candidates were not given a page; Google Safe Browsing was folded into phishing rather than made a sixth — see Rejected, and why.
The five pages
| Page | What a fresh student needs it for | What the corpus can carry (2026-08-27) |
|---|---|---|
| TLS certificates | You are about to measure HTTPS, certificates, or CT logs, and need to know which instrument (active scan, CT log, Censys) answers which question. | 50 papers declare their population unit as certificates (22 of them on the web platform). 133 used a TLS-specific instrument (Censys, ZGrab, crt.sh, CT, sslyze, Qualys SSL, … — not OpenSSL-the-library). A looser full-text probe for TLS and certificates hits 525 papers, 213 web — an upper bound the child page has to narrow. Start with Holz et al. [1Holz, Ralph; Braun, Lothar; Kammenhuber, Nils; Carle, Georg (2011): "The SSL landscape: a thorough analysis of the X.509 PKI using active and passive measurements", in: Proceedings of the ACM Internet Measurement Conference. (DOI)], Durumeric et al. [2Durumeric, Zakir; Kasten, James; Bailey, Michael D.; Halderman, J. Alex (2013): "Analysis of the HTTPS certificate ecosystem", in: Proceedings of the ACM Internet Measurement Conference. (DOI)], Kotzias et al. [3Kotzias, Platon; Razaghpanah, Abbas; Amann, Johanna; Paterson, Kenneth G.; Vallina-Rodriguez, Narseo; Caballero, Juan (2018): "Coming of Age: A Longitudinal Study of TLS Deployment", in: Proceedings of the ACM Internet Measurement Conference. (DOI)], the Censys paper [4Durumeric, Zakir; Adrian, David; Mirian, Ariana; Bailey, Michael D.; Halderman, J. Alex (2015): "A Search Engine Backed by Internet-Wide Scanning", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)], Let's Encrypt [5Aas, Josh; Barnes, Richard; Case, Benton; Durumeric, Zakir; Eckersley, Peter; Flores-López, Alan; Halderman, J. Alex; Hoffman-Andrews, Jacob; Kasten, James; Rescorla, Eric; Schoen, Seth D.; Warren, Brad (2019): "Let's Encrypt: An Automated Certificate Authority to Encrypt the Entire Web", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)], and ten years of ZMap [6Durumeric, Zakir; Adrian, David; Stephens, Phillip; Wustrow, Eric; Halderman, J. Alex (2024): "Ten Years of ZMap", in: Proceedings of the ACM Internet Measurement Conference. (DOI)]. |
| Headers | You are about to crawl for CSP, HSTS, X-Frame-Options, SRI or Trusted Types, and need to know which of those are still worth measuring. | Full-text probe (Content-Security-Policy, CSP plus “header” or “directive”, HSTS, X-Frame-Options, SRI): 317 papers, 176 web. That 176 is an upper bound, not a population — it still contains bibliography hits and homographs. A schema match on header-ish tokens is not usable here — see the provenance page. The child page has to hand-map before any prevalence figure. Start with Weichselbaum et al. [7Weichselbaum, Lukas; Spagnuolo, Michele; Lekies, Sebastian; Janc, Artur (2016): "CSP Is Dead, Long Live CSP! On the Insecurity of Whitelists and the Future of Content Security Policy", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)], Roth et al. [8Roth, Sebastian; Barron, Timothy; Calzavara, Stefano; Nikiforakis, Nick; Stock, Ben (2020): "Complex security policy? A longitudinal analysis of deployed content security policies", in: Proceedings of the 27th Network and Distributed System Security Symposium (NDSS).], Steffens et al. [9Steffens, Marius; Musch, Marius; Johns, Martin; Stock, Ben (2021): "Who’s Hosting the Block Party? Studying Third-Party Blockage of CSP and SRI", in: Proceedings of the Network and Distributed System Security Symposium. (Link)]. |
| VirusTotal | You are about to label files, URLs or domains with VirusTotal, and need to know what a “detected” bit actually is. | 277 papers the extractor marked as used, produced, or drawing a population from VirusTotal (107 of them on the web platform). That is an upper bound on true use: the first sample of eight includes a references-only hit. 262 named it as a tool they used or produced; 254 of those as a classification-service. 64 distinct raw strings (products, feeds, thresholds and combinations, not merely spellings). Of the 209 that used it as a classifier, 86 targeted malware, 42 domains, 37 mobile apps, 16 website-category. The website-category use is already on website_classification; this page is the maliciousness-oracle use. Start with Peng et al. [10Peng, Peng; Yang, Limin; Song, Linhai; Wang, Gang (2019): "Opening the Blackbox of VirusTotal: Analyzing Online Phishing Scan Engines", in: Proceedings of the ACM Internet Measurement Conference. (DOI)]. |
| Phishing | You are about to crawl phishing sites or evaluate a feed, and need to know what the feed does not contain. | Schema union (detection, classification, population, or slug matching “phish”): 139 papers, 92 web. PhishTank is the named source in 21. A full-text /phish/ sweep hits 1,097 papers — that is a fact about these being security venues, not a population. Start with Zhang et al. [11Zhang, Penghui; Oest, Adam; Cho, Haehyun; Sun, Zhibo; Johnson, RC; Wardman, Brad; Sarker, Shaown; Kapravelos, Alexandros; Bao, Tiffany; Wang, Ruoyu; Shoshitaishvili, Yan; Doupé, Adam; Ahn, Gail-Joon (2021): "CrawlPhish: Large-scale Analysis of Client-side Cloaking Techniques in Phishing", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] (cloaking against anti-phishing crawlers) and Peng et al. [10Peng, Peng; Yang, Limin; Song, Linhai; Wang, Gang (2019): "Opening the Blackbox of VirusTotal: Analyzing Online Phishing Scan Engines", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] (VirusTotal's phishing engines). |
| Web vulnerabilities | You are about to look for XSS, CSRF, clickjacking or SOP bypass on live sites, and need methods and denominators — not the ethics checklist. | 880 papers classify a vulnerability; 209 of them on the web platform; 99 both crawled and web. The 99 is a paper-level conjunction — it does not by itself mean the crawl is how the vulnerability was found. 49.2% of the 880 are offline (program analysis of software). The child page's first job is to hand-map the web/crawled slice, not to treat 880 or 99 as a method count. Ethics of scanning live sites is ethics; telling the operator is notifying_websites. Start with Steffens et al. [12Steffens, Marius; Rossow, Christian; Johns, Martin; Stock, Ben (2019): "Don't Trust The Locals: Investigating the Prevalence of Persistent Client-Side Cross-Site Scripting in the Wild", in: Proceedings of the Network and Distributed System Security Symposium. (Link)]. |
Where this namespace stops
These pages already exist and already cover the overlapping question. Do not recreate them under security::
- ethics — the scanning-ethics checklist (Hantke et al.), acceptable-use, identifying your crawler. A “vulnerability scanning ethics” child was considered and rejected for this reason.
- notifying_websites — how to tell an operator, and the response rates in this corpus.
- website_classification — VirusTotal as a topic classifier (Vallina et al., vendor taxonomies, the public-API rate cap already dated there). VirusTotal is the other use of the same API.
- mobile_and_app_measurement — certificate pinning and TLS interception inside apps.
- ip_classification — reputation and geolocation of addresses, including some of the same feeds.
- javascript — script-level analysis; XSS-as-a-JavaScript-phenomenon belongs with Web vulnerabilities when the question is “is this live site exploitable”, and there when the question is “what did this script do”.
Rejected, and why
Three candidates from the original brief were measured and not given a page. Google Safe Browsing was measured too and folded into Phishing rather than rejected:
| Candidate | What the corpus showed | Why not a page |
|---|---|---|
| Vulnerability scanning ethics | Real topic, real papers (Hantke, Ramulu, Wu). | Already a section of ethics. A second copy would drift. |
| Malware datasets as a standalone page | 159 papers classify malware; 51 of them on the web platform. | Two thirds are not web measurement. The label source is VirusTotal; mobile malware is mobile_and_app_measurement. |
| A nuclei / nikto / w3af / openvas scanner-tool page | Full-text hits: 24 papers. | Too thin to carry a page. The web-vulnerabilities child can name them as a residue. |
Google Safe Browsing (full-text 110) sits on Phishing next to PhishTank, not as a sixth page: it is a feed, and the phishing page is about feeds.
Methodology and limitations of these figures
Every number in the table above is a count of papers, from the 5,859-paper extraction, with the denominator named in the same cell. Full-text probes read paper.cols.txt (4 of 5,859 have none and are counted as negatives). 2025–2026 venue-years are provisional — see corpus. The queries, the folds, the residue, and the unedited report output are on security.
- [1]
- Holz, Ralph; Braun, Lothar; Kammenhuber, Nils; Carle, Georg (2011): "The SSL landscape: a thorough analysis of the X.509 PKI using active and passive measurements", in: Proceedings of the ACM Internet Measurement Conference. (DOI)
- [2]
- Durumeric, Zakir; Kasten, James; Bailey, Michael D.; Halderman, J. Alex (2013): "Analysis of the HTTPS certificate ecosystem", in: Proceedings of the ACM Internet Measurement Conference. (DOI)
- [3]
- Kotzias, Platon; Razaghpanah, Abbas; Amann, Johanna; Paterson, Kenneth G.; Vallina-Rodriguez, Narseo; Caballero, Juan (2018): "Coming of Age: A Longitudinal Study of TLS Deployment", in: Proceedings of the ACM Internet Measurement Conference. (DOI)
- [4]
- Durumeric, Zakir; Adrian, David; Mirian, Ariana; Bailey, Michael D.; Halderman, J. Alex (2015): "A Search Engine Backed by Internet-Wide Scanning", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)
- [5]
- Aas, Josh; Barnes, Richard; Case, Benton; Durumeric, Zakir; Eckersley, Peter; Flores-López, Alan; Halderman, J. Alex; Hoffman-Andrews, Jacob; Kasten, James; Rescorla, Eric; Schoen, Seth D.; Warren, Brad (2019): "Let's Encrypt: An Automated Certificate Authority to Encrypt the Entire Web", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)
- [6]
- Durumeric, Zakir; Adrian, David; Stephens, Phillip; Wustrow, Eric; Halderman, J. Alex (2024): "Ten Years of ZMap", in: Proceedings of the ACM Internet Measurement Conference. (DOI)
- [7]
- Weichselbaum, Lukas; Spagnuolo, Michele; Lekies, Sebastian; Janc, Artur (2016): "CSP Is Dead, Long Live CSP! On the Insecurity of Whitelists and the Future of Content Security Policy", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)
- [8]
- Roth, Sebastian; Barron, Timothy; Calzavara, Stefano; Nikiforakis, Nick; Stock, Ben (2020): "Complex security policy? A longitudinal analysis of deployed content security policies", in: Proceedings of the 27th Network and Distributed System Security Symposium (NDSS).
- [9]
- Steffens, Marius; Musch, Marius; Johns, Martin; Stock, Ben (2021): "Who’s Hosting the Block Party? Studying Third-Party Blockage of CSP and SRI", in: Proceedings of the Network and Distributed System Security Symposium. (Link)
- [10]
- Peng, Peng; Yang, Limin; Song, Linhai; Wang, Gang (2019): "Opening the Blackbox of VirusTotal: Analyzing Online Phishing Scan Engines", in: Proceedings of the ACM Internet Measurement Conference. (DOI)
- [11]
- Zhang, Penghui; Oest, Adam; Cho, Haehyun; Sun, Zhibo; Johnson, RC; Wardman, Brad; Sarker, Shaown; Kapravelos, Alexandros; Bao, Tiffany; Wang, Ruoyu; Shoshitaishvili, Yan; Doupé, Adam; Ahn, Gail-Joon (2021): "CrawlPhish: Large-scale Analysis of Client-side Cloaking Techniques in Phishing", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)
- [12]
- Steffens, Marius; Rossow, Christian; Johns, Martin; Stock, Ben (2019): "Don't Trust The Locals: Investigating the Prevalence of Persistent Client-Side Cross-Site Scripting in the Wild", in: Proceedings of the Network and Distributed System Security Symposium. (Link)
