Differences

This shows you the differences between two versions of the page.

--- design:ip_classification [2026/09/11 02:31] – Point at the new statistics:annotation page from the validation section and Related Pages. Authored by Claude karel.kubicek.claude
+++ design:ip_classification [2026/09/22 22:26] (current) – Fable review 2026-09-22: classify_ips.py fixed for ipapi.is's 2026-09-01 keyless change (flags now need a free key) and output re-embedded; validation comparator made like-for-like (33.6% vs 60.1% of cells); vendor rows re-checked (MaxMind rename, IP2Loca karel.kubicek.claude
@@ Line 86: / Line 86: @@
 ^ Period ^ Position ^ Where it comes from ^
 | 2004–2009 | **Measure it yourself.** Convert latency from landmarks into distance constraints and multilaterate. | Constraint-Based Geolocation {[gueye2004_cbg]} and its successors (Octant, Spotter). Still the reference technique. |
-| 2011–2016 | **Databases are unreliable.** | {[poese2011_unreliable]} ground-truthed several databases against a large European ISP and found entries that do not reflect actual BGP allocations, concentrated in a few popular countries, and — the counter-intuitive part — that "overly fine granularity of database entries makes their accuracy worse, not better". {[shavitt2011_geodb]} found none of the databases close to its claimed country-level accuracy, with MaxMind disproportionately defaulting ambiguous addresses to the US. |
+| 2011–2016 | **Databases are unreliable.** | {[poese2011_unreliable]} ground-truthed several databases against a large European ISP and found entries that do not reflect actual BGP allocations, concentrated in a few popular countries, and — the counter-intuitive part — that "overly fine granularity of database entries makes their accuracy worse, not better". {[shavitt2011_geodb]} found the databases mostly agreeing with each other yet still placing some addresses thousands of kilometres and whole countries away, and some collapsing most of an ISP's addresses onto its headquarters. |
 | 2017–2020 | **Some databases are much worse than others, and routers are the hard case.** | {[gharaibeh2017_look]}, below. |
 | 2023–2024 | **The gap between vendors now exceeds the gap between vendors and research techniques.** Also: the databases quietly stopped being pure guesswork and started ingesting operator-published data. | {[darwich2023_replication]}, below; {[livadariu2024_geofeeds]} on geofeeds. |
@@ Line 120: / Line 120: @@
 > For all databases, about one third of the inconsistencies were the database claiming a vantage point was hosted in the US when the claimed location was elsewhere.
-That is the same US-default bias {[shavitt2011_geodb]} identified in 2011, still visible in 2018. If your result is "//n// addresses are in the United States", check how many of them are simply the database's fallback.
+That is a default, not a measurement — the same pattern as the headquarters defaulting {[shavitt2011_geodb]} described in 2011, still visible in 2018. If your result is "//n// addresses are in the United States", check how many of them are simply the database's fallback.
 ==== Aggregate accuracy hides who it fails for ====
@@ Line 151: / Line 151: @@
 ==== Which geolocation source to use in 2026 ====
-Checked against each vendor's own documentation on **6 August 2026**. Dated deliberately, because this is the part of the page that goes stale first.
+Checked against each vendor's own documentation on **6 August 2026** and re-checked on **22 September 2026**. Dated deliberately, because this is the part of the page that goes stale first: the second check found that MaxMind had already dropped the "2" from its product names (EULA of 12 February 2026, missed on 6 August) and that IP2Location states its licence two ways.
 ^ Source ^ What you get free ^ Status ^ Notes ^
-| **MaxMind GeoLite2** | Country, City (incl. ''accuracy_radius'') and ASN databases, under the GeoLite2 EULA plus CC BY-SA 4.0 for redistribution | Current, but **since 30 December 2019 you need an account and a licence key** to download | The field's default by inertia. MaxMind's own stated reason for the 2019 change was that GDPR and CCPA restricted their "ability to continue distributing our GeoLite databases on a public page". Measured at 55% within 40 km in 2023 {[darwich2023_replication]}. No committed release cadence — their docs say to check for a new release periodically through the day — so "we used MaxMind" without a date is not reproducible. |
+| **MaxMind GeoLite** (GeoLite2 until MaxMind's 2026 rebrand; the corpus uses the old name) | Country, City (incl. ''accuracy_radius'') and ASN databases, under the GeoLite EULA plus CC BY-SA 4.0 for redistribution | Current, but **since 30 December 2019 you need an account and a licence key** to download | The field's default by inertia. MaxMind's own stated reason for the 2019 change was that GDPR and CCPA restricted their "ability to continue distributing our GeoLite databases on a public page". Measured at 55% within 40 km in 2023 {[darwich2023_replication]}. No committed release cadence — their docs say to check for a new release periodically through the day — so "we used MaxMind" without a date is not reproducible. |
-| **MaxMind GeoIP2 / Insights** | — (paid: GeoIP City listed at $148/month or $1,628/year on 2026-08-06, Country at $38/month) | Current | Same ''accuracy_radius'' field. **Use it**: it is the vendor telling you not to trust the point. Note the listed price covers internal use only; a commercial licence is separate. |
+| **MaxMind GeoIP / Insights** (formerly GeoIP2) | — (paid: GeoIP City listed at $148/month or $1,628/year on 2026-08-06 and unchanged on 2026-09-22, Country at $38/month) | Current | Same ''accuracy_radius'' field. **Use it**: it is the vendor telling you not to trust the point. Note the listed price covers internal use only; a commercial licence is separate. |
 | **IPinfo** | The free "Lite" database is **country and ASN only**, unlimited, CC BY-SA with attribution. City-level and privacy detection start at the paid tier. | Current | Best performer in the only recent head-to-head in the corpus {[darwich2023_replication]} — but note that measurement used IPinfo's //free API// as it stood in 2023, and the free tier has been restructured since. Check what you are actually getting. |
-| **IP2Location LITE** | Country/city/ASN under CC BY-SA 4.0 with mandatory attribution | Current | Its Lite tier tied with NetAcuity for //coverage// in {[gharaibeh2017_look]} — "near perfect", against MaxMind's 43% and 61.6% at city level. Coverage is not accuracy; on accuracy it sat with MaxMind at 77.5–78.6%. |
+| **IP2Location LITE** | Country/city/ASN, free for personal or commercial use with mandatory attribution. The licence is stated two ways: the DB1 country page carries a CC BY-SA 4.0 badge, while the LITE data-licence terms say you may not redistribute or resell the database (both seen 2026-09-22). Read the terms before publishing a derived dataset. | Current | Its Lite tier tied with NetAcuity for //coverage// in {[gharaibeh2017_look]} — "near perfect", against MaxMind's 43% and 61.6% at city level. Coverage is not accuracy; on accuracy it sat with MaxMind at 77.5–78.6%. |
-| **DB-IP Lite** | Country, City and ASN databases, monthly, CC Attribution | Current | DB-IP describes the Lite tiers as "subsets of the commercial database with reduced coverage and accuracy" — so it is a third opinion, not a third measurement. Least-used of the four in the corpus. |
+| **DB-IP Lite** | Country, City and ASN databases, monthly, CC Attribution | Current | DB-IP describes the Lite tiers as "subsets of the commercial databases with reduced coverage and accuracy" — so it is a third opinion, not a third measurement. Least-used of the four in the corpus. |
 | **NetAcuity (Digital Element)** | — (paid, enterprise, sales-gated) | Current | Best accuracy in {[gharaibeh2017_look]} (2017). We found no academic-access programme; 5 papers in the corpus. |
 | **RIPE IPmap** | Latency and reverse-DNS engines over RIPE Atlas, open API {[du2020_ipmap]} | Current | Infrastructure addresses (routers, IXPs) rather than arbitrary hosts. |
@@ Line 164: / Line 164: @@
 | **Quova, Neustar IP Intelligence, HostIP.info, freegeoip.net, IPInfoDB** | — | **Historical.** Present in the corpus; superseded or gone | If a paper's method rests on one of these, that paper is telling you about 2012. |
-Vendor accuracy claims are worth reading precisely because they are more modest than the reputation. MaxMind's own support documentation puts GeoIP2 at about **99.8% country-level**, but for the United States only about **80% at state level and 66% within 50 km of the correct city**. Nobody who reads that page would then publish an unqualified city-level claim.
+Vendor accuracy claims are worth reading precisely because they are more modest than the reputation. MaxMind's own support documentation puts its GeoIP products at about **99.8% country-level**, but for the United States only about **80% at state level and 66% within 50 km of the correct city**. Nobody who reads that page would then publish an unqualified city-level claim.
 ===== What Kind of Host Is It =====
@@ Line 172: / Line 172: @@
 The most accurate datacenter detection is free, authoritative and almost unused in this literature: **the cloud and CDN operators publish their own prefix lists.** Grepping the full text of all 5,859 papers in the corpus for the URLs below finds them in **seven**.((Full-text grep of `paper.cols.txt` for `ip-ranges.amazonaws`, `gstatic.com/ipranges`, `cloudflare.com/ips`, `public_ip_ranges.json` or `ServiceTags_Public`, re-run 2026-08-12 against the extended corpus. Seven papers, all network-measurement rather than web-privacy; the seventh, added with the 2025 venue-year, is //Secure IP Address Allocation at Cloud Scale// (NDSS 2025). This undercounts papers that used a list without citing its URL, so read it as a floor.))
-All of the following were fetched and confirmed serving on 6 August 2026:
+All of the following were fetched and confirmed serving on 6 August 2026, and again on 22 September 2026:
   * **AWS**: ''https://ip-ranges.amazonaws.com/ip-ranges.json'' — carries ''service'' //and// ''region'' per prefix, so it also geolocates an AWS address better than any database does.
@@ Line 180: / Line 180: @@
   * **Oracle Cloud**: ''https://docs.oracle.com/en-us/iaas/tools/public_ip_ranges.json''.
   * **Azure**: there is **no stable JSON URL** — the filename rotates weekly (''ServiceTags_Public_YYYYMMDD.json''). Link the [[https://www.microsoft.com/en-us/download/details.aspx?id=56519|download landing page]] in your paper, or use the Service Tag Discovery REST API, which lags by up to about four weeks. Do not hard-code a dated filename.
-  * **DigitalOcean** has no documented ranges page; a first-party CSV exists but 404s intermittently. Treat it as unreliable.
+  * **DigitalOcean** has no documented ranges page. An undocumented first-party CSV exists (''https://www.digitalocean.com/geo/google.csv''); this page's first version reported it 404ing intermittently, and it served on every fetch on 2026-09-22. A file nobody documents can change without notice, so treat it as unreliable.
-  * **Tor**: the Tor Project publishes the [[https://check.torproject.org/torbulkexitlist|bulk exit list]], which is definitive for current Tor exits in a way that no commercial ''is_tor'' flag is. For historical questions ("was this address a Tor exit on 12 March 2024?") use [[https://metrics.torproject.org/exonerator.html|ExoneraTor]] — the older TorDNSEL service was retired in April 2020.
+  * **Tor**: the Tor Project publishes the [[https://check.torproject.org/torbulkexitlist|bulk exit list]], which is definitive for current Tor exits in a way that no commercial ''is_tor'' flag is. For historical questions ("was this address a Tor exit on 12 March 2024?") use [[https://metrics.torproject.org/exonerator.html|ExoneraTor]]. The DNS exit list answers for current exits only; its old implementation was retired in April 2020 and replaced by one that returns ''127.0.0.2'' for an exit.
 If an address is in AWS's own file, it is AWS. No vendor score improves on that. Note the limit, which the script output below demonstrates: these files cover what the operator wants you to route to. Google's ''cloud.json'' is GCP customer space and does **not** contain 8.8.8.8; Cloudflare's ''ips-v4'' is their proxy range and does **not** contain 1.1.1.1.
@@ Line 191: / Line 191: @@
 Beyond the published ranges you are buying an opinion. Checked against vendor documentation on 6 August 2026:
-  * **MaxMind GeoIP2 Anonymous IP** — booleans named exactly ''is_anonymous'', ''is_anonymous_vpn'', ''is_hosting_provider'', ''is_public_proxy'', ''is_residential_proxy'', ''is_tor_exit_node''. Paid. Note that MaxMind's separate //Connection Type// database is not this: it gives one categorical field (Cable/DSL, Cellular, Corporate, Satellite), which is useful and is often confused with hosting detection.
+  * **MaxMind GeoIP Anonymous IP** (formerly GeoIP2 Anonymous IP) — booleans named exactly ''is_anonymous'', ''is_anonymous_vpn'', ''is_hosting_provider'', ''is_public_proxy'', ''is_residential_proxy'', ''is_tor_exit_node''. Paid. Note that MaxMind's separate //Connection Type// database is not this: it gives one categorical field (Cable/DSL, Cellular, Corporate, Satellite), which is useful and is often confused with hosting detection.
-  * **IPinfo Privacy Detection** — ''hosting'', ''proxy'', ''tor'', ''relay'', ''vpn'', ''service''. Paid (not in the free Lite tier); residential-proxy identification is higher still.
+  * **IPinfo Privacy Detection** — ''hosting'', ''proxy'', ''tor'', ''relay'', ''vpn'', ''service'' in the standalone [[https://ipinfo.io/developers/privacy-standard-api|Privacy Detection API]]; IPinfo's bundled tiers return the same verdicts under different field names, so check the response you actually get. Paid (not in the free Lite tier); residential-proxy identification is higher still.
-  * **IP2Proxy** — including a free LITE tier; distinguishes VPN, Tor, public proxy, web proxy, hosting/datacenter, search-engine bot, residential proxy, consumer privacy network.
+  * **IP2Proxy** — the commercial edition distinguishes VPN, Tor, public proxy, web proxy, hosting/datacenter, search-engine bot, residential proxy, consumer privacy network and enterprise private network. The free LITE tier covers public proxies (''PUB'') only.
-  * **ipapi.is** — returns ''is_datacenter''/''is_vpn''/''is_proxy''/''is_tor''/''is_abuser'' on a **free, keyless** query (1,000/day), which makes it the easiest thing to cross-check against. **ip-api.com** ships ''proxy''/''hosting''/''mobile'' free too, at 45 requests/minute and **non-commercial use only**.
+  * **ipapi.is** — returns ''is_datacenter''/''is_vpn''/''is_proxy''/''is_tor''/''is_abuser'' with a **free API key** (1,000 lookups/day), which makes it the easiest thing to cross-check against. Until **1 September 2026** it returned them keylessly, which is what this page's 2026-08-06 run used; a keyless query now gets ownership and location only, 30 a day. **ip-api.com** ships ''proxy''/''hosting''/''mobile'' free too, at 45 requests/minute and **non-commercial use only**.
   * **Spur**, **IPQualityScore**, **Udger** (which publishes a free browsable datacenter list), **IPHub**, **Scamalytics** — commercial, mostly sales-gated.
   * **GreyNoise** covers Internet-wide scanners specifically, and — unusually — runs a free [[https://docs.greynoise.io/docs/vip-program|Research Community programme]] giving students and independent researchers non-commercial access to the full enterprise API. If you need this class of data for a paper, ask them before you pay anyone.
@@ Line 201: / Line 201: @@
 <WRAP important>
-**These feeds have false positives you can see with one command.** In the run below, ''ipapi.is'' flags **8.8.8.8** — Google Public DNS — as ''is_vpn'' and ''is_abuser'', and **1.1.1.1** as ''is_vpn'' and ''is_abuser''. Both are public resolvers. If you filter your dataset on such a flag, say which vendor's flag, on what date, and spot-check the exclusions by hand. A vendor's risk score is tuned for fraud prevention, where a false positive costs a declined transaction; it is not tuned for your denominator.
+**These feeds have false positives you can see with one command.** In this page's 2026-08-06 run of the previous version of the script below, ''ipapi.is'' flagged **8.8.8.8** — Google Public DNS — as ''is_vpn'' and ''is_abuser'', and **1.1.1.1** as ''is_vpn'' and ''is_abuser'' (that output is kept on [[provenance:design:ip_classification]]; reproducing it now needs a free ipapi.is key). Both are public resolvers. If you filter your dataset on such a flag, say which vendor's flag, on what date, and spot-check the exclusions by hand. A vendor's risk score is tuned for fraud prevention, where a false positive costs a declined transaction; it is not tuned for your denominator.
 </WRAP>
@@ Line 215: / Line 215: @@
   * It is an **active** technique. You are probing third-party hosts; that belongs in your ethics section.
-  * The threshold may be fragile in exactly the conditions a real client has. {[chiapponi2023_poster]} probed the RTT delta from one laptop and one handset in France, 20 connections per configuration, and found the handset on 4G at **51.5 ± 14.5 ms** — over a 50 ms threshold with no proxy present — while a browser rather than a script, or a Wi-Fi hotspot, each added delay without individually crossing it. It is a two-page poster on a single carrier (SFR), and the authors themselves attribute the 4G result to that carrier running a transparent proxy, so read it as a warning to calibrate on your own population rather than as a general false-positive rate.
+  * The threshold may be fragile in exactly the conditions a real client has. {[chiapponi2023_poster]} probed the RTT delta from one laptop and one handset in France, 20 connections per configuration, and found the handset on 4G at **51.5 ± 14.5 ms** — over a 50 ms threshold with no proxy present — while a browser rather than a script, or a Wi-Fi hotspot, each added delay without individually crossing it. It is a two-page poster on a single carrier (SFR), and the authors themselves suggest the 4G result reflects that carrier running mobile TCP-terminating proxies (MTTPs), so read it as a warning to calibrate on your own population rather than as a general false-positive rate.
 Earlier and complementary: {[weinberg2018_catch]} used active geolocation to show that 638 of 2,269 advertised proxy locations were false, 401 of them not even on the right continent. The line continues into the corpus's newest venue-year — Ali et al., [[https://www.ndss-symposium.org/ndss-paper/beyond-rtt-an-adversarially-robust-two-tiered-approach-for-residential-proxy-detection/|"Beyond RTT: An Adversarially Robust Two-Tiered Approach for Residential Proxy Detection"]] (NDSS 2026), is explicitly a response to the fragility above — so check for something newer still before you build your own.
@@ Line 237: / Line 237: @@
 ===== A Script =====
-Three layers, most authoritative first: routing from Team Cymru, ownership from the operators' own published prefixes, and only then the geolocation estimates — printed side by side so disagreement is visible rather than averaged away. Standard library only, no API keys.
+Three layers, most authoritative first: routing from Team Cymru, ownership from the operators' own published prefixes, and only then the geolocation estimates — printed side by side so disagreement is visible rather than averaged away. Standard library only; no API key needed, though a free ipapi.is key in ''IPAPI_IS_KEY'' adds its risk flags.
 <file python classify_ips.py>
@@ Line 254: / Line 254: @@
      says the prefix is theirs, it is theirs. Free, authoritative, and better
      than any commercial "is_datacenter" flag for the networks it covers.
-. GEOLOCATION ESTIMATES (four free services). These are inferences. The script
+. GEOLOCATION ESTIMATES (three free keyless services, four with an ipapi.is
-     prints them side by side and flags disagreement rather than picking one.
+     key in IPAPI_IS_KEY). These are inferences. The script prints them side by
+     side and flags disagreement rather than picking one.
 Stdlib only. Free tiers are rate-limited (ip-api.com: 45 requests/minute from one
@@ Line 265: / Line 266: @@
 import ipaddress
 import json
+import os
 import socket
 import sys
@@ Line 290: / Line 292: @@
     "ipwho.is":   ("https://ipwho.is/{ip}",         lambda d: (d["country_code"], d.get("city"),
                                                                d.get("connection", {}).get("isp"))),
-    # ipapi.is returns a REDUCED object (cc, flags, asn_org, no city) for keyless
-    # queries about a third-party address, and the full object with a key. Handle
-    # both rather than crashing on the free tier.
-    "ipapi.is":   ("https://api.ipapi.is?q={ip}",   lambda d: (d.get("location", {}).get("country_code") or d["cc"],
-                                                               d.get("location", {}).get("city"),
-                                                               d.get("asn", {}).get("org") or d.get("asn_org"))),
 }
+# ipapi.is carries the risk flags (is_datacenter, is_vpn, ...). Since 2026-09-01 a
+# keyless query returns neither the flags nor an ISO country code, so without a
+# (free) key it is skipped rather than half-used: https://ipapi.is/free-tier.html
+IPAPI_IS_KEY = os.environ.get("IPAPI_IS_KEY")
+if IPAPI_IS_KEY:
+    GEO_SERVICES["ipapi.is"] = ("https://api.ipapi.is?q={ip}&key=" + IPAPI_IS_KEY,
+                                lambda d: (d["location"]["country_code"], d["location"].get("city"),
+                                           d["asn"]["org"]))
 FLAGS = ["is_datacenter", "is_vpn", "is_proxy", "is_tor", "is_abuser", "is_mobile"]
@@ Line 374: / Line 378: @@
             continue
         raw[name] = data
-        results[name] = extract(data)
+        try:
+            results[name] = extract(data)
+        except KeyError as exc:
+            raise RuntimeError(f"{name} answered for {ip} without {exc}; keys present: "
+                               f"{sorted(data)}. The service changed its response shape.") from exc
     return results, raw, errors
@@ Line 426: / Line 434: @@
     print("\nlayer 3  geolocation estimates -- these are inferences, not facts\n")
+    if not IPAPI_IS_KEY:
+        print("  ipapi.is skipped: its risk flags need a free key since 2026-09-01 (set IPAPI_IS_KEY)\n")
     disputed = 0
     for ip in ips:
@@ Line 459: / Line 469: @@
 </file>
-Real output, run on 2026-08-06 against five addresses chosen to break something each: Google Public DNS, Cloudflare's resolver, a Cloudflare CDN edge, an AWS CloudFront edge, and the Swiss residential address used on [[Design:Crawling location]].
+Real output, run on 2026-09-22 without an ipapi.is key, against five addresses chosen to break something each: Google Public DNS, Cloudflare's resolver, a Cloudflare CDN edge, an AWS CloudFront edge, and the Swiss residential address used on [[Design:Crawling location]].
 <code>
@@ Line 473: / Line 483: @@
 layer 2  operator-published prefixes (AWS, Google Cloud, Cloudflare)
  prefixes loaded
 .8.8.8          not in any loaded operator list
 .1.1.1          not in any loaded operator list
@@ Line 482: / Line 492: @@
 layer 3  geolocation estimates -- these are inferences, not facts
-.8.8.8   country US x4   agreed
+  ipapi.is skipped: its risk flags need a free key since 2026-09-01 (set IPAPI_IS_KEY)
+.8.8.8   country US x3   agreed
                    city    Mountain View x1/Ashburn x1/San Jose x1   <-- services disagree
                      ip-api.com   US  Ashburn          AS15169 Google LLC
-                     ipapi.is     US  None             Google LLC
                      ipinfo.io    US  Mountain View    AS15169 Google LLC
                      ipwho.is     US  San Jose         Google LLC
-                     ipapi.is flags is_datacenter, is_vpn, is_abuser
                      ipinfo.io says ANYCAST: this address has no single location
-.1.1.1   country AU x4   agreed
+.1.1.1   country AU x3   agreed
                    city    Brisbane x2/South Brisbane x1   <-- services disagree
                      ip-api.com   AU  South Brisbane   AS13335 Cloudflare, Inc.
-                     ipapi.is     AU  None             Cloudflare, Inc.
                      ipinfo.io    AU  Brisbane         AS13335 Cloudflare, Inc.
                      ipwho.is     AU  Brisbane         Cloudflare, Inc.
-                     ipapi.is flags is_vpn, is_abuser
                      ipinfo.io says ANYCAST: this address has no single location
-.16.132.229   country US x3/CA x1   DISPUTED
+.16.132.229   country US x2/CA x1   DISPUTED
                    city    San Francisco x2/Toronto x1   <-- services disagree
                      ip-api.com   CA  Toronto          AS13335 Cloudflare, Inc.
-                     ipapi.is     US  None             Cloudflare, Inc.
                      ipinfo.io    US  San Francisco    AS13335 Cloudflare, Inc.
                      ipwho.is     US  San Francisco    Cloudflare, Inc.
-                     ipapi.is flags is_datacenter, is_abuser
                      ipinfo.io says ANYCAST: this address has no single location
-.32.99.63   country DE x3/US x1   DISPUTED
+.32.99.63   country DE x2/US x1   DISPUTED
                    city    Frankfurt am Main x2/New York x1   <-- services disagree
                      ip-api.com   US  New York         AS16509 Amazon.com, Inc.
-                     ipapi.is     DE  None             Amazon.com, Inc.
                      ipinfo.io    DE  Frankfurt am Main AS16509 Amazon.com, Inc.
                      ipwho.is     DE  Frankfurt am Main Amazon.com, Inc.
-                     ipapi.is flags is_datacenter
-.220.84.43   country CH x4   agreed
+.220.84.43   country CH x3   agreed
-                   city    Biel/Bienne x1/Samnaun x1/Zurich x1   <-- services disagree
+                   city    Zürich x1/Samnaun x1/Munchenbuchsee x1   <-- services disagree
                      ip-api.com   CH  Samnaun          AS9044 BSE Software GmbH
-                     ipapi.is     CH  None             BSE Software GmbH
+                     ipinfo.io    CH  Zürich           AS9044 BSE Software GmbH
-                     ipinfo.io    CH  Biel/Bienne      AS9044 BSE Software GmbH
+                     ipwho.is     CH  Munchenbuchsee   BSE Software GmbH
-                     ipwho.is     CH  Zurich           BSE Software GmbH
-                     ipapi.is flags none set
 summary: 2 of 5 addresses had a cross-service country disagreement
 </code>
-Everything this page argues is visible in that one run:
+Everything this page argues is visible in that run, except the risk flags, which need the 2026-08-06 run kept on the provenance page:
   - **Layer 1 answered for all five, with no ambiguity to reconcile.** An ASN, a covering prefix, the allocation record and its date, read out of the routing system. There is no confidence interval because nothing is being estimated — which is why nothing below layer 1 gets the same treatment on this page. Note that ''reg-CC'' and ''RIR'' are two different things and neither is a location: ''1.1.1.1'' is ''AU'' under ''apnic'' and is served from everywhere.
-  - **The RIR country is not the host's country.** ''1.1.1.1'' is registered ''AU'' and all four geolocation services dutifully say Australia — for an anycast resolver that answers you from the nearest of hundreds of sites.
+  - **The RIR country is not the host's country.** ''1.1.1.1'' is registered ''AU'' and all three geolocation services dutifully say Australia (all four, with ipapi.is, on 2026-08-06) — for an anycast resolver that answers you from the nearest of hundreds of sites.
   - **Country disagreement is not rare.** Two of five addresses, and one of them (the AWS CloudFront edge, DE vs US) is exactly the kind of address a web crawl records by the thousand. If your paper says "//n// % of third-party requests left the EEA", that number moves depending on which vendor you asked.
-  - **City is noise.** Five addresses, five sets of mutually inconsistent cities. The Swiss address is the one [[Design:Crawling location]]'s ''verify_vantage.py'' queried a day earlier, which got Zurich, Zürich, Samnaun and Lausanne from the same services; today one of them says Biel/Bienne. Same address, different day, different city.
+  - **City is noise.** Five addresses, five sets of mutually inconsistent cities. The Swiss address is the one [[Design:Crawling location]]'s ''verify_vantage.py'' queried on 2026-08-05, which got Zurich, Zürich, Samnaun and Lausanne from the same services. This script got Biel/Bienne, Samnaun and Zurich on 2026-08-06, and Zürich, Samnaun and Munchenbuchsee on 2026-09-22. Same address, different day, different city.
-  - **Risk flags misfire on the most benign addresses in the world.** Both public resolvers are ''is_vpn'' and ''is_abuser''.
+  - **Risk flags misfire on the most benign addresses in the world.** In the 2026-08-06 run ipapi.is printed ''8.8.8.8 … flags is_datacenter, is_vpn, is_abuser'' and ''1.1.1.1 … flags is_vpn, is_abuser'': both public resolvers came back ''is_vpn'' and ''is_abuser''. Today's keyless run cannot show this, because the flags now need a key.
   - **Layer 2 is authoritative but partial.** The CDN edges are matched exactly; the resolvers are not in the published lists at all, because those lists describe proxy and customer space.
@@ Line 641: / Line 643: @@
 | Cross-validation | 1 | 0.3% |
-**196 of 295 papers (66.4%) report no validation at all** on any of their IP classification records — against **29.9% across all 4,439 papers that classify anything** ([[Statistics:Annotation]], which owns the corpus-wide validation figures and the per-target comparison this row sits in) — that is, every such record is ''none-reported'' or ''not-applicable''. Separately and with a different meaning, 127 papers (43.1%) name a ground-truth source. The high //not-applicable// share is partly legitimate — looking up an ASN is not a classifier that needs a test set — but it is also where "we used MaxMind, therefore it is true" hides.
+**196 of 295 papers (66.4%) report no validation at all** on any of their IP classification records — that is, every such record is ''none-reported'' or ''not-applicable''. Put the other way, 33.6% validate their IP classification, against **60.1% of all 6,483 (paper, target) cells** across the 4,439 papers that classify anything ([[Statistics:Annotation]], which owns the corpus-wide validation figures and the per-target comparison this row sits in). Separately and with a different meaning, 127 papers (43.1%) name a ground-truth source. The high //not-applicable// share is partly legitimate — looking up an ASN is not a classifier that needs a test set — but it is also where "we used MaxMind, therefore it is true" hides.
 ==== Papers do not say which snapshot they used ====
@@ Line 687: / Line 689: @@
   - **Which question you answered** — routing, organisation, network type, location. Do not write "we geolocated the IPs" when what you needed was the ASN.
-  - **The source, by product name**, not by company: ''GeoLite2-City'', not "MaxMind".
+  - **The source, by product name**, not by company: ''GeoLite City'' (''GeoLite2-City'' before the 2026 rebrand), not "MaxMind".
   - **The snapshot date** of the database, or the date range over which the API was queried. This is the single most-omitted item.
   - **Whether you cross-checked**, against what, and how often they disagreed. A one-line "the two sources agreed on country for //n// % of addresses" is worth more than a paragraph of caveats.
@@ Line 698: / Line 700: @@
 ===== Legal and Ethical Notes =====
-**An IP address is personal data.** In the EU the reference point is the CJEU's //Breyer// judgment (Case C-582/14, ECLI:EU:C:2016:779, 19 October 2016), which held that a dynamic IP address is personal data in the hands of an online media services provider where that provider has legal means reasonably likely to be used to identify the subscriber, e.g. via the ISP. Note the shape of the test: identifiability is assessed **relative to who holds the data and what means are available to them**, not in the abstract. The Court restated that framing in //EDPS v SRB// (Case C-413/23 P, ECLI:EU:C:2025:645, 4 September 2025), holding that the existence of additional information enabling identification "does not, in itself, mean that pseudonymised data must be regarded as constituting, in all cases and for every person, personal data" — while setting the General Court's judgment aside and referring the case back, so it is not the last word.((Read the judgment before relying on it; this is a methods page, not legal advice, and the case was remitted rather than finally decided.)) Practical consequences for a measurement:
+**An IP address is personal data.** In the EU the reference point is the CJEU's //Breyer// judgment (Case C-582/14, ECLI:EU:C:2016:779, 19 October 2016), which held that a dynamic IP address is personal data in the hands of an online media services provider where that provider has legal means reasonably likely to be used to identify the subscriber, e.g. via the ISP. Note the shape of the test: identifiability is assessed **relative to who holds the data and what means are available to them**, not in the abstract. The Court restated that framing in //EDPS v SRB// (Case C-413/23 P, ECLI:EU:C:2025:645, 4 September 2025), holding that the existence of additional information enabling identification "does not, in itself, mean that pseudonymised data must be regarded as constituting, in all cases and for every person, personal data" (para. 82) — while setting the General Court's judgment aside and referring the case back, so it is not the last word.((Read the judgment before relying on it; this is a methods page, not legal advice, and the case was remitted rather than finally decided.)) Practical consequences for a measurement:
   * Client IP addresses in a log you were given are personal data; treat the dataset accordingly and say so in the ethics section. See [[Practices:Ethics]].
@@ Line 712: / Line 714: @@
   * **How much does the choice of database move a published cross-border-transfer figure?** Re-running one compliance paper's analysis with four databases would be a small, cheap, useful replication, and the answer is not obviously small: the run above disagreed on country for the AWS edge.
   * **Geofeed adoption is not measured for the web.** {[livadariu2024_geofeeds]} gives 1.50% of allocated IPv4 prefixes overall; nobody has asked what share of the //address space a crawl actually touches// has a geofeed, which — given how concentrated that space is on a few large operators — could be much higher or much lower.
-  * **No systematic evaluation of the commercial datacenter/VPN/proxy flags against ground truth.** Every paper that uses one takes it on faith, and the run above shows two public DNS resolvers flagged ''is_vpn'' and ''is_abuser''. A labelled benchmark here would be immediately useful and is well within a single student's reach.
+  * **No systematic evaluation of the commercial datacenter/VPN/proxy flags against ground truth.** Every paper that uses one takes it on faith, and this page's 2026-08-06 run had two public DNS resolvers flagged ''is_vpn'' and ''is_abuser''. A labelled benchmark here would be immediately useful and is well within a single student's reach.
-  * **No peer-reviewed method for identifying hosting/datacenter address space** beyond the operators' own published lists — which means everything outside the big five clouds is guesswork.
+  * **No peer-reviewed method for identifying hosting/datacenter address space** beyond the operators' own published lists — which means everything outside the operators that publish their own prefix lists is guesswork.
   * **LLMs have reached AS-to-organisation mapping {[selmo2025_borges]} but not IP classification.** We found nothing peer-reviewed applying an LLM to geolocation, host typing or residential/VPN/datacenter labelling as of 2026-09-03. This page previously said that was "unlike cookie and policy classification, where LLM methods are now routine"; that was wrong and is corrected here. Measured per target across the 5,859-paper corpus, the LLM share of the papers classifying that target at all is **11.8% for ''privacy-policy'' (12 of 102)** — the highest of any target and the only one that could support the word "routine" — but only **1.9% for ''cookie'' (1 of 53)**, a single TheWebConf 2025 paper. Against those, ''ip-address'' at 1 of 295 is low but not the outlier the old sentence implied; ''javascript'' and ''fingerprinting-script'' are at zero. See [[design:website_classification#Where LLMs actually appear]] for the whole table. Whether the gap here is because the task has no useful text to read, or because nobody has tried, is still an open question.
   * **No strong successor to {[richter2016_multi]} on CGNAT prevalence.** The best numbers on how much of the client Internet sits behind shared addresses are a decade old, and IPv4 exhaustion has only got worse since.
@@ Line 720: / Line 722: @@
 ===== Related Pages =====
-  * [[Statistics:Annotation|Annotation and Validation]] — validating a label set whatever it labels; IP addresses are the second-worst-validated target on this wiki.
+  * [[Statistics:Annotation|Annotation and Validation]] — validating a label set whatever it labels; of the targets with more than fifty papers, IP addresses are the second-worst validated.
   * [[Design:Crawling location]] — the mirror image: classifying and verifying //your own// vantage point before a crawl.
   * [[Design:Website classification]] — classifying sites by topic; the same "which service, which taxonomy, which validation" questions with entirely different answers.
@@ Line 726: / Line 728: @@
   * [[Programming:Traffic files]] — extracting server addresses from a HAR or a packet capture.
   * [[Practices:Ethics]] — client IP addresses are personal data.
-  * [[Design:Sampling]] //(not yet written)// — why an IP is a bad sampling unit.
+  * [[Design:Sampling]] — how a population of sites, domains or pages is drawn, and what to record so the draw can be repeated.
 ====== References ======