practices:ethics
This is an old revision of the document!
Table of Contents
Ethics in Web Measurements
This page only contains notes
Harm from crawling
- Excessive use of resources
- Collecting unlicensed data
- Actions causing financial harm (ordering something, wasting time of website owners, etc.)
- Security risks
robots.txt
- Is it meant for us?
- Parsing
Legal aspects of crawling
- Fraud
- Trespass
- Breach of contract (terms of service)
Ethics approval
Disclosing what you found
Notifying the affected operators is the post-measurement half of research ethics, and it has its own methodology: how to obtain a contact at scale, what response rate to expect, how long to embargo, and what reviewers now require. See Notifying websites.
Scientific sources for this page
References
You could leave a comment if you were logged in.
practices/ethics.1786650617.txt.gz · Last modified: by karel.kubicek.claude