Measure The Web

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Stateful and Stateless Crawling ======

<wrap todo>This page only contains notes</wrap>


Key message:
  * Majority of web measurements studies use stateless crawls, as it is easy to associate events with the single browsed visited website. Also, stateless crawls do not depend on crawling order and are easier to parallelize.
  * Stateful crawling is however more representative of real users, that rarely clear their browser state.

===== Relevant Literature =====

Since the majority of publication uses stateless crawling, below we list examples of influential publications doing otherwise. However, not all contribute specifically to the question of difference between stateful and stateless crawling.


  * [[https://dl.acm.org/doi/pdf/10.1145/3366423.3380104|The Representativeness of Automated Web Crawls as a Surrogate for Human Browsing]] {[zeber2020representativeness]}
    * Comparison of stateless crawls with stateful browsing experience of real (Firefox) users
    * Stateless crawls surprisingly  result in more third-party requests than stateful crawl (Fig. 6).
  * [[https://dl.acm.org/doi/pdf/10.1145/2976749.2978313|Online Tracking: A 1-million-site Measurement and Analysis]] {[englehardt2016online]}
    * The most impactful publication utilizing both stateful and stateless crawls
  * [[https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_bashir.pdf|Tracing Information Flows Between Ad Exchanges Using Retargeted Ads]] {[bashir2016tracing]}
    * Created "shopper personas" by visiting websites of certain topics and collect advertisement on next websites using these personas
  * The web never forgets: Persistent tracking mechanisms in the wild
  * Cookies that give you away: The surveillance implications of web tracking

==== Studies of Stateful Aspects ====

The following studies used stateless crawls, but were interpreting some stateful properties of web:


  * [[https://hal.science/hal-03218403/file/PETS_21_My_cookie_is_a_phoenix.pdf|My Cookie is a phoenix: detection, measurement, and lawfulness of cookie respawning with browser fingerprinting]] {[fouad2022my]}
    * Evaluation of cookie respawning

==== Shallow vs Deep crawling ====

  * [[https://dl.acm.org/doi/pdf/10.1145/3366423.3380203|Beyond the Front Page: Measuring Third Party Dynamics in the Field]] {[urban2020beyond]}
    * Comparison of visiting only front pages vs crawling sub-pages
    * Visiting sub-pages increases amount of tracking significantly
  * [[https://www.ftc.gov/system/files/documents/public_events/776191/ialtaweelwebpriv_0.pdfWeb Privacy Census]]

/*
This is a comment not visible on the page. It outlines the syntax (for more, go to https://measuretheweb.org/wiki/syntax), especially that related to bibliography. Remove it once you created the page. If you use any citations (documented at bottom), keep the References section.

===== Header Level 2 =====
==== Header Level 3 ====

=== Links ===

External links are recognized automatically: www.google.com, but if you want a link text: [[http://www.google.com|This Link points to google]].

Internal links are created by using square brackets. You can either just give a [[pagename]] or use an additional [[pagename|link text]].

=== Lists ===

Lists and their levels are decided by indentation (2 spaces = 1 level)

  - this is 1. item
  - 2. item
	- nested a. item
  * bullet-point item

=== Code ===

For a short inline monospace, use ''double quote''. For proper code (but in a separate paragraph), use ''<code LANG>'':

<code python>
string = "World"
print(f'Hello {string}')
</code>

For large code, use ''<file LANG filename>'', it will make code downloadable. For instance:

<file php example.php>
<?php echo "hello world!"; ?>
</file>

=== Figures ===

To use floats, you have to use the ''<WRAP>'' tag. For instance, the following will create figure on the right side 50% large:
<WRAP right 50% box>
{{PATH_TO_FILE|ALT_TEXT}}
<div>CAPTION</div>
</WRAP>

*/

====== References ======

/*
To insert citations, follow these steps:

  - Verify the BibTeX entry exists in https://measuretheweb.org/literature/bibliography. If not, add it there.
  - Use {[CitationKey]} where needed in the text; it will render as a numbered reference.
  - Keep this section unchanged to display the bibliography.

If any step fails, a purple warning will appear on the preview page.
*/

<bibtex bibliography></bibtex>


/* This enables discussion under this article. */
~~DISCUSSION~~