TY - GEN
T1 - EYEORG
T2 - 12th ACM Conference on Emerging Networking Experiments and Technologies, ACM CoNEXT 2016
AU - Varvello, Matteo
AU - Blackburn, Jeremy
AU - Naylor, David
AU - Papagiannaki, Kostantina
PY - 2016/12/6
Y1 - 2016/12/6
N2 - Tremendous effort has gone into the ongoing battle to make webpages load faster. This effort has culminated in new protocols (QUIC, SPDY, and HTTP/2) as well as novel content delivery mechanisms. In addition, companies like Google and SpeedCurve investigated how to measure "page load time" (PLT) in a way that captures human perception. In this paper we present Eyeorg [12], a platform for crowdsourcing web quality of experience measurements. Eyeorg overcomes the scaling and automation challenges of recruiting users and collecting consistent user-perceived quality measurements. We validate Eyeorg's capabilities via a set of 100 trusted participants. Next, we showcase its functionalities via three measurement campaigns, each involving 1,000 paid participants, to 1) study the quality of several PLT metrics, 2) compare HTTP/1.1 and HTTP/2 performance, and 3) assess the impact of online advertisements and ad blockers on user experience. We find that commonly used, and even novel and sophisticated PLT metrics fail to represent actual human perception of PLT, that the performance gains from HTTP/2 are imperceivable in some circumstances, and that not all ad blockers are created equal.
AB - Tremendous effort has gone into the ongoing battle to make webpages load faster. This effort has culminated in new protocols (QUIC, SPDY, and HTTP/2) as well as novel content delivery mechanisms. In addition, companies like Google and SpeedCurve investigated how to measure "page load time" (PLT) in a way that captures human perception. In this paper we present Eyeorg [12], a platform for crowdsourcing web quality of experience measurements. Eyeorg overcomes the scaling and automation challenges of recruiting users and collecting consistent user-perceived quality measurements. We validate Eyeorg's capabilities via a set of 100 trusted participants. Next, we showcase its functionalities via three measurement campaigns, each involving 1,000 paid participants, to 1) study the quality of several PLT metrics, 2) compare HTTP/1.1 and HTTP/2 performance, and 3) assess the impact of online advertisements and ad blockers on user experience. We find that commonly used, and even novel and sophisticated PLT metrics fail to represent actual human perception of PLT, that the performance gains from HTTP/2 are imperceivable in some circumstances, and that not all ad blockers are created equal.
KW - Adblockers
KW - Crowdsourcing
KW - Http/2
KW - Quality of experience
KW - Web measurements
UR - https://www.scopus.com/pages/publications/85009813151
U2 - 10.1145/2999572.2999590
DO - 10.1145/2999572.2999590
M3 - Conference contribution
T3 - CoNEXT 2016 - Proceedings of the 12th International Conference on Emerging Networking EXperiments and Technologies
SP - 399
EP - 412
BT - CoNEXT 2016 - Proceedings of the 12th International Conference on Emerging Networking EXperiments and Technologies
PB - Association for Computing Machinery, Inc
Y2 - 12 December 2016 through 15 December 2016
ER -