From a4cb94f5f6a7da70ad3e3ea27ab878abec0932dc Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Tue, 9 Apr 2024 18:42:13 +0200 Subject: [PATCH 01/17] doc: explain the throttling methodology - outline the current methodology; - explain its theoretical assumptions; - describe limitations; - mention possible future directions. Part of https://github.com/ooni/ooni.org/issues/1406. --- docs/design/dd-006-throttling.md | 195 +++++++++++++++++++++++++++++++ 1 file changed, 195 insertions(+) create mode 100644 docs/design/dd-006-throttling.md diff --git a/docs/design/dd-006-throttling.md b/docs/design/dd-006-throttling.md new file mode 100644 index 0000000000..017830ca17 --- /dev/null +++ b/docs/design/dd-006-throttling.md @@ -0,0 +1,195 @@ +# Throttling measurement methodology + +| | | +|--------------|------------------------------------------------| +| Author | [@bassosimone](https://github.com/bassosimone) | +| Last-Updated | 2024-04-09 | +| Reviewed-by | N/A | +| Status | ready for review | + +This document explains the throttling measurement methodology implemented inside +the [ooni/probe-cli](https://github.com/ooni/probe-cli) repository. + +We are publishing this document as part of this repository for discussion. A future +version of this document may be moved into the [ooni/spec](https://github.com/ooni/spec) +repository. + +## Problem statement + +We are interested to detect cases of _extreme throttling_. We say that throttling is +_extreme_ when the speed to access web resources is _significantly reduced_ (10x or more) +compared to what is _typically_ observed. We care about extreme throttling because we +are interested in cases in which the performance impact is such to make the website +_unlikely_ to work as intended for web users in a country. + +We, and other researchers, have documented such issues in the past. See, for example: + +1. [our blog post documenting twitter throttling in Russia]( +https://ooni.org/post/2022-russia-blocks-amid-ru-ua-conflict/), which is the +first instance in which we tested this methodology. + +2. ["Throttling Twitter: an emerging censorship technique in Russia" by Xue et al.]( +https://censorbib.nymity.ch/#Xue2021a). + +OONI Probe measures websites as part of the [Web Connectivity experiment]( +https://github.com/ooni/spec/blob/master/nettests/ts-017-web-connectivity.md) and +these measurements contain peformance metrics. + +The next section explains which performance metrics we collect and how these can +be useful to document episodes of extreme throttling. + +## Methodology + +The overall idea of our methodology is that we're not concerned with _how_ throttling +is implemented, rather we aim to show clearly degraded network performance. + +We aim to detect such a degradation by comparing metrics collected by OONI Probe instances +running in a country and network with measurements previously collected by users and/or with +concurrent measurements towards different targets. + +### Network Events + +Web Connectivity v0.5 collects the first 64 [network events]( +https://github.com/ooni/spec/blob/master/data-formats/df-008-netevents.md). These events +include "read" and "write" events, which map directly to network I/O operations. The basic +structure of a "read" network events is the following: + +```JSON +{ + "address": "1.1.1.1:443", + "failure": null, + "num_bytes": 4114, + "operation": "read", + "proto": "tcp", + "t0": 1.001, + "t": 1.174, + "tags": [], + "transaction_id": 1, +} +``` + +Through these events, we know when "read" returned (`t`), for how much time it was blocked +(`t - t0`), and the number of bytes sent or received. + +The slope of the integral of "read" events, threfore, provides information about the speed +at which we were receiving data from the network. Slow downs in the stream either correspond +to reordering and retransmission events (where there is head of line blocking) or to +timeout events (where the TCP pipe is empty). + +Additionally, network events contain events such as `"tls_handshake_start"` and +`"tls_handshake_done`", which look like the following: + +```JSON +{ + "address": "1.1.1.1:443", + "failure": null, + "num_bytes": 0, + "operation": "tls_handshake_start", + "proto": "tcp", + "t0": 1.001, + "t": 1.001, + "tags": [], + "transaction_id": 1, +} +``` + +These events allow us to know when we started and we stopped handshaking. + +Now, considering that the amount of bytes transferred by a TLS handshake with the +same server using similar client code is not far from being constant (i.e., it's a relatively +narrow gaussian with small sigma), we can model the problem of TLS handshaking as +the problem of downloading a ~fixed amount of data. + +With many users measuring popular websites using OONI Probe in a given country +and network, we can therefore establish comparisons of current performance metrics with +previous performance metrics. In case of extreme throttling, where the download speed +is reduced of, at least 10x, we would notice a performance difference. The _time_ +required to complete the TLS handshake should be a sufficient metric (and, in fact, +_is_ a performance metric used by speed tests such as +[speed.cloudflare.com](https://speed.cloudflare.com/)). + +### Download speed metrics + +Additionally, Web Connectivity v0.5 collects download speed samples for connections +used to access websites. We use the same methodology used by [ndt7]( +https://github.com/m-lab/ndt-server/blob/main/spec/ndt7-protocol.md). We measure +the cumulative number of bytes received by a connection using a truncated exponential +distribution to decide when to collect samples. By not collecting samples at fixed +intervals, we [should have PASTA properties](https://en.wikipedia.org/wiki/Arrival_theorem#Theorem_for_arrivals_governed_by_a_Poisson_process). + +The total TLS handshaking, HTTP round trip and body fetching time is bounded by a fixed amount of +seconds (currently ten seconds for the handshake and ten additional seconds for HTTP). Additionally, +there is a cap on the maximum amount of body bytes to download (`1<<19`). + +The expected size of a downloaded webpage should be pretty constant for clients +attempting to fetch such a webpage from the same country and network. Therefore, we +can model handshaking plus fetching the body as asking the question of how much +time it takes to download `handshake_size + min(body_size, 1<<19)` bytes in ~twenty seconds. + +If we assume that the server is not going to throttle downloads (which is still +an hypothesis worth considering), save for some (healthy) packet pacing, significant +changes in the _time_ required to perform the whole set of operations would be +an indication of extreme throttling. However, in using time as the metric, or any +other metric, we need to remember to classify measurements that time out (i.e., are +not able to fetch the whole body) apart from the ones that complete successfully. + +## Discussion + +This methodology leverages existing performance metrics inside of Web Connectivity +v0.5 to passively detect extreme throttling. Because this methodology models +the TLS handshake and fetching the body as speed tests, it is, however, not possible +to provide users with clear indication of throttling after a single run. We will, +instead, need to collect several samples over time and cross compare them using +the [ooni/data](https://github.com/ooni/data) measurement pipeline. + +Throttling could be caused by policers and shapers as well as by forcing specific +users to pass through a congested path. When policers and shapers are used, we +expect the speed to likely converge to predictable values (e.g., 128 kbit/s). On the +contrary, when throttling is driven by congestion, we expect to see higher variance +in the results, possibly correlated with daily usage patterns. + +## Digital Divide Implications + +By collecting passive performance metrics, we are not only equipped to detect +extreme throttling, but we are also gathering information about the performance +achievable by clients in several world regions for reaching specific websites. The +availability of HTTP headers and the practice of some CDNs of annotating the +responses with headers indicating which specific cache is being used could also +be exploited to make interesting network-performance statements. + +## Future Work + +With network events, we can also collect some ~baseline RTT samples. The `t - t0` time +of the TCP connect event provides an upper bound of the path RTT _unless_ there is a +retransmission of the `SYN` segment. The TLS handshake also involves sending TCP segments +back and forth in such a fashion that it's possible to extract RTT metrics. Howewer, we +should be careful to exclude segments sent back to back. + +In general, detecting more precisely the characteristics of throttling either +requires additional research aimed at classifying the stream of events emitted +by a receiving socket under specific throttling conditions. A possible starting +point for this research could be [Strengthening measurements from the edges: +application-level packet loss rate estimation by Basso et al.]( +https://www.sigcomm.org/sites/default/files/ccr/papers/2013/July/2500098-2500104.pdf). + +An alternative approach would require the possibility of providing OONI experiments +with "richer input" parameters or dynamic experiments, aimed at answering more +specific research questions. For example, if there are reports that a website is +throttled by SNI, we could perform a download from a given test server with +certificate verification disabled, using the offending SNI and an innocuous SNI. + +Because HTTP/3 used QUIC and because QUIC operates in userspace, there is +also the possibility of instrumenting the QUIC library to periodically collect +snapshots about the receiver's state. However, in general, sender stats are +much more useful to understand QUIC performance. This fact implies that we could +instrument a QUIC library to observe the sender's state and gather information +about throttling uploads. (However, the whole design of Web Connectivity is not +such that we upload resources, therefore we would need to figure out whether +it is possible to overcome this fundamental limitation first.) + +In the same vein, our Web Connectivity methodology does not currently factor in +the possibility of measuring upload speed throttling for HTTP/1.1 and HTTP2. However, +anecdotal evidence exists that some countries may throttle the upload path or just +have poor upstream connectivity towards interesting websites. A technique that +has sometimes been applied is that of including very large headers into the request +body, even though servers may not necessarily accept such headers. From 4bd4f4960ff2042eff3a3d6e8a99067d1524ad06 Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Tue, 9 Apr 2024 18:46:17 +0200 Subject: [PATCH 02/17] x --- docs/design/dd-006-throttling.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/design/dd-006-throttling.md b/docs/design/dd-006-throttling.md index 017830ca17..432d72fadb 100644 --- a/docs/design/dd-006-throttling.md +++ b/docs/design/dd-006-throttling.md @@ -71,7 +71,7 @@ structure of a "read" network events is the following: Through these events, we know when "read" returned (`t`), for how much time it was blocked (`t - t0`), and the number of bytes sent or received. -The slope of the integral of "read" events, threfore, provides information about the speed +The slope of the integral of "read" events, therefore, provides information about the speed at which we were receiving data from the network. Slow downs in the stream either correspond to reordering and retransmission events (where there is head of line blocking) or to timeout events (where the TCP pipe is empty). @@ -168,8 +168,8 @@ should be careful to exclude segments sent back to back. In general, detecting more precisely the characteristics of throttling either requires additional research aimed at classifying the stream of events emitted by a receiving socket under specific throttling conditions. A possible starting -point for this research could be [Strengthening measurements from the edges: -application-level packet loss rate estimation by Basso et al.]( +point for this research could be ["Strengthening measurements from the edges: +application-level packet loss rate estimation" by Basso et al.]( https://www.sigcomm.org/sites/default/files/ccr/papers/2013/July/2500098-2500104.pdf). An alternative approach would require the possibility of providing OONI experiments From ff65b93e2f65e1e9728c65a022f549551e94b629 Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Tue, 4 Jun 2024 13:58:06 +0200 Subject: [PATCH 03/17] Rename dd-006-throttling.md to dd-007-throttling.md --- docs/design/{dd-006-throttling.md => dd-007-throttling.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename docs/design/{dd-006-throttling.md => dd-007-throttling.md} (100%) diff --git a/docs/design/dd-006-throttling.md b/docs/design/dd-007-throttling.md similarity index 100% rename from docs/design/dd-006-throttling.md rename to docs/design/dd-007-throttling.md From 44da975e0956f8055a00de3ff4263235daff1671 Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Fri, 21 Jun 2024 11:23:43 +0200 Subject: [PATCH 04/17] fix(throttling): address review comments Additionally, mention comments that were provided by @agrabeli, @AlbertoDainotti, et al., during a recent gathering focusing on network measurements where I presented this work. --- docs/design/dd-007-throttling.md | 95 ++++++++++++++++++++++++-------- 1 file changed, 72 insertions(+), 23 deletions(-) diff --git a/docs/design/dd-007-throttling.md b/docs/design/dd-007-throttling.md index 432d72fadb..ab3fcecd44 100644 --- a/docs/design/dd-007-throttling.md +++ b/docs/design/dd-007-throttling.md @@ -4,8 +4,8 @@ |--------------|------------------------------------------------| | Author | [@bassosimone](https://github.com/bassosimone) | | Last-Updated | 2024-04-09 | -| Reviewed-by | N/A | -| Status | ready for review | +| Reviewed-by | [@DecFox](https://github.com/DecFox) | +| Status | accepted | This document explains the throttling measurement methodology implemented inside the [ooni/probe-cli](https://github.com/ooni/probe-cli) repository. @@ -22,13 +22,25 @@ compared to what is _typically_ observed. We care about extreme throttling becau are interested in cases in which the performance impact is such to make the website _unlikely_ to work as intended for web users in a country. -We, and other researchers, have documented such issues in the past. See, for example: +Additionally, as recently discussed with [@inetintel](https://github.com/InetIntel/) +researchers et al., we are interested to detect cases of _targeted throttling_. That is +cases where a set of specific websites gets significantly worse performance while the +overall users' internet experience is unchanged. This kind of throttling is in opposition +to _generalized throttling_ where the internet experience is degraded regardless of the +website compared to the previous internet experience (see [Dimming the Internet by Collin +Anderson](https://arxiv.org/pdf/1306.4361v1) for seminal work on this topic). + +We, and other researchers, have documented extreme, targeted throttling in the +past. See, for example: 1. [our blog post documenting twitter throttling in Russia]( https://ooni.org/post/2022-russia-blocks-amid-ru-ua-conflict/), which is the first instance in which we tested this methodology. -2. ["Throttling Twitter: an emerging censorship technique in Russia" by Xue et al.]( +2. [our blog post documenting throttling in Kazakhstan]( +https://ooni.org/post/2023-throttling-kz-elections/). + +3. ["Throttling Twitter: an emerging censorship technique in Russia" by Xue et al.]( https://censorbib.nymity.ch/#Xue2021a). OONI Probe measures websites as part of the [Web Connectivity experiment]( @@ -36,12 +48,12 @@ https://github.com/ooni/spec/blob/master/nettests/ts-017-web-connectivity.md) an these measurements contain peformance metrics. The next section explains which performance metrics we collect and how these can -be useful to document episodes of extreme throttling. +be useful to document episodes of extreme, targeted throttling. ## Methodology The overall idea of our methodology is that we're not concerned with _how_ throttling -is implemented, rather we aim to show clearly degraded network performance. +is implemented, rather we aim to show clearly-degraded network performance. We aim to detect such a degradation by comparing metrics collected by OONI Probe instances running in a country and network with measurements previously collected by users and/or with @@ -50,9 +62,13 @@ concurrent measurements towards different targets. ### Network Events Web Connectivity v0.5 collects the first 64 [network events]( -https://github.com/ooni/spec/blob/master/data-formats/df-008-netevents.md). These events -include "read" and "write" events, which map directly to network I/O operations. The basic -structure of a "read" network events is the following: +https://github.com/ooni/spec/blob/master/data-formats/df-008-netevents.md) occurring +on a given TCP connection. These events include "read" and "write" events, which +map directly to network I/O operations (i.e., the `recv` and `send` syscalls +respectively). We focus on throttling in the download direction, therefore we're +mostly interested into "read" events. + +The basic structure of a "read" network events is the following: ```JSON { @@ -69,11 +85,11 @@ structure of a "read" network events is the following: ``` Through these events, we know when "read" returned (`t`), for how much time it was blocked -(`t - t0`), and the number of bytes sent or received. +(`t - t0`), and the number of bytes received (`num_bytes`). -The slope of the integral of "read" events, therefore, provides information about the speed +The slope of the integral of "read" events, provides information about the speed at which we were receiving data from the network. Slow downs in the stream either correspond -to reordering and retransmission events (where there is head of line blocking) or to +to reordering and retransmission events (where there is head-of-line blocking) or to timeout events (where the TCP pipe is empty). Additionally, network events contain events such as `"tls_handshake_start"` and @@ -103,11 +119,16 @@ the problem of downloading a ~fixed amount of data. With many users measuring popular websites using OONI Probe in a given country and network, we can therefore establish comparisons of current performance metrics with previous performance metrics. In case of extreme throttling, where the download speed -is reduced of, at least 10x, we would notice a performance difference. The _time_ +is reduced of at least 10x, we would notice a performance difference. The _time_ required to complete the TLS handshake should be a sufficient metric (and, in fact, _is_ a performance metric used by speed tests such as [speed.cloudflare.com](https://speed.cloudflare.com/)). +As of Web Connectivity v0.4, we limit our data collection process to the TLS handshake, +but Web Connectivity v0.5 will extend the scope and collect up to 64 "read" events +including when we are downloading the body. The availability of more data will allow +us to better investigate throttling by analysing more "read" event samples. + ### Download speed metrics Additionally, Web Connectivity v0.5 collects download speed samples for connections @@ -124,7 +145,8 @@ there is a cap on the maximum amount of body bytes to download (`1<<19`). The expected size of a downloaded webpage should be pretty constant for clients attempting to fetch such a webpage from the same country and network. Therefore, we can model handshaking plus fetching the body as asking the question of how much -time it takes to download `handshake_size + min(body_size, 1<<19)` bytes in ~twenty seconds. +time it takes to download `handshake_size + min(body_size, 1<<19)` bytes in up to +~twenty seconds. If we assume that the server is not going to throttle downloads (which is still an hypothesis worth considering), save for some (healthy) packet pacing, significant @@ -133,6 +155,11 @@ an indication of extreme throttling. However, in using time as the metric, or an other metric, we need to remember to classify measurements that time out (i.e., are not able to fetch the whole body) apart from the ones that complete successfully. +Those measurements, in fact, should not be considered "failed" for the purpose of +measuring throttling. Rather, if the TCP connection could progress into the handshake +and possibily into downloading a webpage, these measurements would possibly be +an additional indication of extreme, targeted throttling. + ## Discussion This methodology leverages existing performance metrics inside of Web Connectivity @@ -142,6 +169,26 @@ to provide users with clear indication of throttling after a single run. We will instead, need to collect several samples over time and cross compare them using the [ooni/data](https://github.com/ooni/data) measurement pipeline. +More specifically, we would need to compare current measurements with past +measurements collected for the same target website by users living in the same +country and using the same autonomous system. Alternatively, we could compare +measurements collected during the same time frame towards different websites, even +though this signal is weaker because it can just be caused by interconnection +issues. In any case, these considerations imply that our methodology rests +on the assumption that we will have several measurements for the targer websites, +and our confidence would clearly lower with little available data. + +In analysing the data, it would also be useful to consider the possibility of +checking whether specific HTTP headers or the host name (after a redirect) clearly +indicate specific geographic locations. For example, Cloudflare includes a +`cf-ray` header indicating the specific cache that is serving the content using +the name of the nearest airport. + +Additionally, with the availability of richer input, it would become possible to +run custom `urlgetter` experiments where we use possibly offending and possibly not +offending SNIs with target addresses and possibly-unrelated addresses, thus giving +us a chance to narrow down the cause of throttling to, say, the SNI being used. + Throttling could be caused by policers and shapers as well as by forcing specific users to pass through a congested path. When policers and shapers are used, we expect the speed to likely converge to predictable values (e.g., 128 kbit/s). On the @@ -151,11 +198,12 @@ in the results, possibly correlated with daily usage patterns. ## Digital Divide Implications By collecting passive performance metrics, we are not only equipped to detect -extreme throttling, but we are also gathering information about the performance +extreme, targeted throttling, but we are also gathering information about the performance achievable by clients in several world regions for reaching specific websites. The availability of HTTP headers and the practice of some CDNs of annotating the -responses with headers indicating which specific cache is being used could also -be exploited to make interesting network-performance statements. +responses with headers indicating which specific cache is being (as mentioned above +in the case of Cloudflare) used could also be exploited to make interesting +digital-divide statements. ## Future Work @@ -172,11 +220,12 @@ point for this research could be ["Strengthening measurements from the edges: application-level packet loss rate estimation" by Basso et al.]( https://www.sigcomm.org/sites/default/files/ccr/papers/2013/July/2500098-2500104.pdf). -An alternative approach would require the possibility of providing OONI experiments -with "richer input" parameters or dynamic experiments, aimed at answering more -specific research questions. For example, if there are reports that a website is -throttled by SNI, we could perform a download from a given test server with -certificate verification disabled, using the offending SNI and an innocuous SNI. +An alternative approach, already mentioned above, would require the possibility +of providing OONI experiments such as `urlgetter` with "richer input" parameters that +could provide additional data to answer more-narrow research questions. For example, if +there are reports that a website is throttled by SNI, we could perform a download +from a given test server with certificate verification disabled, using the offending +SNI and an innocuous SNI. Because HTTP/3 used QUIC and because QUIC operates in userspace, there is also the possibility of instrumenting the QUIC library to periodically collect @@ -188,7 +237,7 @@ such that we upload resources, therefore we would need to figure out whether it is possible to overcome this fundamental limitation first.) In the same vein, our Web Connectivity methodology does not currently factor in -the possibility of measuring upload speed throttling for HTTP/1.1 and HTTP2. However, +the possibility of measuring upload speed throttling for HTTP/1.1 and HTTP/2. However, anecdotal evidence exists that some countries may throttle the upload path or just have poor upstream connectivity towards interesting websites. A technique that has sometimes been applied is that of including very large headers into the request From ca788cbca9f518b670da42fa7db046cbef1fe875 Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Fri, 21 Jun 2024 11:25:38 +0200 Subject: [PATCH 05/17] x --- docs/design/dd-007-throttling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/dd-007-throttling.md b/docs/design/dd-007-throttling.md index ab3fcecd44..768bd8b736 100644 --- a/docs/design/dd-007-throttling.md +++ b/docs/design/dd-007-throttling.md @@ -3,7 +3,7 @@ | | | |--------------|------------------------------------------------| | Author | [@bassosimone](https://github.com/bassosimone) | -| Last-Updated | 2024-04-09 | +| Last-Updated | 2024-06-21 | | Reviewed-by | [@DecFox](https://github.com/DecFox) | | Status | accepted | From be3f40f2d0004832c68414621fe99bd7c5ed51b5 Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Fri, 21 Jun 2024 11:27:05 +0200 Subject: [PATCH 06/17] x --- docs/design/dd-007-throttling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/dd-007-throttling.md b/docs/design/dd-007-throttling.md index 768bd8b736..2dd52b7dc6 100644 --- a/docs/design/dd-007-throttling.md +++ b/docs/design/dd-007-throttling.md @@ -28,7 +28,7 @@ cases where a set of specific websites gets significantly worse performance whil overall users' internet experience is unchanged. This kind of throttling is in opposition to _generalized throttling_ where the internet experience is degraded regardless of the website compared to the previous internet experience (see [Dimming the Internet by Collin -Anderson](https://arxiv.org/pdf/1306.4361v1) for seminal work on this topic). +Anderson](https://censorbib.nymity.ch/#Anderson2013a) for seminal work on this topic). We, and other researchers, have documented extreme, targeted throttling in the past. See, for example: From a5fd1d9a4ab79969a4a763c5832a6238e4f9a849 Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Fri, 21 Jun 2024 11:27:55 +0200 Subject: [PATCH 07/17] x --- docs/design/dd-007-throttling.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/design/dd-007-throttling.md b/docs/design/dd-007-throttling.md index 2dd52b7dc6..04d69aae23 100644 --- a/docs/design/dd-007-throttling.md +++ b/docs/design/dd-007-throttling.md @@ -52,8 +52,9 @@ be useful to document episodes of extreme, targeted throttling. ## Methodology -The overall idea of our methodology is that we're not concerned with _how_ throttling -is implemented, rather we aim to show clearly-degraded network performance. +The overall idea of our methodology is that, as a first approximation, +we're not concerned with _how_ throttling is implemented, rather we aim to +show clearly-degraded network performance. We aim to detect such a degradation by comparing metrics collected by OONI Probe instances running in a country and network with measurements previously collected by users and/or with From ca8d4720a419ee4f281c263dd2f3401778e7b7ab Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Fri, 21 Jun 2024 11:31:41 +0200 Subject: [PATCH 08/17] x --- docs/design/dd-007-throttling.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/docs/design/dd-007-throttling.md b/docs/design/dd-007-throttling.md index 04d69aae23..b7eeee80a6 100644 --- a/docs/design/dd-007-throttling.md +++ b/docs/design/dd-007-throttling.md @@ -125,10 +125,15 @@ required to complete the TLS handshake should be a sufficient metric (and, in fa _is_ a performance metric used by speed tests such as [speed.cloudflare.com](https://speed.cloudflare.com/)). -As of Web Connectivity v0.4, we limit our data collection process to the TLS handshake, -but Web Connectivity v0.5 will extend the scope and collect up to 64 "read" events -including when we are downloading the body. The availability of more data will allow -us to better investigate throttling by analysing more "read" event samples. +Additionally, in Web Connectivity v0.5, the "read" events data collection does not +stop after the TLS handshake, therefore, we will have several post-handshake data +points we could also use to make statements about throttling. The size of the webpage +fetched from a given country and network, in fact, should also be pretty constant, +so a reasoning similar to the one made above for the TLS handshake also applies to the +process of handshadking and then downloading a web page. However, because very long +downloads could collect lots of "read" events, and because we want to limit the maximum +amount of "read" events we collected to 64, we have also introduced the following, +complementary metrics to investigate throttling. ### Download speed metrics From eb3150716fd7d1eb49b476ddeba8bbe5b93ae896 Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Fri, 21 Jun 2024 11:32:15 +0200 Subject: [PATCH 09/17] x --- docs/design/dd-007-throttling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/dd-007-throttling.md b/docs/design/dd-007-throttling.md index b7eeee80a6..ddc5fb7b98 100644 --- a/docs/design/dd-007-throttling.md +++ b/docs/design/dd-007-throttling.md @@ -133,7 +133,7 @@ so a reasoning similar to the one made above for the TLS handshake also applies process of handshadking and then downloading a web page. However, because very long downloads could collect lots of "read" events, and because we want to limit the maximum amount of "read" events we collected to 64, we have also introduced the following, -complementary metrics to investigate throttling. +complementary metric to investigate throttling. ### Download speed metrics From 057ebf32e4b6607f225c34e9e5360f4c0bd38da1 Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Fri, 21 Jun 2024 11:32:37 +0200 Subject: [PATCH 10/17] x --- docs/design/dd-007-throttling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/dd-007-throttling.md b/docs/design/dd-007-throttling.md index ddc5fb7b98..07d69b1b1e 100644 --- a/docs/design/dd-007-throttling.md +++ b/docs/design/dd-007-throttling.md @@ -137,7 +137,7 @@ complementary metric to investigate throttling. ### Download speed metrics -Additionally, Web Connectivity v0.5 collects download speed samples for connections +Web Connectivity v0.5 also collects download speed samples for connections used to access websites. We use the same methodology used by [ndt7]( https://github.com/m-lab/ndt-server/blob/main/spec/ndt7-protocol.md). We measure the cumulative number of bytes received by a connection using a truncated exponential From e24ad635e80f3a2fc08304e4b89fa0b75b8ec19f Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Fri, 21 Jun 2024 11:35:31 +0200 Subject: [PATCH 11/17] x --- docs/design/dd-007-throttling.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/design/dd-007-throttling.md b/docs/design/dd-007-throttling.md index 07d69b1b1e..acce9fd534 100644 --- a/docs/design/dd-007-throttling.md +++ b/docs/design/dd-007-throttling.md @@ -190,7 +190,9 @@ indicate specific geographic locations. For example, Cloudflare includes a `cf-ray` header indicating the specific cache that is serving the content using the name of the nearest airport. -Additionally, with the availability of richer input, it would become possible to +Additionally, with the availability of +[richer input](https://github.com/ooni/probe-cli/blob/master/docs/design/dd-008-richer-input.md), +it would become possible to run custom `urlgetter` experiments where we use possibly offending and possibly not offending SNIs with target addresses and possibly-unrelated addresses, thus giving us a chance to narrow down the cause of throttling to, say, the SNI being used. @@ -227,7 +229,9 @@ application-level packet loss rate estimation" by Basso et al.]( https://www.sigcomm.org/sites/default/files/ccr/papers/2013/July/2500098-2500104.pdf). An alternative approach, already mentioned above, would require the possibility -of providing OONI experiments such as `urlgetter` with "richer input" parameters that +of providing OONI experiments such as `urlgetter` with +[richer input](https://github.com/ooni/probe-cli/blob/master/docs/design/dd-008-richer-input.md) +parameters that could provide additional data to answer more-narrow research questions. For example, if there are reports that a website is throttled by SNI, we could perform a download from a given test server with certificate verification disabled, using the offending From 34842cb874cfc52a34e4f8caee754811095094f5 Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Fri, 21 Jun 2024 11:37:20 +0200 Subject: [PATCH 12/17] x --- docs/design/dd-007-throttling.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/design/dd-007-throttling.md b/docs/design/dd-007-throttling.md index acce9fd534..b624362256 100644 --- a/docs/design/dd-007-throttling.md +++ b/docs/design/dd-007-throttling.md @@ -195,7 +195,9 @@ Additionally, with the availability of it would become possible to run custom `urlgetter` experiments where we use possibly offending and possibly not offending SNIs with target addresses and possibly-unrelated addresses, thus giving -us a chance to narrow down the cause of throttling to, say, the SNI being used. +us a chance to narrow down the cause of throttling to, say, the SNI being used. This +kind of A/B experiments would basically replicate the functionality of [the prototype +that we originally wrote to investigate throttling](https://github.com/ooni/probe-cli/pull/684). Throttling could be caused by policers and shapers as well as by forcing specific users to pass through a congested path. When policers and shapers are used, we From 92a07532b9d39e783f34a8480d36cf2c76303ac9 Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Fri, 21 Jun 2024 11:38:58 +0200 Subject: [PATCH 13/17] x --- docs/design/dd-007-throttling.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/design/dd-007-throttling.md b/docs/design/dd-007-throttling.md index b624362256..aec624dd46 100644 --- a/docs/design/dd-007-throttling.md +++ b/docs/design/dd-007-throttling.md @@ -211,8 +211,8 @@ By collecting passive performance metrics, we are not only equipped to detect extreme, targeted throttling, but we are also gathering information about the performance achievable by clients in several world regions for reaching specific websites. The availability of HTTP headers and the practice of some CDNs of annotating the -responses with headers indicating which specific cache is being (as mentioned above -in the case of Cloudflare) used could also be exploited to make interesting +responses with headers indicating which specific cache is being used (as mentioned above +in the case of Cloudflare) could also be exploited to make interesting digital-divide statements. ## Future Work From 8e1b89fdbd6100a490108ce5f95a4371854f2db0 Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Fri, 21 Jun 2024 11:39:47 +0200 Subject: [PATCH 14/17] x --- docs/design/dd-007-throttling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/dd-007-throttling.md b/docs/design/dd-007-throttling.md index aec624dd46..2d562c5961 100644 --- a/docs/design/dd-007-throttling.md +++ b/docs/design/dd-007-throttling.md @@ -223,7 +223,7 @@ retransmission of the `SYN` segment. The TLS handshake also involves sending TCP back and forth in such a fashion that it's possible to extract RTT metrics. Howewer, we should be careful to exclude segments sent back to back. -In general, detecting more precisely the characteristics of throttling either +In general, detecting more precisely the characteristics of throttling requires additional research aimed at classifying the stream of events emitted by a receiving socket under specific throttling conditions. A possible starting point for this research could be ["Strengthening measurements from the edges: From df7e63daf292757bb3e3ad768120a7b5a26cbb19 Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Fri, 21 Jun 2024 11:40:31 +0200 Subject: [PATCH 15/17] x --- docs/design/dd-007-throttling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/dd-007-throttling.md b/docs/design/dd-007-throttling.md index 2d562c5961..e053682cb9 100644 --- a/docs/design/dd-007-throttling.md +++ b/docs/design/dd-007-throttling.md @@ -244,7 +244,7 @@ also the possibility of instrumenting the QUIC library to periodically collect snapshots about the receiver's state. However, in general, sender stats are much more useful to understand QUIC performance. This fact implies that we could instrument a QUIC library to observe the sender's state and gather information -about throttling uploads. (However, the whole design of Web Connectivity is not +about throttling uploads. (Yet, the whole design of Web Connectivity is not such that we upload resources, therefore we would need to figure out whether it is possible to overcome this fundamental limitation first.) From cd97bd888f64706d5e119001962f4a812efa3164 Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Fri, 21 Jun 2024 11:41:55 +0200 Subject: [PATCH 16/17] x --- docs/design/dd-007-throttling.md | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/docs/design/dd-007-throttling.md b/docs/design/dd-007-throttling.md index e053682cb9..54becfd1b1 100644 --- a/docs/design/dd-007-throttling.md +++ b/docs/design/dd-007-throttling.md @@ -244,13 +244,11 @@ also the possibility of instrumenting the QUIC library to periodically collect snapshots about the receiver's state. However, in general, sender stats are much more useful to understand QUIC performance. This fact implies that we could instrument a QUIC library to observe the sender's state and gather information -about throttling uploads. (Yet, the whole design of Web Connectivity is not +about throttling uploads. + +Yet, the whole design of Web Connectivity is not such that we upload resources, therefore we would need to figure out whether -it is possible to overcome this fundamental limitation first.) - -In the same vein, our Web Connectivity methodology does not currently factor in -the possibility of measuring upload speed throttling for HTTP/1.1 and HTTP/2. However, -anecdotal evidence exists that some countries may throttle the upload path or just -have poor upstream connectivity towards interesting websites. A technique that -has sometimes been applied is that of including very large headers into the request -body, even though servers may not necessarily accept such headers. +it is possible to overcome this fundamental limitation for HTTP/1.1 and HTTP/2 +first. A technique that has sometimes been applied is that of including very +large headers into the request body, even though servers may not +necessarily accept such headers. From 23ce507b0a6611ee7b1fdfd557fb85298be167d2 Mon Sep 17 00:00:00 2001 From: Simone Basso Date: Fri, 21 Jun 2024 11:58:03 +0200 Subject: [PATCH 17/17] Update docs/design/dd-007-throttling.md Co-authored-by: DecFox <33030671+DecFox@users.noreply.github.com> --- docs/design/dd-007-throttling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/dd-007-throttling.md b/docs/design/dd-007-throttling.md index 54becfd1b1..b1d3345d66 100644 --- a/docs/design/dd-007-throttling.md +++ b/docs/design/dd-007-throttling.md @@ -181,7 +181,7 @@ country and using the same autonomous system. Alternatively, we could compare measurements collected during the same time frame towards different websites, even though this signal is weaker because it can just be caused by interconnection issues. In any case, these considerations imply that our methodology rests -on the assumption that we will have several measurements for the targer websites, +on the assumption that we will have several measurements for the target websites, and our confidence would clearly lower with little available data. In analysing the data, it would also be useful to consider the possibility of