Create throttling measurement methodology

[bassosimone]

This issue is about creating and adding a throttling measurement methodology to ooniprobe. The general idea to move forward is build upon our recent throttling measurements in Russia.

• webconnectivity LTE collects additional metrics to facilitate throttling measurements #1297
• engine: start sketching out throttling integration testing tool probe#2424
• webconnectivity: API driven A/B testing for webconnectivity LTE
• deployment of webconnectivity LTE with throttling metrics
• preliminary analysis of throttling data collected using webconnectivity LTE

[agrabeli]

For more context, this is what is written in our DRL proposal:

Bandwidth throttling is commonly being reported around the world (particularly during elections and large-scale protests), but an open methodology for measuring it in an automated way from local vantage points does not exist yet. Last year, we collaborated with Censored Planet on measuring the throttling of Twitter in Russia, but this was the result of manual tests run by in-country volunteer technologists and included the analysis of packet captures.

As part of this activity, we will implement a new open methodology for measuring throttling that will be shipped as part of the OONI Probe apps and run automatically from local network vantage points (without requiring any technical expertise from the user, nor the collection of packet captures). This will enable our global community to automatically measure throttling every day and contribute measurements as open data in real-time, supporting the efforts of advocacy groups worldwide (such as the global #KeepItOn campaign).

We have already started experimenting with some possible approaches for measuring the presence of bandwidth throttling. We recently, for example, investigated the throttling of Twitter in Russia amid the war in Ukraine through the analysis of data already present in OONI measurements. Through our analysis, we were able to infer that Russia’s throttling infrastructure appears to be centrally managed.

We plan to further systematize the collection and analysis of throttling-related information to detect this form of censorship at scale. In developing our methodology, we plan to collaborate with our partner, the Internet Outage Detection and Analysis (IODA) project of Georgia Tech, who are also planning on extending their platform to measure the presence of bandwidth throttling. OONI and IODA datasets and measurement approaches are different, and will therefore be complementary, offering richer insight into bandwidth throttling.

[agrabeli]

I have set a tentative timeline on zenhub which probably needs to be adjusted.

[bassosimone]

Here's additional contextual information from: ooni/probe#1680

Over the years, community members have frequently expressed the need for a new nettest designed to measure throttling. This was also more recently requested during RightsCon sessions.

Currently (to my knowledge), an open methodology for measuring throttling doesn't exist yet, and defining and measuring throttling remains an ongoing research question.

Some things to take into consideration:

• What do we mean by throttling? Are we referring to throttling towards specific services? We first need to come up with a specification and based on that evaluate which methodology makes sense (and is feasible).

• Community members frequently report throttling towards popular social media services, such as Facebook, Twitter, YouTube, Instagram, etc.

• Sometimes community members also report "generic throttling" to communicate unusually slow internet speed (without being specific to services). However, these reports often emerge during protests and other events, when it's possible that this could be explained by network congestion and other factors.

Currently, we have the NDT and DASH tests which are run against M-Lab servers. However, these tests are not sufficient for measuring throttling because:

• We don't have a baseline to compare these measurements against;

• These tests are usually user-initiated, and there is often testing bias involved (i.e. people often only run tests when they notice that their internet connectivity is unusually slow, instead of running them all the time).

We would ideally need a large volume of stable measurements (collected from the same AS network) over a long period of time in order to be able to derive statistically-useful information. While the OONI Probe apps support automated testing, the NDT and DASH tests are (currently) not included in automated runs because that would be too bandwidth-intensive. So it remains quite unclear how to collect large volumes of stable measurements from user-initiated tests.

If we were to measure throttling towards a specific service (e.g. Facebook), we would need that company to host a measurement backend that we can run tests against. In this scenario, the OONI Probe DASH test (for example) would run tests against M-Lab servers + Facebook's measurement backend. That way, we would be able to compare the 2 measurements in order to be able to start measuring potential throttling. Convincing companies to host such a measurement backend though can be challenging.

I am going to close ooni/probe#1680 because it's been superseded by this issue, but I also wanted to include the original Epic text to have all the context into the same place.