This document records interesting code that we've deleted for the sake of discoverability for the future.
This dataset was no longer being actively used, and removing it allows us to
limit airflow's access to payload_bytes_raw.
We've removed the CI task from mozilla-pipeline-schemas that used this document sample, so there is no further need for the ETL to support it.
We no longer send data to Amplitude, so these views and scripts were no longer being used.
This pipeline was no longer being actively used. There may be need for a similar pipeline for an upcoming survey, so this removed code can serve as a useful reference in that effort.
AET was never released except for a short test in the beta population, and now the project has been decommissioned, so there is no longer any need for these derived tables.
The deviations_v1 table was used to understand the change of Firefox
desktop usage during Covid-19 pandemic in 2020. The data is no longer being
actively used.
We are now using dynamically generated queries for generic Glean ETL on top of baseline pings, so we have deprecated previous versions of daily and last_seen tables.
The smoot_usage_*_v1* tables used a python file to generate the desktop,
nondesktop, and FxA variants, but have been replaced by v2 tables that make
some different design decisions. One of the main drawbacks of v1 was that
we had to completely recreate the final smoot_usage_all_mtr table for all
history every day, which had started to take on order 1 hour to run. The
v2 tables instead define a day_0 view and a day_13 view and relies on
the Growth and Usage Dashboard (GUD) to query them separately and join the
results together at query time.
For telemetry_stable.main_v4 shredder used SELECT statements over single
clusters, then combined the result to remove rows from the table. This was an
attempt to improve performance so that reserved slots would be cheaper than
on-demand pricing, but it turned out to be slower than using DELETE
statements for whole partitions.