VPN-6324: Telemetry for web based authentication #10171
Conversation
Request for data collection review formAll questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.
These are metrics to learn more about how sign in is successful (or not) as we switch back to web-based authentication. How often do users have issues when signing in?
We want to ensure we don't have bad outcomes as we switch how we do authentication for the VPN.
Not collecting this data and just relying on user feedback - we want more confidence than that, however.
No, unfortunately.
Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.
This collection is documented in the Glean Dictionary at https://dictionary.telemetry.mozilla.org/"
1.5 years, with potential of requesting renewal down the line. [email protected]
No filters - all channels, countries, and locales, unless the user has opted out of data collection on that device.
When launching the app for the first time, a user is given an option for whether to allow data collection. After this initial set up screen, a user can always toggle data collection permissions in the System Preferences screen.
Dashboards that engineers and others will consult on a regular basis.
Within the Mozilla VPN team and Infrastructure team.
No |
|
Adding @travis79 for data review. Thanks! |
There was a problem hiding this comment.
Request for data collection review form
All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.
1. What questions will you answer with this data?These are metrics to learn more about how sign in is successful (or not) as we switch back to web-based authentication. How often do users have issues when signing in?
2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?We want to ensure we don't have bad outcomes as we switch how we do authentication for the VPN.
3. What alternative methods did you consider to answer these questions? Why were they not sufficient?Not collecting this data and just relying on user feedback - we want more confidence than that, however.
4. Can current instrumentation answer these questions?No, unfortunately.
5. List all proposed measurements and indicate the category of data collection for each measurement, using the [Firefox data collection categories](https://wiki.mozilla.org/Data_Collection) found on the Mozilla wiki.Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.
Measurement Name Measurement Description Data Collection Category Tracking Bug #
web_authentication::started Event when web-based auth commences. It includes an extra of a UUID, to be able to connect the start to success/fail metrics. (And the absence of a connected success/fail metric is considered an incomplete authentication.) interaction VPN-6324
web_authentication::successful Event when web-based auth succeeds. Includes an extra of a UUID (see above). interaction VPN-6324
web_authentication::failed Event when web-based auth fails. Includes an extra of a UUID (see above). interaction VPN-63246. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.This collection is documented in the Glean Dictionary at https://dictionary.telemetry.mozilla.org/"
7. How long will this data be collected? Choose one of the following:1.5 years, with potential of requesting renewal down the line. [email protected]
8. What populations will you measure?No filters - all channels, countries, and locales, unless the user has opted out of data collection on that device.
9. If this data collection is default on, what is the opt-out mechanism for users?When launching the app for the first time, a user is given an option for whether to allow data collection. After this initial set up screen, a user can always toggle data collection permissions in the System Preferences screen.
10. Please provide a general description of how you will analyze this data.Dashboards that engineers and others will consult on a regular basis.
11. Where do you intend to share the results of your analysis?Within the Mozilla VPN team and Infrastructure team.
12. Is there a third-party tool (i.e. not Glean or Telemetry) that you are proposing to use for this data collection?No
Data Review Form
- Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?
Yes
- Is there a control mechanism that allows the user to turn the data collection on and off?
Yes, via the application settings
- If the request is for permanent data collection, is there someone who will monitor the data over time?
NA
- Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
Category 2 - Interaction Data
- Is the data collection request for default-on or default-off?
Default on
- Does the instrumentation include the addition of any new identifiers ?
The events contain a "flow-id" that only links the events and doesn't specifically identify the user in any way.
- Is the data collection covered by the existing Firefox privacy notice?
Yes
- Does the data collection use a third-party collection tool?
No
Result
data-review+
| @@ -32,6 +34,7 @@ class TaskAuthenticate final : public Task { | |||
| AuthenticationListener* m_authenticationListener = nullptr; | |||
| AuthenticationListener::AuthenticationType m_authenticationType = | |||
| AuthenticationListener::AuthenticationInBrowser; | |||
| QUuid metricUuid = QUuid::createUuid(); | |||
There was a problem hiding this comment.
Hungarian notation might be oldschool but: m_metricUuid would be neat :)
There was a problem hiding this comment.
Good call, updated
Description
Add telemetry described in VPN-6323.
I also included links to some data reviews that had been previously neglected.
Reference
VPN-6324
Checklist