Crash reporter should prompt user before sending each crash report

[cpeterso]

Why/User Benefit/User Problem

What / Requirements

Acceptance Criteria (how do I know when I’m done?)

---

The Mozilla Data Stewards team analyzed Google's new Play Store requirements and GeckoView's crash report PII and concluded that:

1. Crash reporting can be enabled by default (i.e. out-out) only if we prompt the user for permission before sending each crash report. The prompt may have an "Always allow" option, as long as the app settings have an option to opt out again.
2. Alternatively, crash reporting should be disabled by default (i.e. opt-in), but our new user onboarding can ask users if they would like to opt in.

Prompting before sending each crash report would be user-friendlier. We would not need to clutter our onboarding and users could choose to send crash reports for some sites but not others.

[brampitoyo]

Generally, our product is well enough built that we hope for crashes to only be rare exceptions rather than the norm. Because of this, users care about crashes at the (hopefully rare) moments when they happen (“I’ve lost my work, tabs, links”) and shortly afterwards (“How can I recover my work, tabs, links?”).

Our goal is for every user to turn crash reporting on. However, if a crash has never happened before, they may not care enough to turn crash reporting on as an individual item.

However, we can communicate things differently. For example, if our onboarding message is “Make Firefox better” or “Control How Firefox collects and uses your data”, then the user can relate to crash reporting enough to want to turn it on.

In this onboarding scenario, we can either:

1. Combine all the data collection toggles into one checkbox that says “Help make Firefox better by sending anonymous usage data and crash reports to Mozilla”.
2. Or, split the data collection into two separate toggles. One says “Send usage data”. The other says “Send crash reports”.

For new users, this may be a good path to pursue.

For existing users who have already gone past onboarding, it’s trickier. How do we communicate the fact that crash reporting is a setting that they should turn on, without disrupting the browsing activities they’re about to engage? Showing even a modal window can make users mildly interrupted and annoyed (at best) or angry (at worst).

So the approach that I’d recommend is either:

1. Different behaviours for new and existing users
   • For new users, crash reporting is a part of onboarding.
   • For existing users, crash reporting is a dialogue they only see after the first crash happened.
2. Same behaviours for both new and existing users: crash reporting is a dialogue they only see after the first crash happened.

Having the same behaviour (approach 2) is simpler, although it would be awkward to implement in GeckoView.

The crash reporting dialogue can explain the crash and offer users the chance to Send or Don’t Send a report (with a toggle to always do it in the future – we can check it by default).

If things are set to “Always Send” or “Never Send”, we can simply try to restore the user’s tabs without prompting them for confirmation ever again. Minimal disruption.

[cpeterso]

Thanks! Implementing the same behavior for both new and existing users (approach 2) makes sense. There's no need to bother new users during onboarding if we are already implementing the crash prompt for existing users.

Regarding the settings UI:

Focus+WebView currently has just one setting checkbox: "Send usage data. Mozilla strives to collect only what we need to provide and improve Firefox Focus for everyone." It is enabled by default in Focus (though opt-in for Klar) in both Focus+WebView and Focus+GeckoView because the telemetry does not include PII.

I think Focus+GeckoView will need separate data collection checkboxes for telemetry and crash reporting (like your "Send usage data" and "Send crash reports" example). Focus+WebView's crash reporting is enabled by default because neither the Google Play Store nor Sentry crash reports contain any PII. GeckoView's crash reports, however, contain stack memory that could conceivably contain PII, so Focus+GeckoView can't enable crash reporting by default.

Focus+GeckoView could default "Send usage data" to true and "Send crash reports automatically" to false (until the user selects "Always Send" on the crash prompt). If we wanted to get fancy, "Send crash reports" could offer multiple options (e.g. "Always Ask", "Always Send", "Never Send") instead of a checkbox.

[brampitoyo]

Focus+GeckoView could default "Send usage data" to true and "Send crash reports automatically" to false (until the user selects "Always Send" on the crash prompt).

Agreed.

I’ve made the crash reporter into an in-content page that will show up after the crash.

1. Is this a better approach than showing a dialogue? If I recall correctly, Android 9.0 Pie will no longer show a dialog when the app crashes, so in-content seems like a better idea.
2. Is it better to simplify the Ask/Send/Don’t Send/Always/Never choice into a simple checkbox? It’s easy to understand, and will directly mirror the “Send crash reports” checkbox that we will put under Settings.

Here’s an initial mockup. Thoughts?

[cpeterso]

1. Is this a better approach than showing a dialogue? If I recall correctly, Android 9.0 Pie will no longer show a dialog when the app crashes, so in-content seems like a better idea.

I don't have an opinion, but the in-content page looks good! :)

2. Is it better to simplify the Ask/Send/Don’t Send/Always/Never choice into a simple checkbox? It’s easy to understand, and will directly mirror the “Send crash reports” checkbox that we will put under Settings.

Would "Send crash reports" checked mean "Always Send" (without asking) and unchecked mean "Always Ask"? Or would "Send crash reports" checked mean "Always Ask" and unchecked mean "Never Send"?

If you would like the "Send crash reports" setting to be a simple checkbox, then I think the wording should be something like "Send crash reports automatically".

Here’s an initial mockup. Thoughts?

Would checking "Report crashes like this to help Mozilla improve Firefox Focus" mean "Always Send" and auto-restore a crashed session without showing the "Restore Session" page? Or does the checkbox just mean "Report this current crash"?

What would happen if the user pressed the ◀️ back button? Go to the Focus home screen with no open tabs?

[brampitoyo]

@cpeterso Let’s go with the in-content page proposal, then.

Clarifying a few details:

Would checking "Report crashes like this to help Mozilla improve Firefox Focus" mean "Always Send" and auto-restore a crashed session without showing the "Restore Session" page? Or does the checkbox just mean "Report this current crash"?

Checking the box means “Always send report”, not “Report only this crash”.

Originally, I thought that we should try to auto-restore and skip this page.

Meaning:

• If checkbox is ON and you tap “Restore Session”: in future crashes, previous session will be restored and skipping this page, and a report will be sent.
• If checkbox is OFF and you tap “Restore Session”: in future crashes, previous session will be restored and skipping this page, but no report will be sent.

But in a private browser like Firefox Focus, we should give users a choice to either restore or start from scratch (ie. tap the URL bar to enter an address). So this page should not be skipped.

We hope that crashes don’t happen a lot, so that this page (which is a “speed bump” towards the user’s destination) only appears rarely.

What would happen if the user pressed the ◀️ back button? Go to the Focus home screen with no open tabs?

Pressing the back button can either:

1. Quit the app, erasing previous stored session
2. Or, return to the start screen with the keyboard up and URL bar focused – ready for input

I recommend against quitting the app. The reason being, when the user sees this Restore Session page, the app had just quit (forcefully), so they’re unlikely to want to re-quit the app.

So we should instead go back to the start screen. And if the checkbox was turned ON, then a report would’ve been sent even if the “Restore Session” button isn’t pressed.

[cpeterso]

But in a private browser like Firefox Focus, we should give users a choice to either restore or start from scratch (ie. tap the URL bar to enter an address). So this page should not be skipped.

Sounds good. Summarizing your comment:

• Focus should always show the crash reporter prompt so the user has the option restore their crashed session or tap in the address bar to load a new page. Thus Focus would not have an "Always Send Reports" setting.
• Pressing the back button should instead go back to the start screen. And if the checkbox was turned ON, then a report would’ve been sent even if the “Restore Session” button isn’t pressed.

[pocmo]

Some comments:

• If we intent to build this as a component (Component Idea: Crash reporting (Sentry / Socorro) android-components#506) for all apps then I'd prefer if we start with a generic UI like Fennec's crash report dialog.

• Note that we need to spawn a completely new process (and "launch a new app") when crashing. We can't really show an in-content crash reporter message. Instead this means we would launch something new that looks like the browser and shows something in content. That's complex and with complexity comes the possibility of this code crashing.

• Because of the privacy nature of Focus we do not save anything to disk. This means we can't restore anything because the app process just crashed and with that the data is gone.

Focus+GeckoView could default "Send usage data" to true and "Send crash reports automatically" to false (until the user selects "Always Send" on the crash prompt).

Wait. This means we will never get any crashes in Sentry without the user enabling this. Or do we want to always prompt - even for non-native code crashes?

[pocmo]

That's btw. the crash screen of Fennec:

[cpeterso]

Because of the privacy nature of Focus we do not save anything to disk. This means we can't restore anything because the app process just crashed and with that the data is gone.

Does the crash reporter process have access to the URL from the crash report? If so, the reporter could relaunch the original app (e.g. Focus) with the URL. OTOH, any other Focus tabs or session state would still be lost, so the extra work of restoring the one crashing tab might be not be worth the effort.

Focus+GeckoView could default "Send usage data" to true and "Send crash reports automatically" to false (until the user selects "Always Send" on the crash prompt).

Wait. This means we will never get any crashes in Sentry without the user enabling this. Or do we want to always prompt - even for non-native code crashes?

The Mozilla Data Stewards say we don't have to show a prompt for non-native code crashes (that don't include PII), but it would be nice to give the user the choice. Prompting for some crashes but not others could make the user experience more confusing.

[brampitoyo]

@pocmo wrote:

Because of the privacy nature of Focus we do not save anything to disk. This means we can't restore anything because the app process just crashed and with that the data is gone.

If we have no way of restoring previously opened tabs, this issue changes significantly. This means that there’s no user benefit of seeing the Crash Reporter UI every time Focus crashes, because they don’t get anything out of it. No possibility of restoring their previous sessions.

[…] we need to spawn a completely new process (and "launch a new app") when crashing. We can't really show an in-content crash reporter message. Instead this means we would launch something new that looks like the browser and shows something in content.

I wonder if we can show a simple dialogue after the crash, that will say (copy not final):

Sorry. Firefox Focus had a problem and crashed.

As a private browser, we never save and cannot restore your last browsing session.

[x] Report crashes like this to help Mozilla improve Firefox Focus

[Restart]

The “Close” button is unnecessary, because the user – who just had their task interrupted – just want to get back to their task as quickly as possible.

And once you’ve set a reporting preference, we can just restart as soon as the app has crashed, without showing any dialogue. There will be very minimal interruption that way.

Thoughts?

[cpeterso]

This means that there’s no user benefit of seeing the Crash Reporter UI every time Focus crashes, because they don’t get anything out of it.

Users might not want to submit crash reports for sensitive sites, like their bank. The crash reporter UI would give them a choice for each site.

[brampitoyo]

Users might not want to submit crash reports for sensitive sites, like their bank.

Good point, @cpeterso.

If sensitive personal information are sent along with crash reports and there’s a reason for users to be worried about this, we should give them an option to send (or not to send) every time. After all, our users are more privacy-sensitive than users of other browsers.

On the other hand, if we’re sure that our crash report data is sanitised and anynomised upon sending (so our users really have nothing to worry about, even after visiting sensitive sites), then asking them to reconfirm the “Send report” action seems like putting a roadblock where none needs to exist?

(Maybe I’m overcomplicating this problem, and showing a “We’re sorry” page is the right move to make).

[bbinto]

@pocmo - is the prompt UI going to be part of the Android components?

[pocmo]

@pocmo - is the prompt UI going to be part of the Android components?

Yeah, the plan is to have a generic prompt UI in the component and offer the app ways to customize this later on.

[vesta0]

@brampitoyo will you be working on the crash reporter UI?

[brampitoyo]

@pocmo @cpeterso Do we need to have our own in-content page that appears after the Focus is restarted, or can we show a native system dialogue immediately after the crash (without opening the app)?

You’ve already seen this in-content page:

And here’s a mockup of the system dialogue, which I’ve talked about before:

Any thoughts to our preferred approach?

[sv-ohorvath]

Verified on Nightly.