Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #5001: Set Up Firestore and Upload Free Form Responses #5098

Merged
merged 92 commits into from
Jan 26, 2024

Conversation

adhiamboperes
Copy link
Collaborator

@adhiamboperes adhiamboperes commented Jul 21, 2023

Explanation

Fixes #5001.
This is PR 5 of 6 Planned PRs.

We are adding a whole new infrastructure for uploading data to Firebase Firestore.

Key Changes

  • Introducing a FirestoreDataController that handles Anonymous Authentication and logging to Firestore, based on connectivity status.
  • Adding the Firestore, Auth and AppCheck dependencies, and reppining these maven dependencies.
  • New event logger for converting event logs to documents(hashmaps) and uploading them to firestore.
  • Introduce AuthenticationController
    This new controller provides functionality to retrieve the current signed in user and also sign in a user anonymously, and can be called from anywhere. It has a fake that is to be used in tests.
  • Adding the newly created utility module files to CODEOWNERS
  • Creating wrappers for FirebaseAuth, and Firestore in order to allow switching out with fakes in tests and development versions using dagger.

The implementation relies on an AuthenticationWrapper to switch out the real and fake authentication controllers for testing purposes. To do this, I created a dagger module to provide the correct listener where needed, hence the test file changes in the app module tests.

Technical Decisions

  • I opted to create a new cache for events that will eventually be uploaded to Firestore. This means they will not be accidentally uploaded to Firebase.
  • I opted to use EventLogs and convert them to Documents on the fly as opposed to creating an entirely new model and logging infrastruture that is Firestore-specific.
  • I calculated the FirestoreLogStorageCacheSize based on the average length of a Google Review and the storage size computation specs defined by Firebase. I computed it based on 10 profiles on a device over a 3 month period.
  • I opted to use existing log upload infrastructure to upload to firebase, as opposed to creating a whole new upload workflow so I can reuse the existing test infrastructure as well.

Dependency Updates

  • I added dependencies for Firestore and Auth. The selected versions were both the earliest compatible with the rest of our existing dependencies and the latest that didn't cause proguarding issues. There were incidental updates to transitive dependencies, both major and minor.

Proguard

Added new -dontwarns for the following:
-dontwarn javax.naming.** Reference

Essential Checklist

  • The PR title and explanation each start with "Fix #bugnum: " (If this PR fixes part of an issue, prefix the title with "Fix part of #bugnum: ...".)
  • Any changes to scripts/assets files have their rationale included in the PR explanation.
  • The PR follows the style guide.
  • The PR does not contain any unnecessary code changes from Android Studio (reference).
  • The PR is made from a branch that's not called "develop" and is up-to-date with "develop".
  • The PR is assigned to the appropriate reviewers (reference).

Upload Screenshots

Firestore Upload

Screenshot 2024-01-23 at 11 18 26

Querying Synced Firestore Data on BigQuery

Screenshot 2024-01-23 at 11 26 26 Screenshot 2024-01-23 at 11 26 44

Some of the production code was updated in the previous commit.
These scenarios are covered in other tests so I'm fine with not trying to make them work.
This commit sets up the logging infrastructure, including tests.

Since Firestore is introduced, some tests in the app module may be broken, and will be fixed in the next commit.

This commit soes not handle actual upload to firestore yet, because firebase auth is not yet properly configured.
@adhiamboperes adhiamboperes requested review from a team as code owners July 21, 2023 14:04
@adhiamboperes adhiamboperes requested a review from seanlip July 21, 2023 14:04
@adhiamboperes adhiamboperes self-assigned this Jul 21, 2023
@BenHenning BenHenning removed their assignment Jan 11, 2024
The comment on timeout possibility and creating a sealed class for fakeauthenticationwrapperimp have not been addressed in this commit.
@oppiabot oppiabot bot assigned BenHenning and unassigned adhiamboperes Jan 15, 2024
Copy link

oppiabot bot commented Jan 15, 2024

Unassigning @adhiamboperes since a re-review was requested. @adhiamboperes, please make sure you have addressed all review comments. Thanks!

Copy link
Member

@BenHenning BenHenning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @adhiamboperes! Just had a few follow-up comments. Overall the PR LGTM (so I've approved it as such), but feel free to assign me to take another pass on the final deltas if you'd like (I'm fine with you resolving my follow-up comments, as well).

@oppiabot oppiabot bot added the PR: LGTM label Jan 26, 2024
Copy link

oppiabot bot commented Jan 26, 2024

Hi @adhiamboperes, this PR is ready to be merged. Please address any remaining comments prior to merging, and feel free to merge this PR once the CI checks pass and you're happy with it. Thanks!

- refactor FakeAuthState from sealed to enum class
- refactor auth timeout to constant
- fix remove uploaded logs logic
- fix firebase authwrapper test set up and naming
- refactor event log viewmodel list mapping
@adhiamboperes
Copy link
Collaborator Author

@BenHenning, I have resolved all the conversations as I was able to verify the changes myself. This PR is ready to be merged, thanks!

@adhiamboperes adhiamboperes enabled auto-merge (squash) January 26, 2024 21:59
@adhiamboperes adhiamboperes merged commit 164f33b into develop Jan 26, 2024
43 checks passed
@adhiamboperes adhiamboperes deleted the nps-optional-response-upload branch January 26, 2024 22:31
BenHenning added a commit that referenced this pull request Feb 8, 2024
## Explanation
Fixes #5334
Fixes #3618

The root issue is that ``//instrumentation`` was updated in #5098 to
depend on ``//testing``. However, the current dexer that we use in Bazel
has troubles dexing certain dependencies (including Robolectric &
Mockito), and since ``//testing`` transitively pulls in these test
dependencies, the instrumentation test binaries fail to build.

The main fix is isolating the new Firebase testing dependencies to their
own package instead of needing to pull in ``//testing``. This resulted
in some broad but trivial changes in other tests throughout the
codebase. Note that I opted for just updating ``//testing`` rather than
all affected deps lists to keep things a bit simpler (stricter deps will
fix this in a later PR as part of the Bazel chain using automation to
make things a lot simpler). This addresses issue #5334.

To keep this from happening again in the future, new smoke build tests
were added for instrumentation tests and the top-level
``//instrumentation:oppia_test`` to ensure that our
``ComputeAffectedTests`` utility correctly identifies and picks up these
tests as part of the CI run when relevant (historically, instrumentation
tests have been completely ignored in CI since they can't be run yet;
note this doesn't include the instrumentation package's unit tests since
these use Robolectric and _are_ included in CI runs). This addresses
issue #3618. Note that this was verified as working using an initial
commit to this PR that only added the new smoke tests and verifying that
``//instrumentation:oppia_test_binary_smoke_test`` is now picked up and
fails, per the log output (& CI results):

```
BAZEL_TEST_TARGETS: //instrumentation:oppia_test_binary_smoke_test
```

Finally, note that there are a couple of new test targets added under
``//domain/src/test/java/org/oppia/android/domain/auth`` since it was
noticed during the development of #5138 that these were missed in an
earlier commit to develop (and should be run in both Bazel & Gradle test
passes).

## Essential Checklist
- [x] The PR title and explanation each start with "Fix #bugnum: " (If
this PR fixes part of an issue, prefix the title with "Fix part of
#bugnum: ...".)
- [x] Any changes to
[scripts/assets](https://github.com/oppia/oppia-android/tree/develop/scripts/assets)
files have their rationale included in the PR explanation.
- [x] The PR follows the [style
guide](https://github.com/oppia/oppia-android/wiki/Coding-style-guide).
- [x] The PR does not contain any unnecessary code changes from Android
Studio
([reference](https://github.com/oppia/oppia-android/wiki/Guidance-on-submitting-a-PR#undo-unnecessary-changes)).
- [x] The PR is made from a branch that's **not** called "develop" and
is up-to-date with "develop".
- [x] The PR is **assigned** to the appropriate reviewers
([reference](https://github.com/oppia/oppia-android/wiki/Guidance-on-submitting-a-PR#clarification-regarding-assignees-and-reviewers-section)).

## For UI-specific PRs only
N/A -- This change only targets & fixes test-only infrastructure.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Cloud Firestore to the App
5 participants