CameraX update to rc02 (part 1)

[mzorz]

Description:

This PR makes all the necessary changes to bring CameraX in the Stories library from alpha-06 to latest release candidate at then moment of this writing, Rc02

In particular, this PR implements the SurfaceProvider interface, and we used PreviewView's source code and MediaPipe's implementation as examples.

Reasons to upgrade

• We’re currently on alpha06, which dates back to October 19, 2019. 17 other releases came after that one and they have a release candidate now so it’s time to check back in.
• Several bugfixes and a crash because of the CameraX library registering on app startup as a ContentProvider
• We’re generally looking at stabilizing the feature and studying how people are using it, and we don’t currently have any pressing feature updates so it’s a good moment to give it some bandwidth.
• Apart from the crash cited above, we’re looking into some cropping issues we think may be related to camera capture, so better have the basics covered before we attempt any further workarounds on that side.
• Another major reason to upgrade is described in this issue: restoring the RECORD_AUDIO permission for API < 23

Plan of action

Given we were several versions behind, but found alpha06 quite stable for the use cases we implemented (with some workarounds as well), we made a plan to make the upgrade in steps, to contain changes in a way that makes the refactor easier to understand.

The original plan after analyzing what each version brought in terms of changes was along these lines:

• alpha-07
• alpha-10 (which contains mostly renames and some changes to file saving interfaces for captured images)
• beta-07
• beta-10
• 1.0.0-rc-01, 1.0.0-rc-02

With several of the releases having some breaking API changes, no doubt the one that demanded more refactoring was alpha-07, which coincidentally was the first one to be tackled in the plan. However, after wrangling the needed modifications for alpha-07, we realized we weren't really that far from moving on with the rest of the versions all the way up to rc-02, so instead of generating unnecessary overhead by writing different stacked PRs for each version we decided to finish writing all the changes in one go, which would also make testing a lot easier. In the end, most of the changes were due to the surface provider pattern introduced in alpha-07 and how to go about it with our own BackgroundSurfaceManager.

History

In the Stories library architecture, we have a BackgroundSurfaceManager, which is essential to how it all works in Stories. One of the main needs we had when architecting the library was to keep the ability to have one Surface / TextureView and being able to switch between live preview / video recording / video playback easily, in a way that would feel snappy to the user.

This optimization (keeping the Surface and being able to serve it different data streams as it was attached to live camera preview or a pre-recorded video player, for instance), is in essence different from the way things were architected in the CameraX library to operate.

CameraX offers a new PreviewView component which encapsulates all of the problems live previewing or image capturing from a camera have, and then also take care of handling the rendering without much hurdle for the developer, at the expense of making it arguably harder to extend. There's a door open though, which is the Surface provider pattern introduced in alpha-07, which is the way in which CameraX allows developers to override and come up with their own defined behavior.

Citing the documentation here (https://developer.android.com/jetpack/androidx/releases/camera#camera-core-1.0.0-alpha07):

The Preview use case has been updated to accept a surface created and managed by the application to ensure Android best practices. It is highly recommended to use the PreviewView view class provided in the camera-view package.

Stories' BackgroundSurfaceManager operates at a horizontal level, while CameraX's PreviewView crosses all layers from the View layer to the hardware layer..

CameraX's PreviewView

The recommended way forward when using the CameraX library is to use PreviewView, but this would pose some bigger refactors on our side, given PreviewView is a vertical block (view, view handler, model, library stack) and it would be difficult to use with our own BackgroundSurfaceManager. Besides that, we would have to implement a SurfaceProvider ourselves and then provide the surface to CameraX library but most of the joy of PreviewView would then be left unused.

We did implement our own surface provider in this PR, but knowing in then long run it would be nice to stick to CameraX's recommended practices PreviewView

PreviewView is tested on Google's test labs (https://medium.com/androiddevelopers/camerax-a-glimpse-into-the-past-present-and-future-19e22d8d2748), which is certainly one of the main goals which gave birth to the the CameraX library effort in the first place. For this reason, we'll look forward into further refactoring BackgroundSurfaceManager to include PreviewView, once the VideoCapture use case is stabilized as well.

Current status

As of now, and as we'll see in subsequent PRs, the VideoCapture (undocumented) use case showed some regressions as per our tests on different handsets, so we're leaving this PR as a draft and documenting our findings so we can resume work on this later on.

To summarize:
Tested front / rear cameras switching use cases from ImageCapture to VideoCapture on then following handsets running Android 8.x / 9 / 10: Pixel 2, Pixel 3, Samsung J2, Samsung A31, Samsung Tab S5e, Xaiomi Redmi Note 5.

Video in front camera on the Samsung J2 & Samsung Tab S5e:

• stretched live preview https://issuetracker.google.com/u/1/issues/180121821
• mirrored recorded video
• sometimes just does not work (live preview gets stuck and the recorded video file is corrupt, about 50-60% of the times on the Samsung J2) https://issuetracker.google.com/u/1/issues/177278523, https://issuetracker.google.com/u/1/issues/177812686

General list in the issue tracker for CameraX:
https://issuetracker.google.com/u/1/issues?q=componentid:618491%20status:open%20video

Note: We also tried adding the use case to the basic CameraX sample app but results showed the same issues on the handsets we used to test.

[peril-automattic]

You can test the changes on this Pull Request by downloading the APK here.