[New arch] Migrate OCUpload to new database

[abelgardep]

To keep working on our way to new architecture, we need to migrate the upload list to the room database.

To keep an eye on how it could be done, we can check OCFile Migration: #2955

New table name: transfers

Initially, we will use it only for uploads, keeping the legacy behaviour. It will be ready to be used by other transfers (downloads)

Fields from the SQLite table to migrate to new table:

_id -> incremental row id
local_path -> content URI from device
remote_path -> path to store in server
account_name -> user@serverURI
file_size -> in bytes
status -> (0, 1, 2, 3) -> QUEUED, IN_PROGRESS, FAILED, SUCCEEDED
local_behaviour -> (0, 1) -> COPY, MOVE
force_overwrite -> (0, 1) -> false, true
upload_end_timestamp -> when did it finish -> nullable
last_result -> nullable
created_by -> kind of upload
transfer_id -> nowadays not used, but could be used for worker id -> nullable

Fields to be removed:

upload_time
is_create_remote_folder

TASKS

• Research (if needed)
• Create branch new_arch/migrate_uploads_database
• Development tasks
  • Create a new TransferStatus in domain layer
  • Create a new TransferResult in domain layer
  • Create a new OCTransfer in domain layer
  • Create a new TransferRepository interface in domain layer
  • Create a new OCTransferEntity in data layer
  • Create a new TransferDao in data layer with CRUD operations
  • Add new entity and new Dao to Room database
  • Create a new LocalTransferDataSource interface in data layer
  • Create a new OCLocalTransferDataSource implementation in data layer
  • Create a new OCTransferRepository implementation in data layer
  • Add dependency injection of the new components
  • Deprecate old OCUpload datamodel
  • Replace old model uses with the new one and use of UploadsStorageManager
  • Remove unnecessary methods from old UploadsStorageManager
• Code review and apply changes requested
• Merge branch new_arch/migrate_uploads_database into new_arch/synchronization

[abelgardep]

Currently the uploads pending are stored in the old database.

Uploads are stored, retrieved, updated etc via UploadsStorageManager

Uploads data is in filelist database in list_of_uploads table. We need to migrate the data to the owncloud_database and create a new table there.

Domain

Model

Current class to represent an upload is OCUpload in the datamodel package in the ownCloudApp module.

We need to create a new data class in the domain layer in kotlin that represent those uploads.

Also, we need to move the UploadResult to the domain layer, which is basically the way to persist the result. Also, we should consider adding a way to persist not only a code, but also a message or something to show why. For example, when a virus is detected

Usecases

Use cases represent all the actions we can do with the app. In this case, we need to create a new usecase for each potential use for the uploads feature.

We can check the UploadsStorageManager and see which are the expected and useful usecases:

• Save upload
• Update upload
• Remove upload by id
• Remove upload by account
• Get uploads

Repository interface

The repository interface belongs to the domain layer and its just a facade that conects with the data layer

Data layer

In the data layer, we will add all the classes related with the datasources.
In this case, we won't have a remote datasource since the uploads is something internal. So we will focus on the local datasource.

Entity

We need to create a new entity that encapsulates all the data we want to store about an upload. Normally, it would be pretty similar to the domain model, but with basic types: Strings, booleans, ints...

Dao

It will basically have the CRUD operations needed to create, retrieve, update and delete data.

Room Database

Also, we need to update the room database introducing the new entity and also the new dao.

Normally, we would create a new MIGRATION file here but, in this case, we would use the same migration file as the OCFile migration one: Migration 37

To fill that migration, we can take advantage of the autogenerated file based on the entities. If we did the previous steps correctly, a new asset should be autogenerated in the assets folder. If we check the latest one, we should find the info for the new table, and we could use the "createSql" sentence to fill the migration.

Local Datasource

The local datasource will link the repository with the dao and also it will map the domain to the entity, and the entity to the domain.

We can check this local datasource as example

Repository implementation

Normally, the repository links and synchronizes the local and remote datasources. But in this case, we don't have a remote datasource, so it will communicate directly with the local datasource.

App layer

Now that we have finished with the domain and data layer implementation, we can go back to the app layer.

We need to add every new component to the dependency injection. https://github.com/owncloud/android/tree/new_arch/synchronization/owncloudApp/src/main/java/com/owncloud/android/dependecyinjection

To begin with, we could annotate the old datamodel with @deprecated. That way, we can identify easily where we are using the old model and where we are not.

Let's use the new usecases that we have created. For each call to UploadsStorageManager, we would replace with the new usecase.

For example, storeUpload function:

It is used in the camera uploads worker
Let's use the new domain model and the new storeUpload usecase in that file.

Repeat that for each storeUpload use from UploadsStorageManager.

Check that new items are being added to the new database. (App inspection in AS)

Repeat this for each function in UploadStorageManager.

Once we finished, the uploads view should show the data from the new database. Probably not 100% accurate, but most of it. UI improvements and progress and so on will be done in another issue.

Migrate old database data (in another issue)

Now that we use the new database to persist the uploads, we can migrate the data we had in the old database to the new one.

The logic we need to update is in the FileContentProvider

If we increased the database version in the process, we would need to introduce a new if to check the new and old version and migrate the data accordingly.

In this case, we didn't increased the version, since we are migrating the files and the uploads at the same time. so we can use that to migrate all together.

We will retrieve all the uploads from the old database, and introduce them into the new one.