Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set usesEntities to true if there is an entityList attachment in the manifest #6494

Merged
merged 4 commits into from
Nov 6, 2024

Conversation

grzesiek2010
Copy link
Member

@grzesiek2010 grzesiek2010 commented Nov 4, 2024

Closes #6493

Why is this the best possible solution? Were any other approaches considered?

As discussed in the comments and the issue, we need to mark forms downloaded before the upgrade as using entities if they indeed rely on entities. Doing this when new entities arrive seems to be the simplest and most effective approach.

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

This update modifies the database (specifically the useEntities column) when new entities are being downloaded to ensure that forms using entities are correctly marked, even if they existed prior to the introduction of this column. I don’t foresee any particular risks or areas needing additional careful testing.

Do we need any specific form for testing your changes? If so, please attach one.

No.

Does this change require updates to documentation? If so, please file an issue here and include the link below.

No.

Before submitting this PR, please make sure you have:

  • added or modified tests for any new or changed behavior
  • run ./gradlew connectedAndroidTest (or ./gradlew testLab) and confirmed all checks still pass
  • added a comment above any new strings describing it for translators
  • added any new strings with date formatting to DateFormatsTest
  • verified that any code or assets from external sources are properly credited in comments and/or in the about file.
  • verified that any new UI elements use theme colors. UI Components Style guidelines

false,
false,
new ManifestFile("", List.of(
new MediaFile("file1", Md5.getMd5Hash(new ByteArrayInputStream("contents-updated".getBytes())), "http://file1", true)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contents don't necessarily need to be updated! It looks like when processing an update, if type=entityList is seen in the manifest, the list is processed to a database even when the contents haven't changed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I briefly thought that an alternative might be to move the check for an Entity List up so that it only happens if the media file contents have changed but actually we can't do that right now because the md5 hash will never match the manifest hash. We're relying on HTTP caching to not duplicate downloads. This will be improved as part of broader form download improvements.

Copy link
Member

@lognaturel lognaturel Nov 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I looked at the details of how and where entityAttachmentsDetected is set and I believe that will work as desired. The test should be updated to make it clear that a content change is not necessary.

Too bad there are two passes over the manifest but we can likely improve that as part of download improvements as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I looked at the details of how and where entityAttachmentsDetected is set and I believe that will work as desired. The test should be updated to make it clear that a content change is not necessary.

The logic needs to be updated as well if it should work in this way.

Copy link
Member

@lognaturel lognaturel Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, that makes sense. I see that you've now added a special branch just for having detected any Entity List which matches https://github.com/getodk/collect/blob/master/collect_app/src/main/java/org/odk/collect/android/formmanagement/ServerFormUseCases.kt#L108

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too bad there are two passes over the manifest but we can likely improve that as part of download improvements as well.

Now there is one see: 1d92ec3

@lognaturel lognaturel added the high priority Should be looked at before other PRs/issues label Nov 5, 2024
@grzesiek2010 grzesiek2010 marked this pull request as ready for review November 5, 2024 19:07
@lognaturel lognaturel changed the title Update the usesEntities column if there are new attachments with entities Set usesEntities to true if there is an entityList attachment in the manifest Nov 6, 2024
@seadowg seadowg merged commit e54753f into getodk:master Nov 6, 2024
6 checks passed
@dbemke
Copy link

dbemke commented Nov 6, 2024

We need to test upgrades to verify the issue- do we need the released master apk or some other apk might work?

@lognaturel
Copy link
Member

It would be best to use a master release APK. I’ll make one as soon as I can. Unfortunately I think it will be in a couple of hours.

@dbemke
Copy link

dbemke commented Nov 6, 2024

It would be best to use a master release APK. I’ll make one as soon as I can. Unfortunately I think it will be in a couple of hours.

Thank you. We'll test other PRs and this one probably tomorrow

@dbemke
Copy link

dbemke commented Nov 7, 2024

I'm not sure if registration form (with an old entity spec) works as expected after upgrading.
Steps to reproduce:

  1. Install the store version of Collect.
  2. Scan user "trees” Qr code (with old entities spec forms and entities in the dataset).
    https://test.getodk.cloud/#/projects/576/app-users
  3. Upgrade to the current master version.
  4. Go to the db files and check forms.db column "usesEntities” (values: follow-up "true", registration- "null").
  5. Go to "Start new form” and tap the arrow to refresh the list of blank forms.
  6. Go to the db files and check forms.db column "usesEntities” (values: follow-up "true", registration- "null").
  7. In Central add a new entity, then go to "Start new form” and tap the arrow to refresh the list of blank forms.
  8. Go to the db files and check forms.db column "usesEntities” (values: follow-up "true", registration- "null").

When I add a new entity and refresh the list of blank forms only follow-up form gets updated, registration form doesn't get updated cause for that form nothing changed because of the new entity so this means that in case of registration forms publishing the new version of the form would change value to "true".

@lognaturel
Copy link
Member

lognaturel commented Nov 7, 2024

The usesEntities values look as I expect them to.

As you said, we're not going to detect that a form that only creates/updates Entities is entity-related until we get an update to the form itself. That's ok because forms with the old spec version don't have any offline effects. The only way to get offline effects is to update the Entity spec version. At that point, usesEntities should be set to true.

This PR only targets Entity List attachments. If you think there might be an issue with pure registration forms that don't have Entity List attachments, please file another issue.

@seadowg
Copy link
Member

seadowg commented Nov 7, 2024

As you said, we're not going to detect that a form that only creates/updates Entities is entity-related until we get an update to the form itself. That's ok because forms with the old spec version don't have any offline effects. The only way to get offline effects is to update the Entity spec version. At that point, usesEntities should be set to true.

The only crack in that I can see is if someone has a newer spec entity form on their device in v2024.2 and then updates to v2024.3. I think that form wouldn't have opened in v20242 so it's a pretty out there scenario, but I want to acknowledge that in that situation their form wouldn't be blocked during updates until they updated the form itself.

@lognaturel
Copy link
Member

lognaturel commented Nov 7, 2024

I think that form wouldn't have opened in v20242 so it's a pretty out there scenario

That's a good point! This is one of the reasons we want to leave at least a month between the Collect and Central releases -- it will reduce the chance of this happening.

@dbemke
Copy link

dbemke commented Nov 8, 2024

Tested with Success!

Verified on a device with Android 10

Verified cases:

@WKobus
Copy link

WKobus commented Nov 8, 2024

Tested with Success!

Verified on a device with Android 15

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
behavior verified high priority Should be looked at before other PRs/issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"usesEntities" in forms.db after upgrading Collect
5 participants