-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
⬆️ Upgrade to the latest API versions on android and iOS #934
Comments
These were some API differences I noticed between API 32 and 33:1. Android.location
2. Permissions
3. Android.gms.location
4. android.content.contrast
|
You might want to check out our plugins: (from the project package.json)
to see which APIs we use and whether they are in fact affected Also, you should see:
|
@niccolopaganini where did you get these API changes from? They are not listed in The only one that I see from this list ( Ah but we already ask for unrestricted background operation, so this should be covered. Should test to verify |
@shankari https://developer.android.com/sdk/api_diff/33/changes - this was the website I primarily referred to. |
I went through the plugins again. As @shankari mentioned previously, these were the following plugins @shankari and team had written:
Edit: I cross-referenced some packages that caught my eye. |
@niccolopaganini that's good. can you outline the method that you used to verify this? a listing of the unique packages used in each plugin and a link to the related API doc page would be helpful. |
Update on the aforementioned packages:1.cordova-plugin-em-datacollectionandroid->location->OPGeofence->ExitActivityIntentService &WalkExitWorker has java.util.locale; TripDiaryStateMachineForegroundService.java has NotificationManager; Verification->SensorControlChecks has NotificationManager; wrapper -> Battery.java has BatteryManager 3. cordova-plugin-em-server-communicationjava.net.URL 4. cordova-plugin-em-serversyncNotification Manager 5. cordova-plugin-em-settingsjava.net.URL 6. cordova-plugin-em-unifiedloggerjava.io.File; NotificationManager 2. cordova-plugin-em-opcodeauth & 6. cordova-plugin-em-unifiedloggerNone |
I "cross-verified" the API update and narrowed them down to these packages:
I have determined these packages based on the potential use cases that can take advantage of. Edit: Based on one of your comments regarding the "android.gms.location," I didn't find anything that might directly have a direct influence in this migration update |
I am instead looking for something like: "I searched for all strings starting with "package". Then I went to URL
Also what are the changes in these packages? Do they affect the functionality that our plugins use?
I am not sure what you mean by this. Are you saying that the only packages that
I am not sure what you mean by this. We may use the cordova splashscreen, but we definitely do not use the camera or geolocation plugins. Where did you find references to them? |
Process:I started out with the links that you suggested: 1. https://developer.android.com/about/versions/13/behavior-changes-13 2. https://developer.android.com/about/versions/13/behavior-changes-allOther links that I used: 3. https://developer.android.com/sdk/api_diff/33/changes- I used the "package.cordovabuild.json" file, and then navigated to the src directory for all plugins to locate the files with the packages.(entered by @shankari on behalf of @niccolopaganini ) From link 1, I identified the areas of android that were affected by the migration to APi 33 For instance, link 1 had indicated changes to foreground notification(s). The "data-collection" plugin had (the) "NotificationManager" (as part of the android.app package). This is why I added it to the list of packages that might be affected by the migration update. They have added a new method: https://developer.android.com/reference/android/app/NotificationManager.html#matchesCallFilter(android.net.Uri) |
I have three high-level comments on that process:
|
I am currently working on the script. I was able to extract from the website. However, I am somewhat unsuccessful in extracting text from the Java files. I am getting an output but also an error which I trying to iron out. Hopefully, I will be able to figure it out by 8/3/2023 afternoon Edit: Approach:
I'm currently troubleshooting this error. |
Update on scripts: I have worked on putting together two scripts: one that extracts the packages with classes from the API_diffs website (https://developer.android.com/sdk/api_diff/33/changes/alldiffs_index_changes) and one to use the extracted information from the previously-mentioned website to scan and match against the contents of the java files in a given directory. I had some issues with the web-scraping part where it would store as "hidden link." I was able to fix this issue but extracting the hyperlinks as texts and storing the information with the packages' respective classes. For the second python script, I tried the RE approach but since the CSV had multiple instances of special characters, I had to take an alternative approach to avoid problems mentioned in #934 (comment). I do this by manually iterating over each file in a given directory, check if it is a java file, and then cross-check the contents against the CSV. Although the code is reproducible, one thing to consider is that PATH defined for the script that matches the classes in JAVA files are absolute. To be more precise, you'll have to be very specific in defining the PATH that the script performs its operation on as it ignores directories. Edit: This way of approach is reproducible and therefore, minimal time is consumed in actually finding the classes. As this DOES NOT scan for the methods used, that work still have to be done manually, but by taking this approach, you'll reduce time as opposed to completely checking it manually. |
Update on Native Plugins List:Running the scripts through the directories, we were able to extract packages more easily and efficiently; and the whole process in reproducible. Amongst the several packages/ classes harvested, 27 were found to be unique and requires more attention. They are as follows:
The script to extract the packages and classes works. With this, I was able to extract the methods and compile them in a CSV. To not work on a code from scratch, I ran the CSV with the same python script to extract the methods present in the java file and I am troubleshooting it. This was the script that I used to find classes present in our plugins:
I tried modifying the code to find the methods and the data types:
In this case, the code does not return any error but also does not output anything at the moment. I was hoping it would return matching methods in a given directory and use that as a foundation to build for the data types but it seems like this approach may not be the best. |
I am not sure how you came to that conclusion. I don't see any debugging of the functions. What have you done to debug it? I don't even see print statements in here. |
I thought of looping over the list of methods; where the loop checks for the java classes. If there was a match, then the loop would print the method and directory where it was found |
I have modified the code to scan for the existing methods that might be deprecated, restored, etc.
I am doing the same for the present data types currently but I am not getting any matches. Fixing the code currently. |
Yes, but that doesn't address how you are debugging the issue.
Fixing the script, or fixing the android code? |
I was working on the script (on 8/11/2023). Current status: working. Process descriptionBy suggestion of @shankari, I started working on scripts (currently in Python, will have to be updated to shell in the future) - First script for webscraping that finds packages and its respective classes. Everything is compiled in a CSV. The CSV is then used in with the script that compare the CSV against the java files in a given directory. I created a common method:
You can call the function by:
Sample Output (that can be expected from the code):It takes absolute Paths, first one being CSV and the latter for the directory. One can use the classes and then narrow down the exact methods. I compiled the methods in a CSV stripping that can be counted as "common entity" (Screenshot below) This is also done for the previous CSV file. I do this for simplicity. For example: Before:After:It is to be noted that both CSVs had multiple counts of the same packages/ classes/ methods. Now, it can be solved with code but Excel's UNIQUE function is much easier which is what I used to simplify the list. Now continuing with the methods extraction, I modified the previous code as follows:
to return the methods In the CSV. The function is called with the following code:
Sample Output (An output that can be expected from the code): The classes that seem to be affected for this migration therefore are as follows:
|
@niccolopaganini high level comments:
Wait, so you are saying that you can't just run the scripts one after the other? The process involves opening the file in Excel in the middle? That is not even listed as a step in your process. We are not going to use a manual process using point and click in Excel as part of this flow. |
@niccolopaganini discussed this issue and I am stepping in to help out. It also looks like the there are not significant changes, based on the API diff lists that @niccolopaganini generated, so I am optimistic that we can still get this done by the end of the month. @niccolopaganini will continue with developing the script, which will be useful in the future to fix deprecated methods over the course of the year, until the next API upgrade season is upon us 😄 @niccolopaganini please file another issue for the script and copy your comments in there. I have collapsed them here to avoid overwhelming this issue. |
Looking through the list of behavior changes:
|
Verifications:
|
ok! I don't think we need any upgrades to our native code. Yay! Maybe we have finally gotten over the hump wrt privacy updates. Per the cordova blog Cordova has been upgraded to:
|
For disabling Android battery optimizations, why can't we show this popup? It's really bad UX to make people go through their phone's battery optimizations, find OpenPATH in the list, and disable it themselves. I actually think ALL of the permission requests should be popups instead of Settings shortcuts. But this one seems higher priority because it's the biggest inconvenience |
I agree that this is bad UX, and we do show popups whenever we can. If you try to install OpenPATH on an Android 6 or Android 7 phone, for example, it will have a simple popup for the location permission. However, Android has become progressively stricter about background permissions, particularly for location. The battery optimization popup, in particular, is because of this change in android 12 If we don't have a foreground service, we will only receive location updates "few times an hour" (https://developer.android.com/about/versions/oreo/background-location-limits) If our foreground service is killed, we cannot restart it from the background unless we qualify for one of the exceptions. The exceptions include:
We invoke the I am not sure what API the other app is using. But it is not what the android documentation indicates that we should use. I don't want to use an undocumented API for obvious reasons, but if you can find an alternate documented method, we can easily make the change. |
By this release do you mean the one on staging right now? If so, I can take care of updating the existing strings in en/es.json and reach out about lo.json if we have time, I'm not super familiar with Android yet, but I can probably research and/or test out instructions since I have test phones. I can also roll wider updates of any text into the work I'm doing with permissions in Profile, though that isn't used for the onboarding flow (yet) that is still the older page. |
Video of location permission popup on android 7 😄 android-7-permissions.mov |
Yeah I have observed that Android has gotten stricter. But that doesn't mean we can't show popups. Tons of other apps do. For location perms we will need a series of popups instead of just 1, because they make you specifically ask for "Precise location" and "all the time".
I think if we request for |
I know this would probably require native changes and I'm not necessarily prepared to dig into the Java code myself. |
That's a great find, @JGreenlee - I am happy to make that particular change along with these API updates. I wish the Android documentation would tell us the user-friendly method to use. |
Something to investigate Adds runtime permission. For your app to send non-exempt notifications, the user must grant this permission to your app. https://developer.android.com/develop/ui/views/notifications/notification-permission
We currently prompt the user to enable notifications from the app settings page. It seems like once they have done that, we would not need to request again. @niccolopaganini we should definitely test this. |
Before I continue, let's quickly check whether any of the other plugins require updates.
@niccolopaganini as you can see, none of these other plugins have changed. But maybe they should have been, and are only unchanged because they are unmaintained. We need to test all their functionality to make sure that it still works. So this is super urgent and critical to get right. |
Do we need to update the SDK tools? It looks like we already have SDK 33 support
However, the SDK manager now seems to have updates, including for the API 33 system image
We might as well update to test against the latest images, and now get some preliminary API 24 support as well.
Note that the cmdtools version itself has been upgraded to 10406996 |
FYI, this requires an additional permission
|
The local notification plugin seems to already have some of this?
|
Looking, at that plugin, I see this note:
Looks like at least in olden times, requesting the permission and launching the prompt could get you banned. And there are some reports that using the dialog does not put the app in the list. Although it looks like they may be less strict now as seen in react native permissions Flagging @niccolopaganini for testing issues reported in stackoverflow. |
@JGreenlee I am tempted to not add the permission and use the prompt given that I don't want to risk being banned so close to the API upgrade requirement. I think that the chances of being banned are low since we ask for all kinds of other permissions. And when they launch and test the app, they must know that we ask for the permission from the But it may be that adding the permission flags the app in a way that the manual review doesn't. And I really want to get the API update out without hitches. We can add the permission in the release after the API upgrade and see what happens. Thoughts? |
First level of upgrade: to e-mission-data-collection: |
Note that we should also upgrade the frameworks that we use - primarily the Google Play Services APIs for location and motion activity but again, not going to take that on now to minimize update footprint.
@niccolopaganini for visibility into future changes. |
Pending tasks:
Once those are done, we just need to check that the instruction text in the permission screen works for Android 13, and create another conditional in case it does not. |
Sure, if that's what you think is best. I am not familiar with the app release/ update process, or what constitutes a risk of being flagged, so I'd just defer to your expertise on that |
Given that we are also going to have a redesigned permissions screen in the next release. I am going to postpone this change also until then. |
Following up on this to find the dependencies for the file plugin which we updated. The plugin itself is Find all the exports in the file plugin and then see where they exist in our code.
services.js:
emailService.js
uploadService.js
If |
|
This addresses updating the SDK tools to the latest versions: e-mission/e-mission-docs#934 (comment) And removing the obsolete HAXM package: e-mission/e-mission-docs#958 (comment) It also updates the README to indicate the required java version after the upgrade e-mission#1016 (comment) e-mission#1016 (comment) Testing done: After upgrading to the most recent version of OpenJDK (17) ``` $ java --version openjdk 17.0.8.1 2023-08-24 OpenJDK Runtime Environment Temurin-17.0.8.1+1 (build 17.0.8.1+1) OpenJDK 64-Bit Server VM Temurin-17.0.8.1+1 (build 17.0.8.1+1, mixed mode, sharing) ``` Android SDK install succeeds ``` $ bash setup/prereq_android_sdk_install.sh --------------------------------------- Accept? (y/N): Y [=======================================] 100% Unzipping... android-12/zipalign END: Done with android SDK download, exiting script ``` And an android build in an existing checked-out repo succeeds ``` $ npm run build-dev-android BUILD SUCCESSFUL in 33s 52 actionable tasks: 12 executed, 40 up-to-date Built the following apk(s): .../platforms/android/app/build/outputs/apk/debug/app-debug.apk ``` Albeit with several deprecated APIs ``` w: /Users/kshankar/in-house/openpath-phone/platforms/android/app/src/main/java/com/adobe/phonegap/push/FCMService.kt: (169, 17): 'get(String!): Any?' is deprecated. Deprecated in Java w: /Users/kshankar/in-house/openpath-phone/platforms/android/app/src/main/java/com/adobe/phonegap/push/FCMService.kt: (316, 20): 'get(String!): Any?' is deprecated. Deprecated in Java w: /Users/kshankar/in-house/openpath-phone/platforms/android/app/src/main/java/com/adobe/phonegap/push/FCMService.kt: (627, 33): 'constructor Builder(Context)' is deprecated. Deprecated in Java w: /Users/kshankar/in-house/openpath-phone/platforms/android/app/src/main/java/com/adobe/phonegap/push/FCMService.kt: (1190, 37): 'fromHtml(String!): Spanned!' is deprecated. Deprecated in Java w: /Users/kshankar/in-house/openpath-phone/platforms/android/app/src/main/java/com/adobe/phonegap/push/PushPlugin.kt: (97, 25): 'get(String!): Any?' is deprecated. Deprecated in Java ```
Updates to the text for Android unused app settings here: 3435e12 |
We are currently at API 32 (upgraded in #838)
We need to be at API 33 (Android 13) by Sept 2023
https://developer.android.com/google/play/requirements/target-sdk
So, like we did before, we need to identify possible changes to background operation of the app, including background location, background motion activity and background launches in general for syncing the data. And then we need to address them.
Behavior changes list: https://developer.android.com/about/versions/13/behavior-changes-13
https://developer.android.com/about/versions/13/behavior-changes-all
The text was updated successfully, but these errors were encountered: