⬆️ Upgrade to latest everything 🎉

[shankari]

At long last, after clearing the decks with
e-mission/e-mission-phone#799
e-mission/e-mission-phone#800
e-mission/e-mission-phone#801
e-mission/e-mission-phone#802

I'm ready to start the annual ritual of upgrading to the most recent version of everything.
I will also plan to address #678
and add silent push notifications to android as well to address #677

[shankari]

Switched to cordova android@10.1.0, which is the most recent version (https://cordova.apache.org/announcements/2021/08/16/cordova-android-10.1.0.html).

It generated a warning about the targetSDK version being 30+, which is also required for any updates to the play store.

Looking at the behavior changes between android 10 (API 29) and android 11 (API 30), we have:
https://developer.android.com/about/versions/11/behavior-changes-all

• One-time permissions: Gives users the option to grant more temporary access to location, microphone, and camera permissions.
  • not significant, just another failure case which we need to handle
• Permission dialog visibility: Repeated denials of a permission implies "don't ask again."
  • not significant, we already have to open the app settings to get our "Always" permission anyway
• Auto-reset permissions from unused apps
  • this is new and annoying. Details at https://developer.android.com/topic/performance/app-hibernation. At first glance, seems to be similar to the "Always" re-check and prompt on iOS
• Background location access:
  • need to request foreground and background separately: not an issue since we don't actually request foreground access
  • "Allow all the time" not visible in system dialog; need to go to the app settings screen to fix: not an issue since we already do this

[shankari]

ok, so the hibernation changes are pretty evil and are going to get more evil in android 12 (API 31). In API 31, if the app is not launched (in the foreground) "for a few months", the app hibernates.

In API 30, the app runtime permissions (including location) are reset.
In API 31:

• the app runtime permissions are reset
• the app cannot run jobs or alerts from the background,
• the app cannot receive push notifications
• files in the app cache (including the DB?) are removed.

This is pretty darn terrible. The location permission will be reset, and we won't have the ability to notify the user that anything is wrong through notifications.

Apparently, if your app is primarily targets with background operation, you can ask users to exempt the app from hibernation. We are going to have to go down this route; fortunately, there is a programmatic way to do this.
https://developer.android.com/topic/performance/app-hibernation#exemption

Maybe we should just upgrade to API 31 right now and be a bit more future-proof. Let's see what other changes API 31 needs.

[shankari]

Note also that we can't put this off forever because, in Dec 2021, this reset permission is going to come to devices that: "Runs Android 6.0 (API level 23) to Android 10 (API level 29), inclusive, and is powered by Google Play services" as long as the targetSDK is 11. And we need targetSDK 11 for any native code updates.

So we may be able to limp along for a bit but will have to deal with it fully sometime.

[shankari]

Maybe we should just upgrade to API 31 right now and be a bit more future-proof. Let's see what other changes API 31 needs.
https://developer.android.com/about/versions/12/behavior-changes-all

• Bunch of UI improvements that we are not going to take advantage of just yet
• "restricted app standby bucket": needs further investigation
• microphone/camera visibility: unused
• package visibility into other apps: unused
• crypto algorithms: unused (directly)
• clipboard access: unused
• system dialogs: unused
• untrusted touch events/overlay: unused
• pressing back button: no custom implementation; should test
• custom notifications: unused
• approximate location: user may give only approximate; what does this do for background?
• motion sensor rate limiting
• data handling attribution access
• pending intent mutability
• foreground service launch restrictions

[shankari]

The foreground service launch restrictions are the worst. Basically, we can't restart the foreground service except for exemptions. Most of those don't apply.

• The most relevant option is: Your app receives an event that's related to geofencing or activity recognition transition. We should then be able to start the app when we do need the fine grained location access. But what happens if the service is killed while we are on the trip? That is not a geofence transition, and we are in the background. Or do they argue that this never happens.

• There is also an option to turn off battery optimizations for the app. We may have to fallback to that if the foreground service can indeed be killed during a trip (and I don't see any guarantees that it won't be).

[shankari]

Given the number of changes in android 12, including the foreground service changes, and the ominous "motion sensor rate limiting", I am going to stick with android 11 for now.

I will add the hibernation check for now, to get that out of the way, although it is less important with target API 30, where background operations and notifications are not restricted.

[shankari]

First, upgrading android...

• changed the targetSDKVersion to 30

$ npx cordova platform rm android
Removing android from cordova.platforms array in package.json
$ rm -rf plugins/
$ rm -rf platforms/

• reinstalled, so far, so good

$ npx cordova platform add android@10.1.0
Installing "cordova-plugin-local-notification" for android
android-sdk version check failed (/Users/kshankar/e-mission/e-mission-phone/platforms/android/cordova/android_sdk_version), continuing anyways.
...
Subproject Path: CordovaLib
Subproject Path: app
Adding phonegap-plugin-barcodescanner to package.json
config file *-Info.plist requested for changes not found at /Users/kshankar/e-mission/e-mission-phone/platforms/android/*-Info.plist, ignoring
config file *-Info.plist requested for changes not found at /Users/kshankar/e-mission/e-mission-phone/platforms/android/*-Info.plist, ignoring
config file *-Info.plist requested for changes not found at /Users/kshankar/e-mission/e-mission-phone/platforms/android/*-Info.plist, ignoring
cordova-plugin-androidx-adapter: Processed 160 source files in 5938ms

• build fails

$ npx cordova build android
> Task :app:compileDebugJavaWithJavac
/Users/kshankar/e-mission/e-mission-phone/platforms/android/app/src/main/java/org/apache/cordova/whitelist/WhitelistPlugin.java:25: error: cannot find symbol
import org.apache.cordova.Whitelist;
                         ^
  symbol:   class Whitelist
  location: package org.apache.cordova
/Users/kshankar/e-mission/e-mission-phone/platforms/android/app/src/main/java/org/apache/cordova/whitelist/WhitelistPlugin.java:32: error: cannot find symbol
    private Whitelist allowedNavigations;

[shankari]

Whitelist is deprecated in cordova android 10.x and greater (https://github.com/apache/cordova-plugin-whitelist). However, removing it requires updating the ionic plugin.

$ npx cordova plugin rm cordova-plugin-whitelist
The plugin 'cordova-plugin-whitelist' is required by (cordova-plugin-ionic), skipping uninstallation. (try --force if trying to update)

[shankari]

We are currently at cordova-plugin-ionic 5.4.7, currently latest is 5.5.1, so hopefully we won't have too much of an interface change. And the whitelist dependency was removed in ionic-team/cordova-plugin-ionic@34a202e#diff-6813833f33103dfcaa583d4f09e06dc84c0ce8ddd34c8b567b56c9728be58389 in v5.5.0, so upgrading to v5.5.1 should be good enough.

Builds now!!

BUILD SUCCESSFUL in 11s
49 actionable tasks: 18 executed, 31 up-to-date
Built the following apk(s):
	/Users/kshankar/e-mission/e-mission-phone/platforms/android/app/build/outputs/apk/debug/app-debug.apk

[shankari]

Note other android 10 updates (https://cordova.apache.org/announcements/2021/07/20/cordova-android-10.0.0.html), notably:

• migration away from file:// URLs (which we should not be using anyway). Does this mean that we don't need a custom webview and/or the http adapter plugin?
• update plugins to androidx. we should probably:
  • update our plugins to androidx
  • update other plugins to the most recent version
  • we do still have the androidx adapter in case the other plugins have not been adapted
• the release generates an aab instead of an apk by default, consistent with the new packaging requirements
  (https://android-developers.googleblog.com/2020/11/new-android-app-bundle-and-target-api.html). We will need to upgrade our packaging scripts.

[shankari]

wrt file:// plugins, we use them only in the context of the post-trip prompt.

www//js/incident/post-trip-prompt.js|51| icon: 'file://img/icon.png',
www//js/survey/multilabel/post-trip-prompt.js|54| icon: 'file://img/icon.png',
www//js/control/general-settings.js|242| icon: 'file://img/icon.png',

so we need to see how the local notifications work with it.
If this is too complex, I may drop support for the trip end notifications plugin.

Fortunately, it is pretty clear that since we were using the webview plugin all along, it won't really affect us in many other ways.
apache/cordova-android#1137 (comment)

I'm going to downgrade the priority of this.

[shankari]

Android Studio cannot open the project, though.
"Gradle sync failed: This version of the Android Support plugin for IntelliJ IDEA (or Android Studio) cannot open this project, please retry with version 4.2 or newer."

Trying to figure out how to upgrade the android support plugin. Might just upgrade to a newer version of Android studio if it is easier.

[shankari]

Looks like we can fix it either by downgrading gradle or upgrading android studio.
Since a major upgrade to gradle 7.1.1 was part of the cordova 10 upgrade, we will upgrade android studio.

UPDATE: Runs fine after updating

.

[shankari]

Now, let's upgrade the external plugins.

cordova-plugin-advanced-http 3.0.0 "Advanced HTTP plugin"   -> 3.2.2
cordova-plugin-androidx-adapter 1.1.3 "cordova-plugin-androidx-adapter" -> unchanged
cordova-plugin-app-version 0.1.9 "AppVersion" -> 0.1.12 (deprecated, do we need?)
cordova-plugin-badge 0.8.8 "Badge"
cordova-plugin-customurlscheme 5.0.1 "Custom URL scheme" -> 5.0.2
cordova-plugin-device 2.0.1 "Device" -> 2.0.3

cordova-plugin-em-datacollection 1.6.0 "DataCollection"
cordova-plugin-em-jwt-auth 1.6.4 "JWTAuth"
cordova-plugin-em-server-communication 1.2.3 "ServerComm"
cordova-plugin-em-serversync 1.2.5 "ServerSync"
cordova-plugin-em-settings 1.2.2 "ConnectionSettings"
cordova-plugin-em-transition-notify 1.2.6 "TransitionNotification"
cordova-plugin-em-unifiedlogger 1.3.3 "UnifiedLogger"
cordova-plugin-em-usercache 1.1.3 "UserCache"


cordova-plugin-email-composer 0.9.2 "EmailComposer" -> unchanged (remove? now that we have upload functionality)
cordova-plugin-file 6.0.2 "File" -> unchanged (remove? I think we use this primarily for the email)
cordova-plugin-inappbrowser 4.0.0 "InAppBrowser" -> 5.0.0
cordova-plugin-ionic-keyboard 2.2.0 "cordova-plugin-ionic-keyboard" -> unchanged (do we even need this?)
cordova-plugin-ionic-webview 5.0.0 "cordova-plugin-ionic-webview" -> unchanged
cordova-plugin-ionic 5.4.7 "cordova-plugin-ionic" -> 5.5.1
cordova-plugin-local-notification 0.9.0-beta.3 "LocalNotification" -> unchanged (remove or migrate to the maintained fork?)
cordova-plugin-x-socialsharing 6.0.0 "SocialSharing" -> 6.0.3
cordova-support-google-services 1.3.2 "cordova-support-google-services"
es6-promise-plugin 4.2.2 "Promise"
phonegap-plugin-barcodescanner 8.1.0 "BarcodeScanner" -> see below
phonegap-plugin-multidex 1.0.0 "Multidex"
phonegap-plugin-push 2.3.0 "PushPlugin" -> see below

The two complicated upgrades are:

• BarcodeScanner currently archived (https://github.com/phonegap/phonegap-plugin-barcodescanner). No suggested replacement. Current version on GitHub is 8.1.0. Current version on npm is 8.1.2 and is published by "@modjoulteam". They claim to have their own repo (https://github.com/Modjoul-Team/phonegap-plugin-barcodescanner), but it doesn't really exist. Switching to installing from GitHub to avoid hijacking. May have to fork and upgrade instead.
• PushPlugin not under active development, replace with https://github.com/havesource/cordova-plugin-push. This has had two releases - 1.0.0 and 2.00 and is currently testing 3.0.0-dev. Both 1 and 2 seems to have had "breaking changes" but they primarily involve bumping up libraries. let's try this and fall back to the old plugin if this doesn't work.

[shankari]

Getting some errors with building pods

Failed to install 'cordova-plugin-em-jwt-auth': Error: pod: Command failed with exit code 31
    at ChildProcess.whenDone (/Users/kshankar/e-mission/e-mission-phone/node_modules/cordova-common/src/superspawn.js:136:25)
    at ChildProcess.emit (events.js:314:20)
    at maybeClose (internal/child_process.js:1051:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
Failed to restore plugin "cordova-plugin-em-jwt-auth". You might need to try adding it again. Error: Error: pod: Command failed with exit code 31

and

Failed to install 'cordova-plugin-em-datacollection': Error: pod: Command failed with exit code 31
    at ChildProcess.whenDone (/Users/kshankar/e-mission/e-mission-phone/node_modules/cordova-common/src/superspawn.js:136:25)
    at ChildProcess.emit (events.js:314:20)
    at maybeClose (internal/child_process.js:1051:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
Failed to restore plugin "cordova-plugin-em-datacollection". You might need to try adding it again. Error: Error: pod: Command failed with exit code 31

Before resetting everything, I got a similar error with the new @havesource/cordova-plugin-push plugin.
Let's try to investigate this further.

I am still getting the same error for that plugin.

$ npx cordova plugin rm @havesource/cordova-plugin-push
Uninstalling @havesource/cordova-plugin-push from android
Subproject Path: CordovaLib
Subproject Path: app
Uninstalling @havesource/cordova-plugin-push from ios
Running command: pod install --verbose
Removing "@havesource/cordova-plugin-push"
Removing @havesource/cordova-plugin-push from package.json
kshankar-35069s:e-mission-phone kshankar$ npx cordova plugin add @havesource/cordova-plugin-push
Installing "@havesource/cordova-plugin-push" for android
Subproject Path: CordovaLib
Subproject Path: app
Installing "@havesource/cordova-plugin-push" for ios
Running command: pod install --verbose
Failed to install '@havesource/cordova-plugin-push': Error: pod: Command failed with exit code 31
    at ChildProcess.whenDone (/Users/kshankar/e-mission/e-mission-phone/node_modules/cordova-common/src/superspawn.js:136:25)
    at ChildProcess.emit (events.js:314:20)
    at maybeClose (internal/child_process.js:1051:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
pod: Command failed with exit code 31