A bunch of miscellaneous upgrades #581

TacoTheDank · 2020-06-30T22:52:22Z

Sorry this is so big, didn't mean for that.

1st commit

Delete some unnecessary files
Clean up duplicates in .gitignore

2nd commit

Update gradle wrapper to 6.5 using git bash

3rd commit

Reformat and rearrange gradles to be easier to read
Update Android Studio 3.6.1 -> 4.0.0 (changelog)
Update bintray plugin 2.8.4 -> 2.8.5 (changelog)
Update coveralls plugin 2.4.0 -> 2.8.3 (changelog)
Grouped dependencies for easier reading

4th commit

libopac:
- Updated Jsoup 1.8.3 -> 1.13.1 (changelog)
- Updated org.apache.httpcomponents:httpmime 4.5.2 -> 4.5.12 (changelog)
- Updated joda-time 2.9.9 -> 2.10.6 (changelog)
- Updated OkHttp3 versions -> 3.12.12 (changelog)
- Updated streamsupport-cfuture 1.6.0 -> 1.7.2 (changelog)
- Updated JUnit 4.12 -> 4.13 (changelog)
  - Fixed deprecations in tests
- Updated hamcrest-library 1.3 -> 2.2 (changelog)
  - Fixed deprecations in tests
meaningdetector:
- Updated org.bouncycastle:bcprov-jdk15on 1.51 -> 1.65.01 (changelog)
opacapp:
- Updated AndroidX Multidex 2.0.0 -> 2.0.1 (changelog)
- Updated sentry-android 1.7.16 -> 1.7.30 (can't find changelog, but nothing broke)
- Updated Jsoup 1.8.3 -> 1.13.1 (changelog)
- Updated joda-time 2.9.9 -> 2.10.6 (changelog)
- Remove unused joda-convert
- Updated su.j2e:rv-joiner 1.0.6 -> 1.0.9 (changelog)
- Updated Updated OkHttp3 versions -> 3.12.12 (changelog)
- Remove unused okhttp-urlconnection
- Updated Retrofit 2.3.0 -> 2.6.4 (changelog)
- Updated JMustache 1.13 -> 1.15 (changelog)
- Replace obsolete material-intro fork with its upstream
  - Updated material-intro 1.6 (the fork version) -> 2.0.0 (changelog)
- Remove unused streamsupport-cfuture
- Updated play-services-safetynet 15.0.1 -> play-services-basement 17.3.0 (where the actual used class is from) (changelog)
- Updated Stetho 1.5.0 -> 1.5.1 (changelog)
- Remove unused LeakCanary
- Updated Mockito 3.2.0 -> 3.3.3 (changelog)
- Updated JUnit 4.12 -> 4.13 (changelog)
- Updated commons-io 2.5 -> 2.7 (changelog)
tests:
- Updated JUnit 4.12 -> 4.13 (changelog)
- Updated org.bouncycastle:bcprov-jdk15on 1.51 -> 1.65.01 (changelog)
- Updated Jsoup 1.8.3 -> 1.13.1 (changelog)

5th commit

Migrate Sentry to 2.x (changelog) (I hope I did it right)

6th commit

Replace deprecated legacy AndroidX

7th commit

Updated AndroidX AppCompat 1.0.2 -> 1.1.0 (changelog)
Updated AndroidX Preference 1.0.0 -> 1.1.1 (changelog)
Updated (and added) AndroidX SwipeRefreshLayout 1.0.0 -> 1.1.0 (changelog)
Updated AndroidX Work 2.2.0 -> 2.3.4 (changelog)

8th commit

Replace preference (deprecated in API 29) with AndroidX preference

9th commit

Fix a bunch of other deprecations

10th commit

Fix travis not getting SDK licenses

TacoTheDank · 2020-06-30T22:53:34Z

Accidentally created before finishing the comment above. Oops.

TacoTheDank · 2020-06-30T23:14:08Z

Done. Fixed travis not getting licenses as well.

TacoTheDank · 2020-06-30T23:35:31Z

So it failed again, but to a different SDK license error. I think that'll have to be on your end.

so that it is only initialized in release builds

it is used in libopac, e.g. in HttpClientFactory:190 (JavaNetCookieJar class)

johan12345 · 2020-07-05T20:53:43Z

Hi,
first of all, thank you very much 😍- this must have taken some time to go through all dependencies!

I started to test the updated version and found a bunch of things:

The app wasn't starting because Sentry was complaining that the DSN is missing. As we only use Sentry in release builds, this should not happen. I disabled Sentry's auto-init functionality in 27fc34b, which should solve this. We still need to think about how we supply the DSN to Sentry now (previously we used a sentry.properties file, which was not checked in to this repo, but that method seems to be deprecated with sentry-android 2.0) -> Cc @raphaelm
You removed the okttp-urlconnection dependency, but it was actually used in libopac (e.g. in HttpClientFactory:190 referencing the JavaNetCookieJar class). I have re-added the dependency in b233821.
In a few places, you have replaced getDefaultEncoding() or a hardcoded "UTF-8" with Charsets.UTF_8. Two problems with this:
- In some library API's in libopac, getDefaultEncoding() might actually return something else than "UTF-8". Replacing it with a fixed Charsets.UTF_8 might cause problems. Instead, we could put something like Charset.forName() inside getDefaultEncoding() to convert the String to a Charset instance.
- I am getting crashes due to java.lang.NoClassDefFoundError: Failed resolution of: Lorg/apache/commons/codec/Charsets;. It seems that the library including the Charsets class is missing.
The fork of material-intro included some layout changes. We should check how the intro looks now with the upstream library and whether we can live with that or need to update our fork.
I think the Travis build should be fixed now with a08f23f.

TacoTheDank · 2020-07-05T22:20:58Z

@johan12345 Yeah, sorry about all those.

I think sentry needs this to work for 2.x.

This migration guide will tell you all about migrating to the new 2.x.

I'll fix up the Charset stuff.

TacoTheDank · 2020-07-06T00:41:56Z

@johan12345 It's just because parse using String is deprecated for API 22 and up, and it wants people to use charsets instead. I'll revert all those just to be safe.

TacoTheDank · 2020-07-06T01:10:17Z

The built apks still crash with auto-init disabled. I believe the next commit I push will solve this. EDIT: It didn't, but at least the crashlog is different.

Here it is now:

FATAL EXCEPTION: main
Process: de.geeksfactory.opacclient, PID: 17741
java.lang.RuntimeException: Unable to get provider io.sentry.android.core.SentryInitProvider: io.sentry.core.InvalidDsnException: java.lang.IllegalArgumentException: Invalid DSN: No public key provided.
	at android.app.ActivityThread.installProvider(ActivityThread.java:7413)
	at android.app.ActivityThread.installContentProviders(ActivityThread.java:6953)
	at android.app.ActivityThread.handleBindApplication(ActivityThread.java:6844)
	at android.app.ActivityThread.access$1400(ActivityThread.java:268)
	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1982)
	at android.os.Handler.dispatchMessage(Handler.java:107)
	at android.os.Looper.loop(Looper.java:237)
	at android.app.ActivityThread.main(ActivityThread.java:7811)
	at java.lang.reflect.Method.invoke(Native Method)
	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1076)
Caused by: io.sentry.core.InvalidDsnException: java.lang.IllegalArgumentException: Invalid DSN: No public key provided.
	at io.sentry.core.Dsn.<init>(Dsn.java:26)
	at io.sentry.core.Sentry.init(Sentry.java:14)
	at io.sentry.core.Sentry.init(Sentry.java:5)
	at io.sentry.android.core.SentryAndroid.init(SentryAndroid.java:6)
	at io.sentry.android.core.SentryAndroid.init(SentryAndroid.java:2)
	at io.sentry.android.core.SentryInitProvider.onCreate(SentryInitProvider.java:5)
	at android.content.ContentProvider.attachInfo(ContentProvider.java:2113)
	at android.content.ContentProvider.attachInfo(ContentProvider.java:2087)
	at io.sentry.android.core.SentryInitProvider.attachInfo(SentryInitProvider.java:2)
	at android.app.ActivityThread.installProvider(ActivityThread.java:7408)
	... 10 more
Caused by: java.lang.IllegalArgumentException: Invalid DSN: No public key provided.
	at io.sentry.core.Dsn.<init>(Dsn.java:25)
	... 19 more

I'll undo the commit, so here's reference: 3645bf8

Sentry really wants a key, and will take no bullshit lol.

TacoTheDank · 2020-07-06T01:15:24Z

@johan12345 What's interesting is that upgrading Sentry to 2.x saves almost a megabyte in APK size (811 KB, the other 300 or so KB is due to other stuff).

You can check out exactly the differences by decompiling the APKs with https://github.com/iBotPeaches/Apktool.

Get 6.2.8 from F-Droid, and compare it to a built foss release APK from master.

TacoTheDank · 2020-07-06T01:41:04Z

@johan12345 Sorry for so many mentions lol, but here's what the material-intro looks like.

Your fork:

2.0.0:

This obviously isn't intended. I'll revert the change for now. You should make a new branch from 2.0.0 and make your own changes on top of it.

johan12345 · 2020-07-07T14:34:24Z

@TacoTheDank

The built apks still crash with auto-init disabled.

For the release build? The debug build is working for me now. For the release build we definitely need to supply the DSN. I'll ask @raphaelm how we can do that, maybe we just need to put it in the public repository.

I found one more thing: The tests in :libopac:tests were failing with

java.lang.NoSuchMethodError: org.hamcrest.Matchers.in(Ljava/util/Collection;)Lorg/hamcrest/Matcher;

that is fixed with f786ab2. But now there are some more test failures. That may be due to changes in JSoup, so we need to check what happened there.

johan12345 · 2020-07-07T15:05:56Z

@TacoTheDank with 7f6a297 I have emulated the previous Sentry behavior, i.e. loading the DSN from sentry.properties if it exists and ignoring it otherwise.

TacoTheDank · 2020-07-11T03:33:15Z

@johan12345 Awesome 👍

johan12345 · 2020-07-13T14:51:39Z

I reverted the JSoup upgrade for now, tests are passing now.

Changes in JSoup are critical, as we are parsing HTML pages of many different library catalogs. We will check the problems occurring with updated JSoup separately.

I will merge this PR now, thanks again to @TacoTheDank !

…ved by #581)

TacoTheDank added 9 commits June 29, 2020 16:37

Delete unneeded files

3abadd8

Update gradle wrapper

e6693b0

Update Studio and plugins, rearrange gradles

f4279f3

Update a bunch of dependencies

e8e5b57

Migrate Sentry to 2.x

10c49f4

Replace AndroidX legacy

6570ddd

Update AndroidX libraries

4096327

Replace deprecated preference with AndroidX preference

38a1cbe

Fix a bunch of deprecations

f214267

TacoTheDank changed the title ~~A bunch of miscellaneous upgrades~~ WIP: A bunch of miscellaneous upgrades Jun 30, 2020

TacoTheDank changed the title ~~WIP: A bunch of miscellaneous upgrades~~ A bunch of miscellaneous upgrades Jun 30, 2020

Fix travis

17e3146

johan12345 added 3 commits July 5, 2020 22:25

disable Sentry auto-init

27fc34b

so that it is only initialized in release builds

re-add okhttp-urlconnection dependency

b233821

it is used in libopac, e.g. in HttpClientFactory:190 (JavaNetCookieJar class)

travis: install build-tools-29.0.2

a08f23f

Fix error in manifest

af05fdb

TacoTheDank added 2 commits July 5, 2020 21:07

Revert some deprecation fixes from f214267

4ad72aa

Update gradle wrapper, add built apk directories to .gitignore

177e3e6

TacoTheDank and others added 2 commits July 5, 2020 21:49

Revert material-intro update for now

4e63c28

fix hamcrest upgrade in libopac

f786ab2

load Sentry DSN from sentry.properties

7f6a297

revert Jsoup upgrade for now

ec09fbd

johan12345 merged commit 8fcfa71 into opacapp:master Jul 13, 2020

This was referenced Jul 13, 2020

[WIP] JSoup upgrade #584

Open

Fix: No signature of method: ...cleanDirectory() #587

Closed

johan12345 added a commit that referenced this pull request Jul 23, 2020

re-add streamsupport-cfuture (it is a dependency of libopac, was remo…

fcb08a1

…ved by #581)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

A bunch of miscellaneous upgrades #581

A bunch of miscellaneous upgrades #581

TacoTheDank commented Jun 30, 2020 •

edited

Loading

TacoTheDank commented Jun 30, 2020

TacoTheDank commented Jun 30, 2020

TacoTheDank commented Jun 30, 2020

johan12345 commented Jul 5, 2020

TacoTheDank commented Jul 5, 2020 •

edited

Loading

TacoTheDank commented Jul 6, 2020

TacoTheDank commented Jul 6, 2020 •

edited

Loading

TacoTheDank commented Jul 6, 2020 •

edited

Loading

TacoTheDank commented Jul 6, 2020 •

edited

Loading

johan12345 commented Jul 7, 2020

johan12345 commented Jul 7, 2020

TacoTheDank commented Jul 11, 2020

johan12345 commented Jul 13, 2020

A bunch of miscellaneous upgrades #581

A bunch of miscellaneous upgrades #581

Conversation

TacoTheDank commented Jun 30, 2020 • edited Loading

1st commit

2nd commit

3rd commit

4th commit

5th commit

6th commit

7th commit

8th commit

9th commit

10th commit

TacoTheDank commented Jun 30, 2020

TacoTheDank commented Jun 30, 2020

TacoTheDank commented Jun 30, 2020

johan12345 commented Jul 5, 2020

TacoTheDank commented Jul 5, 2020 • edited Loading

TacoTheDank commented Jul 6, 2020

TacoTheDank commented Jul 6, 2020 • edited Loading

TacoTheDank commented Jul 6, 2020 • edited Loading

TacoTheDank commented Jul 6, 2020 • edited Loading

johan12345 commented Jul 7, 2020

johan12345 commented Jul 7, 2020

TacoTheDank commented Jul 11, 2020

johan12345 commented Jul 13, 2020

TacoTheDank commented Jun 30, 2020 •

edited

Loading

TacoTheDank commented Jul 5, 2020 •

edited

Loading

TacoTheDank commented Jul 6, 2020 •

edited

Loading

TacoTheDank commented Jul 6, 2020 •

edited

Loading

TacoTheDank commented Jul 6, 2020 •

edited

Loading