Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

connectivity issue since letsencrypt cert renewal (samsung s6 android 7.0) #4778

Open
duperray opened this issue Dec 20, 2021 · 20 comments
Open
Labels
T-Defect Something isn't working: bugs, crashes, hangs and other reported problems X-Needs-Info

Comments

@duperray
Copy link

Steps to reproduce

opening element app shows a red banner "connectivity to the server has been lost"

I'm using a homeserver (other client work fine, nexus android same version of element, web client etc...)

next is a exception from logcat (not sure if it's related)

12-20 18:06:37.601  3636 30788 E Parcel  : Class not found when unmarshalling: im.vector.app.features.home.HomeActivityArgs
12-20 18:06:37.601  3636 30788 E Parcel  : java.lang.ClassNotFoundException: im.vector.app.features.home.HomeActivityArgs
12-20 18:06:37.601  3636 30788 E Parcel  :      at java.lang.Class.classForName(Native Method)
12-20 18:06:37.601  3636 30788 E Parcel  :      at java.lang.Class.forName(Class.java:400)
12-20 18:06:37.601  3636 30788 E Parcel  :      at android.os.Parcel.readParcelableCreator(Parcel.java:2517)
12-20 18:06:37.601  3636 30788 E Parcel  :      at android.os.Parcel.readParcelable(Parcel.java:2471)
12-20 18:06:37.601  3636 30788 E Parcel  :      at android.os.Parcel.readValue(Parcel.java:2374)
12-20 18:06:37.601  3636 30788 E Parcel  :      at android.os.Parcel.readArrayMapInternal(Parcel.java:2727)
12-20 18:06:37.601  3636 30788 E Parcel  :      at android.os.BaseBundle.unparcel(BaseBundle.java:269)
12-20 18:06:37.601  3636 30788 E Parcel  :      at android.os.BaseBundle.getString(BaseBundle.java:992)
12-20 18:06:37.601  3636 30788 E Parcel  :      at android.content.Intent.getStringExtra(Intent.java:6727)
12-20 18:06:37.601  3636 30788 E Parcel  :      at com.android.server.am.ActivityStarter.startActivityLocked(ActivityStarter.java:336)
12-20 18:06:37.601  3636 30788 E Parcel  :      at com.android.server.am.ActivityStarter.startActivityMayWait(ActivityStarter.java:1094)
12-20 18:06:37.601  3636 30788 E Parcel  :      at com.android.server.am.ActivityManagerService.startActivityAsUser(ActivityManagerService.java:5822)
12-20 18:06:37.601  3636 30788 E Parcel  :      at com.android.server.am.ActivityManagerService.startActivity(ActivityManagerService.java:5627)
12-20 18:06:37.601  3636 30788 E Parcel  :      at android.app.ActivityManagerNative.onTransact(ActivityManagerNative.java:176)
12-20 18:06:37.601  3636 30788 E Parcel  :      at com.android.server.am.ActivityManagerService.onTransact(ActivityManagerService.java:3613)
12-20 18:06:37.601  3636 30788 E Parcel  :      at android.os.Binder.execTransact(Binder.java:573)
12-20 18:06:37.601  3636 30788 E Parcel  : Caused by: java.lang.ClassNotFoundException: im.vector.app.features.home.HomeActivityArgs
12-20 18:06:37.601  3636 30788 E Parcel  :      at java.lang.Class.classForName(Native Method)
12-20 18:06:37.601  3636 30788 E Parcel  :      at java.lang.BootClassLoader.findClass(ClassLoader.java:1346)
12-20 18:06:37.601  3636 30788 E Parcel  :      at java.lang.BootClassLoader.loadClass(ClassLoader.java:1406)
12-20 18:06:37.601  3636 30788 E Parcel  :      at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
12-20 18:06:37.601  3636 30788 E Parcel  :      ... 16 more
12-20 18:06:37.601  3636 30788 E Parcel  : Caused by: java.lang.NoClassDefFoundError: Class not found using the boot class loader; no stack trace available
12-20 18:06:37.602  3636 30788 W Bundle  : Failed to parse Bundle, but defusing quietly
12-20 18:06:37.602  3636 30788 W Bundle  : android.os.BadParcelableException: ClassNotFoundException when unmarshalling: im.vector.app.features.home.HomeActivityArgs
12-20 18:06:37.602  3636 30788 W Bundle  :      at android.os.Parcel.readParcelableCreator(Parcel.java:2545)
12-20 18:06:37.602  3636 30788 W Bundle  :      at android.os.Parcel.readParcelable(Parcel.java:2471)
12-20 18:06:37.602  3636 30788 W Bundle  :      at android.os.Parcel.readValue(Parcel.java:2374)
12-20 18:06:37.602  3636 30788 W Bundle  :      at android.os.Parcel.readArrayMapInternal(Parcel.java:2727)
12-20 18:06:37.602  3636 30788 W Bundle  :      at android.os.BaseBundle.unparcel(BaseBundle.java:269)
12-20 18:06:37.602  3636 30788 W Bundle  :      at android.os.BaseBundle.getString(BaseBundle.java:992)
12-20 18:06:37.602  3636 30788 W Bundle  :      at android.content.Intent.getStringExtra(Intent.java:6727)
12-20 18:06:37.602  3636 30788 W Bundle  :      at com.android.server.am.ActivityStarter.startActivityLocked(ActivityStarter.java:336)
12-20 18:06:37.602  3636 30788 W Bundle  :      at com.android.server.am.ActivityStarter.startActivityMayWait(ActivityStarter.java:1094)
12-20 18:06:37.602  3636 30788 W Bundle  :      at com.android.server.am.ActivityManagerService.startActivityAsUser(ActivityManagerService.java:5822)
12-20 18:06:37.602  3636 30788 W Bundle  :      at com.android.server.am.ActivityManagerService.startActivity(ActivityManagerService.java:5627)
12-20 18:06:37.602  3636 30788 W Bundle  :      at android.app.ActivityManagerNative.onTransact(ActivityManagerNative.java:176)
12-20 18:06:37.602  3636 30788 W Bundle  :      at com.android.server.am.ActivityManagerService.onTransact(ActivityManagerService.java:3613)
12-20 18:06:37.602  3636 30788 W Bundle  :      at android.os.Binder.execTransact(Binder.java:573)
12-20 18:06:37.603  3636 30788 I ActivityManager: START u0 {act=null typ=null flg=0x0 cmp=ComponentInfo{im.vector.app/im.vector.app.features.home.HomeActivity}} from uid 10204 on display 0

Outcome

connection with the server

Your phone model

Samsung S6

Operating system version

Android 7.0

Application version and app store

Elements Android 1.3.9 [40103092] (G-b4722) Matrix SDK 1.3.9 (581c557) olm 3.2.4 (from playstore)

Homeserver

personal server

Will you send logs?

No

@duperray duperray added the T-Defect Something isn't working: bugs, crashes, hangs and other reported problems label Dec 20, 2021
@ouchadam
Copy link
Contributor

does this still happen on 1.3.12?

@duperray
Copy link
Author

on 1.3.12 [40103122] (G-b5040) with SDK 1.3.12 (1695fde), olm 3.2.7
same connection issue.
the upper mentionned exception also occurs (don't really now if it's related).
logs are really long.
Wondering why I'm the only samsung s6 owner with this error ... (owner of 2 samsungs s6 with this error to be precise ;) )

@ouchadam
Copy link
Contributor

ouchadam commented Dec 22, 2021

I think the exception you've sent is likely the issue, to be extra sure is it possible to submit logs? (assuming the app doesn't crash before allowing them to be sent...)

Parcel  : java.lang.ClassNotFoundException: im.vector.app.features.home.HomeActivityArgs
Parcel  :      at java.lang.Class.classForName(Native Method)

it's a very strange issue, I've only seen this when compiling locally and there's a stale build cache 🤔
if the released apks were really missing classes everyone would be seeing the same error

this SO seems to suggest it could be a bug in some samsung android versions, there's a workaround suggested but it'll be a blind fix from our side as I'm unable to reproduce the issue locally

are any android OS updates available to you?

@duperray
Copy link
Author

I read the SO you mentionned, I'm not able to build the app myself, if you blind fix I would be pleased to test.
it seems that not only old tab3 but also some s8 device are affected.
there is no android OS updates available since ages (in the software info section of the phone, last security fix was provided june the 1rst 2018).
I wonder why this bug occurs now (till 1.3.8 worked like a charm), and why no samsung users complains about this here.
My phone have stock rom, perhaps people with such old device flashed a Lineageos more recent rom (although not supported anymore).
I was not able to downgrade to 1.3.8 (for unknown reason the apk is said to be corrupted by the installation system, which I doubt, as I downloaded it 3 times).
If I clear the cache of the application, will the user have to reconnect/recreate a session and reexchange all the keys?

@duperray
Copy link
Author

duperray commented Dec 22, 2021

sure I can send logcat logs, is it what you are asking for?
element is not crashing (just not connecting)

@ouchadam
Copy link
Contributor

sure I can send logcat logs, is it what you are asking for? element is not crashing (just not connecting)

the usual flow is a popup when the app launches after a crash but it sounds like in your case the app never gets that far, so it may not be possible

POPUP REPORT A BUG
Screenshot_20211222_131311 Screenshot_20211222_131323

1.3.8 was never released, that release was rejected by the play store, which would make your previous working release 1.3.7 (assuming you had the latest at the time), I assume the issue is caused by our Android 12 target update as the actual area of code that's crashing for you hasn't been changed since 1.3.2~ (but the android code behind the scenes will change due to the target change)

unfortunately it's not possible to downgrade without uninstalling, which means if you don't have another verified session or key backup you'll end up losing encrypted message

I read the SO you mentionned, I'm not able to build the app myself, if you blind fix I would be pleased to test.

Fantastic! I can provide debug builds which can be installed alongside the production releases from the play store/fdroid. We generate APKs on every code change via https://buildkite.com/matrix-dot-org/element-android/

If you could install this debug build (the very latest code from our develop branch) https://buildkite.com/organizations/matrix-dot-org/pipelines/element-android/builds/5041/jobs/bb7c3b2e-df69-46b8-9049-d6e488075091/artifacts/8df6de2b-b9ae-4d5f-9ba7-2967ef3077e8 and confirm that the app still crashes then I'll raise a change that applies the workaround from the SO post and we can try again 🤞

@ouchadam
Copy link
Contributor

ouchadam commented Dec 22, 2021

and here's a debug build with the workaround in #4798 applied https://buildkite.com/organizations/matrix-dot-org/pipelines/element-android/builds/5053/jobs/9143e8ad-4c23-4e89-a0d9-b155e2dc6342/artifacts/cd426fd5-bf59-476b-835a-9cf3aa0a2664

the workaround is only applied to home screen and may~ need applying to every screen in the app, so there may still be crashes in other screens

EDIT: should have mentioned this in the first post, always be careful when installing random builds from the internet and do your own due diligence!

@duperray
Copy link
Author

I installed the first debug build you submitted, installation worked, but upon the startup when I provide my server url, it tells:
ssl error. (my server is up and running with other client connected, I triple checked the url).
is it a limitation of debug build?

@ouchadam
Copy link
Contributor

ouchadam commented Dec 22, 2021

oooh! maybe the crash log is a red herring, is your homeserver using a letsencrypt certificate? as you're on 7.0 you've been hit by the root CA expiration #4253 (comment)

@duperray
Copy link
Author

yup, my server use letsencrypt, the openssl s_client -showcerts -connect my.home.server:443 | grep DST command returned

 i:O = Digital Signature Trust Co., CN = DST Root CA X3

I read the 4253 comment but I'm not sure what to do to workaround this ssl issue now.
You mean, from the begining the connectivity issue is related to this ssl error and the exception raised in the logs is
a consequence of this (the main home screen exception is raised because of the red banner connectivity issue)?

@ouchadam
Copy link
Contributor

ouchadam commented Dec 22, 2021

You mean, from the begining the connectivity issue is related to this ssl error and the exception raised in the logs is
a consequence of this (the main home screen exception is raised because of the red banner connectivity issue)?

yep as the app isn't closing it would seem the exception in the logs is only a logged warning rather than a crash and the real issue is ssl related

in the openssl s_client output, can you also confirm that the Peer signature type: is RSA? as Android 7.0 doesn't support ECDSA

EDIT: the default lets encrypt certbot configuration is to use RSA

@duperray
Copy link
Author

Yep I can confirm that Peer Signature type is RSA

@duperray duperray changed the title connectivity issue since 1.3.9 update (samsung s6) connectivity issue since letsencrypt cert renewal (samsung s6 android 7.0) Dec 23, 2021
@duperray
Copy link
Author

another strange thing, I have another samsung s6 device with android 7.0 (we acquired ability to change the broken screens of s6, that's why we all have s6 in da tribe ;) and an old 1.3.2 elements version running.
Believe it or not it connect without problem to my homeserver...
I think I really don't get it. does it mean it's not only a deprecated certificate path, but the client also changed something?

@ouchadam
Copy link
Contributor

ouchadam commented Dec 24, 2021

In 1.3.7 we also restricted some of the TLS ciphers we support #4192

// TLSv1.3.
CipherSuite.TLS_AES_128_GCM_SHA256,
CipherSuite.TLS_AES_256_GCM_SHA384,
CipherSuite.TLS_CHACHA20_POLY1305_SHA256,

// TLSv1.0, TLSv1.1, TLSv1.2.
CipherSuite.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
CipherSuite.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
CipherSuite.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
CipherSuite.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
CipherSuite.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,
CipherSuite.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256

https://github.com/square/okhttp/blob/master/okhttp/src/jvmMain/kotlin/okhttp3/ConnectionSpec.kt#L272

could you confirm the Protocol and cipher from the openssl command?

for example, matrix.org and my personal homeserver (using the default letsencrypt config via the ansisble matrix workbook) output this...

$ openssl s_client -showcerts -connect matrix-client.matrix.org:443    

...
 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   i:O = Digital Signature Trust Co., CN = DST Root CA X3
...
Peer signature type: RSA-PSS
...
Protocol  : TLSv1.3
Cipher    : TLS_AES_256_GCM_SHA384

to double check, are you able to sign in/up on 1.3.9+ on a matrix.org account?

@duperray
Copy link
Author

about protocol and cipher from openssl command, I can confirm:

...
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
...
---
No client certificate CA names sent
Peer signing digest: SHA512
Peer signature type: RSA
Server Temp Key: ECDH, P-384, 384 bits
---
SSL handshake has read 4572 bytes and written 471 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 28EC45CF080A81DE9E4429826C52AA38E2DC79208A3E5E58E5AE8C0C894A778E
    Session-ID-ctx: 
    Master-Key: CD19B839AA704A914B617E4D64B187126AE27A6809285DFEC212FD94A7659630EBDC4F8242207369C86FF20EBB1B6CD6
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1640363272
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes
---
closed

So, differences with your configuration:
signature digest: SHA256 for you and SHA512 for me,
Peer signature type: RSA-PSS for you and RSA for me,
Server Temp Key: X25519, 253 bits for you and ECDH, P-384, 384 bits for me,
...
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384 for you and New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256 for me.

according to your comment, the ECDSA P-384 keys I use are incompatible with my android 7.0 os.

I don't use any ansible playbook but it's a manual install with a default certbot configuration. Unfortunately,
I don't know how to tune my certbot configuration (is it possible) and ask for my specific matrix node compatible certificate...

for the double checking, using the debug build you sent me:

  • I can't access my homeserver (with an ssl error)
  • I can connect to matrix.org public node

@ouchadam
Copy link
Contributor

the server tmp key using the android 7.0 unsupported P-384 could be the problem 🤔 Server Temp Key: ECDH, P-384, 384 bits, which might be related to the TLS difference.

the ansible script (as far as I can tell) is also using the certbot default config, it could be your http server/proxy nginx/apache not enabling tls 1.3 that's causing the trouble~

@duperray
Copy link
Author

duperray commented Dec 28, 2021

ok, unfortunately my nginx/openssl version is too old and not supporting TLSv1.3.
I think now I will need major upgrade to make the things work correctly.
I have no xperience in migrating synapse instance, I hope it is possible
Thank you so much for your kindness and your time spent on this uninteresting old hardware issue.
Don't hesitate to ping me if you need any testing on android 7.0 or samsung S6.
I will keep you informed when I manage to upgrade os + nginx + synapse and if it finally solve the problem (matrix.org seems to work with my client so, hopefully
there is light at the end of the tunnel 😃 ).

@ouchadam
Copy link
Contributor

ouchadam commented Jan 5, 2022

🤞 hopefully the upgrades do the trick!

Thank you so much for your kindness and your time spent on this uninteresting old hardware issue.

Always happy to help out, thanks for helping debug

@edwardcwang
Copy link

Still happens - causes an "SSL Error" upon certificate renewal.

@bagvand
Copy link

bagvand commented Oct 1, 2022

Issue will happen when obfuscate = true.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T-Defect Something isn't working: bugs, crashes, hangs and other reported problems X-Needs-Info
Projects
None yet
Development

No branches or pull requests

4 participants