Add sync routes for v3.7

[nemunaire]

Hi,

There are new things in the release 3.7.

We have new routes to retrieve files and metadata. And apparently it seems to have a new permission/scope in the JWT.

The new route I was able to identify is /sync/v3/files/<FILE-HASH>. I think I handle it the right way (but I'm unable to finish the sync process due to the JWT issue.

For the JWT part, the server is spammed with requests on. We can see this in the logs:

time="2023-10-17T19:24:33Z" level=info msg="Using sync 1.5"
time="2023-10-17T19:24:33Z" level=info msg="setting scopes: intgr screenshare hwcmail:-1 mail:-1 sync:tortoise"
time="2023-10-17T19:24:33Z" level=info msg="[GIN] 2023/10/17 - 19:24:33 | 200 |    3.611147ms |      172.17.0.1 | POST     \"/token/json/2/user/new\""

Oct 17 19:13:10 reMarkable xochitl[1981]: 19:13:10.002 rm.network.usertoken     Scheduled token update at 2023-10-18 19:08:09, 86098997ms from now (scheduleUpdate /__w/xochitl/xochitl/src/network/src/usertoken.cpp:139)
Oct 17 19:13:10 reMarkable xochitl[1981]: 19:13:10.003 rm.network.usertoken     UserToken: setting a new userToken ("fqKdbGciDdJIUzd1QiIe"...) (setUserToken /__w/xochitl/xochitl/src/network/src/usertoken.cpp:74)

I'm trying to activate a cloud account on the official remarkable website, but for the moment it's not a success.

Can someone share the decoded payload (using https://jwt.io/) of a JWT user token, with personal information redacted?

[nemunaire]

With the new route, sync is back working, all features works well.

I noticed that sync scope changed from tortoise to fox:

-  "scopes": "intgr screenshare hwcmail:-1 mail:-1 sync:tortoise",
+  "scopes": "screenshare hwcmail:-1 mail:-1 intgr sync:fox",

But it doesn't seem to affect the synchronization in any way.

Also my rmfakecloud is still spammed with renewed user token, until xochitl restart... but it reappears randomly (also not noticeable for the user).

[zcy85611]

With the new route, sync is back working, all features works well.

I noticed that sync scope changed from tortoise to fox:

-  "scopes": "intgr screenshare hwcmail:-1 mail:-1 sync:tortoise",
+  "scopes": "screenshare hwcmail:-1 mail:-1 intgr sync:fox",

But it doesn't seem to affect the synchronization in any way.

Also my rmfakecloud is still spammed with renewed user token, until xochitl restart... but it reappears randomly (also not noticeable for the user).

thanks for the work, may I ask if the screen sharing still worked? mine is not working since recent updates

[nemunaire]

Hi @zcy85611, regarding the screen sharing feature, I haven't had the chance to test it since I don't have/use the reMarkable app at the moment. Therefore, I'm unable to confirm its current functionality with the latest updates :(

[zcy85611]

With the new route, sync is back working, all features works well.
I noticed that sync scope changed from tortoise to fox:

-  "scopes": "intgr screenshare hwcmail:-1 mail:-1 sync:tortoise",
+  "scopes": "screenshare hwcmail:-1 mail:-1 intgr sync:fox",

But it doesn't seem to affect the synchronization in any way.
Also my rmfakecloud is still spammed with renewed user token, until xochitl restart... but it reappears randomly (also not noticeable for the user).

thanks for the work, may I ask if the screen sharing still worked? mine is not working since recent updates

I've tested on latest firmware, it's not working, no error log on both server and client side, it happens couple of versions ago

[fuerst]

I confirm this PR is working with rM v3.6 as well as v3.7. Thanks for the work!

[suppenkaspar109]

Hi @nemunaire, would it be possible to add a linux/x86_64 image to your docker hub image at https://hub.docker.com/r/nemunaire/rmfakecloud/tags, so I can test the changes easily?

[Eeems]

@ddvk

[feurer98]

I tested it with version 3.5.2.1807 and it looks like everything works. not right away but that was probably due to me.

Edit: Also tested with version 3.7.0.1930. No errors in the logs and uplode and downloade works. worked here straight away
I currently have the account and the subscription is also displayed. However, a limit is displayed for a file that I haven't used for a long time.

[nemunaire]

I also have the cloud limit message on old files. I need to investigate this issue.

[nemunaire]

Ok, first regarding my server being spammed with new token requests, this was due to ping.remarkable.com not redirected to my rmfakecloud instance (until now, it seems they didn't check the JWT or didn't respond with 401). After adding the new host in /etc/hosts, the problem disappear. (The documented procedure states that ping.remarkable.com has to be in the /etc/hosts, there is no particular change, this was just my fault.)

Regarding the screen sharing feature, cc @zcy85611, the reMarkable doesn't contact rmfakecloud. It's not a missing route. This need more investigations.

Regarding the cloud storage limit, in fact sync:fox is for non-Connect account. I revert to sync:tortoise. (For those who try the PR before this message, you can run rmfakecloud with the new commit, your tablet will sync back all files within 3 hours, or re-pair your account.)

[nemunaire]

Thank you all for testing and giving feedback!