Single Sign On E2E test concept

[stefan-niedermann]

This document is subject of change at all time based on feedback in the comments.

ℹ️ Terms

• SSO lib: Nextcloud Single Sign On library
• 3rd party app: Any third party Android app the uses the SSO lib
• files app: The Nextcloud Android app
• server: A running Nextcloud instance that is accessible via Browser, REST, etc. ...

🙋‍♂ Motivation

The files app 3.18-RC1 introduced an issue which caused all 3rd party apps stop working. The aftermaths were complaining users, increased support efforts and 1-star ratings. We should avoid this in the future.

🏁 Goal

1. When changes in the files app or in the SSO lib lead to a breaking change, we must get notified
2. Having an real end to end test, which covers the flow of a 3rd party app authenticating and using the latest SSO lib in combination with the latest beta files app
3. Having an real end to end test, which covers the flow of a 3rd party app authenticating and using the latest stable SSO sample app in combination with the latest files app

➡️ Proposal

In order to achieve goal 2 and 3, i came to the conclusion that we eventually need two separate test flows. One should be located in the repository of the SSO lib and one in the repository of the files app.

Testing flows

We already know that those tests will have a quite long runtime due to the usage of an AVD emulator. Ideally they will run on each push event, but i am also fine with setting up a nightly running job. Maybe listening to push events makes sense for the SSO lib (where less commits are made, and we usually have more time), while setting up a nightly job for the files app.

Files app

1. Setup a temporary server using the Docker image
2. Run emulator
3. Fetch (or clone) and install the latest stable SSO sample app, for example via GitHub, jitpack.io or F-Droid (given the sample app will be published there)
4. Build and install the current files app
5. Run actual test

SSO lib

1. Setup a temporary server using the Docker image
2. Run emulator
3. Fetch and install the latest beta files app
4. Build and install the current files app
5. Run actual test

---

I would love to get some feedback from @tobiasKaminsky @David-Development @desperateCoder - also feel free to ping everyone who might be interested, wants to join forces or can provide further information (for example regarding Docker.

[tobiasKaminsky]

Regarding CI: If you prefer Github Actions, do it, as you said probably others are more into it.

You can start right away, and just ping me anytime if you need help.
When/how often we start this can then be discussed later :)

Maybe it is best to start within SSO repo with stable/dev Files app, as this can be directly downloaded?

[stefan-niedermann]

I already started, but in the Deck app.

Maybe it is best to start within SSO repo with stable/dev Files app, as this can be directly downloaded?

Exactly what i thought - the issue is, that we would have to transform the SSO repp into a multi module project first, so the sample app can be in the same repository. That's a problem that i'd like to discuss first.

If i get it working on the Deck apo, though, we can totally easily move / copy the tests to the SSO lib later 🙂.

[tobiasKaminsky]

Exactly what i thought - the issue is, that we would have to transform the SSO repp into a multi module project first, so the sample app can be in the same repository. That's a problem that i'd like to discuss first.

Karumi/shot is doing the same:
https://github.com/pedrovgs/Shot

shot-* are sample projects, used also for testing
core is real library

Maybe not best example, as they use scala for their plugin 🙈

[stefan-niedermann]

Yep, exactly what i had in mind. We are using multi module projects for Deck, News, Notes (and the nextcloud-commons lib) as well, but to transform the SSO lib, we will need some help. There are many config files for translations, tests, reports and stuff which need to work once we are done. Also ideally we need to keep the coordinates the same, so 3rd party apps do not have to change their dependencies.

[tobiasKaminsky]

I suggest you start with a PR and I can jump in when you need help :)