Notes are not loading (user has ~5k notes). HTTP 504 Gateway Timeout.

[petrstepanov]

Server times out (HTTP 504) when loading notes in the web UI. Tested with over 5000 files.

I migrated to Nextcloud Notes from SimpleNote for the sake of better security, encryption. I've been using SimpleNote for a while and have over 5000 note text files. When loading notes in Nextcloud web UI, Notes app takes 30 sec to 1 minute. The request will eventually time out.

Steps to reproduce

1. Clean install the Nextcloud. Tested v.28.0.2.
2. SSH as root to the Nextcloud server and navigate to the user's files directory.
3. In Terminal create a test folder with 10000 random notes:

wget https://gist.githubusercontent.com/petrstepanov/fd60e1eec0b56ca18b6a0999fbf24a1c/raw/generate-notes.sh
chmod +x ./generate-notes.sh
./generate-notes.sh

4. In Terminal, navigate to the Nextcloud installation folder and update file cache:

cd /var/www/html
sudo -u www-data php occ files:scan --all

5. Set Notes app to use the "NotesTest" folder. In the Nextcloud web UI open "Notes" tab, go to "Notes settings". Under the "NotesPath" select the "NotesTest" folder.
6. Reload the web page.
7. Notes app takes forever to load. App shows error "Fetching notes has failed. See JavaScript console and server log for details."

Expected behaviour

Notes should load in the web interface within a reasonable timeframe.

Actual behaviour

Notes are not loading. Error message is displayed.

Possible solution

Implement lazy loading?

Screenshots

Server

Please complete the following information.

• Notes app version: 4.9.2
• Nextcloud version: 28.0.2
• OS: Ubuntu 22.05 Kernel 5.15.0-92-generic
• Web server: nginx/1.24.0
• PHP version: 8.1.27
• Database: mariadb

Nextcloud configuration:

Details

{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "149.248.33.245",
            "cloud.petrstepanov.com"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "28.0.2.5",
        "overwrite.cli.url": "http:\/\/localhost",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "memcache.local": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        }
    }
}

Client

Please complete the following information.

• Browser (incl. version): Mozilla Firefox 121.0.1
• OS: Fedora 39

Log files

Add relevant parts of your nextcloud.log and/or your browser's JavaScript console here.

[joshtrichards]

Hi @petrstepanov - Can you confirm your php-fpm configuration is reasonable?

https://docs.nextcloud.com/server/latest/admin_manual/installation/server_tuning.html#tune-php-fpm

The default php-fpm config in most distributions is ultra low (e.g. 5 for pm.max_children usually).

Unless your server is very underpowered and/or showing extremely high loads, FPM is likely barely using your server's capacity.

That is likely the cause of the 504 gateway timeouts.