Skip to content

[Bug]: php-fpm eats all cpus executing select from oc_schedulingobjects #39758

Closed

Description

⚠️ This issue respects the following points: ⚠️

Bug description

pgtop shows

  420410 nextcloud           local  nextcloud         9  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  414268 nextcloud           local  nextcloud         3  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  437007 nextcloud           local  nextcloud         3  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  447077 nextcloud           local  nextcloud         3  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  437316 nextcloud           local  nextcloud         3  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  432905 nextcloud           local  nextcloud         3  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  429016 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  411164 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  424782 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  432759 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  415449 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  428065 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  440222 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  440182 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  423448 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  434707 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  457716 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  457776 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  460434 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  446917 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  464630 nextcloud           local  nextcloud         2  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  450377 nextcloud           local  nextcloud         1  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  405526 nextcloud           local  nextcloud         1  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  449394 nextcloud           local  nextcloud         1  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  411148 nextcloud           local  nextcloud         1  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  426976 nextcloud           local  nextcloud         1  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  435448 nextcloud           local  nextcloud         1  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  456022 nextcloud           local  nextcloud         1  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  458547 nextcloud           local  nextcloud         1  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  443428 nextcloud           local  nextcloud         1  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  465470 nextcloud           local  nextcloud         1  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  460783 nextcloud           local  nextcloud         1  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  465909 nextcloud           local  nextcloud         1  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  457755 nextcloud           local  nextcloud         1  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        
  465608 nextcloud           local  nextcloud         1  Query SELECT "uri", "calendardata", "lastmodified", "etag", "size" FROM "oc_schedulingobjects" WHERE "principaluri" = $1        

apachetop shows ~7-8 req/sec most of them are not about calendar

Steps to reproduce

On my installation with ~500 users and 20-30 shared calendars (some of them shared for all) you need just wait for some time. For me it does not look like nextcloud is highly loaded, it looks like some infinite loops execute sql.

oc_schedulingobjects contains more than 1 000 000 records.

Expected behavior

Do not eat all cpu cores on less than 10 http requests per second

Installation method

Other Community project

Nextcloud Server version

27

Operating system

Other

PHP engine version

PHP 8.1

Web server

Apache (supported)

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 22 to 23)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other

Configuration report

{
    "system": {
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "27.0.1.2",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "loglevel": 2,
        "log_type": "file",
        "logfile": "\/var\/log\/nextcloud\/nextcloud.log",
        "log_query": false,
        "log_rotate_size": 268435456,
        "lastupdatedat": "1328756567.0597",
        "forcessl": true,
        "enablebackup": false,
        "maxZipInputSize": "20971520",
        "allowZipDownload": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "ldapIgnoreNamingRules": false,
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "defaultapp": "dashboard",
        "maintenance": false,
        "theme": "",
        "overwrite.cli.url": "https:\/\/cloud.dsr-corporation.com",
        "overwritehost": "cloud.dsr-corporation.com",
        "overwritewebroot": "\/",
        "trusted_domains": [
            "cloud.dsr-corporation.com"
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "appstoreenabled": false,
        "secret": "***REMOVED SENSITIVE VALUE***",
        "debug": false,
        "filelocking.enabled": true,
        "trashbin_retention_obligation": "disabled",
        "versions_retention_obligation": "disabled",
        "htaccess.RewriteBase": "\/",
        "ldapProviderFactory": "\\OCA\\User_LDAP\\LDAPProviderFactory",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "sendmail",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauthtype": "LOGIN",
        "simpleSignUpLink.shown": false,
        "onlyoffice": {
            "verify_peer_off": true
        },
        "app_install_overwrite": [
            "files_markdown",
            "onlyoffice",
            "tasks",
            "rainloop",
            "social",
            "keeweb",
            "impersonate",
            "talk_matterbridge",
            "twofactor_totp",
            "polls",
            "calendar_resource_management",
            "end_to_end_encryption",
            "forms",
            "previewgenerator"
        ],
        "mysql.utf8mb4": true,
        "mail_sendmailmode": "pipe",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "default_phone_region": "US"
    }
}

List of activated Apps

Enabled:
  - activity: 2.19.0
  - admin_audit: 1.17.0
  - bookmarks: 13.0.1
  - calendar: 4.4.4
  - calendar_resource_management: 0.5.0
  - circles: 27.0.1
  - cloud_federation_api: 1.10.0
  - comments: 1.17.0
  - contacts: 5.3.2
  - contactsinteraction: 1.8.0
  - dashboard: 7.7.0
  - dav: 1.27.0
  - deck: 1.10.0
  - end_to_end_encryption: 1.13.1
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_external: 1.19.0
  - files_markdown: 2.3.6
  - files_pdfviewer: 2.8.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - forms: 3.3.0
  - impersonate: 1.14.0
  - keeweb: 0.6.13
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - notes: 4.8.1
  - notifications: 2.15.0
  - notify_push: 0.6.3
  - oauth2: 1.15.1
  - onlyoffice: 8.1.0
  - photos: 2.3.0
  - polls: 3.4.1
  - previewgenerator: 5.1.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - related_resources: 1.2.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - spreed: 17.0.3
  - suspicious_login: 5.0.0
  - systemtags: 1.17.0
  - talk_matterbridge: 1.26.0
  - tasks: 0.15.0
  - text: 3.8.0
  - theming: 2.2.0
  - twofactor_backupcodes: 1.16.0
  - twofactor_totp: 9.0.0
  - user_ldap: 1.17.0
  - user_status: 1.7.0
  - viewer: 2.1.0
  - weather_status: 1.7.0
  - workflowengine: 2.9.0
Disabled:
  - bruteforcesettings: 2.7.0
  - encryption: 2.15.0 (installed 2.12.0)
  - firstrunwizard: 2.16.0 (installed 1.1)
  - mail: 3.2.5 (installed 3.2.1)
  - nextcloud_announcements: 1.16.0 (installed 1.1)
  - password_policy: 1.17.0 (installed 1.2.2)
  - recommendations: 1.6.0 (installed 0.5.0)
  - serverinfo: 1.17.0 (installed 1.7.0)
  - support: 1.10.0 (installed 1.0.1)
  - survey_client: 1.15.0 (installed 1.0.0)
  - updatenotification: 1.17.0 (installed 0.1.0)

Nextcloud Signing status

Technical information
=====================
The following list covers which files have failed the integrity check. Please read
the previous linked documentation to learn more about the errors and how to fix
them.

Results
=======
- core
	- INVALID_HASH
		- .htaccess
		- 3rdparty/sabre/vobject/lib/Reader.php

Reader.php contains one line change:

-        $result = $parser->parse($data, $options);
+        $result = $parser->parse($data, Reader::OPTION_IGNORE_INVALID_LINES | Reader::OPTION_FORGIVING);

Nextcloud Logs

It is over 200M, but I can found only

> More than 1000 expressions in a list are not allowed on Oracle.

messages that can be somehow related

Additional info

Nextcloud intallation from ArchLinux repos

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    0. Needs triagePending check for reproducibility or if it fits our roadmap27-feedbackbugfeature: caldavRelated to CalDAV internalsneeds infostaleTicket or PR with no recent activity

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions