[Bug]: `hash_hkdf(): Argument #2 ($key) cannot be empty` / `HMAC does not match`

[j-ed]

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on Github (I've searched it).
• Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• Nextcloud Server is running on 64bit capable CPU, PHP and OS.
• I agree to follow Nextcloud's Code of Conduct.

Bug description

After upgrading my server from Nextcloud v24.0.4 to v24.0.5 and also upgrading to PHP8 (I don't know if this is of importance), the Nextcloud cron job generates the following error output:

ValueError: hash_hkdf(): Argument #2 ($key) cannot be empty in .../lib/private/Security/Crypto.php:97
Stack trace:
#0 .../nextcloud/lib/private/Security/Crypto.php(97): hash_hkdf('sha512', '')
#1 .../nextcloud/lib/private/Security/IdentityProof/Manager.php(110): OC\Security\Crypto->encrypt('-----BEGIN PRIV...')
#2 .../nextcloud/lib/private/Security/IdentityProof/Manager.php(133): OC\Security\IdentityProof\Manager->generateKey('user-juergen')
#3 .../nextcloud/lib/private/Security/IdentityProof/Manager.php(146): OC\Security\IdentityProof\Manager->retrieveKey('user-juergen')
#4 .../nextcloud/lib/private/Security/IdentityProof/Signer.php(64): OC\Security\IdentityProof\Manager->getKey(Object(OC\User\User))
#5 .../nextcloud/apps/lookup_server_connector/lib/BackgroundJobs/RetryJob.php(150): OC\Security\IdentityProof\Signer->sign('lookupserver', Array, Object(OC\User\User))
#6 .../nextcloud/lib/public/BackgroundJob/Job.php(79): OCA\LookupServerConnector\BackgroundJobs\RetryJob->run(Array)
#7 .../nextcloud/apps/lookup_server_connector/lib/BackgroundJobs/RetryJob.php(113): OCP\BackgroundJob\Job->execute(Object(OC\BackgroundJob\JobList), Object(OC\Log))
#8 .../nextcloud/cron.php(151): OCA\LookupServerConnector\BackgroundJobs\RetryJob->execute(Object(OC\BackgroundJob\JobList), Object(OC\Log))
#9 {main}

Steps to reproduce

I don't know how to reproduce it at the moment but it seems to occur if notification should be send-out.

Expected behavior

Execute background job without running into an error condition.

Installation method

Community Manual installation with Archive

Operating system

No response

PHP engine version

PHP 8.0

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Updated from a minor version (ex. 22.2.3 to 22.2.4)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

## Environment
#### Server Configuration
OS: Linux 5.15.64
Web server: Apache2 2.4.53
Database: MariaDB 10.3.35
PHP version: 8.0.20
Nextcloud version: 24.0.5

#### Client Configuration
Browser: Mozilla Firefox 104.0.2
Operating system: Windows 10

List of activated Apps

Enabled:
  - accessibility: 1.10.0
  - activity: 2.16.0
  - admin_audit: 1.14.0
  - announcementcenter: 6.3.1
  - apporder: 0.15.0
  - audioplayer: 3.3.0
  - bookmarks: 11.0.1
  - bruteforcesettings: 2.4.0
  - calendar: 3.5.0
  - circles: 24.0.1
  - cloud_federation_api: 1.7.0
  - comments: 1.14.0
  - contacts: 4.2.0
  - contactsinteraction: 1.5.0
  - dav: 1.22.0
  - event_update_notification: 1.5.0
  - external: 4.0.0
  - federatedfilesharing: 1.14.0
  - federation: 1.14.0
  - files: 1.19.0
  - files_accesscontrol: 1.14.1
  - files_antivirus: 3.3.1
  - files_automatedtagging: 1.14.0
  - files_downloadactivity: 1.13.0
  - files_external: 1.16.1
  - files_pdfviewer: 2.5.0
  - files_photospheres: 1.24.1
  - files_retention: 1.13.2
  - files_rightclick: 1.3.0
  - files_sharing: 1.16.2
  - files_trashbin: 1.14.0
  - files_versions: 1.17.0
  - files_videoplayer: 1.13.0
  - firstrunwizard: 2.13.0
  - groupfolders: 12.0.1
  - guests: 2.2.0
  - impersonate: 1.11.0
  - logreader: 2.9.0
  - lookup_server_connector: 1.12.0
  - mail: 1.13.8
  - maps: 0.2.1
  - metadata: 0.16.0
  - news: 18.1.1
  - nextcloud_announcements: 1.13.0
  - notes: 4.5.1
  - notifications: 2.12.1
  - notify_push: 0.4.0
  - oauth2: 1.12.0
  - password_policy: 1.14.0
  - photos: 1.6.0
  - previewgenerator: 5.0.0
  - privacy: 1.8.0
  - provisioning_api: 1.14.0
  - serverinfo: 1.14.0
  - settings: 1.6.0
  - sharebymail: 1.14.0
  - smb_test: 0.3.4
  - spreed: 14.0.4
  - suspicious_login: 4.2.0
  - systemtags: 1.14.0
  - tasks: 0.14.4
  - text: 3.5.1
  - theming: 1.15.0
  - twofactor_backupcodes: 1.13.0
  - twofactor_gateway: 0.20.0
  - twofactor_totp: 6.4.0
  - twofactor_webauthn: 0.3.1
  - unsplash: 1.2.5
  - updatenotification: 1.14.0
  - user_status: 1.4.0
  - viewer: 1.8.0
  - workflow_script: 1.9.0
  - workflowengine: 2.6.0
Disabled:
  - dashboard: 7.0.0
  - encryption
  - epubreader: 1.4.7
  - files_trackdownloads: 1.11.0
  - recommendations: 0.6.0
  - support: 1.4.0
  - survey_client: 1.1.0
  - twofactor_admin: 3.2.0
  - user_ldap
  - weather_status: 1.0.0

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

.

Additional info

.

[solracsf]

Do you have a 'secret' value on your config.php file?

[j-ed]

@solracsf Thank you, it seems that the parameter got lost somehow during one of the last couple of updates. I've now added the parameter again and will report if the error reappears again.

[CarlSchwan]

If secret is missing, you will see this error. Once you add it back, it should work again.

Please reopen this issue if it happens again after adding the secret parameter

[j-ed]

@CarlSchwan The problem seems to be more complex than I expected at the beginning. I found out that the secret parameter hasn't been set in the past at all on the server so that an empty value was used for encrypting passwords etc. The system excepted that setting and never complained about an empty value before.
If I set 'secret' => '', in the configuration the displayed exception changes as follow - what is understandable somehow.

... -","app":"cron","method":"","url":"/cron.php","message":"hash_hkdf(): Argument #2 ($key) cannot be empty",

Next I've provided a value to the secret parameter to get rid of the reported messages. As expected this had a bigger impact on the server because all application passwords and mount passwords hd to be changed. I tried to get this done and was able to successfully set new app passwords but I was unable to access my personal external storage anymore. As soon as I access the external storage configuration page I get an exception displayed on the screen with the recommendation to check the server log file.
The server log files shows a HMAC does not match. exception, independently if mounts exists or not. So by setting a value for the secret parameter external storages got unusable. Now I'm sitting between two chairs and can only tell you that the incompatibility seems to have been introduced with NC 24.0.5.

[solracsf]

Secret is used in many places.
There are PRs ongoing about this:

#31499
#31492

[CarlSchwan]

#31499 is working and you can apply the patch. The reason why it is wip is that we wanted to have a migration step in the background adding the secret if it was missing :(

I'll try to revive the branches

[j-ed]

@CarlSchwan Unfortunately I've already fixed the problem on my server. After setting the 'secret' parameter in the configuration I truncated the `oc_storages_credentials' table, which allowed me to access the external storage settings again and to enter new credentials. After that I had to set all application passwords again because the previously generated once didn't work anymore. Additionally I had to reconnect the TOTP applications of my users again. Finally I had to drop all mail tables and reinstalled the mail app to get it up-and-running again. For now it seems to work. Nevertheless the provided patch will be the right way to go, because otherwise many systems will most likely left in an insecure state.

[CarlSchwan]

I merged the related PR

[denics]

I don't know if this should be completely closed, I am on version 25.0.2.3 and I still get the above error. This happened after upgrade from 25.0.2.2 to 25.0.2.3.

{"reqId":"Y5Z8lKVLqNqTC905ESs4pgAAAFA","level":3,"time":"2022-12-12T00:57:57+00:00","remoteAddr":"192.168.1.200","user":"denix","app":"index","method":"POST","url":"/login","message":"hash_hkdf(): Argument #2 ($key) cannot be empty in file '/var/www/nextcloud/lib/private/Security/Crypto.php' line 160","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:107.0) Gecko/20100101 Firefox/107.0","version":"25.0.2.3","exception":{"Exception":"Exception","Message":"hash_hkdf(): Argument #2 ($key) cannot be empty in file '/var/www/nextcloud/lib/private/Security/Crypto.php' line 160","Code":0,"Trace":[{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":172,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":298,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":1047,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":36,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","Line":165,"Previous":{"Exception":"ValueError","Message":"hash_hkdf(): Argument #2 ($key) cannot be empty","Code":0,"Trace":[{"file":"/var/www/nextcloud/lib/private/Security/Crypto.php","line":160,"function":"hash_hkdf"},{"file":"/var/www/nextcloud/lib/private/Security/Crypto.php","line":134,"function":"decryptWithoutSecret","class":"OC\\Security\\Crypto","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/apps/files_external/lib/Service/DBConfigService.php","line":530,"function":"decrypt","class":"OC\\Security\\Crypto","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/apps/files_external/lib/Service/DBConfigService.php","line":510,"function":"decryptValue","class":"OCA\\Files_External\\Service\\DBConfigService","type":"->","args":["*** sensitive parameters replaced ***"]},{"function":"OCA\\Files_External\\Service\\{closure}","class":"OCA\\Files_External\\Service\\DBConfigService","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/apps/files_external/lib/Service/DBConfigService.php","line":508,"function":"array_map"},{"function":"createKeyValueMap","class":"OCA\\Files_External\\Service\\DBConfigService","type":"->"},{"file":"/var/www/nextcloud/apps/files_external/lib/Service/DBConfigService.php","line":486,"function":"array_map"},{"file":"/var/www/nextcloud/apps/files_external/lib/Service/DBConfigService.php","line":421,"function":"getConfigForMounts","class":"OCA\\Files_External\\Service\\DBConfigService","type":"->"},{"file":"/var/www/nextcloud/apps/files_external/lib/Service/DBConfigService.php","line":116,"function":"getMountsFromQuery","class":"OCA\\Files_External\\Service\\DBConfigService","type":"->"},{"file":"/var/www/nextcloud/apps/files_external/lib/Service/UserGlobalStoragesService.php","line":194,"function":"getMountsForUser","class":"OCA\\Files_External\\Service\\DBConfigService","type":"->"},{"file":"/var/www/nextcloud/apps/files_external/lib/Config/ConfigAdapter.php","line":120,"function":"getAllStoragesForUser","class":"OCA\\Files_External\\Service\\UserGlobalStoragesService","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Config/MountProviderCollection.php","line":124,"function":"getMountsForUser","class":"OCA\\Files_External\\Config\\ConfigAdapter","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/SetupManager.php","line":226,"function":"addMountForUser","class":"OC\\Files\\Config\\MountProviderCollection","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/SetupManager.php","line":318,"function":"OC\\Files\\{closure}","class":"OC\\Files\\SetupManager","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Files/SetupManager.php","line":227,"function":"setupForUserWith","class":"OC\\Files\\SetupManager","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Filesystem.php","line":379,"function":"setupForUser","class":"OC\\Files\\SetupManager","type":"->"},{"file":"/var/www/nextcloud/lib/private/Cache/File.php","line":57,"function":"initMountPoints","class":"OC\\Files\\Filesystem","type":"::"},{"file":"/var/www/nextcloud/lib/private/Cache/File.php","line":178,"function":"getStorage","class":"OC\\Cache\\File","type":"->"},{"file":"/var/www/nextcloud/lib/base.php","line":851,"function":"gc","class":"OC\\Cache\\File","type":"->"},{"function":"{closure}","class":"OC","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Hooks/EmitterTrait.php","line":106,"function":"call_user_func_array"},{"file":"/var/www/nextcloud/lib/private/Hooks/PublicEmitter.php","line":40,"function":"emit","class":"OC\\Hooks\\BasicEmitter","type":"->"},{"file":"/var/www/nextcloud/lib/private/User/Session.php","line":400,"function":"emit","class":"OC\\Hooks\\PublicEmitter","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/CompleteLoginCommand.php","line":44,"function":"completeLogin","class":"OC\\User\\Session","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/ALoginCommand.php","line":40,"function":"process","class":"OC\\Authentication\\Login\\CompleteLoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/LoggedInCheckCommand.php","line":60,"function":"processNextOrFinishSuccessfully","class":"OC\\Authentication\\Login\\ALoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/ALoginCommand.php","line":40,"function":"process","class":"OC\\Authentication\\Login\\LoggedInCheckCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/EmailLoginCommand.php","line":70,"function":"processNextOrFinishSuccessfully","class":"OC\\Authentication\\Login\\ALoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/ALoginCommand.php","line":40,"function":"process","class":"OC\\Authentication\\Login\\EmailLoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/UidLoginCommand.php","line":54,"function":"processNextOrFinishSuccessfully","class":"OC\\Authentication\\Login\\ALoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/ALoginCommand.php","line":40,"function":"process","class":"OC\\Authentication\\Login\\UidLoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/UserDisabledCheckCommand.php","line":58,"function":"processNextOrFinishSuccessfully","class":"OC\\Authentication\\Login\\ALoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/ALoginCommand.php","line":40,"function":"process","class":"OC\\Authentication\\Login\\UserDisabledCheckCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/PreLoginHookCommand.php","line":53,"function":"processNextOrFinishSuccessfully","class":"OC\\Authentication\\Login\\ALoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/Chain.php","line":108,"function":"process","class":"OC\\Authentication\\Login\\PreLoginHookCommand","type":"->"},{"file":"/var/www/nextcloud/core/Controller/LoginController.php","line":318,"function":"process","class":"OC\\Authentication\\Login\\Chain","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":225,"function":"tryLogin","class":"OC\\Core\\Controller\\LoginController","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":133,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":172,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":298,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":1047,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":36,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/lib/private/Security/Crypto.php","Line":160},"CustomMessage":"--"}}

Secret is there, as well a new updater.secret. I tried to remove that one, but I did not notice any change.
Thanks a lot!