Encryption using occ takes too much time while consuming too much resources.

[SergioBertolinSG]

Steps to reproduce

Having a massive installation with 1000000 ldap users:

1. Enable encryption app.
2. Enable encryption.
3. Enable LDAP and set up the 1 million users one.
4. Run occ encryption:encrypt-all.

Expected behaviour

Not sure, of course there are limitations. But i'd expect not to have cpu full busy and being able to encrypt many users after a day.

Actual behaviour

While having very few files per user, just the creation of key pairs takes many days.

Also it puts cpu to 100% until it finishes.

Server configuration

Operating system:
Ubuntu 14.04

Web server:
Apache

Database:
MySQL

PHP version:
5.5.9

ownCloud version: (see ownCloud admin page)
{"installed":true,"maintenance":false,"version":"8.2.0.6","versionstring":"8.2 beta1","edition":"Enterprise"}

Updated from an older ownCloud or fresh install:
Fresh

List of activated apps:
Default in enterprise edition.
Encryption.

The content of config/config.php:

Are you using external storage, if yes which one: local/smb/sftp/...
No

Are you using encryption:
Yes

[SergioBertolinSG]

Collecting @schiesbn comments regarding this:

Generally speaking: It is expensive to encrypt all files for all users. That's why in general "encryption yes/no" should always be a decision by the admin at the beginning. This scripts are only there if someone really, really wants to switch a existing installation and can't live with the default behavior (files get encrypted the next time you touch it). It is nothing which we want the admins to turn it on and off all the time. I don't see how this can be speed up substantially. We need to read every file, encrypt it and write it again. Also 1 million sounds quite academic to me. I think even the largest installations are far away from 1 million users.

But let's look at the numbers and see if we can improve at least some parts. (But I fear that 1 million users, all have a few MB or even GB of data will always take looooong)

and

As said, I fear that we can't do much about it.

To illustrate it I wrote this small test script:

$res = openssl_pkey_new();
openssl_pkey_export($res, $privateKey);
$keyDetails = openssl_pkey_get_details($res);
$publicKey = $keyDetails['key'];
openssl_encrypt($privateKey, 'AES-256-CFB', 'password', false, '>+ln5iNDghie+imWy');

It just perform the basic openSSL operations to create the key pair and to encrypt >it.

This script takes something between 0.055 ans 0.24 seconds if I run it multiple times. For now let's assume that it takes 0.1 seconds in average for every keypair (which is probably to optimistic, but anyway).

For 1 million users this means 1666 minutes ~ 27h (worst case (0.24s/key): 4000 >minutes ~ 66h)
For 50.000 users this means 83 minutes (worst case (0.24s/key): 200 minutes)

There is nothing we can do. That's just openssl. In our case we need to write the keys to the hard disc and update the file cache (maybe something we could defer). But we would always have 50 minutes + writing 100.000 files to the disc (private/public key for 50.000 users).

[SergioBertolinSG]

@DeepDiver1975 @cmonteroluque thoughts about this?

I think the lack of performance is important. But no idea if this can be improved for 8.2 or should be moved to 9.0 .

[schiessle]

Just to be clear about this: This is a huge process to go from no encryption to encryption (and the other way) with all files encrypted at once. The number above are just the key creation. But think about the case where you have really 1.000.000 active users how many TB of data you will have. Encrypting all this files will take even much longer. That's why we removed the whole "encrypt all" / "decrypt all" logic for 8.1. It is and always will be the recommend way to decide about encryption right at the beginning, just like you decide which database you want to use, which primary storage, etc. This script are there for people who (think that they) really need to go from one state to another. But in this case they need to know what it means. That's probably OK for a system which is just a few days in production with not that much files and active users. But if you want to do it for a huge system then you need to be prepared to wait a long time. Still it is good to have a way to get all your encrypted data out, even if it would take a week on a separate system.

[schiessle]

@SergioBertolinSG I think you meant @cmonteroluque 😉

[SergioBertolinSG]

Oops yes. Thanks. Edited.

I understand the point. Encryption should be enabled at the beginning in general. But If we want this tool to be usable it should be as quicker as possible. If it can't be quicker then it only should be used in very very limited situations, and it should be warned to the users in the documentation.

[oparoz]

What about offering the option to queue the process and using something like nice. The admin would be able to get the status of the migration and would be notified once it's over.

I think the use case (rare occasions for encrypt all / decrypt all and in many cases at the earliest stages of deployment) make sense. However, it's also true that the current process is extremely slow. Can the system be running for days (perhaps weeks) simultaneously while the encryption of all the data is happening? What happens to the user experience during the process? I think we could justify having a system down for a day, maybe even a weekend (again because of the likely rarity of the event) but beyond that, the system has to be usable even if this effort is planned

[PVince81]

@schiesbn doesn't the new encryption code supports working with non-encrypted files ? It should then already be possible to use the system while other files are being encrypted.

In the event where a user uploads a file and overwrites a non-encrypted file, that file will automatically become encrypted. So we need to make sure that the encryption command skips such files if discovered afterwards. (no direct failure)

Would be good to test this.

[PVince81]

@SergioBertolinSG is this still an issue ?

[SergioBertolinSG]

@PVince81 yes. Lots of resources used and a huge lack of performance.