Per-VM encryption

[qubesuser]

Currently volatile.img is stored unencrypted and includes swap, which means that anyone compromising dom0 or getting past full disk encryption can potentially read things that were in application memory of any VM at any point in past time, if it happened to be swapped out and the sectors were not overwritten.

This can be avoided by mounting volatile.img using DM crypt and a randomly generated key on each bootup that is then naturally destroyed on shutdown.

Likewise, not encrypting private.img means that it is not possible to securely delete a VM in a way that is fully resistant to future compromises since the data may be preserved in unused or reallocated sectors. In this case there is the additional problem of having to encrypt the keys with a master key stored in a way that can be securely deleted/changed (options could be on paper or in the TPM).

There is a performance downside due to having to encrypt the data twice (once for the volatile.img/private.img encryption, once for full disk encryption); this could theoretically be avoided, but it seems that would require major changes.

[marmarek]

Some relevant photos:
https://twitter.com/rootkovska/status/541980196849872896
https://twitter.com/rootkovska/status/650732336330489856

Anyway this is something we consider in "Untrusted Storage Domain" design page 33.

[qubesuser]

I think just shredding like in the screenshots is potentially insecure (against someone obtaining the full disk encryption password), since the SSD or hard disk might have reallocated the blocks, and the filesystem might have done that as well (e.g. if using btrfs).

With XTS blocks are encrypted depending on the disk position, but one could in principle try all encryption positions on all raw disk blocks not part of files and then use entropy, compressibility, presence of long ASCII strings, etc. to guess at the correct plaintext.

Anyway this is something we consider in "Untrusted Storage Domain" design page 33.

Yes, that specific design for an untrusted storage domain would solve this issue too (assuming there is a way to securely destroy the per-AppVM keys), although they actually seem orthogonal in general.

One could implement an untrusted storage domain with the same key for all AppVM, as long as the decryption is done in dom0 as opposed to directly connecting the AppVMs to the storage domain and forward secrecy is not a goal.

An untrusted storage domain threat model also has other requirements like the need to defend against active manipulation, which means that something like the LUKS XTS FDE is not enough, and you'd also need to authenticate everything with a Merkle hash tree of all the disk sectors and then authenticate the root using a secret not shared with the untrusted storage domain (otherwise the storage domain can replace sectors with older encrypted versions) and probably generate and store a new random IV for each encryption to prevent leaking data correlation over time to the untrusted storage domain (with XTS or CBC ESSIV if you write A then B then C to the same sector, then the storage domain would know whether A = C or not because the IV/tweak and thus the ciphertext is the same).

[marmarek]

One could implement an untrusted storage domain with the same key for all AppVM, as long as the decryption is done in dom0 as opposed to directly connecting the AppVMs to the storage domain and forward secrecy is not a goal.

The point about Storage Domain is that dom0 would not touch VM data at all.

Regarding data authentication - yes, it is covered in that design. We haven't decided about precise implementation used for that - one candidate is dm-verity, but maybe there are alternatives.

Some subset of above - anti-forensics data handling - is considered for DispVMs - #904 . There is a chance that we'll implement it for R3.1.

[qubesuser]

The problem of dm-verity is that it seems to currently have no write support, which means it would obviously not work for defending mutable AppVM storage against a malicious storage domain.

Not sure if there is any alternative, it's possible that there is no easier way to do this than doing custom development on dm-verity (which is not so trivial because it needs to be crash-proof).

BTW, directly attaching AppVMs to an untrusted storage domain can be problematic, because a malicious or misconfigured AppVM could leak all its data by simply not encrypting anything (and viceversa the untrusted storage domain could send arbitrary plaintext), creating a very high-bandwidth covert channel that can be accessed even by someone that can only read/write a snapshot of the storage.

[marmarek]

On Thu, Oct 08, 2015 at 01:11:27PM -0700, qubesuser wrote:

BTW, directly attaching AppVMs to an untrusted storage domain can be problematic, because a malicious or misconfigured AppVM could leak all its data by simply not encrypting anything (and viceversa the untrusted storage domain could send arbitrary plaintext), creating a very high-bandwidth covert channel that can be accessed even by someone that can only read/write a snapshot of the storage.

When the AppVM is compromised, it means that the data there is no
longer safe. The whole point about isolation is that only one
particular VM would be affected. In case of cooperating compromised
domains, x86 architecture provide sufficiently enough covert channels.
Yes, some of them have lower bandwidth, some higher, but still -
currently it's hard to solve, at least on on x86. I guess, in most
cases, the most precious data is quite small anyway (keys, etc).

Regarding storage domain sending arbitrary plaintext - it shouldn't be a
problem as long as AppVM properly verify the data. And when not, it's
already game over for that VM.

And generally the whole idea about Untrusted Storage Domain is to take
some more pieces out of TCB. Currently we need to trust disk controller
firmware (because it can effectively own dom0 using DMA) and assume
block backends are bug-free. We want to take those parts out of TCB.
Yes, some details needs to be sorted out, but in current design shape,
it will not be worse than the current approach (where the physical
storage handling is basically ultimately trusted).

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

I'm closing this one as a duplicate of #904.
For "Untrusted Storage Domain" there will be a separate ticket.

[andrewdavidwong]

@marmarek: I can't find an existing issue that tracks per-VM encryption. This is the closest one. Should we re-open this issue to continue to track that, or open a new one?

[DemiMarie]

What could I do to implement this? For swap devices, LUKS should be up to the task.

[DemiMarie]

Libvirt provides an API for storage encryption using LUKS. I could try to make a PR, but I would not be able to test it locally.

[tasket]

How much overlap is there between this request and current storage pool feature?

IMO, they seem mostly the same.

[marmarek]

It's definitely possible to implement this as a qubes storage pool driver.

As for libvirt API for that, we've looked at it before and actually considered to use that, instead of our own storage pools implementation. But in the end rejected because of various limitations. For example it doesn't support LVM thin provisioning (only non-thin LVM), or btrfs reflink copy (copy-on-write). Those (and other) limitations are the reason why we have our own storage pool drivers.

And it looks like libvirt storage encryption API is can be used only for volumes managed by libvirt storage pools, so it's useless for us unfortunately. It's a pity, because that encryption API already have for example ephemeral secrets - perfect for volatile volumes.

[DemiMarie]

Could we instead improve the libvirt implementation? We have a bunch of patches to libvirt anyway.

…

On Sat, Jan 26, 2019, 6:44 PM Marek Marczykowski-Górecki < ***@***.*** wrote: It's definitely possible to implement this as a qubes storage pool driver. As for libvirt API for that, we've looked at it before and actually considered to use that, *instead of* our own storage pools implementation. But in the end rejected because of various limitations. For example it doesn't support LVM thin provisioning (only non-thin LVM), or btrfs reflink copy (copy-on-write). Those (and other) limitations are the reason why we have our own storage pool drivers. And it looks like libvirt storage encryption API is can be used only for volumes managed by libvirt storage pools, so it's useless for us unfortunately. It's a pity, because that encryption API already have for example ephemeral secrets - perfect for volatile volumes. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1293 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGGWB9K_gCzQT__E29ZiEFmETQRCMsC9ks5vHOhxgaJpZM4GK2ny> .

[cm157]

@cm157 Volatile volume encryption is already on my task list. The volume will be encrypted in dom0 using AES-XTS and a randomly generated key. dom0 swap will also be encrypted.

This is fantastic news. I will send you an email to figure out how to handle the bounty.

[ArtificialAmateur]

• Are all LVs of a VM encrypted or just private / volatile?

I would think private / volatile for AppVMs, maybe more for StandaloneVMs.

• Do we use a separate encryption key for each LV or do we use 1 key for all VM related LVs?

Separate key for each LV at first seems better, but what situtation would it provide more security if someone attained one key?

• Where are the master keys to each LV stored and how are they secured in particular?

This is also an implementation question im wanting to figure out. @DemiMarie @marmarek , ideas?

• Do we encrypt all VMs by default or only VMs which were created with some --encryptvm param?

I think it might be best to encrypt by default (StandaloneVM and AppVM) using keyfiles and then have an option (--encryptvm and gui checkbox) to provide a custom passphrase. I think DisposableVMs should follow #904.

It's definitely possible to implement this as a qubes storage pool driver.

I think this is the way to go using dm-crypt (cryptsetup) after volume creation.

[DemiMarie]

@cm157 Volatile volume encryption is already on my task list. The volume will be encrypted in dom0 using AES-XTS and a randomly generated key. dom0 swap will also be encrypted.

This is fantastic news. I will send you an email to figure out how to handle the bounty.

QubesOS/qubes-core-admin#396 will implement volatile volume encryption. It will not implement per-VM encryption, which will need to be handled separately.

Note that data written to the root volume in AppVMs and DispVMs will not be protected by this. I filed #6625 to track making root volumes default to read-only in AppVMs and DispVMs, which would cause data written to the root filesystem to land on the encrypted volatile volume. DispVM private volumes will still need to be handled separately.

[cm157]

This bounty is now at $3500. As mentioned, i open to idea of making this payment upfront or to mutually agreed intermediary in advance of completed work.

[cm157]

If a VM is not active its data should be in a secure state while at rest. Only when a VM is running should its data be in a unencrypted state. Its naive to assume you will have opportunity to completely shutdown your system prior to an attack.

Also, another plus as per recommendations from NSA the benefit of cascading two different encryption implementations should one be compromised... ie eggs one basket.

[ghost]

I'm working on it right now @cm157 proper UI integration and Admin API Integration in Python (the language Admin API is written in with @ function handlers), prompts for passphrase to dom0 via RPC. Can be done just technically or nice with Qt UI (Qubes Settings, Qubes Manager) GUI Checkboxes as well, so it takes a few hours longer. Would be glad to talk about your Yubikey / Nitrokey ideas to make it even better.

[ghost]

Since per VM encryption feature is being discussed, and even funding is available, which has the proper desires and input (support for 2FA keys, disk encryption per Qube, open to RAMDisk, probably not allergic to ISR), it would be beneficial to not just implement a proof of concept, which can be done, but neat QubesOS integration with UX in mind, even if it means a lot of Qt / PyQt, which I started to get comfortable with for some hours. https://github.com/fairlightcx/qubes-manager/tree/encryptdisk

Can this be moved from a feature that backups (and restore) somewhat templates on integration, and LUKS self-explains on storage, to an actual small project with partial up front funding? @cm157 @marmarek Semi-offtopic, I consider qubes-builder highly important to prioritize and currently building vvm_xen is really slow, the structure of qubes- and Marek's privilege as Chief of Git, deserve some rework too.

[ghost]

And, I would like to add VPN (or Tor) per Qube, if not per Application per Qube, to this ticket, after disk encryption is finely done, and Ramdisk maybe ISR, was at least shortly discussed.

[andrewdavidwong]

And, I would like to add VPN (or Tor) per Qube, if not per Application per Qube, to this ticket, after disk encryption is finely done, and Ramdisk maybe ISR, was at least shortly discussed.

I'm confused. Doesn't that already exist, and how is it relevant to this issue?

[cm157]

Hey Ghost, are you officially taking up the bounty? If so, would you like me to deposit the funds into escrow and if yes can you please nominate someone from the QubesOS project?

I'm working on it right now @cm157 proper UI integration and Admin API Integration in Python (the language Admin API is written in with @ function handlers), prompts for passphrase to dom0 via RPC. Can be done just technically or nice with Qt UI (Qubes Settings, Qubes Manager) GUI Checkboxes as well, so it takes a few hours longer. Would be glad to talk about your Yubikey / Nitrokey ideas to make it even better.

Yes, i would love to discuss that i will make a separate post on it tomorrow. Please note, 2FA is not a requirement for the bounty. Its just an idea. The only deliverable is per vm encryption.

[cm157]

And, I would like to add VPN (or Tor) per Qube, if not per Application per Qube, to this ticket, after disk encryption is finely done, and Ramdisk maybe ISR, was at least shortly discussed.

I'm confused. Doesn't that already exist, and how is it relevant to this issue?

I am also confused about that. Do you mean you want to design a simpler native QubesOS process for users to setup and assign a vpn config to a VM?

And what has that got to do with the $3500 bounty? Its out of scope.

[ninavizz]

I mean, this is a lot more than $3500 of work.

[cm157]

I mean, this is a lot more than $3500 of work.

Most of what is being discussed has nothing to do with the bounty i offered. I have no idea what is going on.

The bounty is simple per vm encryption feature consistent with QubesOS project best practices and design principles. I have no other prerequisite conditions.

Please keep it simple folks some of these posts are not making any sense.

[ninavizz]

@cm157 I'm just a UX practicioner... but, my understanding of a "bounty," is that it's an incentive for community folks to find bugs that could cause harm to users. Developing a feature is a non-trivial and costly process. As it is, Qubes OS is a very under-funded project. My contributions to this conversation, and about half the time with my Qubes stuff, is all volunteer, because of this.

I'm certain that Marek and other leadership with Invisible Things Labs would be happy to consider prioritizing specific features if those requesting them paid for their development... but "a small feature" typically costs no less than a $20k to design, develop, and QA. Which I don't share as a finger-wag to your good spirited bounty offer! Just my own thoughts, as a friend of ITL and a contributor to this and other underfunded projects.

@valereznya You're not behaving on par with our Code of Conduct. You also posted an off-topic nit on some protocol, on another Issue I opened to track a different project with. Please either adhere to our code of conduct and our contributing guidelines, or leave. Critical and sometimes intense dialog is entirely possible, without disparaging others.

[cm157]

I mean, this is a lot more than $3500 of work.

Most of what is being discussed has nothing to do with the bounty i offered. I have no idea what is going on.

The bounty is simple per vm encryption feature consistent with QubesOS project best practices and design principles.

dom0 message passing, leading to a prompt, is pretty invasive, and I would take the load of encryption and re-encryption when a VM is idle not too lighlty, especially not when more than one window is being used. I foree this will cause major load on the CPU, and the prompts can be an annoyance, the passphrase must be wiped, interrupting in re-encryption can mess up the file system. And integration with the OS. if you want that done well, as I said, I need a new laptop, send half upfront. Thx.

I am open to the idea of providing at part of the bounty upfront and direct. You would need to be a respected member of the QubesOS development community and have the endorsements of core member of QubesOS team.

As mentioned before I am also open to providing the full bounty upfront to any core member of the QubesOS team to manage.

[cm157]

@cm157 I'm just a UX practicioner... but, my understanding of a "bounty," is that it's an incentive for community folks to find bugs that could cause harm to users. Developing a feature is a non-trivial and costly process. As it is, Qubes OS is a very under-funded project. My contributions to this conversation, and about half the time with my Qubes stuff, is all volunteer, because of this.

I'm certain that Marek and other leadership with Invisible Things Labs would be happy to consider prioritizing specific features if those requesting them paid for their development... but "a small feature" typically costs no less than a $20k to design, develop, and QA. Which I don't share as a finger-wag to your good spirited bounty offer! Just my own thoughts, as a friend of ITL and a contributor to this and other underfunded projects.

The semantics of what constitutes a bounty aside i do understand and agree that the value of this bounty does not reflect the true value of work necessary to implement this feature. My only hope is that someone with the time and ability to implement this and who might see personal value in its (immense) benefits could be motivated by it.

I will contact Marek and see what his thoughts are with prioritizing efforts for per vm encryption feature would be based on a non-binding donation.

Thank you

@valereznya You're not behaving on par with our Code of Conduct. You also posted an off-topic nit on some protocol, on another Issue I opened to track a different project with. Please either adhere to our code of conduct and our contributing guidelines, or leave. Critical and sometimes intense dialog is entirely possible, without disparaging others.

[andrewdavidwong]

Hey Ghost, are you officially taking up the bounty?

Take a look at that user profile. @ghost just takes the place of deleted user accounts.

I'm certain that Marek and other leadership with Invisible Things Labs would be happy to consider prioritizing specific features if those requesting them paid for their development...

...I will contact Marek and see what his thoughts are with prioritizing efforts for per vm encryption feature would be based on a non-binding donation.

We don't try to stop the community from posting and collecting bounties, but our policy is not to implement specific features in exchange for donations:

https://www.qubes-os.org/donate/#do-donors-get-personalized-support-or-special-feature-requests