shiftfs incompatible with overlayfs without specific overlayfs version

[hxx3000]

There seems to be an additional kernel patch in conjunction with shiftfs when used above overlayfs:

https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/focal/commit/fs/overlayfs?id=d24b8a547be1578cb5a200ad9ac57a258f0a9de1

So it should be noted in the documentation not to use this dkms in conjunction with overlayfs.

[toby63]

For everyone reading this issue I want to give a summary:

Note: The following information might be outdated.

Problem:
The problem is that overlayfs cannot be used inside a container, when the container software uses shiftfs.
Note: This is only a problem for special use cases, e.g. running Docker with the overlayfs-storage-driver inside an LXD container.

Solution:
A solution is available in form of kernel patches for the overlayfs module (see ubuntu kernel commit).
But these patches are not included in the mainline kernel.
To my knowledge only Ubuntu included them in their kernel.

Main (potential) Workaround:

• patch the kernel module "overlayfs" via dkms:
  (or more correct: install a modified/patched version of overlayfs via dkms)
  You can use a similar dkms package (such as this one) and try to edit it for overlayfs.
  Note: This approach is untested, so it might not work (e.g. some modules cannot be "managed" with dkms).

Other Workarounds:

• use Ubuntu:
  Regarding Derivates: it is unknown to me whether Ubuntu derivates (such as mint) include the ubuntu kernel

• convince your distro's kernel maintainers to include the patches

• (not recommended) patch the kernel sources and rebuild the kernel manually (this might take a long time to compile)

---

Original Post:
@hxx-fetch:
You are right.
Although this problem is mentioned in a post I linked to in the Readme (see "Further information on shiftfs"), I agree that I should clarify it here in the Readme too.
Quote from the LXD Forum:

Limitations

The only limitation we’re currently aware of is that shiftfs will prevent the use of overlayfs inside the container. This may break Docker users. If this applies to you, you should stay away from shiftfs for now.

I added a note in the Readme (see commit).

[toby63]

I was not aware of a patch for this though, interesting 🤔 .

@brauner: Some questions about this:

1. Just for clarity: This problem only occurs inside containers?
   So using overlayfs on the host still works (e.g. using docker on the host)?

2. Will the patch(es) be in the mainline kernel?
   If yes, what is the status of implementation?

3. I assume it is not possible or "easy" to put the patch(es) into a kernel module?

[brauner]

I was not aware of a patch for this though, interesting .

@brauner: Some questions about this:

1. Just for clarity: This problem only occurs inside containers?
   So using overlayfs on the host still works (e.g. using docker on the host)?

Yes.

2. Will the patch(es) be in the mainline kernel?
   If yes, what is the status of implementation?

We're working on an upstreamable version of shiftfs but it'll be a little while.

3. I assume it is not possible or "easy" to put the patch(es) into a kernel module?

Not impossible but unlikely to be accepted.

[toby63]

@brauner: Thank you for the information.

3. I assume it is not possible or "easy" to put the patch(es) into a kernel module?

Not impossible but unlikely to be accepted.

Sry for asking, but that means:
a) accepted by the kernel maintainers?
b) accepted by the system/kernel (for example my system)?

And it would require a rewrite I assume?

I was just thinking of implementing that into a dkms skript once again 😄 .

[brauner]

@brauner: Thank you for the information.

1. I assume it is not possible or "easy" to put the patch(es) into a kernel module?

Not impossible but unlikely to be accepted.

Sry for asking, but that means:
a) accepted by the kernel maintainers?

Yes, unlikely to be accepted upstream because we're (as in us, the kernel maintainers) how exactly we want this feature to look upstream.

b) accepted by the system/kernel (for example my system)?

No, I meant a). I don't know what your maintainers would say. :)

And it would require a rewrite I assume?

I was just thinking of implementing that into a dkms skript once again .

If you can find a way for that patch to be included in the dkms script, then sure. :)

[toby63]

@brauner:

If you can find a way for that patch to be included in the dkms script, then sure. :)

1. Can dkms alter existing files in the kernel?
   1.a) Or replace existing files or modules?
   I found inconsistent answers so far, but some suggest it is possible (but only for modules).
   A potencial way of doing so, is in comment 7.

2. Or should I find a workaround like including the patch lines in some other way (e.g. new files)?
   Imo not possible without rewrite, because programs are using overlayfs, so overlayfs has to include the patches.

3. Are these two (a b) the only necessary commits?
   Edit: I see now; so commit d24b8a547be1 is a replacement of commit 37430e430a14 ?
   And that commit/patch is the only one needed?