Repository navigation
Publish an image that is ready for scalable cloud deployments #2044
Description
Activity
The
/var/www/htmlvolume is defined in the base image and cannot be undefined. Nextcloud itself does not really work on read-only systems. The config is only one example, apps and themes are also just loaded into the source folder.
That's unfortunately very common for PHP applications but consistent with many other PHP Docker images.The
rsyncscript makes it possible to update the sources when you update the Docker image to a new tag. Otherwise you would have to rely on the internal Nextcloud updater that changes your sources regardless of the Docker image tag.Reacted by Przemysław RomanikReacted by Christian Jürges, Jac, Aaron Steed, Javier Arias, D, Tim Lindholm, Mike Cronce and 0xA1F1EWhile we're at it -- a full nextcloud k8s operator would be amazing ;).
Reacted by Przemysław Romanik, Stepan Kazakov, D and Matthias SchröderYou might be also interested in https://github.com/nextcloud/helm/
Reacted by Przemysław Romanik and suse-coderWhile we're at it -- a full nextcloud k8s operator would be amazing ;).
yeah - but only if the nextcloud docker image removes the rsync stuff and tries to embrace the proper way how a docker image should be done.
And btw. We also need to build our own images because of the rsync stuff...
Reacted by D and Javier AriasThe
/var/www/htmlvolume is defined in the base image and cannot be undefined. Nextcloud itself does not really work on read-only systems. The config is only one example, apps and themes are also just loaded into the source folder. That's unfortunately very common for PHP applications but consistent with many other PHP Docker images.The
rsyncscript makes it possible to update the sources when you update the Docker image to a new tag. Otherwise you would have to rely on the internal Nextcloud updater that changes your sources regardless of the Docker image tag.if you check how the owncloud people did the docker image, you will see that it's not so difficult to create a proper docker image without the need of rsync.
So your thesis "Nextcloud itself does not really work on read-only systems" is simply wrong.
Reacted by Javier Arias, D, xyhhx and 0xA1F1EThe Owncloud images are build completely different and are not part of the official library anymore. AFAIK they're also moving away from PHP.
With this image you can add additional files, configs, hook scripts or volumes. You can also just overwrite the entrypoint of this image to skip rsync completely. If you need to customize it even further, its totally fine to build your own image. This image might be used as a base for all the required runtime libraries.
AFAIK they're also moving away from PHP.
ownCloud itself will always be PHP. ownCloud Infinite Scale is a complete rewrite in Go.
Just want to understand this one better, as the rsync also is annoying in my setup (it takes pretty long)
The
/var/www/htmlvolume is defined in the base image and cannot be undefined.So what you are saying is, the path is defined and people us it. Changing the volume definition would be a breaking change. Is that correct?
From a technical perspective I don't see a reason why the image can't drop the volume definition, copies the Nextcloud files to
/var/www/htmlas part of the image build. As a user I still can mount my persistent folders like custom_apps, themes, config, … into the correct subfolder. The docs for the image even specify this case.volumeMounts: - name: volume-name mountPath: /var/www/html/custom_apps subPath: customapps - name: volume-name mountPath: /var/www/html/config subPath: config - name: volume-name mountPath: /var/www/html/data subPath: data
I am wondering if we could introduce an env variable to solve this.
DO_RSYNC(better name required) defaults totrue. We keep the existing use case with rsync and volume of/var/www/html. But if set to false we do not rsync and can mount fine-grained. And we drop the unnamed volume for/var/www/htmldefinition in the Dockerfile
Draw back looks to me to have the Nextcloud file twice in the image. But I would swallow that pill (have not checked the size tho)Reacted by Boyi Cour solution on building customised one volume nextcloud docker images using symlinks, can be found here:
We use several tricks to overcome the design flaws of the current nextcloud docker image.
Besides installing some more tools into the docker image, we do the following:
- create a new /custom.upgrade.exclude file containing all file and directory names sited on /usr/src/nextcloud, rm /var/www/html and move /usr/src/nextcloud to /var/www/html (where it belongs) (line 23 to 25)
- remove /usr/src/nextcloud and use /upgrade.exclude list to move back only non static files to /usr/src/nextcloud
- create a new symlink friendly /upgrade.exclude (excluding all now static nextcloud src files and folders)
- replace rsync flags -rlDog by -rlKDog to keep symlinks on rsync (line 36)
- copy custom entrypoint shell shell scripts doing the symlink creation (line 43)
- reduce the image size by starting over from scratch (line 49)
- using python code to extract env vars from original nextcloud docker image and replace $more_env with with it
The rest of the magic is done in https://gitlab.eqipe.ch/k8s/nextcloud-image/-/blob/main/eqipe/apache-symlinked/k8s_init.sh?ref_type=heads
It's using /upgrade.exclude.org to create the symlinks needed by nextcloud and restoring version.php and must be used instead of the default /entrypoint.sh
checkout docker-compose.yml to understand how to bring up this symlinked nc docker image.
I hope, sharing our knowledge, would give some ideas to @J0WI how to improve the way nextcloud docker image is done.
All the rearrange steps described above wouldn't be needed of the official docker images created in a more clever way and hopefully in the feature we get a better nc docker image some day...
Reacted by Jonas, lenaulrich, WhiteBahamut and Javier AriasI'm not sure I buy the whole "php doesn't play well with immutable file systems." How does the Nextcloud snap perform upgrades? I'm looking through their code base right now and I can't find anywhere that they copy the Nextcloud code onto a mutable file-system except in their data export script. I'm thinking there should be a way to accomplish building a Docker container that doesn't perform a copy before exporting.
If I didn't have other things to do in my spare time, I'd take a crack at building this container myself.
After a bit of toying around, I managed to get a stateless container working, including a very rudimentary upgrade process which handles database upgrades in-container: https://github.com/thefirstofthe300/docker/tree/master/28/apache-stateless
It breaks several of the assumptions made by the existing containers (namely mounts have to be placed on the config, data, custom_apps, and themes directories), but for those who want to adopt a proper immutable container paradigm, this basic container appears to work.
Reacted by Felix Prasse, Aaron Steed and Javier AriasYou may also want to reference some stuff from https://github.com/Inveniem/nextcloud-azure-aks
After a bit of toying around, I managed to get a stateless container working, including a very rudimentary upgrade process which handles database upgrades in-container: https://github.com/thefirstofthe300/docker/tree/master/28/apache-stateless
It breaks several of the assumptions made by the existing containers (namely mounts have to be placed on the config, data, custom_apps, and themes directories), but for those who want to adopt a proper immutable container paradigm, this basic container appears to work.
Any chance to get a PR into the official one?
I ended up getting around this by building a new container with the source mounted in
/var/www/htmlinstead of an incorrect locationFROM nextcloud:29.0.2-fpm RUN mv /usr/src/nextcloud/* /usr/src/nextcloud/.* /var/www/html ENTRYPOINT [] CMD ["php-fpm"]Unfortunately since /var/www/html is declared as a volume in the dockerfile, it can't be linked, and this does increase the image size but that is preferable to maintaining an entirely new image. I manage upgrades separately so I don't need to use the entrypoint script.
I have it paired with an nginx container:
FROM nextcloud:29.0.2-fpm as source FROM nginx/nginx-unprivileged:1.27-bookworm USER root RUN mkdir -p /var/www/html USER nginx COPY --from=source /usr/src/nextcloud/ /var/www/html/This is far preferable to rsyncing the application source between into a shared volume which is the only reason someone might want the entrypoint script doing that.
The
/var/www/htmlvolume is defined in the base image and cannot be undefined. Nextcloud itself does not really work on read-only systems. The config is only one example, apps and themes are also just loaded into the source folder. That's unfortunately very common for PHP applications but consistent with many other PHP Docker images.The
rsyncscript makes it possible to update the sources when you update the Docker image to a new tag. Otherwise you would have to rely on the internal Nextcloud updater that changes your sources regardless of the Docker image tag.https://github.com/docker-library/php/blob/master/8.3/bookworm/apache/Dockerfile
now The/var/www/htmlis not a volumehttps://github.com/docker-library/php/blob/master/8.3/bookworm/apache/Dockerfile now The
/var/www/htmlis not a volumehttps://github.com/nextcloud/docker/blob/master/31/fpm/Dockerfile#L151
It still gets declared as a volume in the final build.
Is a stand-alone variant wanted by the maintainers?
We are running a stand-alone version in production for some month now and would be happy to upstream it.Reacted by Javier Arias and Victor Roque@johannwagner can you please share how are you doing it?
Is this something project maintainers would be willing to accept from an external contributor? I expect it would be an additional image, so that there's no
VOLUMEdeclaration on/var/www/html(having it on specific subdirectories -/var/www/html/data,/var/www/html/config,/var/www/html/custom_apps,/var/www/html/themes- maybe others?) and so that the codebase is placed directly in/var/www/html, but it wouldn't be a huge body of work that I wouldn't mind contributing, as long as there's an appetite to accept it.Reacted by DThat turned out to be surprisingly easy...I'll submit the PR tomorrow, but in case anybody happens to come in here looking for them either before then or if the changes don't end up being merged, they're on dockerhub as
mcronce/nextcloud:30.0.17-apache,mcronce/nextcloud:31.0.11-apache, andmcronce/nextcloud:32.0.2-apache. Currently have 32.0.2 running in the official helm chart, with the/var/wwwand/var/www/htmlmounts removed. (Will submit those changes as well if the container image changes are merged.)Upgrades went from ~2 hours on my cephfs to ~5 minutes - with most of those 5 minutes spent running
ALTER TABLE ... ROW_FORMAT = DYNAMICon every table twice for each versionThemes, custom apps, config, and data do still all need to live in shared mutable storage, so not sure whether or not this will satisfy OP's original requirement for scalable cloud deployments, but if not, maybe it's at least a step toward that
@mcronce That sounds great! I had ended up solving this by creating a pair of simple custom images:
FROM nextcloud:32.0.2-fpm RUN mv /usr/src/nextcloud/.htaccess /usr/src/nextcloud/.htaccess ;\ mv /usr/src/nextcloud/.user.ini /var/www/html/.user.ini ;\ mv /usr/src/nextcloud/* /var/www/html USER www-data ENTRYPOINT [] CMD ["php-fpm"]
and
FROM library/nextcloud:32.0.2-fpm as source FROM nginxinc/nginx-unprivileged:1.29-bookworm USER root RUN mkdir -p /var/www/html USER nginx COPY --from=source /usr/src/nextcloud/ /usr/src/nextcloud/.user.ini /usr/src/nextcloud/.htaccess /var/www/html/Combined with overriding the entrypoints/command, and running the upgrade job as a one-shot as needed, I ended up with a more sane deployment, at the cost of a much larger image for nextcloud itself. So your approach would make that first dockerfile redundant. For the case of the second dockerfile, I actually think I can use this newer kubernetes feature and just mount the nextcloud image into the docker container.
Hi Folks,
thank you for providing a next cloud container image!
Unfortunately I spent a lot of time to get nextcloud running in my Kubernetes cluster. Unfortunately it's still not running and I would like to consider/discuss the following aspects.
rsync on startup?
As far as I can see, the entrypoint script
rsyncs files from the container image to a different location:docker/docker-entrypoint.sh
Line 177 in 7bd3b7b
In my eyes, a containerized application should be self-containing and source code should be ran from the image (ideally stored there in read-only mode, immutable for security reasons).
Can someone explain to me, why this image copies source files (even third party code, like the aws library) to another location before starting? For me, this seems to be an anti-pattern in the world of container images.
I read something about upgrading, but I think upgrading in the container world means - not to compare any version.php files residing in a persistent file system - but to change the tag of the image. The application itself (again running from a readonly part of the filesystem) should detect its current version and compare this to the database. Then it has to handle the upgrade process (if required) with lockfiles, leases, migrations, etc. Also from a performance perspective I disagree with the approach to rsync application files before application startup. There could be slower persistent media like NFS and startup takes a long time, then you get into trouble with the readinessProbe (i know, I can set a startupProbe - but it feels dirty)...
instanceid?
The running application writes an
instanceidto the configuration which has to be persisted, otherwise the application will try to re-install after pod restarts.Can someone explain to me, why this
instanceidis needed and why it has to be written into the configuration? I would expect it in the database, but not in the file system. The containerized application should not get any state (we have a database for that) and if it has to, it should be stored in thedatadirectory. Otherwise scaling the application is a pain as well.I would be really happy if someone could shed some light at this
rsyncidea. Until then, I'll have to build my own image.Thank you very much :)
BR Matthias
Related issues:
#1006
#1582
#1050
nextcloud/helm#590