Added experimental helm/minikube/kubernetes setup

[Pchelolo]

This is just a proof of concept and it's very limited - a lot of copy-pasta, only mediawiki and mysql are deployed, no Nginx no nothing, tested only on OSX, doesn't properly create all log folders, doesn't set proper permissions, the default DB is hard-coded, doesn't play with host machine hosts files etc.

Creating a PR mostly to get some comments on the general approach, cause some of the decisions are controversial, for example:

• Mounting LocalSettings.php into /etc/mediawiki via ConfigMap
• Mounting installation scripts into /bin/mediawiki via ConfigMap
• Using container postStart hook for installing composer dependencies, waiting for DB (that part should be done via Kubernetes), creating the database
• There must be a much better way to reference the config files then prepending all of them with helm template definition
• And much more...

Instructions:

1. Install minikube, kubectl, helm, start minikube with --memory=4096, initialize helm. I won't write up instructions for that cause it's OS-specific and is much better described in the tools doc
2. Do all the preparation steps from the main README - clone mediawiki and skins, set DOCKER_MW_PATH env variable, set the path in LocalSettings in the mediawiki repo to /etc/mediawiki/LocalSettings.php
3. Run ./kubernetes/init.sh - this will create a mount from your local machine to minikube VM.
4. Run helm install ./kubernetes - this will render all the configs and upload them to minikube
5. Wait, periodically checking kubectl get pods - when both pods get to the Running state - all is done.
6. Run minikube service mediawiki-svc - this will open a browser window and hopefully will show a main page of MediaWiki.

I'm not intending for this PR to be merged as is, creating it more for discussion purposes.

cc @d00rman @lavagetto

[addshore]

@Pchelolo this is perfect timing as I have just got minikube setup locally :)
I'll be sure to give it a test over the coming days.

[addshore]

Install minikube, kubectl, helm, start minikube with --memory=4096, initialize helm. I won't write up instructions for that cause it's OS-specific and is much better described in the tools doc

It would be worth stating that if you have had minikube running before you'll need to do minikube delete first or your VM won't end up with the correct memory allocation!
start with --memory when a VM already exists, even when stopped won't result in the memory allocation changing.

Do all the preparation steps from the main README - clone mediawiki and skins, set DOCKER_MW_PATH env variable, set the path in LocalSettings in the mediawiki repo to /etc/mediawiki/LocalSettings.php
Run ./kubernetes/init.sh - this will create a mount from your local machine to minikube VM.

So, on windows at least setting this env var just seems like a bit of a hassle, setting it in the shell that I run the init script from doesn't seem to work I have to go and set it globally (using the UI) for windows :(

Is there any way to make the script pick this up from a .env file similar to what was done before?
I know this isn't really a standard thing for kubernetes, but it increases the usability cross platform quite a bit.

Anyway, this is the step that I am stuck at for now as I can't seem to make the mount command work :/

[Pchelolo]

@Pchelolo this is perfect timing as I have just got minikube setup locally :)
I'll be sure to give it a test over the coming days.

Haha awesome. I thought I'd better make it before Christmas to have something to play with over the holidays too.

Is there any way to make the script pick this up from a .env file similar to what was done before?
I know this isn't really a standard thing for kubernetes, but it increases the usability cross platform quite a bit.

The init script is just a single mount command, nothing fancy :) You can run the command and substitute the env var with your actual path in order to test this. I've only tested on OSX and didn't even have Windows in mind, so with Windows-specific minikube issues I can't help you unfortunately.

[addshore]

So I never managed to make the init script work.
But I don't think that is anything to do with the script or this PR but more kubernetes itself.

In Fact I struggled to get anything to mount at all with minikube either complaining that it couldn't find or didn't have access to files.
While poking around I also filed kubernetes/minikube#2357 which doesn't appear to be working. Probably not a direct cause of my issues here but more than likely related.

I'll probably take another stab at getting this to work for my setup over the coming days.

[d00rman]

Nice work @Pchelolo ! I wonder, though, since we are trying to converge the two efforts (this one and mediawiki-containers), whether work should be done on the mediawiki-containers side to do so rather than here. Ultimately, the idea is to have a full development environment for mediawiki and the accompanying services.

cc @Krinkle @akosiaris

[Pchelolo]

Oh... I've spent quite a while trying to make a windows setup only to discover that Hyper-V on windows is not supported inside a VirtualBox VM.. Seems like I won't be able to test this on windows.

[addshore]

I wonder, though, since we are trying to converge the two efforts (this one and mediawiki-containers), whether work should be done on the mediawiki-containers side to do so rather than here.

It would probably make sense to try to move the mediawiki-containers images to the production repo for images the wmf has? And get them into a productionish ready state?

Once there they could easily be used by this setup.

Right now on the mediawiki-containers side of things, I'm not a fan of the bundling of services in wikimedia/mediawiki-node-services

[d00rman]

It would probably make sense to try to move the mediawiki-containers images to the production repo for images the wmf has? And get them into a productionish ready state?

There is no such thing as the wmf mediawiki [docker] production image, so I'm not sure I understand the suggestion. I do think that our (wider) collaboration should bring us into a place where the wmf production image becomes a thing :) I merely wanted to emphasise that we should work together in one place, I wasn't implying there is already a solution to this out there ;)

Right now on the mediawiki-containers side of things, I'm not a fan of the bundling of services in wikimedia/mediawiki-node-services

Only two services are there - RESTBase and Parsoid. But, I agree that for the development use case, they should be separated. I created a task for it and it should be tackled soon. For context, these were bundled together to support the low-resource use cases, such as third-party installs on small VPS.

[addshore]

There is no such thing as the wmf mediawiki [docker] production image

But I have no doubt that there will be. I more meant for the node services.

There is a wmf repo for production ready images @ https://github.com/wikimedia/operations-docker-images-production-images (actually on gerrit).

Also wmf base images that we probably want to base things on https://github.com/wikimedia/operations-docker-images-production-images/blob/5cc5c52b654d02fd434dc3cdad92c74210598983/config.yaml#L2

[d00rman]

But I have no doubt that there will be. I more meant for the node services.

There is a wmf repo for production ready images @ https://github.com/wikimedia/operations-docker-images-production-images (actually on gerrit).

You don't need to worry about this, images in mediawiki-containers will be based on these, but we are not discussing node images in this PR, are we?

[addshore]

You don't need to worry about this, images in mediawiki-containers will be based on these, but we are not discussing node images in this PR, are we?

I think we went off topic a while ago in regard to this PR.

I'd like to see the node images as part of the setup, and I don't see much point in duplicating effort, for example, making the image in mediawiki-containers, when we already know we will need a production ready image to land in the ops repo at some point. :)

I'll have another go at kicking minikube to mount things correctly over the next days for this PR.

[d00rman]

I'd like to see the node images as part of the setup,

The mediawiki-containers effort was started long before the images themselves. I should also note that these images are not used anywhere yet, so we are still in an early stage. Rome wan't built in a day ;)

and I don't see much point in duplicating effort,

Hence my proposal to have a unified place where we are working on stuff, instead of two which might potentially diverge, which is not desirable for anybody.

for example, making the image in mediawiki-containers, when we already know we will need a production ready image to land in the ops repo at some point. :)

Production images for Mediawiki will not be available, tested nor used for quite a while still, so having an intermediate step is a requirement here. Besides, if we do it right, the effort won't be really duplicated since ideally we would swap out one image for another and tweak things accordingly. Again. this conversation is not about what to put, but where to put it.

[addshore]

Hence my proposal to have a unified place where we are working on stuff, instead of two which might potentially diverge, which is not desirable for anybody.

Well, I'm going to have to continue working with this system & setup for now as minikube doesn't work correctly for me.

Also both this and mediawiki-containers are starting as totally separate / diverged projects, it would be hard to diverge further really.

It probably make sense to keep working on images @ https://github.com/wikimedia/mediawiki-docker
I should file some tickets about creating tags of the mediawiki image with different versions of PHP and also with dev versions. Once that happens, this dev environment can switch to use the mediawiki images instead of the webdevops images.

The images for services that can surround mediawiki extensions, such as parsoid, restbase, ores etc could probably also live there? Although it might make more sense for them to be published under the wikimedia org on docker hub instead of the library, but they could all live in the same repo

We could restructure the mediawiki-docker repo to contain a library dir and a wikimedia dir.
This such as https://github.com/wikimedia/mediawiki-node-services could then be bundeled into the repo.

Right now I still believe https://github.com/wikimedia/mediawiki-containers and this repo are aiming for different things. This repo is strictly aiming at development, as far as I can tell from the readme and other documentation in tickets mediawiki-containers isn't really trying to do that.

I feel like all of this discussion would better placed in a phabricator ticket.
Perhaps https://phabricator.wikimedia.org/T122992 ?

[addshore]

Well, I have managed to get kubernetes all set up on a small group of VMs with kubeadm, but minikube still doesn't work for me...
I'll come back to this in another month or so and see if anything has moved in minikube to make it work!

[addshore]

Just signed up for the "docker-for-windows" kubernetes beta.
Might be able to push for a switch once I have a local kubernetes.

[kostajh]

For those who haven't seen it yet: https://gerrit.wikimedia.org/r/plugins/gitiles/releng/local-charts/