Move ark server & minio to heptio-ark-server ns

[ncdc]

Move ark server deployment & minio deployment to a separate namespace
from the backups/schedules/restores/config because backups now have a
finalizer. If everything lives in one namespace, you have to delete all
the backups and wait for the GC controller to process them and remove the
finalizer from each before deleting the namespace.

By moving the server into a separate namespace, users can now delete the
heptio-ark namespace the normal way (kubectl delete), and once that
namespace is fully removed, they can delete the heptio-ark-server
namespace.

Signed-off-by: Andy Goldstein andy.goldstein@gmail.com

[ncdc]

Still need to update the docs, as they say you can just kubectl delete -f examples/common, which is still going to be a problem for the moment.

[ncdc]

Docs updated

[Bradamant3]

"Prior" makes me cringe, but it works. Saving the fiddly stuff for another time :-)

[ncdc]

Prior -> Before

[nrb]

Haven't run this yet, but does this somewhat nullify running Ark in any namespace? It's more like the backups are now managed in any namespace and the server will always run in heptio-ark-server, isn't it?

These questions are more clarifying docs and terminology than the code.

[ncdc]

@nrb not exactly. You can run the server in whatever namespace you want (assuming you adjust the yaml appropriately). And you can point the server at whatever namespace you want for the backups/schedules/restores/config. I debated renaming the --namespace flag for ark server to something like --backup-namespace, but I don't think that name is entirely clear about its purpose, plus that only mentions backups and not the rest of the custom resources.

This again highlights the need for a simple way to put the server in 1 ns and the backups etc in another one. Something like ark server install --server-namespace foo --data-namespace bar could do it.

[ncdc]

@Bradamant3 looks like I probably should update docs/namespace.yaml too, since we now have 2 namespaces to possibly change instead of 1.

[nrb]

Credentials namespace in https://github.com/heptio/ark/blob/master/docs/aws-config.md would need to be changed.

e.g.

kubectl create secret generic cloud-credentials \
    --namespace <ARK_NAMESPACE> \
    --from-file cloud=credentials-ark

The secret needs to be in heptio-ark-server or wherever the server is.

[ncdc]

@nrb correct, thanks for pointing that out

[Bradamant3]

edit:
WARNING: It is recommended to run the Ark server in one namespace, and your backups, schedules, restores, and config in a different namespace. You might encounter issues with deleting a single Ark namespace that runs everything.
(edits for clarity only)

[Bradamant3]

one edit for clarity, otherwise /lgtm

[ncdc]

Thanks, I've updated (with a minor edit to your suggestion).

[Bradamant3]

/lgtm (yeah, I tripped a bit over that wording, didn't I?)

[skriss]

why's the RBAC stuff going away?

[ncdc]

Because our service account has cluster-admin (granted above)

[skriss]

Given that someone could still end up "stuck" if they remove Ark in the wrong way (i.e. delete the server NS first), it'd probably be helpful to add an entry to the Troubleshooting section in the docs about what to do if that happens (including your cmd @ncdc ?)

Also, @Bradamant3 can we add a link to the GH pages site near the top of the README? How do folks find it otherwise?

[skriss]

Did some testing on GKE and code/config changes LGTM. One other observation is that using the jq snippet from @ncdc to remove finalizers doesn't seem to work if there's only a single backup.

[ncdc]

Oh that's probably because it returns/prints a single one instead of a list

[ncdc]

We can use kubectl -n heptio-ark get backup instead of ark backup get and it will always return a list

[skriss]

yep, confirmed that works

[Bradamant3]

LGTM

[ncdc]

Going to merge as-is and @Bradamant3 will submit a follow-up for any additional docs edits.