Support creating a Cortex cluster in a pre-existing VPC/subnets

[deliahu]

Description

See https://eksctl.io/usage/vpc-networking/

Blog post with an example: https://joachim8675309.medium.com/create-eks-with-an-existing-vpc-8e31d95ccc5b

[j3lev]

Hey there, will this be implemented at some point?

[deliahu]

@josephlevenson we have been considering this. Our hesitation is that Cortex resources will be mixed with your existing resources in the same network, which can create confusion and has the risk of having side-effects on your existing infra. Keeping them separate allows for a better separation of concerns, and a "clean" environment that Cortex is guaranteed to be able to configure properly. What is your motivation for deploying Cortex in an existing VPC?

[j3lev]

@deliahu Understood. Our use case is pretty simple, we currently have a thin microservice architecture in a single VPC. The cortex cluster interfaces with resources (DB/caching layer that will be used by other services). It would be nice to have our predict service encapsulated within the same VPC without having to set up peering. Does that make sense?

[deliahu]

@josephlevenson that makes sense. Have you tried to set up peering, without success? Or did you get it working, but it's inconvenient?

[j3lev]

We just deployed the cluster today, have not tried peering yet but will likely end up going that route. It does seem a bit inconvenient to have the one service sitting in its own VPC with the rest of the architecture in another. FWIW, Cortex is great and has otherwise made our lives easier so your hard work is much appreciated!

[deliahu]

Thanks for the kind words, I'm glad to hear you've enjoyed using Cortex!

We have this guide to help with VPC peering. Keep us posted on how it goes; it'd be nice to hear your thoughts once you get it set up (you can also connect with us on gitter or email us at dev@cortex.dev).