Running setup on a specific role shouldn't setup accessories

[nickhammond]

The use case for this is to be able to bring a new server online with an existing kamal deployed setup.

Given I have one web role, one accessory:

servers:
  web:
    host: 123.456.78.9
accessories:
  db:
    host: 123.456.78.9

And I want to add another web server into the mix so servers becomes:

servers:
  web:
    host: 123.456.78.9
  web_new:
    host: 111.222.333.444
accessories:
  db:
    host: 123.456.78.9

To bring the server online you'd need to run kamal setup -r web_new so that it pushes up your env files first but that fails with:

docker stderr: docker: Error response from daemon: Conflict. The container name "/hey-db" is already in use by container "7939d26efb157e547d6a70bf0e713a5ccc9079a016ca8261518dbfd6323da5cc". You have to remove (or rename) that container to be able to reuse that name.

I understand that hey-db has already been provisioned and you can work around this by just running kamal deploy -r web_new after that first failure but I think it probably shouldn't try to setup any accessories since they're not being targeted. In this example the accessory is also on a different host so it's not a host target.

So currently it's:

kamal setup -r web_new
# fails at existing container for accessory
kamal deploy -r web_new
# deploys successfully

Thinking we could adjust so that there's a usable server after the setup command finishes running.

[dihmeetree]

Yep ran into this issue as well! Definitely a +1 for this!

[a3kov]

IMHO we don't need multiple "app" roles - in your case you should add the server to the web role.

[nickhammond]

@a3kov There's more context that I left out from that example, such as routing rules, env, and labels for the new role. It's not just a duplicate of the exact same web service, it runs with a different setup but the same app.

For example, one web role has more resources(cpu/memory) and uses more DB_POOL connections. Another example, we use host-based routing so we can direct traffic to a specific container for logging, scalability, and many other reasons.

[a3kov]

Well to support this case Kamal should have explicit app roles configuration. Currently I got the idea that there's one "primary" role ("web") that has the app deployed. Otherwise having pure-accessory servers wouldn't be possible. Always deploying app to every server doesn't make sense

[nickhammond]

@a3kov It does, here's an expanded example with role-specific configuration. You can have hosts or host within each role.

servers:
  web:
    host: 123.456.78.9
    env:
      clear:
        DB_POOL: 50
    options:
      memory: "6GB"

  web_new:
    host: 111.222.333.444
    env:
      clear:
        DB_POOL: 10
    options:
      memory: "2GB"

accessories:
  db:
    host: 123.456.78.9

Also, the "primary" role is just the one that gets connected to kamal-proxy by default but you can always enable it on additional roles. There's a few commands that specifically run on the primary role.

[nickhammond]

And you can target which --host or --role to specifically deploy to which brings it back to the specific issue that this is talking about.

[a3kov]

It does, here's an expanded example with role-specific configuration. You can have hosts or host within each role.

And it's not good enough, because you can't define a pure accessory-only server this way if you assume that every role is an "app" role

Also, the "primary" role is just the one that gets connected to kamal-proxy by default but you can always enable it on additional roles. There's a few commands that specifically run on the primary role.

Is it documented anywhere ?

And you can target which --host or --role to specifically deploy to which brings it back to the specific issue that this is talking about.

And it's broken also, see #1159

[nickhammond]

@a3kov You can define a pure accessory-only server, you just don't use deploy commands, just the accessory commands. https://www.fromthekeyboard.com/using-kamal-to-deploy-and-manage-your-services/

Yes, primary is documented here https://kamal-deploy.org/docs/configuration/roles/#custom-role-configuration

#1159 is the same issue as this one.

[a3kov]

Thanks! I've missed those accessory commands

Yes, primary is documented here https://kamal-deploy.org/docs/configuration/roles/#custom-role-configuration

Well I remember reading that, but in the end I got the impression only web role will run the app. Maybe the docs need improvements on this..

[nickhammond]

@a3kov No prob, feel free to tweak the docs with a PR here https://github.com/basecamp/kamal-site.

[a3kov]

You can define a pure accessory-only server, you just don't use deploy commands, just the accessory commands.

Wait, you mean if I want to have that, I should only run targeted deploys from now on, i.e. kamal deploy -r ? That is not a very good UX..
Besides it's very easy to forget to use the option and to run wrong deployment.
I think having it explicitly set up in the config file would still be the best.

Anyway, it's off-topic, I'm derailing your issue, sorry for that

[nickhammond]

@a3kov No, if you're just using accessories you wouldn't ever call deploy. It's sort of a hack, it's not really meant to be used that way. It's assumed that you're always deploying an app.

[a3kov]

Ah lol, no. that is not my case. I want to run accessories in addition to the app, I just don't always want to run them on the same servers.

[nickhammond]

@a3kov It sounds like you actually want destinations and not just roles.

[a3kov]

Whatever you call it. Kamal supports VMs, not only bare metal. Running multiple containers including the app on a small VM should not be a requirement. I personally think it's a ridiculous requirement. And if I need to use a separate software for that - I would rather switch to it completely, instead of using 2 tools

[nickhammond]

@a3kov That's not a requirement of Kamal. Anyways, this is off-topic. Best of luck with whatever solution you end up with.