Skip to content

Feature Request: Assign replicas to GPUs #11863

@kheyer

Description

@kheyer

Description

I have a service that runs on a single GPU. If I have multiple GPUs available, I would like to create one replica of this service for each GPU available.

Currently, I can only do this by explicitly creating the service multiple times in the docker-compose file and changing the device_ids section of the resources

  resources:
    reservations:
      devices:
      - driver: "nvidia"
        device_ids: ["0"]
        capabilities: [gpu]

Having to create 8 near-identical replicas of the same service in the config file is unwieldy.

I would like to specify the service once and set the replicas with

replicas: ${GPU_COUNT}

However this requires some way for each replica to know what GPU to use. My understanding is currently there is no way to do this, and there isn't a good way to go off a "replica index" for each container (see #9153).

Some method of mapping replicas of a service to different GPUs would be helpful.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions