Skip to content

Rolling deployments for repo updates #2853

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 30, 2025
Merged

Conversation

jvstme
Copy link
Collaborator

@jvstme jvstme commented Jun 30, 2025

  • Support rolling deployments for services when
    the run repo is updated: new commits are added,
    the branch is changed, uncommitted files are
    updated, etc. Switching from one repo to another
    is not yet supported.

    Note: A side effect of this change is that
    if the run configuration file is stored in the
    same repo that is used for the run, any changes
    to the configuration file will also be
    considered a change to the repo
    (repo_code_hash), and hence require a rolling
    deployment for services or a full restart for
    tasks and dev environments. Previously, this was
    not the case, since changes to repo_code_hash
    were ignored for existing jobs. This new
    behavior makes it more difficult to avoid
    redeployment when changing some configuration
    properties, namely priority,
    inactivity_duration, replicas, and
    scaling. However, we consider this acceptable,
    since changing these properties in-place is an
    advanced use case and can still be achieved by
    moving the configuration file out of the repo.

  • Improve run plan output in dstack apply when
    attempting an in-place update:

    • Show not only the list of changed
      configuration properties, but also other
      changes from the run spec, such as
      repo-related changes.

      $ dstack apply -f test-service.dstack.yml
      
      Active run test-service already exists. Detected changes that can be updated in-place:
      - Repo state (branch, commit, or other)
      - Repo files
      - Configuration properties:
        - env
      
      Update the run? [y/n]:
    • Show the list of changes not only when
      in-place update is possible, but also when it
      is not. This will help users understand why a
      run cannot be updated in-place.

      $ dstack apply -f test-service.dstack.yml
      
      Active run test-service already exists. Detected changes that cannot be updated in-place:
      - Repo files
      - Configuration properties:
        - gateway
      
      Stop and override the run? [y/n]:

      Currently, all detected changes are listed
      together. An area for future improvement is
      highlighting the changes that prevent an
      in-place update.

#2180

- Support rolling deployments for services when
  the run repo is updated: new commits are added,
  the branch is changed, uncommitted files are
  updated, etc. Switching from one repo to another
  is not yet supported.

  > **Note**: A side effect of this change is that
  if the run configuration file is stored in the
  same repo that is used for the run, any changes
  to the configuration file will also be
  considered a change to the repo
  (`repo_code_hash`), and hence require a rolling
  deployment for services or a full restart for
  tasks and dev environments. Previously, this was
  not the case, since changes to `repo_code_hash`
  were ignored for existing jobs. This new
  behavior makes it more difficult to avoid
  redeployment when changing some configuration
  properties, namely `priority`,
  `inactivity_duration`, `replicas`, and
  `scaling`. However, we consider this acceptable,
  since changing these properties in-place is an
  advanced use case and can still be achieved by
  moving the configuration file out of the repo.

- Improve run plan output in `dstack apply` when
  attempting an in-place update:

  - Show not only the list of changed
    configuration properties, but also other
    changes from the run spec, such as
    repo-related changes.

    ```shell
    $ dstack apply -f test-service.dstack.yml

    Active run test-service already exists. Detected changes that can be updated in-place:
    - Repo state (branch, commit, or other)
    - Repo files
    - Configuration properties:
      - env

    Update the run? [y/n]:
    ```

  - Show the list of changes not only when
    in-place update is possible, but also when it
    is not. This will help users understand why a
    run cannot be updated in-place.

    ```shell
    $ dstack apply -f test-service.dstack.yml

    Active run test-service already exists. Detected changes that cannot be updated in-place:
    - Repo files
    - Configuration properties:
      - gateway

    Stop and override the run? [y/n]:
    ```

    Currently, all detected changes are listed
    together. An area for future improvement is
    highlighting the changes that prevent an
    in-place update.
@jvstme jvstme requested a review from un-def June 30, 2025 07:36
@jvstme jvstme merged commit ea79096 into master Jun 30, 2025
25 checks passed
@jvstme jvstme deleted the issue_2180_repo_deployments branch June 30, 2025 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants