feat: Add sindri upgrade command for in-place version upgrades

[pacphi]

Sindri Upgrade Feature - Technical Implementation Plan

This document outlines the technical implementation plan for adding in-place upgrade capability to Sindri instances.

Overview

The sindri upgrade command enables users to upgrade a running Sindri instance from one version to another without requiring a full container rebuild. This applies to both local development environments and deployed instances (Fly.io, Docker, DevPod).

Goals

1. Seamless upgrades: Upgrade Sindri CLI, extensions, and schemas in-place
2. Safe rollback: Automatic backup with one-command rollback capability
3. Extension upgrades: Upgrade installed extensions and their package dependencies
4. Testable in CI: Verify upgrade paths before releases
5. Minimal downtime: No container restart required for most upgrades

Non-Goals (Phase 1)

• Upgrading system binaries (mise, claude) - these require container rebuild
• Cross-major-version migrations with breaking schema changes
• Automatic extension compatibility resolution

---

Architecture

Components to Upgrade

Based on Dockerfile lines 90-92, upgrades target:

/docker/lib/       # Extension definitions, schemas, common.sh, profiles, registry
/docker/cli/       # sindri, extension-manager, VERSION
/docker/deploy/    # Provider adapters (docker, fly, devpod, k8s)

Upgrade Flow

┌─────────────────────────────────────────────────────────────────────┐
│                        sindri upgrade                               │
├─────────────────────────────────────────────────────────────────────┤
│  1. Check Context (inside container vs local dev)                   │
│  2. Read current VERSION (/docker/cli/VERSION or cli/VERSION)       │
│  3. Query GitHub API for latest release                             │
│  4. Compare versions (semver)                                       │
│  5. If newer version available:                                     │
│     a. Download release tarball                                     │
│     b. Extract to temp directory                                    │
│     c. Backup current installation                                  │
│     d. Apply new files                                              │
│     e. Validate installation                                        │
│     f. Upgrade installed extensions (optional)                      │
│  6. Report success/failure                                          │
└─────────────────────────────────────────────────────────────────────┘

Directory Structure

/docker/
├── .backup/                    # Versioned backups (NEW)
│   └── 1.5.0/                  # Backup of version 1.5.0
│       ├── lib/
│       ├── cli/
│       └── deploy/
├── .upgrade/                   # Temporary upgrade staging (NEW)
│   └── sindri-1.6.0/           # Extracted release
├── lib/                        # Extension system (upgraded in-place)
├── cli/                        # CLI tools (upgraded in-place)
└── deploy/                     # Adapters (upgraded in-place)

---

Implementation

Phase 1: Core Upgrade Command

File: cli/sindri (additions)

# New command: upgrade
#   sindri upgrade              # Upgrade to latest version
#   sindri upgrade --check      # Check for updates without applying
#   sindri upgrade --rollback   # Restore previous version
#   sindri upgrade --version X  # Upgrade to specific version
#   sindri upgrade --extensions # Also upgrade installed extensions

New Module: cli/upgrade.sh

Primary upgrade logic module with functions:

Function	Description
check_upgrade_available	Query GitHub API, compare versions
download_release	Download and extract release tarball
backup_current	Create versioned backup of current installation
apply_upgrade	Copy new files over existing installation
validate_upgrade	Run sanity checks post-upgrade
rollback	Restore from backup
upgrade_extensions	Trigger extension upgrades

Version Comparison

# Semantic version comparison
# Returns: 0 if equal, 1 if v1 > v2, 2 if v1 < v2
version_compare() {
    local v1="$1" v2="$2"
    # Strip 'v' prefix if present
    v1="${v1#v}"
    v2="${v2#v}"
    # Compare major.minor.patch
    ...
}

GitHub API Integration

# Query latest release
get_latest_release() {
    local repo="pacphi/sindri"
    gh api "repos/${repo}/releases/latest" --jq '.tag_name' 2>/dev/null || \
    curl -s "https://api.github.com/repos/${repo}/releases/latest" | jq -r '.tag_name'
}

# Query specific release
get_release_info() {
    local version="$1"
    gh api "repos/${repo}/releases/tags/v${version}" 2>/dev/null || \
    curl -s "https://api.github.com/repos/${repo}/releases/tags/v${version}"
}

Phase 2: Release Artifact Enhancement

Current Release Assets

The release workflow (.github/workflows/release.yml) currently produces:

• install.sh - Fresh installation script
• QUICK_REFERENCE.md - Documentation

New Release Asset: sindri-upgrade-{version}.tar.gz

Add to release workflow:

- name: Create upgrade tarball
  run: |
    version="${{ needs.validate-tag.outputs.version }}"

    # Create minimal upgrade package
    mkdir -p sindri-upgrade-${version}
    cp -a docker/lib sindri-upgrade-${version}/
    cp -a cli sindri-upgrade-${version}/
    cp -a deploy sindri-upgrade-${version}/

    # Include upgrade manifest
    cat > sindri-upgrade-${version}/UPGRADE_MANIFEST.json << EOF
    {
      "version": "${version}",
      "created": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
      "components": ["lib", "cli", "deploy"],
      "schemaVersion": "1.0",
      "minUpgradeFrom": "1.0.0",
      "checksums": {
        "lib": "$(find sindri-upgrade-${version}/lib -type f -exec sha256sum {} \; | sha256sum | cut -d' ' -f1)",
        "cli": "$(find sindri-upgrade-${version}/cli -type f -exec sha256sum {} \; | sha256sum | cut -d' ' -f1)",
        "deploy": "$(find sindri-upgrade-${version}/deploy -type f -exec sha256sum {} \; | sha256sum | cut -d' ' -f1)"
      }
    }
    EOF

    tar czf sindri-upgrade-${version}.tar.gz sindri-upgrade-${version}/

- name: Create GitHub Release
  uses: softprops/action-gh-release@v2
  with:
    files: |
      install.sh
      QUICK_REFERENCE.md
      sindri-upgrade-${{ needs.validate-tag.outputs.version }}.tar.gz

Phase 3: Extension Upgrade Integration

Extension Upgrade Schema (Already Exists)

From docker/lib/schemas/extension.schema.json:

upgrade:
  strategy: automatic|manual|none
  mise:
    upgradeAll: true
    tools: [tool1, tool2]
  apt:
    packages: [pkg1, pkg2]
    updateFirst: true
  script:
    path: upgrade.sh
    timeout: 600

New Module: cli/extension-manager-modules/upgrade.sh

# Upgrade single extension
upgrade_extension() {
    local ext_name="$1"
    local ext_dir="${DOCKER_LIB}/extensions/${ext_name}"

    # Read upgrade strategy
    local strategy
    strategy=$(load_yaml "${ext_dir}/extension.yaml" ".upgrade.strategy" 2>/dev/null || echo "automatic")

    case "$strategy" in
        automatic)
            upgrade_extension_automatic "$ext_name"
            ;;
        manual)
            upgrade_extension_manual "$ext_name"
            ;;
        none)
            print_warning "Extension $ext_name does not support upgrades"
            ;;
    esac
}

# Automatic upgrade (mise-based)
upgrade_extension_automatic() {
    local ext_name="$1"
    local ext_dir="${DOCKER_LIB}/extensions/${ext_name}"

    # Check for mise upgrade config
    if load_yaml "${ext_dir}/extension.yaml" ".upgrade.mise" &>/dev/null; then
        local upgrade_all
        upgrade_all=$(load_yaml "${ext_dir}/extension.yaml" ".upgrade.mise.upgradeAll" 2>/dev/null || echo "true")

        if [[ "$upgrade_all" == "true" ]]; then
            mise upgrade --yes
        else
            local tools
            tools=$(load_yaml "${ext_dir}/extension.yaml" ".upgrade.mise.tools[]" 2>/dev/null)
            for tool in $tools; do
                mise upgrade "$tool" --yes
            done
        fi
    fi

    # Check for apt upgrade config
    if load_yaml "${ext_dir}/extension.yaml" ".upgrade.apt" &>/dev/null; then
        local update_first
        update_first=$(load_yaml "${ext_dir}/extension.yaml" ".upgrade.apt.updateFirst" 2>/dev/null || echo "true")

        [[ "$update_first" == "true" ]] && sudo apt-get update

        local packages
        packages=$(load_yaml "${ext_dir}/extension.yaml" ".upgrade.apt.packages[]" 2>/dev/null)
        if [[ -n "$packages" ]]; then
            sudo apt-get upgrade -y $packages
        fi
    fi
}

# Manual upgrade (script-based)
upgrade_extension_manual() {
    local ext_name="$1"
    local ext_dir="${DOCKER_LIB}/extensions/${ext_name}"

    local script_path
    script_path=$(load_yaml "${ext_dir}/extension.yaml" ".upgrade.script.path" 2>/dev/null)

    if [[ -n "$script_path" && -f "${ext_dir}/${script_path}" ]]; then
        local timeout
        timeout=$(load_yaml "${ext_dir}/extension.yaml" ".upgrade.script.timeout" 2>/dev/null || echo "600")

        timeout "$timeout" bash "${ext_dir}/${script_path}"
    fi
}

New Command: extension-manager upgrade

Add to cli/extension-manager:

upgrade)
    if [[ $# -eq 0 ]]; then
        # Upgrade all installed extensions
        local installed
        installed=$(get_active_extensions)
        for ext in $installed; do
            upgrade_extension "$ext"
        done
    else
        # Upgrade specific extension
        upgrade_extension "$1"
    fi
    ;;

Phase 4: CI/CD Testing

New Workflow: .github/workflows/test-upgrade.yml

name: Test Upgrade Path

on:
  pull_request:
    branches: [main]
    paths:
      - 'cli/**'
      - 'docker/**'
      - 'deploy/**'
  workflow_dispatch:
    inputs:
      from_version:
        description: 'Version to upgrade from'
        required: true
        default: 'latest'
      to_version:
        description: 'Version to upgrade to'
        required: false
        default: 'current'

jobs:
  test-upgrade-local:
    name: Test Local Upgrade
    runs-on: ubuntu-latest
    steps:
      - name: Checkout current branch
        uses: actions/checkout@v6
        with:
          path: current

      - name: Determine versions
        id: versions
        run: |
          # Get 'from' version (previous release or specified)
          if [[ "${{ inputs.from_version }}" == "latest" || -z "${{ inputs.from_version }}" ]]; then
            from_version=$(gh api repos/${{ github.repository }}/releases/latest --jq .tag_name | tr -d 'v')
          else
            from_version="${{ inputs.from_version }}"
          fi

          # Get 'to' version (current branch or specified)
          if [[ "${{ inputs.to_version }}" == "current" || -z "${{ inputs.to_version }}" ]]; then
            to_version=$(cat current/cli/VERSION)
          else
            to_version="${{ inputs.to_version }}"
          fi

          echo "from=$from_version" >> $GITHUB_OUTPUT
          echo "to=$to_version" >> $GITHUB_OUTPUT
        env:
          GH_TOKEN: ${{ github.token }}

      - name: Checkout 'from' version
        uses: actions/checkout@v6
        with:
          ref: v${{ steps.versions.outputs.from }}
          path: from-version

      - name: Set up Docker
        uses: docker/setup-buildx-action@v3

      - name: Build 'from' version image
        run: |
          cd from-version
          docker build -t sindri:from .

      - name: Start container with 'from' version
        run: |
          docker run -d --name sindri-upgrade-test \
            -v sindri-test-home:/alt/home/developer \
            sindri:from sleep infinity

      - name: Install test extensions (baseline)
        run: |
          docker exec sindri-upgrade-test \
            /docker/cli/extension-manager install nodejs
          docker exec sindri-upgrade-test \
            /docker/cli/extension-manager status

      - name: Copy 'to' version into container
        run: |
          # Simulate upgrade tarball
          docker cp current/docker/lib sindri-upgrade-test:/tmp/upgrade-lib
          docker cp current/cli sindri-upgrade-test:/tmp/upgrade-cli
          docker cp current/deploy sindri-upgrade-test:/tmp/upgrade-deploy

      - name: Run upgrade
        run: |
          docker exec sindri-upgrade-test bash -c '
            # Backup current
            mkdir -p /docker/.backup/$(cat /docker/cli/VERSION)
            cp -a /docker/lib /docker/.backup/$(cat /docker/cli/VERSION)/
            cp -a /docker/cli /docker/.backup/$(cat /docker/cli/VERSION)/
            cp -a /docker/deploy /docker/.backup/$(cat /docker/cli/VERSION)/

            # Apply upgrade
            rsync -a /tmp/upgrade-lib/ /docker/lib/
            rsync -a /tmp/upgrade-cli/ /docker/cli/
            rsync -a /tmp/upgrade-deploy/ /docker/deploy/

            echo "Upgrade complete"
          '

      - name: Validate upgrade
        run: |
          # Check version updated
          docker exec sindri-upgrade-test cat /docker/cli/VERSION

          # Check CLI works
          docker exec sindri-upgrade-test /docker/cli/sindri --version

          # Check extension-manager works
          docker exec sindri-upgrade-test /docker/cli/extension-manager list

          # Check installed extensions still valid
          docker exec sindri-upgrade-test /docker/cli/extension-manager validate nodejs

      - name: Test rollback
        run: |
          docker exec sindri-upgrade-test bash -c '
            # Rollback to previous version
            backup_version=$(ls /docker/.backup/ | head -1)
            cp -a /docker/.backup/${backup_version}/lib/* /docker/lib/
            cp -a /docker/.backup/${backup_version}/cli/* /docker/cli/
            cp -a /docker/.backup/${backup_version}/deploy/* /docker/deploy/

            echo "Rollback complete"
          '

          # Verify rollback
          docker exec sindri-upgrade-test /docker/cli/sindri --version

      - name: Cleanup
        if: always()
        run: |
          docker rm -f sindri-upgrade-test || true
          docker volume rm sindri-test-home || true

  test-upgrade-fly:
    name: Test Fly.io Upgrade
    runs-on: ubuntu-latest
    if: github.event_name == 'workflow_dispatch'
    steps:
      - name: Checkout
        uses: actions/checkout@v6

      - name: Setup flyctl
        uses: superfly/flyctl-actions/setup-flyctl@master

      - name: Deploy 'from' version
        run: |
          # Deploy older version to test app
          echo "Deploy from version and test upgrade via SSH"
          # flyctl deploy --app sindri-upgrade-test --image ghcr.io/pacphi/sindri:${{ inputs.from_version }}
        env:
          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}

      - name: Run upgrade via SSH
        run: |
          echo "SSH into app and run: sindri upgrade"
          # flyctl ssh console --app sindri-upgrade-test -C "/docker/cli/sindri upgrade"

Integration with Release Workflow

Add upgrade testing to .github/workflows/release.yml:

# New job: Test upgrade before publishing
test-upgrade-path:
  runs-on: ubuntu-latest
  needs: [validate-tag, build-image]
  steps:
    - uses: actions/checkout@v6

    - name: Get previous version
      id: prev
      run: |
        prev_tag=$(git describe --tags --abbrev=0 HEAD^ 2>/dev/null || echo "")
        echo "version=${prev_tag#v}" >> $GITHUB_OUTPUT

    - name: Test upgrade from previous release
      if: steps.prev.outputs.version != ''
      run: |
        # Pull previous version
        docker pull ghcr.io/${{ github.repository }}:${{ steps.prev.outputs.version }}

        # Start container
        docker run -d --name upgrade-test \
          ghcr.io/${{ github.repository }}:${{ steps.prev.outputs.version }} \
          sleep infinity

        # Copy new version files
        docker cp docker/lib upgrade-test:/tmp/new-lib
        docker cp cli upgrade-test:/tmp/new-cli
        docker cp deploy upgrade-test:/tmp/new-deploy

        # Apply upgrade
        docker exec upgrade-test bash -c '
          rsync -a /tmp/new-lib/ /docker/lib/
          rsync -a /tmp/new-cli/ /docker/cli/
          rsync -a /tmp/new-deploy/ /docker/deploy/
        '

        # Validate
        docker exec upgrade-test /docker/cli/sindri --version
        docker exec upgrade-test /docker/cli/extension-manager list

---

CLI Interface

sindri upgrade

Usage: sindri upgrade [OPTIONS]

Upgrade Sindri to a newer version.

Options:
    --check             Check for updates without applying
    --version <ver>     Upgrade to specific version (default: latest)
    --extensions        Also upgrade installed extensions
    --rollback          Restore previous version from backup
    --force             Skip confirmation prompts
    --dry-run           Show what would be upgraded without making changes
    -h, --help          Show this help message

Examples:
    sindri upgrade              # Upgrade to latest version
    sindri upgrade --check      # Check if update available
    sindri upgrade --version 1.6.0  # Upgrade to specific version
    sindri upgrade --extensions # Upgrade Sindri and all extensions
    sindri upgrade --rollback   # Restore previous version

Environment:
    SINDRI_UPGRADE_REPO     Override GitHub repository (default: pacphi/sindri)
    SINDRI_UPGRADE_CHANNEL  Release channel: stable, beta, alpha (default: stable)

extension-manager upgrade

Usage: extension-manager upgrade [EXTENSION]

Upgrade extensions to latest versions.

Arguments:
    EXTENSION           Specific extension to upgrade (optional)
                        If omitted, upgrades all installed extensions

Options:
    --dry-run           Show what would be upgraded
    --force             Skip confirmation prompts
    -h, --help          Show this help message

Examples:
    extension-manager upgrade           # Upgrade all extensions
    extension-manager upgrade nodejs    # Upgrade nodejs extension only

---

Rollback Strategy

Backup Format

/docker/.backup/
└── 1.5.0/
    ├── lib/
    │   ├── extensions/
    │   ├── schemas/
    │   ├── profiles.yaml
    │   └── ...
    ├── cli/
    │   ├── sindri
    │   ├── extension-manager
    │   └── VERSION
    └── deploy/
        └── adapters/

Rollback Process

sindri upgrade --rollback

# 1. List available backups
# 2. Select version to restore (default: most recent)
# 3. Validate backup integrity
# 4. Copy backup over current installation
# 5. Validate restored installation

Backup Retention

• Keep last 3 versions by default
• Configurable via SINDRI_BACKUP_RETAIN environment variable
• Automatic cleanup during upgrade

---

Error Handling

Pre-upgrade Checks

Check	Action on Failure
Network connectivity	Abort with helpful message
Disk space (100MB free)	Abort with warning
Write permissions to /docker/	Abort with permission instructions
GitHub API rate limit	Retry with backoff, then abort
Invalid current VERSION	Warn, attempt upgrade anyway

Upgrade Failure Recovery

1. Download fails: Clean temp, report error, no changes made
2. Backup fails: Abort upgrade, report disk space issue
3. Apply fails: Automatic rollback from backup
4. Validation fails: Prompt for manual rollback

Validation Steps

validate_upgrade() {
    local errors=0

    # Check VERSION file updated
    [[ -f /docker/cli/VERSION ]] || ((errors++))

    # Check sindri CLI works
    /docker/cli/sindri --version >/dev/null 2>&1 || ((errors++))

    # Check extension-manager works
    /docker/cli/extension-manager list >/dev/null 2>&1 || ((errors++))

    # Check schemas are valid JSON
    for schema in /docker/lib/schemas/*.json; do
        jq empty "$schema" 2>/dev/null || ((errors++))
    done

    return $errors
}

---

Testing Strategy

Unit Tests

Test	Location
Version comparison	test/unit/upgrade/version_compare.bats
Backup creation	test/unit/upgrade/backup.bats
Rollback	test/unit/upgrade/rollback.bats
GitHub API parsing	test/unit/upgrade/github_api.bats

Integration Tests

Test	Location
Full upgrade cycle	test/integration/upgrade_cycle.bats
Extension upgrade	test/integration/extension_upgrade.bats
Cross-version compat	CI workflow test-upgrade.yml

Manual Testing Matrix

From Version	To Version	Provider	Test Result
1.4.0	1.5.0	Docker
1.4.0	1.5.0	Fly.io
1.5.0	1.6.0	Docker
1.5.0	1.6.0	Fly.io
1.5.0	1.6.0	DevPod

---

Implementation Phases

Phase 1: Core Upgrade (Week 1-2)

• Create cli/upgrade.sh module
• Add upgrade command to cli/sindri
• Implement version check and comparison
• Implement download and extraction
• Implement backup and restore
• Add basic validation

Phase 2: Release Integration (Week 2-3)

• Add upgrade tarball to release workflow
• Create UPGRADE_MANIFEST.json format
• Add checksum verification
• Document upgrade in release notes

Phase 3: Extension Upgrades (Week 3-4)

• Create cli/extension-manager-modules/upgrade.sh
• Add upgrade command to extension-manager
• Implement mise-based upgrades
• Implement apt-based upgrades
• Implement script-based upgrades

Phase 4: CI Testing (Week 4-5)

• Create test-upgrade.yml workflow
• Add upgrade tests to release workflow
• Create upgrade test matrix
• Document testing procedures

Phase 5: Documentation & Polish (Week 5-6)

• Update CLI.md with upgrade commands
• Update RELEASE.md with upgrade testing
• Create UPGRADING.md guide
• Add troubleshooting section

---

Security Considerations

1. Release Verification: Verify release checksums before applying
2. GitHub API: Use authenticated requests when possible
3. File Permissions: Preserve ownership and permissions during upgrade
4. Backup Security: Backups contain same files, no additional exposure
5. Network Security: HTTPS-only for all downloads

---

Open Questions

1. Minimum supported upgrade path: Should we support upgrading from any version, or only N-1?
2. Extension compatibility: How to handle extension schema changes between versions?
3. Partial upgrades: Should we support upgrading only CLI without extensions?
4. Automatic upgrades: Should there be a --auto flag for unattended upgrades?
5. Notification: Should Sindri check for updates on startup and notify users?

---

References

• RELEASE.md - Current release process
• extension.schema.json - Extension upgrade schema
• Dockerfile - Build process and file locations
• GitHub Releases API

[pacphi]

Simpler Fly.io Upgrade Path Discovered

Today while debugging a JVM extension installation issue, we validated a simpler upgrade mechanism for Fly.io deployments:

# From local sindri repo (with updated code)
flyctl deploy -a <app-name>

What happens:

1. Builds new Docker image with updated /docker/lib, /docker/cli, /docker/deploy
2. Pushes to Fly.io registry
3. Replaces the machine with the new image
4. Preserves the existing volume at /alt/home/developer (workspace, installed extensions, user configs, SDKMAN installations)

Key insight

The volume is a separate Fly.io resource from the machine. flyctl deploy only replaces the machine/image while reattaching the existing volume. This is by design in the fly-adapter (line 307: "Volume is automatically created by fly deploy if it doesn't exist" - existing volumes are reused).

This approach:

• ✅ Updates all immutable system files (scripts, schemas, extension definitions)
• ✅ Preserves user data (home directory, workspace, installed tools)
• ✅ Preserves extension installations (they're in $HOME/.local/share/mise on the volume)
• ✅ Simple, no new code required
• ⚠️ Does require rebuilding the image (not true in-place upgrade)
• ⚠️ Brief downtime during machine replacement (~30-60 seconds)

Suggestion

Consider adding this as a "Quick Upgrade (Fly.io)" section in the docs as an immediate solution, while the full sindri upgrade command (for in-place upgrades without image rebuild) remains the longer-term goal.

---

🤖 Generated with Claude Code