fix(providers): improve version source consistency and reliability

# Issue #140: Version Source Consistency & Self-Hosting Strategy

## Problem Summary

The current provider implementations have version source inconsistencies:
- `ListAvailable()` fetches versions from one source
- `Install()` downloads from a different source
- This causes users to see versions they can't actually install

### Current Issues by Runtime

| Runtime | Problem |
|---------|---------|
| Python | `ListAvailable()` scrapes python.org (all versions), `Install()` uses python-build-standalone with hardcoded date `20240814` |
| Ruby | Hardcoded OS versions in asset names (`ubuntu-22.04`, `macos-13`) break when runners update |
| Node.js | Works correctly, but should use manifest for consistency |

---

## Solution: Manifest-Based Architecture

**All runtimes** (Python, Ruby, Node.js, and any future additions) will use the same manifest-based approach:

1. dtvem hosts curated manifest files listing ALL available versions
2. Each version/platform entry contains a direct URL to the best pre-built binary
3. SHA256 checksums included for download verification
4. Versions without pre-builts marked explicitly, with option to request builds

This ensures a consistent experience across all runtimes - same code path, same behavior.

---

## Architecture Overview

```
┌─────────────────────────────────────────────────────────────────────┐
│                     dtvem Manifest System                          │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  ┌─────────────┐     ┌─────────────────────────────────────────┐   │
│  │   dtvem     │────▶│  manifests.dtvem.io (Cloudflare R2)     │   │
│  │   CLI       │     │    ├── python.json                      │   │
│  └─────────────┘     │    ├── ruby.json                        │   │
│                      │    └── node.json                        │   │
│                      └─────────────────────────────────────────┘   │
│                                      │                              │
│                         Contains direct URLs to:                    │
│                                      ▼                              │
│  ┌───────────────────────────────────────────────────────────────┐ │
│  │                    Upstream Binary Sources                    │ │
│  ├───────────────────────────────────────────────────────────────┤ │
│  │ Python:                                                       │ │
│  │   • python.org (Windows .zip)                                │ │
│  │   • astral-sh/python-build-standalone (macOS/Linux .tar.gz)  │ │
│  │   • builds.dtvem.io (custom builds for gaps)                 │ │
│  ├───────────────────────────────────────────────────────────────┤ │
│  │ Ruby:                                                         │ │
│  │   • ruby-builder (macOS/Linux .tar.gz)                       │ │
│  │   • RubyInstaller2 (Windows .7z)                             │ │
│  │   • builds.dtvem.io (custom builds for gaps)                 │ │
│  ├───────────────────────────────────────────────────────────────┤ │
│  │ Node.js:                                                      │ │
│  │   • nodejs.org/dist (all platforms .tar.gz/.zip)             │ │
│  └───────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```

**Request Flow:**
1. `dtvem list-all python` → Fetch manifest from `manifests.dtvem.io/python.json` (cached locally)
2. `dtvem install python 3.13.1` → Use cached manifest, download binary from URL
3. Verify SHA256 checksum before extracting

---

## Manifest Schema

### Example: `python.json`

```json
{
  "schema": 1,
  "versions": {
    "3.13.1": {
      "windows-amd64": {
        "url": "https://www.python.org/ftp/python/3.13.1/python-3.13.1-embed-amd64.zip",
        "sha256": "a1b2c3d4e5f6..."
      },
      "darwin-amd64": {
        "url": "https://github.com/astral-sh/python-build-standalone/releases/download/20251209/cpython-3.13.1%2B20251209-x86_64-apple-darwin-install_only.tar.gz",
        "sha256": "c3d4e5f6a1b2..."
      },
      "linux-amd64": {
        "url": "https://github.com/astral-sh/python-build-standalone/releases/download/20251209/cpython-3.13.1%2B20251209-x86_64-unknown-linux-gnu-install_only.tar.gz",
        "sha256": "e5f6a1b2c3d4..."
      }
    },
    "3.7.17": {
      "windows-amd64": {
        "url": "https://www.python.org/ftp/python/3.7.17/python-3.7.17-embed-amd64.zip",
        "sha256": "..."
      },
      "darwin-amd64": false,
      "linux-amd64": false
    },
    "3.6.15": {
      "windows-amd64": false,
      "darwin-amd64": false,
      "linux-amd64": false
    }
  }
}
```

### Platform Value Types

| Value | Meaning | CLI Behavior |
|-------|---------|--------------|
| `{ "url": "...", "sha256": "..." }` | Pre-built available | Download and install |
| `false` | Version exists, no pre-built | Prompt to request build |
| Key missing from manifest | Version doesn't exist | Error: "not a known version" |

### Design Decisions

| Decision | Choice | Rationale |
|----------|--------|-----------|
| Direct URLs | Yes | No additional lookups needed |
| SHA256 checksums | Yes | Security verification on download |
| Unavailable flag | `false` | Distinguishes "no pre-built" from "doesn't exist" |
| Metadata (LTS, EOL) | No (for now) | Keep manifest small; add later if needed |
| Archive formats only | Yes | No MSI/EXE installers; only .zip, .tar.gz, .7z |

### Platform Keys

Matches Go's `runtime.GOOS-GOARCH`:
- `windows-amd64`, `windows-arm64`, `windows-386`
- `darwin-amd64`, `darwin-arm64`
- `linux-amd64`, `linux-arm64`, `linux-arm`, `linux-386`

---

## New Commands

### `dtvem update`

Force refresh manifest cache (bypass 24-hour TTL):

```
$ dtvem update

Updating manifests...
  Python: 142 versions (3 new)
  Ruby: 87 versions (1 new)
  Node.js: 823 versions (2 new)

Done.
```

### `dtvem request`

Request a build for an unavailable version (opens browser to pre-filled GitHub issue):

```
$ dtvem request python 3.6.15

Opening browser to request build for Python 3.6.15 on darwin-amd64...

If browser doesn't open, visit:
https://github.com/dtvem/dtvem/issues/new?template=build-request.yml&title=build(python):+3.6.15+darwin-amd64&labels=build-request,python,darwin-amd64
```

### Enhanced `dtvem install`

When a version is unavailable, prompt to request:

```
$ dtvem install python 3.6.15

Error: Python 3.6.15 is not available as a pre-built binary for darwin-amd64.

Would you like to request a build? [y/N] y

Opening browser to request build...

If browser doesn't open, visit:
https://github.com/dtvem/dtvem/issues/new?template=build-request.yml&title=build(python):+3.6.15+darwin-amd64&labels=build-request,python,darwin-amd64
```

---

## Implementation Plan

### Phase 1: Manifest Infrastructure

Create `internal/manifest/` package:

```go
type Manifest struct {
    Schema   int                            `json:"schema"`
    Versions map[string]map[string]*Download `json:"versions"`
}

type Download struct {
    URL    string `json:"url"`
    SHA256 string `json:"sha256"`
}

type Client struct {
    baseURL    string  // https://manifests.dtvem.io
    httpClient *http.Client
    cacheDir   string  // ~/.dtvem/cache/
    cacheTTL   time.Duration  // 24 hours
}

func (c *Client) GetManifest(runtime string) (*Manifest, error)
func (c *Client) ForceRefresh() error
```

Add SHA256 verification to download package.

### Phase 2: Update Providers

- **Python**: Use manifest for `ListAvailable()` and `Install()`
- **Ruby**: Use manifest, remove hardcoded OS versions
- **Node.js**: Migrate to manifest for consistency

### Phase 3: Manifest Generation

Script-assisted with human review:

1. Scheduled GitHub Action runs daily
2. Script fetches versions from upstream sources
3. Computes SHA256 checksums
4. Creates PR with manifest changes
5. Human reviews and merges
6. On merge, Action uploads to R2

### Phase 4: Build Request System

#### User Flow

1. `dtvem request` command opens browser to pre-filled GitHub issue URL
2. User submits issue (requires GitHub account)
3. Issue gets `build-request` label automatically
4. Maintainer reviews and adds `build-approved` label
5. GitHub Action triggers, builds ONLY the requested runtime + version + platform
6. Binary uploaded to `builds.dtvem.io`
7. Manifest PR created automatically
8. Issue closed with success comment

```go
// cmd/request.go - opens browser with pre-filled issue URL
func openBuildRequest(runtime, version, platform string) error {
    url := fmt.Sprintf(
        "https://github.com/dtvem/dtvem/issues/new?template=build-request.yml"+
        "&title=build(%s):+%s+%s&labels=build-request,%s,%s",
        runtime, version, platform, runtime, platform,
    )
    return browser.OpenURL(url)  // uses github.com/pkg/browser or similar
}
```

#### Automated Build Workflow

```yaml
# .github/workflows/build-request.yml
name: Build Requested Version
on:
  issues:
    types: [labeled]

jobs:
  build:
    if: github.event.label.name == 'build-approved'
    runs-on: ubuntu-latest
    steps:
      - name: Parse request from issue
        id: parse
        run: |
          # Title format: build(python): 3.6.15 darwin-amd64
          TITLE="${{ github.event.issue.title }}"
          RUNTIME=$(echo "$TITLE" | sed -n 's/build($[^)]*$).*/\1/p')
          VERSION=$(echo "$TITLE" | sed -n 's/.*): $[^ ]*$.*/\1/p')
          PLATFORM=$(echo "$TITLE" | sed -n 's/.* $[^ ]*$$/\1/p')
          echo "runtime=$RUNTIME" >> $GITHUB_OUTPUT
          echo "version=$VERSION" >> $GITHUB_OUTPUT
          echo "platform=$PLATFORM" >> $GITHUB_OUTPUT

      - name: Validate request
        run: |
          # Verify version format is valid (prevent injection)
          # Check runtime is supported (python, ruby, node)

      - name: Trigger platform-specific build
        uses: actions/github-script@v7
        with:
          script: |
            await github.rest.actions.createWorkflowDispatch({
              owner: context.repo.owner,
              repo: context.repo.repo,
              workflow_id: 'build-${{ steps.parse.outputs.runtime }}.yml',
              ref: 'main',
              inputs: {
                version: '${{ steps.parse.outputs.version }}',
                platform: '${{ steps.parse.outputs.platform }}',
                issue_number: '${{ github.event.issue.number }}'
              }
            })
```

#### Platform-Specific Build Workflows

```yaml
# .github/workflows/build-python.yml
name: Build Python
on:
  workflow_dispatch:
    inputs:
      version:
        required: true
      platform:
        required: true
      issue_number:
        required: true

jobs:
  build:
    runs-on: ${{ inputs.platform == 'linux-amd64' && 'ubuntu-latest' ||
                 inputs.platform == 'linux-arm64' && 'ubuntu-24.04-arm' ||
                 inputs.platform == 'darwin-amd64' && 'macos-13' ||
                 inputs.platform == 'darwin-arm64' && 'macos-14' ||
                 inputs.platform == 'windows-amd64' && 'windows-latest' ||
                 'ubuntu-latest' }}
    steps:
      - name: Build Python ${{ inputs.version }}
        run: |
          # Download source from python.org
          # ./configure --prefix=... && make && make install
          # Package as tar.gz/zip

      - name: Calculate SHA256
        id: checksum
        run: |
          SHA=$(sha256sum artifact.tar.gz | cut -d' ' -f1)
          echo "sha256=$SHA" >> $GITHUB_OUTPUT

      - name: Upload to R2
        run: |
          # Upload to builds.dtvem.io/python/${{ inputs.version }}/${{ inputs.platform }}.tar.gz

      - name: Update manifest
        run: |
          # Clone repo, update data/python.json
          # Change: "platform": false → "platform": { "url": "...", "sha256": "..." }
          # Create PR

      - name: Close issue
        run: |
          gh issue close ${{ inputs.issue_number }} \
            --comment "✅ Build complete for Python ${{ inputs.version }} on ${{ inputs.platform }}!

          The binary is now available. Run \`dtvem update\` to refresh your manifest, then:
          \`\`\`
          dtvem install python ${{ inputs.version }}
          \`\`\`"
```

#### Runner Matrix

| Platform | GitHub Runner | Cost |
|----------|---------------|------|
| linux-amd64 | `ubuntu-latest` | Free |
| linux-arm64 | `ubuntu-24.04-arm` | Free |
| darwin-amd64 | `macos-13` | Free (10x minutes) |
| darwin-arm64 | `macos-14` | Free (10x minutes) |
| windows-amd64 | `windows-latest` | Free (2x minutes) |
| windows-arm64 | Cross-compile or skip | TBD |

#### Security: Label Permissions

GitHub's permission model protects against abuse:

| Label | Applied By | Who Can Add |
|-------|------------|-------------|
| `build-request` | Issue template (auto) | Auto-applied, users can't manually add |
| `build-approved` | Maintainer (manual) | **Only maintainers** |

**Why this is secure:**
- `build-request` is auto-applied by the issue template - just for organization
- Regular users **cannot** manually add or remove labels on issues
- `build-approved` is the security gate - only maintainers can add it
- Builds only trigger when `build-approved` is added
- Even if someone crafts a malicious issue, they can't trigger builds without maintainer approval

**Flow:**
```
User creates issue from template
        ↓
[build-request auto-applied by GitHub]
        ↓
User CANNOT add build-approved (no write permission)
        ↓
Maintainer reviews request
        ↓
[Maintainer adds build-approved]  ← Only maintainers can do this
        ↓
GitHub Action triggers build
```

### Phase 5: Hosting (Cloudflare R2)

```
manifests.dtvem.io          (R2 bucket)
├── python.json
├── ruby.json
└── node.json

builds.dtvem.io             (R2 bucket)
├── python/3.6.15/
│   ├── darwin-amd64.tar.gz
│   └── linux-amd64.tar.gz
├── ruby/...
└── node/...
```

---

## Key Decisions

| Topic | Decision |
|-------|----------|
| **Hosting** | Cloudflare R2 with custom domains (manifests.dtvem.io, builds.dtvem.io) |
| **Manifest updates** | Script-assisted with human review |
| **Local caching** | 24-hour TTL, `dtvem update` to force refresh |
| **Unavailable versions** | Marked as `false`, users can request builds |
| **Archive formats** | Only .zip, .tar.gz, .7z (no MSI/EXE installers) |
| **Cost** | ~$1-2/month for R2 |

---

## Success Criteria

1. `dtvem list-all <runtime>` only shows versions available for current platform
2. `dtvem install <runtime> <version>` works for any listed version
3. Downloaded binaries verified via SHA256
4. Unavailable versions show helpful message with option to request build
5. Manifest updates don't require new dtvem release

---

## References

- [python-build-standalone releases](https://github.com/astral-sh/python-build-standalone/releases)
- [ruby-builder releases](https://github.com/ruby/ruby-builder/releases)
- [Node.js dist index](https://nodejs.org/dist/index.json)
- [Cloudflare R2 pricing](https://developers.cloudflare.com/r2/pricing/)


Topic	Decision
Hosting	Cloudflare R2 with custom domains (manifests.dtvem.io, builds.dtvem.io)
Manifest updates	Script-assisted with human review
Local caching	24-hour TTL, `dtvem update` to force refresh
Unavailable versions	Marked as `false`, users can request builds
Archive formats	Only .zip, .tar.gz, .7z (no MSI/EXE installers)
Cost	~$1-2/month for R2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(providers): improve version source consistency and reliability #140

Issue #140: Version Source Consistency & Self-Hosting Strategy

Problem Summary

Current Issues by Runtime

Solution: Manifest-Based Architecture

Architecture Overview

Manifest Schema

Example: `python.json`

Platform Value Types

Design Decisions

Platform Keys

New Commands

`dtvem update`

`dtvem request`

Enhanced `dtvem install`

Implementation Plan

Phase 1: Manifest Infrastructure

Phase 2: Update Providers

Phase 3: Manifest Generation

Phase 4: Build Request System

User Flow

Automated Build Workflow

Platform-Specific Build Workflows

Runner Matrix

Security: Label Permissions

Phase 5: Hosting (Cloudflare R2)

Key Decisions

Success Criteria

References

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Runtime	Problem
Python	`ListAvailable()` scrapes python.org (all versions), `Install()` uses python-build-standalone with hardcoded date `20240814`
Ruby	Hardcoded OS versions in asset names (`ubuntu-22.04`, `macos-13`) break when runners update
Node.js	Works correctly, but should use manifest for consistency

Value	Meaning	CLI Behavior
`{ "url": "...", "sha256": "..." }`	Pre-built available	Download and install
`false`	Version exists, no pre-built	Prompt to request build
Key missing from manifest	Version doesn't exist	Error: "not a known version"

Decision	Choice	Rationale
Direct URLs	Yes	No additional lookups needed
SHA256 checksums	Yes	Security verification on download
Unavailable flag	`false`	Distinguishes "no pre-built" from "doesn't exist"
Metadata (LTS, EOL)	No (for now)	Keep manifest small; add later if needed
Archive formats only	Yes	No MSI/EXE installers; only .zip, .tar.gz, .7z

Platform	GitHub Runner	Cost
linux-amd64	`ubuntu-latest`	Free
linux-arm64	`ubuntu-24.04-arm`	Free
darwin-amd64	`macos-13`	Free (10x minutes)
darwin-arm64	`macos-14`	Free (10x minutes)
windows-amd64	`windows-latest`	Free (2x minutes)
windows-arm64	Cross-compile or skip	TBD

Label	Applied By	Who Can Add
`build-request`	Issue template (auto)	Auto-applied, users can't manually add
`build-approved`	Maintainer (manual)	Only maintainers

fix(providers): improve version source consistency and reliability #140

Description

Issue #140: Version Source Consistency & Self-Hosting Strategy

Problem Summary

Current Issues by Runtime

Solution: Manifest-Based Architecture

Architecture Overview

Manifest Schema

Example: python.json

Platform Value Types

Design Decisions

Platform Keys

New Commands

dtvem update

dtvem request

Enhanced dtvem install

Implementation Plan

Phase 1: Manifest Infrastructure

Phase 2: Update Providers

Phase 3: Manifest Generation

Phase 4: Build Request System

User Flow

Automated Build Workflow

Platform-Specific Build Workflows

Runner Matrix

Security: Label Permissions

Phase 5: Hosting (Cloudflare R2)

Key Decisions

Success Criteria

References

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Example: `python.json`

`dtvem update`

`dtvem request`

Enhanced `dtvem install`