-
Notifications
You must be signed in to change notification settings - Fork 0
feat: add GitHub Marketplace readiness, bug fixes, and expanded test coverage #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
e283e96
change to github action template
dongjiang1989 0c0bdb6
fix(entrypoint): use bash array for args, remove redundant output blocks
dongjiang1989 7cce9f7
fix: address PR #1 review comments from coderabbitai
dongjiang1989 aa72a6e
fix(ci): harden Docker dry-run smoke test
dongjiang1989 a419df1
fix(ci): use empty config for Docker smoke test — no real tokens needed
dongjiang1989 ab1310e
fix(modelscope): use correct SDK methods for file listing
dongjiang1989 0675963
fix(ci): use --entrypoint to verify Docker image
dongjiang1989 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| name: Bug Report | ||
| description: Report a bug or unexpected behavior in HF-MS Sync | ||
| title: "[Bug]: " | ||
| labels: ["bug"] | ||
| assignees: [] | ||
|
|
||
| body: | ||
| - type: markdown | ||
| attributes: | ||
| value: | | ||
| Thanks for taking the time to report a bug! Please fill out the information below. | ||
|
|
||
| - type: textarea | ||
| id: description | ||
| attributes: | ||
| label: What happened? | ||
| description: A clear description of what the bug is. | ||
| placeholder: "When I run the sync, it fails with..." | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: expected | ||
| attributes: | ||
| label: What did you expect? | ||
| description: What should have happened instead? | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: reproduction | ||
| attributes: | ||
| label: Steps to reproduce | ||
| description: How can we reproduce this issue? | ||
| placeholder: | | ||
| 1. Create config with... | ||
| 2. Run workflow... | ||
| 3. See error... | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: config | ||
| attributes: | ||
| label: Configuration | ||
| description: | | ||
| Your `sync_config.yaml` (redact any sensitive tokens): | ||
| render: yaml | ||
|
|
||
| - type: textarea | ||
| id: logs | ||
| attributes: | ||
| label: Logs | ||
| description: | | ||
| Relevant log output from the GitHub Actions run: | ||
| render: text | ||
|
|
||
| - type: dropdown | ||
| id: direction | ||
| attributes: | ||
| label: Sync direction | ||
| options: | ||
| - hf_to_ms | ||
| - ms_to_hf | ||
| - bidirectional | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: dropdown | ||
| id: version | ||
| attributes: | ||
| label: Action version | ||
| options: | ||
| - v1 (latest) | ||
| - main branch | ||
| - specific tag (specify below) | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: input | ||
| id: runner | ||
| attributes: | ||
| label: Runner OS | ||
| placeholder: "ubuntu-latest, ubuntu-22.04, etc." | ||
|
|
||
| - type: textarea | ||
| id: additional | ||
| attributes: | ||
| label: Additional context | ||
| description: Any other information that might help. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| name: Feature Request | ||
| description: Suggest a new feature or enhancement for HF-MS Sync | ||
| title: "[Feature]: " | ||
| labels: ["enhancement"] | ||
| assignees: [] | ||
|
|
||
| body: | ||
| - type: markdown | ||
| attributes: | ||
| value: | | ||
| Have an idea for improving HF-MS Sync? We'd love to hear it! | ||
|
|
||
| - type: textarea | ||
| id: problem | ||
| attributes: | ||
| label: Problem or use case | ||
| description: What problem are you trying to solve? What's the use case? | ||
| placeholder: "I need to sync models but..." | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: solution | ||
| attributes: | ||
| label: Proposed solution | ||
| description: How do you think this should be solved? | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: alternatives | ||
| attributes: | ||
| label: Alternatives considered | ||
| description: Have you considered other approaches? | ||
|
|
||
| - type: dropdown | ||
| id: category | ||
| attributes: | ||
| label: Category | ||
| options: | ||
| - New sync direction or mode | ||
| - Performance improvement | ||
| - Better error handling | ||
| - Configuration enhancement | ||
| - Documentation | ||
| - CI/CD integration | ||
| - Other | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: additional | ||
| attributes: | ||
| label: Additional context | ||
| description: Mockups, links, or any other relevant information. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| ## Description | ||
|
|
||
| <!-- Briefly describe what this PR does and why --> | ||
|
|
||
| ## Type of change | ||
|
|
||
| - [ ] Bug fix (non-breaking change that fixes an issue) | ||
| - [ ] New feature (non-breaking change that adds functionality) | ||
| - [ ] Breaking change (fix or feature that would cause existing functionality to change) | ||
| - [ ] Documentation update | ||
| - [ ] CI/CD or infrastructure change | ||
|
|
||
| ## Changes | ||
|
|
||
| <!-- List the specific changes made --> | ||
|
|
||
| - | ||
| - | ||
| - | ||
|
|
||
| ## Testing | ||
|
|
||
| - [ ] Unit tests pass (`pytest tests/ -v`) | ||
| - [ ] Lint passes (`ruff check src/ tests/`) | ||
| - [ ] Added/updated tests for new functionality | ||
| - [ ] Tested with dry-run mode | ||
| - [ ] Tested E2E with a small model (if applicable) | ||
|
|
||
| ## Checklist | ||
|
|
||
| - [ ] My code follows the project's style guidelines | ||
| - [ ] I have performed a self-review of my code | ||
| - [ ] I have updated the documentation (if applicable) | ||
| - [ ] My changes generate no new warnings | ||
| - [ ] New and existing tests pass with my changes | ||
|
|
||
| ## Screenshots / Logs | ||
|
|
||
| <!-- If applicable, add screenshots or log output --> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,151 @@ | ||
| name: CI | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main, master] | ||
| pull_request: | ||
| branches: [main, master] | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| lint: | ||
| name: Lint | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - uses: actions/setup-python@v5 | ||
|
dongjiang1989 marked this conversation as resolved.
|
||
| with: | ||
| python-version: '3.11' | ||
|
|
||
| - name: Install ruff | ||
| run: pip install ruff | ||
|
|
||
| - name: Ruff check | ||
| run: ruff check src/ tests/ | ||
|
|
||
| - name: Ruff format check | ||
| run: ruff format --check src/ tests/ | ||
|
|
||
| test: | ||
| name: Test (Python ${{ matrix.python-version }}) | ||
| runs-on: ubuntu-latest | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| python-version: ['3.10', '3.11', '3.12'] | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
| cache: 'pip' | ||
|
|
||
| - name: Install dependencies | ||
| run: pip install -e ".[dev]" | ||
|
|
||
| - name: Run tests | ||
| run: pytest tests/ -v --tb=short --cov=src --cov-report=term-missing | ||
|
|
||
| action-validation: | ||
| name: Validate Action | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - name: Validate action.yml | ||
| run: | | ||
| echo "━━━ Checking action.yml ━━━" | ||
| # action.yml must exist | ||
| test -f action.yml || { echo "FAIL: action.yml not found"; exit 1; } | ||
|
|
||
| # Must have required fields | ||
| grep -q "^name:" action.yml || { echo "FAIL: missing name"; exit 1; } | ||
| grep -q "^description:" action.yml || { echo "FAIL: missing description"; exit 1; } | ||
| grep -q "^inputs:" action.yml || { echo "FAIL: missing inputs"; exit 1; } | ||
| grep -q "^outputs:" action.yml || { echo "FAIL: missing outputs"; exit 1; } | ||
| grep -q "using: 'docker'" action.yml || { echo "FAIL: not a docker action"; exit 1; } | ||
| grep -q "image: 'Dockerfile'" action.yml || { echo "FAIL: missing image"; exit 1; } | ||
|
|
||
| echo "action.yml validation PASSED" | ||
|
|
||
| - name: Validate Dockerfile | ||
| run: | | ||
| echo "━━━ Checking Dockerfile ━━━" | ||
| test -f Dockerfile || { echo "FAIL: Dockerfile not found"; exit 1; } | ||
| grep -q "ENTRYPOINT" Dockerfile || { echo "FAIL: missing ENTRYPOINT"; exit 1; } | ||
| grep -q "entrypoint.sh" Dockerfile || { echo "FAIL: missing entrypoint.sh"; exit 1; } | ||
|
|
||
| echo "Dockerfile validation PASSED" | ||
|
|
||
| - name: Validate entrypoint.sh | ||
| run: | | ||
| echo "━━━ Checking entrypoint.sh ━━━" | ||
| test -f entrypoint.sh || { echo "FAIL: entrypoint.sh not found"; exit 1; } | ||
| test -x entrypoint.sh || { echo "FAIL: entrypoint.sh not executable"; exit 1; } | ||
| head -1 entrypoint.sh | grep -q "^#!/bin/bash" || { echo "FAIL: missing shebang"; exit 1; } | ||
| grep -q "GITHUB_OUTPUT" entrypoint.sh || { echo "FAIL: missing GITHUB_OUTPUT"; exit 1; } | ||
|
|
||
| echo "entrypoint.sh validation PASSED" | ||
|
|
||
| - name: Validate LICENSE | ||
| run: | | ||
| test -f LICENSE || { echo "FAIL: LICENSE not found"; exit 1; } | ||
| echo "LICENSE validation PASSED" | ||
|
|
||
| docker-build: | ||
| name: Docker Build | ||
| runs-on: ubuntu-latest | ||
| needs: [lint, test, action-validation] | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - name: Build Docker image | ||
| run: docker build -t hf-ms-sync:ci . | ||
|
|
||
| - name: Verify Docker image | ||
| run: | | ||
| # Verify the image has the entrypoint (override ENTRYPOINT to run cat) | ||
| docker run --rm --entrypoint cat hf-ms-sync:ci /entrypoint.sh > /dev/null | ||
| echo "Docker image built and verified" | ||
|
|
||
| - name: Dry-run test in Docker | ||
| run: | | ||
| # Use ci_smoke_test.yaml (empty models/datasets) so the full | ||
| # dry-run flow completes without real API tokens. | ||
| docker run --rm \ | ||
| -v ${{ github.workspace }}/config:/app/config \ | ||
| -e INPUT_CONFIG=config/ci_smoke_test.yaml \ | ||
| -e INPUT_DIRECTION=hf_to_ms \ | ||
| -e INPUT_DRY_RUN=true \ | ||
| -e INPUT_LOG_LEVEL=INFO \ | ||
| -e INPUT_STATE_DIR=/tmp/sync_state \ | ||
| -e INPUT_TARGET= \ | ||
| -e INPUT_HF_TOKEN= \ | ||
| -e INPUT_MODELSCOPE_TOKEN= \ | ||
| -e GITHUB_OUTPUT=/tmp/github_output \ | ||
| -e GITHUB_STEP_SUMMARY=/tmp/github_step_summary \ | ||
| hf-ms-sync:ci 2>&1 | tee /tmp/docker_output.log | ||
|
|
||
| # Verify entrypoint started and parsed arguments correctly | ||
| grep -q "Config:.*ci_smoke_test.yaml" /tmp/docker_output.log \ | ||
| || { echo "FAIL: entrypoint did not print config summary"; exit 1; } | ||
| grep -q "Dry run:.*true" /tmp/docker_output.log \ | ||
| || { echo "FAIL: dry_run flag not propagated to entrypoint"; exit 1; } | ||
| grep -q "Direction:.*hf_to_ms" /tmp/docker_output.log \ | ||
| || { echo "FAIL: direction flag not propagated"; exit 1; } | ||
| grep -q "No sync items configured" /tmp/docker_output.log \ | ||
| || { echo "FAIL: sync engine did not reach empty-config path"; exit 1; } | ||
|
|
||
| echo "Docker smoke test PASSED" | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.