Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for sparse checkouts #1369

Merged
merged 4 commits into from
Jun 9, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Add support for sparse checkouts
  • Loading branch information
dfdez authored and dscho committed Jun 3, 2023
commit 9f59c817cf084aca21b15575c800c5e747674b61
13 changes: 13 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,19 @@ jobs:
shell: bash
run: __test__/verify-side-by-side.sh

# Sparse checkout
- name: Sparse checkout
uses: ./
with:
sparse-checkout: |
__test__
.github
dist
path: sparse-checkout

- name: Verify sparse checkout
run: __test__/verify-sparse-checkout.sh

# LFS
- name: Checkout LFS
uses: ./
Expand Down
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,11 @@ When Git 2.18 or higher is not in your PATH, falls back to the REST API to downl
# Default: true
clean: ''

# Do a sparse checkout on given patterns. Each pattern should be separated with
# new lines
# Default: null
sparse-checkout: ''

# Number of commits to fetch. 0 indicates all history for all branches and tags.
# Default: 1
fetch-depth: ''
Expand Down Expand Up @@ -106,6 +111,8 @@ When Git 2.18 or higher is not in your PATH, falls back to the REST API to downl

# Scenarios

- [Fetch only the root files](#Fetch-only-the-root-files)
- [Fetch only the root files and `.github` and `src` folder](#Fetch-only-the-root-files-and-github-and-src-folder)
- [Fetch all history for all tags and branches](#Fetch-all-history-for-all-tags-and-branches)
- [Checkout a different branch](#Checkout-a-different-branch)
- [Checkout HEAD^](#Checkout-HEAD)
Expand All @@ -116,6 +123,24 @@ When Git 2.18 or higher is not in your PATH, falls back to the REST API to downl
- [Checkout pull request on closed event](#Checkout-pull-request-on-closed-event)
- [Push a commit using the built-in token](#Push-a-commit-using-the-built-in-token)

## Fetch only the root files

```yaml
- uses: actions/checkout@v3
with:
sparse-checkout: .
```

## Fetch only the root files and `.github` and `src` folder

```yaml
- uses: actions/checkout@v3
with:
sparse-checkout: |
.github
src
```

## Fetch all history for all tags and branches

```yaml
Expand Down
2 changes: 2 additions & 0 deletions __test__/git-auth-helper.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -727,6 +727,7 @@ async function setup(testName: string): Promise<void> {
branchDelete: jest.fn(),
branchExists: jest.fn(),
branchList: jest.fn(),
sparseCheckout: jest.fn(),
checkout: jest.fn(),
checkoutDetach: jest.fn(),
config: jest.fn(
Expand Down Expand Up @@ -800,6 +801,7 @@ async function setup(testName: string): Promise<void> {
authToken: 'some auth token',
clean: true,
commit: '',
sparseCheckout: [],
fetchDepth: 1,
lfs: false,
submodules: false,
Expand Down
1 change: 1 addition & 0 deletions __test__/git-directory-helper.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -462,6 +462,7 @@ async function setup(testName: string): Promise<void> {
branchList: jest.fn(async () => {
return []
}),
sparseCheckout: jest.fn(),
checkout: jest.fn(),
checkoutDetach: jest.fn(),
config: jest.fn(),
Expand Down
1 change: 1 addition & 0 deletions __test__/input-helper.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ describe('input-helper tests', () => {
expect(settings.clean).toBe(true)
expect(settings.commit).toBeTruthy()
expect(settings.commit).toBe('1234567890123456789012345678901234567890')
expect(settings.sparseCheckout).toBe(undefined)
expect(settings.fetchDepth).toBe(1)
expect(settings.lfs).toBe(false)
expect(settings.ref).toBe('refs/heads/some-ref')
Expand Down
63 changes: 63 additions & 0 deletions __test__/verify-sparse-checkout.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#!/bin/sh

# Verify .git folder
if [ ! -d "./sparse-checkout/.git" ]; then
echo "Expected ./sparse-checkout/.git folder to exist"
exit 1
fi

# Verify sparse-checkout
cd sparse-checkout

SPARSE=$(git sparse-checkout list)

if [ "$?" != "0" ]; then
echo "Failed to validate sparse-checkout"
exit 1
fi

# Check that sparse-checkout list is not empty
if [ -z "$SPARSE" ]; then
echo "Expected sparse-checkout list to not be empty"
exit 1
fi

# Check that all folders of the sparse checkout exist
for pattern in $SPARSE
do
if [ ! -d "$pattern" ]; then
echo "Expected directory '$pattern' to exist"
exit 1
fi
done

checkSparse () {
if [ ! -d "./$1" ]; then
echo "Expected directory '$1' to exist"
exit 1
fi

for file in $(git ls-tree -r --name-only HEAD $1)
do
if [ ! -f "$file" ]; then
echo "Expected file '$file' to exist"
exit 1
fi
done
}

# Check that all folders and their children have been checked out
checkSparse __test__
checkSparse .github
checkSparse dist

# Check that only sparse-checkout folders have been checked out
for pattern in $(git ls-tree --name-only HEAD)
do
if [ -d "$pattern" ]; then
if [[ "$pattern" != "__test__" && "$pattern" != ".github" && "$pattern" != "dist" ]]; then
echo "Expected directory '$pattern' to not exist"
exit 1
fi
fi
done
5 changes: 5 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,11 @@ inputs:
clean:
description: 'Whether to execute `git clean -ffdx && git reset --hard HEAD` before fetching'
default: true
sparse-checkout:
description: >
Do a sparse checkout on given patterns.
Each pattern should be separated with new lines
default: null
fetch-depth:
description: 'Number of commits to fetch. 0 indicates all history for all branches and tags.'
default: 1
Expand Down
36 changes: 30 additions & 6 deletions dist/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -574,6 +574,11 @@ class GitCommandManager {
return result;
});
}
sparseCheckout(sparseCheckout) {
return __awaiter(this, void 0, void 0, function* () {
yield this.execGit(['sparse-checkout', 'set', ...sparseCheckout]);
});
}
checkout(ref, startPoint) {
return __awaiter(this, void 0, void 0, function* () {
const args = ['checkout', '--progress', '--force'];
Expand Down Expand Up @@ -615,15 +620,18 @@ class GitCommandManager {
return output.exitCode === 0;
});
}
fetch(refSpec, fetchDepth) {
fetch(refSpec, options) {
return __awaiter(this, void 0, void 0, function* () {
const args = ['-c', 'protocol.version=2', 'fetch'];
if (!refSpec.some(x => x === refHelper.tagsRefSpec)) {
args.push('--no-tags');
}
args.push('--prune', '--progress', '--no-recurse-submodules');
if (fetchDepth && fetchDepth > 0) {
args.push(`--depth=${fetchDepth}`);
if (options.filter) {
args.push(`--filter=${options.filter}`);
}
if (options.fetchDepth && options.fetchDepth > 0) {
args.push(`--depth=${options.fetchDepth}`);
}
else if (fshelper.fileExistsSync(path.join(this.workingDirectory, '.git', 'shallow'))) {
args.push('--unshallow');
Expand Down Expand Up @@ -1210,20 +1218,24 @@ function getSource(settings) {
}
// Fetch
core.startGroup('Fetching the repository');
const fetchOptions = {};
if (settings.sparseCheckout)
fetchOptions.filter = 'blob:none';
if (settings.fetchDepth <= 0) {
// Fetch all branches and tags
let refSpec = refHelper.getRefSpecForAllHistory(settings.ref, settings.commit);
yield git.fetch(refSpec);
yield git.fetch(refSpec, fetchOptions);
// When all history is fetched, the ref we're interested in may have moved to a different
// commit (push or force push). If so, fetch again with a targeted refspec.
if (!(yield refHelper.testRef(git, settings.ref, settings.commit))) {
refSpec = refHelper.getRefSpec(settings.ref, settings.commit);
yield git.fetch(refSpec);
yield git.fetch(refSpec, fetchOptions);
}
}
else {
fetchOptions.fetchDepth = settings.fetchDepth;
const refSpec = refHelper.getRefSpec(settings.ref, settings.commit);
yield git.fetch(refSpec, settings.fetchDepth);
yield git.fetch(refSpec, fetchOptions);
}
core.endGroup();
// Checkout info
Expand All @@ -1238,6 +1250,12 @@ function getSource(settings) {
yield git.lfsFetch(checkoutInfo.startPoint || checkoutInfo.ref);
core.endGroup();
}
// Sparse checkout
if (settings.sparseCheckout) {
core.startGroup('Setting up sparse checkout');
yield git.sparseCheckout(settings.sparseCheckout);
core.endGroup();
}
// Checkout
core.startGroup('Checking out the ref');
yield git.checkout(checkoutInfo.ref, checkoutInfo.startPoint);
Expand Down Expand Up @@ -1673,6 +1691,12 @@ function getInputs() {
// Clean
result.clean = (core.getInput('clean') || 'true').toUpperCase() === 'TRUE';
core.debug(`clean = ${result.clean}`);
// Sparse checkout
const sparseCheckout = core.getMultilineInput('sparse-checkout');
if (sparseCheckout.length) {
result.sparseCheckout = sparseCheckout;
core.debug(`sparse checkout = ${result.sparseCheckout}`);
}
// Fetch depth
result.fetchDepth = Math.floor(Number(core.getInput('fetch-depth') || '1'));
if (isNaN(result.fetchDepth) || result.fetchDepth < 0) {
Expand Down
27 changes: 23 additions & 4 deletions src/git-command-manager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ export interface IGitCommandManager {
branchDelete(remote: boolean, branch: string): Promise<void>
branchExists(remote: boolean, pattern: string): Promise<boolean>
branchList(remote: boolean): Promise<string[]>
sparseCheckout(sparseCheckout: string[]): Promise<void>
checkout(ref: string, startPoint: string): Promise<void>
checkoutDetach(): Promise<void>
config(
Expand All @@ -25,7 +26,13 @@ export interface IGitCommandManager {
add?: boolean
): Promise<void>
configExists(configKey: string, globalConfig?: boolean): Promise<boolean>
fetch(refSpec: string[], fetchDepth?: number): Promise<void>
fetch(
refSpec: string[],
options: {
filter?: string
fetchDepth?: number
}
): Promise<void>
getDefaultBranch(repositoryUrl: string): Promise<string>
getWorkingDirectory(): string
init(): Promise<void>
Expand Down Expand Up @@ -154,6 +161,10 @@ class GitCommandManager {
return result
}

async sparseCheckout(sparseCheckout: string[]): Promise<void> {
await this.execGit(['sparse-checkout', 'set', ...sparseCheckout])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to have extra check to make sure git is greater than certain version that support sparse-checkout?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @TingluoHuang git sparse-checkout is supported since the git version 2.26.0 and the current version on actions/checkout of git is 2.40.1 so I don't think this is something necessary 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current Git version on hosted runners is v2.40.1. But actions/checkout also supports self-hosted runners where users are often not at liberty to bump the Git version, e.g. because they're stuck on old Ubuntu LTS versions which simply refuse to move the needle.

I added c4602b4 to address this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uu nice! I understand perfect ✨

}

async checkout(ref: string, startPoint: string): Promise<void> {
const args = ['checkout', '--progress', '--force']
if (startPoint) {
Expand Down Expand Up @@ -202,15 +213,23 @@ class GitCommandManager {
return output.exitCode === 0
}

async fetch(refSpec: string[], fetchDepth?: number): Promise<void> {
async fetch(
refSpec: string[],
options: {filter?: string; fetchDepth?: number}
): Promise<void> {
const args = ['-c', 'protocol.version=2', 'fetch']
if (!refSpec.some(x => x === refHelper.tagsRefSpec)) {
args.push('--no-tags')
}

args.push('--prune', '--progress', '--no-recurse-submodules')
if (fetchDepth && fetchDepth > 0) {
args.push(`--depth=${fetchDepth}`)

if (options.filter) {
args.push(`--filter=${options.filter}`)
}

if (options.fetchDepth && options.fetchDepth > 0) {
args.push(`--depth=${options.fetchDepth}`)
} else if (
fshelper.fileExistsSync(
path.join(this.workingDirectory, '.git', 'shallow')
Expand Down
16 changes: 13 additions & 3 deletions src/git-source-provider.ts
Original file line number Diff line number Diff line change
Expand Up @@ -153,23 +153,26 @@ export async function getSource(settings: IGitSourceSettings): Promise<void> {

// Fetch
core.startGroup('Fetching the repository')
const fetchOptions: {filter?: string; fetchDepth?: number} = {}
if (settings.sparseCheckout) fetchOptions.filter = 'blob:none'
if (settings.fetchDepth <= 0) {
// Fetch all branches and tags
let refSpec = refHelper.getRefSpecForAllHistory(
settings.ref,
settings.commit
)
await git.fetch(refSpec)
await git.fetch(refSpec, fetchOptions)

// When all history is fetched, the ref we're interested in may have moved to a different
// commit (push or force push). If so, fetch again with a targeted refspec.
if (!(await refHelper.testRef(git, settings.ref, settings.commit))) {
refSpec = refHelper.getRefSpec(settings.ref, settings.commit)
await git.fetch(refSpec)
await git.fetch(refSpec, fetchOptions)
}
} else {
fetchOptions.fetchDepth = settings.fetchDepth
const refSpec = refHelper.getRefSpec(settings.ref, settings.commit)
await git.fetch(refSpec, settings.fetchDepth)
await git.fetch(refSpec, fetchOptions)
}
core.endGroup()

Expand All @@ -191,6 +194,13 @@ export async function getSource(settings: IGitSourceSettings): Promise<void> {
core.endGroup()
}

// Sparse checkout
if (settings.sparseCheckout) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what will happen if sparseCheckout is an empty array?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That shouldn't happen, you can see that the length is checked when setting up the sparseCheckout variable

You can find this here

core.startGroup('Setting up sparse checkout')
await git.sparseCheckout(settings.sparseCheckout)
core.endGroup()
}

// Checkout
core.startGroup('Checking out the ref')
await git.checkout(checkoutInfo.ref, checkoutInfo.startPoint)
Expand Down
5 changes: 5 additions & 0 deletions src/git-source-settings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,11 @@ export interface IGitSourceSettings {
*/
clean: boolean

/**
* The array of folders to make the sparse checkout
*/
sparseCheckout: string[]

/**
* The depth when fetching
*/
Expand Down
7 changes: 7 additions & 0 deletions src/input-helper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,13 @@ export async function getInputs(): Promise<IGitSourceSettings> {
result.clean = (core.getInput('clean') || 'true').toUpperCase() === 'TRUE'
core.debug(`clean = ${result.clean}`)

// Sparse checkout
const sparseCheckout = core.getMultilineInput('sparse-checkout')
if (sparseCheckout.length) {
result.sparseCheckout = sparseCheckout
core.debug(`sparse checkout = ${result.sparseCheckout}`)
}

// Fetch depth
result.fetchDepth = Math.floor(Number(core.getInput('fetch-depth') || '1'))
if (isNaN(result.fetchDepth) || result.fetchDepth < 0) {
Expand Down