Skip to content

Add a filter parameter for larger repositories#142

Closed
CGA1123 wants to merge 1 commit into
bufbuild:mainfrom
freshaengineering:cga1123/filter
Closed

Add a filter parameter for larger repositories#142
CGA1123 wants to merge 1 commit into
bufbuild:mainfrom
freshaengineering:cga1123/filter

Conversation

@CGA1123

@CGA1123 CGA1123 commented Mar 24, 2025

Copy link
Copy Markdown

This will generally always be tree:0 in order to create a treeless clone and sparse checkout of only the input directory.

This improves performance of breaking change checks significantly for larger repositories.

This option was added in v1.50.0 which was released only a couple of months ago.

I think it could be nice to automatically add this filter by default or perhaps just to set the default value to tree:0?

This would break for anyone using an older version of the CLI.

Not sure of a way that this could be introduced in a backwards compatible manner, maybe we can just run buf --version and parse that? Or have installer.ts pass through the installed version number?

This will generally always be `tree:0` in order to create a treeless
clone and sparse checkout of only the input directory.

This improves performance of breaking change checks significantly for
larger repositories.
@CLAassistant

CLAassistant commented Mar 24, 2025

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@emcfarlane

emcfarlane commented Mar 24, 2025

Copy link
Copy Markdown
Collaborator

Hey @CGA1123 thanks for putting this together! We do have a version restriction here, which could then be used to fix the default case. Although very unlikely, we should enforce upgrading of buf to 1.50.1 for security reasons too: https://github.com/bufbuild/buf/releases/tag/v1.50.1 . So this version restriction would serve two purposes, and remove the need for the explicit filter parameter.

@CGA1123

CGA1123 commented Mar 24, 2025

Copy link
Copy Markdown
Author

Sounds great.

So to confirm, bump the required version to >= 1.50.1 and make filter=tree:0 the default?

CGA1123 added a commit to CGA1123/buf that referenced this pull request Mar 24, 2025
When using `filter` `buf` will fail to run `git checkout` as the
credentials aren't passed to `git`.

This results in failures such as:

```
Failure: could not clone https://github.com/foo/bar.git: exit status 128
fatal: could not read Username for 'https://github.com': No such device or address
fatal: could not fetch <SHA> from promisor remote
```

I noticed this while testing bufbuild/buf-action#142 which relies on
overriding `credential.helper` for authentication in GitHub Actions.
@CGA1123

CGA1123 commented Mar 24, 2025

Copy link
Copy Markdown
Author

Relatedly, I think in order to make this transition smooth, something like bufbuild/buf#3707 will need to be merged in.

Currently, using filter will fail when running git checkout because no auth credentials are passed to that step.

@emcfarlane

Copy link
Copy Markdown
Collaborator

Thanks for getting bufbuild/buf#3707 fixed! We talked through adding the version limit internally and decided that might be too big a restriction for users. The approach we are looking for is to add a check to see if buf version supports the filter and add it by default. Where a user sets the breaking_against parameter, the filter should not be added. Let me know if you want to look into this. Otherwise converting to an issue I can help pick up.

@CGA1123

CGA1123 commented Mar 25, 2025

Copy link
Copy Markdown
Author

That makes sense 👍 I've created #147 to help track this.

Thanks for your time 🙇

@CGA1123 CGA1123 closed this Mar 25, 2025
@CGA1123 CGA1123 deleted the cga1123/filter branch March 25, 2025 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants