Skip to content

Fix #6960 - LFS OID urls uses unusual content-type header #7005

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

zeripath
Copy link
Contributor

@zeripath zeripath commented May 20, 2019

This PR drops the unusual content-type requirement of "application/vnd.git-lfs" for repo.git/info/lfs/:oid, and just relies on the content-type not being "application/vnd.git-lfs+json".

Fixes #6960

Of interest, I have been unable to find a definite use for the meta version of this url.

@codecov-io
Copy link

codecov-io commented May 20, 2019

Codecov Report

Merging #7005 into master will increase coverage by 0.04%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #7005      +/-   ##
=========================================
+ Coverage   41.45%   41.5%   +0.04%     
=========================================
  Files         441     440       -1     
  Lines       59699   59450     -249     
=========================================
- Hits        24747   24673      -74     
+ Misses      31727   31560     -167     
+ Partials     3225    3217       -8
Impacted Files Coverage Δ
modules/lfs/server.go 45% <100%> (+0.58%) ⬆️
models/unit.go 62.16% <0%> (-5.41%) ⬇️
models/webhook.go 56.73% <0%> (-3.03%) ⬇️
models/repo_list.go 72.08% <0%> (-1.02%) ⬇️
models/repo_indexer.go 67.79% <0%> (-0.85%) ⬇️
models/gpg_key.go 55.83% <0%> (-0.84%) ⬇️
modules/httplib/httplib.go
modules/log/colors_router.go 87.5% <0%> (+4.16%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 02542a2...63f21d5. Read the comment docs.

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label May 20, 2019
@lafriks lafriks added the type/refactoring Existing code has been cleaned up. There should be no new functionality. label May 21, 2019
@@ -415,7 +413,7 @@ func Represent(rv *RequestVars, meta *models.LFSMetaObject, download, upload boo
func ContentMatcher(r macaron.Request) bool {
mediaParts := strings.Split(r.Header.Get("Accept"), ";")
mt := mediaParts[0]
return mt == contentMediaType
return mt != metaMediaType
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does it checks not equal now? imho it should still be:

Suggested change
return mt != metaMediaType
return mt == metaMediaType

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was previously checking if mt == "application/vnd.git-lfs"

now it checks if mt != "application/vnd.git-lfs+json"

Copy link
Member

@lafriks lafriks May 21, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really correct then as any invalid type would also match?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that the spec says much at all about this - what it doesn't do is specify that it must be the current content type. @slonopotamus any thoughts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe accepting any content type for blob is OK. It allows downloading blobs with just a browser. You already have "filename" parameter that skips content-type check. What worries me is verify url. Does it still have ContentMatcher? If yes, this goes against the spec

Copy link
Contributor

@slonopotamus slonopotamus May 22, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You want to say that you pass tests without installing any Accept header for verify and having ContentMatcher in it? Oh my. I start to suspect that git-lfs does not install Accept header for verify unless told by server to do so, even though spec says that verify has to be +json.

We may possibly want to wait for reaction from LFS devs to git-lfs/lfs-test-server#85.

Copy link
Contributor Author

@zeripath zeripath May 22, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's ok I've just pushed changes to the branch as above.

Now it's probably the case that our tests are insufficient. For example in #6999 and #6961 I noticed that SSH locks weren't working despite tests (written by myself I should say - but when I understood LFS even less than I do now) suggesting that they were working. I don't think we attempt an LFS checkout either.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm interestingly that breaks the media endpoint...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reported git-lfs/git-lfs#3662 to git-lfs so they start to send Accept: application/vnd.git-lfs+json for /verify URL.

Copy link
Contributor

@slonopotamus slonopotamus May 22, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suddenly understood that it is only me using "/verify URL" term. This is because in git-as-svn we use completely different URLs for verification and upload/download, thus avoiding the need to determine what to do based on Accept HTTP header.

@slonopotamus
Copy link
Contributor

slonopotamus commented May 22, 2019

I think I know how to fix this mess.

  1. Only add "Accept: +json" for /verify endpoint. Actually, this shouldn't be needed at all, but seems like git-lfs has a bug and doesn't do it itself. I'll report it to them today.

  2. Drop ContentMatcher checks completely.

  3. Do not add any Accept header for other links (upload/download)

also ensure header map is unique for verify
@slonopotamus
Copy link
Contributor

Okay, I've created #7015 that does exactly what I described.

@zeripath zeripath closed this May 23, 2019
@zeripath zeripath deleted the fix-#6960-drop-unusual-header branch May 28, 2019 12:50
@go-gitea go-gitea locked and limited conversation to collaborators Nov 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. type/refactoring Existing code has been cleaned up. There should be no new functionality.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Gitea attempts to replace Accept header for LFS requests
5 participants