GetRepositoryByOwnerAndName fails when running natively on HTTPS

[joohoi]

• Gitea version (or commit ref): 1.7

When running gitea with configuration like:

PROTOCOL         = https
DOMAIN           = example.com
HTTP_PORT        = 443
ROOT_URL         = https://example.com/

ENABLE_LETSENCRYPT = true

And pushing to a repository (in my case over SSH), gitea errors with message:

[T] GetRepositoryByOwnerAndName: https://localhost:443/api/internal/repo/username/reponame
[...io/gitea/cmd/serv.go:194 runServ()] [F] Failed to get repository: Get https://localhost:443/api/internal/repo/username/reponame: remote error: tls: internal error

This happens as localhost naturally isn't a valid domain name in the certificate provided by the HTTP server component.

Why is localhost selected for a domain then? In modules/setting/setting.go:824-832 the variable HTTP_ADDR : (the address to bind the web service to) is used, and if it's set to 0.0.0.0, "localhost" will be used as domain for the internal HTTP API call.

This issue can be alleviated by setting variable LOCAL_ROOT_URL to include the FQDN that the x509 certificate is issued for.

However the documentation regarding this issue isn't too clear:

LOCAL_ROOT_URL: %(PROTOCOL)s://%(HTTP_ADDR)s:%(HTTP_PORT)s/: Local (DMZ) URL for Gitea workers (such as SSH update) accessing web service. In most cases you do not need to change the default value. Alter it only if your SSH server node is not the same as HTTP node. Do not set this variable if PROTOCOL is set to unix.

My suggestion is to clarify the documentation, and to either make the local API URL generation more sophisticated to include this use case, or to use

&http.Transport{
    TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}

to omit the certificate verification when accessing a localhost URL.

[zeripath]

Thank you @joohoi for this detailed bug report. I haven't personally duplicated your bug but from your information provided I'm going to set the reviewed/confirmed tag.

I'm not sure which approach to take.

Clearly this can be fixed for the majority by using the configuration options that are already there - so fixing the documentation is an option. However that's slightly annoying and goes against the principle that Gitea should be painless.

On the other hand, it could be very difficult to assure that the remote URL is definitely reachable from the localhost. It's certainly possible that a device may have an external domain name that it itself cannot connect to.

I also think setting Gitea to disregard TLS errors is probably the wrong thing to do. We already attempt to do this.

I wonder is there any way to provide a preresolved ip i.e. http_addr or 127.0.0.1 and the fqdn derived from app_url to the request?

[joohoi]

I also think setting Gitea to disregard TLS errors is probably the wrong thing to do.

My idea for disregarding TLS errors was strictly for connections made over TLS to localhost. The only case where localhost certificate could be verified, would be the case where user has explicitly created their own CA certificate that has been added to system certificate store, trusted and a certificate that's used by Gitea has been signed with this CA cert. It's an extremely rare situation.

To keep Gitea painless, I still do think that making an exception for TLS cetificate validation if the host we're connecting is localhost is a good approach here.

[zeripath]

OK. So I've just looked at the code base, all requests using setting.LocalUrl send their requests to modules/private/internal.go

func newInternalRequest(url, method string) *httplib.Request {
	req := newRequest(url, method).SetTLSClientConfig(&tls.Config{
		InsecureSkipVerify: true,
	})
	if setting.Protocol == setting.UnixSocket {
		req.SetTransport(&http.Transport{
			Dial: func(_, _ string) (net.Conn, error) {
				return net.Dial("unix", setting.HTTPAddr)
			},
		})
	}
	return req
}

So just confirm for me that the problem is definitely still on master and v1.7? It seems like we've already decided to skip verification on internal lookup but we're just not doing it properly.

It looks like a quick PR that changes that SetTLSClientConfig to an appropriately written SetTransport as per your suggestion would work. I'd happily ACK it if you put one up that does that.

[joohoi]

I can confirm that the issue still exists in 1.7 and master branch.

I dug into the code, and InsecureSkipVerify was indeed set correctly. Turned out that the TLS error was caused by the upstream golang.org/x/crypto/acme/autocert package that handles the Let's Encrypt certificate automation. Autocert terminates the handshake if the requested domain (in SNII) isn't whitelisted (and hence it can't get a certificate for it).

This was a pretty straightforward fix however, and no new functionality was required.