Skip to content

Commit

Permalink
Allow blank port for minio client to work with S3 (kubeflow#2996)
Browse files Browse the repository at this point in the history
Background
---
As of current, supplying a minioServiceHost of `s3.amazonaws.com` with
any port incurs an error.

Per the results of:

```
testBucket := func(hostPort string) error {
	client, _ := minio.New(hostPort, accessKey, secretKey, false)
	_, err := client.BucketExists(bucket)
	return err
}

for _, endpoint := range []string{
	"s3.amazonaws.com:443",
	"s3.amazonaws.com:80",
	"s3.amazonaws.com:",
	"s3.amazonaws.com",
} {
	fmt.Printf("Endpoint: %s, Error: %v\n", endpoint, testBucket(endpoint))
}
```

```
Endpoint: s3.amazonaws.com:443, Error: Get http://s3.amazonaws.com:443/kflow-test/?location=: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x15\x00\x00\x00\x02\x01\x00"
Endpoint: s3.amazonaws.com:80, Error: Head http://s3.amazonaws.com/kflow-test/: 301 response missing Location header
Endpoint: s3.amazonaws.com:, Error: Head http://s3.amazonaws.com/kflow-test/: 301 response missing Location header
Endpoint: s3.amazonaws.com, Error: <nil>
```

Only the connection without a port specified works.

This change allows passing a blank port to support this.
  • Loading branch information
pavanaiyar authored and Jeffwan committed Dec 9, 2020
1 parent c7f704d commit 879fa34
Showing 1 changed file with 11 additions and 1 deletion.
12 changes: 11 additions & 1 deletion backend/src/apiserver/client/minio.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import (

func CreateMinioClient(minioServiceHost string, minioServicePort string,
accessKey string, secretKey string) (*minio.Client, error) {
minioClient, err := minio.New(fmt.Sprintf("%s:%s", minioServiceHost, minioServicePort),
minioClient, err := minio.New(joinHostPort(minioServiceHost, minioServicePort),
accessKey, secretKey, false /* Secure connection */)
if err != nil {
return nil, errors.Wrapf(err, "Error while creating minio client: %+v", err)
Expand Down Expand Up @@ -54,3 +54,13 @@ func CreateMinioClientOrFatal(minioServiceHost string, minioServicePort string,
}
return minioClient
}

// joinHostPort combines host and port into a network address of the form "host:port".
//
// An empty port value results in "host" instead of "host:" (which net.JoinHostPort would return)
func joinHostPort(host, port string) string {
if port == "" {
return host
}
return fmt.Sprintf("%s:%s", host, port)
}

0 comments on commit 879fa34

Please sign in to comment.