non amazonaws endpoint support (+ force PATH_STYLE)

[blajoie]

Hi,

Using the new v1.4 (current master)

It looks like when both ~/.s3cfg and ~/.aws credentials exist, then ~/.aws credentials will be used and ~/.s3cfg will be ignored.

We (@NHJohnson, @egor-dolzhenko) have a need to use the new host_base value extracted from the ~/.s3cfg (for non amazonaws endpoint), but we also do not want to force users to remove/disable ~/.aws credentials (this looks to be the only way to force htslib to always read (host_base) from ~/.s3cfg?)

This PR makes a few changes to hfile_s3.c.

1. always read from both ~/.s3cfg and ~/.aws/credentials, placing priority on ~/.aws for id/secret/token. This allows host_base to always be extracted and used. We are using a local S3 and specifying a non-amazonaws endpoint is required.
2. Following the logic outlined here (Disable dns support for buckets when "host_bucket" hostname doesn't have a %(bucket)s item in it. s3tools/s3cmd#416), we also have the need to always force PATH_STYLE over VIRTUAL_STYLE urls. Our s3 resource does not support VIRTUAL_STYLE urls. To achieve this, a user can set host_base and host_bucket to be equal, which then forces PATH_STYLE urls within htslib.

Any feedback is welcomed!

Thanks

[blajoie]

Sorry - just realized this was made to master instead of develop - will resubmit.