Not able to make this work with minio! #21
Description
Hi Team,
I am trying to make this utility work with minio and having some issues. Any help or pointers to resolve this is highly appreciated.
I have installed the minio via docker on the same server where my airflow is installed. I am able to access minio properly via the publicip:9001. I then created a bucket by the name dags and able to upload files.
Now I want to sync the dags volume of the airflow container with the minio. For test purposes I am trying to run it with the following arguments but always getting error. I am having hardtime uderstanding how to fix this.
objinsync pull --once --disable-ssl --s3-endpoint http://localhost:9000 s3://dags/*.* ~/Projects/airflow/dags
{"level":"info","ts":1657540886.0054412,"caller":"objinsync/main.go:87","msg":"SENTRY_DSN not found, sentry integration disabled."}
{"level":"info","ts":1657540886.009082,"caller":"objinsync/main.go:142","msg":"Pulling from s3://dags/*.* to /home/lyridadmin/Projects/airflow/dags..."}
{"level":"info","ts":1657540886.0091188,"caller":"objinsync/main.go:126","msg":"Pull started."}
ERROR: failed to pull objects from remote store: Failed to detect AWS region: EC2MetadataRequestError: failed to get EC2 instance identity document
caused by: RequestError: send request failed
caused by: Get "http://169.254.169.254/latest/dynamic/instance-identity/document": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Tried with this as well - but getting the same error as above.
objinsync pull --once --disable-ssl --s3-endpoint http://publicip:9000 s3://dags/*.* ~/Projects/airflow/dags
Edit:
Also, how do I put the details of the accessKey and SecretKey of the Minio?