ECR DNS not resolvable

[williamsbdev]

I am attempting to use this image to proxy our ECR with our docker.example.net DNS.

When I attempt to pull an image from my Mac OS X laptop through the proxy to ECR, I see the logs are giving me the following error:

2017/06/22 13:02:43 [error] 68#68 12345678901.dkr.ecr.us-east-1.amazonaws.com could not be resolved (110: Operation timed out), client 10.X.X.X, server: _, request: "GET /v2/ HTTP/1.1", host: "docker.example.net"
10.X.X.X - - [22/Jun/2017:13:02:43 +0000] "GET /v2/ HTTP/1.1" 502 173 "-" "docker/17.06.0-ce-rc5 go/go1.8.3 git-commit b7e4173 kernel/4.9.31-moby os/linux arch/amd64 UpstreamClient(Docker-Client/17.06.0-ce-rc5 \x5C(darwin\x5C))"

So it appears that we're getting a 502 status code attempting to contact our ECR.

** note: AWS account ID and IPs have been changed

[catalinpan]

Can I know how the container is started?
Are the credentials passed as variables or as a file?
Are the ssl certificates valid for the domain used?

[williamsbdev]

1. The container was started with environment variables:

docker run -p 80:80 -e "AWS_SECRET=value" -e "AWS_KEY=value" -e "REGION=us-east-1" -v `pwd`/default.key:/etc/nginx/ssl/default.key:ro -v `pwd`/default.crt:/etc/nginx/ssl/default.crt:ro --name ecr-proxy -p 443:443 --rm catalinpan/aws-ecr-proxy

2. credentials are passed as variables
3. The SSL certificates are self signed and not valid for the domain. I am fronting the container with an ALB that does have a valid certificate for our docker.example.net domain (wildcard certificate *.example.net) then the ALB does not care about a self signed certificate and is connecting to 443.

[catalinpan]

If the container will use the default ssl certificates the -v option for certificates can be skipped. The container has it's own self signed certificates.

docker run -p 80:80 -e "AWS_SECRET=value" -e "AWS_KEY=value" -e "REGION=us-east-1" --name ecr-proxy -p 443:443 --rm catalinpan/aws-ecr-proxy

Try to point the ALB to port 80 instead of 443.

docker client-------443---->ALB-----80---->docker-proxy------443----->aws-ecr

Also from the container cmd test if you can actually connect to the AWS ECR
curl https://123456781234.dkr.ecr.us-east-1.amazonaws.com/ping
This should return Healthy if everything is ok.

[williamsbdev]

Thank you for that suggestion. I had tried it earlier in the morning but still was mapping in the SSL volumes and was getting the same error.

I just tried it again (with the docker run above that you provide) we are getting the same error as above.

[williamsbdev]

@catalinpan - How are you running this? Behind an ALB? Going to port 80 or 443?

[catalinpan]

The container was tested with valid certificates directly mounted via -v option

Client---443--->docker-proxy---443--->aws-ecr

And with a load balancer
Client---443--->loadBalancer---80--->docker-proxy---443--->aws-ecr

Did you managed to test the curl command from container command line? If the command returns healthy it means the proxy works and the problem is somewhere else.

[williamsbdev]

@catalinpan - Thank you so much for all your help! We've decided to not go the route of using the proxy. This seems like a great project and I'm sure all the problems I was having was the way that I was setting it up that was causing me issues.

[akramfares]

I had the exact same issue, especially if you're in a VPC. I changed the resolver IP to 169.254.169.253 and it started working.

More details: https://docs.aws.amazon.com/vpc/latest/userguide/vpc-dns.html

[Richard-Payne]

Opened PR to allow the resolvers to be configured:
#12