Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have the zookeeper backend use the host port for the service paths, allow publishing services if the base service path already exists, and allow publishing into the root of zookeeper. #367

Merged
merged 1 commit into from
Apr 19, 2016

Conversation

jmeichle
Copy link
Contributor

This PR solves three problems with the initially committed zookeeper backend.

Problem 1: Znode name for a registrator service is published as the PrivatePort not the PublicPort (or service.Port). This was a mistake when originally writing this and is a pretty bad flaw. The reason this is severe is because we cannot run more than a single container with the same service name and private port, since they cannot both be published in zookeeper.

A quick example, running this ubuntu container:

docker run -p 80 -it ubuntu:14.04 sleep 36000

with registrator configured for zookeeper://zk:2181/services publishes this in zookeeper:

--/services/ubuntu has 1 children
----/services/ubuntu/80 data: {"Name":"ubuntu","IP":"","PublicPort":32771,"PrivatePort":80,"ContainerID":"a9147e4a0901","Tags":[],"Attrs":{}}

The path is the base chroot (/services), the service name (ubuntu), then the "PrivatePort" (which is service.Origin.ExposedPort in registrator). This needs to be service.Port.

Problem 2: registrator cannot publish into zookeeper without a chroot style base path. When running this docker-compose.yml:

zookeeper:
  image: garland/zookeeper
  ports:
   - "2181:2181"
   - "2888:2888"
   - "3888:3888"
  environment:
   - SERVCE_2181_name=zookeeper
   - SERVCE_2888_name=zookeeper
   - SERVCE_3888_name=zookeeper
registrator:
  image: gliderlabs/registrator:master
  volumes:
   - /var/run/docker.sock:/tmp/docker.sock
  command: zookeeper://zk:2181/
  links:
   - zookeeper:zk
ubuntu:
  image: ubuntu:14.04
  ports:
   - "80"
  environment:
   - SERVICE_80_name=ubuntu
  command: sleep 36000

I get this error

registrator_1 | 2016/03/12 21:15:19 Listening for Docker events ...
registrator_1 | 2016/03/12 21:15:19 Syncing services on 3 containers
zookeeper_1   | 2016-03-12 21:15:19,122 [myid:] - INFO  [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@358] - Invalid path //ubuntu with session 0x1536caef2f60000
zookeeper_1   | 2016-03-12 21:15:19,123 [myid:] - INFO  [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x1536caef2f60000 type:create cxid:0x4 zxid:0x2 txntype:-1 reqpath:n/a Error Path://ubuntu Error:KeeperErrorCode = BadArguments for //ubuntu
registrator_1 | 2016/03/12 21:15:19 zookeeper: failed to create base service node:  zk: unknown error
registrator_1 | 2016/03/12 21:15:19 added: afe6f01c1095 d3aafb922ff2:zkservicescompose_ubuntu_1:80
registrator_1 | 2016/03/12 21:15:19 ignored: d3aafb922ff2 no published ports

Registrator is attempting create the service node at //ubuntu instead of /ubuntu and fails.

Problem 3: The only the first container for a service name is published

This is due to the zookeeper plugin checking if the 'base path' in zookeeper exists (chroot + service name). If it didn't exist, zookeeper creates the node, and then continues to create the service port entries in zookeeper under the service name path. However, the nested logic prevented any publishing from occuring if the base service name path existed. This is incorrect because if the path for the service name exists, registrator should still attempt to publish services for this name.

This PR addresses all three problems. Quick example here for the above docker-compose.yml which was tweaked to run a second ubuntu container (with the same SERVICE_80_name) and modified for a test build of registrator that this PR is for:

jmeichle@jkm-ThinkPad-T440s:~/registrator$ zk-tool list /ubuntu
--/ubuntu data: 
--/ubuntu has 2 children
----/ubuntu/32785 data: {"Name":"ubuntu","IP":"","PublicPort":32785,"PrivatePort":80,"ContainerID":"911beae425bc","Tags":[],"Attrs":{}}
----/ubuntu/32784 data: {"Name":"ubuntu","IP":"","PublicPort":32784,"PrivatePort":80,"ContainerID":"c5c212cb508c","Tags":[],"Attrs":{}}

The service name is now published correctly into the root of zookeeper (/ubuntu) the service znode's name is now the PublicPort instead of the PrivatePort, and we see the services from both containers sharing the name 'ubuntu'. This enables me to run multiple containers for the same service name, after fixing the fact the second container wouldn't publish at all, and the path would have a collision since it used the PrivatePort.

@jmeichle
Copy link
Contributor Author

@progrium ping

exists, _, err := r.client.Exists(r.path + "/" + service.Name)
basePath := r.path + "/" + service.Name
if (r.path == "/") {
basePath = r.path + service.Name
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This addresses problem 2

…llow publishing services if the base service path already exists, and allow publishing into the root of zookeeper.
@jmeichle jmeichle mentioned this pull request Mar 14, 2016
Merged
@mattatcha mattatcha merged commit 04f52e0 into gliderlabs:master Apr 19, 2016
mattatcha added a commit that referenced this pull request Apr 19, 2016
* fix where providing a SERVICE_NAME for a container with multiple ports exposed would cause services to overwrite each other

* Synchornize etcd cluster in registrator on service registration

* note on docker hub tags

* link to boot2docker

* analytics

* Default to tcp for PortType if not provided

* Allow DEV_RUN_OPTS to be used when calling make dev

* Add new version checker

Checks for new versions with the "usage" service and automatically displays a
standard version message when the "--version" flag is passed.

* Adding retries to backend service in the startup

Signed-off-by: Marcelo Salazar R <chelosalazar@gmail.com>

* Added retry parameters documentation

Signed-off-by: Marcelo Salazar R <chelosalazar@gmail.com>

* Upgrade to alpine:3.2 and go 1.4

go 1.4 is now required (miekg/dns#197)

* Refactor bridge for better testability

bridge.New no longer attempts to ping an adapter, caller must now use the bridge Ping method.

A few simple tests have been added to the bridge pkg

* Removed unused attributes

* prebump

* Adding documentation link

* update wording for Weave product family

* updating documentation & CHANGELOG

* Fix specific port names not overriding port suffix

* Actually check metadata from port. Fix ENV variable order dependency

* Use exit status to determine if container was killed

Instead of using the "kill" and "stop" events, this uses the exit status to
check whether the container was terminated via a signal. This will be more
reliable since the "kill" event can also be sent for non-fatal signals such as
SIGHUP.

Fixes #248

* Fix releases link in README

* Align SPONSORS text

* Add more detailed usage regarding options placement

Go's "flag" module only parses options up until the first non-option argument,
so additional arguments are left unparsed in "flag.Args()". We only expect one
argument, but additional arguments were ignored, leading to some confusion
about options that were ignored.

This updates the Usage() message with the syntax showing the options before the
registry URI, as well as more detail if the registry argument is missing, or
additional arguments are found.

* Cleanup dangling services

When a service was previously registered into the service registry
and registrator exits without unregistering, registrator now queries
the backend to see which services were registered, and checks against
it's internal list to determine which should be unregistered.

* Support for Docker multi host networking

When using the Docker multi-host networking, IPAddress under NetworkSettings is set to an empty string and the container IP can be retrieved from NetworkSettings.Networks.

At this point it is assumed a single Docker network is associated with the container

* Update util.go

* Using NewVersionedClientFromEnv to create docker client

* Initial basic zookeeper backend for registrator

* Small docs refurbishment

* Note for ignoring individual service on container

* Add support for Consul unix sockets

* Change default port for etc2 backend to default 2379

* Update docs for etcd backend for default port

* *servicePort method
  add support hostip for overlay network

* gofmt bridge

* added Consul TCP Health Check

* removed sentence that was copied from HTTP health check

* Update CHANGELOG

* Add image size to readme.

Closes #290

* Release prep

* Add image size to docs

* bump

* Have the zookeeper backend use the host port for the service paths, allow publishing services if the base service path already exists, and allow publishing into the root of zookeeper. (#367)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants