Use agent docker image

[mtojek]

This PR adds support for Elastic Agent image.

Docker-compose allows for scaling agent instances.

[ruflin]

I did not get this working on my end. In Kibana I get:

request to http://package-registry:8080/search?package=base&internal=true failed, reason: getaddrinfo ENOTFOUND package-registry package-registry:8080

And in the logs I see:

elastic-agent_1     | fail to enroll: fail to execute request to Kibana: Status code: 401, Kibana returned an error: Unauthorized, message: [security_exception] missing authentication credentials for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0="ApiKey" & 1="Basic realm=\"security\" charset=\"UTF-8\"" } } }
environments_elastic-agent_1 exited with code 1

[ruflin]

@kuisathaverat Can we make this image public?

[mtojek]

I did not get this working on my end. In Kibana I get:

request to http://package-registry:8080/search?package=base&internal=true failed, reason: getaddrinfo ENOTFOUND package-registry package-registry:8080

And in the logs I see:

elastic-agent_1     | fail to enroll: fail to execute request to Kibana: Status code: 401, Kibana returned an error: Unauthorized, message: [security_exception] missing authentication credentials for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0="ApiKey" & 1="Basic realm=\"security\" charset=\"UTF-8\"" } } }
environments_elastic-agent_1 exited with code 1

Did you take down all running containers? It looks like it referred to not existing one.

[kuisathaverat]

yep, we can make it public, Which namespace you want to push it?

[ruflin]

First time I run this I got:

Creating environments_package-registry_1 ... done
Creating environments_elasticsearch_1    ... done

ERROR: for kibana  Container "82665ad5fd87" is unhealthy.
ERROR: Encountered errors while bringing up the project.

I assume the timeout was too low.

[mtojek]

Hmm.. you can try to wipe out docker images. Anyway, this one is weird. If it's not a one-time issue, would be good to reproduce it.

[ruflin]

Would be nice to have a mage command for this, easier to remember (at least for me).

[mtojek]

Well it all depends what do you need, either entire fleet with enrolled agent or you prefer to keep the agent aside on your host.

I can add one or two default mage targets, but it will limit configurability of available docker-compose options.

[ruflin]

Lets wait with it and see what most of us use frequently.

[mtojek]

I noticed other issue. I don't see any metricbeat and filebeat processes running:

[root@0c60bbe45399 elastic-agent]# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.1 559748 15772 ?        Ssl  08:39   0:00 elastic-agent run
root       190  0.0  0.0  11836  2908 pts/0    Ss   08:40   0:00 /bin/bash
root       241  0.0  0.0  51760  3528 pts/0    R+   08:51   0:00 ps aux

@michalpristas does it look like a bug or should I enable something else?

[michalpristas]

depends on a config, can you get a config retrieved?
./elastic-agent inspect

[mtojek]

depends on a config, can you get a config retrieved?
./elastic-agent inspect

[root@0c60bbe45399 elastic-agent]# ./elastic-agent inspect
Error: unknown command "inspect" for "elastic-agent"
Run 'elastic-agent --help' for usage.
unknown command "inspect" for "elastic-agent"

Does it mean the agent docker image is not published on regular basis?

-rwxr-x--- 1 root elastic-agent 21907211 Apr 28 11:48 elastic-agent

[michalpristas]

this seems rather old
i will test in on my end as well but i need to do some testing on 7.x today

^^ @kuisathaverat how often is docker image published?

[ruflin]

There is also docker.elastic.co/beats/elastic-agent:8.0.0-SNAPSHOT, I wonder if this helps.

[kuisathaverat]

^^ @kuisathaverat how often is docker image published?

we push with every correct merge to master, 7.x, and 7.7 to docker.elastic.co/observability-ci/elastic-agent:BRANCH_NAME and docker.elastic.co/observability-ci/elastic-agent:SHA1 you can check the images at https://container-library.elastic.co/r/observability-ci/elastic-agent

This namespace is private, if you need them on public we can make it

[mtojek]

@michalpristas @ruflin ^

any suggestions regarding the public namespace/project? "beats/elastic-agent"?

BTW It would great to clean all other namespaces from old agent's images.

[kuisathaverat]

BTW It would great to clean all other namespaces from old agent's images.

we do not have permission we will open an infra issue for that.

[ruflin]

@kuisathaverat How is this different from docker.elastic.co/beats/elastic-agent:8.0.0-SNAPSHOT?

[kuisathaverat]

Those images (docker.elastic.co/beats/elastic-agent:8.0.0-SNAPSHOT) are released by the snapshot process from the RElease team, they are generated once a day

[ruflin]

But the content of the file is the same? My assumption would be for the use case here from @mtojek once a day should normally be enough?

[mtojek]

But the content of the file is the same? My assumption would be for the use case here from @mtojek once a day should normally be enough?

Absolutely, it works for me.

[ruflin]

Figured out why it didn't work. I had a fresh checkout of the repo and never run mage build so the registry stopped because of no packages.

[mtojek]

Figured out why it didn't work. I had a fresh checkout of the repo and never run mage build so the registry stopped because of no packages.

I added a hint to the README.