Embedded Otel Collector into Agent #3800
Conversation
michalpristas
added
enhancement
|
Duplicating the test requirements here so we know when this is done, at least the first test should exist as part of this PR. The second could be a follow up.
|
| // not otel continue as usual | ||
| return runElasticAgent(ctx, cancel, override, stop, testingMode, fleetInitTimeout, modifiers...) | ||
|
|
||
| } |
There was a problem hiding this comment.
Would really like to see otel integrated into the coordinator and runtime system. That way we get all the status reporting, and the ability to run both at the same time.
There was a problem hiding this comment.
requirement for now was not to do that: https://github.com/elastic/ingest-dev/issues/2612#issue-1976319527
as i understood it.
if we decide to run both at the same time we need to figure out/revisit requirements regarding control ports, logging (you cannot pass logger to collector so far) and making sure otel config is embeddable into our config
There was a problem hiding this comment.
@joshdover what was your idea of the scope for the initial integration?
There was a problem hiding this comment.
Even if we didn't want to allow both to be ran at the same time, by not using the coordinator and runtime means that none of the status reporting will be correct. Don't we want elastic-agent status to work?
There was a problem hiding this comment.
My idea was to keep it as simple as possible for now to allow us to experiment hands on with it. We don't have short-term plans to integrate OTel Collector into Agent mode, it will come later.
I think we can discuss how elastic-agent status should work - not against supporting that and using the coordinator to do it. But I still think we can merge this PR with this gap, aiming for very small chunks of work here.
|
@michalpristas Left a couple of very minor comments/questions, mostly looks like some missing cleanup. |
|
e2e failing on endpoint not being able to connect to ES |
|
For reference this PR increases the size of the elastic-agent binary by 22 MB on my Mac. |
|
26.7% Coverage on New Code
|
failing on unrelated e2e again |
|
Both test failures are known issues. I'm going to force merge this one. |
What does this PR do?
This PR adds an ability to start agent in collector mode.
So far logic is simple. When specifying config using
-cflag it checks the name of the config.If config is
otel,otlp,otlpcolthen it starts in otel mode.When name does not match but config contains
serviceandexporters,importersthen we assume collector config and start in otel mode.Otherwise we start in a standard way.
On top of that, we provide custom config provider which serves a default value for logging output ==
stdoutinstead of otel defaultstderr.Tested on darwin and windows. when changing
elastic-agent.ymlto contain otel config even service is started in otel modeDoD from issue
stdoutChecklist
./changelog/fragmentsusing the changelog tool