Error Starting Fluentd

[wooxudong]

Problem

The fluend container would not be able to start with following errors

...

Steps to replicate

FROM bitnami/fluentd:1.14.0

RUN fluent-gem install 'fluent-plugin-aws-elasticsearch-service' -v 2.4.1

Expected Behavior or What you need to ask

Fluentd to start correctly
...

Using Fluentd and ES plugin versions

Fluentd: v1.14.0 fluent-plugin-aws-elasticsearch-service: v 2.4.1

[pavel-trubitsyn]

Hi, I have the same issue with fluent/fluentd:v1.14.1.

[pavel-trubitsyn]

Solution uken/fluent-plugin-elasticsearch#912 (comment)

[wooxudong]

@pavel-trubitsyn , thanks for the solution.

I think it shall be a proper fix for this plugin to solve the dependency chain ?

[wooxudong]

Any updates on this ?

[DaemonDude23]

I've been watching this problem overall for the last year. Today I got it working after learning about an OpenSearch-specific plugin from the Fluentd maintainers.

I had previously built a container with plugins from this repo to push logs to AWS's OpenSearch, but when trying to update it, ran into these dependency conflicts.
I upgraded from: fluent/fluentd:v1.13.3-debian-1.0 to fluent/fluentd:v1.14.5-debian-1.0. The Dockerfile I kept the same, other than removing any elasticsearch plugins and replacing it with: fluent-plugin-opensearch:1.0.1 (https://github.com/fluent/fluent-plugin-opensearch).

Then I modified the config to change the output @type aws-elasticsearch-service to opensearch. I removed 2 parameters that are not supported by this plugin (log_es_400_reason and suppress_type_name) and it's working just fine.
I see there are many issues in this repo that relate to this problem. Hopefully this will help someone resolve it.
EDIT: There's a bug with the Opensearch plugin: fluent/fluent-plugin-opensearch#24