Support otel_logs codec in S3 source (#5028)

[danhli]

Description

Add a new codec option, otel_logs, in S3 source. With this option, OTLP JSON files can be ingested and processed natively, similar to ingesting through http endpoint using otel_logs_source.

Issues Resolved

Resolves #5028

Check List

• New functionality includes testing.
• New functionality has a documentation issue. Please link to it in this PR.
  • New functionality has javadoc added
• Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[dlvenable]

Can you rename this to OTelLogsCodec? What was the reason for Json in the name?

[danhli]

Renamed it to OTelLogsInputCodec.

[dlvenable]

Maybe this should be otel_json_logs or otel_logs_json?

[danhli]

Changed it to otel_logs.

[dlvenable]

Thank you @danhli . I have a few comments to help with documentation.

[dlvenable]

This should be an enum. It will help with our auto-generated schema and documentation.

I've submitted a few PRs to correct these, such as #5025 and #5035.

Please see these:

https://github.com/opensearch-project/data-prepper/blob/main/data-prepper-plugins/key-value-processor/src/main/java/org/opensearch/dataprepper/plugins/processor/keyvalue/WhitespaceOption.java

https://github.com/opensearch-project/data-prepper/blob/main/data-prepper-plugins/key-value-processor/src/test/java/org/opensearch/dataprepper/plugins/processor/keyvalue/WhitespaceOptionTest.java

This is where it is used:

data-prepper/data-prepper-plugins/key-value-processor/src/main/java/org/opensearch/dataprepper/plugins/processor/keyvalue/KeyValueProcessorConfig.java

Lines 129 to 133 in 6e51f51

	@JsonProperty("whitespace")
	@JsonPropertyDescription("Specifies whether to be lenient or strict with the acceptance of " +
	"unnecessary white space surrounding the configured value-split sequence. " +
	"In this case, strict means that whitespace is trimmed and lenient means it is retained in the key name and in the value." +
	"Default is <code>lenient</code>.")

[dlvenable]

Please add documentation:

See an example:

data-prepper/data-prepper-plugins/flatten-processor/src/main/java/org/opensearch/dataprepper/plugins/processor/flatten/FlattenProcessorConfig.java

Lines 19 to 20 in 6e51f51

	@JsonPropertyOrder
	@JsonClassDescription("The <code>flatten</code> processor transforms nested objects inside of events into flattened structures.")

[dlvenable]

Please add documentation:

Examples:

data-prepper/data-prepper-plugins/key-value-processor/src/main/java/org/opensearch/dataprepper/plugins/processor/keyvalue/KeyValueProcessorConfig.java

Lines 129 to 135 in 6e51f51

	@JsonProperty("whitespace")
	@JsonPropertyDescription("Specifies whether to be lenient or strict with the acceptance of " +
	"unnecessary white space surrounding the configured value-split sequence. " +
	"In this case, strict means that whitespace is trimmed and lenient means it is retained in the key name and in the value." +
	"Default is <code>lenient</code>.")
	@NotNull
	private WhitespaceOption whitespace = DEFAULT_WHITESPACE;