Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pkg/translator/loki] Split conversion logic into a package #14106

Merged
merged 8 commits into from
Sep 16, 2022

Conversation

jpkrohling
Copy link
Member

As a second step of the Loki exporter refactoring, this change splits the conversion logic into its own package, so that it can be consumed by Loki itself in the future.

Signed-off-by: Juraci Paixão Kröhling juraci@kroehling.de

@jpkrohling jpkrohling force-pushed the jpkrohling/issue13649 branch from f899dc6 to 7a5510f Compare September 13, 2022 19:48
@jpkrohling jpkrohling requested a review from kovrus September 13, 2022 19:49
@jpkrohling jpkrohling force-pushed the jpkrohling/issue13649 branch 2 times, most recently from d18659a to 4675174 Compare September 14, 2022 11:48
@jpkrohling
Copy link
Member Author

@kovrus, I'm still unsure if we should consume the logproto directly from Loki or if we should just copy it here. If we just copy, we probably won't be able to use this translator directly on Loki, as the proto would cause a duplicate registration on their side.

@cyriltovena and @owen-d, I would love to hear your opinions as well.

@jpkrohling
Copy link
Member Author

I'll rebase this after the release, but I consider this PR ready to be reviewed.

pkg/translator/loki/encode.go Outdated Show resolved Hide resolved
"go.uber.org/zap"
)

func LogsToLoki(logger *zap.Logger, ld plog.Logs) (pr *logproto.PushRequest) {
Copy link
Member

@kovrus kovrus Sep 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having zap.Logger as a parameter of this method would enforce users of pkg/translator/loki to use go.uber.org/zap. I am wondering whether we can remove this logger method parameter and make this method return an error as well. Then we can drop the go.uber.org/zap dep of this package altogether, since it is not used anywhere else. Alternatively, we can add some generic logger wrapper, but it seems like an overkill.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing would make sense, I believe

unreleased/13649-loki-convert.yaml Outdated Show resolved Hide resolved
Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>
Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>
@jpkrohling jpkrohling force-pushed the jpkrohling/issue13649 branch from 43fa8d6 to 9fba9ad Compare September 15, 2022 14:47
Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>
Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>
@jpkrohling jpkrohling marked this pull request as ready for review September 15, 2022 15:05
@jpkrohling jpkrohling requested review from a team and mx-psi September 15, 2022 15:05
Copy link
Member

@kovrus kovrus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left one more comment, otherwise, lgtm

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>
Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>
@mx-psi
Copy link
Member

mx-psi commented Sep 16, 2022

@jpkrohling Is this meant to be consumed outside of the contrib repository? (The description makes me think it is, but I want to make sure just in case we can make this internal)

Copy link

@cyriltovena cyriltovena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

@jpkrohling
Copy link
Member Author

@mx-psi, yes, I intend to use this on a PR I'm about to start working on for Loki, to add an OTLP endpoint there.


// some dependencies attempt to bring something like v1.8.2-0.20220303173753-edfe657b5405, which is older than v0.38.0
// at the time of this inclusion, v0.38.0 was the latest version available (also tagged as v2.38.0)
replace github.com/prometheus/prometheus => github.com/prometheus/prometheus v0.38.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add this replace to the contrib distro file on opentelemetry-collector-releases

Copy link
Member

@mx-psi mx-psi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you document the new public API? I think it's just two functions and a struct, so it shouldn't take too much time

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>
@jpkrohling
Copy link
Member Author

Documented! In the process of documenting, I realized that I might need different functions for the OTLP endpoint I'll build, but I'll send smaller PRs if that's the case.

Copy link
Member

@mx-psi mx-psi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, feel free to send the releases PR my way if you need a review :)

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>
@codeboten codeboten merged commit c1a4f58 into open-telemetry:main Sep 16, 2022
djaglowski pushed a commit to djaglowski/opentelemetry-collector-contrib that referenced this pull request Sep 19, 2022
…emetry#14106)

As a second step of the Loki exporter refactoring, this change splits the conversion logic into its own package, so that it can be consumed by Loki itself in the future.
@mar4uk
Copy link
Contributor

mar4uk commented Nov 11, 2022

Closes #13649

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants