fix: Delete invocation_id log field when streaming plugins logs
#366
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
erezrokah
requested review from
marianogappa and
bbernays
and removed request for
a team and
bbernays
marianogappa
approved these changes
Jul 19, 2024
kodiakhq bot
pushed a commit
that referenced
this pull request
Jul 19, 2024
🤖 I have created a release *beep* *boop* --- ## [1.20.7](v1.20.6...v1.20.7) (2024-07-19) ### Bug Fixes * Delete `invocation_id` log field when streaming plugins logs ([#366](#366)) ([148b167](148b167)) * **deps:** Update module github.com/cloudquery/cloudquery-api-go to v1.12.3 ([#364](#364)) ([28fae3f](28fae3f)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
kodiakhq bot
pushed a commit
to cloudquery/cloudquery
that referenced
this pull request
Jul 22, 2024
…#18641) #### Summary Fixes #18443 Also needs cloudquery/plugin-pb-go#366 so the field isn't duplicated in JSON logs
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Part of cloudquery/cloudquery#18443. Both CLI and plugins add the invocation ID field to the log.
When the CLI invokes plugins it streams their logs into its own log so it all shows up in the same file.
This happens here
plugin-pb-go/managedplugin/plugin.go
Line 480 in 28fae3f
We parse the log line as JSON, then log it with the CLI logger.
zerologdoesn't remove duplicate fields (see https://github.com/rs/zerolog?tab=readme-ov-file#field-duplication ) we need to remove the duplicate ourselves (similar tolevel).Please note that the duplicates only show up when passing
--log-format jsonas it seems the text format does remove duplicates (zerologsaves the message to be printed internally in JSON format so probably the conversion to text removes the duplicate somewhere, didn't bother to check).CLI PR in cloudquery/cloudquery#18641
Use the following steps to ensure your PR is ready to be reviewed
go fmt ./...to format your code 🖊golangci-lint run🚨 (install golangci-lint here)