Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[processor/k8sattributes] Add container.image.repo_digests #34031

Merged
merged 11 commits into from
Aug 6, 2024

Conversation

evantorrie
Copy link
Contributor

@evantorrie evantorrie commented Jul 11, 2024

Description:

Adds support for extracting canonical repo digest image references for a container from k8s attributes

Link to tracking Issue: #34029

Testing:

  1. Adds unit tests for extracting the k8s .imageID field from container status API if it is of the reference.Canonical form, and storing it in a ImageRepoDigest string field. (Note: k8s will only ever return one value here, although the semantic conventions for container.image.repo_digests define it to be a Slice. We convert into a Slice lazily when we populate the pcommon data structure.
  2. Adds unit tests for including container.image.repo_digests in rules for extraction.
  3. Includes container.image.repo_digests in the e2e tests for k8sattributes processor.

Documentation:

Added container.image.repo_digests as an example of attributes that can be extracted from a container

@evantorrie
Copy link
Contributor Author

evantorrie commented Jul 11, 2024

Implementation notes:

  1. This is the first k8sattributes e2e test which returns a Slice attribute value. I have not changed the "expectedValue" matcher to explicitly handle slices. Since there is only ever going to be one value in the container.image.repo_digests slice attribute, the existing test code which checks for the "expected value" appearing anywhere in "value.ToString()" will work without having to radically rework the matchers.
  2. Although the k8s API structure returns this value in ImageID, when we process this into our local data structure for later usage, we use a ImageRepoDigest string field - and populate it only if the ImageID field returned by the k8s API is of the correct distribution/reference.Canonical type.

@evantorrie evantorrie marked this pull request as ready for review July 11, 2024 06:05
@evantorrie evantorrie requested a review from a team July 11, 2024 06:05
@ChrsMark
Copy link
Member

So eventually we are not going to follow the

I think one way to proceed is to also use the github.com/distribution/reference package to attempt to parse the output of .imageID from the k8s API response. If it is a canonical reference, then populate the semantic attribute container.image.repo_digests with those values (of which it seems there will only be one from the existing CRI implementations (?)). If it is not, then populate container.image.id.

from #32314 (comment)?

Doesn't this fallback to container.image.id makes sense in case we cannot extract the repo digests?

@evantorrie
Copy link
Contributor Author

Doesn't this fallback to container.image.id makes sense in case we cannot extract the repo digests?

I was trying to simplify this PR by limiting the scope to just populating container.image.repo_digests if what the CRI returns is indeed a valid repo digest.

Including container.image.id could be done in a subsequent PR (I do have a version of this code which does the fallback), although I still am unclear on the use case for someone wanting to know container.image.id in cases where it is not a valid repo reference, and instead is just a sha256 ref that is valid only for the local CRI used by the k8s apiserver.

Yes, for theoretical completeness sake, it seems to make sense, but in practice, I don't know whether someone really would use this. Maybe my imagination (or understanding of what value a sha256 ref provides) is too limited.

@ChrsMark
Copy link
Member

@evantorrie sounds good!

Yes, for theoretical completeness sake, it seems to make sense, but in practice, I don't know whether someone really would use this. Maybe my imagination (or understanding of what value a sha256 ref provides) is too limited.

The only use-case I would see here is if somebody wants to reference an image even if that's not coming from a repo. For a security analysis, generic issue investigation or sth. But yes it can be done later.

@evantorrie
Copy link
Contributor Author

Unrelated unit test failing due to this: #33998

Copy link
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks good to me based on the prior discussions within the various related issues. Codeowners of the processor should also review/confirm.

processor/k8sattributesprocessor/options.go Show resolved Hide resolved
processor/k8sattributesprocessor/README.md Outdated Show resolved Hide resolved
@evantorrie
Copy link
Contributor Author

@open-telemetry/collector-contrib-approvers Could we get another pair of eyes on this pull request? Thanks!

Copy link
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

processor/k8sattributesprocessor/README.md Outdated Show resolved Hide resolved
processor/k8sattributesprocessor/README.md Outdated Show resolved Hide resolved
@ChrsMark
Copy link
Member

@evantorrie the CI failures look unrelated. Consider rebasing as well.

evantorrie and others added 3 commits August 1, 2024 03:25
Direct link to v1.26.0 of semantic conventions

Co-authored-by: Christos Markou <chrismarkou92@gmail.com>
@evantorrie
Copy link
Contributor Author

@TylerHelmuth @fatsheep9146 This has been pretty much complete for a week or so now. Any chance of getting another review on it from the code-owners?

Copy link
Member

@dmitryax dmitryax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Just one question

processor/k8sattributesprocessor/processor.go Show resolved Hide resolved
@dmitryax dmitryax merged commit 528678e into open-telemetry:main Aug 6, 2024
161 checks passed
@github-actions github-actions bot added this to the next release milestone Aug 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants