Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: merge podTemplates filed more specific, such as tolerations, nodeselector and so on. #7851

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

Ma-YuXin
Copy link
Contributor

@Ma-YuXin Ma-YuXin commented Apr 6, 2024

Changes

fixes: #7127

Background:

In the original implementation, template replacement is simply replacing pipelineRun.spec.podTemplate with pipelineRun.spec.taskRunSpecs[].taskPodTemplate,the specific description can be seen here:Issue #6846.

To solve the problem #6846. In pr #6862, the function MergePodTemplateWithDefault is called to merge podtemplate, In this function, two parameters named tpl and defaultTpl are passed in. It only merges volumes and envs, the judgment logic of the remaining fields is if the same field is set on both templates, the value from tpl will overwrite the value from defaultTpl, which leads to the occurrence of problem #7127

Solution:

Here I created a new function named ExpandedMergePodTemplateWithDefault, which can merge fields such as NodeSelector, Tolerations ,ImagePullSecrets and so on. If there is a conflict when these fields are merged, we will use the value in the field corresponding to the incoming parameter with high priority (similar to the implementation of the original function, that is, the second default of the incoming parameter is higher priority). The reason why it is not modified on the original function is that the original function is referenced in multiple places. If it is modified directly, it may cause unknown problems.

In this pr, I replaced the merge function called in function GetTaskRunSpec with my newly defined function and wrote a test function for my newly defined function and added multiple test cases to the original TestPipelineRun_GetTaskRunSpec test function

/kind bug

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

  • Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
  • Has Tests included if any functionality added or changed
  • pre-commit Passed
  • Follows the commit message standard
  • Meets the Tekton contributor standards (including functionality, content, code)
  • Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
  • Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
  • Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

PodTemplate filed such as tolerations, nodeselector will be merged between pipelineRun.spec.podTemplate and pipelineRun.spec.taskRunSpecs[]

@tekton-robot tekton-robot added kind/bug Categorizes issue or PR as related to a bug. release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Apr 6, 2024
@tekton-robot tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Apr 6, 2024
@tekton-robot
Copy link
Collaborator

Hi @Ma-YuXin. Thanks for your PR.

I'm waiting for a tektoncd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@tekton-robot tekton-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Apr 6, 2024
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/pod/template.go 75.3% 80.7% 5.4

@afrittoli
Copy link
Member

/ok-to-test

@tekton-robot tekton-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Apr 8, 2024
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/pod/template.go 75.3% 80.7% 5.4

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/pod/template.go 75.3% 80.7% 5.4

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/pod/template.go 75.3% 80.7% 5.4

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/pod/template.go 75.3% 80.7% 5.4

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/pod/template.go 75.3% 80.7% 5.4

@Ma-YuXin
Copy link
Contributor Author

Ma-YuXin commented Apr 8, 2024

/retest

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/pod/template.go 75.3% 80.7% 5.4

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/pod/template.go 75.3% 80.7% 5.4

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 14, 2024
@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chitrangpatel, vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

// mergeMaps takes two maps with the same key and value types
// and merges them, giving priority to the items in the override map.
func mergeMaps[K comparable, V any](base, overrides map[K]V) map[K]V {
result := make(map[K]V)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may be we can just add the overwrite to the base and done with it, instead of creating new and return that

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think overwriting the base is safe. The base/default can be reused. We should only change tpl.

Copy link
Member

@afrittoli afrittoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR.

There are two places where template get merged:

  • config with PipelineRun.PodTemplate, via SetDefaults, into the PipelineRun spec
  • PipelineRun.PodTemplate with PipelineRun.TaskRunSpecs[].PodTemplate via GetTaskRunSpec into the TaskRun itself

This PR updates only the latter, but for consistency I think it should update both. WDYT?

@chitrangpatel
Copy link
Contributor

@afrittoli's commend still needs to be addressed. Lets move this to the next release unless it can be done now.

@afrittoli
Copy link
Member

@Ma-YuXin thanks for this PR! Are you able to continue working on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/bug Categorizes issue or PR as related to a bug. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

podTemplate.tolerations do not et merged between taskRunTemplate and taskRunSpec[]
7 participants