Skip to content

Add a WithValueTranslator option to Reconciller. #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 25, 2021
Merged

Conversation

porridge
Copy link
Collaborator

This lets the user inject code which produces helm values based on the fetched object itself (unlike Mapper which can only see Values).
This way the custom code can convert the object to a typed one and work with proper structs rather than a tree of maps to interface{}.

One wrinkle I'm not proud of is the conversion between chartutil.Values and
internal/values.Values but I don't think it's ugly enough to put it on the
critical path.

This lets the user inject code which produces helm values based on the fetched object itself (unlike Mapper which can only see Values).
This way the custom code can convert the object to a typed one and work with proper structs rather than a tree of maps to interface{}.

One wrinkle I'm not proud of is the conversion between chartutil.Values and
internal/values.Values but I don't think it's ugly enough to put it on the
critical path.
@@ -362,8 +363,17 @@ func WithPostHook(h hook.PostHook) Option {
}
}

// WithValueTranslator is an Option that configures a function that translates a
// custom resource to the values passed to Helm.
func WithValueTranslator(t values.Translator) Option {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you mention the difference to the Mapper? It could get confusing as they have similar use-cases.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, reading this PR I don't really understand why a custom translator is required in addition to the mapper. Things that come to mind:

  • access to metadata (name, ...) and status fields
  • ability to use Go types by converting the *unstructured.Unstructured into the Go type for a CR.

However, having translator + mapper seems confusing. Perspectively, I think it would make more sense to introduce translator as the first-class citizen, and drop the valueMapper field. WithValueMapper could then return a function that sets the valueTranslator accordingly (relying on FromUnstructured).

Or am I completely off, and the difference is actually whether it gets executed pre/post the application of overrides?

Copy link
Collaborator Author

@porridge porridge May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SimonBaeumer added a couple of sentences.

@misberner the rationale is in the PR description:

This lets the user inject code which produces helm values based on the fetched object itself (unlike Mapper which can only see Values).
This way the custom code can convert the object to a typed one and work with proper structs rather than a tree of maps to interface{}.

Basically it is so that we can have write type-safe code at least to read the CRs.

I'd agree about dropping the Mapper API eventually, but:

  • it's out of scope of this PR, I'm trying to diverge from upstream as little as possible,
  • in general it would need more discussion with upstream, for example there are also these ApplyOverrides and CoalesceValues steps in getValues which I don't fully understand, so there might be use cases where Mapper is a better tool

@@ -32,6 +32,14 @@ type Values struct {
}

func FromUnstructured(obj *unstructured.Unstructured) (*Values, error) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is FromUnstructured() still used somewhere? If not, maybe it should be deleted.

Alternatively, I don't think you need ChartValuesFromUnstructured() function at all. Its only difference from FromUnstructured() is that it doesn't wrap map[string]interface{} into Values. And that's only for DefaultTranslator.

You can do something like

var DefaultTranslator = values.TranslatorFunc(func(u *unstructured.Unstructured) (chartutil.Values, error) {
	blah, err = FromUnstructured(u)
	if err != nil {
		return chartutil.Values{}, err
	}
    return blah.Map(), nil
}

and then ChartValuesFromUnstructured() isn't used anywhere.

I think that's simpler than finding the proper name for ChartValuesFromUnstructured() and adjusting tests for it (which I otherwise would ask).


Maybe authors have an idea to extend Values struct with some other filed. Right now, I think this struct is redundant and the code will gain more clarity if it goes away eventually.

I see that with ChartValuesFromUnstructured() you avoid creating a garbage object but I don't think it is worthwhile goal. Even though the object is returned as pointer, I believe it should be placed on stack in Reconciler.getValues() (due to escape analysis), and so freeing it is cheap.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I was trying to avoid the back-and-forth of New->Map->New, not for performance reasons but for simplicity. But getting rid of ChartValuesFromUnstructured seems worth it. Changed.

Copy link
Contributor

@misberner misberner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minimizing divergence makes sense to me to make our lives easier for future rebases. When we pitch this to upstream, however, I think it would be nice to move applyOverrides + mapper to a "legacy translator", as imho neither makes sense if you have something as flexible as a translator (CoalesceValues seems to be required as it pulls in values defined in the chart itself)

Copy link

@msugakov msugakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to Malte's point about retiring Mapper&Co. in favor of Translator.
What could be a good way for us to not forget mention it when we eventually merge this upstream. Should we add a // TODO: comment on Mapper for this?

@porridge
Copy link
Collaborator Author

TODO added, merging.

@porridge porridge merged commit 88508a2 into main May 25, 2021
@porridge porridge deleted the translate-option branch May 25, 2021 09:25
porridge added a commit that referenced this pull request Dec 2, 2021
Revert "Pass context to Translate(). (#8)"
This reverts commit c3df552.

Revert "Add a WithValueTranslator option to Reconciller. (#6)"
This reverts commit 88508a2.
porridge added a commit that referenced this pull request Dec 15, 2021
Revert "Pass context to Translate(). (#8)"
This reverts commit c3df552.

Revert "Add a WithValueTranslator option to Reconciller. (#6)"
This reverts commit 88508a2.
porridge added a commit that referenced this pull request Dec 17, 2021
* Revert Translator commits.

Revert "Pass context to Translate(). (#8)"
This reverts commit c3df552.

Revert "Add a WithValueTranslator option to Reconciller. (#6)"
This reverts commit 88508a2.

* Add a WithValueTranslator option to Reconciller (cherry-picked from upstream PR).

A Translator is a way to produces helm values based on the fetched custom
resource itself (unlike `Mapper` which can only see `Values`).

This way the code which converts the custom resource to Helm values can first
convert an `Unstructured` into a regular struct, and then rely on Go type
safety rather than work with a tree of maps from `string` to `interface{}`.

Thanks to having access to a `Context`, the code can also safely access the
network, for example in order to retrieve other resources from the k8s cluster,
when they are referenced by the custom resource.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants