Skip to content
This repository has been archived by the owner on Jun 29, 2022. It is now read-only.

contour: Fix the hostPort regression #1342

Merged
merged 6 commits into from
Mar 3, 2021

Conversation

surajssd
Copy link
Member

@surajssd surajssd commented Jan 22, 2021

This change was introduced in 8d44c85 - "don't expose Envoy using hostPorts". But a later commit overrode it: 6a6a3c5 - "contour: Update to v1.7.0". This commit fixes this regression.

Fixes: #1336

@surajssd surajssd force-pushed the surajssd/fix-contour-envoy-regression branch from 14b33db to 968745a Compare January 29, 2021 15:38
@surajssd
Copy link
Member Author

Right now the way these test helper functions are written there is no easy way to write test that verify an error condition. Right now the test fails, right away because we pass test object t.

@surajssd surajssd force-pushed the surajssd/fix-contour-envoy-regression branch from 968745a to 35fd587 Compare January 29, 2021 15:48
pkg/components/velero/component_test.go Outdated Show resolved Hide resolved
pkg/components/internal/testutil/jsonpath.go Outdated Show resolved Hide resolved
pkg/components/internal/testutil/jsonpath.go Outdated Show resolved Hide resolved
pkg/components/contour/component_test.go Show resolved Hide resolved
@surajssd surajssd force-pushed the surajssd/fix-contour-envoy-regression branch from 35fd587 to 1e5d6e3 Compare February 1, 2021 08:31
@surajssd surajssd force-pushed the surajssd/fix-contour-envoy-regression branch from 9f06d73 to d7e4f66 Compare February 4, 2021 07:06
@surajssd
Copy link
Member Author

surajssd commented Feb 4, 2021

@invidian PTAL all the comments have been addressed.

@surajssd surajssd force-pushed the surajssd/fix-contour-envoy-regression branch from d7e4f66 to 56f55b1 Compare February 12, 2021 12:33
Copy link
Member

@invidian invidian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking better :)

pkg/k8sutil/create.go Outdated Show resolved Hide resolved
pkg/k8sutil/create.go Outdated Show resolved Hide resolved
pkg/k8sutil/create.go Show resolved Hide resolved
pkg/components/internal/testutil/jsonpath.go Show resolved Hide resolved
pkg/components/internal/testutil/jsonpath.go Show resolved Hide resolved
// what error to expect.
func JSONPathExists(t *testing.T, yamlConfig string, jsonPath string, errExp string) {
_, err := jsonPathValue(yamlConfig, jsonPath)
if err != nil && errExp == "" { //nolint:gocritic
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would avoid disabling gocritic.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is suggesting usage of a switch case instead of the if else. Given the kinda checks we have switch case is not ideal.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also uses elses a lot, which are discouraged in Go..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can just remove the else's here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also use a const error here, then complexity of the whole thing is reduced to just errors.Is() call, which I think is elegant. I can share the code for that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed else because they are each func is just a way to exit out of the function.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is errors.Is() or any const we create going to help with the error coming from upstream lib? Especially when they have not declared an error var for us to consume but just an error created using fmt.Errorf.

return results, fmt.Errorf("%s is not found", node.Value)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is errors.Is() or any const we create going to help with the error coming from upstream lib? Especially when they have not declared an error var for us to consume but just an error created using fmt.Errorf.

If we set allowMissingKeys for jsonpath, then we get no value, so we can return constant ValueNotFound error.

Though I'm not a big fan of this const error pattern, as it adds bunch of boilerplate. It would look something like:

type NotFoundError string

func (e NotFoundError) Error() string {
  return string(e)
}

const ErrNotFound NotFoundError = "not found"

type KeyNotFoundError struct {
  Key string
}

func (e KeyNotFoundError) Error() string {
  return fmt.Sprintf("%s %s", e.Key, ErrNotFound)
}

func (e KeyNotFoundError) Unwrap() error {
  return ErrNotFound
}

func jsonPathValue(data, key string) (string, error) {
  if !strings.Contains(data, key) {
    return "", KeyNotFoundError{Key: key}
  }

  return "", nil
}

func JSONPathExistsSimplified(t *testing.T, yamlConfig string, jsonPath string, errExp error) {
  _, err := jsonPathValue(yamlConfig, jsonPath)
  if !errors.Is(err, errExp) {
    t.Fatalf("Unexpected error: %v, expected: %v", err, errExp)
  }
}

So I think returning nil value would probably also work fine.

}

if len(v) == 0 || len(v[0]) == 0 {
t.Fatalf("No result found")
return reflect.Value{}, fmt.Errorf("no result found")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than returning a error here, how about we change the return type to *reflect.Value and we return nil, nil.

This way, we know everything went well, including query parsing, but the value was simply not found. This will make checking more idiomatic.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is reworked now. The functionality is not changed just returning an error to bubble up the caller stack.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we still must use strings.Contains, which IMO should be used as a last resort tool, as it test specific implementation.

With approach I suggested, we avoid ambiguity and we work against the specification of function.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree *reflect.Value makes sense to distinguish errors from value not found.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The newly added test helper JSONPathExists is currently relying on the error from this function call:

v, err := jPath.FindResults(obj.Object)

So I don't understand how is the change to the pointer going to help in anything? We will still have to check for the error and match the string.

The error is not coming from this block:

if len(v) == 0 || len(v[0]) == 0 {
	return reflect.Value{}, fmt.Errorf("no result found")
}

Also jPath.FindResults returns [][]reflect.Value which definitely not a array of pointers so we don't need to worry about deep copying it if we convert it to pointer.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@surajssd I think calling jPath.AllowMissingKeys() should help then?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's wrong with getting an error on something that does not exist?

It is possible that user (in this case a developer) might be giving wrong jsonpath and expecting something to exist. Such cases are ignored with AllowMissingKeys or gives wrong results.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

developer would expect certain value, so they must check for nil values. So either we have const error or return nil, nil, both have similar result with slightly different interface. IMO const error is cleaner from API point of view, but it brings more boilerplate for maintenance. I'm not sure which one is better, other than returning nil is simpler.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AllowMissingKeys gives us strong gaurantee on which we rely on, so not making at false. But yes returning nil instead of error, then it is up to the user to figure out if there is an empty result.

Copy link
Member

@knrt10 knrt10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question

pkg/k8sutil/create.go Outdated Show resolved Hide resolved
@surajssd surajssd force-pushed the surajssd/fix-contour-envoy-regression branch from 56f55b1 to e7e25d3 Compare February 15, 2021 09:15
Copy link
Member

@invidian invidian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think k8sutil: Rename YAML imports should be split and rebased.

pkg/k8sutil/create.go Outdated Show resolved Hide resolved
pkg/k8sutil/create.go Outdated Show resolved Hide resolved
pkg/k8sutil/create.go Outdated Show resolved Hide resolved
pkg/k8sutil/create.go Outdated Show resolved Hide resolved
pkg/k8sutil/create.go Show resolved Hide resolved
pkg/components/internal/testutil/common.go Show resolved Hide resolved
pkg/k8sutil/create.go Outdated Show resolved Hide resolved
pkg/components/internal/testutil/jsonpath.go Show resolved Hide resolved
// what error to expect.
func JSONPathExists(t *testing.T, yamlConfig string, jsonPath string, errExp string) {
_, err := jsonPathValue(yamlConfig, jsonPath)
if err != nil && errExp == "" { //nolint:gocritic
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also uses elses a lot, which are discouraged in Go..

}

if len(v) == 0 || len(v[0]) == 0 {
t.Fatalf("No result found")
return reflect.Value{}, fmt.Errorf("no result found")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we still must use strings.Contains, which IMO should be used as a last resort tool, as it test specific implementation.

With approach I suggested, we avoid ambiguity and we work against the specification of function.

pkg/k8sutil/create_test.go Show resolved Hide resolved
pkg/k8sutil/create.go Outdated Show resolved Hide resolved
}

if len(v) == 0 || len(v[0]) == 0 {
t.Fatalf("No result found")
return reflect.Value{}, fmt.Errorf("no result found")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree *reflect.Value makes sense to distinguish errors from value not found.

pkg/components/internal/testutil/jsonpath.go Show resolved Hide resolved
pkg/k8sutil/create.go Outdated Show resolved Hide resolved
// what error to expect.
func JSONPathExists(t *testing.T, yamlConfig string, jsonPath string, errExp string) {
_, err := jsonPathValue(yamlConfig, jsonPath)
if err != nil && errExp == "" { //nolint:gocritic
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can just remove the else's here.

@surajssd surajssd force-pushed the surajssd/fix-contour-envoy-regression branch 3 times, most recently from 77e998e to 01ea884 Compare February 24, 2021 12:17
@surajssd surajssd added this to the v0.7.0 milestone Feb 24, 2021
@surajssd surajssd force-pushed the surajssd/fix-contour-envoy-regression branch from 01ea884 to 0317077 Compare February 25, 2021 08:06
knrt10
knrt10 previously requested changes Feb 25, 2021
Copy link
Member

@knrt10 knrt10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NITs

pkg/k8sutil/create.go Outdated Show resolved Hide resolved
pkg/components/internal/testutil/jsonpath.go Show resolved Hide resolved
@surajssd surajssd force-pushed the surajssd/fix-contour-envoy-regression branch 2 times, most recently from 968085c to 9c206e1 Compare February 26, 2021 10:31
iaguis
iaguis previously approved these changes Mar 3, 2021
Copy link
Contributor

@iaguis iaguis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of nits. Looks good to me in general.

pkg/k8sutil/create.go Outdated Show resolved Hide resolved
pkg/k8sutil/create.go Outdated Show resolved Hide resolved
pkg/components/internal/testutil/common.go Show resolved Hide resolved
@invidian invidian dismissed their stale review March 3, 2021 11:52

Dismissing my review.

This (`k8s.io/apimachinery/pkg/util/yaml`) is renamed to `yamlutil` so
that in the next commit when we import `sigs.k8s.io/yaml` it can simply
be used as `yaml.`.

Signed-off-by: Suraj Deshmukh <suraj@kinvolk.io>
This commit adds an object ObjectMetadata to identify any Kubernetes
object uniquely.

Adds three exported functions. YAMLToUnstructured converts any YAML
object to `unstructured.Unstructured` type. SplitYAMLDocuments splits a
YAML file with multiple YAML documents separated by `---` into separate
YAML configs. YAMLToObjectMetadata takes in a YAML config as string and
returns ObjectMetadata.

Add unit tests for each of the exported functions.

Signed-off-by: Suraj Deshmukh <suraj@kinvolk.io>
This commit removes the local function `unstructredObj` and uses the
function from the `k8sutil` package.

Signed-off-by: Suraj Deshmukh <suraj@kinvolk.io>
This commit modifies the behaviour of passing the test object of type
`testing.T` to the helper functions, instead return the errors.

Signed-off-by: Suraj Deshmukh <suraj@kinvolk.io>
Earlier the conversion tests were using unreliable file name generated
by Helm to find a Kubernetes object's YAML. Now this commit changes that
and the users should now provide Version, Kind and Name to find the
specific object.

This specially solves the problem when a single file has multiple YAML
documents separated by `---`, the new functions provide reliable way of
finding the objects.

Change the signature of ConfigFromMap to accept `k8sutil.ObjectMetadata`
instead of string indicating file name.

Modify the conversion tests to provide `k8sutil.ObjectMetadata` to find
specific object.

Signed-off-by: Suraj Deshmukh <suraj@kinvolk.io>
This change was introduced in `8d44c8513f67512527218c8a538e79a61cb7cd34`
- "don't expose Envoy using hostPorts". But a later commit overrode it:
`6a6a3c5d849b981559c2cc2cf3797bfd6b6c9159` - "contour: Update to
v1.7.0". This commit fixes this regression.

Add test to verify if hostPort does not exist, check if the converted
config does not contain the hostPort.

This commit also adds test to check if a certain JSONPath exists, it is
used in the above unit test.

Signed-off-by: Suraj Deshmukh <suraj@kinvolk.io>
@iaguis
Copy link
Contributor

iaguis commented Mar 3, 2021

AWS CI fail seems legit

@surajssd surajssd dismissed knrt10’s stale review March 3, 2021 16:03

This has been incorporated.

@surajssd surajssd merged commit 2482601 into master Mar 3, 2021
@surajssd surajssd deleted the surajssd/fix-contour-envoy-regression branch March 3, 2021 16:04
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Don't expose envoy hostport
5 participants