schema cty types incompatible with hclwrite

[melinath]

terraform-plugin-sdk has a dependency on github.com/hashicorp/go-cty/cty which is an out-of-date fork of github.com/zclconf/go-cty.

Expected Behavior

Data should be easy to convert from cty -> hcl using hclwrite.

Actual Behavior

schema is defined using an out-of-date forked version of go-cty, so the only way to export with hclwrite is to first convert the schema types manually from github.com/hashicorp/go-cty/cty -> github.com/zclconf/go-cty

Steps to Reproduce

import (
	"encoding/json"

	hashicorpcty "github.com/hashicorp/go-cty/cty"
	"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
	"github.com/zclconf/go-cty/cty"
	ctyjson "github.com/zclconf/go-cty/cty/json"
)

func mapToCtyValWithSchema(m map[string]interface{}, s map[string]*schema.Schema) cty.Value {
	b, err := json.Marshal(&m)
	if err != nil {
		panic(fmt.Errorf("error marshaling map as JSON: %v", err))
	}
	ty := schema.InternalMap(s).CoreConfigSchema().ImpliedType()
	ret, err := ctyjson.Unmarshal(b, ty)
	if err != nil {
		panic(fmt.Errorf("error unmarshaling JSON as cty.Value: %v", err))
	}
	return ret
}

This will fail to compile because ctyjson requires a cty.Type; using ctyjson is necessary because hclwrite requires cty values as input.

References

hashicorp/hcl#546

[bflad]

Please note that I have changed the labels of this issue from bug to enhancement as this does not represent functionality that is broken, but rather an artifact of the previous intentional choice to switch the SDK's dependency on cty to a fork.

That being said, since v2.8.0 the SDK has begun exposing the underlying cty implementation more broadly as part of functionality such as (*helper/schema.ResourceData).GetRawConfig() which can allow provider developers to workaround some of the SDK limitations around its automatic behaviors. It's certainly less than ideal that it is not as interoperable with the general HCL ecosystem for very advanced use cases.

Unfortunately, we cannot just swap out the implementation now to just point to upstream, as that would represent a breaking change for anyone using this existing functionality. Even if we deprecated the existing exported fields/methods and tried to expose new methods using the upstream, the underlying cty data and logic would either need to be wholly copied or perform some non-trivial type conversions.

Given the amount of complexity involved with potentially switching this implementation to the upstream would bring, and the fact that there are no plans for another major version update of this Go module where we could just swap the dependency then, the maintainers are not likely to pick up this enhancement without very strong use cases. Any public contribution must introduce the upstream cty wholesale without breaking any already existing exported hashicorp cty types.