Collector panics on startup for non-nilable config options with unset environment variable

[crobert-1]

Describe the bug

When the collector is started with a configuration that has a non-nilable config option (e.g. int or uint64) set to an environment variable that hasn't been set, the collector panics without any information other than the stack.

Steps to reproduce

Config:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:8518

exporters:
  debug:
    verbosity: detailed
    sampling_initial: ${UNSET_VARIABLE}

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: []
      exporters: [debug]

Start the collector with an int configuration option, in this case the debug exporter's sampling_initial option, set to the environment variable UNSET_VARIABLE, which isn't set.

What did you expect to see?

No panic. I'm not sure if intended behavior here is to error and force the user to set the value of the variable, or to start with a default value, such as 0 for number types.

What did you see instead?

$ ./bin/otelcorecol_darwin_amd64 --config config.yaml 
panic: reflect: call of reflect.Value.Interface on zero Value

goroutine 1 [running]:
reflect.valueInterface({0x0?, 0x0?, 0xc00041c528?}, 0x85?)
	reflect/value.go:1501 +0xfb
reflect.Value.Interface({0x0?, 0x0?, 0xc0003f5400?})
	reflect/value.go:1496 +0x25
go.opentelemetry.io/collector/confmap.decodeConfig.expandNilStructPointersHookFunc.func2({0x0?, 0x0?, 0xa0b8020?}, {0xa0b8020?, 0xc0003b32c8?, 0xc0003f5480?})
	go.opentelemetry.io/collector/confmap@v0.107.0/confmap.go:306 +0x131
github.com/go-viper/mapstructure/v2.cachedDecodeHook.func3({0x0?, 0x0?, 0x5?}, {0xa0b8020?, 0xc0003b32c8?, 0xc00041c650?})
	github.com/go-viper/mapstructure/v2@v2.1.0/decode_hooks.go:54 +0x2c
github.com/go-viper/mapstructure/v2.ComposeDecodeHookFunc.func1({0xa19d580?, 0xc0003f5480?, 0xc00030a270?}, {0xa0b8020?, 0xc0003b32c8?, 0xc00030a270?})
	github.com/go-viper/mapstructure/v2@v2.1.0/decode_hooks.go:98 +0xdc
github.com/go-viper/mapstructure/v2.cachedDecodeHook.func3({0xa19d580?, 0xc0003f5480?, 0xc0003fe601?}, {0xa0b8020?, 0xc0003b32c8?, 0xc00041c700?})
	github.com/go-viper/mapstructure/v2@v2.1.0/decode_hooks.go:54 +0x2c
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc00030a030, {0xa082da9, 0x10}, {0xa19d580?, 0xc0003f5480?}, {0xa0b8020?, 0xc0003b32c8?, 0x22?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:480 +0x12e
github.com/go-viper/mapstructure/v2.(*Decoder).decodeStructFromMap(0xc00030a030, {0x0, 0x0}, {0xa11cb00?, 0xc0003f1b90?, 0xa200180?}, {0xa200180?, 0xc0003b32c0?, 0xc00041d160?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:1462 +0xb10
github.com/go-viper/mapstructure/v2.(*Decoder).decodeStruct(0xc00030a030, {0x0, 0x0}, {0xa11cb00?, 0xc0003f1b90?}, {0xa200180?, 0xc0003b32c0?, 0xc00041d228?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:1278 +0x55f
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc00030a030, {0x0, 0x0}, {0xa11cb00?, 0xc0003f1b90?}, {0xa200180?, 0xc0003b32c0?, 0xc0002a2e98?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:505 +0x27f
github.com/go-viper/mapstructure/v2.(*Decoder).decodePtr(0xc00030a030, {0x0, 0x0}, {0xa11cb00, 0xc0003f1b90}, {0xa10bb80?, 0xc0002a2e98?, 0xc00041d388?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:1104 +0x3f6
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc00030a030, {0x0, 0x0}, {0xa11cb00?, 0xc0003f1b90?}, {0xa10bb80?, 0xc0002a2e98?, 0xc0002a2e98?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:509 +0x255
github.com/go-viper/mapstructure/v2.(*Decoder).decodePtr(0xc00030a030, {0x0, 0x0}, {0xa11cb00, 0xc0003f1b90}, {0xc0003f24c0?, 0xc0002a2e98?, 0xab1e740?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:1110 +0x49d
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc00030a030, {0x0, 0x0}, {0xa11cb00?, 0xc0003f1b90?}, {0xc0003f24c0?, 0xc0002a2e98?, 0xa11cb00?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:509 +0x255
github.com/go-viper/mapstructure/v2.(*Decoder).decodeBasic(0xc00030a030, {0x0, 0x0}, {0xa11cb00, 0xc0003f1b90}, {0xa1313a0?, 0xc0003edf00?, 0xc0003b2e12?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:555 +0x61f
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc00030a030, {0x0, 0x0}, {0xa11cb00?, 0xc0003f1b90?}, {0xa1313a0?, 0xc0003edf00?, 0x0?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:493 +0x23d
github.com/go-viper/mapstructure/v2.(*Decoder).Decode(0xc00030a030, {0xa11cb00, 0xc0003f1b90})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:430 +0xbb
go.opentelemetry.io/collector/confmap.decodeConfig(0xc0003edd50, {0xa088e80, 0xc0003edf00}, 0x1, 0x0)
	go.opentelemetry.io/collector/confmap@v0.107.0/confmap.go:211 +0x2cf
go.opentelemetry.io/collector/confmap.(*Conf).Unmarshal(0xc0003edd50, {0xa088e80, 0xc0003edf00}, {0x0, 0x0, 0x10?})
	go.opentelemetry.io/collector/confmap@v0.107.0/confmap.go:88 +0xa5
go.opentelemetry.io/collector/otelcol/internal/configunmarshaler.(*Configs[...]).Unmarshal(0xa2ccca0, 0xc0003ed330)
	go.opentelemetry.io/collector/otelcol@v0.107.0/internal/configunmarshaler/configs.go:52 +0x285
go.opentelemetry.io/collector/confmap.decodeConfig.unmarshalerHookFunc.func7({0xa11cb00?, 0xc0003d9770?, 0xa1af280?}, {0xa1af280?, 0xc0003db520?, 0xc0003d9770?})
	go.opentelemetry.io/collector/confmap@v0.107.0/confmap.go:422 +0x271
github.com/go-viper/mapstructure/v2.cachedDecodeHook.func3({0xa11cb00?, 0xc0003d9770?, 0x5?}, {0xa1af280?, 0xc0003db520?, 0xc00041dd40?})
	github.com/go-viper/mapstructure/v2@v2.1.0/decode_hooks.go:54 +0x2c
github.com/go-viper/mapstructure/v2.ComposeDecodeHookFunc.func1({0xa11cb00?, 0xc0003d9770?, 0xc0003ed320?}, {0xa1af280?, 0xc0003db520?, 0xc0003ed320?})
	github.com/go-viper/mapstructure/v2@v2.1.0/decode_hooks.go:98 +0xdc
github.com/go-viper/mapstructure/v2.cachedDecodeHook.func3({0xa11cb00?, 0xc0003d9770?, 0xc0003e6601?}, {0xa1af280?, 0xc0003db520?, 0xc00041ddf0?})
	github.com/go-viper/mapstructure/v2@v2.1.0/decode_hooks.go:54 +0x2c
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc0003db5f0, {0xa0712bd, 0x9}, {0xa11cb00?, 0xc0003d9770?}, {0xa1af280?, 0xc0003db520?, 0x16?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:480 +0x12e
github.com/go-viper/mapstructure/v2.(*Decoder).decodeStructFromMap(0xc0003db5f0, {0x0, 0x0}, {0xa11cb00?, 0xc0003d9680?, 0xa230480?}, {0xa230480?, 0xc0003e6480?, 0xc00041e850?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:1462 +0xb10
github.com/go-viper/mapstructure/v2.(*Decoder).decodeStruct(0xc0003db5f0, {0x0, 0x0}, {0xa11cb00?, 0xc0003d9680?}, {0xa230480?, 0xc0003e6480?, 0xc00041e918?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:1278 +0x55f
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc0003db5f0, {0x0, 0x0}, {0xa11cb00?, 0xc0003d9680?}, {0xa230480?, 0xc0003e6480?, 0xc0002a2ba8?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:505 +0x27f
github.com/go-viper/mapstructure/v2.(*Decoder).decodePtr(0xc0003db5f0, {0x0, 0x0}, {0xa11cb00, 0xc0003d9680}, {0xa089400?, 0xc0002a2ba8?, 0xc00029cce2?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:1104 +0x3f6
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc0003db5f0, {0x0, 0x0}, {0xa11cb00?, 0xc0003d9680?}, {0xa089400?, 0xc0002a2ba8?, 0x0?})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:509 +0x255
github.com/go-viper/mapstructure/v2.(*Decoder).Decode(0xc0003db5f0, {0xa11cb00, 0xc0003d9680})
	github.com/go-viper/mapstructure/v2@v2.1.0/mapstructure.go:430 +0xbb
go.opentelemetry.io/collector/confmap.decodeConfig(0xc0003db0e0, {0xa0a2bc0, 0xc0002a2ba8}, 0x1, 0x0)
	go.opentelemetry.io/collector/confmap@v0.107.0/confmap.go:211 +0x2cf
go.opentelemetry.io/collector/confmap.(*Conf).Unmarshal(0xc0003db0e0, {0xa0a2bc0, 0xc0002a2ba8}, {0x0, 0x0, 0xc0002a2b98?})
	go.opentelemetry.io/collector/confmap@v0.107.0/confmap.go:88 +0xa5
go.opentelemetry.io/collector/otelcol.unmarshal(0xc0003db0e0, {0xc000391c20, 0xc000391d10, 0xc000391c80, 0xc000391b30, 0xc000391da0, 0xc000391c50, 0xc000391d40, 0xc000391cb0, 0xc000391b60, ...})
	go.opentelemetry.io/collector/otelcol@v0.107.0/unmarshaler.go:47 +0x2ff
go.opentelemetry.io/collector/otelcol.(*configProvider).Get(0x4?, {0xa2d1d30?, 0xab838a0?}, {0xc000391c20, 0xc000391d10, 0xc000391c80, 0xc000391b30, 0xc000391da0, 0xc000391c50, 0xc000391d40, ...})
	go.opentelemetry.io/collector/otelcol@v0.107.0/configprovider.go:107 +0xb8
go.opentelemetry.io/collector/otelcol.(*Collector).setupConfigurationComponents(0xc000363b00, {0xa2d1d30, 0xab838a0})
	go.opentelemetry.io/collector/otelcol@v0.107.0/collector.go:170 +0x166
go.opentelemetry.io/collector/otelcol.(*Collector).Run(0xc000363b00, {0xa2d1d30, 0xab838a0})
	go.opentelemetry.io/collector/otelcol@v0.107.0/collector.go:272 +0x55
go.opentelemetry.io/collector/otelcol.NewCommand.func1(0xc000356908, {0x9e70af5?, 0x7?, 0x9e6dbc5?})
	go.opentelemetry.io/collector/otelcol@v0.107.0/command.go:37 +0xa7
github.com/spf13/cobra.(*Command).execute(0xc000356908, {0xc00019a190, 0x2, 0x2})
	github.com/spf13/cobra@v1.8.1/command.go:985 +0xaca
github.com/spf13/cobra.(*Command).ExecuteC(0xc000356908)
	github.com/spf13/cobra@v1.8.1/command.go:1117 +0x3ff
github.com/spf13/cobra.(*Command).Execute(0xa2bae30?)
	github.com/spf13/cobra@v1.8.1/command.go:1041 +0x13
main.runInteractive({0xa2bae30, {{0x9e747b3, 0xb}, {0x9ea2c94, 0x33}, {0x9e747be, 0xb}}, 0x0, {{{0x0, 0x0, ...}, ...}}, ...})
	go.opentelemetry.io/collector/cmd/otelcorecol/main.go:49 +0x5d
main.run(...)
	go.opentelemetry.io/collector/cmd/otelcorecol/main_others.go:10
main.main()
	go.opentelemetry.io/collector/cmd/otelcorecol/main.go:42 +0x1b8

What version did you use?

v0.107.0 and v0.106.1

Environment

This has been hit on barebones macos/amd64 and in k8s as well.

[TylerHelmuth]

The problem is happening in expandNilStructPointersHookFunc because the from reflect.Value is nil.

This value is nil because the previous hook, useExpandValue, is converting the expandedValue for sampling_initial to nil, since exp.Value is nil and the target type is not a string.

exp.Value is nil because in expand.go, the ret returned by mr.expandURI(ctx, input) has a nil rawConf. The rawConf is nil because the env var didn't contain anything so the yaml.Unmarshal call here had no bytes to set for &rawConf.

I don't yet see a solution to this issue, but thats the why.

@mx-psi maybe the return value in useExpandValue, or all of the expandedValue type, needs to understand more about default values for primitive types?