Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mdatagen] fix missing/extraneous imports for telemetry codegen #11762

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

diurnalist
Copy link

Description

the telemetry code generation does not include some required imports and also includes some imports that are not used, which leads to build failures. this fixes those issues; re-running mdatagen on the samplereceiver results in a no-op, indicating that somehow there was drift b/w the output codegen and the templates.

Link to tracking issue

Fixes #11761

Testing

  • compiled and re-ran mdatagen on samplereceiver to check output

the telemetry code generation does not include some required imports
and also includes some imports that are not used, which leads to build failures.
this fixes those issues; re-running mdatagen on the samplereceiver results
in a no-op, indicating that somehow there was drift b/w the output codegen
and the templates.
@diurnalist diurnalist requested a review from a team as a code owner November 26, 2024 21:47
@dmathieu dmathieu added the Skip Changelog PRs that do not require a CHANGELOG.md entry label Nov 27, 2024
@dmathieu
Copy link
Member

mdatagen is used by many packages. When making changes to the templates, you should regenerate everything with the make gogenerate command, and ensure every package still works.

@diurnalist
Copy link
Author

@dmathieu thanks! I did run that, and there were no changes registered. I think that it doesn't currently run codegen on the examples, which might explain how this drift happened.

cd cmd/mdatagen && go install .
Running target 'generate' in module 'client'
/Library/Developer/CommandLineTools/usr/bin/make -C client generate
go generate ./...
Running target 'generate' in module 'cmd/builder'
/Library/Developer/CommandLineTools/usr/bin/make -C cmd/builder generate
go generate ./...
Running target 'generate' in module 'cmd/mdatagen'
/Library/Developer/CommandLineTools/usr/bin/make -C cmd/mdatagen generate
go generate ./...
Running target 'generate' in module 'cmd/otelcorecol'
/Library/Developer/CommandLineTools/usr/bin/make -C cmd/otelcorecol generate
go generate ./...
Running target 'generate' in module 'component'
/Library/Developer/CommandLineTools/usr/bin/make -C component generate
go generate ./...
Running target 'generate' in module 'component/componentstatus'
/Library/Developer/CommandLineTools/usr/bin/make -C component/componentstatus generate
go generate ./...
Running target 'generate' in module 'component/componenttest'
/Library/Developer/CommandLineTools/usr/bin/make -C component/componenttest generate
go generate ./...
Running target 'generate' in module 'config/configauth'
/Library/Developer/CommandLineTools/usr/bin/make -C config/configauth generate
go generate ./...
Running target 'generate' in module 'config/configcompression'
/Library/Developer/CommandLineTools/usr/bin/make -C config/configcompression generate
go generate ./...
Running target 'generate' in module 'config/configgrpc'
/Library/Developer/CommandLineTools/usr/bin/make -C config/configgrpc generate
go generate ./...
Running target 'generate' in module 'config/confighttp'
/Library/Developer/CommandLineTools/usr/bin/make -C config/confighttp generate
go generate ./...
Running target 'generate' in module 'config/confignet'
/Library/Developer/CommandLineTools/usr/bin/make -C config/confignet generate
go generate ./...
Running target 'generate' in module 'config/configopaque'
/Library/Developer/CommandLineTools/usr/bin/make -C config/configopaque generate
go generate ./...
Running target 'generate' in module 'config/configretry'
/Library/Developer/CommandLineTools/usr/bin/make -C config/configretry generate
go generate ./...
Running target 'generate' in module 'config/configtelemetry'
/Library/Developer/CommandLineTools/usr/bin/make -C config/configtelemetry generate
go generate ./...
Running target 'generate' in module 'config/configtls'
/Library/Developer/CommandLineTools/usr/bin/make -C config/configtls generate
go generate ./...
Running target 'generate' in module 'config/internal'
/Library/Developer/CommandLineTools/usr/bin/make -C config/internal generate
go generate ./...
Running target 'generate' in module 'confmap'
/Library/Developer/CommandLineTools/usr/bin/make -C confmap generate
go generate ./...
Running target 'generate' in module 'confmap/internal/e2e'
/Library/Developer/CommandLineTools/usr/bin/make -C confmap/internal/e2e generate
go generate ./...
Running target 'generate' in module 'confmap/provider/envprovider'
/Library/Developer/CommandLineTools/usr/bin/make -C confmap/provider/envprovider generate
go generate ./...
Running target 'generate' in module 'confmap/provider/fileprovider'
/Library/Developer/CommandLineTools/usr/bin/make -C confmap/provider/fileprovider generate
go generate ./...
Running target 'generate' in module 'confmap/provider/httpprovider'
/Library/Developer/CommandLineTools/usr/bin/make -C confmap/provider/httpprovider generate
go generate ./...
Running target 'generate' in module 'confmap/provider/httpsprovider'
/Library/Developer/CommandLineTools/usr/bin/make -C confmap/provider/httpsprovider generate
go generate ./...
Running target 'generate' in module 'confmap/provider/yamlprovider'
/Library/Developer/CommandLineTools/usr/bin/make -C confmap/provider/yamlprovider generate
go generate ./...
Running target 'generate' in module 'connector'
/Library/Developer/CommandLineTools/usr/bin/make -C connector generate
go generate ./...
Running target 'generate' in module 'connector/connectorprofiles'
/Library/Developer/CommandLineTools/usr/bin/make -C connector/connectorprofiles generate
go generate ./...
Running target 'generate' in module 'connector/connectortest'
/Library/Developer/CommandLineTools/usr/bin/make -C connector/connectortest generate
go generate ./...
Running target 'generate' in module 'connector/forwardconnector'
/Library/Developer/CommandLineTools/usr/bin/make -C connector/forwardconnector generate
go generate ./...
Running target 'generate' in module 'consumer'
/Library/Developer/CommandLineTools/usr/bin/make -C consumer generate
go generate ./...
Running target 'generate' in module 'consumer/consumererror'
/Library/Developer/CommandLineTools/usr/bin/make -C consumer/consumererror generate
go generate ./...
Running target 'generate' in module 'consumer/consumererror/consumererrorprofiles'
/Library/Developer/CommandLineTools/usr/bin/make -C consumer/consumererror/consumererrorprofiles generate
go generate ./...
Running target 'generate' in module 'consumer/consumerprofiles'
/Library/Developer/CommandLineTools/usr/bin/make -C consumer/consumerprofiles generate
go generate ./...
Running target 'generate' in module 'consumer/consumertest'
/Library/Developer/CommandLineTools/usr/bin/make -C consumer/consumertest generate
go generate ./...
Running target 'generate' in module 'exporter'
/Library/Developer/CommandLineTools/usr/bin/make -C exporter generate
go generate ./...
Running target 'generate' in module 'exporter/debugexporter'
/Library/Developer/CommandLineTools/usr/bin/make -C exporter/debugexporter generate
go generate ./...
Running target 'generate' in module 'exporter/exporterhelper/exporterhelperprofiles'
/Library/Developer/CommandLineTools/usr/bin/make -C exporter/exporterhelper/exporterhelperprofiles generate
go generate ./...
Running target 'generate' in module 'exporter/exporterprofiles'
/Library/Developer/CommandLineTools/usr/bin/make -C exporter/exporterprofiles generate
go generate ./...
Running target 'generate' in module 'exporter/exportertest'
/Library/Developer/CommandLineTools/usr/bin/make -C exporter/exportertest generate
go generate ./...
Running target 'generate' in module 'exporter/nopexporter'
/Library/Developer/CommandLineTools/usr/bin/make -C exporter/nopexporter generate
go generate ./...
Running target 'generate' in module 'exporter/otlpexporter'
/Library/Developer/CommandLineTools/usr/bin/make -C exporter/otlpexporter generate
go generate ./...
Running target 'generate' in module 'exporter/otlphttpexporter'
/Library/Developer/CommandLineTools/usr/bin/make -C exporter/otlphttpexporter generate
go generate ./...
Running target 'generate' in module 'extension'
/Library/Developer/CommandLineTools/usr/bin/make -C extension generate
go generate ./...
Running target 'generate' in module 'extension/auth'
/Library/Developer/CommandLineTools/usr/bin/make -C extension/auth generate
go generate ./...
Running target 'generate' in module 'extension/auth/authtest'
/Library/Developer/CommandLineTools/usr/bin/make -C extension/auth/authtest generate
go generate ./...
Running target 'generate' in module 'extension/experimental/storage'
/Library/Developer/CommandLineTools/usr/bin/make -C extension/experimental/storage generate
go generate ./...
Running target 'generate' in module 'extension/extensioncapabilities'
/Library/Developer/CommandLineTools/usr/bin/make -C extension/extensioncapabilities generate
go generate ./...
Running target 'generate' in module 'extension/extensiontest'
/Library/Developer/CommandLineTools/usr/bin/make -C extension/extensiontest generate
go generate ./...
Running target 'generate' in module 'extension/memorylimiterextension'
/Library/Developer/CommandLineTools/usr/bin/make -C extension/memorylimiterextension generate
go generate ./...
Running target 'generate' in module 'extension/zpagesextension'
/Library/Developer/CommandLineTools/usr/bin/make -C extension/zpagesextension generate
go generate ./...
Running target 'generate' in module 'featuregate'
/Library/Developer/CommandLineTools/usr/bin/make -C featuregate generate
go generate ./...
Running target 'generate' in module 'filter'
/Library/Developer/CommandLineTools/usr/bin/make -C filter generate
go generate ./...
Running target 'generate' in module 'internal/e2e'
/Library/Developer/CommandLineTools/usr/bin/make -C internal/e2e generate
go generate ./...
Running target 'generate' in module 'internal/fanoutconsumer'
/Library/Developer/CommandLineTools/usr/bin/make -C internal/fanoutconsumer generate
go generate ./...
Running target 'generate' in module 'internal/memorylimiter'
/Library/Developer/CommandLineTools/usr/bin/make -C internal/memorylimiter generate
go generate ./...
Running target 'generate' in module 'internal/sharedcomponent'
/Library/Developer/CommandLineTools/usr/bin/make -C internal/sharedcomponent generate
go generate ./...
Running target 'generate' in module 'internal/tools'
/Library/Developer/CommandLineTools/usr/bin/make -C internal/tools generate
go generate ./...
Running target 'generate' in module 'otelcol'
/Library/Developer/CommandLineTools/usr/bin/make -C otelcol generate
go generate ./...
Running target 'generate' in module 'otelcol/otelcoltest'
/Library/Developer/CommandLineTools/usr/bin/make -C otelcol/otelcoltest generate
go generate ./...
Running target 'generate' in module 'pdata'
/Library/Developer/CommandLineTools/usr/bin/make -C pdata generate
go generate ./...
Running target 'generate' in module 'pdata/pprofile'
/Library/Developer/CommandLineTools/usr/bin/make -C pdata/pprofile generate
go generate ./...
Running target 'generate' in module 'pdata/testdata'
/Library/Developer/CommandLineTools/usr/bin/make -C pdata/testdata generate
go generate ./...
Running target 'generate' in module 'pipeline'
/Library/Developer/CommandLineTools/usr/bin/make -C pipeline generate
go generate ./...
Running target 'generate' in module 'pipeline/pipelineprofiles'
/Library/Developer/CommandLineTools/usr/bin/make -C pipeline/pipelineprofiles generate
go generate ./...
Running target 'generate' in module 'processor'
/Library/Developer/CommandLineTools/usr/bin/make -C processor generate
go generate ./...
Running target 'generate' in module 'processor/batchprocessor'
/Library/Developer/CommandLineTools/usr/bin/make -C processor/batchprocessor generate
go generate ./...
Running target 'generate' in module 'processor/memorylimiterprocessor'
/Library/Developer/CommandLineTools/usr/bin/make -C processor/memorylimiterprocessor generate
go generate ./...
Running target 'generate' in module 'processor/processorhelper/processorhelperprofiles'
/Library/Developer/CommandLineTools/usr/bin/make -C processor/processorhelper/processorhelperprofiles generate
go generate ./...
Running target 'generate' in module 'processor/processorprofiles'
/Library/Developer/CommandLineTools/usr/bin/make -C processor/processorprofiles generate
go generate ./...
Running target 'generate' in module 'processor/processortest'
/Library/Developer/CommandLineTools/usr/bin/make -C processor/processortest generate
go generate ./...
Running target 'generate' in module 'receiver'
/Library/Developer/CommandLineTools/usr/bin/make -C receiver generate
go generate ./...
Running target 'generate' in module 'receiver/nopreceiver'
/Library/Developer/CommandLineTools/usr/bin/make -C receiver/nopreceiver generate
go generate ./...
Running target 'generate' in module 'receiver/otlpreceiver'
/Library/Developer/CommandLineTools/usr/bin/make -C receiver/otlpreceiver generate
go generate ./...
Running target 'generate' in module 'receiver/receiverprofiles'
/Library/Developer/CommandLineTools/usr/bin/make -C receiver/receiverprofiles generate
go generate ./...
Running target 'generate' in module 'receiver/receivertest'
/Library/Developer/CommandLineTools/usr/bin/make -C receiver/receivertest generate
go generate ./...
Running target 'generate' in module 'scraper'
/Library/Developer/CommandLineTools/usr/bin/make -C scraper generate
go generate ./...
Running target 'generate' in module 'semconv'
/Library/Developer/CommandLineTools/usr/bin/make -C semconv generate
go generate ./...
Running target 'generate' in module 'service'
/Library/Developer/CommandLineTools/usr/bin/make -C service generate
go generate ./...
Running target 'generate' in module '/Users/jason.anderson/dev/src/github.com/diurnalist/opentelemetry-collector'
/Library/Developer/CommandLineTools/usr/bin/make -C /Users/jason.anderson/dev/src/github.com/diurnalist/opentelemetry-collector generate
go generate ./...
/Library/Developer/CommandLineTools/usr/bin/make fmt
mkdir -p /Users/jason.anderson/dev/src/github.com/diurnalist/opentelemetry-collector/.tools
cd /Users/jason.anderson/dev/src/github.com/diurnalist/opentelemetry-collector/internal/tools && go build -o /Users/jason.anderson/dev/src/github.com/diurnalist/opentelemetry-collector/.tools/goimports -trimpath golang.org/x/tools/cmd/goimports
gofmt -w -s ./
/Users/jason.anderson/dev/src/github.com/diurnalist/opentelemetry-collector/.tools/goimports -w  -local go.opentelemetry.io/collector ./

@diurnalist
Copy link
Author

Ah, I think I may know how this was missed - make gogenerate runs fmt and goimports after it does the codegen, which fixes the bad imports automatically behind the scenes. Hmm, perhaps we should not run that on that target to ensure the output of the codegen is what we expect?

@dmathieu
Copy link
Member

I think gogenerate should keep running gofmt. Templates can be harder to properly format than actual Go files.
Removing goimport from there may make sense though.

Your PR still has failing tests though.

@diurnalist
Copy link
Author

Ah, thanks - I believe I have fixed up the issue in the template. Also added a new test.

Copy link

codecov bot commented Dec 2, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.48%. Comparing base (c52d625) to head (5c2cd99).
Report is 108 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #11762      +/-   ##
==========================================
+ Coverage   91.45%   91.48%   +0.02%     
==========================================
  Files         447      447              
  Lines       23721    23721              
==========================================
+ Hits        21694    21700       +6     
+ Misses       1653     1649       -4     
+ Partials      374      372       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mmorel-35
Copy link
Contributor

mmorel-35 commented Dec 9, 2024

Maybe worth adding alias
noopmetric "go.opentelemetry.io/otel/metric/noop"

As it it done with the other generated files ?

Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added Stale and removed Stale labels Dec 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Skip Changelog PRs that do not require a CHANGELOG.md entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[mdatagen] telemetry codegen outputs illegal Go files
4 participants