copywrite EOL proto headers debugging #19799

… initial xDS… into release/1.15.x (#17361) * backport of commit 530d192 * backport of commit 13f9965 --------- Co-authored-by: DanStough <dan.stough@hashicorp.com>

Co-authored-by: Krastin Krastev <krastin@hashicorp.com>

Co-authored-by: Dan Bond <danbond@protonmail.com>

* backport of commit 0afff9e * backport of commit fe56513 * backport of commit 49a4951 * backport of commit 1995a5f * backport of commit 1f39bdf * backport of commit 5a2e89f --------- Co-authored-by: trujillo-adam <ajosetru@gmail.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

…ease/1.15.x (#17338) * no-op commit due to failed cherry-picking * docs: connect-service-upstreams annotation fixes (#17312) * corrections * fixes * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Switching order of labeled/unlabeled --------- Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* no-op commit due to failed cherry-picking * docs: connect-service-upstreams annotation fixes (#17312) * corrections * fixes * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Update website/content/docs/k8s/annotations-and-labels.mdx Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Switching order of labeled/unlabeled --------- Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * Fix merge conflict --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

Co-authored-by: trujillo-adam <ajosetru@gmail.com>

…17412) * Rename hcp-metrics-collector to consul-telemetry-collector * Fix docs * Fix doc comment --------- Co-authored-by: Connor <connor.kelly@hashicorp.com>

…lver config entries into release/1.15.x (#17397) * no-op commit due to failed cherry-picking * docs: Reference pages for service-router and service-resolver config entries (#17145) * service-resolve configuration entry reference * Updates * missing backtick * service router configuration entry reference * link fixes + tab fixes * link and tab fixes * link fixes * service resolver improvements * hierarchy fixes * spacing * links + formatting * proofing fixes * mmore fixes * Apply suggestions from code review suggestions from code review for service resolver Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * policy sections edits * service router code review * Tables to sections - service router HCL * YAML tables to sections * formatting fixes * converting tables to sections - service resolver * final tables to sections * Adjustments/alignments * nanosecond fix * Update website/content/docs/connect/config-entries/service-router.mdx Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * link to filter example config --------- Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * Update website/content/docs/connect/config-entries/service-resolver.mdx * merge fix --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> Co-authored-by: boruszak <jeffrey.boruszak@hashicorp.com>

…#17417) manual backport of #17416 to 1.15.x

* WIP * ci:upload test results to datadog * fix use of envvar in expression * getting correct permission in reusable-unit.yml * getting correct permission in reusable-unit.yml * fixing DATADOG_API_KEY envvar expresssion * pass datadog-api-key * removing type from datadog-api-key

…e/1.15.x (#17431) * backport of commit 1e48592 * Add integration test for wan fed issue * Add changelog --------- Co-authored-by: Paul Glass <pglass@hashicorp.com>

)

* backport of feat: initial grpc blocking queries (11b1dd2) * backport of changelog and docs update (5ceef20) --------- Co-authored-by: DanStough <dan.stough@hashicorp.com>

Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>

…se/1.15.x (#17458) * backport of commit dc995c1 * backport of commit 169a327 --------- Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>

Co-authored-by: cskh <hui.kang@hashicorp.com>

…mbda (#17432) backport of commit ba18381

Co-authored-by: DanStough <dan.stough@hashicorp.com>

Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>

Co-authored-by: John Murret <john.murret@hashicorp.com>

… into release/1.15.x (#17493) * backport of commit 04b6a90 * backport of commit 13e4057 --------- Co-authored-by: Lincoln Stoll <me@lds.li> Co-authored-by: John Murret <john.murret@hashicorp.com>

* Move hcp client to subpackage hcpclient (#16800) * [HCP Observability] New MetricsClient (#17100) * Client configured with TLS using HCP config and retry/throttle * Add tests and godoc for metrics client * close body after request * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * remove clone * Extract CloudConfig and mock for future PR * Switch to hclog.FromContext * [HCP Observability] OTELExporter (#17128) * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * [HCP Observability] OTELSink (#17159) * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Initialize OTELSink with sync.Map for all the instrument stores. * Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests. * Switch to mutex instead of sync.Map to avoid type assertion * Add gauge store * Clarify comments * return concrete sink type * Fix lint errors * Move gauge store to be within sink * Use context.TODO,rebase and clenaup opts handling * Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1 * Fix imports * Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx * Add lots of documentation to the OTELSink * Fix gauge store comment and check ok * Add select and ctx.Done() check to gauge callback * use require.Equal for attributes * Fixed import naming * Remove float64 calls and add a NewGaugeStore method * Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store * Generate 100 gauge operations * Seperate the labels into goroutines in sink test * Generate kv store for the test case keys to avoid using uuid * Added a race test with 300 samples for OTELSink * Do not pass in waitgroup and use error channel instead. * Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel * Fix nits * [HCP Observability] Init OTELSink in Telemetry (#17162) * Move hcp client to subpackage hcpclient (#16800) * [HCP Observability] New MetricsClient (#17100) * Client configured with TLS using HCP config and retry/throttle * Add tests and godoc for metrics client * close body after request * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * remove clone * Extract CloudConfig and mock for future PR * Switch to hclog.FromContext * [HCP Observability] New MetricsClient (#17100) * Client configured with TLS using HCP config and retry/throttle * Add tests and godoc for metrics client * close body after request * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * remove clone * Extract CloudConfig and mock for future PR * Switch to hclog.FromContext * [HCP Observability] New MetricsClient (#17100) * Client configured with TLS using HCP config and retry/throttle * Add tests and godoc for metrics client * close body after request * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * remove clone * Extract CloudConfig and mock for future PR * Switch to hclog.FromContext * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Initialize OTELSink with sync.Map for all the instrument stores. * Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests. * Switch to mutex instead of sync.Map to avoid type assertion * Add gauge store * Clarify comments * return concrete sink type * Fix lint errors * Move gauge store to be within sink * Use context.TODO,rebase and clenaup opts handling * Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1 * Fix imports * Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx * Add lots of documentation to the OTELSink * Fix gauge store comment and check ok * Add select and ctx.Done() check to gauge callback * use require.Equal for attributes * Fixed import naming * Remove float64 calls and add a NewGaugeStore method * Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store * Generate 100 gauge operations * Seperate the labels into goroutines in sink test * Generate kv store for the test case keys to avoid using uuid * Added a race test with 300 samples for OTELSink * [HCP Observability] OTELExporter (#17128) * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * Do not pass in waitgroup and use error channel instead. * Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel * Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1 * Initialize OTELSink with sync.Map for all the instrument stores. * Added telemetry agent to client and init sink in deps * Fixed client * Initalize sink in deps * init sink in telemetry library * Init deps before telemetry * Use concrete telemetry.OtelSink type * add /v1/metrics * Avoid returning err for telemetry init * move sink init within the IsCloudEnabled() * Use HCPSinkOpts in deps instead * update golden test for configuration file * Switch to using extra sinks in the telemetry library * keep name MetricsConfig * fix log in verifyCCMRegistration * Set logger in context * pass around MetricSink in deps * Fix imports * Rebased onto otel sink pr * Fix URL in test * [HCP Observability] OTELSink (#17159) * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Initialize OTELSink with sync.Map for all the instrument stores. * Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests. * Switch to mutex instead of sync.Map to avoid type assertion * Add gauge store * Clarify comments * return concrete sink type * Fix lint errors * Move gauge store to be within sink * Use context.TODO,rebase and clenaup opts handling * Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1 * Fix imports * Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx * Add lots of documentation to the OTELSink * Fix gauge store comment and check ok * Add select and ctx.Done() check to gauge callback * use require.Equal for attributes * Fixed import naming * Remove float64 calls and add a NewGaugeStore method * Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store * Generate 100 gauge operations * Seperate the labels into goroutines in sink test * Generate kv store for the test case keys to avoid using uuid * Added a race test with 300 samples for OTELSink * Do not pass in waitgroup and use error channel instead. * Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel * Fix nits * pass extraSinks as function param instead * Add default interval as package export * remove verifyCCM func * Add clusterID * Fix import and add t.Parallel() for missing tests * Kick Vercel CI * Remove scheme from endpoint path, and fix error logging * return metrics.MetricSink for sink method * Update SDK * [HCP Observability] Metrics filtering and Labels in Go Metrics sink (#17184) * Move hcp client to subpackage hcpclient (#16800) * [HCP Observability] New MetricsClient (#17100) * Client configured with TLS using HCP config and retry/throttle * Add tests and godoc for metrics client * close body after request * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * remove clone * Extract CloudConfig and mock for future PR * Switch to hclog.FromContext * [HCP Observability] New MetricsClient (#17100) * Client configured with TLS using HCP config and retry/throttle * Add tests and godoc for metrics client * close body after request * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * remove clone * Extract CloudConfig and mock for future PR * Switch to hclog.FromContext * [HCP Observability] New MetricsClient (#17100) * Client configured with TLS using HCP config and retry/throttle * Add tests and godoc for metrics client * close body after request * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * remove clone * Extract CloudConfig and mock for future PR * Switch to hclog.FromContext * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Initialize OTELSink with sync.Map for all the instrument stores. * Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests. * Switch to mutex instead of sync.Map to avoid type assertion * Add gauge store * Clarify comments * return concrete sink type * Fix lint errors * Move gauge store to be within sink * Use context.TODO,rebase and clenaup opts handling * Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1 * Fix imports * Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx * Add lots of documentation to the OTELSink * Fix gauge store comment and check ok * Add select and ctx.Done() check to gauge callback * use require.Equal for attributes * Fixed import naming * Remove float64 calls and add a NewGaugeStore method * Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store * Generate 100 gauge operations * Seperate the labels into goroutines in sink test * Generate kv store for the test case keys to avoid using uuid * Added a race test with 300 samples for OTELSink * [HCP Observability] OTELExporter (#17128) * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * Do not pass in waitgroup and use error channel instead. * Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel * Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1 * Initialize OTELSink with sync.Map for all the instrument stores. * Added telemetry agent to client and init sink in deps * Fixed client * Initalize sink in deps * init sink in telemetry library * Init deps before telemetry * Use concrete telemetry.OtelSink type * add /v1/metrics * Avoid returning err for telemetry init * move sink init within the IsCloudEnabled() * Use HCPSinkOpts in deps instead * update golden test for configuration file * Switch to using extra sinks in the telemetry library * keep name MetricsConfig * fix log in verifyCCMRegistration * Set logger in context * pass around MetricSink in deps * Fix imports * Rebased onto otel sink pr * Fix URL in test * [HCP Observability] OTELSink (#17159) * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Initialize OTELSink with sync.Map for all the instrument stores. * Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests. * Switch to mutex instead of sync.Map to avoid type assertion * Add gauge store * Clarify comments * return concrete sink type * Fix lint errors * Move gauge store to be within sink * Use context.TODO,rebase and clenaup opts handling * Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1 * Fix imports * Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx * Add lots of documentation to the OTELSink * Fix gauge store comment and check ok * Add select and ctx.Done() check to gauge callback * use require.Equal for attributes * Fixed import naming * Remove float64 calls and add a NewGaugeStore method * Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store * Generate 100 gauge operations * Seperate the labels into goroutines in sink test * Generate kv store for the test case keys to avoid using uuid * Added a race test with 300 samples for OTELSink * Do not pass in waitgroup and use error channel instead. * Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel * Fix nits * pass extraSinks as function param instead * Add default interval as package export * remove verifyCCM func * Add clusterID * Fix import and add t.Parallel() for missing tests * Kick Vercel CI * Remove scheme from endpoint path, and fix error logging * return metrics.MetricSink for sink method * Update SDK * Added telemetry agent to client and init sink in deps * Add node_id and __replica__ default labels * add function for default labels and set x-hcp-resource-id * Fix labels tests * Commit suggestion for getDefaultLabels * Fixed server.id, and t.Parallel() * Make defaultLabels a method on the TelemetryConfig object * Rename FilterList to lowercase filterList * Cleanup filter implemetation by combining regex into a single one, and making the type lowercase * Fix append * use regex directly for filters * Fix x-resource-id test to use mocked value * Fix log.Error formats * Forgot the len(opts.Label) optimization) * Use cfg.NodeID instead --------- * remove replic tag (#17484) * [HCP Observability] Add custom metrics for OTEL sink, improve logging, upgrade modules and cleanup metrics client (#17455) * Add custom metrics for Exporter and transform operations * Improve deps logging Run go mod tidy * Upgrade SDK and OTEL * Remove the partial success implemetation and check for HTTP status code in metrics client * Add x-channel * cleanup logs in deps.go based on PR feedback * Change to debug log and lowercase * address test operation feedback * use GetHumanVersion on version * Fix error wrapping * Fix metric names * [HCP Observability] Turn off retries for now until dynamically configurable (#17496) * Remove retries for now until dynamic configuration is possible * Clarify comment * Update changelog * improve changelog --------- Co-authored-by: Joshua Timmons <joshua.timmons1@gmail.com>

Co-authored-by: Morgan Drake <12264057+modrake@users.noreply.github.com>

Co-authored-by: DanStough <dan.stough@hashicorp.com>

Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com>

Co-authored-by: Poonam Jadhav <poonam.jadhav@hashicorp.com>

…luding cpu memory and disk usage into release/1.15.x (#17532) * backport of commit fafb68b * backport of commit baffef4 * backport of commit 369619b * backport of commit b7f5032 * backport of commit ac60717 * backport of commit 2a3dd50 * backport of commit 50621fa * backport of commit 6e3ceb0 * backport of commit bf69a82 --------- Co-authored-by: Nick Ethier <nethier@hashicorp.com>

…into release/1.15.x (#17536) * backport of commit ee9fd90 * backport of commit 6bc25b5 --------- Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>

#17539) backport of commit 8c30455

Co-authored-by: Dan Stough <Daniel.Stough@gmail.com>

Co-authored-by: Luiz Aoqui <luiz@hashicorp.com>

Co-authored-by: Poonam Jadhav <poonam.jadhav@hashicorp.com>

Co-authored-by: Dave Rawks <dave@rawks.io>

…1.15.x (#17473) * backport of commit 6e53750 * backport of commit fbdab01 * trigger ci --------- Co-authored-by: Hans Hasselberg <hans@hashicorp.com>

…changing or a snapshot restore occurring into release/1.15.x (#17567) * backport of commit 82e7d4f * backport of commit 45008e2 * backport of commit bdee9e3 --------- Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>

…tion into release/1.15.x (#17583) * backport of commit 7c2c221 * backport of commit fc0053f --------- Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>

…ics into release/1.15.x (#17580) * backport of commit 81fe71d * backport of commit 8d6b707 --------- Co-authored-by: cskh <hui.kang@hashicorp.com>

Fix some linter warnings before updating the lint-consul-retry code in hashicorp/lint-consul-retry#4 backport of m #17600 onto release/1.15.x Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

…es in synthesis code into release/1.15.x (#17615) * backport of commit 63e8743 * backport of commit 896f117 --------- Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>

Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>

…ease/1.15.x (#17634) * backport of commit fb2f3b6 * backport of commit 178abb8 * backport of commit 77b3998 * backport of commit a245b32 --------- Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>

…che (#17636)

…elease/1.15.x (#17451) * backport of commit ce702d7 * CI: do not upload test metrics to datadog on forks (#17448) * bypass uploading testmetrics to datadog on forks / when api is not available * add exclusions to upgrade tests --------- Co-authored-by: John Murret <john.murret@hashicorp.com>

… release/1.15.x (#17649) * no-op commit due to failed cherry-picking * remove legacy links format --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: Bryce Kalow <bkalow@hashicorp.com>

…/1.15.x (#17466) * no-op commit due to failed cherry-picking * docs: update the defaults for request limits (#17437) --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: John Murret <john.murret@hashicorp.com>

Co-authored-by: Poonam Jadhav <poonam.jadhav@hashicorp.com>

Co-authored-by: Hariram Sankaran <56744845+ramramhariram@users.noreply.github.com>

…ase/1.15.x (#17682) * backport of commit 0191cb1 * backport of commit 6c245e7 --------- Co-authored-by: josh <josh.timmons@hashicorp.com>

…ase/1.15.x (#17590) * backport of commit 8144eb0 * backport of commit 4d3f2e0 --------- Co-authored-by: josh <josh.timmons@hashicorp.com>

Co-authored-by: DanStough <dan.stough@hashicorp.com>

Co-authored-by: cskh <hui.kang@hashicorp.com>

Co-authored-by: Maliz <maliheh.monshizadeh@hashicorp.com>

Co-authored-by: DanStough <dan.stough@hashicorp.com>

…lease/1.15.x (#17717) * backport of commit c02c444 * backport of commit 3237e24 * backport of commit 3bbd88f --------- Co-authored-by: David Yu <dyu@hashicorp.com>

Co-authored-by: Tobias Birkefeld <t@craxs.de>

Co-authored-by: Paul Glass <pglass@hashicorp.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>

Co-authored-by: Chris Chapman <cchapman@hashicorp.com>

Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>

Co-authored-by: John Murret <john.murret@hashicorp.com>

…7766) * no-op commit due to failed cherry-picking * added redirects and updated links (#17764) cherry-picking fixes for broken links --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

Co-authored-by: Mark Campbell-Vincent <mnmvincent@gmail.com>

Co-authored-by: chappie <6537530+chapmanc@users.noreply.github.com>

…ation set to 5m for consul-debug into release/1.15.x (#17727) * backport of commit 9d72a26 * backport of commit 9b9bb8d * backport of commit ba44809 --------- Co-authored-by: Ashesh Vidyut <ashesh.vidyut@hashicorp.com> Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com>

… output of 'consul operator raft list-peers' into release/1.15.x (#17737) * backport of commit 2f94024 * backport of commit 7626d09 * backport of commit 79aabc9 * backport of commit 44eee41 * merge commit fa40654 * fix merge conf * [NET-3865] [Supportability] Additional Information in the output of 'consul operator raft list-peers' (#17582) * init * fix tests * added -detailed in docs * added change log * fix doc * checking for entry in map * fix tests * removed detailed flag * removed detailed flag * revert unwanted changes * removed unwanted changes * updated change log * pr review comment changes * pr comment changes single API instead of two * fix change log * fix tests * fix tests * fix test operator raft endpoint test * Update .changelog/17582.txt Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * nits * updated docs --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * [NET-3865] [Supportability] Additional Information in the output of 'consul operator raft list-peers' (#17582) * init * fix tests * added -detailed in docs * added change log * fix doc * checking for entry in map * fix tests * removed detailed flag * removed detailed flag * revert unwanted changes * removed unwanted changes * updated change log * pr review comment changes * pr comment changes single API instead of two * fix change log * fix tests * fix tests * fix test operator raft endpoint test * Update .changelog/17582.txt Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * nits * updated docs --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * resolve diff * fix merge conf * fix merge conf * removed detailed flag --------- Co-authored-by: Ashesh Vidyut <ashesh.vidyut@hashicorp.com> Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com> Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* backport of commit 2f94024 * backport of commit 7626d09 * backport of commit 79aabc9 * backport of commit 44eee41 * fix conflicts * Updated docs added explanation. (#17751) * init * fix tests * added -detailed in docs * added change log * fix doc * checking for entry in map * fix tests * removed detailed flag * removed detailed flag * revert unwanted changes * removed unwanted changes * updated change log * pr review comment changes * pr comment changes single API instead of two * fix change log * fix tests * fix tests * fix test operator raft endpoint test * Update .changelog/17582.txt Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * nits * updated docs * explanation added --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com> --------- Co-authored-by: Ashesh Vidyut <ashesh.vidyut@hashicorp.com> Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com> Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* backport of commit 2f94024 * backport of commit 7626d09 * backport of commit 79aabc9 * backport of commit 44eee41 * merge conf resolve * merge conf resolve * Fix Docs for Trails Leader By (#17763) * init * fix tests * added -detailed in docs * added change log * fix doc * checking for entry in map * fix tests * removed detailed flag * removed detailed flag * revert unwanted changes * removed unwanted changes * updated change log * pr review comment changes * pr comment changes single API instead of two * fix change log * fix tests * fix tests * fix test operator raft endpoint test * Update .changelog/17582.txt Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * nits * updated docs * explanation added * fix doc * fix docs --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com> --------- Co-authored-by: Ashesh Vidyut <ashesh.vidyut@hashicorp.com> Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com> Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

…nal when updating ACL token into release/1.15.x (#17832) * backport of commit 31d96f5 * backport of commit 78dbcfb * backport of commit de3dcee * backport of commit 2c436e6 --------- Co-authored-by: gbolo <george.bolo@gmail.com>

…ease/1.15.x (#17819) * no-op commit due to failed cherry-picking * Fix formatting for webhook-certs Consul tutorial (#17810) * Fix formatting for webhook-certs Consul tutorial * Make a small grammar change to also pick up whitespace changes necessary for formatting --------- Co-authored-by: David Yu <dyu@hashicorp.com> --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: Steven Zamborsky <97125550+stevenzamborsky@users.noreply.github.com> Co-authored-by: David Yu <dyu@hashicorp.com>

…ase/1.15.x (#17573) * no-op commit due to failed cherry-picking * file not cherrypicked --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: Curt Bushko <cbushko@gmail.com> Co-authored-by: John Murret <john.murret@hashicorp.com>

…handled into release/1.15.x (#17170) * APIGW: Update how status conditions for certificates are handled (#17115) * Move status condition for invalid certifcate to reference the listener that is using the certificate * Fix where we set the condition status for listeners and certificate refs, added tests * Add changelog * Remove unused code from backport * Revert go mod changes --------- Co-authored-by: John Maguire <john.maguire@hashicorp.com> Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

…7844) * APIGW Normalize Status Conditions (#16994) * normalize status conditions for gateways and routes * Added tests for checking condition status and panic conditions for validating combinations, added dummy code for fsm store * get rid of unneeded gateway condition generator struct * Remove unused file * run go mod tidy * Update tests, add conflicted gateway status * put back removed status for test * Fix linting violation, remove custom conflicted status * Update fsm commands oss * Fix incorrect combination of type/condition/status * cleaning up from PR review * Change "invalidCertificate" to be of accepted status * Move status condition enums into api package * Update gateways controller and generated code * Update conditions in fsm oss tests * run go mod tidy on consul-container module to fix linting * Fix type for gateway endpoint test * go mod tidy from changes to api * go mod tidy on troubleshoot * Fix route conflicted reason * fix route conflict reason rename * Fix text for gateway conflicted status * Add valid certificate ref condition setting * Revert change to resolved refs to be handled in future PR * Resolve sneaky merge conflicts --------- Co-authored-by: John Maguire <john.maguire@hashicorp.com> Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

Co-authored-by: V. K <cn007b@gmail.com>

… release/1.15.x (#17864) * backport of commit ad7cbd5 * backport of commit 122b50b * backport of commit fea8da7 --------- Co-authored-by: Iryna Shustava <iryna@hashicorp.com>

Signed-off-by: Dan Bond <danbond@protonmail.com> Co-authored-by: Dan Bond <danbond@protonmail.com>

Co-authored-by: mr-miles <miles.waller@gmail.com>

…ing Consul Telemetry Collector into release/1.15.x (#17900) * backport of commit 94584cf * backport of commit aed6a00 * backport of commit 2eb7a2a --------- Co-authored-by: josh <josh.timmons@hashicorp.com>

Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>

…mage into release/1.15.x (#17919) * backport of commit d490377 * Reference hashicorp/consul instead of consul for Docker image (#17914) * Reference hashicorp/consul instead of consul for Docker image * Update Make targets that pull consul directly --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

Co-authored-by: Dan Bond <danbond@protonmail.com>

) * backport of commit afa1f42 * backport of commit e097002 * backport of commit 2f2aad5 * backport of commit 4a5c9c1 --------- Co-authored-by: Ranjandas <thejranjan@gmail.com> Co-authored-by: Chris S. Kim <kisunji92@gmail.com>

* backport of commit 4f7d21d * backport of commit d1ba0e8 --------- Co-authored-by: David Yu <dyu@hashicorp.com>

Co-authored-by: natemollica-dev <57850649+natemollica-nm@users.noreply.github.com> Co-authored-by: David Yu <dyu@hashicorp.com>

…iguration - enable_debug into release/1.15.x (#17968) * backport of commit 10f500e * merge conf resolved * removed experiments --------- Co-authored-by: Ashesh Vidyut <ashesh.vidyut@hashicorp.com> Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com>

…, nodes, service into release/1.15.x (#17986) * backport to 1.15.x --------- Co-authored-by: cskh <hui.kang@hashicorp.com>

Co-authored-by: Evan Phoenix <evan@phx.io>

…o release/1.15.x (#18013) * backport of commit 954bd6a * backport of commit 85c32d8 * backport of commit 7ea3d62 * backport of commit 127ae69 * backport of commit e04099b * fix import --------- Co-authored-by: DanStough <dan.stough@hashicorp.com>

Co-authored-by: J.C. Jones <jc@letsencrypt.org>

…t annotation into release/1.15.x (#18039) * backport of commit e1bf428 * backport of commit ddf214e * backport of commit e41bd9c * backport of commit b9cfc86 * backport of commit 0ddf013 * backport of commit 1b0b513 * backport of commit 29442ad * backport of commit 5e7ddf5 * backport of commit f2b6fa7 * backport of commit 83b84a9 * backport of commit 56d8173 * backport of commit 0ab44f0 * backport of commit 69c99fb * backport of commit b79e124 * backport of commit fb14419 * backport of commit 3b7b2a0 --------- Co-authored-by: David Yu <dyu@hashicorp.com>

…e/1.15.x (#18042) * manually resolve conflicts * remove extraneous heading * address feedback --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: David Yu <dyu@hashicorp.com>

@trujillo-adam

…/1.15.x (#18018) * no-op commit due to failed cherry-picking * Fix removed service-to-service peering links (#17221) * docs: fix removed service-to-service peering links * docs: extend peering-via-mesh-gateways intro (thanks @trujillo-adam) --------- Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: Michael Hofer <karras@users.noreply.github.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* add helm chart changes * ran again with updated values Co-authored-by: Dan Stough <dan.stough@hashicorp.com>

…rces into release/1.15.x (#18064) * fix(connect): set initial_fetch_time to wait indefinitely * changelog * PR feedback 1 --------- Co-authored-by: DanStough <dan.stough@hashicorp.com>

…elease/1.15.x (#18083) * backport of commit 58ef5fd * backport of commit 857427a --------- Co-authored-by: josh <josh.timmons@hashicorp.com>

Co-authored-by: Krastin Krastev <krastin@hashicorp.com>

…ect CA into release/1.15.x (#18098) * backport of commit 4034bb2 * backport of commit 9c4c3c5 * backport of commit 7282078 --------- Co-authored-by: Tom Davies <thomas.23.davies@bt.com>

…18093) Co-authored-by: Nick Irvine <115657443+nfi-hashicorp@users.noreply.github.com>

…8104) * backport of commit 0d7bee8 * backport of commit 408cbe8 * backport of commit a085478 * backport of commit 71c4c65 * backport of commit 0c060fa --------- Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com> Co-authored-by: David Yu <dyu@hashicorp.com>

Co-authored-by: John Murret <john.murret@hashicorp.com>

Co-authored-by: Dan Bond <danbond@protonmail.com>

Co-authored-by: nv-hashi <80716011+nv-hashi@users.noreply.github.com>

@jmurret

…r because of dependency bugs with go 1.20.6. Pin go to 1.20.5. into release/1.15.x (#18127) ## Backport This PR is auto-generated from #18124 to be assessed for backporting due to the inclusion of the label backport/1.15. :rotating_light: >**Warning** automatic cherry-pick of commits failed. If the first commit failed, you will see a blank no-op commit below. If at least one commit succeeded, you will see the cherry-picked commits up to, _not including_, the commit where the merge conflict occurred. The person who merged in the original PR is: @jmurret This person should manually cherry-pick the original PR into a new backport PR, and close this one when the manual backport PR is merged in. > merge conflict error: POST https://api.github.com/repos/hashicorp/consul/merges: 409 Merge conflict [] The below text is copied from the body of the original PR. --- ### Description The following jobs started failing when go 1.20.6 was released: - `go-test-api-1-19` - `go-test-api-1-20` - `compatibility-integration-tests` - `upgrade-integration-tests` `compatibility-integration-tests` and `compatibility-integration-tests` to this testcontainers issue: testcontainers/testcontainers-go#1359. This issue calls for testcontainers to release a new version when one of their dependencies is fixed. When that is done, we will unpin the go versions in `compatibility-integration-tests` and `compatibility-integration-tests`. ### Testing & Reproduction steps See these jobs broken in CI and then see them work with this PR. --- <details> <summary> Overview of commits </summary> - 747195f - 5164924 - f4d6ca1 - a474071 - 8c03b36 - c50b17c - 7b55f66 - 93ce5fc </details> --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: John Murret <john.murret@hashicorp.com> Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com>

…elease/1.15.x (#18136) ## Backport This PR is auto-generated from #18134 to be assessed for backporting due to the inclusion of the label backport/1.15. The below text is copied from the body of the original PR. --- ### Description - Fix unmatched bracket in the [doc](https://developer.hashicorp.com/consul/docs/services/usage/checks#ttl-check-configuration) (see the following screenshot of the page) <img width="618" alt="Screenshot 2023-07-13 at 9 01 19 PM" src="https://github.com/hashicorp/consul/assets/463631/20707735-906f-4b06-999d-44e6329a9fec"> ### Testing & Reproduction steps  ### Links  ### PR Checklist * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern --- <details> <summary> Overview of commits </summary> - d40243b </details> Co-authored-by: cskh <hui.kang@hashicorp.com>

@jmurret

…st.host that contains socked now error into release/1.15.x (#18144) ## Backport This PR is auto-generated from #18129 to be assessed for backporting due to the inclusion of the label backport/1.15. :rotating_light: >**Warning** automatic cherry-pick of commits failed. If the first commit failed, you will see a blank no-op commit below. If at least one commit succeeded, you will see the cherry-picked commits up to, _not including_, the commit where the merge conflict occurred. The person who merged in the original PR is: @jmurret This person should manually cherry-pick the original PR into a new backport PR, and close this one when the manual backport PR is merged in. > merge conflict error: POST https://api.github.com/repos/hashicorp/consul/merges: 409 Merge conflict [] The below text is copied from the body of the original PR. --- ### Description This is related to #18124 where we pinned the go versions in CI to 1.20.5 and 1.19.10. go 1.20.6 and 1.19.11 now validate request host headers for validity, including the hostname cannot be prefixed with slashes. For local communications (npipe://, unix://), the hostname is not used, but we need valid and meaningful hostname. Prior versions go Go would clean the host header, and strip slashes in the process, but go1.20.6 and go1.19.11 no longer do, and reject the host header. Around the community we are seeing that others are intercepting the req.host and if it starts with a slash or ends with .sock, they changing the host to localhost or another dummy value. [client: define a "dummy" hostname to use for local connections by thaJeztah · Pull Request #45942 · moby/moby](moby/moby#45942) ### Testing & Reproduction steps Check CI tests. ### Links * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern --- <details> <summary> Overview of commits </summary> - 747195f - 5164924 - f4d6ca1 - a474071 - 8c03b36 - c50b17c - cc8eaf8 - ce10138 - 133c7ec - b0bd440 - 8f22308 - f8578b0 - 4452224 - 19634a4 </details> --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: John Murret <john.murret@hashicorp.com>

…ge_max into release/1.15.x (#18155) ## Backport This PR is auto-generated from #18154 to be assessed for backporting due to the inclusion of the label backport/1.15. The below text is copied from the body of the original PR. --- ### Description Addresses #17171 (comment) ### Testing & Reproduction steps  ### Links  ### PR Checklist * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern --- <details> <summary> Overview of commits </summary> - f5a6411 </details> Co-authored-by: David Yu <dyu@hashicorp.com>

… name 1.15.x (#18163) Fix a bug that wrongly trims domains when there is an overlap with DC name (#17160) * Fix a bug that wrongly trims domains when there is an overlap with DC name Before this change, when DC name and domain/alt-domain overlap, the domain name incorrectly trimmed from the query. Example: Given: datacenter = dc-test, alt-domain = test.consul. Querying for "test-node.node.dc-test.consul" will faile, because the code was trimming "test.consul" instead of just ".consul" This change, fixes the issue by adding dot (.) before trimming * trimDomain: ensure domain trimmed without modyfing original domains * update changelog --------- Co-authored-by: Alex Simenduev <shamil.si@gmail.com>

…4743] into release/1.15.x (#18147) Co-authored-by: Nick Irvine <115657443+nfi-hashicorp@users.noreply.github.com>

….15.x (#18188) Backport of [NET-4865] Bump golang.org/x/net to 0.12.0 into release/1.16.x (#18189) Bump golang.org/x/net to 0.12.0 While not necessary to directly address CVE-2023-29406 (which should be handled by using a patched version of Go when building), an accompanying change to HTTP/2 error handling does impact agent code. See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2 change. Bump this dependency across our submodules as well for the sake of potential indirect consumers of `x/net/http`. Manual backport of 84cbf09.

…ease/1.15.x (#18194) backport of commit 93f3209 Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>

…18202) Fix Backport Assistant PR commenting (#18200) * Fix Backport Assistant failure PR commenting For general comments on a PR, it looks like you have to use the `/issue` endpoint rather than `/pulls`, which requires commit/other review-specific target details. This matches the endpoint used in `backport-reminder.yml`. * Remove Backport Reminder workflow This is noisy (even when adding multiple labels, individual comments per label are generated), and likely no longer needed: we haven't had this work in a long time due to an expired GH token, and we now have better automation for backport PR assignment. Manual backport of 1c7fcdf.

…18210) This PR explicitly enables WebSocket upgrades in Envoy's UpgradeConfig for all proxy types. (API Gateway, Ingress, and Sidecar.) Fixes #8283 Co-authored-by: Blake Covarrubias <blake@covarrubi.as>

backport of commit 45700f1 Co-authored-by: Dan Bond <danbond@protonmail.com>

…rs into release/1.15.x (#18232) backport of commit c5522ca Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>

… from running on docs only and ui only changes into release/1.15.x (#18243) backport of commit e0c0efe Co-authored-by: John Murret <john.murret@hashicorp.com>

…orkflows from running on docs only and ui only changes" into release/1.15.x (#18252) backport of commit 029099a Co-authored-by: John Murret <john.murret@hashicorp.com>

….15.x (#18256) NET-1825: More new ACL token creation docs (#18063) Co-authored-by: Paul Glass <pglass@hashicorp.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

…l from the go team. into release/1.15.x (#18265) backport of commit 23f0cfc Co-authored-by: John Murret <john.murret@hashicorp.com>

* add auto lic usage docs for 1.15 * commit actual file * fix usage and typos

backport of commit 785f10e Co-authored-by: Dan Bond <danbond@protonmail.com>

…1.15.x (#18279) member cli: add -filter expression to flags

… 1.0.0 into release/1.15.x (#18297) backport of commit 1584728 Co-authored-by: Paul Glass <pglass@hashicorp.com>

…ease/1.15.x (#18286) * backport of commit 9b02a4c * backport of commit 3cf5e28 * backport of commit ecec4c8 * backport of commit edee177 --------- Co-authored-by: boruszak <jeffrey.boruszak@hashicorp.com>

…ving api-gateway into release/1.15.x (#18294) * backport of commit 972b20a * backport of commit 2756817 * backport of commit 72c8f1e --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

Update list of Envoy versions Update supported envoy versions to 1.22.11, 1.23.12, 1.24.10, 1.25.9.

…ease/1.15.x (#18310) Update list of Envoy versions in docs Update supported Envoy versions across Consul release versions. Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>

…18288) [CC-5718] Remove HCP token requirement during bootstrap

Ensure that all release branches are aligned on content. Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

…18340) backport of commit ae87778 Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>

… into release/1.15.x (#18344) [CC-5719] Add support for builtin global-read-only policy (#18319) * [CC-5719] Add support for builtin global-read-only policy * Add changelog * Add read-only to docs * Fix some minor issues. * Change from ReplaceAll to Sprintf * Change IsValidPolicy name to return an error instead of bool * Fix PolicyList test * Fix other tests * Apply suggestions from code review * Fix state store test for policy list. * Fix naming issues * Update acl/validation.go * Update agent/consul/acl_endpoint.go --------- Co-authored-by: Jeremy Jacobson <jjacobson93@users.noreply.github.com> Co-authored-by: Paul Glass <pglass@hashicorp.com> Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com>

…1.15.x (#18355) * backport of commit 7f0a3db * backport of commit ea350bf * cherry-pick * go mod tidy --------- Co-authored-by: absolutelightning <ashesh.vidyut@hashicorp.com> Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com>

…nfiguration into release/1.15.x (#18360) [HCP Telemetry] Periodic Refresh for Dynamic Telemetry Configuration (#18168) * OTElExporter now uses an EndpointProvider to discover the endpoint * OTELSink uses a ConfigProvider to obtain filters and labels configuration * improve tests for otel_sink * Regex logic is moved into client for a method on the TelemetryConfig object * Create a telemetry_config_provider and update deps to use it * Fix conversion * fix import newline * Add logger to hcp client and move telemetry_config out of the client.go file * Add a telemetry_config.go to refactor client.go * Update deps * update hcp deps test * Modify telemetry_config_providers * Check for nil filters * PR review updates * Fix comments and move around pieces * Fix comments * Remove context from client struct * Moved ctx out of sink struct and fixed filters, added a test * Remove named imports, use errors.New if not fformatting * Remove HCP dependencies in telemetry package * Add success metric and move lock only to grab the t.cfgHahs * Update hash * fix nits * Create an equals method and add tests * Improve telemetry_config_provider.go tests * Add race test * Add missing godoc * Remove mock for MetricsClient * Avoid goroutine test panics * trying to kick CI lint issues by upgrading mod * imprve test code and add hasher for testing * Use structure logging for filters, fix error constants, and default to allow all regex * removed hashin and modify logic to simplify * Improve race test and fix PR feedback by removing hash equals and avoid testing the timer.Ticker logic, and instead unit test * Ran make go-mod-tidy * Use errtypes in the test * Add changelog * add safety check for exporter endpoint * remove require.Contains by using error types, fix structure logging, and fix success metric typo in exporter * Fixed race test to have changing config values * Send success metric before modifying config * Avoid the defer and move the success metric under

…et` to 0.13.0 into release/1.15.x (#18362) backport of commit 905e371 Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>

#18371) * backport of commit 1a9cded * backport of commit cfec746 * backport of commit 8a9db4c * backport of commit ac13bf1 --------- Co-authored-by: Jeremy Jacobson <jeremy.jacobson@hashicorp.com>

backport of commit 7cd00e7 Co-authored-by: R.B. Boyer <rb@hashicorp.com>

….x (#18212) backport of commit 4e8cddd Co-authored-by: cskh <hui.kang@hashicorp.com>

…18379) * no-op commit due to failed cherry-picking * resolve merge conflicts --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>

#18416) Backport 1.16.x Upgrade test: remove outdated test and disable log due to verbosity (… (#18413) Upgrade test: remove outdated test and disable log due to verbosity (#18403) * remove outdated test * disable log since we have too many parallel tests

…ersion in in… into release/1.15.x (#18407) * bump version * remove duplicate test file --------- Co-authored-by: Hui Kang <hui.kang@hashicorp.com>

backport of commit 0f355b0 Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>

… into release/1.15.x (#18471) backport of commit 59bfc81 Co-authored-by: NiniOak <anita.akaeze@hashicorp.com>

Updates Helm docs to better align with style

…lease/1.15.x (#18480) docs: specify manual server metadata intervention Co-authored-by: Dan Bond <danbond@protonmail.com>

… file contains BUSL header into release/1.15.x (#18487) Add license-checker action that fails when any backported file contains BUSL header (#18485) * Add license-checker action that fails when any backported file contains BUSL header * Quote echoed variable to retain line breaks * Add ticket to reference for more details Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

NET-5371 License checker pt2 (#18491) * Update grep command to work on ubuntu * Run license checker when new commits are pushed to PR Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

…into release/1.15.x (#18507) backport of commit 1673852 Co-authored-by: Blake Covarrubias <blake@covarrubi.as>

…to release/1.15.x (#18521) CI Split integration tests to run nightly and every PR (#18518) Co-authored-by: temp <temp@hashicorp.com>

…k into release/1.15.x (#18564) * backport of commit 06d4c72 * backport of commit 42c37bb --------- Co-authored-by: cskh <hui.kang@hashicorp.com>

OSS -> CE (community edition) changes (#18517)

… files are in commit history into release/1.15.x (#18569) backport of commit c833fcc Co-authored-by: NiniOak <anita.akaeze@hashicorp.com>

…nto the TelemetryConfigProvider into release/1.15.x (#18627) * [HCP Telemetry] Move first TelemetryConfig Fetch into the TelemetryConfigProvider (#18318) * Add Enabler interface to turn sink on/off * Use h for hcpProviderImpl vars, fix PR feeback and fix errors * Keep nil check in exporter and fix tests * Clarify comment and fix function name * Use disable instead of enable * Fix errors nit in otlp_transform * Add test for refreshInterval of updateConfig * Add disabled field in MetricsConfig struct * Fix PR feedback: improve comment and remove double colons * Fix deps test which requires a maybe * Update hcp-sdk-go to v0.61.0 * use disabled flag in telemetry_config.go * Handle 4XX errors in telemetry_provider * Fix deps test * Check 4XX instead * Run make go-mod-tidy * Delete test-integ

…1.15.x (#18632) * backport of commit bd136d1 * backport of commit cb770fa --------- Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>

…Version / Datacenter into release/1.15.x (#18639) * Reference hashicorp/consul instead of consul for Docker image (#17914) * Reference hashicorp/consul instead of consul for Docker image * Update Make targets that pull consul directly * Update Consul K8s Upgrade Doc Updates (#17921) Updating upgrade procedures to encompass expected errors during upgrade process from v1.13.x to v1.14.x. * Update sameness-group.mdx (#17915) * Update create-sameness-groups.mdx (#17927) * deps: coredns v1.10.1 (#17912) * Ensure RSA keys are at least 2048 bits in length (#17911) * Ensure RSA keys are at least 2048 bits in length * Add changelog * update key length check for FIPS compliance * Fix no new variables error and failing to return when error exists from validating * clean up code for better readability * actually return value * tlsutil: Fix check TLS configuration (#17481) * tlsutil: Fix check TLS configuration * Rewording docs. * Update website/content/docs/services/configuration/checks-configuration-reference.mdx Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * Fix typos and add changelog entry. --------- Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * docs: Deprecations for connect-native SDK and specific connect native APIs (#17937) * Update v1_16_x.mdx * Update connect native golang page --------- Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * Revert "Add workflow to verify linux release packages (#17904)" (#17942) This reverts commit 3368f14fab500ebe9f6aeab5631dd1d5f5a453e5. * Fixes Secondary ConnectCA update (#17846) This fixes a bug that was identified which resulted in subsequent ConnectCA configuration update not to persist in the cluster. * fixing typo in link to jwt-validations-with-intentions doc (#17955) * Fix streaming backend link (#17958) * Fix streaming backend link * Update health.mdx * Dynamically create jwks clusters for jwt-providers (#17944) * website: remove deprecated agent rpc docs (#17962) * Fix missing BalanceOutboundConnections in v2 catalog. (#17964) * feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug (#17565) * # This is a combination of 9 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same * # This is a combination of 12 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same # This is the commit message #10: made enable debug atomic bool # This is the commit message #11: fix lint # This is the commit message #12: fix test true enable debug * parent 10f500e895d92cc3691ade7b74a33db755d22039 author absolutelightning <ashesh.vidyut@hashicorp.com> 1687352587 +0530 committer absolutelightning <ashesh.vidyut@hashicorp.com> 1687352592 +0530 init without tests change log fix tests fix tests added tests change log breaking change removed breaking change fix test keeping the test behaviour same made enable debug atomic bool fix lint fix test true enable debug using enable debug in agent as atomic bool test fixes fix tests fix tests added update on correct locaiton fix tests fix reloadable config enable debug fix tests fix init and acl 403 * revert commit * Fix formatting codeblocks on APIgw docs (#17970) * fix formatting codeblocks * remove unnecessary indents * Remove POC code (#17974) * update doc (#17910) * update doc * update link * Remove duplicate and unused newDecodeConfigEntry func (#17979) * docs: samenessGroup YAML examples (#17984) * configuration entry syntax * Example config * Add changelog entry for 1.16.0 (#17987) * Fix typo (#17198) servcies => services * Expose JWKS cluster config through JWTProviderConfigEntry (#17978) * Expose JWKS cluster config through JWTProviderConfigEntry * fix typos, rename trustedCa to trustedCA * Integration test for ext-authz Envoy extension (#17980) * Fix incorrect protocol for transparent proxy upstreams. (#17894) This PR fixes a bug that was introduced in: https://github.com/hashicorp/consul/pull/16021 A user setting a protocol in proxy-defaults would cause tproxy implicit upstreams to not honor the upstream service's protocol set in its `ServiceDefaults.Protocol` field, and would instead always use the proxy-defaults value. Due to the fact that upstreams configured with "tcp" can successfully contact upstream "http" services, this issue was not recognized until recently (a proxy-defaults with "tcp" and a listening service with "http" would make successful requests, but not the opposite). As a temporary work-around, users experiencing this issue can explicitly set the protocol on the `ServiceDefaults.UpstreamConfig.Overrides`, which should take precedence. The fix in this PR removes the proxy-defaults protocol from the wildcard upstream that tproxy uses to configure implicit upstreams. When the protocol was included, it would always overwrite the value during discovery chain compilation, which was not correct. The discovery chain compiler also consumes proxy defaults to determine the protocol, so simply excluding it from the wildcard upstream config map resolves the issue. * feat: include nodes count in operator usage endpoint and cli command (#17939) * feat: update operator usage api endpoint to include nodes count * feat: update operator usange cli command to includes nodes count * [OSS] Improve Gateway Test Coverage of Catalog Health (#18011) * fix(cli): remove failing check from 'connect envoy' registration for api gateway * test(integration): add tests to check catalog statsus of gateways on startup * remove extra sleep comment * Update test/integration/consul-container/libs/assert/service.go * changelog * Fixes Traffic rate limitting docs (#17997) * Fix removed service-to-service peering links (#17221) * docs: fix removed service-to-service peering links * docs: extend peering-via-mesh-gateways intro (thanks @trujillo-adam) --------- Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * docs: Sameness "beta" warning (#18017) * Warning updates * .x * updated typo in tab heading (#18022) * updated typo in tab heading * updated tab group typo, too * Document that DNS lookups can target cluster peers (#17990) Static DNS lookups, in addition to explicitly targeting a datacenter, can target a cluster peer. This was added in 95dc0c7b301b70a6b955a8b7c9737c9b86f03df6 but didn't make the documentation. The driving function for the change is `parseLocality` here: https://github.com/hashicorp/consul/blob/0b1299c28d8127129d61310ee4280055298438e0/agent/dns_oss.go#L25 The biggest change in this is to adjust the standard lookup syntax to tie `.<datacenter>` to `.dc` as required-together, and to append in the similar `.<cluster-peer>.peer` optional argument, both to A record and SRV record lookups. Co-authored-by: David Yu <dyu@hashicorp.com> * Add first integration test for jwt auth with intention (#18005) * fix stand-in text for name field (#18030) * removed sameness conf entry from failover nav (#18033) * docs - add service sync annotations and k8s service weight annotation (#18032) * Docs for https://github.com/hashicorp/consul-k8s/pull/2293 * remove versions for enterprise features since they are old --------- Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com> * docs - add jobs use case for service mesh k8s (#18037) * docs - add jobs use case for service mesh k8s * add code blocks * address feedback (#18045) * Add verify server hostname to tls default (#17155) * [OSS] Fix initial_fetch_timeout to wait for all xDS resources (#18024) * fix(connect): set initial_fetch_time to wait indefinitely * changelog * PR feedback 1 * ui: fix typos for peer service imports (#17999) * test: fix FIPS inline cert test message (#18076) * Fix a couple typos in Agent Telemetry Metrics docs (#18080) * Fix metrics docs * Add changelog Signed-off-by: josh <josh.timmons@hashicorp.com> --------- Signed-off-by: josh <josh.timmons@hashicorp.com> * docs updates - cluster peering and virtual services (#18069) * Update route-to-virtual-services.mdx * Update establish-peering.mdx * Update service-mesh-compare.mdx (#17279) grammar change * Update helm docs on main (#18085) * ci: use gotestsum v1.10.1 [NET-4042] (#18088) * Docs: Update proxy lifecycle annotations and consul-dataplane flags (#18075) * Update proxy lifecycle annotations and consul-dataplane flags * Pass configured role name to Vault for AWS auth in Connect CA (#17885) * Docs for dataplane upgrade on k8s (#18051) * Docs for dataplane upgrade on k8s --------- Co-authored-by: David Yu <dyu@hashicorp.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * docs - update upgrade index page to not recommend consul leave. (#18100) * Displays Consul version of each nodes in UI nodes section (#17754) * update UINodes and UINodeInfo response with consul-version info added as NodeMeta, fetched from serf members * update test cases TestUINodes, TestUINodeInfo * added nil check for map * add consul-version in local agent node metadata * get consul version from serf member and add this as node meta in catalog register request * updated ui mock response to include consul versions as node meta * updated ui trans and added version as query param to node list route * updates in ui templates to display consul version with filter and sorts * updates in ui - model class, serializers,comparators,predicates for consul version feature * added change log for Consul Version Feature * updated to get version from consul service, if for some reason not available from serf * updated changelog text * updated dependent testcases * multiselection version filter * Update agent/consul/state/catalog.go comments updated Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> --------- Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * api gw 1.16 updates (#18081) * api gw 1.16 updates * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * update CodeBlockConfig filename * Apply suggestions from code review Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * remove non-standard intentions page * Update website/content/docs/api-gateway/configuration/index.mdx Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * [NET-4103] ci: build s390x (#18067) * ci: build s390x * ci: test s390x * ci: dev build s390x * no GOOS * ent only * build: publish s390x * fix syntax error * fix syntax error again * fix syntax error again x2 * test branch * Move s390x conditionals to step level * remove test branch --------- Co-authored-by: emilymianeil <eneil@hashicorp.com> * :ermahgerd "Sevice Mesh" -> "Service Mesh" (#18116) Just a typo in the docs. * Split pbmesh.UpstreamsConfiguration as a resource out of pbmesh.Upstreams (#17991) Configuration that previously was inlined into the Upstreams resource applies to both explicit and implicit upstreams and so it makes sense to split it out into its own resource. It also has other minor changes: - Renames `proxy.proto` proxy_configuration.proto` - Changes the type of `Upstream.destination_ref` from `pbresource.ID` to `pbresource.Reference` - Adds comments to fields that didn't have them * [NET-4895] ci - api tests and consul container tests error because of dependency bugs with go 1.20.6. Pin go to 1.20.5. (#18124) ### Description The following jobs started failing when go 1.20.6 was released: - `go-test-api-1-19` - `go-test-api-1-20` - `compatibility-integration-tests` - `upgrade-integration-tests` `compatibility-integration-tests` and `compatibility-integration-tests` to this testcontainers issue: https://github.com/testcontainers/testcontainers-go/issues/1359. This issue calls for testcontainers to release a new version when one of their dependencies is fixed. When that is done, we will unpin the go versions in `compatibility-integration-tests` and `compatibility-integration-tests`. ### Testing & Reproduction steps See these jobs broken in CI and then see them work with this PR. --------- Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com> * Add ingress gateway deprecation notices to docs (#18102) ### Description This adds notices, that ingress gateway is deprecated, to several places in the product docs where ingress gateway is the topic. ### Testing & Reproduction steps Tested with a local copy of the website. ### Links Deprecation of ingress gateway was announced in the Release Notes for Consul 1.16 and Consul-K8s 1.2. See: [https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated ) [https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated) ### PR Checklist * [N/A] updated test coverage * [X] external facing docs updated * [X] appropriate backport labels added * [X] not a security concern --------- Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * Add docs for jwt cluster configuration (#18004) ### Description  - Add jwt-provider docs for jwks cluster configuration. The configuration was added here: https://github.com/hashicorp/consul/pull/17978 * Docs: fix unmatched bracket for health checks page (#18134) * NET-4657/add resource service client (#18053) ### Description  Dan had already started on this [task](https://github.com/hashicorp/consul/pull/17849) which is needed to start building the HTTP APIs. This just needed some cleanup to get it ready for review. Overview: - Rename `internalResourceServiceClient` to `insecureResourceServiceClient` for name consistency - Configure a `secureResourceServiceClient` with auth enabled ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Fix bug with Vault CA provider (#18112) Updating RootPKIPath but not IntermediatePKIPath would not update leaf signing certs with the new root. Unsure if this happens in practice but manual testing showed it is a bug that would break mesh and agent connections once the old root is pruned. * [NET-4897] net/http host header is now verified and request.host that contains socked now error (#18129) ### Description This is related to https://github.com/hashicorp/consul/pull/18124 where we pinned the go versions in CI to 1.20.5 and 1.19.10. go 1.20.6 and 1.19.11 now validate request host headers for validity, including the hostname cannot be prefixed with slashes. For local communications (npipe://, unix://), the hostname is not used, but we need valid and meaningful hostname. Prior versions go Go would clean the host header, and strip slashes in the process, but go1.20.6 and go1.19.11 no longer do, and reject the host header. Around the community we are seeing that others are intercepting the req.host and if it starts with a slash or ends with .sock, they changing the host to localhost or another dummy value. [client: define a "dummy" hostname to use for local connections by thaJeztah · Pull Request #45942 · moby/moby](https://github.com/moby/moby/pull/45942) ### Testing & Reproduction steps Check CI tests. ### Links * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * add a conditional around setting LANFilter.AllSegments to make sure it is valid (#18139) ### Description This is to correct a code problem because this assumes all segments, but when you get to Enterprise, you can be in partition that is not the default partition, in which case specifying all segments does not validate and fails. This is to correct the setting of this filter with `AllSegments` to `true` to only occur when in the the `default` partition. ### Testing & Reproduction steps  ### Links  ### PR Checklist * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * chore: bump upgrade integrations tests to 1.15, 116 [NET-4743] (#18130) * re org resource type registry (#18133) * fix: update delegateMock used in ENT (#18149) ### Description  The mock is used in `http_ent_test` file which caused lint failures. For OSS->ENT parity adding the same change here. ### Links  Identified in OSS->ENT [merge PR](https://github.com/hashicorp/consul-enterprise/pull/6328) ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter (#18062) ### Description  - Currently the jwt-auth filter doesn't take into account the service identity when validating jwt-auth, it only takes into account the path and jwt provider during validation. This causes issues when multiple source intentions restrict access to an endpoint with different JWT providers. - To fix these issues, rather than use the JWT auth filter for validation, we use it in metadata mode and allow it to forward the successful validated JWT token payload to the RBAC filter which will make the decisions. This PR ensures requests with and without JWT tokens successfully go through the jwt-authn filter. The filter however only forwards the data for successful/valid tokens. On the RBAC filter level, we check the payload for claims and token issuer + existing rbac rules. ### Testing & Reproduction steps  - This test covers a multi level jwt requirements (requirements at top level and permissions level). It also assumes you have envoy running, you have a redis and a sidecar proxy service registered, and have a way to generate jwks with jwt. I mostly use: https://www.scottbrady91.com/tools/jwt for this. - first write your proxy defaults ``` Kind = "proxy-defaults" name = "global" config { protocol = "http" } ``` - Create two providers ``` Kind = "jwt-provider" Name = "auth0" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjog....." } } ``` ``` Kind = "jwt-provider" Name = "okta" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjogW3...." } } ``` - add a service intention ``` Kind = "service-intentions" Name = "redis" JWT = { Providers = [ { Name = "okta" }, ] } Sources = [ { Name = "*" Permissions = [{ Action = "allow" HTTP = { PathPrefix = "/workspace" } JWT = { Providers = [ { Name = "okta" VerifyClaims = [ { Path = ["aud"] Value = "my_client_app" }, { Path = ["sub"] Value = "5be86359073c434bad2da3932222dabe" } ] }, ] } }, { Action = "allow" HTTP = { PathPrefix = "/" } JWT = { Providers = [ { Name = "auth0" }, ] } }] } ] ``` - generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with different claims than `/workspace` expects and 1 with correct claims - connect to your envoy (change service and address as needed) to view logs and potential errors. You can add: `-- --log-level debug` to see what data is being forwarded ``` consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502 ``` - Make the following requests: ``` curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v Successful request ``` ### TODO * [x] Update test coverage * [ ] update integration tests (follow-up PR) * [x] appropriate backport labels added * Support Consul Connect Envoy Command on Windows (#17694) ### Description Add support for consul connect envoy command on windows. This PR fixes the comments of PR - https://github.com/hashicorp/consul/pull/15114 ### Testing * Built consul.exe from this branch on windows and hosted here - [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Updated the [tutorial](https://developer.hashicorp.com/consul/tutorials/developer-mesh/consul-windows-workloads) and changed the `consul_url.default` value to [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Followed the steps in the tutorial and verified that everything is working as described. ### PR Checklist * [x] updated test coverage * [ ] external facing docs updated * [x] appropriate backport labels added * [x] not a security concern --------- Co-authored-by: Franco Bruno Lavayen <cocolavayen@gmail.com> Co-authored-by: Jose Ignacio Lorenzo <74208929+joselo85@users.noreply.github.com> Co-authored-by: Jose Ignacio Lorenzo <joseignaciolorenzo85@gmail.com> Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> * Change docs to say 168h instead of 7d for server_rejoin_age_max (#18154) ### Description Addresses https://github.com/hashicorp/consul/pull/17171#issuecomment-1636930705 * [OSS] test: improve xDS listener code coverage (#18138) test: improve xDS listener code coverage * Re-order expected/actual for assertContainerState in consul container tests (#18157) Re-order expected/actual, consul container tests * group and document make file (#17943) * group and document make file * Add `testing/deployer` (neé `consul-topology`) [NET-4610] (#17823) Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com> Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com> * [NET-4792] Add integrations tests for jwt-auth (#18169) * Add FIPS reference to consul enterprise docs (#18028) * Add FIPS reference to consul enterprise docs * Update website/content/docs/enterprise/index.mdx Co-authored-by: David Yu <dyu@hashicorp.com> * remove support for ecs client (fips) --------- Co-authored-by: David Yu <dyu@hashicorp.com> * add peering_commontopo tests [NET-3700] (#17951) Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com> Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com> Co-authored-by: NiniOak <anita.akaeze@hashicorp.com> * docs - remove Sentinel from enterprise features list (#18176) * Update index.mdx * Update kv.mdx * Update docs-nav-data.json * delete sentinel.mdx * Update redirects.js --------- Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com> * [NET-4865] Bump golang.org/x/net to 0.12.0 (#18186) Bump golang.org/x/net to 0.12.0 While not necessary to directly address CVE-2023-29406 (which should be handled by using a patched version of Go when building), an accompanying change to HTTP/2 error handling does impact agent code. See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2 change. Bump this dependency across our submodules as well for the sake of potential indirect consumers of `x/net/http`. * Call resource mutate hook before validate hook (NET-4907) (#18178) * [NET-4865] security: Update Go version to 1.20.6 (#18190) Update Go version to 1.20.6 This resolves [CVE-2023-29406] (https://nvd.nist.gov/vuln/detail/CVE-2023-29406) for uses of the `net/http` standard library. Note that until the follow-up to #18124 is done, the version of Go used in those impacted tests will need to remain on 1.20.5. * Improve XDS test coverage: JWT auth edition (#18183) * Improve XDS test coverage: JWT auth edition more tests * test: xds coverage for jwt listeners --------- Co-authored-by: DanStough <dan.stough@hashicorp.com> * update readme.md (#18191) u[date readme.md * Update submodules to latest following 1.16.0 (#18197) Align all our internal use of submodules on the latest versions. * SEC-090: Automated trusted workflow pinning (2023-07-18) (#18174) Result of tsccr-helper -log-level=info -pin-all-workflows . Co-authored-by: hashicorp-tsccr[bot] <hashicorp-tsccr[bot]@users.noreply.github.com> * Fix Backport Assistant PR commenting (#18200) * Fix Backport Assistant failure PR commenting For general comments on a PR, it looks like you have to use the `/issue` endpoint rather than `/pulls`, which requires commit/other review-specific target details. This matches the endpoint used in `backport-reminder.yml`. * Remove Backport Reminder workflow This is noisy (even when adding multiple labels, individual comments per label are generated), and likely no longer needed: we haven't had this work in a long time due to an expired GH token, and we now have better automation for backport PR assignment. * resource: Pass resource to Write ACL hook instead of just resource Id [NET-4908] (#18192) * Explicitly enable WebSocket upgrades (#18150) This PR explicitly enables WebSocket upgrades in Envoy's UpgradeConfig for all proxy types. (API Gateway, Ingress, and Sidecar.) Fixes #8283 * docs: fix the description of client rpc (#18206) * NET-4804: Add dashboard for monitoring consul-k8s (#18208) * [OSS] Improve xDS Code Coverage - Clusters (#18165) test: improve xDS cluster code coverage * NET-4222 take config file consul container (#18218) Net 4222 take config file consul container * Envoy Integration Test Windows (#18007) * [CONSUL-395] Update check_hostport and Usage (#40) * [CONSUL-397] Copy envoy binary from Image (#41) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix --------- Co-authored-by: Jose Ignacio Lorenzo <74208929+joselo85@users.noreply.github.com> Co-authored-by: Franco Bruno Lavayen <cocolavayen@gmail.com> Co-authored-by: Ivan K Berlot <ivanberlot@gmail.com> Co-authored-by: Ezequiel Fernández Ponce <20102608+ezfepo@users.noreply.github.com> Co-authored-by: joselo85 <joseignaciolorenzo85@gmail.com> Co-authored-by: Ezequiel Fernández Ponce <ezequiel.fernandez@southworks.com> Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> * fix typos and update ecs compat table (#18215) * fix typos and update ecs compat table * real info for the ecs compat matrix table * Update website/content/docs/ecs/compatibility.mdx Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com> --------- Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com> * [OSS] proxystate: add proxystate protos (#18216) * proxystate: add proxystate protos to pbmesh and resolve imports and conflicts between message names * ci: don't verify s390x (#18224) * [CC-5718] Remove HCP token requirement during bootstrap (#18140) * [CC-5718] Remove HCP token requirement during bootstrap * Re-add error for loading HCP management token * Remove old comment * Add changelog entry * Remove extra validation line * Apply suggestions from code review Co-authored-by: lornasong <lornasong@users.noreply.github.com> --------- Co-authored-by: lornasong <lornasong@users.noreply.github.com> * [NET-4122] Doc guidance for federation with externalServers (#18207) Doc guidance for federation with externalServers Add guidance for proper configuration when joining to a secondary cluster using WAN fed with external servers also enabled. Also clarify federation requirements and fix formatting for an unrelated value. Update both the Helm chart reference (synced from `consul-k8s`, see hashicorp/consul-k8s#2583) and the docs on using `externalServers`. * [OSS] Improve xDS Code Coverage - Endpoints and Misc (#18222) test: improve xDS endpoints code coverage * Clarify license reporting timing and GDPR compliance (#18237) Add Alicia's edits to clarify log timing and other details * Fix Github Workflow File (#18241) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix * Fix tests --------- Co-authored-by: Ivan K Berlot <ivanberlot@gmail.com> Co-authored-by: Jose Ignacio Lorenzo <74208929+joselo85@users.noreply.github.com> Co-authored-by: Franco Bruno Lavayen <cocolavayen@gmail.com> Co-authored-by: Ezequiel Fernández Ponce <20102608+ezfepo@users.noreply.github.com> Co-authored-by: joselo85 <joseignaciolorenzo85@gmail.com> Co-authored-by: Ezequiel Fernández Ponce <ezequiel.fernandez@southworks.com> Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> * NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236) * Align build arch matrix with enterprise (#18235) Ensure that OSS remains in sync w/ Enterprise by aligning the format of arch matrix args for various build jobs. * Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes" (#18248) Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236)" This reverts commit a11dba710e6ce6f172c0fa6c9b61567cc1efffc8. * resource: Add scope to resource type registration [NET-4976] (#18214) Enables querying a resource type's registration to determine if a resource is cluster, partition, or partition and namespace scoped. * Fix some inconsistencies in jwt docs (#18234) * NET-1825: More new ACL token creation docs (#18063) Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * grafana: add the panel resource usage of connect injector (#18247) * [NET-3700] Backfill changelog entry for c2bbe67 and 7402d06 (#18259) Backfill changelog entry for c2bbe67 and 7402d06 Add a changelog entry for the follow-up PR since it was specific to the fix and references the original change. * NET-4897 - update comment to include the current issue url from the go team. (#18263) * fix typos, style, and improper links (#18269) * member cli: add -filter expression to flags (#18223) * member cli: add -filter expression to flags * changelog * update doc * Add test cases * use quote * go-tests: disable s390x (#18273) * docs: Update ext-authz documentation for kubernetes (#18281) Update ext-authz documentation for kubernetes * docs: Consul on Kubernetes specific upgrade info (#18230) * Compatibility page - dataplanes mention * Upgrading higher-level dataplane mention * `exec=` string callout * More visible for upgrade page * Apply suggestions from code review Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com> --------- Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com> Co-authored-by: John Murret <john.murret@hashicorp.com> * Fix typo in Envoy extensions doc (#18284) * docs: K8s secondary DC requirements (#18280) * Requested edit * Apply suggestions from code review Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --------- Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway (#18291) * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway This fixes a race condition due to our dependency on having the listener(s) from the api-gateway config entry in order to fully and properly process the resources on the bound-api-gateway config entry. * Apply suggestions from code review * Add changelog entry * Update K8s changelog to address cloud auto-join change in 1.0.0 (#18293) * Update list of Envoy versions (#18300) Update supported envoy versions to 1.24.10, 1.25.9, 1.26.4, 1.27.0. * [NET-4904] Update list of Envoy versions in docs (#18306) Update list of Envoy versions in docs Update supported Envoy versions across Consul release versions. * Update actions for TSCCR (#18317) Update action versions before deadline * Fix topology view when displaying mixed connect-native/normal services. (#13023) * Fix topoloy intention with mixed connect-native/normal services. If a service is registered twice, once with connect-native and once without, the topology views would prune the existing intentions. This change brings the code more in line with the transparent proxy behavior. * Dedupe nodes in the ServiceTopology ui endpoint (like done with tags). * Consider a service connect-native as soon as one instance is. * Stop JWT provider from being written in non default namespace (#18325) * docs: Fix some comments (#17118) Signed-off-by: cui fliter <imcusg@gmail.com> * [NET-4703] Prevent partial application of Envoy extensions (#18068) Prevent partial application of Envoy extensions Ensure that non-required extensions do not change xDS resources before exiting on failure by cloning proto messages prior to applying each extension. To support this change, also move `CanApply` checks up a layer and make them prior to attempting extension application, s.t. we avoid unnecessary copies where extensions can't be applied. Last, ensure that we do not allow panics from `CanApply` or `Extend` checks to escape the attempted extension application. * docs: Simplify example jq commands by removing pipes (#18327) Simplify jq command examples by removing pipes to other commands. * fix typo in create a mesh token docs (#18337) * [CC-5719] Add support for builtin global-read-only policy (#18319) * [CC-5719] Add support for builtin global-read-only policy * Add changelog * Add read-only to docs * Fix some minor issues. * Change from ReplaceAll to Sprintf * Change IsValidPolicy name to return an error instead of bool * Fix PolicyList test * Fix other tests * Apply suggestions from code review Co-authored-by: Paul Glass <pglass@hashicorp.com> * Fix state store test for policy list. * Fix naming issues * Update acl/validation.go Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com> * Update agent/consul/acl_endpoint.go --------- Co-authored-by: Paul Glass <pglass@hashicorp.com> Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com> * [NET-5121] proxystate: move protos to subdirectory to avoid conflicts (#18335) * also makes a few protos updates * resource: adding various helpers for working with resources (#18342) This is a bit of a grab bag of helpers that I found useful for working with them when authoring substantial Controllers. Subsequent PRs will make use of them. * Wasm integration tests for local and remote wasm files (#17756) * wasm integration tests for local and remote wasm files refactoring and cleanup for wasm testing remove wasm debug logging PR feedback, wasm build lock correct path pattern for wasm build files Add new helper function to minimize changes to existing test code Remove extra param mod tidy add custom service setup to test lib add wait until static server sidecar can reach nginx sidecar Doc comments PR feedback Update workflows to compile wasm for integration tests Fix docker build path Fix package name for linter Update makefile, fix redeclared function Update expected wasm filename Debug test ls in workflow remove pwd in favor of relative path more debugging Build wasm in compatability tests as well Build wasm directly in ci rather than in container Debug tinygo and llvm version Change wasm file extension Remove tinygo debugging Remove extra comments * Add compiled wasm and build instructions * [HCP Telemetry] Periodic Refresh for Dynamic Telemetry Configuration (#18168) * OTElExporter now uses an EndpointProvider to discover the endpoint * OTELSink uses a ConfigProvider to obtain filters and labels configuration * improve tests for otel_sink * Regex logic is moved into client for a method on the TelemetryConfig object * Create a telemetry_config_provider and update deps to use it * Fix conversion * fix import newline * Add logger to hcp client and move telemetry_config out of the client.go file * Add a telemetry_config.go to refactor client.go * Update deps * update hcp deps test * Modify telemetry_config_providers * Check for nil filters * PR review updates * Fix comments and move around pieces * Fix comments * Remove context from client struct * Moved ctx out of sink struct and fixed filters, added a test * Remove named imports, use errors.New if not fformatting * Remove HCP dependencies in telemetry package * Add success metric and move lock only to grab the t.cfgHahs * Update hash * fix nits * Create an equals method and add tests * Improve telemetry_config_provider.go tests * Add race test * Add missing godoc * Remove mock for MetricsClient * Avoid goroutine test panics * trying to kick CI lint issues by upgrading mod * imprve test code and add hasher for testing * Use structure logging for filters, fix error constants, and default to allow all regex * removed hashin and modify logic to simplify * Improve race test and fix PR feedback by removing hash equals and avoid testing the timer.Ticker logic, and instead unit test * Ran make go-mod-tidy * Use errtypes in the test * Add changelog * add safety check for exporter endpoint * remove require.Contains by using error types, fix structure logging, and fix success metric typo in exporter * Fixed race test to have changing config values * Send success metric before modifying config * Avoid the defer and move the success metric under * NET-4240 - Snapshots are failing on Windows (#18302) * fix go mod * fix go sum * added change log * ran make go mod tidy * Register ProxyStateTemplate Resource (#18316) Also, change the ProxyState.id to identity. This is because we already have the id of this proxy from the resource, and this id should be name-aligned with the workload it represents. It should also have the owner ref set to the workload ID if we need that. And so the id field seems unnecessary. We do, however, need a reference to workload identity so that we can authorize the proxy when it initially connects to the xDS server. * [NET-5146] security: Update Go version to 1.20.7 and `x/net` to 0.13.0 (#18358) * Update Go version to 1.20.7 This resolves [CVE-2023-29409] (https://nvd.nist.gov/vuln/detail/CVE-2023-29409)(`crypto/tls`). * Bump golang.org/x/net to 0.13.0 Addresses [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978) for security scans (non-impacting). * mesh: adding the protobuf types and resources backing mesh config v2 (#18351) * [OSS] test: xds coverage for routes (#18369) test: xds coverage for routes * Fix policy lookup to allow for slashes (#18347) * Fix policy lookup to allow for slashes * Fix suggestions * Fix other test * Revert some lines * add some initial CODEOWNERS (#18346) * Add redirects for mesh-gateway docs (#18377) * expose grpc as http endpoint (#18221) expose resource grpc endpoints as http endpoints * bimapper: fix a bug and add some more test coverage (#18387) * [docs] Fix ServiceDefaults example in distributed tracing (#17212) Fix ServiceDefaults example in distributed tracing. * Fix #17730 - Dev mode has new line (#18367) * adding new line only in case of pretty in url not in dev mode * change log added * chore: make go-mod-tidy (#18388) * resource: Make resource read tenancy aware (#18397) * Not using chmod - fixed integration test for Enterprise (#18401) * Not using chmod * checking icacls * Removed push * replaced ordered list of rate limit ops with flow diagram (#18398) * replaced ordered list of rate limit ops with flow diagram * made diagram font bigger * updated colors for dark mode img * fixed typo * recentered dark mode image * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Upgrade test: remove outdated test and disable log due to verbosity (#18403) * remove outdated test * disable log since we have too many parallel tests * bump testcontainers-go from 0.22.0 and remove pinned go version in in… (#18395) * bump testcontainers-go from 0.22.0 and remove pinned go version in integ test * go mod tidy * Replace deprecated target.Authority with target.URL.Host * Add ServiceEndpoints Mutation hook tests (#18404) * Add ServiceEndpoints Mutation hook tests * Move endpoint owner validation into the validation hook Also there were some minor changes to error validation to account for go-cmp not liking to peer through an errors.errorstring type that get created by errors.New * catalog: add FailoverPolicy mutation and validation hooks (#18390) Add most of the validation and mutation hooks for the FailoverPolicy resource. * NET-5115 Add retry + timeout filters for api-gateway (#18324) * squash, implement retry/timeout in consul core * update tests * catalog: adding a controller to reconcile FailoverPolicy resources (#18399) Add most of the semantic cross-resource validation for FailoverPolicy resources using a new controller. * fix the error in ent repo (#18421) fix the error in ent repo * build: updates for 1.16.1 release (#18415) * build: updates for 1.16.1 release * build: fix missing replace directive for envoyextensions * build: go mod tidy testing/deployer * update ECS links (#18419) * docs: fix incorrect proxy-defaults config in Lua Envoy extension (#18418) fix incorrect proxy-defaults config * resource: Make resource write tenancy aware (#18423) * [NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437) * Add logging to locality policy application In OSS, this is currently a no-op. * Inherit locality when registering sidecars When sidecar locality is not explicitly configured, inherit locality from the proxied service. * NET-4984: Update APIGW Config Entries for JWT Auth (#18366) * Added oss config entries for Policy and JWT on APIGW * Updated structs for config entry * Updated comments, ran deep-copy * Move JWT configuration into OSS file * Add in the config entry OSS file for jwts * Added changelog * fixing proto spacing * Moved to using manually written deep copy method * Use pointers for override/default fields in apigw config entries * Run gen scripts for changed types * k8s compat - Openshift versions (#18307) * Update compatibility.mdx Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * APIGW: Update HTTPRouteConfigEntry for JWT Auth (#18422) * Updated httproute config entry for JWT Filters * Added manual deepcopy method for httproute jwt filter * Fix test * Update JWTFilter to be in oss file * Add changelog * Add build tags for deepcopy oss file * [COMPLIANCE] License changes (#18443) * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * add missing license headers * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * Net-2708/delete resource endpoint (#18420) * feat: add http delete endpoint for resource service * refactor: clean up * Disable deep-copy codegen verification for now. (#18446) Once we figure out what to do with license headers for these files we should re-enable this check. * build: upgrade to latest buf v1.26.0 (#18426) * NET-4952 Add docs for export command (#18425) * Init file for services export command docs * Add link for export command to nav * Add export to list of subcommands for services * Add usage and basic command options * Add API options using includes * Add section for enterprise options * Include argument value in flag format * Add examples * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Net-2712/resource hcl parsing (#18250) * Initial protohcl implementation Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Co-authored-by: Daniel Upton <daniel@floppy.co> * resourcehcl: implement resource decoding on top of protohcl Co-authored-by: Daniel Upton <daniel@floppy.co> * fix: resolve ci failures * test: add additional unmarshalling tests * refactor: update function test to clean protohcl package imports --------- Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Co-authored-by: Daniel Upton <daniel@floppy.co> * read endpoint (#18268) implement http read endpoint to expose resource grpc service read method * Net-2707/list resource endpoint (#18444) feat: list resources endpoint * Fix incorrect yaml in examples (#18463) * NET-5187: Upgrade test timeout due to log producer errors (#18461) Stop log producer before restarting container * [NET-4799] [OSS] xdsv2: listeners L4 support for connect proxies (#18436) * refactor to avoid future import cycles * resource: Make resource list …

test: fix snapshot save w/ append test for ENT (#18656) Co-authored-by: Dan Stough <dan.stough@hashicorp.com>

…1.15.x (#18629) * backport of commit a5ad366 * backport of commit 24e7b9b * NET-3181 - Allow log file naming like Nomad (#18617) * fixes file name for consul * added log file * added tests for rename method --------- Co-authored-by: absolutelightning <ashesh.vidyut@hashicorp.com> Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com>

… nomad - fix bug into release/1.15.x (#18641) * backport of commit a5ad366 * backport of commit 24e7b9b * Net 3181 consul gh issue 15709 allow log file naming like nomad - fix bug (#18631) * fixes file name for consul * added log file * added tests for rename method * append instead of trunc * fix file truncate issue * added changelog * fix for build destros ci * removed changelog * solaris * removed change log --------- Co-authored-by: absolutelightning <ashesh.vidyut@hashicorp.com> Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com>

…pend-filename command and using leader version in version into release/1.15.x (#18685) * backport of commit e8f4bfe * backport of commit 5072958 * Removed nodename and status from consul snapshot save -append-filename command and using leader version in version (#18680) * init * fix tests * fix tests lint * fix api call inside dc * updated doc * address comments --------- Co-authored-by: absolutelightning <ashesh.vidyut@hashicorp.com> Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com>

Reduce runner sizes and concurrency

* Add support for querying tokens by service name. (#18667) Add support for querying tokens by service name The consul-k8s endpoints controller has a workflow where it fetches all tokens. This is not performant for large clusters, where there may be a sizable number of tokens. This commit attempts to alleviate that problem and introduces a new way to query by the token's service name.

update docs

…ressGateway into release/1.15.x (#18718) Adds PassiveHealthCheck Fields to ServiceDefaults and IngressGateway (#18532) Co-authored-by: Melisa Griffin <missylbytes@users.noreply.github.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

fix: emit consul version metric on a regular interval (#18724)

…to release/1.15.x (#18732) backport of commit 1c262f0 Co-authored-by: David Yu <dyu@hashicorp.com>

…/1.15.x (#18746) backport of commit aa8dee9 Co-authored-by: Jeremy Jacobson <jeremy.jacobson@hashicorp.com>

* Show latest config in /v1/agent/self * remove license string in backport

#18762) backport of commit d2a0fe8 Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>

…#18777) backport of commit 8af681d Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>

backport of commit fe58132 Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>

…#18786) * Vault CA provider clean up previous default issuers (#18773) (cherry picked from commit 4dfca64)

…elease/1.15.x (#18845) backport of commit 5be0674 Co-authored-by: Blake Covarrubias <blake@covarrubi.as>

…appens after service deregistration into release/1.15.x (#18856) Fix gateway services cleanup where proxy deregistration happens after service deregistration (#18831) * Fix gateway services cleanup where proxy deregistration happens after service deregistration * Add test * Add changelog * Fix comment

backport of commit aba7a72 Co-authored-by: david3a <49253132+david3a@users.noreply.github.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

…18385) * backport of commit 317585c * backport of commit 7ca2a56 --------- Co-authored-by: boruszak <jeffrey.boruszak@hashicorp.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

…18885) NET-4519 Collecting journald logs in "consul debug" bundle (#18797) * debug since * fix docs * chagelog added * fix go mod * debug test fix * fix test * tabs test fix * Update .changelog/18797.txt --------- Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>

* Update api version to 1.25.0 * Update to 1.21.1 of api

…1.15.x (#18890) docs: Fix HCL, JSON, and YAML syntax errors (#18879) This commit fixes syntax errors in HCL, JSON, and YAML example configurations. In some cases, it replaces the code example with the proper format for the code block. Also fixes HCL formatting and misc opportunistic updates to codeblock. Co-authored-by: Blake Covarrubias <blake@covarrubi.as> Co-authored-by: Tu Nguyen <im2nguyen@gmail.com>

…lease/1.15.x (#18893) Fix code block examples on dns-static-lookups.mdx HCL and JSON configuration examples were being displayed in the same code block. This commit separates the configurations to properly display them as independent configuration examples. Co-authored-by: Blake Covarrubias <blake@covarrubi.as>

…ase/1.15.x (#18905) backport of commit 06c2aaa Co-authored-by: Blake Covarrubias <blake@covarrubi.as>

….x (#18911) backport of commit 30b6777 Co-authored-by: Blake Covarrubias <blake@covarrubi.as>

* Add changelog and put back replace statements * bump version

* no-op commit due to failed cherry-picking * Update base ref property name * Test skip ci (#18924) test_push_merge * cleanup test push code --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: NiniOak <anita.akaeze@hashicorp.com>

…release/1.15.x (#18933) * docs: Fix some comments (#17118) Signed-off-by: cui fliter <imcusg@gmail.com> * [NET-4703] Prevent partial application of Envoy extensions (#18068) Prevent partial application of Envoy extensions Ensure that non-required extensions do not change xDS resources before exiting on failure by cloning proto messages prior to applying each extension. To support this change, also move `CanApply` checks up a layer and make them prior to attempting extension application, s.t. we avoid unnecessary copies where extensions can't be applied. Last, ensure that we do not allow panics from `CanApply` or `Extend` checks to escape the attempted extension application. * docs: Simplify example jq commands by removing pipes (#18327) Simplify jq command examples by removing pipes to other commands. * fix typo in create a mesh token docs (#18337) * [CC-5719] Add support for builtin global-read-only policy (#18319) * [CC-5719] Add support for builtin global-read-only policy * Add changelog * Add read-only to docs * Fix some minor issues. * Change from ReplaceAll to Sprintf * Change IsValidPolicy name to return an error instead of bool * Fix PolicyList test * Fix other tests * Apply suggestions from code review Co-authored-by: Paul Glass <pglass@hashicorp.com> * Fix state store test for policy list. * Fix naming issues * Update acl/validation.go Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com> * Update agent/consul/acl_endpoint.go --------- Co-authored-by: Paul Glass <pglass@hashicorp.com> Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com> * [NET-5121] proxystate: move protos to subdirectory to avoid conflicts (#18335) * also makes a few protos updates * resource: adding various helpers for working with resources (#18342) This is a bit of a grab bag of helpers that I found useful for working with them when authoring substantial Controllers. Subsequent PRs will make use of them. * Wasm integration tests for local and remote wasm files (#17756) * wasm integration tests for local and remote wasm files refactoring and cleanup for wasm testing remove wasm debug logging PR feedback, wasm build lock correct path pattern for wasm build files Add new helper function to minimize changes to existing test code Remove extra param mod tidy add custom service setup to test lib add wait until static server sidecar can reach nginx sidecar Doc comments PR feedback Update workflows to compile wasm for integration tests Fix docker build path Fix package name for linter Update makefile, fix redeclared function Update expected wasm filename Debug test ls in workflow remove pwd in favor of relative path more debugging Build wasm in compatability tests as well Build wasm directly in ci rather than in container Debug tinygo and llvm version Change wasm file extension Remove tinygo debugging Remove extra comments * Add compiled wasm and build instructions * [HCP Telemetry] Periodic Refresh for Dynamic Telemetry Configuration (#18168) * OTElExporter now uses an EndpointProvider to discover the endpoint * OTELSink uses a ConfigProvider to obtain filters and labels configuration * improve tests for otel_sink * Regex logic is moved into client for a method on the TelemetryConfig object * Create a telemetry_config_provider and update deps to use it * Fix conversion * fix import newline * Add logger to hcp client and move telemetry_config out of the client.go file * Add a telemetry_config.go to refactor client.go * Update deps * update hcp deps test * Modify telemetry_config_providers * Check for nil filters * PR review updates * Fix comments and move around pieces * Fix comments * Remove context from client struct * Moved ctx out of sink struct and fixed filters, added a test * Remove named imports, use errors.New if not fformatting * Remove HCP dependencies in telemetry package * Add success metric and move lock only to grab the t.cfgHahs * Update hash * fix nits * Create an equals method and add tests * Improve telemetry_config_provider.go tests * Add race test * Add missing godoc * Remove mock for MetricsClient * Avoid goroutine test panics * trying to kick CI lint issues by upgrading mod * imprve test code and add hasher for testing * Use structure logging for filters, fix error constants, and default to allow all regex * removed hashin and modify logic to simplify * Improve race test and fix PR feedback by removing hash equals and avoid testing the timer.Ticker logic, and instead unit test * Ran make go-mod-tidy * Use errtypes in the test * Add changelog * add safety check for exporter endpoint * remove require.Contains by using error types, fix structure logging, and fix success metric typo in exporter * Fixed race test to have changing config values * Send success metric before modifying config * Avoid the defer and move the success metric under * NET-4240 - Snapshots are failing on Windows (#18302) * fix go mod * fix go sum * added change log * ran make go mod tidy * Register ProxyStateTemplate Resource (#18316) Also, change the ProxyState.id to identity. This is because we already have the id of this proxy from the resource, and this id should be name-aligned with the workload it represents. It should also have the owner ref set to the workload ID if we need that. And so the id field seems unnecessary. We do, however, need a reference to workload identity so that we can authorize the proxy when it initially connects to the xDS server. * [NET-5146] security: Update Go version to 1.20.7 and `x/net` to 0.13.0 (#18358) * Update Go version to 1.20.7 This resolves [CVE-2023-29409] (https://nvd.nist.gov/vuln/detail/CVE-2023-29409)(`crypto/tls`). * Bump golang.org/x/net to 0.13.0 Addresses [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978) for security scans (non-impacting). * mesh: adding the protobuf types and resources backing mesh config v2 (#18351) * [OSS] test: xds coverage for routes (#18369) test: xds coverage for routes * Fix policy lookup to allow for slashes (#18347) * Fix policy lookup to allow for slashes * Fix suggestions * Fix other test * Revert some lines * add some initial CODEOWNERS (#18346) * Add redirects for mesh-gateway docs (#18377) * expose grpc as http endpoint (#18221) expose resource grpc endpoints as http endpoints * bimapper: fix a bug and add some more test coverage (#18387) * [docs] Fix ServiceDefaults example in distributed tracing (#17212) Fix ServiceDefaults example in distributed tracing. * Fix #17730 - Dev mode has new line (#18367) * adding new line only in case of pretty in url not in dev mode * change log added * chore: make go-mod-tidy (#18388) * resource: Make resource read tenancy aware (#18397) * Not using chmod - fixed integration test for Enterprise (#18401) * Not using chmod * checking icacls * Removed push * replaced ordered list of rate limit ops with flow diagram (#18398) * replaced ordered list of rate limit ops with flow diagram * made diagram font bigger * updated colors for dark mode img * fixed typo * recentered dark mode image * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Upgrade test: remove outdated test and disable log due to verbosity (#18403) * remove outdated test * disable log since we have too many parallel tests * bump testcontainers-go from 0.22.0 and remove pinned go version in in… (#18395) * bump testcontainers-go from 0.22.0 and remove pinned go version in integ test * go mod tidy * Replace deprecated target.Authority with target.URL.Host * Add ServiceEndpoints Mutation hook tests (#18404) * Add ServiceEndpoints Mutation hook tests * Move endpoint owner validation into the validation hook Also there were some minor changes to error validation to account for go-cmp not liking to peer through an errors.errorstring type that get created by errors.New * catalog: add FailoverPolicy mutation and validation hooks (#18390) Add most of the validation and mutation hooks for the FailoverPolicy resource. * NET-5115 Add retry + timeout filters for api-gateway (#18324) * squash, implement retry/timeout in consul core * update tests * catalog: adding a controller to reconcile FailoverPolicy resources (#18399) Add most of the semantic cross-resource validation for FailoverPolicy resources using a new controller. * fix the error in ent repo (#18421) fix the error in ent repo * build: updates for 1.16.1 release (#18415) * build: updates for 1.16.1 release * build: fix missing replace directive for envoyextensions * build: go mod tidy testing/deployer * update ECS links (#18419) * docs: fix incorrect proxy-defaults config in Lua Envoy extension (#18418) fix incorrect proxy-defaults config * resource: Make resource write tenancy aware (#18423) * [NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437) * Add logging to locality policy application In OSS, this is currently a no-op. * Inherit locality when registering sidecars When sidecar locality is not explicitly configured, inherit locality from the proxied service. * NET-4984: Update APIGW Config Entries for JWT Auth (#18366) * Added oss config entries for Policy and JWT on APIGW * Updated structs for config entry * Updated comments, ran deep-copy * Move JWT configuration into OSS file * Add in the config entry OSS file for jwts * Added changelog * fixing proto spacing * Moved to using manually written deep copy method * Use pointers for override/default fields in apigw config entries * Run gen scripts for changed types * k8s compat - Openshift versions (#18307) * Update compatibility.mdx Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * APIGW: Update HTTPRouteConfigEntry for JWT Auth (#18422) * Updated httproute config entry for JWT Filters * Added manual deepcopy method for httproute jwt filter * Fix test * Update JWTFilter to be in oss file * Add changelog * Add build tags for deepcopy oss file * [COMPLIANCE] License changes (#18443) * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * add missing license headers * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * Net-2708/delete resource endpoint (#18420) * feat: add http delete endpoint for resource service * refactor: clean up * Disable deep-copy codegen verification for now. (#18446) Once we figure out what to do with license headers for these files we should re-enable this check. * build: upgrade to latest buf v1.26.0 (#18426) * NET-4952 Add docs for export command (#18425) * Init file for services export command docs * Add link for export command to nav * Add export to list of subcommands for services * Add usage and basic command options * Add API options using includes * Add section for enterprise options * Include argument value in flag format * Add examples * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Net-2712/resource hcl parsing (#18250) * Initial protohcl implementation Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Co-authored-by: Daniel Upton <daniel@floppy.co> * resourcehcl: implement resource decoding on top of protohcl Co-authored-by: Daniel Upton <daniel@floppy.co> * fix: resolve ci failures * test: add additional unmarshalling tests * refactor: update function test to clean protohcl package imports --------- Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Co-authored-by: Daniel Upton <daniel@floppy.co> * read endpoint (#18268) implement http read endpoint to expose resource grpc service read method * Net-2707/list resource endpoint (#18444) feat: list resources endpoint * Fix incorrect yaml in examples (#18463) * NET-5187: Upgrade test timeout due to log producer errors (#18461) Stop log producer before restarting container * [NET-4799] [OSS] xdsv2: listeners L4 support for connect proxies (#18436) * refactor to avoid future import cycles * resource: Make resource list tenancy aware (#18475) * [COMPLIANCE] License update (#18479) Update BUSL LICENSE to include licensed product and version. Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * docs: specify manual server metadata intervention (#18477) * docs: Update OpenShift compatibility (#18478) Update compatibility.mdx * resource: Make resource delete tenancy aware (#18476) resource: Make resource delete tenancy awarae * Add license-checker action that fails when any backported file contains BUSL header (#18485) * Add license-checker action that fails when any backported file contains BUSL header * Quote echoed variable to retain line breaks * Add ticket to reference for more details * docs: Update K8s TGW tutorial to reliably obtain role ID (#18474) The `grep` command used to obtain the ID for the terminating gateway role is not reliable in all scenarios. For example, if there is a similarly named role, the command may return the wrong role ID for the active terminating gateway instance. This commit updates the command to use jq to obtain the role ID. If multiple roles are found, jq will raise an error informing the user that it cannot reliably determine the role ID. * NET-5371 License checker pt2 (#18491) * Update grep command to work on ubuntu * Run license checker when new commits are pushed to PR * README - Update KV use case to Dynamic App Configuration (#18301) * Update README.md * Update kv.mdx * Add BUSL badge * README - re-order badges and update hub link (#18498) Update README.md * [NET-5163] Support locality testing in consul-container (#18484) * Support locality testing in consul-container Support including locality in client sidecar config. Also align test config structs with Ent to avoid future conflicts. * Refactor consul-container fortio helpers Refactor fortio test helpers to separate HTTP retries from waiting on fortio result changes due to e.g. service startup and failovers. * NET-4853 - xds v2 - implement base connect proxy functionality for clusters (#18499) * Fix HCL (#18513) * Fix HCL * Update create-sameness-groups.mdx * NET-4932 - xds v2 - implement base connect proxy functionality for endpoints (#18500) * NET-4853 - xds v2 - implement base connect proxy functionality for clusters * NET-4853 - xds v2 - implement base connect proxy functionality for clusters * NET-4932 - xds v2 - implement base connect proxy functionality for endpoints * Update endpoints_test.go * gofmt * Update naming.go * Support custom watches on controller (#18439) * Support custom watches on controller * refactor mapper methods * NET-4858 - xds v2 - implement base connect proxy functionality for routes (#18501) * NET-4853 - xds v2 - implement base connect proxy functionality for clusters * NET-4853 - xds v2 - implement base connect proxy functionality for clusters * NET-4932 - xds v2 - implement base connect proxy functionality for endpoints * Update endpoints_test.go * gofmt * NET-4858 - Make connect proxy route tests pass using xds v2 * Update endpoints_test.go * Update naming.go * use alsoRunTestForV2 * remove unused makeAddress * gofmt * fixing clusters * bimapper: allow to untrack links and support reference or id (#18451) * bimapper: fix data race (#18519) * CI Split integration tests to run nightly and every PR (#18518) * CI Split integration tests to run nightly and every PR * Checkout release branch for nightly test * Fix broken links caught in weekly report (#18522) * fix broken link caught in weekly report * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Create nightly test-integration for consul release branch (#18530) * Create nightly test-integration for consul release branch * test * fix * docs: Fix spelling errors across various pages on the site (#18533) This commit fixes numerous spelling errors across the site and also removes unnecessary whitespace that was present in the edited files. * Make proto-public license MPL (#18531) * Make proto-public license MPL * Add proto-public dir to exclusion list in .copywrite.hcl * NET-4943 - Implement ProxyTracker (#18535) * resource: Make resource watchlist tenancy aware (#18539) * Reduce required type arguments for DecodedResource (#18540) * OSS -> CE (community edition) changes (#18517) * CI: fix envoy versions in CI of release branch (#18538) * CI: fix envoy versions in CI of release branch * - remove steps sds for 1.15 nightly run - checkout the release branch * add default name * fix * Update go-tests.yml * fix checkout code * add comments and revert schedule * catalog: add failover mode enum validation (#18545) * mesh: add validation for the new pbmesh resources (#18410) Adds validation for HTTPRoute, GRPCRoute, TCPRoute, DestinationPolicy, and ComputedRoutes. * xds controller: resolve ServiceEndpoints references in ProxyStateTemp… (#18544) xds controller: resolve ServiceEndpoints references in ProxyStateTemplate * mesh: adding type aliases for mesh resource usage (#18448) Introduces some simple type aliases for DecodedResource[*X] wrappers for each type which cut down on the verbosity * Update trust bundle into proxy-state-template (#18550) * catalog: validating Protocol and Health enums on Service, Workload, and ServiceEndpoints (#18554) * chore: fix missing/incorrect license headers (#18555) * fix for , non presence of consul-version meta (#18464) * fix for #18406 , non presence of consul-version meta * removed redundant checks * updated mock-api to mimic api response for synthetic nodes * added test to test getDistinctConsulVersions method with synthetic-node case * updated typo in comments * added change log * feat: add experiments flag to testserver sdk (#18541) * bug: prevent go routine leakage due to existing DeferCheck (#18558) * bug: prevent go routine leakage due to existing DeferCheck * add changelog * NET-5382 & PLAT-1159: Do not trigger workflow if only doc files are in commit history (#18528) NET-5382: Do not run workflow runs if at least one path matches * UI: community verbiage (#18560) * chore: update community verbiage * chore: add changelog entry * Fix Windows FIPS Build (#18357) * Fix Windows FIPS Build * create new file for windows arm * removed build tag * fix buidl tags * fix multiple go build * CE port of enterprise extension (#18572) CE commit * resource: Make resource listbyowner tenancy aware (#18566) * NET-5147: Added placeholder structs for JWT functionality (#18575) * Added placeholder structs for JWT functionality * Added watches for CE vs ENT * Add license header * Undo plumbing work * Add context arg * resource: Make resource writestatus tenancy aware (#18577) * NET-5338 - NET-5338 - Run a v2 mode xds server (#18579) * NET-5338 - NET-5338 - Run a v2 mode xds server * fix linting * CI: send slack notification on failed nightly job (#18578) * CE commit (#18583) * Fix broken link on sameness group page (#18511) fix broken link * Fix typo in permissive mTLS docs (#18551) Update onboarding-tproxy-mode.mdx * Set concurrency for workflows (#18567) * Set concurrency for workflows * test: run automated tests against Vault 1.11 - 1.14 (#18590) Begin to test against Vault 1.14. Drop tests against Vault 1.10. * Reduce the frequency of metric exports to minutely (#18584) * NET-4944 - wire up controllers with proxy tracker (#18603) Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com> * Watch the ProxyTracker from xDS controller (#18611) * Distinguish v2 catalog Protocol unset from default (#18612) Even though we intend to default to TCP when this field is not explicitly provided, uncluding an `UNSPECIFIED` default enum value allows us to create inheritance chains, e.g. service to workload. * Added OpenTelemetry Access Logging Envoy extension (#18336) * [HCP Telemetry] Move first TelemetryConfig Fetch into the TelemetryConfigProvider (#18318) * Add Enabler interface to turn sink on/off * Use h for hcpProviderImpl vars, fix PR feeback and fix errors * Keep nil check in exporter and fix tests * Clarify comment and fix function name * Use disable instead of enable * Fix errors nit in otlp_transform * Add test for refreshInterval of updateConfig * Add disabled field in MetricsConfig struct * Fix PR feedback: improve comment and remove double colons * Fix deps test which requires a maybe * Update hcp-sdk-go to v0.61.0 * use disabled flag in telemetry_config.go * Handle 4XX errors in telemetry_provider * Fix deps test * Check 4XX instead * Run make go-mod-tidy * update comments and docs about running envoy integration tests with the ENVOY_VERSION set. (#18614) update ENVOY_VERSION and documentation of it used in the bats envoy tests. Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com> * delete all v2 resources type when deleting a namespace (CE) (#18621) * add namespace scope to ServiceV1Alpha1Type * add CE portion of namespace deletion * resource: Allow nil tenancy (#18618) * PLAT-1192 - Run CI on smaller instances (#18624) Use smaller runners * NET-3181 - Allow log file naming like Nomad (#18617) * fixes file name for consul * added log file * added tests for rename method * Add the plumbing for APIGW JWT work (#18609) * Add the plumbing for APIGW JWT work * Remove unneeded import * Add deep equal function for HTTPMatch * Added plumbing for status conditions * Remove unneeded comment * Fix comments * Add calls in xds listener for apigateway to setup listener jwt auth * docs: admin partition and DNS clarification (#18613) * fix: make UNSPECIFIED protocol pass validation (#18634) We explicitly enumerate the allowed protocols in validation, so this change is necessary to use the new enum value. Also add tests for enum validators to ensure they stay aligned to protos unless we explicitly want them to diverge. * NET 1594 - Snapshot Agent Filename Should Include Consul Version / Datacenter (#18625) * init * tests added and few fixes * revert arg message * changelog added * removed var declaration * fix CI * fix test * added node name and status * updated save.mdx * added example * fix tense * fix description * Net 3181 consul gh issue 15709 allow log file naming like nomad - fix bug (#18631) * fixes file name for consul * added log file * added tests for rename method * append instead of trunc * fix file truncate issue * added changelog * fix for build destros ci * removed changelog * solaris * resource: Require scope for resource registration (#18635) * [NET-4958] Fix issue where envoy endpoints would fail to populate after snapshot restore (#18636) Fix issue where agentless endpoints would fail to populate after snapshot restore. Fixes an issue that was introduced in #17775. This issue happens because a long-lived pointer to the state store is held, which is unsafe to do. Snapshot restorations will swap out this state store, meaning that the proxycfg watches would break for agentless. * Add extra logging for mesh health endpoints. (#18647) * feat: Change global-read-only policy to non editable (#18602) * remove stray FIPS file (#18651) * Add known issue notice for #18636. (#18650) * fix: remove Intention test with latency to make consul-enterprise rep… (#18654) fix: remove Intention test with latency to make consul-enterprise repo tests work * fix: add telemetry bind socket to proxyConfiguration proto (#18652) * feat: copy edits for built-in policy alert (#18655) * test: fix snapshot save w/ append test for ENT (#18656) * Net 2714/xw cli read command (#18462) enable `consul resource read` command in cli * change log for audit log (#18668) * change log for audit log * fix file name * breaking change * Add TCP+TLS Healthchecks (#18381) * Begin adding TCPUseTLS * More TCP with TLS plumbing * Making forward progress * Keep on adding TCP+TLS support for healthchecks * Removed too many lines * Unit tests for TCP+TLS * Update tlsutil/config.go Co-authored-by: Samantha <hello@entropy.cat> * Working on the tcp+tls unit test * Updated the runtime integration tests * Progress * Revert this file back to HEAD * Remove debugging lines * Implement TLS enabled TCP socket server and make a successful TCP+TLS healthcheck on it * Update docs * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update agent/checks/check.go Co-authored-by: Samantha <hello@entropy.cat> * Address comments * Remove extraneous bracket * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update the mockTLSServer * Remove trailing newline * Address comments * Fix merge problem * Add changelog entry --------- Co-authored-by: Samantha <hello@entropy.cat> * Fix windows Ent runner (#18683) * fix windows image for enterprise * added quotesT * Removed nodename and status from consul snapshot save -append-filename command and using leader version in version (#18680) * init * fix tests * fix tests lint * fix api call inside dc * updated doc * address comments * update guide to reflect tenancy and scope (#18687) * update guide to reflect tenancy and scope * Apply suggestions from code review Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * update ACLHooks signature * Update docs/resources/guide.md Co-authored-by: Semir Patel <semir.patel@hashicorp.com> --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * Add support for querying tokens by service name. (#18667) Add support for querying tokens by service name The consul-k8s endpoints controller has a workflow where it fetches all tokens. This is not performant for large clusters, where there may be a sizable number of tokens. This commit attempts to alleviate that problem and introduces a new way to query by the token's service name. * restore common-topo peering integration tests, but nightly [NET-5476] (#18673) * dataplane: Allow getting bootstrap parameters when using V2 APIs (#18504) This PR enables the GetEnvoyBootstrapParams endpoint to construct envoy bootstrap parameters from v2 catalog and mesh resources. * Make bootstrap request and response parameters less specific to services so that we can re-use them for workloads or service instances. * Remove ServiceKind from bootstrap params response. This value was unused previously and is not needed for V2. * Make access logs generation generic so that we can generate them using v1 or v2 resources. * switch all client nodes in dc2 to dataplane [NET-4299] (#18608) * Reformat proxy docs refresh (#18623) * first commit; reformat PD conf entry * updated proxies overview page * added Deploy SM proxy usage and removed reg index * moved sidecar proxy usage to main proxy folder * recast sidecar reg page as Deploy sidecar services * fix typos * recast SM reg as conf reference- set the sidebar * add redirects * fix links * add PD conf entry usage to appropro pages * edits to proxy conf ref * fix links on index page * example command to write PD conf entry * updated links to old SM proxy reg page * updated links to sidecar service reg page * tryna fix front matter issues * Apply suggestions from code review Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com> * added paragraph about SM proxies to overivew * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * sidecar-proxy controller: L4 controller with explicit upstreams (NET-3988) (#18352) * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * This commit also changes service endpoints to include workload identity. This made the implementation a bit easier as we don't need to look up as many workloads and instead rely on endpoints data. * test: update sidecarproxy/builder golden tests to use determinstic golden data (#18703) * [APIGW] NET-5017 JWT Cleanup/Status Conditions (#18700) * Fixes issues in setting status * Update golden files for changes to xds generation to not use deprecated methods * Fixed default for validation of JWT for route * Added changelog for jwt features (#18709) * fix: NET-1521 show latest config in /v1/agent/self (#18681) * fix: NET-1521 show latest config in /v1/agent/self * Using larger machine for Enterprise (#18713) using 4x large for ent * [NET-5325] ACL templated policies support in tokens and roles (#18708) * [NET-5325] ACL templated policies support in tokens and roles - Add API support for creating tokens/roles with templated-policies - Add CLI support for creating tokens/roles with templated-policies * adding changelog * Adds PassiveHealthCheck Fields to ServiceDefaults and IngressGateway (#18532) Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * Instantiate secure resource service client after the grpc server (#18712) * ENT port: test-integ/peering: peer through mesh gateway [NET-4609] (#18605) Co-authored-by: Matt Keeler <mjkeeler7@gmail.com> * [CC-6039] Update builtin policy descriptions (#18705) * Net-2713/resource apply command (#18610) * feat: implement apply command * fix: resolve lint issues * refactor: use fixed types * fix: ci lint failures after main rebase * NET-5530 Support response header modifiers on http-route config entry (#18646) * Add response header filters to http-route config entry definitions * Map response header filters from config entry when constructing route destination * Support response header modifiers at the service level as well * Update protobuf definitions * Update existing unit tests * Add response filters to route consolidation logic * Make existing unit tests more robust * Add missing docstring * Add changelog entry * Add response filter modifiers to existing integration test * Add more robust testing for response header modifiers in the discovery chain * Add more robust testing for request header modifiers in the discovery chain * Modify test to verify that service filter modifiers take precedence over rule filter modifiers * fix: emit consul version metric on a regular interval (#18724) * [NET-5330] Support templated policies in Binding rules (#18719) * [NET-5330] Support templated policies in Binding rules * changelog for templated policy support in binding rules * NET-5530 Generate deep-copy code (#18730) * Generate deep-copy code * Undo license header removal * sidecar-proxy controller: Add support for transparent proxy (NET-5069) (#18458) This commit adds support for transparent proxy to the sidecar proxy controller. As we do not yet support inferring destinations from intentions, this assumes that all services in the cluster are destinations. * Prefix sidecar proxy test files with source and destination. (#18620) * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Address PR comments * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * PR review comments * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Make sure endpoint refs route to mesh port instead of an app port * Address PR comments * fixing copyright * tidy imports * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * tidy imports * add copyright headers * Prefix sidecar proxy test files with source and destination. * Update controller_test.go --------- Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com> * k8s: Update audit-logging docs to include K8s examples (#18721) Update audit-logging.mdx * Fix windows integrations tests machine size (#18736) fix windows integrations tests machine size * Increase timeout for integration test windows. (#18737) increase timeout * Net-4292/list command (#18649) * feat: implement list command * refactor: apply command file parsing * Added logs for debugging (#18740) * added logs * added echo * removed pull request * xds: update golden tests to be deterministic (#18707) * Run copyright after running deep-copy as part of the Makefile/CI (#18741) * execute copyright headers after performing deep-copy generation. * fix copyright install * Apply suggestions from code review Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * Apply suggestions from code review Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * rename steps to match codegen naming * remove copywrite install category --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * [NET-5333] Add api to read/list and preview templated policies (#18748) * NET-5132 - Configure multiport routing for connect proxies in TProxy mode (#18606) * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Address PR comments * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * PR review comments * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Make sure endpoint refs route to mesh port instead of an app port * Address PR comments * fixing copyright * tidy imports * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * tidy imports * add copyright headers * Prefix sidecar proxy test files with source and destination. * Update controller_test.go * NET-5132 - Configure multiport routing for connect proxies in TProxy mode * formatting golden files * reverting golden files and adding changes in manually. build implicit destinations still has some issues. * fixing files that were incorrectly repeating the outbound listener * PR comments * extract AlpnProtocol naming convention to getAlpnProtocolFromPortName(portName) * removing address level filtering. * adding license to resources_test.go --------- Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com> * added time stamps (#18757) * fix: write endpoint errors out gracefully (#18743) * [NET-5574] Update Go version to 1.20.8 (#18742) Update Go version to 1.20.8 This resolves several CVEs (see changelog entry). * Add Envoy golden test for OTEL access logging extension (#18760) * mesh: create new routes-controller to reconcile xRoute types into a ComputedRoutes resource (#18460) This new controller produces an intermediate output (ComputedRoutes) that is meant to summarize all relevant xRoutes and related mesh configuration in an easier-to-use format for downstream use to construct the ProxyStateTemplate. It also applies status updates to the xRoute resource types to indicate that they are themselves semantically valid inputs. * xds controller: setup watches for and compute leaf cert references in ProxyStateTemplate, and wire up leaf cert manager dependency (#18756) * Refactors the leafcert package to not have a dependency on agent/consul and agent/cache to avoid import cycles. This way the xds controller can just import the leafcert package to use the leafcert manager. The leaf cert logic in the controller: * Sets up watches for leaf certs that are referenced in the ProxyStateTemplate (which generates the leaf certs too). * Gets the leaf cert from the leaf cert cache * Stores the leaf cert in the ProxyState that's pushed to xds * For the cert watches, this PR also uses a bimapper + a thin wrapper to map leaf cert events to related ProxyStateTemplates Since bimapper uses a resource.Reference or resource.ID to map between two resource types, I've created an internal type for a leaf certificate to use for the resource.Reference, since it's not a v2 resource. The wrapper allows mapping events to resources (as opposed to mapping resources to resources) The controller tests: Unit: Ensure that we resolve leaf cert references Lifecycle: Ensure that when the CA is updated, the leaf cert is as well Also adds a new spiffe id type, and adds workload identity and workload identity URI to leaf certs. This is so certs are generated with the new workload identity based SPIFFE id. * Pulls out some leaf cert test helpers into a helpers file so it can be used in the xds controller tests. * Wires up leaf cert manager dependency * Support getting token from proxytracker * Add workload identity spiffe id type to the authorize and sign functions --------- Co-authored-by: John Murret <john.murret@hashicorp.com> * Test: NET-5592 - update Nomad integration testing (#18768) NET-5592 - update Nomad integration testing * Add workload identity ACL rules (#18769) * Windows Integration Test Changes (#18758) * some changes to debug * revert machines * increased timeout * added sleep 10 seconds before test start * chagne envoy version * removed sleep * revert timeout * replace position * removed date * Revert "[NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437)" This reverts commit 05604eeec1186a4c8a6156e7519a579171659c1a. * fix build * Revert "replace position" This reverts commit 48e6af46a8daae186c283f30d316b1104906993e. * Revert "Revert "[NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437)"" This reverts commit d7c568e2be727b72e6827225782e0e17ac06b74a. * comment out api gateway http hostnames test * fix import * revert integ test run on PR * Add V2 TCP traffic permissions (#18771) Add support for TCP traffic permissions * resource: add helper to normalize inner Reference tenancy during mutate (#18765) When one resource contains an inner field that is of type *pbresource.Reference we want the Tenancy to be reasonably defaulted by the following rules: 1. The final values will be limited by the scope of the referenced type. 2. Values will be inferred from the parent's tenancy, and if that is insufficient then using the default tenancy for the type's scope. 3. Namespace will only be used from a parent if the reference and the parent share a partition, otherwise the default namespace will be used. Until we tackle peering, this hard codes an assumption of peer name being local. The logic for defaulting may need adjustment when that is addressed. * Disable docs change CI skip detection (#18775) There's currently a bug that causes CI to be skipped on all non-PR changes. Until that's fixed and we can be certain the check will fail CI or default to running tests in the case of errors, disabling this check. * delete command ready (#18679) * delete command for resource management * Vault CA provider clean up previous default issuers (#18773) * Fix snapshot creation issue. (#18783) The renaming of files from oss -> ce caused incorrect snapshots to be created due to ce writes now happening prior to ent writes. When this happens various entities will attempt to be restored from the snapshot prior to a partition existing and will cause a panic to occur. * catalog: service endpoints inherits protocol from service when workload doesn't have one (#18792) * add fuzz tests to resourcehcl package and fix some panics (#18798) * add a simple fuzz test for the resourcehcl package and fix some panics uncovered by the test * fix default for null values * Clean up resources in test (#18799) * Updates gateway-class-config docs to include fields for privileged container port-mapping and Openshift Security Context Constraints Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * Removes errant word in docs that was breaking the build (#18804) * Fix flaky test (#18805) * register traffic permission and workload identity types (#18704) * add workload identity and traffic permission protos * register new types * add generated pb code * fix exports.go path * add proto newlines * fix type name Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com> * address review * fix protos and add tests * fix validation constraints * add tests --------- Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com> * [NET-5329] use acl templated policy under the hood for node/service identities (#18813) * [NET-5334] Added CLI commands for templated policies (#18816) * mesh: Wire ComputedRoutes into the ProxyStateTemplate via the sidecar controller (#18752) Reworks the sidecar controller to accept ComputedRoutes as an input and use it to generate appropriate ProxyStateTemplate resources containing L4/L7 mesh configuration. * mesh: call the right mapper (#18818) * fix templated policy cli output (#18821) * resource: default peername to "local" for now (#18822) * Wire up traffic permissions (#18812) Wire up traffic permissions * added consul and envoy version constraints (#18726) * added consul and envoy version constraints * fixed Destination configuraiton and added tproxy requirement * Apply suggestions from code review Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> --------- Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> * Default to tcp protocol when workload protocol is unspecified (#18824) * Retry and timeout test acceptance test (#18791) * retry and timeout test * add docker mirrior * checkpoint * add in error * add in delay * up error rate * fix status code * Use embedded strings for templated policies (#18829) * consul operator raft transfer-leader should send the id (#17107) Fixes #16955 Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> * Allow empty data writes for resources (#18819) * allow nil data writes for resources * update demo to test valid type with no data * Fix nondeterministic test (#18828) * mesh: prevent writing a ComputedRoutes with no ported configs (#18833) * catalog: Default protocol to tcp in catalog.Service if unspecified (#18832) * Adjust metrics test (#18837) * Audit log consul 1.17x changes (#18669) * audit log consul 1.17x changes * added some details * verbose * Fixes for integration tests windows for ENT (#18839) * fixes for integration tests * fix runner size for enterprise * fix spacing * fix spacing * removed branch test run * fix: provide meaningful error messages and add test (#18772) * fix: provide meaningful error messages and add test * fix: return error instead of warning when extra args are provided * add v2 tenancy bridge Flag and v2 Tenancy Bridge initial implementation (#18830) * add v2 tenancy bridge and a feature flag for v2 tenancy * move tenancy bridge v2 under resource package * docs: Fix typo in description for server_addresses (#18838) Change 'If not port' to 'If no port'. Resolves #18553 * grafana: display connected consul-dataplanes (#18842) * Adding Apigee for ext_authz, minor fix in the default ext_authz docs (#18796) * adding apigee for ext_authz, minor fix * adding the Apigee docs to nav * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Blake Covarrubias <blake.covarrubias@gmail.com> * addressing feedback * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Blake Covarrubias <blake.covarrubias@gmail.com> * catalog: normalize/default/validate tenancy components of FailoverPolicy internal References (#18825) FailoverPolicy resources contain inner Reference fields. We want to ensure that components of those reference Tenancy fields left unspecified are defaulted using the tenancy of the enclosing FailoverPolicy resource. As the underlying helper being used to do the normalization calls the function modified in #18822, it also means that the PeerName field will be set to "local" for now automatically to avoid "local" != "" issues downstream. * Fix gateway services cleanup where proxy deregistration happens after service deregistration (#18831) * Fix gateway services cleanup where proxy deregistration happens after service deregistration * Add test * Add changelog * Fix comment * Move ACL templated policies to hcl files (#18853) * mesh: update xds controller to synthesize empty endpoints when no endpoints ref is found (#18835) * mesh: normalize/default/validate tenancy components of mesh internal References (#18827) HTTPRoute, GRPCRoute, TCPRoute, and Upstreams resources contain inner Reference fields. We want to ensure that components of those reference Tenancy fields left unspecified are defaulted using the tenancy of the enclosing resource. As the underlying helper being used to do the normalization calls the function modified in #18822, it also means that the PeerName field will be set to "local" for now automatically to avoid "local" != "" issues downstream. * resource: mutate and validate before acls on write (#18868) * Remove flaky test assertions (#18870) * mesh: only build tproxy outbound listener once per destination (#18836) Previously, when using implicit upstreams, we'd build outbound listener per destination instead of one for all destinations. This will result in port conflicts when trying to send this config to envoy. This PR also makes sure that leaf and root references are always added (before we would only add it if there are inbound non-mesh ports). Also, black-hole traffic when there are no inbound ports other than mesh * NET-4519 Collecting journald logs in "consul debug" bundle (#18797) * debug since * fix docs * chagelog added * fix go mod * debug test fix * fix test * tabs test fix * Update .changelog/18797.txt Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com> --------- Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com> * CI: lint test-integ (#18875) * CI: lint test-integ * fix lint error * Honor Default Traffic Permissions in V2 (#18886) wire up v2 default traffic permissions * docs: Fix HCL, JSON, and YAML syntax errors (#18879) This commit fixes syntax errors in HCL, JSON, and YAML example configurations. In some cases, it replaces the code example with the proper format for the code block. Also fixes HCL formatting and misc opportunistic updates to codeblock. Co-authored-by: Tu Nguyen <im2nguyen@gmail.com> * Fix code block examples on dns-static-lookups.mdx (#18880) HCL and JSON configuration examples were being displayed in the same code block. This commit separates the configurations to properly display them as independent configuration examples. * docs: Fix Kubernetes CRD example configs (#18878) Fixes configuration examples for several Consul Kubernetes CRDs. The CRDs were missing required fields such as `apiVersion`, `metadata`, and `spec`. Co-authored-by: Tu Nguyen <im2nguyen@gmail.com> * docs: Apigee extension backport (#18847) * commit * link text edits * Add operator audit endpoint changes (#18899) * docs: Remove YAML service registration examples (#18877) Remove YAML service registration examples and replace them with JSON. This is because YAML is not a supported configuration format for the Consul's agent configuration, nor is it supported by the HTTP API. This commit replaces the YAML examples with JSON and adds additional JSON examples where they were missing. * Add otel-access-logging Envoy extension integration test (#18898) * Add note about service upstream env var dot broken (#18895) * add note about service upstream env var dot broken * Apply suggestions from code review Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --------- Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * xds: Use downstream protocol when connecting to local app (#18573) Configure Envoy to use the same HTTP protocol version used by the downstream caller when forwarding requests to a local application that is configured with the protocol set to either `http2` or `grpc`. This allows upstream applications that support both HTTP/1.1 and HTTP/2 on a single port to receive requests using either protocol. This is beneficial when the application primarily communicates using HTTP/2, but also needs to support HTTP/1.1, such as to respond to Kubernetes HTTP readiness/liveness probes. Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com> * resource: ensure resource.AuthorizerContext properly strips the local… (#18908) resource: ensure resource.AuthorizerContext properly strips the local peer name * docs: Add complete auth method payloads (#18849) This commit modifies the example payloads for various auth methods to remove 'other fields' and instead use complete example payloads. * agent/config: prevent startup if resource-apis experiment and cloud are enabled (#18876) * v2: various fixes to make K8s tproxy multiport acceptance tests and manual explicit upstreams (single port) tests pass (#18874) Adding coauthors who mobbed/paired at various points throughout last week. Co-authored-by: Dan Stough <dan.stough@hashicorp.com> Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: John Murret <john.murret@hashicorp.com> Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com> Co-authored-by: Michael Wilkerson <mwilkerson@hashicorp.com> * Added changelog entries for 1.14.10, 1.15.6, 1.16.2 (#18917) * [NET-5332] Add nomad server templated policy (#18888) * [NET-5332] Add nomad server templated policy * slksfd * mesh: make FailoverPolicy work in xdsv2 and ProxyStateTemplate (#18900) Ensure that configuring a FailoverPolicy for a service that is reachable via a xRoute or a direct upstream causes an envoy aggregate cluster to be created for the original cluster name, but with separate clusters for each one of the possible destinations. * Fix changelog order (#18918) * Fix changelog order * fix ordering or entries * Add namespace proto and registration (#18848) * add namespace proto and registration * fix proto generation * add missing copywrite headers * fix proto linter errors * fix exports and Type export * add mutate hook and more validation * add more validation rules and tests * Apply suggestions from code review Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * fix owner error and add test * remove ACL for now * add tests around space suffix prefix. * only fait when ns and ap are default, add test for it --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * Update base ref property name (#18851) * Update base ref property name * Test skip ci (#18924) test_push_merge * cleanup test push code * dns token (#17936) * dns token fix whitespace for docs and comments fix test cases fix test cases remove tabs in help text Add changelog Peering dns test Peering dns test Partial implementation of Peered DNS test Swap to new topology lib expose dns port for integration tests on client remove partial test implementation remove extra port exposure remove changelog from the ent pr Add dns token to set-agent-token switch Add enterprise golden file Use builtin/dns template in tests Update ent dns policy Update ent dns template test remove local gen certs fix templated policy specs * add changelog * go mod tidy * NET-5131 - support multiple ported upstreams tests (#18923) * add multiple upstream ports to golden file test for destination builder * NET-5131 - add unit tests for multiple ported upstreams * fix merge conflicts * iterate through array * backport of commit e5531f2c24b3a29186fff775761e5b81946de5e2 * backport of commit 1caef16d58e7c10c0e69ace838f45d1f46f9fff7 --------- Signed-off-by: cui fliter <imcusg@gmail.com> Co-authored-by: cui fliter <imcusg@gmail.com> Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> Co-authored-by: Blake Covarrubias <blake@covarrubi.as> Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com> Co-authored-by: Jeremy Jacobson <jjacobson93@users.noreply.github.com> Co-authored-by: Paul Glass <pglass@hashicorp.com> Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com> Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com> Co-authored-by: John Landa <jonathanlanda@gmail.com> Co-authored-by: Ashvitha <ashvitha.sridharan@hashicorp.com> Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com> Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> Co-authored-by: Dan Stough <dan.stough@hashicorp.com> Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com> Co-authored-by: wangxinyi7 <121973291+wangxinyi7@users.noreply.github.com> Co-authored-by: Andrea Scarpino <andrea@scarpino.dev> Co-authored-by: Semir Patel <semir.patel@hashicorp.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: cskh <hui.kang@hashicorp.com> Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com> Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com> Co-authored-by: Gerard Nguyen <gerard@hashicorp.com> Co-authored-by: John Maguire <john.maguire@hashicorp.com> Co-authored-by: David Yu <dyu@hashicorp.com> Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Poonam Jadhav <poonam.jadhav@hashicorp.com> Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: Daniel Upton <daniel@floppy.co> Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com> Co-authored-by: Anita Akaeze <anita.akaeze@hashicorp.com> Co-authored-by: Dan Bond <danbond@protonmail.com> Co-authored-by: John Murret <john.murret@hashicorp.com> Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com> Co-authored-by: Vijay <vijayraghav22@gmail.com> Co-authored-by: Natalie Smith <nataliemegans@gmail.com> Co-authored-by: Chris S. Kim <ckim@hashicorp.com> Co-authored-by: Curt Bushko <cbushko@gmail.com> Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> Co-authored-by: Joshua Timmons <josh.timmons@hashicorp.com> Co-authored-by: Hardik Shingala <34568645+hdkshingala@users.noreply.github.com> Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> Co-authored-by: Derek Menteer <105233703+hashi-derek@users.noreply.github.com> Co-authored-by: Valeriia Ruban <valeriia.ruban@hashicorp.com> Co-authored-by: skpratt <sarah.pratt@hashicorp.com> Co-authored-by: Phil Porada <pgporada@users.noreply.github.com> Co-authored-by: Samantha <hello@entropy.cat> Co-authored-by: Nick Irvine <115657443+nfi-hashicorp@users.noreply.github.com> Co-authored-by: Melisa Griffin <missylbytes@users.noreply.github.com> Co-authored-by: Matt Keeler <mjkeeler7@gmail.com> Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: Sophie Gairo <97480023+sophie-gairo@users.noreply.github.com> Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com> Co-authored-by: James Hartig <me@jameshartig.com> Co-authored-by: Gautam <gautambaghel93@gmail.com> Co-authored-by: Blake Covarrubias <blake.covarrubias@gmail.com> Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com> Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com> Co-authored-by: Tu Nguyen <im2nguyen@gmail.com> Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com> Co-authored-by: Nick Ethier <nethier@hashicorp.com>

backport of commit 24797c9 Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>

Update helm docs

…sts into release/1.15.x (#18955) backport of commit 89e15e4 Co-authored-by: NiniOak <anita.akaeze@hashicorp.com>

backport of commit cc40e08

…se/1.15.x (#18971) backport of commit 5d0edec Co-authored-by: Blake Covarrubias <blake@covarrubi.as>

…release/1.15.x (#18968) Backport of docs: Change heading to filename in CodeBlockConfig to release/1.15.x backport of commit 4e1e18f Co-authored-by: Blake Covarrubias <blake@covarrubi.as>

…into release/1.15.x (#18974) backport of commit 38c356c Co-authored-by: Andrea Scarpino <andrea@scarpino.dev>

…18976) * Update list of Envoy versions (#18300) Update supported envoy versions to 1.24.10, 1.25.9, 1.26.4, 1.27.0. * [NET-4904] Update list of Envoy versions in docs (#18306) Update list of Envoy versions in docs Update supported Envoy versions across Consul release versions. * Update actions for TSCCR (#18317) Update action versions before deadline * Fix topology view when displaying mixed connect-native/normal services. (#13023) * Fix topoloy intention with mixed connect-native/normal services. If a service is registered twice, once with connect-native and once without, the topology views would prune the existing intentions. This change brings the code more in line with the transparent proxy behavior. * Dedupe nodes in the ServiceTopology ui endpoint (like done with tags). * Consider a service connect-native as soon as one instance is. * Stop JWT provider from being written in non default namespace (#18325) * docs: Fix some comments (#17118) Signed-off-by: cui fliter <imcusg@gmail.com> * [NET-4703] Prevent partial application of Envoy extensions (#18068) Prevent partial application of Envoy extensions Ensure that non-required extensions do not change xDS resources before exiting on failure by cloning proto messages prior to applying each extension. To support this change, also move `CanApply` checks up a layer and make them prior to attempting extension application, s.t. we avoid unnecessary copies where extensions can't be applied. Last, ensure that we do not allow panics from `CanApply` or `Extend` checks to escape the attempted extension application. * docs: Simplify example jq commands by removing pipes (#18327) Simplify jq command examples by removing pipes to other commands. * fix typo in create a mesh token docs (#18337) * [CC-5719] Add support for builtin global-read-only policy (#18319) * [CC-5719] Add support for builtin global-read-only policy * Add changelog * Add read-only to docs * Fix some minor issues. * Change from ReplaceAll to Sprintf * Change IsValidPolicy name to return an error instead of bool * Fix PolicyList test * Fix other tests * Apply suggestions from code review Co-authored-by: Paul Glass <pglass@hashicorp.com> * Fix state store test for policy list. * Fix naming issues * Update acl/validation.go Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com> * Update agent/consul/acl_endpoint.go --------- Co-authored-by: Paul Glass <pglass@hashicorp.com> Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com> * [NET-5121] proxystate: move protos to subdirectory to avoid conflicts (#18335) * also makes a few protos updates * resource: adding various helpers for working with resources (#18342) This is a bit of a grab bag of helpers that I found useful for working with them when authoring substantial Controllers. Subsequent PRs will make use of them. * Wasm integration tests for local and remote wasm files (#17756) * wasm integration tests for local and remote wasm files refactoring and cleanup for wasm testing remove wasm debug logging PR feedback, wasm build lock correct path pattern for wasm build files Add new helper function to minimize changes to existing test code Remove extra param mod tidy add custom service setup to test lib add wait until static server sidecar can reach nginx sidecar Doc comments PR feedback Update workflows to compile wasm for integration tests Fix docker build path Fix package name for linter Update makefile, fix redeclared function Update expected wasm filename Debug test ls in workflow remove pwd in favor of relative path more debugging Build wasm in compatability tests as well Build wasm directly in ci rather than in container Debug tinygo and llvm version Change wasm file extension Remove tinygo debugging Remove extra comments * Add compiled wasm and build instructions * [HCP Telemetry] Periodic Refresh for Dynamic Telemetry Configuration (#18168) * OTElExporter now uses an EndpointProvider to discover the endpoint * OTELSink uses a ConfigProvider to obtain filters and labels configuration * improve tests for otel_sink * Regex logic is moved into client for a method on the TelemetryConfig object * Create a telemetry_config_provider and update deps to use it * Fix conversion * fix import newline * Add logger to hcp client and move telemetry_config out of the client.go file * Add a telemetry_config.go to refactor client.go * Update deps * update hcp deps test * Modify telemetry_config_providers * Check for nil filters * PR review updates * Fix comments and move around pieces * Fix comments * Remove context from client struct * Moved ctx out of sink struct and fixed filters, added a test * Remove named imports, use errors.New if not fformatting * Remove HCP dependencies in telemetry package * Add success metric and move lock only to grab the t.cfgHahs * Update hash * fix nits * Create an equals method and add tests * Improve telemetry_config_provider.go tests * Add race test * Add missing godoc * Remove mock for MetricsClient * Avoid goroutine test panics * trying to kick CI lint issues by upgrading mod * imprve test code and add hasher for testing * Use structure logging for filters, fix error constants, and default to allow all regex * removed hashin and modify logic to simplify * Improve race test and fix PR feedback by removing hash equals and avoid testing the timer.Ticker logic, and instead unit test * Ran make go-mod-tidy * Use errtypes in the test * Add changelog * add safety check for exporter endpoint * remove require.Contains by using error types, fix structure logging, and fix success metric typo in exporter * Fixed race test to have changing config values * Send success metric before modifying config * Avoid the defer and move the success metric under * NET-4240 - Snapshots are failing on Windows (#18302) * fix go mod * fix go sum * added change log * ran make go mod tidy * Register ProxyStateTemplate Resource (#18316) Also, change the ProxyState.id to identity. This is because we already have the id of this proxy from the resource, and this id should be name-aligned with the workload it represents. It should also have the owner ref set to the workload ID if we need that. And so the id field seems unnecessary. We do, however, need a reference to workload identity so that we can authorize the proxy when it initially connects to the xDS server. * [NET-5146] security: Update Go version to 1.20.7 and `x/net` to 0.13.0 (#18358) * Update Go version to 1.20.7 This resolves [CVE-2023-29409] (https://nvd.nist.gov/vuln/detail/CVE-2023-29409)(`crypto/tls`). * Bump golang.org/x/net to 0.13.0 Addresses [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978) for security scans (non-impacting). * mesh: adding the protobuf types and resources backing mesh config v2 (#18351) * [OSS] test: xds coverage for routes (#18369) test: xds coverage for routes * Fix policy lookup to allow for slashes (#18347) * Fix policy lookup to allow for slashes * Fix suggestions * Fix other test * Revert some lines * add some initial CODEOWNERS (#18346) * Add redirects for mesh-gateway docs (#18377) * expose grpc as http endpoint (#18221) expose resource grpc endpoints as http endpoints * bimapper: fix a bug and add some more test coverage (#18387) * [docs] Fix ServiceDefaults example in distributed tracing (#17212) Fix ServiceDefaults example in distributed tracing. * Fix #17730 - Dev mode has new line (#18367) * adding new line only in case of pretty in url not in dev mode * change log added * chore: make go-mod-tidy (#18388) * resource: Make resource read tenancy aware (#18397) * Not using chmod - fixed integration test for Enterprise (#18401) * Not using chmod * checking icacls * Removed push * replaced ordered list of rate limit ops with flow diagram (#18398) * replaced ordered list of rate limit ops with flow diagram * made diagram font bigger * updated colors for dark mode img * fixed typo * recentered dark mode image * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Upgrade test: remove outdated test and disable log due to verbosity (#18403) * remove outdated test * disable log since we have too many parallel tests * bump testcontainers-go from 0.22.0 and remove pinned go version in in… (#18395) * bump testcontainers-go from 0.22.0 and remove pinned go version in integ test * go mod tidy * Replace deprecated target.Authority with target.URL.Host * Add ServiceEndpoints Mutation hook tests (#18404) * Add ServiceEndpoints Mutation hook tests * Move endpoint owner validation into the validation hook Also there were some minor changes to error validation to account for go-cmp not liking to peer through an errors.errorstring type that get created by errors.New * catalog: add FailoverPolicy mutation and validation hooks (#18390) Add most of the validation and mutation hooks for the FailoverPolicy resource. * NET-5115 Add retry + timeout filters for api-gateway (#18324) * squash, implement retry/timeout in consul core * update tests * catalog: adding a controller to reconcile FailoverPolicy resources (#18399) Add most of the semantic cross-resource validation for FailoverPolicy resources using a new controller. * fix the error in ent repo (#18421) fix the error in ent repo * build: updates for 1.16.1 release (#18415) * build: updates for 1.16.1 release * build: fix missing replace directive for envoyextensions * build: go mod tidy testing/deployer * update ECS links (#18419) * docs: fix incorrect proxy-defaults config in Lua Envoy extension (#18418) fix incorrect proxy-defaults config * resource: Make resource write tenancy aware (#18423) * [NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437) * Add logging to locality policy application In OSS, this is currently a no-op. * Inherit locality when registering sidecars When sidecar locality is not explicitly configured, inherit locality from the proxied service. * NET-4984: Update APIGW Config Entries for JWT Auth (#18366) * Added oss config entries for Policy and JWT on APIGW * Updated structs for config entry * Updated comments, ran deep-copy * Move JWT configuration into OSS file * Add in the config entry OSS file for jwts * Added changelog * fixing proto spacing * Moved to using manually written deep copy method * Use pointers for override/default fields in apigw config entries * Run gen scripts for changed types * k8s compat - Openshift versions (#18307) * Update compatibility.mdx Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * APIGW: Update HTTPRouteConfigEntry for JWT Auth (#18422) * Updated httproute config entry for JWT Filters * Added manual deepcopy method for httproute jwt filter * Fix test * Update JWTFilter to be in oss file * Add changelog * Add build tags for deepcopy oss file * [COMPLIANCE] License changes (#18443) * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * add missing license headers * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * Net-2708/delete resource endpoint (#18420) * feat: add http delete endpoint for resource service * refactor: clean up * Disable deep-copy codegen verification for now. (#18446) Once we figure out what to do with license headers for these files we should re-enable this check. * build: upgrade to latest buf v1.26.0 (#18426) * NET-4952 Add docs for export command (#18425) * Init file for services export command docs * Add link for export command to nav * Add export to list of subcommands for services * Add usage and basic command options * Add API options using includes * Add section for enterprise options * Include argument value in flag format * Add examples * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Net-2712/resource hcl parsing (#18250) * Initial protohcl implementation Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Co-authored-by: Daniel Upton <daniel@floppy.co> * resourcehcl: implement resource decoding on top of protohcl Co-authored-by: Daniel Upton <daniel@floppy.co> * fix: resolve ci failures * test: add additional unmarshalling tests * refactor: update function test to clean protohcl package imports --------- Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Co-authored-by: Daniel Upton <daniel@floppy.co> * read endpoint (#18268) implement http read endpoint to expose resource grpc service read method * Net-2707/list resource endpoint (#18444) feat: list resources endpoint * Fix incorrect yaml in examples (#18463) * NET-5187: Upgrade test timeout due to log producer errors (#18461) Stop log producer before restarting container * [NET-4799] [OSS] xdsv2: listeners L4 support for connect proxies (#18436) * refactor to avoid future import cycles * resource: Make resource list tenancy aware (#18475) * [COMPLIANCE] License update (#18479) Update BUSL LICENSE to include licensed product and version. Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * docs: specify manual server metadata intervention (#18477) * docs: Update OpenShift compatibility (#18478) Update compatibility.mdx * resource: Make resource delete tenancy aware (#18476) resource: Make resource delete tenancy awarae * Add license-checker action that fails when any backported file contains BUSL header (#18485) * Add license-checker action that fails when any backported file contains BUSL header * Quote echoed variable to retain line breaks * Add ticket to reference for more details * docs: Update K8s TGW tutorial to reliably obtain role ID (#18474) The `grep` command used to obtain the ID for the terminating gateway role is not reliable in all scenarios. For example, if there is a similarly named role, the command may return the wrong role ID for the active terminating gateway instance. This commit updates the command to use jq to obtain the role ID. If multiple roles are found, jq will raise an error informing the user that it cannot reliably determine the role ID. * NET-5371 License checker pt2 (#18491) * Update grep command to work on ubuntu * Run license checker when new commits are pushed to PR * README - Update KV use case to Dynamic App Configuration (#18301) * Update README.md * Update kv.mdx * Add BUSL badge * README - re-order badges and update hub link (#18498) Update README.md * [NET-5163] Support locality testing in consul-container (#18484) * Support locality testing in consul-container Support including locality in client sidecar config. Also align test config structs with Ent to avoid future conflicts. * Refactor consul-container fortio helpers Refactor fortio test helpers to separate HTTP retries from waiting on fortio result changes due to e.g. service startup and failovers. * NET-4853 - xds v2 - implement base connect proxy functionality for clusters (#18499) * Fix HCL (#18513) * Fix HCL * Update create-sameness-groups.mdx * NET-4932 - xds v2 - implement base connect proxy functionality for endpoints (#18500) * NET-4853 - xds v2 - implement base connect proxy functionality for clusters * NET-4853 - xds v2 - implement base connect proxy functionality for clusters * NET-4932 - xds v2 - implement base connect proxy functionality for endpoints * Update endpoints_test.go * gofmt * Update naming.go * Support custom watches on controller (#18439) * Support custom watches on controller * refactor mapper methods * NET-4858 - xds v2 - implement base connect proxy functionality for routes (#18501) * NET-4853 - xds v2 - implement base connect proxy functionality for clusters * NET-4853 - xds v2 - implement base connect proxy functionality for clusters * NET-4932 - xds v2 - implement base connect proxy functionality for endpoints * Update endpoints_test.go * gofmt * NET-4858 - Make connect proxy route tests pass using xds v2 * Update endpoints_test.go * Update naming.go * use alsoRunTestForV2 * remove unused makeAddress * gofmt * fixing clusters * bimapper: allow to untrack links and support reference or id (#18451) * bimapper: fix data race (#18519) * CI Split integration tests to run nightly and every PR (#18518) * CI Split integration tests to run nightly and every PR * Checkout release branch for nightly test * Fix broken links caught in weekly report (#18522) * fix broken link caught in weekly report * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Create nightly test-integration for consul release branch (#18530) * Create nightly test-integration for consul release branch * test * fix * docs: Fix spelling errors across various pages on the site (#18533) This commit fixes numerous spelling errors across the site and also removes unnecessary whitespace that was present in the edited files. * Make proto-public license MPL (#18531) * Make proto-public license MPL * Add proto-public dir to exclusion list in .copywrite.hcl * NET-4943 - Implement ProxyTracker (#18535) * resource: Make resource watchlist tenancy aware (#18539) * Reduce required type arguments for DecodedResource (#18540) * OSS -> CE (community edition) changes (#18517) * CI: fix envoy versions in CI of release branch (#18538) * CI: fix envoy versions in CI of release branch * - remove steps sds for 1.15 nightly run - checkout the release branch * add default name * fix * Update go-tests.yml * fix checkout code * add comments and revert schedule * catalog: add failover mode enum validation (#18545) * mesh: add validation for the new pbmesh resources (#18410) Adds validation for HTTPRoute, GRPCRoute, TCPRoute, DestinationPolicy, and ComputedRoutes. * xds controller: resolve ServiceEndpoints references in ProxyStateTemp… (#18544) xds controller: resolve ServiceEndpoints references in ProxyStateTemplate * mesh: adding type aliases for mesh resource usage (#18448) Introduces some simple type aliases for DecodedResource[*X] wrappers for each type which cut down on the verbosity * Update trust bundle into proxy-state-template (#18550) * catalog: validating Protocol and Health enums on Service, Workload, and ServiceEndpoints (#18554) * chore: fix missing/incorrect license headers (#18555) * fix for , non presence of consul-version meta (#18464) * fix for #18406 , non presence of consul-version meta * removed redundant checks * updated mock-api to mimic api response for synthetic nodes * added test to test getDistinctConsulVersions method with synthetic-node case * updated typo in comments * added change log * feat: add experiments flag to testserver sdk (#18541) * bug: prevent go routine leakage due to existing DeferCheck (#18558) * bug: prevent go routine leakage due to existing DeferCheck * add changelog * NET-5382 & PLAT-1159: Do not trigger workflow if only doc files are in commit history (#18528) NET-5382: Do not run workflow runs if at least one path matches * UI: community verbiage (#18560) * chore: update community verbiage * chore: add changelog entry * Fix Windows FIPS Build (#18357) * Fix Windows FIPS Build * create new file for windows arm * removed build tag * fix buidl tags * fix multiple go build * CE port of enterprise extension (#18572) CE commit * resource: Make resource listbyowner tenancy aware (#18566) * NET-5147: Added placeholder structs for JWT functionality (#18575) * Added placeholder structs for JWT functionality * Added watches for CE vs ENT * Add license header * Undo plumbing work * Add context arg * resource: Make resource writestatus tenancy aware (#18577) * NET-5338 - NET-5338 - Run a v2 mode xds server (#18579) * NET-5338 - NET-5338 - Run a v2 mode xds server * fix linting * CI: send slack notification on failed nightly job (#18578) * CE commit (#18583) * Fix broken link on sameness group page (#18511) fix broken link * Fix typo in permissive mTLS docs (#18551) Update onboarding-tproxy-mode.mdx * Set concurrency for workflows (#18567) * Set concurrency for workflows * test: run automated tests against Vault 1.11 - 1.14 (#18590) Begin to test against Vault 1.14. Drop tests against Vault 1.10. * Reduce the frequency of metric exports to minutely (#18584) * NET-4944 - wire up controllers with proxy tracker (#18603) Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com> * Watch the ProxyTracker from xDS controller (#18611) * Distinguish v2 catalog Protocol unset from default (#18612) Even though we intend to default to TCP when this field is not explicitly provided, uncluding an `UNSPECIFIED` default enum value allows us to create inheritance chains, e.g. service to workload. * Added OpenTelemetry Access Logging Envoy extension (#18336) * [HCP Telemetry] Move first TelemetryConfig Fetch into the TelemetryConfigProvider (#18318) * Add Enabler interface to turn sink on/off * Use h for hcpProviderImpl vars, fix PR feeback and fix errors * Keep nil check in exporter and fix tests * Clarify comment and fix function name * Use disable instead of enable * Fix errors nit in otlp_transform * Add test for refreshInterval of updateConfig * Add disabled field in MetricsConfig struct * Fix PR feedback: improve comment and remove double colons * Fix deps test which requires a maybe * Update hcp-sdk-go to v0.61.0 * use disabled flag in telemetry_config.go * Handle 4XX errors in telemetry_provider * Fix deps test * Check 4XX instead * Run make go-mod-tidy * update comments and docs about running envoy integration tests with the ENVOY_VERSION set. (#18614) update ENVOY_VERSION and documentation of it used in the bats envoy tests. Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com> * delete all v2 resources type when deleting a namespace (CE) (#18621) * add namespace scope to ServiceV1Alpha1Type * add CE portion of namespace deletion * resource: Allow nil tenancy (#18618) * PLAT-1192 - Run CI on smaller instances (#18624) Use smaller runners * NET-3181 - Allow log file naming like Nomad (#18617) * fixes file name for consul * added log file * added tests for rename method * Add the plumbing for APIGW JWT work (#18609) * Add the plumbing for APIGW JWT work * Remove unneeded import * Add deep equal function for HTTPMatch * Added plumbing for status conditions * Remove unneeded comment * Fix comments * Add calls in xds listener for apigateway to setup listener jwt auth * docs: admin partition and DNS clarification (#18613) * fix: make UNSPECIFIED protocol pass validation (#18634) We explicitly enumerate the allowed protocols in validation, so this change is necessary to use the new enum value. Also add tests for enum validators to ensure they stay aligned to protos unless we explicitly want them to diverge. * NET 1594 - Snapshot Agent Filename Should Include Consul Version / Datacenter (#18625) * init * tests added and few fixes * revert arg message * changelog added * removed var declaration * fix CI * fix test * added node name and status * updated save.mdx * added example * fix tense * fix description * Net 3181 consul gh issue 15709 allow log file naming like nomad - fix bug (#18631) * fixes file name for consul * added log file * added tests for rename method * append instead of trunc * fix file truncate issue * added changelog * fix for build destros ci * removed changelog * solaris * resource: Require scope for resource registration (#18635) * [NET-4958] Fix issue where envoy endpoints would fail to populate after snapshot restore (#18636) Fix issue where agentless endpoints would fail to populate after snapshot restore. Fixes an issue that was introduced in #17775. This issue happens because a long-lived pointer to the state store is held, which is unsafe to do. Snapshot restorations will swap out this state store, meaning that the proxycfg watches would break for agentless. * Add extra logging for mesh health endpoints. (#18647) * feat: Change global-read-only policy to non editable (#18602) * remove stray FIPS file (#18651) * Add known issue notice for #18636. (#18650) * fix: remove Intention test with latency to make consul-enterprise rep… (#18654) fix: remove Intention test with latency to make consul-enterprise repo tests work * fix: add telemetry bind socket to proxyConfiguration proto (#18652) * feat: copy edits for built-in policy alert (#18655) * test: fix snapshot save w/ append test for ENT (#18656) * Net 2714/xw cli read command (#18462) enable `consul resource read` command in cli * change log for audit log (#18668) * change log for audit log * fix file name * breaking change * Add TCP+TLS Healthchecks (#18381) * Begin adding TCPUseTLS * More TCP with TLS plumbing * Making forward progress * Keep on adding TCP+TLS support for healthchecks * Removed too many lines * Unit tests for TCP+TLS * Update tlsutil/config.go Co-authored-by: Samantha <hello@entropy.cat> * Working on the tcp+tls unit test * Updated the runtime integration tests * Progress * Revert this file back to HEAD * Remove debugging lines * Implement TLS enabled TCP socket server and make a successful TCP+TLS healthcheck on it * Update docs * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update agent/checks/check.go Co-authored-by: Samantha <hello@entropy.cat> * Address comments * Remove extraneous bracket * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update the mockTLSServer * Remove trailing newline * Address comments * Fix merge problem * Add changelog entry --------- Co-authored-by: Samantha <hello@entropy.cat> * Fix windows Ent runner (#18683) * fix windows image for enterprise * added quotesT * Removed nodename and status from consul snapshot save -append-filename command and using leader version in version (#18680) * init * fix tests * fix tests lint * fix api call inside dc * updated doc * address comments * update guide to reflect tenancy and scope (#18687) * update guide to reflect tenancy and scope * Apply suggestions from code review Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * update ACLHooks signature * Update docs/resources/guide.md Co-authored-by: Semir Patel <semir.patel@hashicorp.com> --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * Add support for querying tokens by service name. (#18667) Add support for querying tokens by service name The consul-k8s endpoints controller has a workflow where it fetches all tokens. This is not performant for large clusters, where there may be a sizable number of tokens. This commit attempts to alleviate that problem and introduces a new way to query by the token's service name. * restore common-topo peering integration tests, but nightly [NET-5476] (#18673) * dataplane: Allow getting bootstrap parameters when using V2 APIs (#18504) This PR enables the GetEnvoyBootstrapParams endpoint to construct envoy bootstrap parameters from v2 catalog and mesh resources. * Make bootstrap request and response parameters less specific to services so that we can re-use them for workloads or service instances. * Remove ServiceKind from bootstrap params response. This value was unused previously and is not needed for V2. * Make access logs generation generic so that we can generate them using v1 or v2 resources. * switch all client nodes in dc2 to dataplane [NET-4299] (#18608) * Reformat proxy docs refresh (#18623) * first commit; reformat PD conf entry * updated proxies overview page * added Deploy SM proxy usage and removed reg index * moved sidecar proxy usage to main proxy folder * recast sidecar reg page as Deploy sidecar services * fix typos * recast SM reg as conf reference- set the sidebar * add redirects * fix links * add PD conf entry usage to appropro pages * edits to proxy conf ref * fix links on index page * example command to write PD conf entry * updated links to old SM proxy reg page * updated links to sidecar service reg page * tryna fix front matter issues * Apply suggestions from code review Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com> * added paragraph about SM proxies to overivew * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * sidecar-proxy controller: L4 controller with explicit upstreams (NET-3988) (#18352) * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * This commit also changes service endpoints to include workload identity. This made the implementation a bit easier as we don't need to look up as many workloads and instead rely on endpoints data. * test: update sidecarproxy/builder golden tests to use determinstic golden data (#18703) * [APIGW] NET-5017 JWT Cleanup/Status Conditions (#18700) * Fixes issues in setting status * Update golden files for changes to xds generation to not use deprecated methods * Fixed default for validation of JWT for route * Added changelog for jwt features (#18709) * fix: NET-1521 show latest config in /v1/agent/self (#18681) * fix: NET-1521 show latest config in /v1/agent/self * Using larger machine for Enterprise (#18713) using 4x large for ent * [NET-5325] ACL templated policies support in tokens and roles (#18708) * [NET-5325] ACL templated policies support in tokens and roles - Add API support for creating tokens/roles with templated-policies - Add CLI support for creating tokens/roles with templated-policies * adding changelog * Adds PassiveHealthCheck Fields to ServiceDefaults and IngressGateway (#18532) Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * Instantiate secure resource service client after the grpc server (#18712) * ENT port: test-integ/peering: peer through mesh gateway [NET-4609] (#18605) Co-authored-by: Matt Keeler <mjkeeler7@gmail.com> * [CC-6039] Update builtin policy descriptions (#18705) * Net-2713/resource apply command (#18610) * feat: implement apply command * fix: resolve lint issues * refactor: use fixed types * fix: ci lint failures after main rebase * NET-5530 Support response header modifiers on http-route config entry (#18646) * Add response header filters to http-route config entry definitions * Map response header filters from config entry when constructing route destination * Support response header modifiers at the service level as well * Update protobuf definitions * Update existing unit tests * Add response filters to route consolidation logic * Make existing unit tests more robust * Add missing docstring * Add changelog entry * Add response filter modifiers to existing integration test * Add more robust testing for response header modifiers in the discovery chain * Add more robust testing for request header modifiers in the discovery chain * Modify test to verify that service filter modifiers take precedence over rule filter modifiers * fix: emit consul version metric on a regular interval (#18724) * [NET-5330] Support templated policies in Binding rules (#18719) * [NET-5330] Support templated policies in Binding rules * changelog for templated policy support in binding rules * NET-5530 Generate deep-copy code (#18730) * Generate deep-copy code * Undo license header removal * sidecar-proxy controller: Add support for transparent proxy (NET-5069) (#18458) This commit adds support for transparent proxy to the sidecar proxy controller. As we do not yet support inferring destinations from intentions, this assumes that all services in the cluster are destinations. * Prefix sidecar proxy test files with source and destination. (#18620) * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Address PR comments * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * PR review comments * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Make sure endpoint refs route to mesh port instead of an app port * Address PR comments * fixing copyright * tidy imports * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * tidy imports * add copyright headers * Prefix sidecar proxy test files with source and destination. * Update controller_test.go --------- Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com> * k8s: Update audit-logging docs to include K8s examples (#18721) Update audit-logging.mdx * Fix windows integrations tests machine size (#18736) fix windows integrations tests machine size * Increase timeout for integration test windows. (#18737) increase timeout * Net-4292/list command (#18649) * feat: implement list command * refactor: apply command file parsing * Added logs for debugging (#18740) * added logs * added echo * removed pull request * xds: update golden tests to be deterministic (#18707) * Run copyright after running deep-copy as part of the Makefile/CI (#18741) * execute copyright headers after performing deep-copy generation. * fix copyright install * Apply suggestions from code review Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * Apply suggestions from code review Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * rename steps to match codegen naming * remove copywrite install category --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * [NET-5333] Add api to read/list and preview templated policies (#18748) * NET-5132 - Configure multiport routing for connect proxies in TProxy mode (#18606) * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Address PR comments * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * PR review comments * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Make sure endpoint refs route to mesh port instead of an app port * Address PR comments * fixing copyright * tidy imports * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * tidy imports * add copyright headers * Prefix sidecar proxy test files with source and destination. * Update controller_test.go * NET-5132 - Configure multiport routing for connect proxies in TProxy mode * formatting golden files * reverting golden files and adding changes in manually. build implicit destinations still has some issues. * fixing files that were incorrectly repeating the outbound listener * PR comments * extract AlpnProtocol naming convention to getAlpnProtocolFromPortName(portName) * removing address level filtering. * adding license to resources_test.go --------- Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com> * added time stamps (#18757) * fix: write endpoint errors out gracefully (#18743) * [NET-5574] Update Go version to 1.20.8 (#18742) Update Go version to 1.20.8 This resolves several CVEs (see changelog entry). * Add Envoy golden test for OTEL access logging extension (#18760) * mesh: create new routes-controller to reconcile xRoute types into a ComputedRoutes resource (#18460) This new controller produces an intermediate output (ComputedRoutes) that is meant to summarize all relevant xRoutes and related mesh configuration in an easier-to-use format for downstream use to construct the ProxyStateTemplate. It also applies status updates to the xRoute resource types to indicate that they are themselves semantically valid inputs. * xds controller: setup watches for and compute leaf cert references in ProxyStateTemplate, and wire up leaf cert manager dependency (#18756) * Refactors the leafcert package to not have a dependency on agent/consul and agent/cache to avoid import cycles. This way the xds controller can just import the leafcert package to use the leafcert manager. The leaf cert logic in the controller: * Sets up watches for leaf certs that are referenced in the ProxyStateTemplate (which generates the leaf certs too). * Gets the leaf cert from the leaf cert cache * Stores the leaf cert in the ProxyState that's pushed to xds * For the cert watches, this PR also uses a bimapper + a thin wrapper to map leaf cert events to related ProxyStateTemplates Since bimapper uses a resource.Reference or resource.ID to map between two resource types, I've created an internal type for a leaf certificate to use for the resource.Reference, since it's not a v2 resource. The wrapper allows mapping events to resources (as opposed to mapping resources to resources) The controller tests: Unit: Ensure that we resolve leaf cert references Lifecycle: Ensure that when the CA is updated, the leaf cert is as well Also adds a new spiffe id type, and adds workload identity and workload identity URI to leaf certs. This is so certs are generated with the new workload identity based SPIFFE id. * Pulls out some leaf cert test helpers into a helpers file so it can be used in the xds controller tests. * Wires up leaf cert manager dependency * Support getting token from proxytracker * Add workload identity spiffe id type to the authorize and sign functions --------- Co-authored-by: John Murret <john.murret@hashicorp.com> * Test: NET-5592 - update Nomad integration testing (#18768) NET-5592 - update Nomad integration testing * Add workload identity ACL rules (#18769) * Windows Integration Test Changes (#18758) * some changes to debug * revert machines * increased timeout * added sleep 10 seconds before test start * chagne envoy version * removed sleep * revert timeout * replace position * removed date * Revert "[NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437)" This reverts commit 05604eeec1186a4c8a6156e7519a579171659c1a. * fix build * Revert "replace position" This reverts commit 48e6af46a8daae186c283f30d316b1104906993e. * Revert "Revert "[NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437)"" This reverts commit d7c568e2be727b72e6827225782e0e17ac06b74a. * comment out api gateway http hostnames test * fix import * revert integ test run on PR * Add V2 TCP traffic permissions (#18771) Add support for TCP traffic permissions * resource: add helper to normalize inner Reference tenancy during mutate (#18765) When one resource contains an inner field that is of type *pbresource.Reference we want the Tenancy to be reasonably defaulted by the following rules: 1. The final values will be limited by the scope of the referenced type. 2. Values will be inferred from the parent's tenancy, and if that is insufficient then using the default tenancy for the type's scope. 3. Namespace will only be used from a parent if the reference and the parent share a partition, otherwise the default namespace will be used. Until we tackle peering, this hard codes an assumption of peer name being local. The logic for defaulting may need adjustment when that is addressed. * Disable docs change CI skip detection (#18775) There's currently a bug that causes CI to be skipped on all non-PR changes. Until that's fixed and we can be certain the check will fail CI or default to running tests in the case of errors, disabling this check. * delete command ready (#18679) * delete command for resource management * Vault CA provider clean up previous default issuers (#18773) * Fix snapshot creation issue. (#18783) The renaming of files from oss -> ce caused incorrect snapshots to be created due to ce writes now happening prior to ent writes. When this happens various entities will attempt to be restored from the snapshot prior to a partition existing and will cause a panic to occur. * catalog: service endpoints inherits protocol from service when workload doesn't have one (#18792) * add fuzz tests to resourcehcl package and fix some panics (#18798) * add a simple fuzz test for the resourcehcl package and fix some panics uncovered by the test * fix default for null values * Clean up resources in test (#18799) * Updates gateway-class-config docs to include fields for privileged container port-mapping and Openshift Security Context Constraints Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * Removes errant word in docs that was breaking the build (#18804) * Fix flaky test (#18805) * register traffic permission and workload identity types (#18704) * add workload identity and traffic permission protos * register new types * add generated pb code * fix exports.go path * add proto newlines * fix type name Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com> * address review * fix protos and add tests * fix validation constraints * add tests --------- Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com> * [NET-5329] use acl templated policy under the hood for node/service identities (#18813) * [NET-5334] Added CLI commands for templated policies (#18816) * mesh: Wire ComputedRoutes into the ProxyStateTemplate via the sidecar controller (#18752) Reworks the sidecar controller to accept ComputedRoutes as an input and use it to generate appropriate ProxyStateTemplate resources containing L4/L7 mesh configuration. * mesh: call the right mapper (#18818) * fix templated policy cli output (#18821) * resource: default peername to "local" for now (#18822) * Wire up traffic permissions (#18812) Wire up traffic permissions * added consul and envoy version constraints (#18726) * added consul and envoy version constraints * fixed Destination configuraiton and added tproxy requirement * Apply suggestions from code review Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> --------- Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> * Default to tcp protocol when workload protocol is unspecified (#18824) * Retry and timeout test acceptance test (#18791) * retry and timeout test * add docker mirrior * checkpoint * add in error * add in delay * up error rate * fix status code * Use embedded strings for templated policies (#18829) * consul operator raft transfer-leader should send the id (#17107) Fixes #16955 Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> * Allow empty data writes for resources (#18819) * allow nil data writes for resources * update demo to test valid type with no data * Fix nondeterministic test (#18828) * mesh: prevent writing a ComputedRoutes with no ported configs (#18833) * catalog: Default protocol to tcp in catalog.Service if unspecified (#18832) * Adjust metrics test (#18837) * Audit log consul 1.17x changes (#18669) * audit log consul 1.17x changes * added some details * verbose * Fixes for integration tests windows for ENT (#18839) * fixes for integration tests * fix runner size for enterprise * fix spacing * fix spacing * removed branch test run * fix: provide meaningful error messages and add test (#18772) * fix: provide meaningful error messages and add test * fix: return error instead of warning when extra args are provided * add v2 tenancy bridge Flag and v2 Tenancy Bridge initial implementation (#18830) * add v2 tenancy bridge and a feature flag for v2 tenancy * move tenancy bridge v2 under resource package * docs: Fix typo in description for server_addresses (#18838) Change 'If not port' to 'If no port'. Resolves #18553 * grafana: display connected consul-dataplanes (#18842) * Adding Apigee for ext_authz, minor fix in the default ext_authz docs (#18796) * adding apigee for ext_authz, minor fix * adding the Apigee docs to nav * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Blake Covarrubias <blake.covarrubias@gmail.com> * addressing feedback * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Blake Covarrubias <blake.covarrubias@gmail.com> * catalog: normalize/default/validate tenancy components of FailoverPolicy internal References (#18825) FailoverPolicy resources contain inner Reference fields. We want to ensure that components of those reference Tenancy fields left unspecified are defaulted using the tenancy of the enclosing FailoverPolicy resource. As the underlying helper being used to do the normalization calls the function modified in #18822, it also means that the PeerName field will be set to "local" for now automatically to avoid "local" != "" issues downstream. * Fix gateway services cleanup where proxy deregistration happens after service deregistration (#18831) * Fix gateway services cleanup where proxy deregistration happens after service deregistration * Add test * Add changelog * Fix comment * Move ACL templated policies to hcl files (#18853) * mesh: update xds controller to synthesize empty endpoints when no endpoints ref is found (#18835) * mesh: normalize/default/validate tenancy components of mesh internal References (#18827) HTTPRoute, GRPCRoute, TCPRoute, and Upstreams resources contain inner Reference fields. We want to ensure that components of those reference Tenancy fields left unspecified are defaulted using the tenancy of the enclosing resource. As the underlying helper being used to do the normalization calls the function modified in #18822, it also means that the PeerName field will be set to "local" for now automatically to avoid "local" != "" issues downstream. * resource: mutate and validate before acls on write (#18868) * Remove flaky test assertions (#18870) * mesh: only build tproxy outbound listener once per destination (#18836) Previously, when using implicit upstreams, we'd build outbound listener per destination instead of one for all destinations. This will result in port conflicts when trying to send this config to envoy. This PR also makes sure that leaf and root references are always added (before we would only add it if there are inbound non-mesh ports). Also, black-hole traffic when there are no inbound ports other than mesh * NET-4519 Collecting journald logs in "consul debug" bundle (#18797) * debug since * fix docs * chagelog added * fix go mod * debug test fix * fix test * tabs test fix * Update .changelog/18797.txt Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com> --------- Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com> * CI: lint test-integ (#18875) * CI: lint test-integ * fix lint error * Honor Default Traffic Permissions in V2 (#18886) wire up v2 default traffic permissions * docs: Fix HCL, JSON, and YAML syntax errors (#18879) This commit fixes syntax errors in HCL, JSON, and YAML example configurations. In some cases, it replaces the code example with the proper format for the code block. Also fixes HCL formatting and misc opportunistic updates to codeblock. Co-authored-by: Tu Nguyen <im2nguyen@gmail.com> * Fix code block examples on dns-static-lookups.mdx (#18880) HCL and JSON configuration examples were being displayed in the same code block. This commit separates the configurations to properly display them as independent configuration examples. * docs: Fix Kubernetes CRD example configs (#18878) Fixes configuration examples for several Consul Kubernetes CRDs. The CRDs were missing required fields such as `apiVersion`, `metadata`, and `spec`. Co-authored-by: Tu Nguyen <im2nguyen@gmail.com> * docs: Apigee extension backport (#18847) * commit * link text edits * Add operator audit endpoint changes (#18899) * docs: Remove YAML service registration examples (#18877) Remove YAML service registration examples and replace them with JSON. This is because YAML is not a supported configuration format for the Consul's agent configuration, nor is it supported by the HTTP API. This commit replaces the YAML examples with JSON and adds additional JSON examples where they were missing. * Add otel-access-logging Envoy extension integration test (#18898) * Add note about service upstream env var dot broken (#18895) * add note about service upstream env var dot broken * Apply suggestions from code review Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --------- Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * xds: Use downstream protocol when connecting to local app (#18573) Configure Envoy to use the same HTTP protocol version used by the downstream caller when forwarding requests to a local application that is configured with the protocol set to either `http2` or `grpc`. This allows upstream applications that support both HTTP/1.1 and HTTP/2 on a single port to receive requests using either protocol. This is beneficial when the application primarily communicates using HTTP/2, but also needs to support HTTP/1.1, such as to respond to Kubernetes HTTP readiness/liveness probes. Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com> * resource: ensure resource.AuthorizerContext properly strips the local… (#18908) resource: ensure resource.AuthorizerContext properly strips the local peer name * docs: Add complete auth method payloads (#18849) This commit modifies the example payloads for various auth methods to remove 'other fields' and instead use complete example payloads. * agent/config: prevent startup if resource-apis experiment and cloud are enabled (#18876) * v2: various fixes to make K8s tproxy multiport acceptance tests and manual explicit upstreams (single port) tests pass (#18874) Adding coauthors who mobbed/paired at various points throughout last week. Co-authored-by: Dan Stough <dan.stough@hashicorp.com> Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: John Murret <john.murret@hashicorp.com> Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com> Co-authored-by: Michael Wilkerson <mwilkerson@hashicorp.com> * fix docs * Update website/content/docs/agent/config/cli-flags.mdx Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Update website/content/docs/agent/config/cli-flags.mdx Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * backport of commit 6f25963ed361617278d68467c0cfb0d2df0b9015 * backport of commit 05d5d24784302a7f6ec911884c5724fa336edfda * Fix docs for log file name changes (#18913) * fix docs * Update website/content/docs/agent/config/cli-flags.mdx Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Update website/content/docs/agent/config/cli-flags.mdx Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Update website/content/docs/agent/config/cli-flags.mdx Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Signed-off-by: cui fliter <imcusg@gmail.com> Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> Co-authored-by: Curt Bushko <cbushko@gmail.com> Co-authored-by: Florian Apolloner <florian@apolloner.eu> Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com> Co-authored-by: cui fliter <imcusg@gmail.com> Co-authored-by: Blake Covarrubias <blake@covarrubi.as> Co-authored-by: Jeremy Jacobson <jjacobson93@users.noreply.github.com> Co-authored-by: Paul Glass <pglass@hashicorp.com> Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com> Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com> Co-authored-by: John Landa <jonathanlanda@gmail.com> Co-authored-by: Ashvitha <ashvitha.sridharan@hashicorp.com> Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com> Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> Co-authored-by: Dan Stough <dan.stough@hashicorp.com> Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com> Co-authored-by: wangxinyi7 <121973291+wangxinyi7@users.noreply.github.com> Co-authored-by: Andrea Scarpino <andrea@scarpino.dev> Co-authored-by: Semir Patel <semir.patel@hashicorp.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: cskh <hui.kang@hashicorp.com> Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com> Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com> Co-authored-by: Gerard Nguyen <gerard@hashicorp.com> Co-authored-by: John Maguire <john.maguire@hashicorp.com> Co-authored-by: David Yu <dyu@hashicorp.com> Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Poonam Jadhav <poonam.jadhav@hashicorp.com> Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: Daniel Upton <daniel@floppy.co> Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com> Co-authored-by: Anita Akaeze <anita.akaeze@hashicorp.com> Co-authored-by: Dan Bond <danbond@protonmail.com> Co-authored-by: John Murret <john.murret@hashicorp.com> Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com> Co-authored-by: Vijay <vijayraghav22@gmail.com> Co-authored-by: Natalie Smith <nataliemegans@gmail.com> Co-authored-by: Chris S. Kim <ckim@hashicorp.com> Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> Co-authored-by: Joshua Timmons <josh.timmons@hashicorp.com> Co-authored-by: Hardik Shingala <34568645+hdkshingala@users.noreply.github.com> Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> Co-authored-by: Derek Menteer <105233703+hashi-derek@users.noreply.github.com> Co-authored-by: Valeriia Ruban <valeriia.ruban@hashicorp.com> Co-authored-by: skpratt <sarah.pratt@hashicorp.com> Co-authored-by: Phil Porada <pgporada@users.noreply.github.com> Co-authored-by: Samantha <hello@entropy.cat> Co-authored-by: Nick Irvine <115657443+nfi-hashicorp@users.noreply.github.com> Co-authored-by: Melisa Griffin <missylbytes@users.noreply.github.com> Co-authored-by: Matt Keeler <mjkeeler7@gmail.com> Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: Sophie Gairo <97480023+sophie-gairo@users.noreply.github.com> Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com> Co-authored-by: James Hartig <me@jameshartig.com> Co-authored-by: Gautam <gautambaghel93@gmail.com> Co-authored-by: Blake Covarrubias <blake.covarrubias@gmail.com> Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com> Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com> Co-authored-by: Tu Nguyen <im2nguyen@gmail.com> Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com> Co-authored-by: Nick Ethier <nethier@hashicorp.com> Co-authored-by: absolutelightning <ashesh.vidyut@hashicorp.com>

…on into release/1.15.x (#18989) backport of commit 312bb96 Co-authored-by: R.B. Boyer <rb@hashicorp.com>

…into release/1.15.x (#19012) * Fix type of datacenter in Service Resolvers Config Entry (#19004) * fix redirect datacenter doc --------- Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com> Co-authored-by: absolutelightning <ashesh.vidyut@hashicorp.com>

…olverConfigE… into release/1.15.x (#19033) * read endpoint (#18268) implement http read endpoint to expose resource grpc service read method * Net-2707/list resource endpoint (#18444) feat: list resources endpoint * Fix incorrect yaml in examples (#18463) * NET-5187: Upgrade test timeout due to log producer errors (#18461) Stop log producer before restarting container * [NET-4799] [OSS] xdsv2: listeners L4 support for connect proxies (#18436) * refactor to avoid future import cycles * resource: Make resource list tenancy aware (#18475) * [COMPLIANCE] License update (#18479) Update BUSL LICENSE to include licensed product and version. Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * docs: specify manual server metadata intervention (#18477) * docs: Update OpenShift compatibility (#18478) Update compatibility.mdx * resource: Make resource delete tenancy aware (#18476) resource: Make resource delete tenancy awarae * Add license-checker action that fails when any backported file contains BUSL header (#18485) * Add license-checker action that fails when any backported file contains BUSL header * Quote echoed variable to retain line breaks * Add ticket to reference for more details * docs: Update K8s TGW tutorial to reliably obtain role ID (#18474) The `grep` command used to obtain the ID for the terminating gateway role is not reliable in all scenarios. For example, if there is a similarly named role, the command may return the wrong role ID for the active terminating gateway instance. This commit updates the command to use jq to obtain the role ID. If multiple roles are found, jq will raise an error informing the user that it cannot reliably determine the role ID. * NET-5371 License checker pt2 (#18491) * Update grep command to work on ubuntu * Run license checker when new commits are pushed to PR * README - Update KV use case to Dynamic App Configuration (#18301) * Update README.md * Update kv.mdx * Add BUSL badge * README - re-order badges and update hub link (#18498) Update README.md * [NET-5163] Support locality testing in consul-container (#18484) * Support locality testing in consul-container Support including locality in client sidecar config. Also align test config structs with Ent to avoid future conflicts. * Refactor consul-container fortio helpers Refactor fortio test helpers to separate HTTP retries from waiting on fortio result changes due to e.g. service startup and failovers. * NET-4853 - xds v2 - implement base connect proxy functionality for clusters (#18499) * Fix HCL (#18513) * Fix HCL * Update create-sameness-groups.mdx * NET-4932 - xds v2 - implement base connect proxy functionality for endpoints (#18500) * NET-4853 - xds v2 - implement base connect proxy functionality for clusters * NET-4853 - xds v2 - implement base connect proxy functionality for clusters * NET-4932 - xds v2 - implement base connect proxy functionality for endpoints * Update endpoints_test.go * gofmt * Update naming.go * Support custom watches on controller (#18439) * Support custom watches on controller * refactor mapper methods * NET-4858 - xds v2 - implement base connect proxy functionality for routes (#18501) * NET-4853 - xds v2 - implement base connect proxy functionality for clusters * NET-4853 - xds v2 - implement base connect proxy functionality for clusters * NET-4932 - xds v2 - implement base connect proxy functionality for endpoints * Update endpoints_test.go * gofmt * NET-4858 - Make connect proxy route tests pass using xds v2 * Update endpoints_test.go * Update naming.go * use alsoRunTestForV2 * remove unused makeAddress * gofmt * fixing clusters * bimapper: allow to untrack links and support reference or id (#18451) * bimapper: fix data race (#18519) * CI Split integration tests to run nightly and every PR (#18518) * CI Split integration tests to run nightly and every PR * Checkout release branch for nightly test * Fix broken links caught in weekly report (#18522) * fix broken link caught in weekly report * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Create nightly test-integration for consul release branch (#18530) * Create nightly test-integration for consul release branch * test * fix * docs: Fix spelling errors across various pages on the site (#18533) This commit fixes numerous spelling errors across the site and also removes unnecessary whitespace that was present in the edited files. * Make proto-public license MPL (#18531) * Make proto-public license MPL * Add proto-public dir to exclusion list in .copywrite.hcl * NET-4943 - Implement ProxyTracker (#18535) * resource: Make resource watchlist tenancy aware (#18539) * Reduce required type arguments for DecodedResource (#18540) * OSS -> CE (community edition) changes (#18517) * CI: fix envoy versions in CI of release branch (#18538) * CI: fix envoy versions in CI of release branch * - remove steps sds for 1.15 nightly run - checkout the release branch * add default name * fix * Update go-tests.yml * fix checkout code * add comments and revert schedule * catalog: add failover mode enum validation (#18545) * mesh: add validation for the new pbmesh resources (#18410) Adds validation for HTTPRoute, GRPCRoute, TCPRoute, DestinationPolicy, and ComputedRoutes. * xds controller: resolve ServiceEndpoints references in ProxyStateTemp… (#18544) xds controller: resolve ServiceEndpoints references in ProxyStateTemplate * mesh: adding type aliases for mesh resource usage (#18448) Introduces some simple type aliases for DecodedResource[*X] wrappers for each type which cut down on the verbosity * Update trust bundle into proxy-state-template (#18550) * catalog: validating Protocol and Health enums on Service, Workload, and ServiceEndpoints (#18554) * chore: fix missing/incorrect license headers (#18555) * fix for , non presence of consul-version meta (#18464) * fix for #18406 , non presence of consul-version meta * removed redundant checks * updated mock-api to mimic api response for synthetic nodes * added test to test getDistinctConsulVersions method with synthetic-node case * updated typo in comments * added change log * feat: add experiments flag to testserver sdk (#18541) * bug: prevent go routine leakage due to existing DeferCheck (#18558) * bug: prevent go routine leakage due to existing DeferCheck * add changelog * NET-5382 & PLAT-1159: Do not trigger workflow if only doc files are in commit history (#18528) NET-5382: Do not run workflow runs if at least one path matches * UI: community verbiage (#18560) * chore: update community verbiage * chore: add changelog entry * Fix Windows FIPS Build (#18357) * Fix Windows FIPS Build * create new file for windows arm * removed build tag * fix buidl tags * fix multiple go build * CE port of enterprise extension (#18572) CE commit * resource: Make resource listbyowner tenancy aware (#18566) * NET-5147: Added placeholder structs for JWT functionality (#18575) * Added placeholder structs for JWT functionality * Added watches for CE vs ENT * Add license header * Undo plumbing work * Add context arg * resource: Make resource writestatus tenancy aware (#18577) * NET-5338 - NET-5338 - Run a v2 mode xds server (#18579) * NET-5338 - NET-5338 - Run a v2 mode xds server * fix linting * CI: send slack notification on failed nightly job (#18578) * CE commit (#18583) * Fix broken link on sameness group page (#18511) fix broken link * Fix typo in permissive mTLS docs (#18551) Update onboarding-tproxy-mode.mdx * Set concurrency for workflows (#18567) * Set concurrency for workflows * test: run automated tests against Vault 1.11 - 1.14 (#18590) Begin to test against Vault 1.14. Drop tests against Vault 1.10. * Reduce the frequency of metric exports to minutely (#18584) * NET-4944 - wire up controllers with proxy tracker (#18603) Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com> * Watch the ProxyTracker from xDS controller (#18611) * Distinguish v2 catalog Protocol unset from default (#18612) Even though we intend to default to TCP when this field is not explicitly provided, uncluding an `UNSPECIFIED` default enum value allows us to create inheritance chains, e.g. service to workload. * Added OpenTelemetry Access Logging Envoy extension (#18336) * [HCP Telemetry] Move first TelemetryConfig Fetch into the TelemetryConfigProvider (#18318) * Add Enabler interface to turn sink on/off * Use h for hcpProviderImpl vars, fix PR feeback and fix errors * Keep nil check in exporter and fix tests * Clarify comment and fix function name * Use disable instead of enable * Fix errors nit in otlp_transform * Add test for refreshInterval of updateConfig * Add disabled field in MetricsConfig struct * Fix PR feedback: improve comment and remove double colons * Fix deps test which requires a maybe * Update hcp-sdk-go to v0.61.0 * use disabled flag in telemetry_config.go * Handle 4XX errors in telemetry_provider * Fix deps test * Check 4XX instead * Run make go-mod-tidy * update comments and docs about running envoy integration tests with the ENVOY_VERSION set. (#18614) update ENVOY_VERSION and documentation of it used in the bats envoy tests. Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com> * delete all v2 resources type when deleting a namespace (CE) (#18621) * add namespace scope to ServiceV1Alpha1Type * add CE portion of namespace deletion * resource: Allow nil tenancy (#18618) * PLAT-1192 - Run CI on smaller instances (#18624) Use smaller runners * NET-3181 - Allow log file naming like Nomad (#18617) * fixes file name for consul * added log file * added tests for rename method * Add the plumbing for APIGW JWT work (#18609) * Add the plumbing for APIGW JWT work * Remove unneeded import * Add deep equal function for HTTPMatch * Added plumbing for status conditions * Remove unneeded comment * Fix comments * Add calls in xds listener for apigateway to setup listener jwt auth * docs: admin partition and DNS clarification (#18613) * fix: make UNSPECIFIED protocol pass validation (#18634) We explicitly enumerate the allowed protocols in validation, so this change is necessary to use the new enum value. Also add tests for enum validators to ensure they stay aligned to protos unless we explicitly want them to diverge. * NET 1594 - Snapshot Agent Filename Should Include Consul Version / Datacenter (#18625) * init * tests added and few fixes * revert arg message * changelog added * removed var declaration * fix CI * fix test * added node name and status * updated save.mdx * added example * fix tense * fix description * Net 3181 consul gh issue 15709 allow log file naming like nomad - fix bug (#18631) * fixes file name for consul * added log file * added tests for rename method * append instead of trunc * fix file truncate issue * added changelog * fix for build destros ci * removed changelog * solaris * resource: Require scope for resource registration (#18635) * [NET-4958] Fix issue where envoy endpoints would fail to populate after snapshot restore (#18636) Fix issue where agentless endpoints would fail to populate after snapshot restore. Fixes an issue that was introduced in #17775. This issue happens because a long-lived pointer to the state store is held, which is unsafe to do. Snapshot restorations will swap out this state store, meaning that the proxycfg watches would break for agentless. * Add extra logging for mesh health endpoints. (#18647) * feat: Change global-read-only policy to non editable (#18602) * remove stray FIPS file (#18651) * Add known issue notice for #18636. (#18650) * fix: remove Intention test with latency to make consul-enterprise rep… (#18654) fix: remove Intention test with latency to make consul-enterprise repo tests work * fix: add telemetry bind socket to proxyConfiguration proto (#18652) * feat: copy edits for built-in policy alert (#18655) * test: fix snapshot save w/ append test for ENT (#18656) * Net 2714/xw cli read command (#18462) enable `consul resource read` command in cli * change log for audit log (#18668) * change log for audit log * fix file name * breaking change * Add TCP+TLS Healthchecks (#18381) * Begin adding TCPUseTLS * More TCP with TLS plumbing * Making forward progress * Keep on adding TCP+TLS support for healthchecks * Removed too many lines * Unit tests for TCP+TLS * Update tlsutil/config.go Co-authored-by: Samantha <hello@entropy.cat> * Working on the tcp+tls unit test * Updated the runtime integration tests * Progress * Revert this file back to HEAD * Remove debugging lines * Implement TLS enabled TCP socket server and make a successful TCP+TLS healthcheck on it * Update docs * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update agent/checks/check.go Co-authored-by: Samantha <hello@entropy.cat> * Address comments * Remove extraneous bracket * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update the mockTLSServer * Remove trailing newline * Address comments * Fix merge problem * Add changelog entry --------- Co-authored-by: Samantha <hello@entropy.cat> * Fix windows Ent runner (#18683) * fix windows image for enterprise * added quotesT * Removed nodename and status from consul snapshot save -append-filename command and using leader version in version (#18680) * init * fix tests * fix tests lint * fix api call inside dc * updated doc * address comments * update guide to reflect tenancy and scope (#18687) * update guide to reflect tenancy and scope * Apply suggestions from code review Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * update ACLHooks signature * Update docs/resources/guide.md Co-authored-by: Semir Patel <semir.patel@hashicorp.com> --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * Add support for querying tokens by service name. (#18667) Add support for querying tokens by service name The consul-k8s endpoints controller has a workflow where it fetches all tokens. This is not performant for large clusters, where there may be a sizable number of tokens. This commit attempts to alleviate that problem and introduces a new way to query by the token's service name. * restore common-topo peering integration tests, but nightly [NET-5476] (#18673) * dataplane: Allow getting bootstrap parameters when using V2 APIs (#18504) This PR enables the GetEnvoyBootstrapParams endpoint to construct envoy bootstrap parameters from v2 catalog and mesh resources. * Make bootstrap request and response parameters less specific to services so that we can re-use them for workloads or service instances. * Remove ServiceKind from bootstrap params response. This value was unused previously and is not needed for V2. * Make access logs generation generic so that we can generate them using v1 or v2 resources. * switch all client nodes in dc2 to dataplane [NET-4299] (#18608) * Reformat proxy docs refresh (#18623) * first commit; reformat PD conf entry * updated proxies overview page * added Deploy SM proxy usage and removed reg index * moved sidecar proxy usage to main proxy folder * recast sidecar reg page as Deploy sidecar services * fix typos * recast SM reg as conf reference- set the sidebar * add redirects * fix links * add PD conf entry usage to appropro pages * edits to proxy conf ref * fix links on index page * example command to write PD conf entry * updated links to old SM proxy reg page * updated links to sidecar service reg page * tryna fix front matter issues * Apply suggestions from code review Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com> * added paragraph about SM proxies to overivew * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * sidecar-proxy controller: L4 controller with explicit upstreams (NET-3988) (#18352) * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * This commit also changes service endpoints to include workload identity. This made the implementation a bit easier as we don't need to look up as many workloads and instead rely on endpoints data. * test: update sidecarproxy/builder golden tests to use determinstic golden data (#18703) * [APIGW] NET-5017 JWT Cleanup/Status Conditions (#18700) * Fixes issues in setting status * Update golden files for changes to xds generation to not use deprecated methods * Fixed default for validation of JWT for route * Added changelog for jwt features (#18709) * fix: NET-1521 show latest config in /v1/agent/self (#18681) * fix: NET-1521 show latest config in /v1/agent/self * Using larger machine for Enterprise (#18713) using 4x large for ent * [NET-5325] ACL templated policies support in tokens and roles (#18708) * [NET-5325] ACL templated policies support in tokens and roles - Add API support for creating tokens/roles with templated-policies - Add CLI support for creating tokens/roles with templated-policies * adding changelog * Adds PassiveHealthCheck Fields to ServiceDefaults and IngressGateway (#18532) Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * Instantiate secure resource service client after the grpc server (#18712) * ENT port: test-integ/peering: peer through mesh gateway [NET-4609] (#18605) Co-authored-by: Matt Keeler <mjkeeler7@gmail.com> * [CC-6039] Update builtin policy descriptions (#18705) * Net-2713/resource apply command (#18610) * feat: implement apply command * fix: resolve lint issues * refactor: use fixed types * fix: ci lint failures after main rebase * NET-5530 Support response header modifiers on http-route config entry (#18646) * Add response header filters to http-route config entry definitions * Map response header filters from config entry when constructing route destination * Support response header modifiers at the service level as well * Update protobuf definitions * Update existing unit tests * Add response filters to route consolidation logic * Make existing unit tests more robust * Add missing docstring * Add changelog entry * Add response filter modifiers to existing integration test * Add more robust testing for response header modifiers in the discovery chain * Add more robust testing for request header modifiers in the discovery chain * Modify test to verify that service filter modifiers take precedence over rule filter modifiers * fix: emit consul version metric on a regular interval (#18724) * [NET-5330] Support templated policies in Binding rules (#18719) * [NET-5330] Support templated policies in Binding rules * changelog for templated policy support in binding rules * NET-5530 Generate deep-copy code (#18730) * Generate deep-copy code * Undo license header removal * sidecar-proxy controller: Add support for transparent proxy (NET-5069) (#18458) This commit adds support for transparent proxy to the sidecar proxy controller. As we do not yet support inferring destinations from intentions, this assumes that all services in the cluster are destinations. * Prefix sidecar proxy test files with source and destination. (#18620) * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Address PR comments * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * PR review comments * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Make sure endpoint refs route to mesh port instead of an app port * Address PR comments * fixing copyright * tidy imports * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * tidy imports * add copyright headers * Prefix sidecar proxy test files with source and destination. * Update controller_test.go --------- Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com> * k8s: Update audit-logging docs to include K8s examples (#18721) Update audit-logging.mdx * Fix windows integrations tests machine size (#18736) fix windows integrations tests machine size * Increase timeout for integration test windows. (#18737) increase timeout * Net-4292/list command (#18649) * feat: implement list command * refactor: apply command file parsing * Added logs for debugging (#18740) * added logs * added echo * removed pull request * xds: update golden tests to be deterministic (#18707) * Run copyright after running deep-copy as part of the Makefile/CI (#18741) * execute copyright headers after performing deep-copy generation. * fix copyright install * Apply suggestions from code review Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * Apply suggestions from code review Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * rename steps to match codegen naming * remove copywrite install category --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * [NET-5333] Add api to read/list and preview templated policies (#18748) * NET-5132 - Configure multiport routing for connect proxies in TProxy mode (#18606) * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Address PR comments * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * PR review comments * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Make sure endpoint refs route to mesh port instead of an app port * Address PR comments * fixing copyright * tidy imports * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * tidy imports * add copyright headers * Prefix sidecar proxy test files with source and destination. * Update controller_test.go * NET-5132 - Configure multiport routing for connect proxies in TProxy mode * formatting golden files * reverting golden files and adding changes in manually. build implicit destinations still has some issues. * fixing files that were incorrectly repeating the outbound listener * PR comments * extract AlpnProtocol naming convention to getAlpnProtocolFromPortName(portName) * removing address level filtering. * adding license to resources_test.go --------- Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com> * added time stamps (#18757) * fix: write endpoint errors out gracefully (#18743) * [NET-5574] Update Go version to 1.20.8 (#18742) Update Go version to 1.20.8 This resolves several CVEs (see changelog entry). * Add Envoy golden test for OTEL access logging extension (#18760) * mesh: create new routes-controller to reconcile xRoute types into a ComputedRoutes resource (#18460) This new controller produces an intermediate output (ComputedRoutes) that is meant to summarize all relevant xRoutes and related mesh configuration in an easier-to-use format for downstream use to construct the ProxyStateTemplate. It also applies status updates to the xRoute resource types to indicate that they are themselves semantically valid inputs. * xds controller: setup watches for and compute leaf cert references in ProxyStateTemplate, and wire up leaf cert manager dependency (#18756) * Refactors the leafcert package to not have a dependency on agent/consul and agent/cache to avoid import cycles. This way the xds controller can just import the leafcert package to use the leafcert manager. The leaf cert logic in the controller: * Sets up watches for leaf certs that are referenced in the ProxyStateTemplate (which generates the leaf certs too). * Gets the leaf cert from the leaf cert cache * Stores the leaf cert in the ProxyState that's pushed to xds * For the cert watches, this PR also uses a bimapper + a thin wrapper to map leaf cert events to related ProxyStateTemplates Since bimapper uses a resource.Reference or resource.ID to map between two resource types, I've created an internal type for a leaf certificate to use for the resource.Reference, since it's not a v2 resource. The wrapper allows mapping events to resources (as opposed to mapping resources to resources) The controller tests: Unit: Ensure that we resolve leaf cert references Lifecycle: Ensure that when the CA is updated, the leaf cert is as well Also adds a new spiffe id type, and adds workload identity and workload identity URI to leaf certs. This is so certs are generated with the new workload identity based SPIFFE id. * Pulls out some leaf cert test helpers into a helpers file so it can be used in the xds controller tests. * Wires up leaf cert manager dependency * Support getting token from proxytracker * Add workload identity spiffe id type to the authorize and sign functions --------- Co-authored-by: John Murret <john.murret@hashicorp.com> * Test: NET-5592 - update Nomad integration testing (#18768) NET-5592 - update Nomad integration testing * Add workload identity ACL rules (#18769) * Windows Integration Test Changes (#18758) * some changes to debug * revert machines * increased timeout * added sleep 10 seconds before test start * chagne envoy version * removed sleep * revert timeout * replace position * removed date * Revert "[NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437)" This reverts commit 05604eeec1186a4c8a6156e7519a579171659c1a. * fix build * Revert "replace position" This reverts commit 48e6af46a8daae186c283f30d316b1104906993e. * Revert "Revert "[NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437)"" This reverts commit d7c568e2be727b72e6827225782e0e17ac06b74a. * comment out api gateway http hostnames test * fix import * revert integ test run on PR * Add V2 TCP traffic permissions (#18771) Add support for TCP traffic permissions * resource: add helper to normalize inner Reference tenancy during mutate (#18765) When one resource contains an inner field that is of type *pbresource.Reference we want the Tenancy to be reasonably defaulted by the following rules: 1. The final values will be limited by the scope of the referenced type. 2. Values will be inferred from the parent's tenancy, and if that is insufficient then using the default tenancy for the type's scope. 3. Namespace will only be used from a parent if the reference and the parent share a partition, otherwise the default namespace will be used. Until we tackle peering, this hard codes an assumption of peer name being local. The logic for defaulting may need adjustment when that is addressed. * Disable docs change CI skip detection (#18775) There's currently a bug that causes CI to be skipped on all non-PR changes. Until that's fixed and we can be certain the check will fail CI or default to running tests in the case of errors, disabling this check. * delete command ready (#18679) * delete command for resource management * Vault CA provider clean up previous default issuers (#18773) * Fix snapshot creation issue. (#18783) The renaming of files from oss -> ce caused incorrect snapshots to be created due to ce writes now happening prior to ent writes. When this happens various entities will attempt to be restored from the snapshot prior to a partition existing and will cause a panic to occur. * catalog: service endpoints inherits protocol from service when workload doesn't have one (#18792) * add fuzz tests to resourcehcl package and fix some panics (#18798) * add a simple fuzz test for the resourcehcl package and fix some panics uncovered by the test * fix default for null values * Clean up resources in test (#18799) * Updates gateway-class-config docs to include fields for privileged container port-mapping and Openshift Security Context Constraints Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * Removes errant word in docs that was breaking the build (#18804) * Fix flaky test (#18805) * register traffic permission and workload identity types (#18704) * add workload identity and traffic permission protos * register new types * add generated pb code * fix exports.go path * add proto newlines * fix type name Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com> * address review * fix protos and add tests * fix validation constraints * add tests --------- Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com> * [NET-5329] use acl templated policy under the hood for node/service identities (#18813) * [NET-5334] Added CLI commands for templated policies (#18816) * mesh: Wire ComputedRoutes into the ProxyStateTemplate via the sidecar controller (#18752) Reworks the sidecar controller to accept ComputedRoutes as an input and use it to generate appropriate ProxyStateTemplate resources containing L4/L7 mesh configuration. * mesh: call the right mapper (#18818) * fix templated policy cli output (#18821) * resource: default peername to "local" for now (#18822) * Wire up traffic permissions (#18812) Wire up traffic permissions * added consul and envoy version constraints (#18726) * added consul and envoy version constraints * fixed Destination configuraiton and added tproxy requirement * Apply suggestions from code review Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> --------- Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> * Default to tcp protocol when workload protocol is unspecified (#18824) * Retry and timeout test acceptance test (#18791) * retry and timeout test * add docker mirrior * checkpoint * add in error * add in delay * up error rate * fix status code * Use embedded strings for templated policies (#18829) * consul operator raft transfer-leader should send the id (#17107) Fixes #16955 Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> * Allow empty data writes for resources (#18819) * allow nil data writes for resources * update demo to test valid type with no data * Fix nondeterministic test (#18828) * mesh: prevent writing a ComputedRoutes with no ported configs (#18833) * catalog: Default protocol to tcp in catalog.Service if unspecified (#18832) * Adjust metrics test (#18837) * Audit log consul 1.17x changes (#18669) * audit log consul 1.17x changes * added some details * verbose * Fixes for integration tests windows for ENT (#18839) * fixes for integration tests * fix runner size for enterprise * fix spacing * fix spacing * removed branch test run * fix: provide meaningful error messages and add test (#18772) * fix: provide meaningful error messages and add test * fix: return error instead of warning when extra args are provided * add v2 tenancy bridge Flag and v2 Tenancy Bridge initial implementation (#18830) * add v2 tenancy bridge and a feature flag for v2 tenancy * move tenancy bridge v2 under resource package * docs: Fix typo in description for server_addresses (#18838) Change 'If not port' to 'If no port'. Resolves #18553 * grafana: display connected consul-dataplanes (#18842) * Adding Apigee for ext_authz, minor fix in the default ext_authz docs (#18796) * adding apigee for ext_authz, minor fix * adding the Apigee docs to nav * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Blake Covarrubias <blake.covarrubias@gmail.com> * addressing feedback * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Blake Covarrubias <blake.covarrubias@gmail.com> * catalog: normalize/default/validate tenancy components of FailoverPolicy internal References (#18825) FailoverPolicy resources contain inner Reference fields. We want to ensure that components of those reference Tenancy fields left unspecified are defaulted using the tenancy of the enclosing FailoverPolicy resource. As the underlying helper being used to do the normalization calls the function modified in #18822, it also means that the PeerName field will be set to "local" for now automatically to avoid "local" != "" issues downstream. * Fix gateway services cleanup where proxy deregistration happens after service deregistration (#18831) * Fix gateway services cleanup where proxy deregistration happens after service deregistration * Add test * Add changelog * Fix comment * Move ACL templated policies to hcl files (#18853) * mesh: update xds controller to synthesize empty endpoints when no endpoints ref is found (#18835) * mesh: normalize/default/validate tenancy components of mesh internal References (#18827) HTTPRoute, GRPCRoute, TCPRoute, and Upstreams resources contain inner Reference fields. We want to ensure that components of those reference Tenancy fields left unspecified are defaulted using the tenancy of the enclosing resource. As the underlying helper being used to do the normalization calls the function modified in #18822, it also means that the PeerName field will be set to "local" for now automatically to avoid "local" != "" issues downstream. * resource: mutate and validate before acls on write (#18868) * Remove flaky test assertions (#18870) * mesh: only build tproxy outbound listener once per destination (#18836) Previously, when using implicit upstreams, we'd build outbound listener per destination instead of one for all destinations. This will result in port conflicts when trying to send this config to envoy. This PR also makes sure that leaf and root references are always added (before we would only add it if there are inbound non-mesh ports). Also, black-hole traffic when there are no inbound ports other than mesh * NET-4519 Collecting journald logs in "consul debug" bundle (#18797) * debug since * fix docs * chagelog added * fix go mod * debug test fix * fix test * tabs test fix * Update .changelog/18797.txt Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com> --------- Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com> * CI: lint test-integ (#18875) * CI: lint test-integ * fix lint error * Honor Default Traffic Permissions in V2 (#18886) wire up v2 default traffic permissions * docs: Fix HCL, JSON, and YAML syntax errors (#18879) This commit fixes syntax errors in HCL, JSON, and YAML example configurations. In some cases, it replaces the code example with the proper format for the code block. Also fixes HCL formatting and misc opportunistic updates to codeblock. Co-authored-by: Tu Nguyen <im2nguyen@gmail.com> * Fix code block examples on dns-static-lookups.mdx (#18880) HCL and JSON configuration examples were being displayed in the same code block. This commit separates the configurations to properly display them as independent configuration examples. * docs: Fix Kubernetes CRD example configs (#18878) Fixes configuration examples for several Consul Kubernetes CRDs. The CRDs were missing required fields such as `apiVersion`, `metadata`, and `spec`. Co-authored-by: Tu Nguyen <im2nguyen@gmail.com> * docs: Apigee extension backport (#18847) * commit * link text edits * Add operator audit endpoint changes (#18899) * docs: Remove YAML service registration examples (#18877) Remove YAML service registration examples and replace them with JSON. This is because YAML is not a supported configuration format for the Consul's agent configuration, nor is it supported by the HTTP API. This commit replaces the YAML examples with JSON and adds additional JSON examples where they were missing. * Add otel-access-logging Envoy extension integration test (#18898) * Add note about service upstream env var dot broken (#18895) * add note about service upstream env var dot broken * Apply suggestions from code review Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --------- Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> * xds: Use downstream protocol when connecting to local app (#18573) Configure Envoy to use the same HTTP protocol version used by the downstream caller when forwarding requests to a local application that is configured with the protocol set to either `http2` or `grpc`. This allows upstream applications that support both HTTP/1.1 and HTTP/2 on a single port to receive requests using either protocol. This is beneficial when the application primarily communicates using HTTP/2, but also needs to support HTTP/1.1, such as to respond to Kubernetes HTTP readiness/liveness probes. Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com> * resource: ensure resource.AuthorizerContext properly strips the local… (#18908) resource: ensure resource.AuthorizerContext properly strips the local peer name * docs: Add complete auth method payloads (#18849) This commit modifies the example payloads for various auth methods to remove 'other fields' and instead use complete example payloads. * agent/config: prevent startup if resource-apis experiment and cloud are enabled (#18876) * v2: various fixes to make K8s tproxy multiport acceptance tests and manual explicit upstreams (single port) tests pass (#18874) Adding coauthors who mobbed/paired at various points throughout last week. Co-authored-by: Dan Stough <dan.stough@hashicorp.com> Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: John Murret <john.murret@hashicorp.com> Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com> Co-authored-by: Michael Wilkerson <mwilkerson@hashicorp.com> * Added changelog entries for 1.14.10, 1.15.6, 1.16.2 (#18917) * [NET-5332] Add nomad server templated policy (#18888) * [NET-5332] Add nomad server templated policy * slksfd * mesh: make FailoverPolicy work in xdsv2 and ProxyStateTemplate (#18900) Ensure that configuring a FailoverPolicy for a service that is reachable via a xRoute or a direct upstream causes an envoy aggregate cluster to be created for the original cluster name, but with separate clusters for each one of the possible destinations. * Fix changelog order (#18918) * Fix changelog order * fix ordering or entries * Add namespace proto and registration (#18848) * add namespace proto and registration * fix proto generation * add missing copywrite headers * fix proto linter errors * fix exports and Type export * add mutate hook and more validation * add more validation rules and tests * Apply suggestions from code review Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * fix owner error and add test * remove ACL for now * add tests around space suffix prefix. * only fait when ns and ap are default, add test for it --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * Update base ref property name (#18851) * Update base ref property name * Test skip ci (#18924) test_push_merge * cleanup test push code * dns token (#17936) * dns token fix whitespace for docs and comments fix test cases fix test cases remove tabs in help text Add changelog Peering dns test Peering dns test Partial implementation of Peered DNS test Swap to new topology lib expose dns port for integration tests on client remove partial test implementation remove extra port exposure remove changelog from the ent pr Add dns token to set-agent-token switch Add enterprise golden file Use builtin/dns template in tests Update ent dns policy Update ent dns template test remove local gen certs fix templated policy specs * add changelog * go mod tidy * NET-5131 - support multiple ported upstreams tests (#18923) * add multiple upstream ports to golden file test for destination builder * NET-5131 - add unit tests for multiple ported upstreams * fix merge conflicts * Fix for loop in filter_changed_files_go_test script (#18931) * iterate through array * remove comment * Fix create dns token docs (#18927) * [NET-5589] Optimize leaf watch diff on xds controller. (#18921) Optimize leaf watch diff on xds controller. * NET-4884 - Terminating gateway tests for namespaces & partitions (#18820) * Add gateway test to CE * Fix V2 Wildcard RBAC Regular Expressions (#18941) fix wildcard rbac regular expressions * skip flaky test (#18949) * docs: Fix invalid JSON in code examples (#18932) This commit fixes invalid JSON in various code examples. * do not trigger integration tests (#18948) * Dump response body on fail (#18962) * Add protoc generator to emit resource type variables (#18957) The annotations include a little more data than is strictly necessary because we will also have a protoc generator for openapi output. * docs: Change heading to filename in CodeBlockConfig (#18951) Change various CodeBlockConfig objects to use the `filename` attribute instead of `heading` when the code block references a named file. * docs: Replace unicode quotes with ASCII quotes (#18950) Replaces unicode quotation marks with ASCII quotation marks. For code examples, this fixes HCL decoding errors that would otherwise be raised when attempting to read the file. * net-5689 fix disabling panic threshold logic (#18958) * tenancy: split up tenancy `types.go` into CE version (#18966) * [NET-5589] Add jitter to xds v2 leaf cert watches (#18940) Add jitter to xds v2 leaf cert watches. * Fix docs for log file name changes (#18913) * fix docs * Update website/content/docs/agent/config/cli-flags.mdx Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Update website/content/docs/agent/config/cli-flags.mdx Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Update website/content/docs/agent/config/cli-flags.mdx Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * resource: allow for the ACLs.Read hook to request the entire data payload to perform the authz check (#18925) The ACLs.Read hook for a resource only allows for the identity of a resource to be passed in for use in authz consideration. For some resources we wish to allow for the current stored value to dictate how to enforce the ACLs (such as reading a list of applicable services from the payload and allowing service:read on any of them to control reading the enclosing resource). This change update the interface to usually accept a *pbresource.ID, but if the hook decides it needs more data it returns a sentinel error and the resource service knows to defer the authz check until after fetching the data from storage. * catalog: add ACL checks for FailoverPolicy resources (#18919) FailoverPolicy resources are name-aligned with the Service they control. They also contain a list of possible failover destinations that are References to other Services. The ACLs should be: - list: (default) - read: service:<resource_name>:read - write: service:<resource_name>:write + service:<destination_name>:read (for any destination) * catalog: fix for new method argument (#18978) * catalog,mesh,auth: Bump versions to v2beta1 (#18930) * [Docs] Add note to jwt docs to specify the need for ACLs (#18942) * remove now orphaned generated v1alpha1 pb.go files (#18982) * peerstream: fix flaky test related to autopilot integration (#18979) * mesh: add ACL checks for DestinationPolicy resources (#18920) DestinationPolicy resources are name-aligned with the Service they control. The ACLs should be: - list: (default) - read: service:<resource_name>:read - write: service:<resource_name>:write * mesh: add ACL checks for xRoute resources (#18926) xRoute resources are not name-aligned with the Services they control. They have a list of "parent ref" services that they alter traffic flow for, and they contain a list of "backend ref" services that they direct that traffic to. The ACLs should be: - list: (default) - read: - ALL service:<parent_ref_service>:read - write: - ALL service:<parent_ref_service>:write - ALL service:<backend_ref_service>:read * Traffic Permissions Validations (#18907) add TP validations and mutation and add CTP validations * feat: remove resource api client from api module (#18984) * feat: remove resource api client from api module * fix: go mod clean up * mesh: store bound reference pointers on a ComputedRoutes resource and use during reconcile (#18965) xRoute resource types contain a slice of parentRefs to services that they manipulate traffic for. All xRoutes that have a parentRef to given Service will be merged together to generate a ComputedRoutes resource name-aligned with that Service. This means that a write of an xRoute with 2 parent ref pointers will cause at most 2 reconciles for ComputedRoutes. If that xRoute's list of parentRefs were ever to be reduced, or otherwise lose an item, that subsequent map event will only emit events for the current set of refs. The removed ref will not cause the generated ComputedRoutes related to that service to be re-reconciled to omit the influence of that xRoute. To combat this, we will store on the ComputedRoutes resource a BoundResources []*pbresource.Reference field with references to all resources that were used to influence the generated output. When the routes controller reconciles, it will use a bimapper to index this influence, and the dependency mappers for the xRoutes will look themselves up in that index to discover additional (former) ComputedRoutes that need to be notified as well. * Add CLI support for json (#18991) * add cli support for json format * add tests for json parsing * make owner and id pointers. * add copyright header * remove print --------- Co-authored-by: Poonam Jadhav <poonam.jadhav@hashicorp.com> * mesh: compute more of the xRoute features into ComputedRoutes (#18980) Convert more of the xRoutes features that were skipped in an earlier PR into ComputedRoutes and make them work: - DestinationPolicy defaults - more timeouts - load balancer policy - request/response header mutations - urlrewrite - GRPCRoute matches * catalog,mesh,auth: Move resource types to the proto-public module (#18935) * mesh: update various protobuf comments for mesh types (#18993) * docs: add changelog (#18994) * api: add Token field to ServiceRegisterOpts (#18983) Ongoing work to support Nomad Workload Identity for authenticating with Consul will mean that Nomad's service registration sync with Consul will want to use Consul tokens scoped to individual workloads for registering services and checks. The `ServiceRegisterOpts` type in the API doesn't have an option to pass the token in, which prevent us from sharing the same Consul connection for all workloads. Add a `Token` field to match the behavior of `ServiceDeregisterOpts`. * api: add `CheckRegisterOpts` method to Agent API (#18943) Ongoing work to support Nomad Workload Identity for authenticating with Consul will mean that Nomad's service registration sync with Consul will want to use Consul tokens scoped to individual workloads for registering services and checks. The `CheckRegister` method in the API doesn't have an option to pass the token in, which prevent us from sharing the same Consul connection for all workloads. Add a `CheckRegisterOpts` to match the behavior of `ServiceRegisterOpts`. * grafana: fix a query metrics from ent and add consul version (#18998) * v2 explicit destination traffic permissions (#18823) * workload identity boilerplate * notes from discussion with Iryna * WIP traffic permissions controller poc * workload identity, traffic permissions validation, errors, types * traffic permissions mapper framing, traffic permissions controller updates. * more roughing out of the controller * cleanup * controller and mapper logic * tests * refactor mapper logic, add tests * clean up tenancy and integration test stubs * consolidate mapping * cleanup cache leak, revert bimapper changes * address review comments * test fix and rebase * use resource helper --------- Co-authored-by: John Landa <john.landa@hashicorp.com> * mesh: rename Upstreams and UpstreamsConfiguration to Destinations* (#18995) * auth: register auth controllers with the server (#19000) * acl: default tenancy with the no-auth ACL resolver (#19006) When using the no-auth acl resolver (the case for most controllers and the get-envoy-boostrap-params endpoint), ResolveTokenAndDefaultMeta method only returns an acl resolver. However, the resource service relies on the ent meta to be filled in to do the tenancy defaulting and inheriting it from the token when one is present. So this change makes sure that the ent meta defaulting always happens in the ACL resolver. * NET-5657 - consul-containers test for explicit upstreams (#18952) * Explicit container test * remove static resources * fix passing serviceBindPorts * WIP * fix explicit upstream test * use my image in CI until dataplane is fixed. * gofmt * fixing reference to v2beta1 in test-containers * WIP * remove bad references * add missing license headers * allow access internal/resource/resourcetest * fix check-allowed-imports to append array items * use preview image for dataplane * revert some inadverntent comment updates in peering_topology * add building local consul-dataplane image to compatibility-tests CI * fix substitution in CI * change upstreams to destinations based on incoming change * fixing use of upstreams in resource update * remove commented out lines and enable envoy concurrency on dataplane. * changes to addess PR feedback * small fixes --------- Co-authored-by: Eric <eric@haberkorn.co> * Skip the catalog v2 upgrade test (#19005) We intentionally broke api compatibility here as we are not yet maintaining backwards compat for the v2 apis * Fix type of datacenter in Service Resolvers Config Entry (#19004) * docs: Rename Consul OSS to Consul CE (#19009) Rename references of Consul OSS to Consul Community Edition (CE). Co-authored-by: Tu Nguyen <im2nguyen@gmail.com> * Update release notes to indicate folks should upgrade to to 1.16.2 or later (#19002) * Update v1_16_x.mdx * Update upgrade-specific.mdx * Update website/content/docs/upgrading/upgrade-specific.mdx Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Handle Traffic Permissions With Empty Sources Properly (#19024) Fix issues with empty sources * Validate that each permission on traffic permissions resources has at least one source. * Don't construct RBAC policies when there aren't any principals. This resulted in Envoy rejecting xDS updates with a validation error. ``` error= | rpc error: code = Internal desc = Error adding/updating listener(s) public_listener: Proto constraint validation failed (RBACValidationError.Rules: embedded message failed validation | caused by RBACValidationError.Policies[consul-intentions-layer4-1]: embedded message failed validation | caused by PolicyValidationError.Principals: value must contain at least 1 item(s)): rules { ``` * Include RequestTimeout in marshal/unmarshal of ServiceResolverConfigEntry * backport of commit 4ad1a8760d1276ec2f2ce7b9be741badb7abb1f6 * backport of commit 9109bdd79163dbdda4ef1e7dc7e990ba2cab8858 --------- Co-authored-by: wangxinyi7 <121973291+wangxinyi7@users.noreply.github.com> Co-authored-by: Poonam Jadhav <poonam.jadhav@hashicorp.com> Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com> Co-authored-by: Anita Akaeze <anita.akaeze@hashicorp.com> Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> Co-authored-by: Semir Patel <semir.patel@hashicorp.com> Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Dan Bond <danbond@protonmail.com> Co-authored-by: David Yu <dyu@hashicorp.com> Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: Blake Covarrubias <blake@covarrubi.as> Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> Co-authored-by: John Murret <john.murret@hashicorp.com> Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com> Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> Co-authored-by: cskh <hui.kang@hashicorp.com> Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com> Co-authored-by: Vijay <vijayraghav22@gmail.com> Co-authored-by: Dan Stough <dan.stough@hashicorp.com> Co-authored-by: Natalie Smith <nataliemegans@gmail.com> Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com> Co-authored-by: Chris S. Kim <ckim@hashicorp.com> Co-authored-by: John Maguire <john.maguire@hashicorp.com> Co-authored-by: Curt Bushko <cbushko@gmail.com> Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> Co-authored-by: Joshua Timmons <josh.timmons@hashicorp.com> Co-authored-by: Hardik Shingala <34568645+hdkshingala@users.noreply.github.com> Co-authored-by: Ashvitha <ashvitha.sridharan@hashicorp.com> Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> Co-authored-by: Derek Menteer <105233703+hashi-derek@users.noreply.github.com> Co-authored-by: Valeriia Ruban <valeriia.ruban@hashicorp.com> Co-authored-by: skpratt <sarah.pratt@hashicorp.com> Co-authored-by: Phil Porada <pgporada@users.noreply.github.com> Co-authored-by: Samantha <hello@entropy.cat> Co-authored-by: Nick Irvine <115657443+nfi-hashicorp@users.noreply.github.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com> Co-authored-by: Gerard Nguyen <gerard@hashicorp.com> Co-authored-by: Melisa Griffin <missylbytes@users.noreply.github.com> Co-authored-by: Matt Keeler <mjkeeler7@gmail.com> Co-authored-by: Jeremy Jacobson <jjacobson93@users.noreply.github.com> Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com> Co-authored-by: Sophie Gairo <97480023+sophie-gairo@users.noreply.github.com> Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com> Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com> Co-authored-by: James Hartig <me@jameshartig.com> Co-authored-by: Gautam <gautambaghel93@gmail.com> Co-authored-by: Blake Covarrubias <blake.covarrubias@gmail.com> Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com> Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com> Co-authored-by: Tu Nguyen <im2nguyen@gmail.com> Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com> Co-authored-by: Nick Ethier <nethier@hashicorp.com> Co-authored-by: John Landa <jonathanlanda@gmail.com> Co-authored-by: Tim Gross <tgross@hashicorp.com> Co-authored-by: John Landa <john.landa@hashicorp.com> Co-authored-by: Eric <eric@haberkorn.co> Co-authored-by: Chris Thain <chris.m.thain@gmail.com>

Update upgrade-specific.mdx

…nto release/1.15.x (#19073) backport of commit 63a5831 Co-authored-by: absolutelightning <ashesh.vidyut@hashicorp.com>

…into release/1.15.x (#19079) backport of commit 1ef51ad Co-authored-by: cskh <hui.kang@hashicorp.com>

…d clusters NET-3463 into release/1.15.x (#19091) Allow connections through Terminating Gateways from peered clusters NET-3463 (#18959) * Add InboundPeerTrustBundle maps to Terminating Gateway * Add notify and cancelation of watch for inbound peer trust bundles * Pass peer trust bundles to the RBAC creation function * Regenerate Golden Files * add changelog, also adds another spot that needed peeredTrustBundles * Add basic test for terminating gateway with peer trust bundle * Add intention to cluster peered golden test * rerun codegen * update changelog * really update the changelog --------- Co-authored-by: Thomas Eckert <teckert@hashicorp.com> Co-authored-by: Melisa Griffin <melisa.griffin@hashicorp.com>

…18322) (#19116) NET-4135 - Fix NodeMeta filtering Catalog List Services API (#18322) * logs for debugging * Init * white spaces fix * added change log * Fix tests * fix typo * using queryoptionfilter to populate args.filter * tests * fix test * fix tests * fix tests * fix tests * fix tests * fix variable name * fix tests * fix tests * fix tests * Update .changelog/18322.txt * fix change log * address nits * removed unused line * doing join only when filter has nodemeta * fix tests * fix tests * Update agent/consul/catalog_endpoint.go * fix tests * removed unwanted code --------- Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com> Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

…cks into release/1.15.x (#19121) backport of commit ff451f3 Co-authored-by: David Yu <dyu@hashicorp.com>

#19152) into release/1.15.x (#19156) backport of commit 02bfe1d Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

…net` to 0.17.0 into release/1.15.x (#19233) * backport of commit d7d9de9 * backport of commit 0794b1c --------- Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>

… for dev images into release/1.15.x (#19280) Backport of build(docker): always publish full and minor version tags for dev images into release/1.17.x (#19282) backport of commit c6bb4a5 Co-authored-by: DanStough <dan.stough@hashicorp.com>

Vault CA bugfixes (#19285) * Re-add retry logic to Vault token renewal * Fix goroutine leak * Add test for detecting goroutine leak * Add changelog * Rename tests * Add comment

#19347) Backport of Add grpc keepalive configuration into release/1.15.x (#19339) Add grpc keepalive configuration. (#19339) Prior to the introduction of this configuration, grpc keepalive messages were sent after 2 hours of inactivity on the stream. This posed issues in various scenarios where the server-side xds connection balancing was unaware that envoy instances were uncleanly killed / force-closed, since the connections would only be cleaned up after ~5 minutes of TCP timeouts occurred. Setting this config to a 30 second interval with a 20 second timeout ensures that at most, it should take up to 50 seconds for a dead xds connection to be closed.

…ase/1.15.x (#19191) backport of commit 1fffd23 Co-authored-by: Thomas Eckert <teckert@hashicorp.com>

…es are made … into release/1.15.x (#19064)

…elease/1.15.x (#19397) backport of commit 8a0896d Co-authored-by: Tu Nguyen <im2nguyen@gmail.com>

…ase/1.15.x (#19394) Use strict DNS for mesh gateways with hostnames Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>

…9406) Backport of Fix casing in example yaml config to release/1.15.x Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

CVE-2023-44487) to release/1.15.x (#19421) Bump google.golang.org/grpc to 1.56.3 This resolves [CVE-2023-44487](https://nvd.nist.gov/vuln/detail/CVE-2023-44487). Also includes various fixes from later release versions required for tests and linters to pass. See 77f44fa for the majority of these changes. Co-authored-by: Chris Thain <chris.m.thain@gmail.com>

…5.x (#19439) docs: Fix Kubernetes CRD example configs (#18878) Fixes configuration examples for several Consul Kubernetes CRDs. The CRDs were missing required fields such as `apiVersion`, `metadata`, and `spec`. Co-authored-by: Tu Nguyen <im2nguyen@gmail.com>

…ane requirements into release/1.15.x (#19450) Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

…#19464) * no-op commit due to failed cherry-picking * added redirect for conf entries 1.8.x (#19460) fixed merge conflicts in redirect --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

backport of commit d399654 Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

… of going directly to the docker hub into release/1.15.x (#19484) backport of commit 5ab1e98 Co-authored-by: R.B. Boyer <rb@hashicorp.com>

* Consul k8s release 1.1.7 * fix typos

Only show back to hcp link if CONSUL_HCP_URL is present

…15.x (#19518) backport of commit b8a18c4 Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>

…elease/1.15.x (#19521) backport of commit a47995f Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>

* backport of commit 5cb614e * Fix flaky metrics tests. --------- Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>

…#19559) backport of commit d6283a9 Co-authored-by: DanStough <dan.stough@hashicorp.com>

…e/1.15.x (#19424) backport of commit 26cb1f6 Co-authored-by: John Murret <john.murret@hashicorp.com>

…19624) backport of commit d4ba678 Co-authored-by: Joshua Timmons <joshua.timmons1@gmail.com>

…19630) * backport of commit 54ecfa6 * backport of commit a5bc545 --------- Co-authored-by: am-ak <114914687+am-ak@users.noreply.github.com>

…ly hardy chamois 2 (#19623) * feat: Change global-read-only policy to non editable (downgraded styles of alerts)

…lf (#19712) * chore: add suffix to consul version in sidenav

… 4.5.0 into release/1.15.x (#19739) backport of commit fbb8be5 Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>

fix a panic in the CLI when deleting an acl policy with an unknown name (#19679) * fix a panic in the CLI when deleting an acl policy with an unknown name * add changelog

…15.x (#19746) backport of commit 5316676 Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>

docs: Update consul-k8s Helm chart docs (1.1.x) Sync docs for recent changes to the Helm chart from `consul-k8s`.

…19699) (#19749) Revert "[NET-6232] docs: Update consul-k8s Helm chart docs (1.1.x) (#19699)" This reverts commit c4b3ca8.

* [NET-5688] APIGateway UI Topology Fixes (#19657) * Update catalog and ui endpoints to show APIGateway in gateway service topology view * Added initial implementation for service view * updated ui * Fix topology view for gateways * Adding tests for gw controller * remove unused args * Undo formatting changes * Fix call sites for upstream/downstream gw changes * Add config entry tests * Fix function calls again * Move from ServiceKey to ServiceName, cleanup from PR review * Add additional check for length of services in bound apigateway for IsSame comparison * fix formatting for proto * gofmt * Add DeepCopy for retrieved BoundAPIGateway * gofmt * gofmt * Rename function to be more consistent * Remove BUSL license * Fix import

Commits on Dec 1, 2023

run make after copywrite ignore additions

dlaguerta committed Dec 1, 2023

Configuration menu

View commit details

Copy full SHA for 0ab5d8c

Browse repository at this point

Copy the full SHA

0ab5d8c View commit details

Browse the repository at this point in the history

copywrite EOL proto headers debugging #19799

copywrite EOL proto headers debugging #19799

Commits on May 15, 2023

Commits on May 16, 2023

Commits on May 17, 2023

Commits on May 18, 2023

Commits on May 19, 2023

Commits on May 22, 2023

Commits on May 23, 2023

Commits on May 24, 2023

Commits on May 25, 2023

Commits on May 26, 2023

Commits on May 29, 2023

Commits on May 30, 2023

Commits on May 31, 2023

Commits on Jun 1, 2023

Commits on Jun 2, 2023

Commits on Jun 5, 2023

Commits on Jun 6, 2023

Commits on Jun 7, 2023

Commits on Jun 8, 2023

Commits on Jun 9, 2023

Commits on Jun 12, 2023

Commits on Jun 13, 2023

Commits on Jun 14, 2023

Commits on Jun 15, 2023

Commits on Jun 16, 2023

Commits on Jun 17, 2023

Commits on Jun 18, 2023

Commits on Jun 21, 2023

Commits on Jun 22, 2023

Commits on Jun 23, 2023

Commits on Jun 26, 2023

Commits on Jun 27, 2023

Commits on Jun 28, 2023

Commits on Jun 29, 2023

Commits on Jun 30, 2023

Commits on Jul 1, 2023

Commits on Jul 5, 2023

Commits on Jul 6, 2023

Commits on Jul 7, 2023

Commits on Jul 10, 2023

Commits on Jul 11, 2023

Commits on Jul 12, 2023

Commits on Jul 13, 2023

Commits on Jul 14, 2023

Commits on Jul 17, 2023

Commits on Jul 18, 2023

Commits on Jul 19, 2023

Commits on Jul 20, 2023

Commits on Jul 21, 2023

Commits on Jul 23, 2023

Commits on Jul 24, 2023

Commits on Jul 25, 2023

Commits on Jul 26, 2023

Commits on Jul 27, 2023

Commits on Aug 1, 2023

Commits on Aug 2, 2023

Commits on Aug 3, 2023

Commits on Aug 4, 2023

Commits on Aug 8, 2023

Commits on Aug 9, 2023

Commits on Aug 10, 2023

Commits on Aug 11, 2023

Commits on Aug 15, 2023

Commits on Aug 16, 2023

Commits on Aug 17, 2023

Commits on Aug 21, 2023

Commits on Aug 23, 2023

Commits on Aug 28, 2023

Commits on Aug 31, 2023

Commits on Sep 4, 2023

Commits on Sep 6, 2023

Commits on Sep 8, 2023

Commits on Sep 10, 2023

Commits on Sep 11, 2023

Commits on Sep 12, 2023

Commits on Sep 13, 2023

Commits on Sep 18, 2023

Commits on Sep 19, 2023