Add AutoUpdate Client/Cache implementation #46661

vapopov · 2024-09-17T02:53:23Z

In this PR implemented client and cache layer for the auto update service

Related: #39805

api/client/autoupdate/autoupdate.go

lib/auth/authclient/api.go

lib/cache/cache.go

api/client/webclient/webclient.go

lib/client/api.go

lib/authz/permissions.go

lib/cache/cache.go

lib/cache/cache_test.go

lib/cache/collections.go

lib/services/local/events.go

api/client/client.go

lib/cache/cache.go

Rename comments for consistency

* Add AutoUpdate Client/Cache implementation * CR changes * Add permission for proxy to access resources * Rename all occurrences auto update to camelcase * Remove auto update client wrapper * Drop AutoUpdateServiceClient helper Rename comments for consistency

public-teleport-github-review-bot · 2024-09-19T03:04:30Z

@vapopov See the table below for backport results.

Branch	Result
branch/v14	Failed
branch/v15	Failed
branch/v16	Failed

* Add AutoUpdate Client/Cache implementation * CR changes * Add permission for proxy to access resources * Rename all occurrences auto update to camelcase * Remove auto update client wrapper * Drop AutoUpdateServiceClient helper Rename comments for consistency

* Displaying mode and controls to additional participants * Moving SessionControlsInfoBroadcast over to kube/proxy * Transitioning to consistent proxy-emitted mode+controls * Moving message broadcast so new participant wont see it * Possible unit test fix (cant seem to test locally) * Fixed unit test * Adding a line break before messaging the participant * Linter errors * Emitting audit event and controls message for additional parties, i.e. not the session initiator * Revert "Emitting audit event and controls message for additional parties, i.e. not the session initiator" This reverts commit b66ad27. * Add User Tasks resource - protos (#46059) * Add User Integration Tasks resource - protos * add account id * move state to task instead of instance * rename from user integration task to user task * add instance id * Add notice to web UI that users arent equal to MAU (#46686) This adds a dismissible notice to the Users page for usage based billing users that notifies them that the user count here isn't an accurate reflection of MAU * Clarify TLS requirements in the Jira guide (#46484) Closes #45654 - Indicate that certificates for the Jira web server cannot be self signed. - Remove references to Caddy and a `Certificate` resource, which were left over from an attempted change to this guide that was not fully completed. * Remove TXT record validation of custom DNS zones in VNet (#46709) * Remove TXT record validation from custom DNS zones * Remove mentions of TXT records from docs * Outline in the RFD why domain verification was dropped * Update rfd/0163-vnet.md Co-authored-by: Nic Klaassen <nic@goteleport.com> --------- Co-authored-by: Nic Klaassen <nic@goteleport.com> * docs: mention the --days flag when executing an audit log query (#45764) * Update access-monitoring.mdx Include the default date range in the CLI example. This range is otherwise unclear and is hidden in the tctl audit help menu. * Update access-monitoring.mdx * Update docs/pages/admin-guides/access-controls/access-monitoring.mdx Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> --------- Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> * Remove access-graph path resolution, proxy `/enterprise` requests (#46541) * Remove `access-graph` path from tsconfig * Proxy /enterprise requested in dev * update e ref (#46726) * fix: tolerate mismatched key PEM headers (#46725) * fix: tolerate mismatched key PEM headers Issue #43381 introduced a regression where we now fail to parse PKCS8 encoded RSA private keys within an "RSA PRIVATE KEY" PEM block in some cases. This format is somewhat non-standard, usually PKCS8 data should be in a "PRIVATE KEY" PEM block. However, certain versions of OpenSSL and possibly even Teleport in specific cases have generated private keys in this format. This commit updates ParsePrivateKey and ParsePublicKey to be more tolerant of PKCS8, PKCS1, or PKIX key data no matter which PEM header is used. Fixes #46710 changelog: fixed regression in private key parser to handle mismatched PEM headers * fix typo in comment Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com> --------- Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com> * Use dynamic base path for favicon images (#46719) * Add Datadog Incident Management plugin support (#46271) * Implement datadog plugin * Add unit tests * Add fallback recipient config * Rename to Datadog Incident Management * Update tests * Datadog Incident Management * Update tctl resource plugin command * Typos * Lint * Exclude api changes for now * Set channel size * Address feedback - Add PluginShutdownTimeout const - Support api endpoint configuration - Add additional godocs/comments * Comment about datadog client package * Document Datadog API types * Only post resolution message when the AR is resolved * Fix lint * Unused function --------- Co-authored-by: hugoShaka <hugo.hervieux@goteleport.com> * Add AutoUpdate Client/Cache implementation (#46661) * Add AutoUpdate Client/Cache implementation * CR changes * Add permission for proxy to access resources * Rename all occurrences auto update to camelcase * Remove auto update client wrapper * Drop AutoUpdateServiceClient helper Rename comments for consistency * User Tasks: services and clients implementation (#46131) This PR adds the implementation for the User Tasks: - services (backend+cache) - clients (API + tctl) - light validation to set up the path for later PRs * expanding testplan for host user creation (#46729) * Fix operator docs reference generator bug (#46732) In the reference page for one Kubernetes operator resource, some Markdown links are malformed. The issue is that some fields of custom resource definitions used by the operator consist of arrays of anonymous objects with fields that are also objects. When creating docs based on these fields, the operator resource docs generator creates a malformed link reference. This change modifies the generator to replace any spaces with hyphens before outputting link references, causing the resulting internal links to work correctly. This change also does some light refactoring to remove an unnecessary `switch` statement. * [auto] Update AMI IDs for 16.4.0 (#46746) Co-authored-by: GitHub <noreply@github.com> * Remove deprecated HTTP RemoteCluster endpoints (#46756) * Remove deprecated HTTP RemoteCluster endpoints * Remove redundant test * Add `tbot` helm chart to `version.mk` (#46763) * Remove LockConfiguration.LockName (#46772) Cleans up the deprecated config option now that gravitational/teleport.e#5034 has been merged. * adding a reference to to the host user guide (#46765) * Replace more Logrus usage with Slog (#46757) * Remove logrus from lib/auth/machineid * Switch authclient.Config.Log and TunnelAuthDialerConfig.Log to Slog * Add *slog.Logger to auth.Server * Remove logrus usage in `lib/auth/access.go` * Replace logrus with slog in lib/auth/accountrecovery.go * Replace logrus with slog in `lib/auth/apiserver.go` * Add missing logger to auth.Server * Fix test * Update AWS roles ARNs displayed on `tsh app login` for AWS console apps (#44983) * feat(tsh): list aws console logins from server * chore(services): remove unified resources change This is being covered on another PR. * test(tsh): solve TestAzure flakiness by waiting using app servers are ready * fix(tsh): apps with logins were fallingback into using aws arns * refactor(client): use GetEnrichedResources * chore(client): rename function * refactor(tsh): directly resource lisiting for apps and reuse cluster client * chore(client): reset client changes * refactor(tsh): reuse cluster client for fetching allowed logins * chore(tsh): remove unused function param * refactor(tsh): update getApp retry with login * refactor(tsh): use a single function to grab profile and cluste client * refactor(tsh): perform retry with login at caller site * fix(tsh): close auth client * test(tsh): fix test failing due to login misconfiguration * test(tsh): fix lint errors * test(tsh): remove unused imports * bulk audit event export api (#46399) * Reverting back to using the emitSessionJoin boolean * Nits and removing a debug log --------- Co-authored-by: Marco Dinis <marco.dinis@goteleport.com> Co-authored-by: Michael <michael.myers@goteleport.com> Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> Co-authored-by: Rafał Cieślak <rafal.cieslak@goteleport.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Dan Johns <117299936+djohns7@users.noreply.github.com> Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> Co-authored-by: Ryan Clark <ryan.clark@goteleport.com> Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com> Co-authored-by: Bernard Kim <bernard@goteleport.com> Co-authored-by: hugoShaka <hugo.hervieux@goteleport.com> Co-authored-by: Vadym Popov <vadym.popov@goteleport.com> Co-authored-by: Erik Tate <erik.tate@goteleport.com> Co-authored-by: teleport-post-release-automation[bot] <128860004+teleport-post-release-automation[bot]@users.noreply.github.com> Co-authored-by: GitHub <noreply@github.com> Co-authored-by: Noah Stride <noah.stride@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> Co-authored-by: Gabriel Corado <gabriel.oliveira@goteleport.com> Co-authored-by: Forrest <30576607+fspmarshall@users.noreply.github.com>

) * Displaying mode and controls to additional participants * Moving SessionControlsInfoBroadcast over to kube/proxy * Transitioning to consistent proxy-emitted mode+controls * Moving message broadcast so new participant wont see it * Possible unit test fix (cant seem to test locally) * Fixed unit test * Adding a line break before messaging the participant * Linter errors * Emitting audit event and controls message for additional parties, i.e. not the session initiator * Revert "Emitting audit event and controls message for additional parties, i.e. not the session initiator" This reverts commit b66ad27. * Add User Tasks resource - protos (#46059) * Add User Integration Tasks resource - protos * add account id * move state to task instead of instance * rename from user integration task to user task * add instance id * Add notice to web UI that users arent equal to MAU (#46686) This adds a dismissible notice to the Users page for usage based billing users that notifies them that the user count here isn't an accurate reflection of MAU * Clarify TLS requirements in the Jira guide (#46484) Closes #45654 - Indicate that certificates for the Jira web server cannot be self signed. - Remove references to Caddy and a `Certificate` resource, which were left over from an attempted change to this guide that was not fully completed. * Remove TXT record validation of custom DNS zones in VNet (#46709) * Remove TXT record validation from custom DNS zones * Remove mentions of TXT records from docs * Outline in the RFD why domain verification was dropped * Update rfd/0163-vnet.md --------- * docs: mention the --days flag when executing an audit log query (#45764) * Update access-monitoring.mdx Include the default date range in the CLI example. This range is otherwise unclear and is hidden in the tctl audit help menu. * Update access-monitoring.mdx * Update docs/pages/admin-guides/access-controls/access-monitoring.mdx --------- * Remove access-graph path resolution, proxy `/enterprise` requests (#46541) * Remove `access-graph` path from tsconfig * Proxy /enterprise requested in dev * update e ref (#46726) * fix: tolerate mismatched key PEM headers (#46725) * fix: tolerate mismatched key PEM headers Issue #43381 introduced a regression where we now fail to parse PKCS8 encoded RSA private keys within an "RSA PRIVATE KEY" PEM block in some cases. This format is somewhat non-standard, usually PKCS8 data should be in a "PRIVATE KEY" PEM block. However, certain versions of OpenSSL and possibly even Teleport in specific cases have generated private keys in this format. This commit updates ParsePrivateKey and ParsePublicKey to be more tolerant of PKCS8, PKCS1, or PKIX key data no matter which PEM header is used. Fixes #46710 changelog: fixed regression in private key parser to handle mismatched PEM headers * fix typo in comment --------- * Use dynamic base path for favicon images (#46719) * Add Datadog Incident Management plugin support (#46271) * Implement datadog plugin * Add unit tests * Add fallback recipient config * Rename to Datadog Incident Management * Update tests * Datadog Incident Management * Update tctl resource plugin command * Typos * Lint * Exclude api changes for now * Set channel size * Address feedback - Add PluginShutdownTimeout const - Support api endpoint configuration - Add additional godocs/comments * Comment about datadog client package * Document Datadog API types * Only post resolution message when the AR is resolved * Fix lint * Unused function --------- * Add AutoUpdate Client/Cache implementation (#46661) * Add AutoUpdate Client/Cache implementation * CR changes * Add permission for proxy to access resources * Rename all occurrences auto update to camelcase * Remove auto update client wrapper * Drop AutoUpdateServiceClient helper Rename comments for consistency * User Tasks: services and clients implementation (#46131) This PR adds the implementation for the User Tasks: - services (backend+cache) - clients (API + tctl) - light validation to set up the path for later PRs * expanding testplan for host user creation (#46729) * Fix operator docs reference generator bug (#46732) In the reference page for one Kubernetes operator resource, some Markdown links are malformed. The issue is that some fields of custom resource definitions used by the operator consist of arrays of anonymous objects with fields that are also objects. When creating docs based on these fields, the operator resource docs generator creates a malformed link reference. This change modifies the generator to replace any spaces with hyphens before outputting link references, causing the resulting internal links to work correctly. This change also does some light refactoring to remove an unnecessary `switch` statement. * [auto] Update AMI IDs for 16.4.0 (#46746) * Remove deprecated HTTP RemoteCluster endpoints (#46756) * Remove deprecated HTTP RemoteCluster endpoints * Remove redundant test * Add `tbot` helm chart to `version.mk` (#46763) * Remove LockConfiguration.LockName (#46772) Cleans up the deprecated config option now that gravitational/teleport.e#5034 has been merged. * adding a reference to to the host user guide (#46765) * Replace more Logrus usage with Slog (#46757) * Remove logrus from lib/auth/machineid * Switch authclient.Config.Log and TunnelAuthDialerConfig.Log to Slog * Add *slog.Logger to auth.Server * Remove logrus usage in `lib/auth/access.go` * Replace logrus with slog in lib/auth/accountrecovery.go * Replace logrus with slog in `lib/auth/apiserver.go` * Add missing logger to auth.Server * Fix test * Update AWS roles ARNs displayed on `tsh app login` for AWS console apps (#44983) * feat(tsh): list aws console logins from server * chore(services): remove unified resources change This is being covered on another PR. * test(tsh): solve TestAzure flakiness by waiting using app servers are ready * fix(tsh): apps with logins were fallingback into using aws arns * refactor(client): use GetEnrichedResources * chore(client): rename function * refactor(tsh): directly resource lisiting for apps and reuse cluster client * chore(client): reset client changes * refactor(tsh): reuse cluster client for fetching allowed logins * chore(tsh): remove unused function param * refactor(tsh): update getApp retry with login * refactor(tsh): use a single function to grab profile and cluste client * refactor(tsh): perform retry with login at caller site * fix(tsh): close auth client * test(tsh): fix test failing due to login misconfiguration * test(tsh): fix lint errors * test(tsh): remove unused imports * bulk audit event export api (#46399) * Reverting back to using the emitSessionJoin boolean * Nits and removing a debug log --------- Co-authored-by: Marco Dinis <marco.dinis@goteleport.com> Co-authored-by: Michael <michael.myers@goteleport.com> Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> Co-authored-by: Rafał Cieślak <rafal.cieslak@goteleport.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Dan Johns <117299936+djohns7@users.noreply.github.com> Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> Co-authored-by: Ryan Clark <ryan.clark@goteleport.com> Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com> Co-authored-by: Bernard Kim <bernard@goteleport.com> Co-authored-by: hugoShaka <hugo.hervieux@goteleport.com> Co-authored-by: Vadym Popov <vadym.popov@goteleport.com> Co-authored-by: Erik Tate <erik.tate@goteleport.com> Co-authored-by: teleport-post-release-automation[bot] <128860004+teleport-post-release-automation[bot]@users.noreply.github.com> Co-authored-by: GitHub <noreply@github.com> Co-authored-by: Noah Stride <noah.stride@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> Co-authored-by: Gabriel Corado <gabriel.oliveira@goteleport.com> Co-authored-by: Forrest <30576607+fspmarshall@users.noreply.github.com>

* Add AutoUpdate Client/Cache implementation * CR changes * Add permission for proxy to access resources * Rename all occurrences auto update to camelcase * Remove auto update client wrapper * Drop AutoUpdateServiceClient helper Rename comments for consistency

vapopov added backport/branch/v14 no-changelog Indicates that a PR does not require a changelog entry backport/branch/v15 backport/branch/v16 labels Sep 17, 2024

vapopov requested review from sclevine, hugoShaka and rosstimothy September 17, 2024 02:53

github-actions bot requested a review from Joerger September 17, 2024 02:54

github-actions bot added the size/md label Sep 17, 2024

vapopov commented Sep 17, 2024

View reviewed changes

api/client/autoupdate/autoupdate.go Outdated Show resolved Hide resolved

rosstimothy reviewed Sep 17, 2024

View reviewed changes

lib/auth/authclient/api.go Outdated Show resolved Hide resolved

lib/cache/cache.go Outdated Show resolved Hide resolved

api/client/webclient/webclient.go Outdated Show resolved Hide resolved

lib/client/api.go Outdated Show resolved Hide resolved

vapopov added 3 commits September 17, 2024 14:00

Add AutoUpdate Client/Cache implementation

e6d56fc

CR changes

eead83c

Add permission for proxy to access resources

fed1fcd

vapopov force-pushed the vapopov/autoupdate-cache-and-client branch from fce0a1a to fed1fcd Compare September 17, 2024 18:41

rosstimothy reviewed Sep 17, 2024

View reviewed changes

vapopov added 2 commits September 17, 2024 20:30

Rename all occurrences auto update to camelcase

ef27d0d

Remove auto update client wrapper

98af039

vapopov force-pushed the vapopov/autoupdate-cache-and-client branch from e93c731 to 98af039 Compare September 18, 2024 01:13

rosstimothy reviewed Sep 18, 2024

View reviewed changes

api/client/client.go Outdated Show resolved Hide resolved

lib/cache/cache.go Outdated Show resolved Hide resolved

lib/cache/cache.go Outdated Show resolved Hide resolved

Drop AutoUpdateServiceClient helper

4f42c60

Rename comments for consistency

vapopov force-pushed the vapopov/autoupdate-cache-and-client branch from 91bb819 to 4f42c60 Compare September 18, 2024 15:28

rosstimothy approved these changes Sep 18, 2024

View reviewed changes

sclevine approved these changes Sep 18, 2024

View reviewed changes

public-teleport-github-review-bot bot removed request for hugoShaka and Joerger September 18, 2024 22:40

vapopov added this pull request to the merge queue Sep 18, 2024

github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Sep 19, 2024

vapopov added this pull request to the merge queue Sep 19, 2024

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 19, 2024

vapopov added this pull request to the merge queue Sep 19, 2024

Merged via the queue into master with commit f4fc673 Sep 19, 2024
40 checks passed

vapopov deleted the vapopov/autoupdate-cache-and-client branch September 19, 2024 03:02

This was referenced Sep 19, 2024

[v16] Add AutoUpdate Client/Cache implementation #46748

Merged

[v15] Add AutoUpdate Client/Cache implementation #46749

Merged

[v14] Add AutoUpdate Client/Cache implementation #46750

Merged

vapopov mentioned this pull request Sep 20, 2024

Fix interface check for clusterNameExecutor #46818

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add AutoUpdate Client/Cache implementation #46661

Add AutoUpdate Client/Cache implementation #46661

vapopov commented Sep 17, 2024

public-teleport-github-review-bot bot commented Sep 19, 2024

Add AutoUpdate Client/Cache implementation #46661

Add AutoUpdate Client/Cache implementation #46661

Conversation

vapopov commented Sep 17, 2024

public-teleport-github-review-bot bot commented Sep 19, 2024