Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Secrets-Sync/vault-18673-documentation #23189

Merged
merged 39 commits into from
Sep 27, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
f113e17
* secrets sync doc initial setup
maxcoulombe Sep 19, 2023
f0e91be
Add VSCode debug bin to gitignore
robmonte Sep 20, 2023
03895e4
+ added missing endpoint doc
maxcoulombe Sep 20, 2023
43ab088
Merge remote-tracking branch 'origin/main' into sync/vault-18673-docu…
maxcoulombe Sep 20, 2023
bcafe21
Begin writing doc info
robmonte Sep 20, 2023
ea9a959
More docs
robmonte Sep 21, 2023
1417bfe
Merge branch 'sync/vault-18673-documentation' of ssh://github.com/has…
robmonte Sep 21, 2023
185d4ec
* added GCP destination doc
maxcoulombe Sep 22, 2023
2303fa5
* enhanced the TF instructions a bit
maxcoulombe Sep 22, 2023
e39dffb
Merge remote-tracking branch 'origin/sync/vault-18673-documentation' …
maxcoulombe Sep 22, 2023
12f9343
* sections as sentence case
maxcoulombe Sep 22, 2023
d184160
* typo
maxcoulombe Sep 22, 2023
179ad02
More docs
robmonte Sep 25, 2023
cc35a18
Moving AWS and Index pages into separate PR
robmonte Sep 25, 2023
b7294f4
Update website/content/docs/sync/gcpsm.mdx
maxcoulombe Sep 25, 2023
4cc3e9d
Secrets Sync Docs - format clean up suggestions (#23267)
yhyakuna Sep 25, 2023
0c812cd
add secret sync for github docs (#23266)
raymonstah Sep 26, 2023
4516324
add secret sync for azure kv docs (#23255)
raymonstah Sep 26, 2023
d6fb287
+ added Vercel doc (#23261)
maxcoulombe Sep 26, 2023
41b4ca2
Add AWS and Index pages (#23276)
robmonte Sep 26, 2023
f91253f
Merge branch 'main' into sync/vault-18673-documentation
maxcoulombe Sep 26, 2023
65d6574
Update website/content/api-docs/system/secrets-sync.mdx
maxcoulombe Sep 26, 2023
4293919
Format & style updates (#23289)
yhyakuna Sep 26, 2023
19723af
* adjustments
maxcoulombe Sep 26, 2023
c6e9020
Merge remote-tracking branch 'origin/sync/vault-18673-documentation' …
maxcoulombe Sep 26, 2023
a821c7d
Sync docs formatting consistency (#23291)
robmonte Sep 26, 2023
81aaab8
* tidying & normalisation
maxcoulombe Sep 26, 2023
3e0ab1c
Merge remote-tracking branch 'origin/sync/vault-18673-documentation' …
maxcoulombe Sep 26, 2023
e71694e
* fix merge
maxcoulombe Sep 26, 2023
b350b52
Fix quotations on environment variable names and fix sentence to corr…
robmonte Sep 26, 2023
bec112d
* small typos
maxcoulombe Sep 26, 2023
250ed35
Merge remote-tracking branch 'origin/sync/vault-18673-documentation' …
maxcoulombe Sep 26, 2023
c58c2e2
Merge branch 'main' into sync/vault-18673-documentation
maxcoulombe Sep 26, 2023
79a777a
+ add enterprise alert
maxcoulombe Sep 26, 2023
0ed80aa
Minor fixes (#23307)
yhyakuna Sep 27, 2023
4de7b5c
Fix format language tags
robmonte Sep 27, 2023
04859f2
Fix more things dropped from the earlier merge conflict
robmonte Sep 27, 2023
819a802
Update association description
robmonte Sep 27, 2023
16fead6
Remove plural when only 1 step preceeds
robmonte Sep 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add AWS and Index pages (#23276)
* Add AWS and Index pages

* Fix resource constraint

* Typo suggestions

Co-authored-by: Max Coulombe <109547106+maxcoulombe@users.noreply.github.com>

* Merge clean up with additions

---------

Co-authored-by: Max Coulombe <109547106+maxcoulombe@users.noreply.github.com>
  • Loading branch information
robmonte and maxcoulombe authored Sep 26, 2023
commit 41b4ca2930eba4e999c9b28be20874fd4da57ceb
107 changes: 78 additions & 29 deletions website/content/docs/sync/awssm.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@ description: The AWS Secrets Manager destination syncs secrets from Vault to AWS

The AWS Secrets Manager destination enables Vault to sync and unsync secrets of your choosing into
an external AWS account. When configured, Vault will actively maintain the state of each externally-synced
secret in realtime. This includes sending new secrets, updating existing secret values, and removing
secrets when they either get dissocaited from the destination or deleted from Vault.
secret in near-realtime. This includes sending new secrets, updating existing secret values, and removing
secrets when they either get dissociated from the destination or deleted from Vault. This enables the
ability to keep control of all your secrets localized while leveraging the benefits of the AWS Secrets Manager.

## Setup

Expand All @@ -24,41 +25,54 @@ The following is an example policy outlining the required permissions to use sec
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"secretsmanager:Create*",
"secretsmanager:Put*",
"secretsmanager:Update*",
"secretsmanager:Delete*",
"secretsmanager:TagResource",
"secretsmanager:UntagResource"
],
"Resource": "*"
}
{
"Effect": "Allow",
"Action": [
"secretsmanager:Create*",
"secretsmanager:Update*",
"secretsmanager:Delete*",
"secretsmanager:TagResource"
],
"Resource": "arn:aws:secretsmanager:*:*:secret:vault*"
}
]
}
```

## Usage

When AWS credentials have been obtained, Vault can be configured to create an AWS destination, then
begin to associate your secrets with this destination.
Once AWS credentials have been obtained, configure Vault with them to create an AWS destination. If
you do not already have a KVv2 ready to use, mount a new KVv2 secrets engine and create a secret.

1. Store any KV secret into Vault, like so:
1. Mount the secret engine:

```shell-session
$ vault kv put secret/foo mypass=bar
$ vault secrets enable -path=my-kv kv-v2
```

**Output:**

<CodeBlockConfig hideClipboard>

```text
Success! Enabled the kv-v2 secrets engine at: my-kv/
```
=== Secret Path ===
secret/data/foo

</CodeBlockConfig>

1. Create a KV secret:

```shell-session
$ vault kv put -mount=my-kv my-secret foo=bar
```

**Output:**

<CodeBlockConfig hideClipboard>

```text
==== Secret Path ====
my-kv/data/my-secret
======= Metadata =======
Key Value
--- -----
Expand All @@ -68,7 +82,7 @@ begin to associate your secrets with this destination.
destroyed false
version 1
```

</CodeBlockConfig>

1. Configure the sync destination:
Expand All @@ -83,7 +97,7 @@ begin to associate your secrets with this destination.
**Output:**

<CodeBlockConfig hideClipboard>

```text
Key Value
--- -----
Expand All @@ -94,12 +108,12 @@ begin to associate your secrets with this destination.

</CodeBlockConfig>

1. Set a secret to be associated with the configured secret destination:
1. Set a secret to be associated with this AWS SM destination:

```shell-session
$ vault write sys/sync/destinations/aws-sm/my-awssm-1/associations/set \
mount='secret' \
secret_name='foo'
mount='my-kv' \
secret_name='my-secret'
```

**Output:**
Expand All @@ -109,7 +123,35 @@ begin to associate your secrets with this destination.
```
Key Value
--- -----
associated_secrets map[kv_37993f8a/foo:map[accessor:kv_37993f8a secret_name:foo sync_status:SYNCED updated_at:2023-09-19T13:17:35.085581-05:00]]
associated_secrets map[kv_37993f8a/my-secret:map[accessor:kv_37993f8a secret_name:my-secret sync_status:SYNCED updated_at:2023-09-19T13:17:35.085581-05:00]]
store_name aws1
store_type aws-sm
```

</CodeBlockConfig>

1. Navigate to the [Secrets Manager](https://console.aws.amazon.com/secretsmanager/) in the AWS console
to confirm your secret was successfully synced.

Moving forward, any modification on the Vault secret will be propagated to its AWS Secrets Manager
counterpart. Creating a new secret version in Vault will update the one in AWS to the new version. Deleting either
the secret or the association in Vault will delete the secret in your AWS account as well.

1. Delete the secret association:
```shell-session
$ vault write sys/sync/destinations/aws-sm/my-awssm-1/associations/remove \
mount='my-kv' \
secret_name='my-secret'
```

**Output:**

<CodeBlockConfig hideClipboard>

```text
Key Value
--- -----
associated_secrets map[]
store_name aws1
store_type aws-sm
```
Expand All @@ -118,13 +160,20 @@ begin to associate your secrets with this destination.

## Troubleshooting

...
### Synced secrets not appearing in AWS Secrets Manager

Ensure that you are looking at the correct region in AWS, especially if using the web console.
The region you synced your secrets to must match the region you are trying to read them from.

### Error: unauthorized request: UnrecognizedClientException: The security token included in the request is invalid.

Ensure you are not using short-lived credentials with secrets syncing. By nature of the feature,
the credentials used by Vault to keep secrets synced must be long-lived.

## Tutorial

Refer to the [Vault Enterprise Secrets Sync](/vault/tutorials/enterprise/secrets-sync)
tutorial to learn how to configure the secrets sync between Vault and AWS Secrets Manager.
Refer to the [Vault Enterprise Secrets Sync tutorial](/vault/tutorials/enterprise/secrets-sync)
to learn how to configure the secrets sync between Vault and AWS Secrets Manager.

## API

Expand Down
55 changes: 55 additions & 0 deletions website/content/docs/sync/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
---
layout: docs
page_title: Secrets Sync
description: Secrets sync allows you to safely sync Vault-managed secrets with external destinations.
---

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add an Enterprise alert here. It will automatically render with the Vault logo.

I think it's also common on the top of the API docs page for ent features.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup done!

# Secrets Sync

Secrets sync enables users to manage their secrets with Vault while syncing a subset of them into
external secret store destinations. Secrets that will be synced must be individually associated
with each desired destination. This means that associating a secret named `my-secret` with a
destinated named `my-aws` will not automatically sync to second destination named `my-gcp`.

<Note>
Unsyncing a secret either by deleting the secret from Vault or from deleting its association
is a hard-delete option. This means it is immediately removed from the external system and bypasses
any soft-deletion states or recovery mechanisms.
</Note>

## Destinations

Secrets can be synced into various external stores, called destinations. Destinations are configured
separately for each type of external system. The supported destinations are listed:
* [AWS Secrets Manager](/vault/docs/sync/awssm)
* [Azure Key Vault](/vault/docs/sync/azurekv)
* [GCP Secret Manager](/vault/docs/sync/gcpsm)
* [GitHub Repository Actions](/vault/docs/sync/github)
* [Vercel Projects](/vault/docs/sync/vercelproject)

## Assocations

Choosing the secrets you want synced to a given destination is done by creating links between them,
called associations. Once a destination has been successfully configured, you can begin making the
associations to it by providing the secret's name and its mount. Each secret's association keeps
track of its current sync status, the timestamp of its last status change, and the error code of
the last sync or unsync operation, if it failed.


## Sync Statuses

There are several sync statuses which relay information about the outcome of the last sync
operation to have occurred on that secret. The status information is stored inside each
association object returned by the endpoint and, upon failure, includes an error code explaining why.

* `SYNCING` - The sync operation for the secret is currently in-progress.
* `SYNCED` - The sync operation was successful and sent the secret to the external destination.
* `UNSYNCING` - The unsync operation for the secret is currently in-progress.
* `UNSYNCED` - The unsync operation was successful and removed the secret from the external destination.
This is returned when a secret itself is deleted but the association is not.
* `INTERNAL_VAULT_ERROR` - The sync or unsync operation failed due to an issue internal to Vault.
* `CLIENT_SIDE_ERROR` - The sync or unsync operation failed due to a configuration error.
For example, the credentials used to configure the destination expired or were revoked.
* `EXTERNAL_SERVICE_ERROR` - The sync or unsync operation failed due to an issue with the
external service provider's side.
* `UNKNOWN` - Vault is unable to determine the current state of the secret in regards to the external service.