-
Notifications
You must be signed in to change notification settings - Fork 91
[D&M] Remote clusters cleanup #443
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 13 commits
51e3da7
25bceed
41693ad
567fab3
297a4fc
6757e61
12c0e4c
9c78a70
921d25b
d17aa90
bb2b4ff
0cfb20a
05bc37c
a982cdf
3176a62
cb8ec6f
da366d6
913b811
e8377d1
4f59eb7
a8c53d3
c84e9e1
1bf69c9
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,9 +1,49 @@ | ||
# Remote clusters | ||
--- | ||
applies_to: | ||
stack: ga | ||
deployment: | ||
shainaraskas marked this conversation as resolved.
Show resolved
Hide resolved
|
||
ece: ga | ||
eck: ga | ||
ess: ga | ||
self: ga | ||
serverless: unavailable | ||
--- | ||
|
||
# Remote clusters [remote-clusters] | ||
|
||
% What needs to be done: Write from scratch | ||
|
||
% GitHub issue: https://github.com/elastic/docs-projects/issues/345 | ||
|
||
% Scope notes: "Landing page for cross cluster comms, used by CCS and CCR. | ||
We will cover here the raw configuration at Elasticsearch level and the docs to enable remote clusters in ESS / ECE / ECK. | ||
We can include links to the use cases of remote clusters, such as CCR and CCS." | ||
% We will cover here the raw configuration at Elasticsearch level and the docs to enable remote clusters in ESS / ECE / ECK. | ||
% We can include links to the use cases of remote clusters, such as CCR and CCS." | ||
|
||
By setting up **remote clusters**, you can connect an {{es}} cluster to other {{es}} clusters. Remote clusters can be located in different data centers, geographic regions, and run on a different type of environment: {{ech}}, {{ece}}, {{eck}}, or self-managed. | ||
|
||
Remote clusters are especially useful in two cases: | ||
|
||
- **Cross-cluster replication** | ||
With [cross-cluster replication](/deploy-manage/tools/cross-cluster-replication.md), or CCR, you ingest data to an index on a remote cluster. This leader index is replicated to one or more read-only follower indices on your local cluster. Creating a multi-cluster architecture with cross-cluster replication enables you to configure disaster recovery, bring data closer to your users, or establish a centralized reporting cluster to process reports locally. | ||
|
||
- **Cross-cluster search** | ||
[Cross-cluster search](/solutions/search/cross-cluster-search.md), or CCS, enables you to run a search request against one or more remote clusters. This capability provides each region with a global view of all clusters, allowing you to send a search request from a local cluster and return results from all connected remote clusters. For full {{ccs}} capabilities, the local and remote cluster must be on the same [subscription level](https://www.elastic.co/subscriptions). | ||
|
||
::::{note} about terminology | ||
In the case of remote clusters, the {{es}} cluster or deployment initiating the connection and requests is often referred to as the **local cluster**, while the {{es}} cluster or deployment receiving the requests is referred to as the **remote cluster**. | ||
:::: | ||
|
||
## Setup | ||
|
||
Depending on the environment the local and remote clusters are deployed on and the security model you wish to use, the exact details needed to add a remote cluster vary but generally follow the same path: | ||
|
||
1. **Configure trust between clusters.** In the settings of the local deployment or cluster, configure the trust security model that your remote connections will use to access the remote cluster. This step involves specifying API keys or certificates retrieved from the remote clusters. | ||
|
||
2. **Establish the connection.** In {{kib}} on the local cluster, finalize the connection by specifying each remote cluster's details. | ||
|
||
Find the instructions with details on the supported security models and available connection modes for your specific scenario: | ||
|
||
- [Remote clusters with {{ech}}](remote-clusters/ec-enable-ccs.md) | ||
- [Remote clusters with {{ece}}](remote-clusters/ece-enable-ccs.md) | ||
- [Remote clusters with {{eck}}](remote-clusters/eck-remote-clusters.md) | ||
- [Remote clusters with self-managed installations](remote-clusters/remote-clusters-self-managed.md) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
:::::{dropdown} Version compatibility table | ||
|
||
* Any node can communicate with another node on the same major version. For example, 9.0 can talk to any 9.x node. | ||
* Version compatibility is symmetric, meaning that if 7.16 can communicate with 8.0, 8.0 can also communicate with 7.16. The following table depicts version compatibility between local and remote nodes. | ||
|
||
| | | | ||
| --- | --- | | ||
| | Local cluster | | ||
| Remote cluster | 5.0–5.5 | 5.6 | 6.0–6.6 | 6.7 | 6.8 | 7.0 | 7.1–7.16 | 7.17 | 8.0–9.0 | | ||
| 5.0–5.5 |  |  |  |  |  |  |  |  |  | | ||
| 5.6 |  |  |  |  |  |  |  |  |  | | ||
| 6.0–6.6 |  |  |  |  |  |  |  |  |  | | ||
| 6.7 |  |  |  |  |  |  |  |  |  | | ||
| 6.8 |  |  |  |  |  |  |  |  |  | | ||
| 7.0 |  |  |  |  |  |  |  |  |  | | ||
| 7.1–7.16 |  |  |  |  |  |  |  |  |  | | ||
| 7.17 |  |  |  |  |  |  |  |  |  | | ||
| 8.0–9.0 |  |  |  |  |  |  |  |  |  | | ||
|
||
|
||
::::{important} | ||
Elastic only supports {{ccs}} on a subset of these configurations. See [Supported {{ccs}} configurations](../../../solutions/search/cross-cluster-search.md#ccs-supported-configurations). | ||
:::: | ||
|
||
::::: | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,19 +1,24 @@ | ||
--- | ||
applies_to: | ||
deployment: | ||
ess: ga | ||
eck: ga | ||
navigation_title: With {{eck}} | ||
mapped_pages: | ||
- https://www.elastic.co/guide/en/cloud/current/ec-enable-ccs-for-eck.html | ||
--- | ||
|
||
# Enabling CCS/R between Elasticsearch Service and ECK [ec-enable-ccs-for-eck] | ||
# Remote clusters between {{ech}} and ECK [ec-enable-ccs-for-eck] | ||
florent-leborgne marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
These steps describe how to configure remote clusters between an {{es}} cluster in Elasticsearch Service and an {{es}} cluster running within [Elastic Cloud on Kubernetes (ECK)](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-overview.html). Once that’s done, you’ll be able to [run CCS queries from {{es}}](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-cross-cluster-search.html) or [set up CCR](https://www.elastic.co/guide/en/elasticsearch/reference/current/ccr-getting-started-tutorial.html). | ||
These steps describe how to configure remote clusters between an {{es}} cluster in {{ech}} and an {{es}} cluster running within [{{eck}} (ECK)](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-overview.html). Once that’s done, you’ll be able to [run CCS queries from {{es}}](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-cross-cluster-search.html) or [set up CCR](https://www.elastic.co/guide/en/elasticsearch/reference/current/ccr-getting-started-tutorial.html). | ||
|
||
|
||
## Establish trust between two clusters [ec_establish_trust_between_two_clusters] | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. line 81: confused about what the user is supposed to do on the target page. the establishing trust procedures above also refer to these pages. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If that's what you're referring to: Those page used to be maintained by the ECK team. Only thing I know is that doing it on ECK is very close to how it's done with self-managed, which isn't very concrete information... We can make a note to improve this later. |
||
|
||
The first step is to establish trust between the two clusters. | ||
|
||
|
||
### Establish trust in the Elasticsearch Service cluster [ec_establish_trust_in_the_elasticsearch_service_cluster] | ||
### Establish trust in the {{ech}} cluster [ec_establish_trust_in_the_elasticsearch_service_cluster] | ||
|
||
1. Save the ECK CA certificate to a file. For a cluster named `quickstart`, run: | ||
|
||
|
@@ -22,7 +27,7 @@ The first step is to establish trust between the two clusters. | |
``` | ||
|
||
|
||
1. Update the trust settings for the Elasticsearch Service deployment. Follow the steps provided in [Access clusters of a self-managed environment](ec-remote-cluster-self-managed.md), and specifically the first three steps in **Specify the deployments trusted to be used as remote clusters** using TLS certificate as security model. | ||
1. Update the trust settings for the {{ech}} deployment. Follow the steps provided in [Access clusters of a self-managed environment](ec-remote-cluster-self-managed.md), and specifically the first three steps in **Specify the deployments trusted to be used as remote clusters** using TLS certificate as security model. | ||
florent-leborgne marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
* Use the certificate file saved in the first step. | ||
* Select the {{ecloud}} pattern and enter `default.es.local` for the `Scope ID`. | ||
|
@@ -32,7 +37,7 @@ The first step is to establish trust between the two clusters. | |
|
||
### Establish trust in the ECK cluster [ec_establish_trust_in_the_eck_cluster] | ||
|
||
1. Upload the Elasticsearch Service certificate (that you downloaded in the last step of the previous section) as a Kubernetes secret. | ||
1. Upload the {{ech}} certificate (that you downloaded in the last step of the previous section) as a Kubernetes secret. | ||
|
||
```sh | ||
kubectl create secret generic ce-aws-cert --from-file=<path to certificate file> | ||
|
@@ -75,14 +80,14 @@ The first step is to establish trust between the two clusters. | |
|
||
## Setup CCS/R [ec_setup_ccsr] | ||
|
||
Now that trust has been established, you can set up CCS/R from the ECK cluster to the Elasticsearch Service cluster or from the Elasticsearch Service cluster to the ECK cluster. | ||
Now that trust has been established, you can set up CCS/R from the ECK cluster to the {{ech}} cluster or from the {{ech}} cluster to the ECK cluster. | ||
|
||
|
||
### ECK Cluster to Elasticsearch Service cluster [ec_eck_cluster_to_elasticsearch_service_cluster] | ||
### ECK Cluster to {{ech}} cluster [ec_eck_cluster_to_elasticsearch_service_cluster] | ||
|
||
Configure the ECK cluster [using certificate based authentication](ec-remote-cluster-self-managed.md). | ||
|
||
|
||
### Elasticsearch Service cluster to ECK Cluster [ec_elasticsearch_service_cluster_to_eck_cluster] | ||
### {{ech}} cluster to ECK Cluster [ec_elasticsearch_service_cluster_to_eck_cluster] | ||
|
||
Follow the steps outlined in the [ECK documentation](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-remote-clusters.html#k8s_configure_the_remote_cluster_connection_through_the_elasticsearch_rest_api). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a perfect page????