You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/developer/advanced/development-es-snapshots.asciidoc
+22-22Lines changed: 22 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,32 +1,32 @@
1
1
[[development-es-snapshots]]
2
-
===Daily Elasticsearch Snapshots
2
+
== Daily {es} Snapshots
3
3
4
-
For local development and CI, {kib}, by default, uses Elasticsearch snapshots that are built daily when running tasks that require Elasticsearch (e.g. functional tests).
4
+
For local development and CI, {kib}, by default, uses {es} snapshots that are built daily when running tasks that require {es} (e.g. functional tests).
5
5
6
-
A snapshot is just a group of tarballs, one for each supported distribution/architecture/os of Elasticsearch, and a JSON-based manifest file containing metadata about the distributions.
6
+
A snapshot is just a group of tarballs, one for each supported distribution/architecture/os of {es}, and a JSON-based manifest file containing metadata about the distributions.
7
7
8
-
https://ci.kibana.dev/es-snapshots[A dashboard] is available that shows the current status and compatibility of the latest Elasticsearch snapshots.
8
+
https://ci.kibana.dev/es-snapshots[A dashboard] is available that shows the current status and compatibility of the latest {es} snapshots.
9
9
10
-
==== Process Overview
10
+
=== Process Overview
11
11
12
-
1. Elasticsearch snapshots are built for each current tracked branch of {kib}.
12
+
1. {es} snapshots are built for each current tracked branch of {kib}.
13
13
2. Each snapshot is uploaded to a public Google Cloud Storage bucket, `kibana-ci-es-snapshots-daily`.
14
14
** At this point, the snapshot is not automatically used in CI or local development. It needs to be tested/verified first.
15
15
3. Each snapshot is tested with the latest commit of the corresponding {kib} branch, using the full CI suite.
16
16
4. After CI
17
17
** If the snapshot passes, it is promoted and automatically used in CI and local development.
18
-
** If the snapshot fails, the issue must be investigated and resolved. A new incompatibility may exist between Elasticsearch and {kib}.
18
+
** If the snapshot fails, the issue must be investigated and resolved. A new incompatibility may exist between {es} and {kib}.
19
19
20
-
==== Using the latest snapshot
20
+
=== Using the latest snapshot
21
21
22
-
When developing locally, you may wish to use the most recent Elasticsearch snapshot, even if it's failing CI. To do so, prefix your commands with the follow environment variable:
22
+
When developing locally, you may wish to use the most recent {es} snapshot, even if it's failing CI. To do so, prefix your commands with the follow environment variable:
23
23
24
24
["source","bash"]
25
25
-----------
26
26
KBN_ES_SNAPSHOT_USE_UNVERIFIED=true
27
27
-----------
28
28
29
-
You can use this flag with any command that downloads and runs Elasticsearch snapshots, such as `scripts/es` or the FTR.
29
+
You can use this flag with any command that downloads and runs {es} snapshots, such as `scripts/es` or the FTR.
30
30
31
31
For example, to run functional tests with the latest snapshot:
32
32
@@ -35,7 +35,7 @@ For example, to run functional tests with the latest snapshot:
Currently, there is not a way to run your pull request with the latest unverified snapshot without a code change. You can, however, do it with a small code change.
41
41
@@ -45,9 +45,9 @@ Currently, there is not a way to run your pull request with the latest unverifie
45
45
46
46
Your pull request should then use the latest snapshot the next time that it runs. Just don't merge the change to `Jenkinsfile`!
47
47
48
-
==== Google Cloud Storage buckets
48
+
=== Google Cloud Storage buckets
49
49
50
-
===== kibana-ci-es-snapshots-daily
50
+
==== kibana-ci-es-snapshots-daily
51
51
52
52
This bucket stores snapshots that are created on a daily basis, and is the primary location used by `kbn-es` to download snapshots.
53
53
@@ -61,7 +61,7 @@ The file structure for this bucket looks like this:
This bucket stores only the most recently promoted snapshot for each version. Old snapshots are only deleted when new ones are uploaded.
67
67
@@ -73,18 +73,18 @@ The file structure for this bucket looks like this:
73
73
* `<version>/*.tar.gz.sha512`
74
74
* `<version>/manifest.json`
75
75
76
-
==== How snapshots are built, tested, and promoted
76
+
=== How snapshots are built, tested, and promoted
77
77
78
-
Each day, a https://kibana-ci.elastic.co/job/elasticsearch+snapshots+trigger/[Jenkins job] runs that triggers Elasticsearch builds for each currently tracked branch/version. This job is automatically updated with the correct branches whenever we release new versions of {kib}.
78
+
Each day, a https://kibana-ci.elastic.co/job/elasticsearch+snapshots+trigger/[Jenkins job] runs that triggers {es} builds for each currently tracked branch/version. This job is automatically updated with the correct branches whenever we release new versions of {kib}.
79
79
80
-
===== Build
80
+
==== Build
81
81
82
-
https://kibana-ci.elastic.co/job/elasticsearch+snapshots+build/[This Jenkins job] builds the Elasticsearch snapshots and uploads them to GCS.
82
+
https://kibana-ci.elastic.co/job/elasticsearch+snapshots+build/[This Jenkins job] builds the {es} snapshots and uploads them to GCS.
83
83
84
84
The Jenkins job pipeline definition is https://github.com/elastic/kibana/blob/master/.ci/es-snapshots/Jenkinsfile_build_es[in the {kib} repo].
85
85
86
-
1. Checkout Elasticsearch repo for the given branch/version.
87
-
2. Run `./gradlew -p distribution/archives assemble --parallel` to create all of the Elasticsearch distributions.
86
+
1. Checkout {es} repo for the given branch/version.
87
+
2. Run `./gradlew -p distribution/archives assemble --parallel` to create all of the {es} distributions.
88
88
3. Create a tarball for each distribution.
89
89
4. Create a manifest JSON file containing info about the distribution, as well as its download URL.
90
90
5. Upload the tarballs and manifest to a unique location in the GCS bucket `kibana-ci-es-snapshots-daily`.
@@ -93,9 +93,9 @@ The Jenkins job pipeline definition is https://github.com/elastic/kibana/blob/ma
93
93
** This allows the `KBN_ES_SNAPSHOT_USE_UNVERIFIED` flag to work.
94
94
7. Trigger the verification job, to run the full {kib} CI test suite with this snapshot.
95
95
96
-
===== Verification and Promotion
96
+
==== Verification and Promotion
97
97
98
-
https://kibana-ci.elastic.co/job/elasticsearch+snapshots+verify/[This Jenkins job] tests the latest Elasticsearch snapshot with the full {kib} CI pipeline, and promotes if it there are no test failures.
98
+
https://kibana-ci.elastic.co/job/elasticsearch+snapshots+verify/[This Jenkins job] tests the latest {es} snapshot with the full {kib} CI pipeline, and promotes if it there are no test failures.
99
99
100
100
The Jenkins job pipeline definition is https://github.com/elastic/kibana/blob/master/.ci/es-snapshots/Jenkinsfile_verify_es[in the {kib} repo].
0 commit comments