Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add is-write-index flag to aliases #30942

Merged
merged 29 commits into from
Jun 15, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
5838717
add is-write-index flag to aliases
talevy May 29, 2018
7baf86b
cleanup
talevy May 31, 2018
19315ff
fix unit tests
talevy Jun 1, 2018
440fed4
update docs and tests
talevy Jun 1, 2018
135d7c1
change to validate at MetaData.build
talevy Jun 4, 2018
5bb7217
Merge branch 'master' into is-write-index-setting
talevy Jun 5, 2018
847bbe6
Merge branch 'master' into is-write-index-setting
talevy Jun 5, 2018
32ca015
add writeIndex to ToAndFromJsonMetaDataTests
talevy Jun 5, 2018
07188f3
Merge branch 'master' into is-write-index-setting
talevy Jun 12, 2018
01fab60
refactor as to not re-build aliasAndIndexLookup
talevy Jun 12, 2018
c7a3a96
migrate validation tests and setonce writeIndex
talevy Jun 12, 2018
8df9bc8
Merge branch 'master' into is-write-index-setting
talevy Jun 12, 2018
4e898c1
fix SharedClusterSnapshotRestoreIT#testRenameOnRestore
talevy Jun 12, 2018
1aac672
Merge branch 'master' into is-write-index-setting
talevy Jun 12, 2018
5ea9725
fix rest spec tests
talevy Jun 12, 2018
1c9e28f
change validation strategy and no defaulting to indexmetadata
talevy Jun 12, 2018
4755a10
update to reflect no defaulting values
talevy Jun 12, 2018
ae2b0c9
fix unit tests
talevy Jun 12, 2018
545c801
fix doc tests
talevy Jun 12, 2018
a614010
add version guard for 7.0 in IndicesAliasesRequest
talevy Jun 13, 2018
ee4142d
Merge branch 'master' into is-write-index-setting
talevy Jun 13, 2018
8c33d23
Merge branch 'master' into is-write-index-setting
talevy Jun 13, 2018
0025a51
further cleanup
talevy Jun 13, 2018
373c1e5
add randomization to AliasMetaDataTests.assertEqualInstances
talevy Jun 13, 2018
ba1162c
Merge branch 'master' into is-write-index-setting
talevy Jun 13, 2018
a1391b2
split out is_write_index check in yaml test and skip if run before 7.0
talevy Jun 13, 2018
91deba4
add test for write index swapping
talevy Jun 13, 2018
1629aef
Merge branch 'master' into is-write-index-setting
talevy Jun 14, 2018
f8dba0c
remove unused imports
talevy Jun 14, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions docs/reference/indices/aliases.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,94 @@ GET /alias2/_search?q=user:kimchy&routing=2,3
// CONSOLE
// TEST[continued]

[float]
[[aliases-write-index]]
==== Write Index

It is possible to associate the index pointed to by an alias as the write index.
When specified, all index and update requests against an alias that point to multiple

This comment was marked as resolved.

indices will attempt to resolve to the one index that is the write index.
Only one index per alias can be assigned to be the write index at a time. If no write index is specified
and there are multiple indices referenced by an alias, then writes will not be allowed.

It is possible to specify an index associated with an alias as a write index using both the aliases API
and index creation API.

[source,js]
--------------------------------------------------
POST /_aliases
{
"actions" : [
{
"add" : {
"index" : "test",
"alias" : "alias1",
"is_write_index" : true
}
}
]
}
--------------------------------------------------
// CONSOLE
// TEST[s/^/PUT test\n/]

In this example, we associate the alias `alias1` to both `test` and `test2`, where
`test` will be the index chosen for writing to.

[source,js]
--------------------------------------------------
PUT /alias1/_doc/1
{
"foo": "bar"
}
--------------------------------------------------
// CONSOLE
// TEST[continued]

The new document that was indexed to `/alias1/_doc/1` will be indexed as if it were
`/test/_doc/1`.

[source,js]
--------------------------------------------------
GET /test/_doc/1
--------------------------------------------------
// CONSOLE
// TEST[continued]

To swap which index is the write index for an alias, the Aliases API can be leveraged to
do an atomic swap. The swap is not dependent on the ordering of the actions.

[source,js]
--------------------------------------------------
POST /_aliases
{
"actions" : [
{
"add" : {
"index" : "test",
"alias" : "alias1",
"is_write_index" : true
}
}, {
"add" : {
"index" : "test2",
"alias" : "alias1",
"is_write_index" : false
}
}
]
}
--------------------------------------------------
// CONSOLE
// TEST[s/^/PUT test\nPUT test2\n/]

[IMPORTANT]
=====================================
Aliases that do not explicitly set `is_write_index: true` for an index, and
only reference one index, will have that referenced index behave as if it is the write index
until an additional index is referenced. At that point, there will be no write index and
writes will be rejected.
=====================================

[float]
[[alias-adding]]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,14 +79,37 @@
indices.get_alias:
index: test_index

- match: {test_index.aliases.test_alias: {}}
- match: {test_index.aliases.test_blias.search_routing: b}
- match: {test_index.aliases.test_blias.index_routing: b}
- is_false: test_index.aliases.test_blias.filter
- match: {test_index.aliases.test_clias.filter.term.field: value}
- is_false: test_index.aliases.test_clias.index_routing
- is_false: test_index.aliases.test_clias.search_routing

---
"Create index with write aliases":
- skip:
version: " - 6.99.99"
reason: is_write_index is not implemented in ES <= 6.x
- do:
indices.create:
index: test_index
body:
aliases:
test_alias: {}
test_blias:
is_write_index: false
test_clias:
is_write_index: true

- do:
indices.get_alias:
index: test_index

- is_false: test_index.aliases.test_alias.is_write_index
- is_false: test_index.aliases.test_blias.is_write_index
- is_true: test_index.aliases.test_clias.is_write_index

---
"Create index with no type mappings":
- do:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
package org.elasticsearch.action.admin.indices.alias;

import org.elasticsearch.ElasticsearchGenerationException;
import org.elasticsearch.Version;
import org.elasticsearch.common.Nullable;
import org.elasticsearch.common.ParseField;
import org.elasticsearch.common.Strings;
Expand Down Expand Up @@ -49,6 +50,7 @@ public class Alias implements Streamable, ToXContentFragment {
private static final ParseField ROUTING = new ParseField("routing");
private static final ParseField INDEX_ROUTING = new ParseField("index_routing", "indexRouting", "index-routing");
private static final ParseField SEARCH_ROUTING = new ParseField("search_routing", "searchRouting", "search-routing");
private static final ParseField IS_WRITE_INDEX = new ParseField("is_write_index");

private String name;

Expand All @@ -61,6 +63,9 @@ public class Alias implements Streamable, ToXContentFragment {
@Nullable
private String searchRouting;

@Nullable
private Boolean writeIndex;

private Alias() {

}
Expand Down Expand Up @@ -167,6 +172,21 @@ public Alias searchRouting(String searchRouting) {
return this;
}

/**
* @return the write index flag for the alias
*/
public Boolean writeIndex() {
return writeIndex;
}

/**
* Sets whether an alias is pointing to a write-index
*/
public Alias writeIndex(@Nullable Boolean writeIndex) {
this.writeIndex = writeIndex;
return this;
}

/**
* Allows to read an alias from the provided input stream
*/
Expand All @@ -182,6 +202,11 @@ public void readFrom(StreamInput in) throws IOException {
filter = in.readOptionalString();
indexRouting = in.readOptionalString();
searchRouting = in.readOptionalString();
if (in.getVersion().onOrAfter(Version.V_7_0_0_alpha1)) {
writeIndex = in.readOptionalBoolean();
} else {
writeIndex = null;
}
}

@Override
Expand All @@ -190,6 +215,9 @@ public void writeTo(StreamOutput out) throws IOException {
out.writeOptionalString(filter);
out.writeOptionalString(indexRouting);
out.writeOptionalString(searchRouting);
if (out.getVersion().onOrAfter(Version.V_7_0_0_alpha1)) {
out.writeOptionalBoolean(writeIndex);
}
}

/**
Expand Down Expand Up @@ -219,6 +247,10 @@ public static Alias fromXContent(XContentParser parser) throws IOException {
} else if (SEARCH_ROUTING.match(currentFieldName, parser.getDeprecationHandler())) {
alias.searchRouting(parser.text());
}
} else if (token == XContentParser.Token.VALUE_BOOLEAN) {
if (IS_WRITE_INDEX.match(currentFieldName, parser.getDeprecationHandler())) {
alias.writeIndex(parser.booleanValue());
}
}
}
return alias;
Expand All @@ -245,6 +277,8 @@ public XContentBuilder toXContent(XContentBuilder builder, Params params) throws
}
}

builder.field(IS_WRITE_INDEX.getPreferredName(), writeIndex);

builder.endObject();
return builder;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
package org.elasticsearch.action.admin.indices.alias;

import org.elasticsearch.ElasticsearchGenerationException;
import org.elasticsearch.Version;
import org.elasticsearch.action.ActionRequestValidationException;
import org.elasticsearch.action.AliasesRequest;
import org.elasticsearch.action.support.IndicesOptions;
Expand Down Expand Up @@ -84,6 +85,7 @@ public static class AliasActions implements AliasesRequest, Writeable, ToXConten
private static final ParseField ROUTING = new ParseField("routing");
private static final ParseField INDEX_ROUTING = new ParseField("index_routing", "indexRouting", "index-routing");
private static final ParseField SEARCH_ROUTING = new ParseField("search_routing", "searchRouting", "search-routing");
private static final ParseField IS_WRITE_INDEX = new ParseField("is_write_index");

private static final ParseField ADD = new ParseField("add");
private static final ParseField REMOVE = new ParseField("remove");
Expand Down Expand Up @@ -179,6 +181,7 @@ private static ObjectParser<AliasActions, Void> parser(String name, Supplier<Ali
ADD_PARSER.declareField(AliasActions::routing, XContentParser::text, ROUTING, ValueType.INT);
ADD_PARSER.declareField(AliasActions::indexRouting, XContentParser::text, INDEX_ROUTING, ValueType.INT);
ADD_PARSER.declareField(AliasActions::searchRouting, XContentParser::text, SEARCH_ROUTING, ValueType.INT);
ADD_PARSER.declareField(AliasActions::writeIndex, XContentParser::booleanValue, IS_WRITE_INDEX, ValueType.BOOLEAN);
}
private static final ObjectParser<AliasActions, Void> REMOVE_PARSER = parser(REMOVE.getPreferredName(), AliasActions::remove);
private static final ObjectParser<AliasActions, Void> REMOVE_INDEX_PARSER = parser(REMOVE_INDEX.getPreferredName(),
Expand Down Expand Up @@ -215,6 +218,7 @@ private static ObjectParser<AliasActions, Void> parser(String name, Supplier<Ali
private String routing;
private String indexRouting;
private String searchRouting;
private Boolean writeIndex;

public AliasActions(AliasActions.Type type) {
this.type = type;
Expand All @@ -231,6 +235,9 @@ public AliasActions(StreamInput in) throws IOException {
routing = in.readOptionalString();
searchRouting = in.readOptionalString();
indexRouting = in.readOptionalString();
if (in.getVersion().onOrAfter(Version.V_7_0_0_alpha1)) {
writeIndex = in.readOptionalBoolean();
}
}

@Override
Expand All @@ -242,6 +249,9 @@ public void writeTo(StreamOutput out) throws IOException {
out.writeOptionalString(routing);
out.writeOptionalString(searchRouting);
out.writeOptionalString(indexRouting);
if (out.getVersion().onOrAfter(Version.V_7_0_0_alpha1)) {
out.writeOptionalBoolean(writeIndex);
}
}

/**
Expand Down Expand Up @@ -401,6 +411,18 @@ public AliasActions filter(QueryBuilder filter) {
}
}

public AliasActions writeIndex(Boolean writeIndex) {
if (type != AliasActions.Type.ADD) {
throw new IllegalArgumentException("[is_write_index] is unsupported for [" + type + "]");
}
this.writeIndex = writeIndex;
return this;
}

public Boolean writeIndex() {
return writeIndex;
}

@Override
public String[] aliases() {
return aliases;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,18 @@ public IndicesAliasesRequestBuilder addAlias(String index, String alias, QueryBu
return this;
}

/**
* Adds an alias to the index.
*
* @param index The index
* @param alias The alias
* @param writeIndex write index flag
*/
public IndicesAliasesRequestBuilder addAlias(String index, String alias, boolean writeIndex) {
request.addAliasAction(AliasActions.add().index(index).alias(alias).writeIndex(writeIndex));
return this;
}

/**
* Removes an alias from the index.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,8 @@ protected void masterOperation(final IndicesAliasesRequest request, final Cluste
switch (action.actionType()) {
case ADD:
for (String alias : concreteAliases(action, state.metaData(), index)) {
finalActions.add(new AliasAction.Add(index, alias, action.filter(), action.indexRouting(), action.searchRouting()));
finalActions.add(new AliasAction.Add(index, alias, action.filter(), action.indexRouting(),
action.searchRouting(), action.writeIndex()));
}
break;
case REMOVE:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ public void onFailure(Exception e) {
static IndicesAliasesClusterStateUpdateRequest prepareRolloverAliasesUpdateRequest(String oldIndex, String newIndex,
RolloverRequest request) {
List<AliasAction> actions = unmodifiableList(Arrays.asList(
new AliasAction.Add(newIndex, request.getAlias(), null, null, null),
new AliasAction.Add(newIndex, request.getAlias(), null, null, null, null),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can do this in the follow up but we shouldn't forget to carry over the index write flag of the alias.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, that is a whole nother thing for Rollover to leverage this feature

new AliasAction.Remove(oldIndex, request.getAlias())));
final IndicesAliasesClusterStateUpdateRequest updateRequest = new IndicesAliasesClusterStateUpdateRequest(actions)
.ackTimeout(request.ackTimeout())
Expand Down
Loading