Skip to content

Adding script support for version parts #62953

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Adding script support for version parts #62953

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions docs/reference/mapping/types/version.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,3 +68,62 @@ This field type isn't optimized for heavy wildcard, regex or fuzzy searches. Whi
type of queries work in this field, you should consider using a regular `keyword` field if
you strongly rely on these kind of queries.

==== Script support

The `version` fields offers some specialized access to detailed information derived from
valid version strings like the Major, Minor or Patch release number, whether the version value
is valid according to Semver or if it is a pre-release version. This can be helpful when e.g.
filtering for only released versions or running aggregations on parts of the version.
The following query, for example, filters for released versions and groups them by Major version
using a `terms` aggregation:

[source,console]
--------------------------------------------------
POST my-index-000001/_search
{
"query": {
"bool": {
"filter": [
{
"script": {
"script": {
"source": "doc['my_version'].isRelease() == true"
}
}
}
]
}
},
"aggs": {
"group_major": {
"terms": {
"script": { "source": "doc['my_version'].getMajor()"},
"order": {
"_key": "asc"
}
}
}
}
}

--------------------------------------------------
// TEST[continued]

Functions available on via doc values in scripting are:

[horizontal]

isValid()::
Returns `true` if the field contains a version thats legal according to the Semantic Versioning rules

isRelease()::
Returns `true` if the field contains a valid release version, `false` if it is a pre-release version or invalid.

getMajor()::
Returns an Integer value of the Major version if the version is valid, or null otherwise

getMinor()::
Returns an Integer value of the Minor version if the version is valid, or null otherwise.

getPatch()::
Returns an Integer value of the Patch version if the version is valid, or null otherwise.
60 changes: 60 additions & 0 deletions ...er-version/src/main/java/org/elasticsearch/xpack/versionfield/VersionScriptDocValues.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import org.apache.lucene.index.SortedSetDocValues;
import org.apache.lucene.util.ArrayUtil;
import org.elasticsearch.index.fielddata.ScriptDocValues;
import org.elasticsearch.xpack.versionfield.VersionEncoder.VersionParts;

import java.io.IOException;

Expand Down Expand Up @@ -55,4 +56,63 @@ public String get(int index) {
public int size() {
return count;
}

public boolean isValid() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VersionScriptDocValues is a list so I don't think this is the right level to apply the functions. We accept multiple values so there should be a way to apply these functions to the first, second or last entry in the list too ?
These functions could also be applied to any string (keyword family) field ? A static function likesemverIsRelease(String) that would work on the decoded version could also be helpful imo, we don't need to put these functions under the new script values.

return VersionEncoder.legalVersionString(VersionParts.ofVersion(getValue()));
}

public boolean isRelease() {
VersionParts parts = VersionParts.ofVersion(getValue());
return VersionEncoder.legalVersionString(parts) && parts.preRelease == null;
}

public Integer getMajor() {
VersionParts parts = VersionParts.ofVersion(getValue());
if (VersionEncoder.legalVersionString(parts) && parts.mainVersion != null) {
int firstDot = parts.mainVersion.indexOf(".");
if (firstDot > 0) {
return Integer.valueOf(parts.mainVersion.substring(0, firstDot));
} else {
return Integer.valueOf(parts.mainVersion);
}
}
return null;
}

public Integer getMinor() {
VersionParts parts = VersionParts.ofVersion(getValue());
Integer rc = null;
if (VersionEncoder.legalVersionString(parts) && parts.mainVersion != null) {
int firstDot = parts.mainVersion.indexOf(".");
if (firstDot > 0) {
int secondDot = parts.mainVersion.indexOf(".", firstDot + 1);
if (secondDot > 0) {
rc = Integer.valueOf(parts.mainVersion.substring(firstDot + 1, secondDot));
} else {
rc = Integer.valueOf(parts.mainVersion.substring(firstDot + 1));
}
}
}
return rc;
}

public Integer getPatch() {
VersionParts parts = VersionParts.ofVersion(getValue());
Integer rc = null;
if (VersionEncoder.legalVersionString(parts) && parts.mainVersion != null) {
int firstDot = parts.mainVersion.indexOf(".");
if (firstDot > 0) {
int secondDot = parts.mainVersion.indexOf(".", firstDot + 1);
if (secondDot > 0) {
int thirdDot = parts.mainVersion.indexOf(".", secondDot + 1);
if (thirdDot > 0) {
rc = Integer.valueOf(parts.mainVersion.substring(secondDot + 1, thirdDot));
} else {
rc = Integer.valueOf(parts.mainVersion.substring(secondDot + 1));
}
}
}
}
return rc;
}
}
5 changes: 5 additions & 0 deletions ...ugin/mapper-version/src/main/resources/org/elasticsearch/xpack/versionfield/whitelist.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,9 @@
class org.elasticsearch.xpack.versionfield.VersionScriptDocValues {
String get(int)
String getValue()
boolean isValid()
boolean isRelease()
Integer getMajor()
Integer getMinor()
Integer getPatch()
}
103 changes: 103 additions & 0 deletions x-pack/plugin/src/test/resources/rest-api-spec/test/versionfield/20_scripts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,106 @@ setup:
- match: { hits.hits.0._source.version: "3.1.0" }
- match: { hits.hits.1._source.version: "1.1.12" }
- match: { hits.hits.2._source.version: "2.0.0-beta" }

---
"Filter script for illegal versions":
- do:
bulk:
refresh: true
body:
- '{ "index" : { "_index" : "test_index", "_id" : "4" } }'
- '{"version": "a123.2.2-beta" }'

- do:
search:
index: test_index
body:
query: { "bool" : { "filter" : [{ "script" : { "script" : {"source": "doc['version'].isValid() == false"}}}] }}

- match: { hits.total.value: 1 }
- match: { hits.hits.0._source.version: "a123.2.2-beta" }

- do:
search:
index: test_index
body:
query: { "bool" : { "filter" : [{ "script" : { "script" : {"source": "doc['version'].isValid()"}}}] }}

- match: { hits.total.value: 3 }
- match: { hits.hits.0._source.version: "1.1.12" }
- match: { hits.hits.1._source.version: "2.0.0-beta" }
- match: { hits.hits.2._source.version: "3.1.0" }

---
"Filter script for pre-release versions":
- do:
bulk:
refresh: true
body:
- '{ "index" : { "_index" : "test_index", "_id" : "4" } }'
- '{"version": "a123.2.2-beta" }'

- do:
search:
index: test_index
body:
query: { "bool" : { "filter" : [{ "script" : { "script" : {"source": "doc['version'].isRelease()"}}}] }}

- match: { hits.total.value: 2 }
- match: { hits.hits.0._source.version: "1.1.12" }
- match: { hits.hits.1._source.version: "3.1.0" }

---
"Aggregate using script on major, minor, patch versions":

- do:
bulk:
refresh: true
body:
- '{ "index" : { "_index" : "test_index", "_id" : "4" } }'
- '{"version": "3" }'
- '{ "index" : { "_index" : "test_index", "_id" : "5" } }'
- '{"version": "3.1" }'
- '{ "index" : { "_index" : "test_index", "_id" : "6" } }'
- '{"version": "illegal3.1" }'

- do:
search:
index: test_index
body:
size: 0
aggs: { "majors": { "terms": { "script": { "source": "doc['version'].getMajor()", "lang": "painless"}}}}

- length: { aggregations.majors.buckets: 3 }
- match: { aggregations.majors.buckets.0.key: "3" }
- match: { aggregations.majors.buckets.0.doc_count: 3 }
- match: { aggregations.majors.buckets.1.key: "1" }
- match: { aggregations.majors.buckets.1.doc_count: 1 }
- match: { aggregations.majors.buckets.2.key: "2" }
- match: { aggregations.majors.buckets.2.doc_count: 1 }

- do:
search:
index: test_index
body:
size: 0
aggs: { "majors": { "terms": { "script": { "source": "doc['version'].getMinor()", "lang": "painless"}}}}

- length: { aggregations.majors.buckets: 2 }
- match: { aggregations.majors.buckets.0.key: "1" }
- match: { aggregations.majors.buckets.0.doc_count: 3 }
- match: { aggregations.majors.buckets.1.key: "0" }
- match: { aggregations.majors.buckets.1.doc_count: 1 }

- do:
search:
index: test_index
body:
size: 0
aggs: { "majors": { "terms": { "script": { "source": "doc['version'].getPatch()", "lang": "painless"}}}}

- length: { aggregations.majors.buckets: 2 }
- match: { aggregations.majors.buckets.0.key: "0" }
- match: { aggregations.majors.buckets.0.doc_count: 2 }
- match: { aggregations.majors.buckets.1.key: "12" }
- match: { aggregations.majors.buckets.1.doc_count: 1 }