Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update @elastic/elasticsearch to 8.0.0-canary13 #98266

Merged
merged 100 commits into from
Jun 8, 2021
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
501dc8f
bump @elastic/elasticsearch to canary.7
mshustov Apr 26, 2021
f54192e
address errors in core
mshustov Apr 26, 2021
2db55e3
address errors in data plugin
mshustov Apr 26, 2021
e8f4424
address errors in Alerting team plugins
mshustov Apr 26, 2021
5a06616
remove outdated messages in Lens
mshustov Apr 26, 2021
2201e25
remove unnecessary comments in ML
mshustov Apr 26, 2021
2815faa
address errors in Observability plugin
mshustov Apr 26, 2021
1171f07
address errors in reporting plugin
mshustov Apr 26, 2021
adcb00f
address errors in Rule registry plugin
mshustov Apr 26, 2021
911bdd3
fix errors in Security plugins
mshustov Apr 26, 2021
c6ccefb
fix errors in ES-UI plugin
mshustov Apr 26, 2021
a79ded4
remove unnecessary union.
mshustov Apr 26, 2021
78e0b57
update core tests
mshustov Apr 26, 2021
00d8356
fix kbn-es-archiver
mshustov Apr 26, 2021
eb93d74
update to canary 8
mshustov May 4, 2021
b73a043
Merge branch 'master' into update-es-to-canary-7
mshustov May 14, 2021
10fc716
bump to v9
mshustov May 14, 2021
5b2f816
use new typings
mshustov May 14, 2021
7d66a3d
fix new errors in core
mshustov May 14, 2021
4f3d917
Merge branch 'master' into update-es-to-canary-7
mshustov May 18, 2021
1bcdc62
fix errors in core typeings
mshustov May 18, 2021
67d12d5
fix type errors in data plugin
mshustov May 18, 2021
e777436
fix type errors in telemetray plugin
mshustov May 18, 2021
9e7cdc5
fix data plugin tests
mshustov May 18, 2021
c90d623
fix search examples type error
mshustov May 18, 2021
c6e56de
fix errors in discover plugin
mshustov May 18, 2021
ea14ba1
fix errors in index_pattern_management
mshustov May 18, 2021
41a2a37
fix type errors in vis_type_*
mshustov May 18, 2021
d4d3bd7
fix errors in typings/elasticsearch
mshustov May 18, 2021
dc45dc4
fix type errors in actions plugin
mshustov May 18, 2021
6041055
fix type errors in alerting and apm plugins
mshustov May 18, 2021
e5866ce
fix type errors in canvas and cases
mshustov May 18, 2021
9e83b4c
fix errors in event_log
mshustov May 18, 2021
2f39a5b
fix type errors in ILM and ingest_pipelines
mshustov May 18, 2021
3ed8434
fix errors in lens plugin
mshustov May 18, 2021
5ba2bbd
fix errors in lists plugin
mshustov May 18, 2021
50941bd
fix errors in logstash
mshustov May 18, 2021
c9c7b32
fix errors in metrics_entities
mshustov May 18, 2021
4deea47
fix errors in o11y
mshustov May 18, 2021
a02b37c
fix errors in watcher
mshustov May 18, 2021
4f4f435
fix errors in uptime
mshustov May 18, 2021
3ec7ea4
fix errors in upgrade_assistant
mshustov May 18, 2021
498a5a9
fix errors in task_manager
mshustov May 18, 2021
e66294b
fix errors in stack_alerts
mshustov May 18, 2021
6071ab8
fix errors in security_solution
mshustov May 18, 2021
af3d7a6
fix errors in rule_registry
mshustov May 18, 2021
4f6a79d
fix errors in snapshot_restore
mshustov May 18, 2021
39c5e1c
fix remaining errors
mshustov May 18, 2021
7d6ad5c
fix search intergration tests
mshustov May 18, 2021
19b8fe0
adjust assetion
mshustov May 18, 2021
2d0c51e
Merge branch 'master' into update-es-to-canary-7
mshustov May 19, 2021
00a4af4
bump version to canary.10
mshustov May 19, 2021
812195f
adapt code to new naming schema
mshustov May 19, 2021
287745b
use mapping types provided by the client library
mshustov May 19, 2021
f85e9ba
Revert "adjust assetion"
mshustov May 19, 2021
2f58edb
fix so intergration tests
mshustov May 19, 2021
6043554
fix http integration tests
mshustov May 19, 2021
87e7786
Merge branch 'master' into update-es-to-canary-7
mshustov May 19, 2021
63ff8c7
Merge branch 'master' into update-es-to-canary-7
mshustov May 22, 2021
194bdb2
bump version to canary 11
mshustov May 22, 2021
f4731c4
fix login test
mshustov May 22, 2021
cdb2017
fix http integration test
mshustov May 22, 2021
4d70bd0
fix apm test
mshustov May 22, 2021
9e5aa9b
update docs
mshustov May 22, 2021
93e5d3e
fixing some ml types
jgowdyelastic May 26, 2021
572da7e
Merge branch 'master' into update-es-to-canary-7
mshustov Jun 3, 2021
a449332
fix new errors in data plugin
mshustov Jun 3, 2021
e0afe3c
fix new errors in alerting plugin
mshustov Jun 3, 2021
4adebdd
fix new errors in lists plugin
mshustov Jun 3, 2021
d0f0a27
fix new errors in reporting
mshustov Jun 3, 2021
0245d27
fix or mute errors in rule_registry plugin
mshustov Jun 3, 2021
21f196b
Merge remote-tracking branch 'origin/update-es-to-canary-7' into upda…
mshustov Jun 3, 2021
a02b056
more ML type fixes
jgowdyelastic Jun 3, 2021
8e91339
bump to canary 12
mshustov Jun 3, 2021
8123a98
Merge remote-tracking branch 'origin/update-es-to-canary-7' into upda…
mshustov Jun 3, 2021
689e765
Merge branch 'master' into update-es-to-canary-7
mshustov Jun 3, 2021
2231476
fix errors after merge conflict
mshustov Jun 3, 2021
ce581f6
additional ML fixes
jgowdyelastic Jun 4, 2021
1e7fab5
Merge remote-tracking branch 'origin/update-es-to-canary-7' into upda…
mshustov Jun 4, 2021
c068c1b
bump to canary 13
mshustov Jun 4, 2021
c07f8e8
fix errors in apm plugin
mshustov Jun 4, 2021
f2f2749
fix errors in fleet plugin
mshustov Jun 4, 2021
0b46cbb
fix errors in infra plugin
mshustov Jun 4, 2021
5a39655
fix errors in monitoring plugin
mshustov Jun 4, 2021
596e8f4
fix errors in osquery plugin
mshustov Jun 4, 2021
4e5e3d4
fix errors in security solution plugins
mshustov Jun 4, 2021
4d562e5
fix errors in transform plugin
mshustov Jun 4, 2021
7eec0dd
Update type imports for ES
dgieselaar Jun 7, 2021
ec4469f
Merge remote-tracking branch 'origin/update-es-to-canary-7' into upda…
mshustov Jun 7, 2021
50a8117
Merge branch 'master' into update-es-to-canary-7
mshustov Jun 7, 2021
b36574a
fix errors in x-pack plugins
mshustov Jun 7, 2021
cd8a885
fix errors in tests
mshustov Jun 7, 2021
3909a74
update docs
mshustov Jun 7, 2021
635a2c3
fix errors in x-pack/test
mshustov Jun 7, 2021
d05b773
Merge branch 'master' into update-es-to-canary-7
mshustov Jun 7, 2021
5e6ec78
update error description
mshustov Jun 7, 2021
a87d4dd
Merge branch 'master' into update-es-to-canary-7
mshustov Jun 8, 2021
7304b6c
fix errors after master merge
mshustov Jun 8, 2021
13cca1f
update comment in infra plugin
mshustov Jun 8, 2021
6e549ed
fix new errors on xpack tests/
mshustov Jun 8, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@
"@elastic/apm-rum-react": "^1.2.5",
"@elastic/charts": "29.0.0",
"@elastic/datemath": "link:bazel-bin/packages/elastic-datemath/npm_module",
"@elastic/elasticsearch": "npm:@elastic/elasticsearch-canary@^8.0.0-canary.4",
"@elastic/elasticsearch": "npm:@elastic/elasticsearch-canary@^8.0.0-canary.7",
"@elastic/ems-client": "7.13.0",
"@elastic/eui": "32.1.0",
"@elastic/filesaver": "1.1.2",
Expand Down
2 changes: 1 addition & 1 deletion packages/kbn-es-archiver/src/lib/indices/delete_index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ export async function waitForSnapshotCompletion(
const { body: repositoryMap } = await client.snapshot.getRepository({} as any);
for (const repository of Object.keys(repositoryMap)) {
const allInProgress = await getInProgressSnapshots(repository);
const found = allInProgress.find((s: any) => s.indices.includes(index));
const found = allInProgress?.find((s: any) => s.indices.includes(index));

if (!found) {
continue;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ const MAPPINGS = {
},
},
},
};
} as const;

function runTest(key: string | string[], mapping: IndexMapping | SavedObjectsFieldMapping) {
expect(typeof key === 'string' || Array.isArray(key)).toBeTruthy();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ test(`returns single object with properties`, () => {
properties: {},
},
},
};
} as const;

const result = getRootPropertiesObjects(mappings);
expect(result).toEqual({
Expand All @@ -32,7 +32,7 @@ test(`returns single object with type === 'object'`, () => {
type: 'object',
},
},
};
} as const;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: why casting as const instead of const mappings: IndexMapping = ... ? (in some other test files too)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When TS infers a type for an object { type: 'keyword' }, it ends up with { type: string } since the object can be mutated. as const asserts that the object is immutable and the final type will be { type: "keyword" }.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can do const mappings: IndexMapping =, but it found as const easier to understand 🤷‍♂️


const result = getRootPropertiesObjects(mappings);
expect(result).toEqual({
Expand Down Expand Up @@ -75,7 +75,7 @@ test(`returns two objects with type === 'object'`, () => {
type: 'object',
},
},
};
} as const;

const result = getRootPropertiesObjects(mappings);
expect(result).toEqual({
Expand All @@ -95,7 +95,7 @@ test(`excludes objects without properties and type of keyword`, () => {
type: 'keyword',
},
},
};
} as const;

const result = getRootPropertiesObjects(mappings);
expect(result).toEqual({});
Expand All @@ -111,7 +111,7 @@ test(`excludes two objects without properties and type of keyword`, () => {
type: 'keyword',
},
},
};
} as const;

const result = getRootPropertiesObjects(mappings);
expect(result).toEqual({});
Expand All @@ -127,7 +127,7 @@ test(`includes one object with properties and excludes one object without proper
type: 'keyword',
},
},
};
} as const;

const result = getRootPropertiesObjects(mappings);
expect(result).toEqual({
Expand All @@ -147,7 +147,7 @@ test(`includes one object with type === 'object' and excludes one object without
type: 'keyword',
},
},
};
} as const;

const result = getRootPropertiesObjects(mappings);
expect(result).toEqual({
Expand All @@ -170,7 +170,7 @@ test('excludes references and migrationVersion which are part of the blacklist',
type: 'object',
},
},
};
} as const;
const result = getRootPropertiesObjects(mappings);
expect(result).toEqual({
foo: {
Expand Down
24 changes: 12 additions & 12 deletions src/core/server/saved_objects/mappings/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
* Side Public License, v 1.
*/

import type { estypes } from '@elastic/elasticsearch';
/**
* Describe a saved object type mapping.
*
Expand Down Expand Up @@ -120,18 +121,17 @@ export interface IndexMappingMeta {
*
* @public
*/
export interface SavedObjectsCoreFieldMapping {
type: string;
null_value?: number | boolean | string;
index?: boolean;
Comment on lines -123 to -126
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's probably a good thing to finally get rid of these manually maintained types

doc_values?: boolean;
fields?: {
[subfield: string]: {
type: string;
ignore_above?: number;
};
};
}
export type SavedObjectsCoreFieldMapping = estypes.Property & {
/**
* The dynamic property of the mapping, either `false` or `'strict'`. If
* unspecified `dynamic: 'strict'` will be inherited from the top-level
* index mappings.
*
* Note: To limit the number of mapping fields Saved Object types should
* *never* use `dynamic: true`.
*/
dynamic?: false | 'strict';
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@delvedor We still have a problem with TypeMapping type - it doesn’t allow Object field type. In the current type definitions, it requires type: 'object' property to be set. It’s not the case in our code, https://github.com/elastic/kibana/blob/f54192ec3e5f58fa63c0abf06892ddd306b1da2c/src/core/server/saved_objects/mappings/lib/get_property.test.ts nor in the ES docs https://www.elastic.co/guide/en/elasticsearch/reference/current/object.html

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be fixed in the next types release.

};

/**
* See {@link SavedObjectsFieldMapping} for documentation.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,19 +19,19 @@ describe('buildActiveMappings', () => {
const properties = {
aaa: { type: 'text' },
bbb: { type: 'long' },
};
} as const;

expect(buildActiveMappings(properties)).toMatchSnapshot();
});

test('disallows duplicate mappings', () => {
const properties = { type: { type: 'long' } };
const properties = { type: { type: 'long' } } as const;

expect(() => buildActiveMappings(properties)).toThrow(/Cannot redefine core mapping \"type\"/);
});

test('disallows mappings with leading underscore', () => {
const properties = { _hm: { type: 'keyword' } };
const properties = { _hm: { type: 'keyword' } } as const;

expect(() => buildActiveMappings(properties)).toThrow(
/Invalid mapping \"_hm\"\. Mappings cannot start with _/
Expand Down Expand Up @@ -60,7 +60,7 @@ describe('buildActiveMappings', () => {
aaa: { type: 'keyword', fields: { a: { type: 'keyword' }, b: { type: 'text' } } },
bbb: { fields: { b: { type: 'text' }, a: { type: 'keyword' } }, type: 'keyword' },
ccc: { fields: { b: { type: 'text' }, a: { type: 'text' } }, type: 'keyword' },
};
} as const;

const mappings = buildActiveMappings(properties);
const hashes = mappings._meta!.migrationMappingPropertyHashes!;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ test('mappings without index pattern goes to default index', () => {
type1: {
properties: {
field1: {
type: 'string',
type: 'text',
},
},
},
Expand All @@ -48,7 +48,7 @@ test('mappings without index pattern goes to default index', () => {
type1: {
properties: {
field1: {
type: 'string',
type: 'text',
},
},
},
Expand All @@ -69,7 +69,7 @@ test(`mappings with custom index pattern doesn't go to default index`, () => {
type1: {
properties: {
field1: {
type: 'string',
type: 'text',
},
},
},
Expand All @@ -81,7 +81,7 @@ test(`mappings with custom index pattern doesn't go to default index`, () => {
type1: {
properties: {
field1: {
type: 'string',
type: 'text',
},
},
},
Expand All @@ -103,7 +103,7 @@ test('creating a script gets added to the index pattern', () => {
type1: {
properties: {
field1: {
type: 'string',
type: 'text',
},
},
},
Expand All @@ -116,7 +116,7 @@ test('creating a script gets added to the index pattern', () => {
type1: {
properties: {
field1: {
type: 'string',
type: 'text',
},
},
},
Expand Down Expand Up @@ -144,18 +144,18 @@ test('throws when two scripts are defined for an index pattern', () => {
type1: {
properties: {
field1: {
type: 'string',
type: 'text',
},
},
},
type2: {
properties: {
field1: {
type: 'string',
type: 'text',
},
},
},
};
} as const;
expect(() =>
createIndexMap({
kibanaIndexName: defaultIndex,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ describe('ElasticIndex', () => {
return elasticsearchClientMock.createSuccessTransportRequestPromise({
[index]: {
aliases: { foo: index },
mappings: { dynamic: 'strict', properties: { a: 'b' } },
mappings: { dynamic: 'strict', properties: { a: 'b' } as any },
settings: {},
},
} as estypes.GetIndexResponse);
Expand Down Expand Up @@ -175,7 +175,7 @@ describe('ElasticIndex', () => {
dynamic: 'strict' as const,
properties: { foo: { type: 'keyword' } },
},
};
} as const;

await Index.convertToAlias(
client,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,7 @@ export async function createIndex(
mappings?: IndexMapping
) {
await client.indices.create({
// @ts-expect-error @elastic-elasticsearch types don't support nested properties
body: { mappings, settings },
index,
});
Expand All @@ -283,6 +284,7 @@ export async function convertToAlias(
script?: string
) {
await client.indices.create({
// @ts-expect-error @elastic-elasticsearch types don't support nested properties
body: { mappings: info.mappings, settings },
index: info.indexName,
});
Expand Down Expand Up @@ -406,7 +408,6 @@ async function reindex(
task_id: String(task),
});

// @ts-expect-error @elastic/elasticsearch GetTaskResponse doesn't contain `error` property
const e = body.error;
if (e) {
throw new Error(`Re-index failed [${e.type}] ${e.reason} :: ${JSON.stringify(e)}`);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ describe('disableUnknownTypeMappingFields', () => {
},
},
},
};
} as const;
const activeMappings = {
_meta: {
migrationMappingPropertyHashes: {
Expand All @@ -46,7 +46,7 @@ describe('disableUnknownTypeMappingFields', () => {
},
},
},
};
} as const;
const targetMappings = disableUnknownTypeMappingFields(activeMappings, sourceMappings);

it('disables complex field mappings from unknown types in the source mappings', () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -362,7 +362,11 @@ const mockV2MigrationOptions = () => {
)
);
options.client.indices.addBlock.mockReturnValue(
elasticsearchClientMock.createSuccessTransportRequestPromise({ acknowledged: true })
elasticsearchClientMock.createSuccessTransportRequestPromise({
acknowledged: true,
shards_acknowledged: true,
indices: [],
})
);
options.client.reindex.mockReturnValue(
elasticsearchClientMock.createSuccessTransportRequestPromise({
Expand Down
7 changes: 4 additions & 3 deletions src/core/server/saved_objects/migrationsv2/actions/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -305,7 +305,7 @@ export const cloneIndex = (
};

interface WaitForTaskResponse {
error: Option.Option<{ type: string; reason: string; index: string }>;
error: Option.Option<{ type: string; reason: string; index?: string }>;
completed: boolean;
failures: Option.Option<any[]>;
description?: string;
Expand Down Expand Up @@ -367,7 +367,6 @@ const waitForTask = (
const failures = body.response?.failures ?? [];
return Either.right({
completed: body.completed,
// @ts-expect-error @elastic/elasticsearch GetTaskResponse doesn't declare `error` property
error: Option.fromNullable(body.error),
failures: failures.length > 0 ? Option.some(failures) : Option.none,
description: body.task.description,
Expand Down Expand Up @@ -523,7 +522,7 @@ export const waitForReindexTask = flow(
if (res.error.value.type === 'index_not_found_exception') {
return TaskEither.left({
type: 'index_not_found_exception' as const,
index: res.error.value.index,
index: res.error.value.index!,
});
} else {
throw new Error('Reindex failed with the following error:\n' + JSON.stringify(res.error));
Expand Down Expand Up @@ -720,6 +719,7 @@ export const createIndex = (
// started
timeout: DEFAULT_TIMEOUT,
body: {
// @ts-expect-error @elastic-elasticsearch types don't support nested properties
mappings,
aliases: aliasesObject,
settings: {
Expand Down Expand Up @@ -813,6 +813,7 @@ export const updateAndPickupMappings = (
.putMapping({
index,
timeout: DEFAULT_TIMEOUT,
// @ts-expect-error @elastic-elasticsearch types don't support nested properties
body: mappings,
})
.then((res) => {
Expand Down
4 changes: 2 additions & 2 deletions src/core/server/saved_objects/migrationsv2/model.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ describe('migrations v2 model', () => {
disabled_saved_object_type: '7997cf5a56cc02bdc9c93361bde732b0',
},
},
};
} as const;

test('INIT -> OUTDATED_DOCUMENTS_SEARCH if .kibana is already pointing to the target index', () => {
const res: ResponseType<'INIT'> = Either.right({
Expand Down Expand Up @@ -657,7 +657,7 @@ describe('migrations v2 model', () => {
disabled_saved_object_type: '7997cf5a56cc02bdc9c93361bde732b0',
},
},
};
} as const;

const waitForYellowSourceState: WaitForYellowSourceState = {
...baseState,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ export class SavedObjectTypeRegistry {
throw new Error(`Type '${type.name}' is already registered`);
}
validateType(type);
this.types.set(type.name, deepFreeze(type));
this.types.set(type.name, deepFreeze(type) as SavedObjectsType);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What change caused this cast to be required?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it's the fact that SavedObjectsFieldMapping is the union type from now on.

}

/**
Expand Down
Loading