Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Search Sessions] Client side search cache #92439

Merged
merged 104 commits into from
Apr 16, 2021
Merged
Show file tree
Hide file tree
Changes from 99 commits
Commits
Show all changes
104 commits
Select commit Hold shift + click to select a range
40548e4
dev docs
Feb 10, 2021
5e20068
sessions tutorial
Feb 11, 2021
e3a6342
title
Feb 11, 2021
4a4e6aa
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
3e81b18
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
572fc8a
Merge branch 'master' of github.com:elastic/kibana into sessions/dev-…
Feb 22, 2021
5364643
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
0e05128
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
8c6451f
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
57303b3
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
5c8c415
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
f929a23
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
5d78b1f
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
ace6fcb
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
11b8018
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
bb11a01
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
7698bc7
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
b8d5172
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
a2a31f2
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
db06563
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
d283cb9
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
58c9555
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
2d95717
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
d38135b
Merge branch 'sessions/dev-docs' of github.com:lizozom/kibana into se…
Feb 22, 2021
52b53a6
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
7f67919
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
04c7065
Update dev_docs/tutorials/data/search.mdx
lizozom Feb 22, 2021
4bc8a64
Merge branch 'sessions/dev-docs' of github.com:lizozom/kibana into se…
Feb 22, 2021
31f7655
Code review
Feb 22, 2021
27c93e2
Merge branch 'master' of github.com:elastic/kibana into sessions/dev-…
Feb 22, 2021
fc6a00d
client cache
Feb 23, 2021
0afdecb
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Mar 15, 2021
49b0751
mock utils
Mar 15, 2021
127c70e
improve code
Mar 15, 2021
02af580
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Mar 16, 2021
5620ecf
Use cacheOnClient in Lens
Mar 16, 2021
bdd137d
mock
Mar 16, 2021
0bdc211
docs and types
Mar 16, 2021
4fe003e
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Mar 16, 2021
94e3a64
unit tests!
Mar 18, 2021
5743645
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Mar 18, 2021
2253276
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Mar 22, 2021
1ab4ca2
Search response cache + tests
Mar 24, 2021
9bd5cee
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Mar 24, 2021
6059ba9
remove cacheOnClient
Mar 24, 2021
f7c7694
test ts
Mar 24, 2021
436af44
shouldCacheOnClient + improve tests
Mar 24, 2021
d14b2b8
remove unused
Mar 24, 2021
4e7dca4
clear subs
Mar 25, 2021
fe998a7
dont unsubscribe on setItem
Mar 25, 2021
afb47af
caching mess
Mar 25, 2021
e963be0
t
Mar 25, 2021
5a0ab41
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Mar 29, 2021
8156cd4
fix jest
Mar 29, 2021
8a0901b
add size to bfetch response @ppisljar
Mar 29, 2021
fa6f783
ts
Mar 29, 2021
fbb8f33
ts
Mar 29, 2021
b245982
yarn kbn bootstrapMerge branch 'master' of github.com:elastic/kibana …
Mar 29, 2021
570fe84
docs
Mar 29, 2021
9735765
simplify abort controller logic and extract it into a class
Mar 29, 2021
e1574dc
docs
Mar 29, 2021
9554535
delete unused tests
Mar 29, 2021
3656089
Merge branch 'sessions/extract-search-abort-controller' into sessions…
Mar 29, 2021
301c826
use addAbortSignal
Mar 29, 2021
512bd7e
code review
Mar 30, 2021
765c942
Merge branch 'master' of github.com:elastic/kibana into sessions/extr…
Mar 30, 2021
f1634dd
Merge branch 'sessions/extract-search-abort-controller' into sessions…
Mar 30, 2021
453aa0f
Use shareReplay, fix tests
Mar 30, 2021
af40401
Merge branch 'master' of github.com:elastic/kibana into sessions/extr…
Mar 30, 2021
1de1a63
Merge branch 'sessions/extract-search-abort-controller' into sessions…
Mar 30, 2021
ed1f6f0
code review
Mar 30, 2021
645c7fb
Merge branch 'sessions/extract-search-abort-controller' into sessions…
Mar 30, 2021
9b3d67f
bfetch test
Mar 30, 2021
5097a6e
code review
Mar 30, 2021
fa4eedc
Merge branch 'sessions/extract-search-abort-controller' into sessions…
Mar 30, 2021
49bade6
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Mar 30, 2021
3fae68f
Leave the bfetch changes out
Mar 30, 2021
5cda382
docs + isRestore
Mar 30, 2021
885de71
make sure to clean up properly
Mar 30, 2021
92ea7b5
Make sure that aborting in cache works correctly
Mar 31, 2021
4c71f89
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Mar 31, 2021
19d09f2
fix test
Mar 31, 2021
ba360f9
Merge branch 'master' into sessions/client-side-cache
kibanamachine Mar 31, 2021
700aba6
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Mar 31, 2021
6679e80
Merge branch 'sessions/client-side-cache' of github.com:lizozom/kiban…
Mar 31, 2021
7e74935
Merge branch 'master' into sessions/client-side-cache
kibanamachine Apr 1, 2021
79f8907
import
Apr 1, 2021
1cfbd69
Merge branch 'sessions/client-side-cache' of github.com:lizozom/kiban…
Apr 1, 2021
6820678
Merge branch 'master' into sessions/client-side-cache
kibanamachine Apr 4, 2021
26bb24b
Merge branch 'master' into sessions/client-side-cache
kibanamachine Apr 5, 2021
836d8fa
Merge branch 'master' into sessions/client-side-cache
kibanamachine Apr 5, 2021
7e394d4
Merge branch 'master' into sessions/client-side-cache
kibanamachine Apr 7, 2021
a8efa69
Merge branch 'master' into sessions/client-side-cache
kibanamachine Apr 12, 2021
6671fbe
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Apr 13, 2021
07423c1
code review round 1
Apr 13, 2021
eb4147b
ts
Apr 13, 2021
bdc3c9a
Merge branch 'sessions/client-side-cache' of github.com:lizozom/kiban…
Apr 13, 2021
3757cc8
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Apr 14, 2021
15cf1e0
Added functional test for search request caching
Apr 14, 2021
0058dd0
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Apr 15, 2021
b9a82c5
test
Apr 15, 2021
3591a94
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Apr 16, 2021
b8c4358
Merge branch 'master' of github.com:elastic/kibana into sessions/clie…
Apr 16, 2021
869e0c7
skip before codefreeze
Apr 16, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-plugins-data-public](./kibana-plugin-plugins-data-public.md) &gt; [SearchInterceptor](./kibana-plugin-plugins-data-public.searchinterceptor.md) &gt; [getSerializableOptions](./kibana-plugin-plugins-data-public.searchinterceptor.getserializableoptions.md)

## SearchInterceptor.getSerializableOptions() method

<b>Signature:</b>

```typescript
protected getSerializableOptions(options?: ISearchOptions): Pick<ISearchOptions, "strategy" | "sessionId" | "isStored" | "isRestore" | "legacyHitsTotal">;
```

## Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| options | <code>ISearchOptions</code> | |

<b>Returns:</b>

`Pick<ISearchOptions, "strategy" | "sessionId" | "isStored" | "isRestore" | "legacyHitsTotal">`

Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ export declare class SearchInterceptor

| Method | Modifiers | Description |
| --- | --- | --- |
| [getSerializableOptions(options)](./kibana-plugin-plugins-data-public.searchinterceptor.getserializableoptions.md) | | |
| [getTimeoutMode()](./kibana-plugin-plugins-data-public.searchinterceptor.gettimeoutmode.md) | | |
| [handleSearchError(e, options, isTimeout)](./kibana-plugin-plugins-data-public.searchinterceptor.handlesearcherror.md) | | |
| [search(request, options)](./kibana-plugin-plugins-data-public.searchinterceptor.search.md) | | Searches using the given <code>search</code> method. Overrides the <code>AbortSignal</code> with one that will abort either when the request times out, or when the original <code>AbortSignal</code> is aborted. Updates <code>pendingCount$</code> when the request is started/finalized. |
Expand Down
1 change: 1 addition & 0 deletions examples/search_examples/common/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ export interface IMyStrategyRequest extends IEsSearchRequest {
}
export interface IMyStrategyResponse extends IEsSearchResponse {
cool: string;
executed_at: number;
}

export const SERVER_SEARCH_ROUTE_PATH = '/api/examples/search';
67 changes: 61 additions & 6 deletions examples/search_examples/public/search/app.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ export const SearchExamplesApp = ({
setSelectedNumericField(fields?.length ? getNumeric(fields)[0] : null);
}, [fields]);

const doAsyncSearch = async (strategy?: string) => {
const doAsyncSearch = async (strategy?: string, sessionId?: string) => {
if (!indexPattern || !selectedNumericField) return;

// Construct the query portion of the search request
Expand All @@ -138,6 +138,7 @@ export const SearchExamplesApp = ({
const searchSubscription$ = data.search
.search(req, {
strategy,
sessionId,
})
.subscribe({
next: (res) => {
Expand All @@ -148,19 +149,30 @@ export const SearchExamplesApp = ({
? // @ts-expect-error @elastic/elasticsearch no way to declare a type for aggregation in the search response
res.rawResponse.aggregations[1].value
: undefined;
const isCool = (res as IMyStrategyResponse).cool;
const executedAt = (res as IMyStrategyResponse).executed_at;
const message = (
<EuiText>
Searched {res.rawResponse.hits.total} documents. <br />
The average of {selectedNumericField!.name} is{' '}
{avgResult ? Math.floor(avgResult) : 0}.
<br />
Is it Cool? {String((res as IMyStrategyResponse).cool)}
{isCool ? `Is it Cool? ${isCool}` : undefined}
<br />
<EuiText data-test-subj="requestExecutedAt">
{executedAt ? `Executed at? ${executedAt}` : undefined}
</EuiText>
</EuiText>
);
notifications.toasts.addSuccess({
title: 'Query result',
text: mountReactNode(message),
});
notifications.toasts.addSuccess(
{
title: 'Query result',
text: mountReactNode(message),
},
{
toastLifeTimeMs: 30000,
}
);
searchSubscription$.unsubscribe();
} else if (isErrorResponse(res)) {
// TODO: Make response error status clearer
Expand Down Expand Up @@ -227,6 +239,10 @@ export const SearchExamplesApp = ({
doAsyncSearch('myStrategy');
};

const onClientSideSessionCacheClickHandler = () => {
doAsyncSearch('myStrategy', data.search.session.getSessionId());
};

const onServerClickHandler = async () => {
if (!indexPattern || !selectedNumericField) return;
try {
Expand Down Expand Up @@ -374,6 +390,45 @@ export const SearchExamplesApp = ({
</EuiButtonEmpty>
</EuiText>
<EuiSpacer />
<EuiTitle size="s">
<h3>Client side search session caching</h3>
</EuiTitle>
<EuiText>
<EuiButtonEmpty
size="xs"
onClick={() => data.search.session.start()}
iconType="alert"
data-test-subj="searchExamplesStartSession"
>
<FormattedMessage
id="searchExamples.startNewSession"
defaultMessage="Start a new session"
/>
</EuiButtonEmpty>
<EuiButtonEmpty
size="xs"
onClick={() => data.search.session.clear()}
iconType="alert"
data-test-subj="searchExamplesClearSession"
>
<FormattedMessage
id="searchExamples.clearSession"
defaultMessage="Clear session"
/>
</EuiButtonEmpty>
<EuiButtonEmpty
size="xs"
onClick={onClientSideSessionCacheClickHandler}
iconType="play"
data-test-subj="searchExamplesCacheSearch"
>
<FormattedMessage
id="searchExamples.myStrategyButtonText"
defaultMessage="Request from low-level client via My Strategy"
/>
</EuiButtonEmpty>
</EuiText>
<EuiSpacer />
<EuiTitle size="s">
<h3>Using search on the server</h3>
</EuiTitle>
Expand Down
1 change: 1 addition & 0 deletions examples/search_examples/server/my_strategy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ export const mySearchStrategyProvider = (
map((esSearchRes) => ({
...esSearchRes,
cool: request.get_cool ? 'YES' : 'NOPE',
executed_at: new Date().getTime(),
}))
),
cancel: async (id, options, deps) => {
Expand Down
2 changes: 1 addition & 1 deletion src/plugins/data/common/search/tabify/tabify.ts
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ export function tabifyAggResponse(
const write = new TabbedAggResponseWriter(aggConfigs, respOpts || {});
const topLevelBucket: AggResponseBucket = {
...esResponse.aggregations,
doc_count: esResponse.hits.total,
doc_count: esResponse.hits?.total,
lizozom marked this conversation as resolved.
Show resolved Hide resolved
};

collectBucket(aggConfigs, write, topLevelBucket, '', 1);
Expand Down
2 changes: 2 additions & 0 deletions src/plugins/data/public/public.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -2354,6 +2354,8 @@ export class SearchInterceptor {
// (undocumented)
protected readonly deps: SearchInterceptorDeps;
// (undocumented)
protected getSerializableOptions(options?: ISearchOptions): Pick<ISearchOptions, "strategy" | "sessionId" | "isStored" | "isRestore" | "legacyHitsTotal">;
// (undocumented)
protected getTimeoutMode(): TimeoutErrorMode;
// Warning: (ae-forgotten-export) The symbol "KibanaServerError" needs to be exported by the entry point index.d.ts
// Warning: (ae-forgotten-export) The symbol "AbortError" needs to be exported by the entry point index.d.ts
Expand Down
28 changes: 17 additions & 11 deletions src/plugins/data/public/search/search_interceptor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -113,20 +113,14 @@ export class SearchInterceptor {
}
}

/**
* @internal
* @throws `AbortError` | `ErrorLike`
*/
protected runSearch(
request: IKibanaSearchRequest,
options?: ISearchOptions
): Promise<IKibanaSearchResponse> {
const { abortSignal, sessionId, ...requestOptions } = options || {};
protected getSerializableOptions(options?: ISearchOptions) {
const { sessionId, ...requestOptions } = options || {};

const serializableOptions: ISearchOptionsSerializable = {};
const combined = {
...requestOptions,
...this.deps.session.getSearchOptions(sessionId),
};
const serializableOptions: ISearchOptionsSerializable = {};

if (combined.sessionId !== undefined) serializableOptions.sessionId = combined.sessionId;
if (combined.isRestore !== undefined) serializableOptions.isRestore = combined.isRestore;
Expand All @@ -135,10 +129,22 @@ export class SearchInterceptor {
if (combined.strategy !== undefined) serializableOptions.strategy = combined.strategy;
if (combined.isStored !== undefined) serializableOptions.isStored = combined.isStored;

return serializableOptions;
}

/**
* @internal
* @throws `AbortError` | `ErrorLike`
*/
protected runSearch(
request: IKibanaSearchRequest,
options?: ISearchOptions
): Promise<IKibanaSearchResponse> {
const { abortSignal } = options || {};
return this.batchedFetch(
{
request,
options: serializableOptions,
options: this.getSerializableOptions(options),
},
abortSignal
);
Expand Down
2 changes: 1 addition & 1 deletion src/plugins/data/public/search/session/session_service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ export interface SearchSessionIndicatorUiConfig {
}

/**
* Responsible for tracking a current search session. Supports only a single session at a time.
* Responsible for tracking a current search session. Supports a single session at a time.
*/
export class SessionService {
public readonly state$: Observable<SearchSessionState>;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,21 +21,24 @@ describe('search abort controller', () => {
test('immediately aborts when passed an aborted signal in the constructor', () => {
const controller = new AbortController();
controller.abort();
const sac = new SearchAbortController(controller.signal);
const sac = new SearchAbortController();
sac.addAbortSignal(controller.signal);
expect(sac.getSignal().aborted).toBe(true);
});

test('aborts when input signal is aborted', () => {
const controller = new AbortController();
const sac = new SearchAbortController(controller.signal);
const sac = new SearchAbortController();
sac.addAbortSignal(controller.signal);
expect(sac.getSignal().aborted).toBe(false);
controller.abort();
expect(sac.getSignal().aborted).toBe(true);
});

test('aborts when all input signals are aborted', () => {
const controller = new AbortController();
const sac = new SearchAbortController(controller.signal);
const sac = new SearchAbortController();
sac.addAbortSignal(controller.signal);

const controller2 = new AbortController();
sac.addAbortSignal(controller2.signal);
Expand All @@ -48,7 +51,8 @@ describe('search abort controller', () => {

test('aborts explicitly even if all inputs are not aborted', () => {
const controller = new AbortController();
const sac = new SearchAbortController(controller.signal);
const sac = new SearchAbortController();
sac.addAbortSignal(controller.signal);

const controller2 = new AbortController();
sac.addAbortSignal(controller2.signal);
Expand All @@ -60,7 +64,8 @@ describe('search abort controller', () => {

test('doesnt abort, if cleared', () => {
const controller = new AbortController();
const sac = new SearchAbortController(controller.signal);
const sac = new SearchAbortController();
sac.addAbortSignal(controller.signal);
expect(sac.getSignal().aborted).toBe(false);
sac.cleanup();
controller.abort();
Expand All @@ -77,15 +82,15 @@ describe('search abort controller', () => {
});

test('doesnt abort on timeout, if cleared', () => {
const sac = new SearchAbortController(undefined, 100);
const sac = new SearchAbortController(100);
expect(sac.getSignal().aborted).toBe(false);
sac.cleanup();
timeTravel(100);
expect(sac.getSignal().aborted).toBe(false);
});

test('aborts on timeout, even if no signals passed in', () => {
const sac = new SearchAbortController(undefined, 100);
const sac = new SearchAbortController(100);
expect(sac.getSignal().aborted).toBe(false);
timeTravel(100);
expect(sac.getSignal().aborted).toBe(true);
Expand All @@ -94,7 +99,8 @@ describe('search abort controller', () => {

test('aborts on timeout, even if there are unaborted signals', () => {
const controller = new AbortController();
const sac = new SearchAbortController(controller.signal, 100);
const sac = new SearchAbortController(100);
sac.addAbortSignal(controller.signal);

expect(sac.getSignal().aborted).toBe(false);
timeTravel(100);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,7 @@ export class SearchAbortController {
private destroyed = false;
private reason?: AbortReason;

constructor(abortSignal?: AbortSignal, timeout?: number) {
if (abortSignal) {
this.addAbortSignal(abortSignal);
}

constructor(timeout?: number) {
Dosant marked this conversation as resolved.
Show resolved Hide resolved
if (timeout) {
this.timeoutSub = timer(timeout).subscribe(() => {
this.reason = AbortReason.Timeout;
Expand All @@ -41,6 +37,7 @@ export class SearchAbortController {
};

public cleanup() {
if (this.destroyed) return;
this.destroyed = true;
this.timeoutSub?.unsubscribe();
this.inputAbortSignals.forEach((abortSignal) => {
Expand Down
Loading