Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move buildEsQuery to a package #23345

Merged
merged 42 commits into from
Nov 22, 2018
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
e27ba53
fix: move buildEsQuery to utils
lukasolson Sep 19, 2018
1308202
fix: tests that I broke
lukasolson Sep 21, 2018
0480b9c
Merge branch 'master' into fix/filtersEverywhere
lukasolson Sep 24, 2018
72fbe52
Merge branch 'master' into fix/filtersEverywhere
lukasolson Oct 1, 2018
7c93a4b
fix: add back link to the docs
lukasolson Oct 1, 2018
391060f
fix: don't export from ui/ and link to utils
lukasolson Oct 2, 2018
fb63b64
Merge branch 'master' into fix/filtersEverywhere
lukasolson Oct 17, 2018
dafa660
Merge branch 'master' into fix/filtersEverywhere
lukasolson Oct 17, 2018
a96841a
fix: move to a package
lukasolson Oct 18, 2018
09a69ed
fix: move error to errors.js
lukasolson Oct 18, 2018
844037d
Merge branch 'master' into fix/filtersEverywhere
lukasolson Oct 18, 2018
96f133f
fix: paths for peg task
lukasolson Oct 18, 2018
a2fb932
Merge branch 'master' into fix/filtersEverywhere
lukasolson Oct 18, 2018
8f2480c
fix: update reference to kuery
lukasolson Oct 19, 2018
63d230d
fix: build step for transpilation
lukasolson Oct 22, 2018
3884518
fix: add typescript declaration file
lukasolson Oct 22, 2018
cb0aea2
Merge branch 'master' into fix/filtersEverywhere
lukasolson Oct 23, 2018
0a0a6d6
Merge branch 'master' into fix/filtersEverywhere
lukasolson Nov 2, 2018
09e4331
Merge branch 'master' into fix/filtersEverywhere
lukasolson Nov 2, 2018
b783ea9
fix: test
lukasolson Nov 2, 2018
36b4259
tmp: debug individual tests
lukasolson Nov 5, 2018
f124678
debug: add debug stuff for reporting tests
lukasolson Nov 5, 2018
d8bfa36
try to debug test
lukasolson Nov 6, 2018
414c311
Merge branch 'master' into fix/filtersEverywhere
lukasolson Nov 7, 2018
a13a749
Merge branch 'master' into fix/filtersEverywhere
lukasolson Nov 12, 2018
4fbac25
Merge branch 'master' into lukasolson/fix/filtersEverywhere
markov00 Nov 13, 2018
887c97a
Testing splitting reporting jobs in two
markov00 Nov 13, 2018
103ee2b
Testing splitting each job
markov00 Nov 13, 2018
d1d0970
Fix ci yaml
markov00 Nov 13, 2018
59f978f
Skipping job to check failing test
markov00 Nov 13, 2018
e36d7ab
debug - adding a catch to jobResponseHandler on report
markov00 Nov 14, 2018
f0149b1
Testing a different job and enabling verbose mode
markov00 Nov 14, 2018
99a8bc0
Testing verbose on phantom_api skipping other CI tests
markov00 Nov 14, 2018
245f72c
Fix script mode
markov00 Nov 14, 2018
991d46f
fix: try running tests in chromium
lukasolson Nov 14, 2018
04f5e37
fix: move out of devDependencies
lukasolson Nov 15, 2018
9fd38fe
fix: remove commented test
lukasolson Nov 15, 2018
7db5961
Revert "fix: try running tests in chromium"
lukasolson Nov 15, 2018
df364f0
Revert testing changes
markov00 Nov 19, 2018
408ddb4
Merge branch 'master' into pr/23345
markov00 Nov 19, 2018
ffa8657
Fixing build for phantomjs
markov00 Nov 20, 2018
ee4ddc2
Revert CI configuration to master. Remove verbose logging for tests
markov00 Nov 20, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ bower_components
/src/fixtures/vislib/mock_data
/src/ui/public/angular-bootstrap
/src/ui/public/flot-charts
/src/ui/public/kuery/ast/kuery.js
/src/ui/public/kuery/ast/legacy_kuery.js
/src/utils/kuery/ast/kuery.js
/src/utils/kuery/ast/legacy_kuery.js
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of interest: why are we ignoring those two files for linting? :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They're generated by PEG

/test/fixtures/scenarios
/src/core_plugins/console/public/webpackShims
/src/core_plugins/console/public/tests/webpackShims
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@

import _ from 'lodash';
import { FilterManager } from './filter_manager.js';
import { buildPhraseFilter } from 'ui/filter_manager/lib/phrase';
import { buildPhrasesFilter } from 'ui/filter_manager/lib/phrases';
import { buildPhraseFilter, buildPhrasesFilter } from '../../../../../utils/filters';

export class PhraseFilterManager extends FilterManager {
constructor(controlId, fieldName, indexPattern, queryFilter) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

import _ from 'lodash';
import { FilterManager } from './filter_manager.js';
import { buildRangeFilter } from 'ui/filter_manager/lib/range';
import { buildRangeFilter } from '../../../../../utils/filters';

// Convert slider value into ES range filter
function toRange(sliderValue) {
Expand Down
2 changes: 1 addition & 1 deletion src/core_plugins/vega/public/vega_view/vega_base_view.js
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import * as vegaLite from 'vega-lite';
import { Utils } from '../data_model/utils';
import { VISUALIZATION_COLORS } from '@elastic/eui';
import { TooltipHandler } from './vega_tooltip';
import { buildQueryFilter } from 'ui/filter_manager/lib';
import { buildQueryFilter } from '../../../../utils/filters';

vega.scheme('elastic', VISUALIZATION_COLORS);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,7 @@
*/

import _ from 'lodash';
import { buildExistsFilter } from '../../filter_manager/lib/exists';
import { buildPhrasesFilter } from '../../filter_manager/lib/phrases';
import { buildExistsFilter, buildPhrasesFilter } from '../../../../utils/filters';
import { buildQueryFromFilters } from '../../courier';

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
*/

import moment from 'moment';
import { buildRangeFilter } from '../../../filter_manager/lib/range';
import { buildRangeFilter } from '../../../../../utils/filters';

export function createFilterDateHistogram(agg, key) {
const start = moment(key);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

import chrome from '../../../chrome';
import { dateRange } from '../../../utils/date_range';
import { buildRangeFilter } from '../../../filter_manager/lib/range';
import { buildRangeFilter } from '../../../../../utils/filters';

const config = chrome.getUiSettingsClient();

Expand Down
2 changes: 1 addition & 1 deletion src/ui/public/agg_types/buckets/create_filter/filters.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
* under the License.
*/

import { buildQueryFilter } from '../../../filter_manager/lib/query';
import { buildQueryFilter } from '../../../../../utils/filters';
import _ from 'lodash';

export function createFilterFilters(aggConfig, key) {
Expand Down
2 changes: 1 addition & 1 deletion src/ui/public/agg_types/buckets/create_filter/histogram.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
* under the License.
*/

import { buildRangeFilter } from '../../../filter_manager/lib/range';
import { buildRangeFilter } from '../../../../../utils/filters';

export function createFilterHistogram(aggConfig, key) {
const value = parseInt(key, 10);
Expand Down
2 changes: 1 addition & 1 deletion src/ui/public/agg_types/buckets/create_filter/ip_range.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
*/

import { CidrMask } from '../../../utils/cidr_mask';
import { buildRangeFilter } from '../../../filter_manager/lib/range';
import { buildRangeFilter } from '../../../../../utils/filters';

export function createFilterIpRange(aggConfig, key) {
let range;
Expand Down
2 changes: 1 addition & 1 deletion src/ui/public/agg_types/buckets/create_filter/range.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
* under the License.
*/

import { buildRangeFilter } from '../../../filter_manager/lib/range';
import { buildRangeFilter } from '../../../../../utils/filters';

export function createFilterRange(aggConfig, key) {
return buildRangeFilter(
Expand Down
4 changes: 1 addition & 3 deletions src/ui/public/agg_types/buckets/create_filter/terms.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,7 @@
* under the License.
*/

import { buildPhraseFilter } from '../../../filter_manager/lib/phrase';
import { buildPhrasesFilter } from '../../../filter_manager/lib/phrases';
import { buildExistsFilter } from '../../../filter_manager/lib/exists';
import { buildPhraseFilter, buildPhrasesFilter, buildExistsFilter } from '../../../../../utils/filters';

export function createFilterTerms(aggConfig, key, params) {
const field = aggConfig.params.field;
Expand Down
3 changes: 0 additions & 3 deletions src/ui/public/courier/search_source/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,3 @@
*/

export { SearchSourceProvider } from './search_source';
export { migrateFilter } from './migrate_filter';
export { decorateQuery } from './decorate_query';
export { buildQueryFromFilters, luceneStringToDsl } from './build_query';
9 changes: 7 additions & 2 deletions src/ui/public/courier/search_source/search_source.js
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,8 @@ import { searchRequestQueue } from '../search_request_queue';
import { FetchSoonProvider } from '../fetch';
import { FieldWildcardProvider } from '../../field_wildcard';
import { getHighlightRequest } from '../../../../core_plugins/kibana/common/highlight';
import { BuildESQueryProvider } from './build_query';
import { BuildESQueryProvider } from '../../../../utils/es_query';
import { KbnError } from '../../errors';

const FIELDS = [
'type',
Expand Down Expand Up @@ -601,7 +602,11 @@ export function SearchSourceProvider(Promise, Private, config) {
_.set(flatData.body, '_source.includes', remainingFields);
}

flatData.body.query = buildESQuery(flatData.index, flatData.query, flatData.filters);
try {
flatData.body.query = buildESQuery(flatData.index, flatData.query, flatData.filters);
} catch (e) {
throw new KbnError(e.message, KbnError);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok now that I see this, I'm assuming that's why you removed the wildcard error type?

}

if (flatData.highlightAll != null) {
if (flatData.highlightAll && flatData.body.query) {
Expand Down
2 changes: 1 addition & 1 deletion src/ui/public/filter_editor/filter_editor.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ import {
areIndexPatternsProvided,
isFilterPinned
} from './lib/filter_editor_utils';
import * as filterBuilder from '../filter_manager/lib';
import * as filterBuilder from '../../../utils/filters';
import { keyMap } from '../utils/key_map';

const module = uiModules.get('kibana');
Expand Down
2 changes: 1 addition & 1 deletion src/ui/public/filter_manager/filter_manager.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

import _ from 'lodash';
import { FilterBarQueryFilterProvider } from '../filter_bar/query_filter';
import { getPhraseScript } from './lib/phrase';
import { getPhraseScript } from '../../../utils/filters';

// Adds a filter to a passed state
export function FilterManagerProvider(Private) {
Expand Down
4 changes: 1 addition & 3 deletions src/ui/public/kuery/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,4 @@
* under the License.
*/

export * from './ast';
export * from './filter_migration';
export * from './node_types';
export * from '../../../utils/kuery';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just delete this module now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above, we could, but we'd have to update all of the references, which is fine by me if you think it's best.

2 changes: 1 addition & 1 deletion src/ui/public/utils/brush_event.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

import _ from 'lodash';
import moment from 'moment';
import { buildRangeFilter } from '../filter_manager/lib/range';
import { buildRangeFilter } from '../../../utils/filters';
import { timefilter } from 'ui/timefilter';

export function onBrushEvent(event, $state) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import { BuildESQueryProvider } from '../build_es_query';
import StubbedLogstashIndexPatternProvider from 'fixtures/stubbed_logstash_index_pattern';
import ngMock from 'ng_mock';
import { expectDeepEqual } from '../../../../../../test_utils/expect_deep_equal.js';
import { fromKueryExpression, toElasticsearchQuery } from '../../../../kuery';
import { fromKueryExpression, toElasticsearchQuery } from '../../../../../../utils/kuery';
import { luceneStringToDsl } from '../lucene_string_to_dsl';
import { decorateQuery } from '../../decorate_query';

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ import StubbedLogstashIndexPatternProvider from 'fixtures/stubbed_logstash_index
import ngMock from 'ng_mock';
import { expectDeepEqual } from '../../../../../../test_utils/expect_deep_equal.js';
import expect from 'expect.js';
import { fromKueryExpression, toElasticsearchQuery } from '../../../../kuery';
import { fromKueryExpression, toElasticsearchQuery } from '../../../../../../utils/kuery';

let indexPattern;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@
*/

import { groupBy, has } from 'lodash';
import { decorateQuery } from '../decorate_query';
import { buildQueryFromKuery } from './from_kuery';
import { buildQueryFromFilters } from './from_filters';
import { buildQueryFromLucene } from './from_lucene';
Expand All @@ -35,7 +34,7 @@ export function BuildESQueryProvider(config) {
const queriesByLanguage = groupBy(validQueries, 'language');

const kueryQuery = buildQueryFromKuery(indexPattern, queriesByLanguage.kuery, config);
const luceneQuery = buildQueryFromLucene(queriesByLanguage.lucene, decorateQuery);
const luceneQuery = buildQueryFromLucene(queriesByLanguage.lucene, config);
const filterQuery = buildQueryFromFilters(filters, indexPattern);

return {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,13 @@
*/

import _ from 'lodash';
import chrome from '../../chrome';

const config = chrome.getUiSettingsClient();

/**
* Decorate queries with default parameters
* @param {query} query object
* @returns {object}
*/
export function decorateQuery(query) {
export function decorateQuery(query, config) {
const queryOptions = config.get('query:queryString:options');

if (_.has(query, 'query_string.query')) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
*/

import _ from 'lodash';
import { migrateFilter } from '../migrate_filter';
import { migrateFilter } from './migrate_filter';

/**
* Create a filter that can be reversed for filters with negate set
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,35 +17,11 @@
* under the License.
*/

import { fromLegacyKueryExpression, fromKueryExpression, toElasticsearchQuery, nodeTypes } from '../../../kuery';
import { documentationLinks } from '../../../documentation_links';
import { NoLeadingWildcardsError } from '../../../kuery/errors';

const queryDocs = documentationLinks.query;
import { fromKueryExpression, toElasticsearchQuery, nodeTypes } from '../kuery';

export function buildQueryFromKuery(indexPattern, queries = [], config) {
const allowLeadingWildcards = config.get('query:allowLeadingWildcards');

const queryASTs = queries.map((query) => {
try {
return fromKueryExpression(query.query, { allowLeadingWildcards });
}
catch (parseError) {
if (parseError instanceof NoLeadingWildcardsError) {
throw parseError;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was this condition doing and why was it removed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because I got rid of the NoLeadingWildcardsError since it relied on KbnError in public. Instead, we just throw a regular error. This condition wasn't actually really necessary anyway since we end up throwing the error anyway below (unless it parses successfully in the old syntax). So in theory, the only thing that changes by removing this condition is when something would parse successfully in the old grammar but also has a leading wildcard, which I can't imagine of a scenario.


try {
fromLegacyKueryExpression(query.query);
}
catch (legacyParseError) {
throw parseError;
}
throw new Error(
`It looks like you're using an outdated Kuery syntax. See what changed in the [docs](${queryDocs.kueryQuerySyntax})!`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove the link to the docs? Without any direction I think it'll be impossible for a user to find the right information.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't want it to rely on something in public/ (which is where documentationLinks is). I guess I could just make the invoking function in public/ check for this type of error and add the links.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation_links module could also probably be moved to utils?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That module relies on ui/metadata which I'm not sure could be moved to utils...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It only uses the branch name though. I think the stuff in metadata ultimately comes from the server, so the branch name should be accessible somewhere.

);
}
});
const queryASTs = queries.map(query => fromKueryExpression(query.query, { allowLeadingWildcards }));
return buildQuery(indexPattern, queryASTs);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,13 @@
*/

import _ from 'lodash';
import { decorateQuery } from './decorate_query';
import { luceneStringToDsl } from './lucene_string_to_dsl';

export function buildQueryFromLucene(queries, decorateQuery) {
export function buildQueryFromLucene(queries, config) {
const combinedQueries = _.map(queries, (query) => {
const queryDsl = luceneStringToDsl(query.query);
return decorateQuery(queryDsl);
return decorateQuery(queryDsl, config);
});

return {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
*/

import _ from 'lodash';
import { getConvertedValueForField } from '../../filter_manager/lib/phrase';
import { getConvertedValueForField } from '../../utils/filters';

export function migrateFilter(filter, indexPattern) {
if (filter.match) {
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@
* under the License.
*/

export { buildExistsFilter } from './exists';
export { buildPhraseFilter } from './phrase';
export { buildPhrasesFilter } from './phrases';
export { buildQueryFilter } from './query';
export { buildRangeFilter } from './range';
export * from './exists';
export * from './phrase';
export * from './phrases';
export * from './query';
export * from './range';
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@

import _ from 'lodash';
import { nodeTypes } from '../node_types/index';
import * as errors from '../errors';
import { parse as parseKuery } from './kuery';
import { parse as parseLegacyKuery } from './legacy_kuery';

Expand Down Expand Up @@ -47,7 +46,7 @@ function fromExpression(expression, parseOptions = {}, parse = parseKuery) {

parseOptions = {
...parseOptions,
helpers: { nodeTypes, errors }
helpers: { nodeTypes }
};

return parse(expression, parseOptions);
Expand Down
File renamed without changes.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

// Initialization block
{
const { parseCursor, cursorSymbol, allowLeadingWildcards = true, helpers: { nodeTypes, errors } } = options;
const { parseCursor, cursorSymbol, allowLeadingWildcards = true, helpers: { nodeTypes } } = options;
const buildFunctionNode = nodeTypes.function.buildNodeWithArgumentNodes;
const buildLiteralNode = nodeTypes.literal.buildNode;
const buildWildcardNode = nodeTypes.wildcard.buildNode;
Expand Down Expand Up @@ -163,7 +163,7 @@ Value
if (value.type === 'cursor') return value;

if (!allowLeadingWildcards && value.type === 'wildcard' && nodeTypes.wildcard.hasLeadingWildcard(value)) {
throw new errors.NoLeadingWildcardsError();
error('Leading wildcards are disabled. See query:allowLeadingWildcards in Advanced Settings.');
}

const isPhrase = buildLiteralNode(false);
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import _ from 'lodash';
import * as ast from '../ast';
import * as literal from '../node_types/literal';
import * as wildcard from '../node_types/wildcard';
import { getPhraseScript } from '../../filter_manager/lib/phrase';
import { getPhraseScript } from '../../filters';
import { getFields } from './utils/get_fields';

export function buildNodeParams(fieldName, value, isPhrase = false) {
Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
import _ from 'lodash';
import { nodeTypes } from '../node_types';
import * as ast from '../ast';
import { getRangeScript } from '../../filter_manager/lib/range';
import { getRangeScript } from '../../filters';
import { getFields } from './utils/get_fields';

export function buildNodeParams(fieldName, params) {
Expand Down
Loading