Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sanitize create acceleration queries and direct queries #1605

Merged
merged 1 commit into from
Mar 21, 2024
Merged

Sanitize create acceleration queries and direct queries #1605

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions common/constants/data_sources.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ export const ACCELERATION_ADD_FIELDS_TEXT = '(add fields here)';
export const ACCELERATION_INDEX_NAME_REGEX = /^[a-z0-9_]+$/;
export const ACCELERATION_S3_URL_REGEX = /^(s3|s3a):\/\/[a-zA-Z0-9.\-]+/;
export const SPARK_HIVE_TABLE_REGEX = /Provider:\s*hive/;
export const SANITIZE_QUERY_REGEX = /\s+/g;
export const TIMESTAMP_DATATYPE = 'timestamp';

export const ACCELERATION_INDEX_TYPES = [
Expand Down
11 changes: 8 additions & 3 deletions public/components/common/search/direct_search.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,15 @@
EuiPopoverFooter,
EuiToolTip,
} from '@elastic/eui';
import { i18n } from '@osd/i18n';
import { isEmpty, isEqual } from 'lodash';
import React, { useEffect, useState } from 'react';
import { batch, useDispatch, useSelector } from 'react-redux';
import { i18n } from '@osd/i18n';
import { ASYNC_POLLING_INTERVAL, QUERY_LANGUAGE } from '../../../../common/constants/data_sources';
import {
ASYNC_POLLING_INTERVAL,
QUERY_LANGUAGE,
SANITIZE_QUERY_REGEX,
} from '../../../../common/constants/data_sources';
import {
APP_ANALYTICS_TAB_ID_REGEX,
RAW_QUERY,
Expand Down Expand Up @@ -60,7 +64,7 @@
tempQuery: string;
handleQueryChange: (query: string) => void;
handleQuerySearch: () => void;
dslService: any;

Check warning on line 67 in public/components/common/search/direct_search.tsx

View workflow job for this annotation

GitHub Actions / Lint 

Unexpected any. Specify a different type
}

export interface IDatePickerProps {
Expand All @@ -70,11 +74,11 @@
setEndTime: () => void;
setTimeRange: () => void;
setIsOutputStale: () => void;
handleTimePickerChange: (timeRange: string[]) => any;

Check warning on line 77 in public/components/common/search/direct_search.tsx

View workflow job for this annotation

GitHub Actions / Lint 

Unexpected any. Specify a different type
handleTimeRangePickerRefresh: () => any;

Check warning on line 78 in public/components/common/search/direct_search.tsx

View workflow job for this annotation

GitHub Actions / Lint 

Unexpected any. Specify a different type
}

export const DirectSearch = (props: any) => {

Check warning on line 81 in public/components/common/search/direct_search.tsx

View workflow job for this annotation

GitHub Actions / Lint 

Unexpected any. Specify a different type
const {
query,
tempQuery,
Expand Down Expand Up @@ -115,7 +119,7 @@
error: pollingError,
startPolling,
stopPolling,
} = usePolling<any, any>((params) => {

Check warning on line 122 in public/components/common/search/direct_search.tsx

View workflow job for this annotation

GitHub Actions / Lint 

Unexpected any. Specify a different type

Check warning on line 122 in public/components/common/search/direct_search.tsx

View workflow job for this annotation

GitHub Actions / Lint 

Unexpected any. Specify a different type
return sqlService.fetchWithJobId(params);
}, ASYNC_POLLING_INTERVAL);

Expand Down Expand Up @@ -223,9 +227,10 @@
);
});
const sessionId = getAsyncSessionId(explorerSearchMetadata.datasources[0].label);
const requestQuery = tempQuery || query;
const requestPayload = {
lang: lang.toLowerCase(),
query: tempQuery || query,
query: requestQuery.replaceAll(SANITIZE_QUERY_REGEX, ' '),
datasource: explorerSearchMetadata.datasources[0].label,
} as DirectQueryRequest;

Expand Down Expand Up @@ -329,20 +334,20 @@
})
);
}
}, [pollingResult, pollingError]);

Check warning on line 337 in public/components/common/search/direct_search.tsx

View workflow job for this annotation

GitHub Actions / Lint 

React Hook useEffect has missing dependencies: 'dispatch', 'dispatchOnGettingHis', 'stopPollingWithStatus', and 'tabId'. Either include them or remove the dependency array

useEffect(() => {
return () => {
stopPolling();
};
}, []);

Check warning on line 343 in public/components/common/search/direct_search.tsx

View workflow job for this annotation

GitHub Actions / Lint 

React Hook useEffect has a missing dependency: 'stopPolling'. Either include it or remove the dependency array

useEffect(() => {
if (!explorerSearchMetadata.isPolling) {
stopPolling();
setIsQueryRunning(false);
}
}, [explorerSearchMetadata.isPolling]);

Check warning on line 350 in public/components/common/search/direct_search.tsx

View workflow job for this annotation

GitHub Actions / Lint 

React Hook useEffect has missing dependencies: 'setIsQueryRunning' and 'stopPolling'. Either include them or remove the dependency array

return (
<div className="globalQueryBar">
Expand Down
3 changes: 2 additions & 1 deletion ...ts/manage/accelerations/create_accelerations_flyout/create/create_acceleration_button.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

import { EuiButton } from '@elastic/eui';
import React, { useEffect, useState } from 'react';
import { SANITIZE_QUERY_REGEX } from '../../../../../../../../common/constants/data_sources';
import { CreateAccelerationForm } from '../../../../../../../../common/types/data_connections';
import {
DirectQueryLoadingStatus,
Expand Down Expand Up @@ -39,7 +40,7 @@

const requestPayload: DirectQueryRequest = {
lang: 'sql',
query: accelerationQueryBuilder(accelerationFormData),
query: accelerationQueryBuilder(accelerationFormData).replaceAll(SANITIZE_QUERY_REGEX, ' '),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know I have code that's subject to the same issue, but this will also replace instances of multiple spaces in query strings and prevents things like adding single-line comments to queries.

It might be beyond the scope of this PR, but if we're doing this repeatedly, I think we should make a centralized sanitizeSql method that can handle detecting spaces in strings and remove comments before it tries to normalize whitespace.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. We need also centralize direct query calls so that we keep sanitizeSql method used only in one location. Have added this hook that can be used at all places from cache loaders, integrations to https://github.com/opensearch-project/dashboards-observability/blob/76d206c18bf412d9c4c541f06c991843a113ac7d/public/framework/datasources/direct_query_hook.tsx

Issue: #1503

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added comments to the issue we are tracking

datasource: accelerationFormData.dataSource,
};

Expand All @@ -60,7 +61,7 @@
setIsLoading(false);
setToast('Create acceleration query failed', 'success');
}
}, [directqueryLoadStatus]);

Check warning on line 64 in public/components/datasources/components/manage/accelerations/create_accelerations_flyout/create/create_acceleration_button.tsx

View workflow job for this annotation

GitHub Actions / Lint 

React Hook useEffect has missing dependencies: 'resetFlyout' and 'setToast'. Either include them or remove the dependency array. If 'resetFlyout' changes too often, find the parent component that defines it and wrap that definition in useCallback

return (
<EuiButton onClick={createAcceleration} fill isLoading={isLoading}>
Expand Down
Loading