Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add ssh tunneling to dynamic form for Database Connection UI #22689

Merged
merged 161 commits into from
Jan 20, 2023
Merged
Show file tree
Hide file tree
Changes from 155 commits
Commits
Show all changes
161 commits
Select commit Hold shift + click to select a range
830a283
save
hughhhh Oct 19, 2022
2c1e736
create migration
hughhhh Oct 20, 2022
f78df83
created schema and rename
hughhhh Oct 21, 2022
d482df4
linting
hughhhh Oct 21, 2022
9edb581
fix encrpytions
hughhhh Oct 21, 2022
da27d8f
remove map tabl
hughhhh Oct 24, 2022
773a6c8
fix linting
hughhhh Oct 25, 2022
2f2dda2
add constraint
hughhhh Oct 25, 2022
fd0d7f2
add fk to migration
hughhhh Oct 25, 2022
158da8d
init
hughhhh Oct 26, 2022
face73f
update all the examples
hughhhh Oct 26, 2022
95d079e
change remaining bits
hughhhh Oct 26, 2022
d5926e3
add id
hughhhh Oct 27, 2022
f7a6a41
use factory instead
hughhhh Oct 28, 2022
30e380a
Merge branch 'master' of https://github.com/apache/superset into crea…
hughhhh Oct 28, 2022
87c0d79
Merge branch 'master' into ref-get-sqla-engine-2
hughhhh Oct 28, 2022
11b240b
fix confict
hughhhh Oct 28, 2022
1bfdbda
fix conflict
hughhhh Oct 28, 2022
4146d5a
setup return value for contextmanager
hughhhh Oct 31, 2022
f8b877d
add sshtunnel pip
hughhhh Oct 31, 2022
54fc147
updates test
hughhhh Nov 1, 2022
fdc6ca3
fix linting
hughhhh Nov 2, 2022
66c0801
renaming function
hughhhh Nov 3, 2022
1f9ec5e
fix test
hughhhh Nov 5, 2022
c698cf4
Merge branch 'ref-get-sqla-engine-2' of https://github.com/apache/sup…
hughhhh Nov 7, 2022
41bd19b
add schema to test_connection api
hughhhh Nov 7, 2022
8811a99
fix get engine to return contextmanager
hughhhh Nov 7, 2022
82d7532
why
hughhhh Nov 7, 2022
1f829ac
yerp
hughhhh Nov 7, 2022
d53d116
update typing
hughhhh Nov 7, 2022
752161d
update comment
hughhhh Nov 7, 2022
8c4b081
Merge branch 'ref-get-sqla-engine-2' of https://github.com/apache/sup…
hughhhh Nov 7, 2022
1a19a97
save
hughhhh Nov 8, 2022
58b9cce
save
hughhhh Nov 8, 2022
0ac6fb1
Merge branch 'master' of https://github.com/apache/superset into ref-…
hughhhh Nov 8, 2022
31f3c1d
fix pylint
hughhhh Nov 8, 2022
a0b30e6
Merge branch 'ref-get-sqla-engine-2' of https://github.com/apache/sup…
hughhhh Nov 8, 2022
e089a8d
last one
hughhhh Nov 8, 2022
81b2f88
Merge branch 'ref-get-sqla-engine-2' of https://github.com/apache/sup…
hughhhh Nov 8, 2022
45686b7
update naming on ssh tunnel
hughhhh Nov 9, 2022
7ce5836
Merge branch 'master' into ref-get-sqla-engine-2
hughhhh Nov 9, 2022
d9c8d0d
Merge branch 'ref-get-sqla-engine-2' of https://github.com/apache/sup…
hughhhh Nov 9, 2022
ec27b80
fix renaming
hughhhh Nov 10, 2022
65e3e29
fix renaming 2
hughhhh Nov 10, 2022
9fa9db5
Merge branch 'master' of https://github.com/apache/superset into crea…
hughhhh Nov 10, 2022
1a11ff4
oops
hughhhh Nov 10, 2022
3f0dae1
fix linting errors
hughhhh Nov 10, 2022
2777807
feat(ssh_tunnel): DAO Changes for SSH Tunnel (#22120)
Antonio-RiveroMartnez Nov 15, 2022
8ed02cd
fix merge conflicts
hughhhh Nov 16, 2022
6a68147
Merge branch 'create-sshtunnelconfig-tbl' of https://github.com/apach…
hughhhh Nov 16, 2022
6bd32e8
feat(ssh_tunnel): Delete command & exceptions (#22131)
Antonio-RiveroMartnez Nov 16, 2022
8a3ee35
Merge branch 'master' of https://github.com/apache/superset into crea…
hughhhh Nov 16, 2022
bc89194
Merge branch 'create-sshtunnelconfig-tbl' of https://github.com/apach…
hughhhh Nov 16, 2022
adb9451
fix indenting for superset/databases/commands/validate.py
hughhhh Nov 17, 2022
16d960b
change tablename
hughhhh Nov 17, 2022
d2ab4a6
feat(ssh_tunnel): DELETE SSH Tunnels API (#22153)
Antonio-RiveroMartnez Nov 17, 2022
fb2acd0
Revert "feat(ssh_tunnel): DELETE SSH Tunnels API" (#22156)
hughhhh Nov 17, 2022
4d807c9
feat(ssh_tunnel): Update command & exceptions (#22132)
Antonio-RiveroMartnez Nov 17, 2022
dc0c848
forgot server_port
hughhhh Nov 17, 2022
21fcdf0
bind_port + bind_host :)
hughhhh Nov 17, 2022
68cb75f
oops
hughhhh Nov 17, 2022
44ca56b
fix linting
hughhhh Nov 17, 2022
7e1461e
feat(ssh_tunnel): SSH Tunnel updates from Code Review (#22182)
Antonio-RiveroMartnez Nov 21, 2022
92e41f1
Merge branch 'master' of https://github.com/apache/superset into crea…
hughhhh Nov 22, 2022
6c59663
feat(ssh_tunnel): Create command & exceptions (#22148)
hughhhh Nov 22, 2022
466703a
Update schemas.py
hughhhh Nov 28, 2022
554de53
Merge branch 'master' of https://github.com/apache/superset into crea…
hughhhh Nov 30, 2022
4448739
chore(ssh-tunnel): create `contextmanager` for sql.inspect (#22251)
hughhhh Nov 30, 2022
bb78055
fix lint
hughhhh Nov 30, 2022
f507385
fix migrations
hughhhh Nov 30, 2022
45aa022
Merge branch 'master' of https://github.com/apache/superset into crea…
hughhhh Dec 1, 2022
3d3b71b
Merge branch 'master' of https://github.com/apache/superset into crea…
hughhhh Dec 1, 2022
86436b6
Revert "chore(ssh-tunnel): create `contextmanager` for sql.inspect (#…
hughhhh Dec 1, 2022
54a8d7f
debugging
hughhhh Dec 1, 2022
3f6afec
fix(ssh_tunnel): Address Base PR comments from peer review (#22306)
Antonio-RiveroMartnez Dec 5, 2022
7625566
fix pre-commit
hughhhh Dec 5, 2022
0578a8e
working changes
hughhhh Dec 6, 2022
ec20429
refactor bind_host and bind_port
hughhhh Dec 6, 2022
1f57d4a
refactor create flow for temp ssh tunnels
hughhhh Dec 7, 2022
ed19a3e
remove logger
hughhhh Dec 8, 2022
852c8bb
chore(ssh_tunnel): Add extra tests to SSHTunnel commands (#22372)
Antonio-RiveroMartnez Dec 8, 2022
be5c005
add flush to allow database.id to be populated
hughhhh Dec 8, 2022
948f748
Merge branch 'create-sshtunnelconfig-tbl' of https://github.com/apach…
hughhhh Dec 8, 2022
c636ce7
make sure to use inspector with context
hughhhh Dec 9, 2022
908896f
remove id and database_id
hughhhh Dec 12, 2022
e3ef835
uselist
hughhhh Dec 12, 2022
a4b1c28
refactor bind_host and bind_port
hughhhh Dec 6, 2022
828d45c
debugging
hughhhh Dec 1, 2022
862288a
SSH Tunnel:
Antonio-RiveroMartnez Nov 22, 2022
5ca158e
SSH Tunnel:
Antonio-RiveroMartnez Nov 25, 2022
ade44d3
SSH Tunnel:
Antonio-RiveroMartnez Nov 27, 2022
bae36a0
SSH Tunnel:
Antonio-RiveroMartnez Nov 27, 2022
e3f99d2
SSH Tunnel:
Antonio-RiveroMartnez Nov 28, 2022
6641f97
SSH Tunnel:
Antonio-RiveroMartnez Nov 28, 2022
c7dc836
SSH Tunnel:
Antonio-RiveroMartnez Nov 29, 2022
89c20eb
SSH Tunnel:
Antonio-RiveroMartnez Nov 29, 2022
f4b34a2
SSH Tunnel:
Antonio-RiveroMartnez Nov 30, 2022
ce842a6
SSH Tunnel:
Antonio-RiveroMartnez Dec 1, 2022
3ae8c86
SSH Tunnel:
Antonio-RiveroMartnez Dec 1, 2022
3faf78c
SSH Tunnel:
Antonio-RiveroMartnez Dec 2, 2022
66fb789
SSH Tunnel:
Antonio-RiveroMartnez Dec 8, 2022
cc243bf
SSH Tunnel:
Antonio-RiveroMartnez Dec 8, 2022
8e910f1
SSH Tunnel:
Antonio-RiveroMartnez Dec 12, 2022
a5cf0e4
SSH Tunnel:
Antonio-RiveroMartnez Dec 12, 2022
c5c50ed
feat(ssh-tunnel): ssh manager config + feature flag (#22201)
hughhhh Dec 15, 2022
06e115b
update kwarg function name
hughhhh Dec 15, 2022
13ed50d
chore(ssh-tunnel): Move SSHManager to extensions pattern (#22433)
hughhhh Dec 16, 2022
54d51e2
add flag to indicate ssh tunneling is enabled for this engine
hughhhh Dec 16, 2022
53eaa63
Update superset/migrations/versions/2022-10-20_10-48_f3c2d8ec8595_cre…
hughhhh Dec 19, 2022
8f8faff
Update superset/migrations/versions/2022-10-20_10-48_f3c2d8ec8595_cre…
hughhhh Dec 19, 2022
607c682
fix linting
hughhhh Dec 20, 2022
367efe6
Merge branch 'create-sshtunnelconfig-tbl' into api_ssh_tunnel
Antonio-RiveroMartnez Dec 20, 2022
dffc266
SSH Tunnel:
Antonio-RiveroMartnez Dec 20, 2022
65d3fc9
SSH Tunnel:
Antonio-RiveroMartnez Dec 20, 2022
e1815da
SSH Tunnel:
Antonio-RiveroMartnez Dec 21, 2022
e16c52b
SSH Tunnel:
Antonio-RiveroMartnez Dec 21, 2022
9da6987
SSH Tunnel:
Antonio-RiveroMartnez Dec 21, 2022
110e0ba
SSH Tunnel:
Antonio-RiveroMartnez Dec 22, 2022
b328427
SSH Tunnel UI:
Antonio-RiveroMartnez Dec 22, 2022
1ea0e8b
Merge branch 'master' of https://github.com/apache/superset into crea…
hughhhh Dec 22, 2022
d5b4ad2
Merge branch 'create-sshtunnelconfig-tbl' into api_ssh_tunnel
Antonio-RiveroMartnez Dec 22, 2022
5fc7e09
Merge master into ssh_sqla_form
Antonio-RiveroMartnez Dec 22, 2022
7cc7bc8
fix requirements
hughhhh Dec 22, 2022
227029d
Merge branch 'create-sshtunnelconfig-tbl' into api_ssh_tunnel
Antonio-RiveroMartnez Dec 22, 2022
422a86e
Merge branch 'api_ssh_tunnel' into ssh_sqla_form
Antonio-RiveroMartnez Dec 22, 2022
28342a4
SSH Tunnel UI:
Antonio-RiveroMartnez Dec 22, 2022
7c539d2
Merge branch 'master' of https://github.com/apache/superset into crea…
hughhhh Jan 3, 2023
643d25e
SSH Tunnel:
Antonio-RiveroMartnez Jan 3, 2023
846815e
SSH Tunnel:
Antonio-RiveroMartnez Jan 3, 2023
394afc1
get df with get_raw_connection function
hughhhh Jan 3, 2023
052eee1
SSH Tunnel:
Antonio-RiveroMartnez Jan 3, 2023
9b09fc7
feat(ssh_tunnel): APIs for SSH Tunnels (#22199)
Antonio-RiveroMartnez Jan 3, 2023
af66b34
Merge branch 'api_ssh_tunnel' into ssh_sqla_form
Antonio-RiveroMartnez Jan 3, 2023
2bb1140
Merge branch 'create-sshtunnelconfig-tbl' into api_ssh_tunnel
Antonio-RiveroMartnez Jan 3, 2023
1170a34
Merge branch 'api_ssh_tunnel' into ssh_sqla_form
Antonio-RiveroMartnez Jan 3, 2023
857e041
SSH Tunnel:
Antonio-RiveroMartnez Jan 3, 2023
27587ec
Merge master
Antonio-RiveroMartnez Jan 3, 2023
1cb7384
SSH Tunnel UI:
Antonio-RiveroMartnez Jan 4, 2023
86db39d
Merge branch 'master' into ssh_sqla_form
Antonio-RiveroMartnez Jan 4, 2023
6a03887
SSH Tunnel UI:
Antonio-RiveroMartnez Jan 4, 2023
ff5ec7f
SSH Tunnel UI:
Antonio-RiveroMartnez Jan 4, 2023
213f57a
Merge branch 'master' into ssh_sqla_form
Antonio-RiveroMartnez Jan 4, 2023
1e0161d
Merge branch 'master' into ssh_sqla_form
Antonio-RiveroMartnez Jan 10, 2023
e394e7a
SSH Tunnel UI:
Antonio-RiveroMartnez Jan 10, 2023
a096122
SSH Tunnel UI:
Antonio-RiveroMartnez Jan 10, 2023
8df08f9
SSH Tunnel UI:
Antonio-RiveroMartnez Jan 10, 2023
ffd9a56
functionality works
hughhhh Jan 11, 2023
ec9788d
fix styling
hughhhh Jan 11, 2023
3e8ae1c
style changes
hughhhh Jan 11, 2023
fa0e932
Merge branch 'master' of https://github.com/apache/superset into hxgh…
hughhhh Jan 11, 2023
1f5f8c0
cast as bool
hughhhh Jan 11, 2023
8adb25f
casting type
hughhhh Jan 11, 2023
80628d0
add test for dynamic form
hughhhh Jan 12, 2023
f4b0b3a
add right postgres value
hughhhh Jan 17, 2023
032c938
fix test
hughhhh Jan 17, 2023
c007b25
remove logs
hughhhh Jan 17, 2023
7da78fa
rename variable
hughhhh Jan 17, 2023
02facfb
Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/i…
hughhhh Jan 18, 2023
cb6f761
Merge branch 'master' of https://github.com/apache/superset into hxgh…
hughhhh Jan 18, 2023
276802f
fix lint
hughhhh Jan 18, 2023
830b787
Merge branch 'master' of https://github.com/apache/superset into hxgh…
hughhhh Jan 19, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -1198,6 +1198,42 @@ describe('DatabaseModal', () => {
});

describe('SSH Tunnel Form interaction', () => {
test('properly interacts with SSH Tunnel form textboxes for dynamic form', async () => {
userEvent.click(
screen.getByRole('button', {
name: /postgresql/i,
}),
);
screen.logTestingPlaygroundURL();
lyndsiWilliams marked this conversation as resolved.
Show resolved Hide resolved
expect(await screen.findByText(/step 2 of 3/i)).toBeInTheDocument();
const SSHTunnelingToggle = screen.getByTestId('ssh-tunnel-switch');
userEvent.click(SSHTunnelingToggle);
const SSHTunnelServerAddressInput = screen.getByTestId(
'ssh-tunnel-server_address-input',
);
expect(SSHTunnelServerAddressInput).toHaveValue('');
userEvent.type(SSHTunnelServerAddressInput, 'localhost');
expect(SSHTunnelServerAddressInput).toHaveValue('localhost');
const SSHTunnelServerPortInput = screen.getByTestId(
'ssh-tunnel-server_port-input',
);
expect(SSHTunnelServerPortInput).toHaveValue('');
userEvent.type(SSHTunnelServerPortInput, '22');
expect(SSHTunnelServerPortInput).toHaveValue('22');
const SSHTunnelUsernameInput = screen.getByTestId(
'ssh-tunnel-username-input',
);
expect(SSHTunnelUsernameInput).toHaveValue('');
userEvent.type(SSHTunnelUsernameInput, 'test');
expect(SSHTunnelUsernameInput).toHaveValue('test');
const SSHTunnelPasswordInput = screen.getByTestId(
'ssh-tunnel-password-input',
);
expect(SSHTunnelPasswordInput).toHaveValue('');
userEvent.type(SSHTunnelPasswordInput, 'pass');
expect(SSHTunnelPasswordInput).toHaveValue('pass');
});

test('properly interacts with SSH Tunnel form textboxes', async () => {
userEvent.click(
screen.getByRole('button', {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,12 @@ const ErrorAlertContainer = styled.div`
`};
`;

const SSHTunnelContainer = styled.div`
${({ theme }) => `
padding: 0px ${theme.gridUnit * 4}px;
`};
`;

interface DatabaseModalProps {
addDangerToast: (msg: string) => void;
addSuccessToast: (msg: string) => void;
Expand Down Expand Up @@ -549,7 +555,10 @@ const DatabaseModal: FunctionComponent<DatabaseModalProps> = ({
DB.backend === db?.engine || DB.engine === db?.engine,
) as DatabaseObject
)?.engine_information?.allow_ssh_tunneling;
const sshTunneling = isFeatureEnabled(FeatureFlag.SSH_TUNNELING);
const sshTunneling =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: would be more clear to use name like: sshTunnelingEnabled or isSshTunnelingEnabled

isFeatureEnabled(FeatureFlag.SSH_TUNNELING) &&
engineAllowsSSHTunneling !== undefined &&
engineAllowsSSHTunneling;
hughhhh marked this conversation as resolved.
Show resolved Hide resolved
const hasAlert =
connectionAlert || !!(db?.engine && engineSpecificAlertMapping[db.engine]);
const useSqlAlchemyForm =
Expand Down Expand Up @@ -1287,6 +1296,37 @@ const DatabaseModal: FunctionComponent<DatabaseModalProps> = ({
});
};

const renderSSHTunnelForm = () => (
<SSHTunnelForm
isEditMode={isEditMode}
sshTunneling={sshTunneling}
db={db as DatabaseObject}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not new to this PR so maybe not in scope, but is there any validation on db to ensure it is a DatabaseObject? Forcing a coercion here and on line 1304 basically removes any useful type checking. We may want to add a type guard or other validation that db does have the structure SSHTunnelForm expects.

Copy link
Member

@Antonio-RiveroMartnez Antonio-RiveroMartnez Jan 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be wrong but, isn't this:

const [db, setDB] = useReducer<
    Reducer<Partial<DatabaseObject> | null, DBReducerActionType>
  >(dbReducer, null);

forcing db to be Partial<DatabaseObject> | null ? thus would match the type in SSHTunnelForm?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eric-briscoe i say we cut a ticket for this under tech debt so we can revisit doing a proper refactor of this component

dbFetched={dbFetched as DatabaseObject}
onSSHTunnelParametersChange={({
target,
}: {
target: HTMLInputElement | HTMLTextAreaElement;
}) =>
onChange(ActionType.parametersSSHTunnelChange, {
type: target.type,
name: target.name,
value: target.value,
})
}
setSSHTunnelLoginMethod={(method: AuthType) =>
setDB({
type: ActionType.setSSHTunnelLoginMethod,
payload: { login_method: method },
})
}
removeSSHTunnelConfig={() =>
setDB({
type: ActionType.removeSSHTunnelConfig,
})
}
/>
);

const renderCTABtns = () => (
<StyledBtns>
<Button
Expand Down Expand Up @@ -1493,36 +1533,7 @@ const DatabaseModal: FunctionComponent<DatabaseModalProps> = ({
testConnection={testConnection}
testInProgress={testInProgress}
>
{sshTunneling && engineAllowsSSHTunneling && (
<SSHTunnelForm
isEditMode={isEditMode}
sshTunneling={sshTunneling}
db={db as DatabaseObject}
dbFetched={dbFetched as DatabaseObject}
onSSHTunnelParametersChange={({
target,
}: {
target: HTMLInputElement | HTMLTextAreaElement;
}) =>
onChange(ActionType.parametersSSHTunnelChange, {
type: target.type,
name: target.name,
value: target.value,
})
}
setSSHTunnelLoginMethod={(method: AuthType) =>
setDB({
type: ActionType.setSSHTunnelLoginMethod,
payload: { login_method: method },
})
}
removeSSHTunnelConfig={() =>
setDB({
type: ActionType.removeSSHTunnelConfig,
})
}
/>
)}
{sshTunneling && renderSSHTunnelForm()}
</SqlAlchemyForm>
{isDynamic(db?.backend || db?.engine) && !isEditMode && (
<div css={(theme: SupersetTheme) => infoTooltip(theme)}>
Expand Down Expand Up @@ -1796,6 +1807,11 @@ const DatabaseModal: FunctionComponent<DatabaseModalProps> = ({
validationErrors={validationErrors}
getPlaceholder={getPlaceholder}
/>
{sshTunneling && (
<SSHTunnelContainer>
{renderSSHTunnelForm()}
</SSHTunnelContainer>
)}
<div css={(theme: SupersetTheme) => infoTooltip(theme)}>
{dbModel.engine !== Engines.GSheet && (
<>
Expand Down