Open
Description
Is there an existing issue for this?
- I have searched the existing issues
Problem statement
We are currently not utilizing the output_table
and quarantine_table
specified in the config, to actually write any data out. It's up to the user to persist the data after quality checking. We only use quarantine_table
in the dashboard but it's up to the user to write the data out.
Note: This is only relevant if DQX is installed in the workspace and not used purely as a library.
Proposed Solution
Provide an option to write the data out to tables as per output_table
and quarantine_table
field in the config after quality checking.
Additional Context
No response
Activity