Skip to content

[FEATURE]: Provide option to write output and quarantine data to tables as defined in the config #167

@mwojtyczka

Description

@mwojtyczka

Is there an existing issue for this?

  • I have searched the existing issues

Problem statement

We are currently not utilizing the output_table and quarantine_table specified in the config, to actually write any data out. It's up to the user to persist the data after quality checking. We only use quarantine_table in the dashboard but it's up to the user to write the data out.

Note: This is only relevant if DQX is installed in the workspace and not used purely as a library.

Proposed Solution

Provide an option to write the data out to tables as per output_table and quarantine_table field in the config after quality checking.

Additional Context

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions