feat: move clinvar to new pattern #5100
Conversation
| if existing_version_obj and ClinvarAllVariantsSnvIndel.objects.filter(version=existing_version_obj.version).exists(): | ||
| clinvar_run_sql( | ||
| Template(f"ALTER TABLE `$reference_genome/$dataset_type/clinvar_all_variants` DROP PARTITION '{new_version}';") | ||
| Template(f"ALTER TABLE `$reference_genome/$dataset_type/reference_data/clinvar/all_variants` DROP PARTITION '{new_version}';") |
There was a problem hiding this comment.
You can actually get the table name off of the model's meta field which would make this code more resilient - you just use ClinvarAllVariantsSnvIndel._meta.db_table. It would also get rid of the need to template out variables
hanars
left a comment
hanars
left a comment
There was a problem hiding this comment.
This looks good, I just want to confirm that after the migration runs the new materialized views will automatically populate with the correct clinvar data without needing to run the reload clinvar command or a 'SYSTEM REFRESH VIEW
| ('GRCh38', 'SNV_INDEL'), | ||
| ('GRCh38', 'MITO'), | ||
| ]: | ||
| cursor.execute(sql.substitute(reference_genome=reference_genome, dataset_type=dataset_type)) |
There was a problem hiding this comment.
not particularly related to this PR, but this file should really be in the clickhouse_search/management/commands folder not the seqr commands folder
|
I think this is fine, but there is a potential edge case where the two materialized views we create here run concurrently upon creation; the second one (the One hacky resolution would be to add a RunPython that runs |
|
Having an empty clinvar join table at the end of this is not really an acceptable situation so we will need this to deterministically result with all the views populated. I feel like the best way to do this would be adding |
| FROM `$reference_genome/$dataset_type/reference_data/clinvar/seqr_variants` | ||
| """) | ||
|
|
||
| def build_materialized_view(reference_genome: str, dataset_type: str, materialized_view: str): |
There was a problem hiding this comment.
added a mechanism for refreshing and waiting for each view.
This reverts commit ee311c0.
No description provided.