refactor/feature: remove nextclade | add eqn to reads #187
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gordonkoehn
changed the title
There was a problem hiding this comment.
Pull Request Overview
This PR refactors the code to remove nextclade integration and add enhancements like computing eqn to reads, while also updating tests and documentation accordingly.
- Removed the nextclade-dependent translation function and its associated tests.
- Added new tests for Insertion equality and improved BAM file sorting and indexing.
- Updated documentation and development environment configuration to reflect the removal of nextclade.
Reviewed Changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/process/test_translate_align.py | Removed nextclade translation test and added TODO markers for new tests; note potential misspelling in test function names. |
| tests/process/test_interface.py | Added tests for equality comparison in Insertion and AAInsertionSet classes. |
| tests/process/test_convert.py | Introduced a new test for BAM file sorting and indexing using a monkeypatched pysam.sort. |
| tests/conftest.py | Added fixture for generating a FASTA query from a BAM file. |
| src/sr2silo/process/translate_align.py | Removed the translate_nextclade function to deprecate nextclade usage. |
| src/sr2silo/process/interface.py | Implemented eq methods for Insertion and AAInsertionSet classes. |
| src/sr2silo/process/convert.py | Added index-creation logic in bam_to_fasta_query and updated sort_and_index_bam. |
| src/sr2silo/process/init.py | Updated all with the addition of bam_to_fasta_query. |
| src/sr2silo/import_to_loculus.py | Updated comment to reflect metadata reference updates. |
| resources/sars-cov-2/README.md | Revised README comment to reflect new reference naming. |
| environments/dev-environment.yml | Removed nextclade dependency and cleaned up comments. |
| README.md | Updated instructions to remove reference to nextclade in development setup. |
Comments suppressed due to low confidence (2)
tests/process/test_translate_align.py:64
- The function name 'test_read_in_AligendReads_nuc_seq' appears to have a spelling error; consider renaming it to 'test_read_in_AlignedReads_nuc_seq' for clarity. Similar corrections may be needed for other test function names containing 'AligendReads'.
def test_read_in_AligendReads_nuc_seq():
src/sr2silo/process/convert.py:491
- The code checks for an index file with the '.bai' extension but then writes an index with the extension '.bam.bai'. Consider using a consistent extension (e.g., '.bai') to avoid potential issues with index detection.
output_bam_fp.with_suffix(".bam.bai").write_bytes(input_bam_fp.with_suffix(".bam.bai").read_bytes())
12 tasks
get insertions form Nextclade get insertions form Nextclade get insertions form Nextclade WIP get nuc sequence WIP: parsesd nextclade to read objects remove trash WIP: failing tests Fix: indexing of bam black formating WIP - got the results to compare WIP all __eq__ adding Nuc comparison WIP making Nextclade use the reference from sr2silo WIP changed sr2silo ref to nextclade - still not workign add sorting by coordinate config feature: add paired read merging on alignments (#125) * copy over the script * update by manual copy * WIP * ignore .bai index files - should be freshly generated * add smallgenomutils conda package * add read pair merger * add sam * WIP * WIP: adding checks * move to temp * move to temp * move constants * Reset wokrflows * fix test * add proper test with check fo sorting * add test coverage * up test coverage * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * lower the test coverage * lower the test coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: fix in CI for test coverage (#153) * test: test coverage in CI (#152) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fixing CI --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fixing CI test: test coverage in CI (#152) (#154) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fix CI test coverage (#155) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> backtack: the CI issues are bad (#156) * fix CI test coverage * fix aciton * backtrack on code coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: adjusting pytest coverage in CI (#159) * fix CI test coverage * fix aciton * backtrack on code coverage * fixing format of pipeline * fixing format of pipeline * fixing format of pipeline --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> feature: integrate paired-end read merging into CLI (#160) * adding sam_to_bam * add conversion tests * remove comments * specify sorting * rought workflow * add sorting by coordinate * WIP move aux functions to convert * working start-to-end with merging * black it * fix workflwotest * fix workflwotest * fix ruff * ruff ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: test coverage in CI (#152) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> history cleanup for dev --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fixing CI test: test coverage in CI (#152) (#154) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fix CI test coverage (#155) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> backtack: the CI issues are bad (#156) * fix CI test coverage * fix aciton * backtrack on code coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: adjusting pytest coverage in CI (#159) * fix CI test coverage * fix aciton * backtrack on code coverage * fixing format of pipeline * fixing format of pipeline * fixing format of pipeline --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: test coverage in CI (#152) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fix test to temp dir declutter temp dir handling in main (#166) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> add sr2silo in namespace for workflow setup (#165) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> centralize workflow config (#167) feature / fix: temporary directory handling for amino acid aligment (#179) * Introduces TMPDIR support in both main and translate_align modules * Refactors enrich_read_with_metadata into curry_read_with_metadata and updates related tests * Updates documentation and configuration files for resource requirements and CI environment handling --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> test: validate one/zero based coords (#181) * WUP * WIP tests * support for skipped char * add support for soft clipping * add support for hardclipping * adding Padding support * fix test read 02 * fix last read * move to script * note on coord system * fix black * cleanup test to tmp_path * remove trash * bam_to_fasta_query - support for soft clippings + validation * add test for create index and move test_bam_to_fasta_query to temp * update parse_translate_align text fixture after corrections * reformat ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> resolve changing base issues ram ref nextclade change ref to same as V-Pipe fix: Insertions JSON format (#183) * change insertion strings * add validator to pydantic model * update front to end tests * Update src/sr2silo/silo_read_schema.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * satisfy ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> reformat black nextclade test cannot work skip
add tests for insertions Update tests/conftest.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
gordonkoehn
force-pushed
the
feature/salvage_nextclade
branch
from
aec568f to
317485d
Compare
This was referenced Mar 27, 2025
gordonkoehn
added a commit
that referenced
this pull request
Mar 28, 2025
* feature: add paired read merging on alignments (#125) * copy over the script * update by manual copy * WIP * ignore .bai index files - should be freshly generated * add smallgenomutils conda package * add read pair merger * add sam * WIP * WIP: adding checks * move to temp * move to temp * move constants * Reset wokrflows * fix test * add proper test with check fo sorting * add test coverage * up test coverage * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * lower the test coverage * lower the test coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * test: fix in CI for test coverage (#153) * test: test coverage in CI (#152) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fixing CI --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fixing CI * test: test coverage in CI (#152) (#154) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix CI test coverage (#155) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * backtack: the CI issues are bad (#156) * fix CI test coverage * fix aciton * backtrack on code coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * test: adjusting pytest coverage in CI (#159) * fix CI test coverage * fix aciton * backtrack on code coverage * fixing format of pipeline * fixing format of pipeline * fixing format of pipeline --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * feature: integrate paired-end read merging into CLI (#160) * adding sam_to_bam * add conversion tests * remove comments * specify sorting * rought workflow * add sorting by coordinate * WIP move aux functions to convert * working start-to-end with merging * black it * fix workflwotest * fix workflwotest * fix ruff * ruff ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * history cleanup for dev --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fixing CI test: test coverage in CI (#152) (#154) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fix CI test coverage (#155) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> backtack: the CI issues are bad (#156) * fix CI test coverage * fix aciton * backtrack on code coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: adjusting pytest coverage in CI (#159) * fix CI test coverage * fix aciton * backtrack on code coverage * fixing format of pipeline * fixing format of pipeline * fixing format of pipeline --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * test: test coverage in CI (#152) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fix test to temp dir * declutter temp dir handling in main (#166) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * add sr2silo in namespace for workflow setup (#165) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * centralize workflow config (#167) * feature / fix: temporary directory handling for amino acid aligment (#179) * Introduces TMPDIR support in both main and translate_align modules * Refactors enrich_read_with_metadata into curry_read_with_metadata and updates related tests * Updates documentation and configuration files for resource requirements and CI environment handling --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * test: validate one/zero based coords (#181) * WUP * WIP tests * support for skipped char * add support for soft clipping * add support for hardclipping * adding Padding support * fix test read 02 * fix last read * move to script * note on coord system * fix black * cleanup test to tmp_path * remove trash * bam_to_fasta_query - support for soft clippings + validation * add test for create index and move test_bam_to_fasta_query to temp * update parse_translate_align text fixture after corrections * reformat ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * fix: Insertions JSON format (#183) * change insertion strings * add validator to pydantic model * update front to end tests * Update src/sr2silo/silo_read_schema.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * satisfy ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * refactor/feature: remove nextclade | add eqn to reads (#187) * layout orthogonal test * Salvage Nextclade Orth Test Changes get insertions form Nextclade get insertions form Nextclade get insertions form Nextclade WIP get nuc sequence WIP: parsesd nextclade to read objects remove trash WIP: failing tests Fix: indexing of bam black formating WIP - got the results to compare WIP all __eq__ adding Nuc comparison WIP making Nextclade use the reference from sr2silo WIP changed sr2silo ref to nextclade - still not workign add sorting by coordinate config feature: add paired read merging on alignments (#125) * copy over the script * update by manual copy * WIP * ignore .bai index files - should be freshly generated * add smallgenomutils conda package * add read pair merger * add sam * WIP * WIP: adding checks * move to temp * move to temp * move constants * Reset wokrflows * fix test * add proper test with check fo sorting * add test coverage * up test coverage * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * lower the test coverage * lower the test coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: fix in CI for test coverage (#153) * test: test coverage in CI (#152) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fixing CI --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fixing CI test: test coverage in CI (#152) (#154) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fix CI test coverage (#155) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> backtack: the CI issues are bad (#156) * fix CI test coverage * fix aciton * backtrack on code coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: adjusting pytest coverage in CI (#159) * fix CI test coverage * fix aciton * backtrack on code coverage * fixing format of pipeline * fixing format of pipeline * fixing format of pipeline --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> feature: integrate paired-end read merging into CLI (#160) * adding sam_to_bam * add conversion tests * remove comments * specify sorting * rought workflow * add sorting by coordinate * WIP move aux functions to convert * working start-to-end with merging * black it * fix workflwotest * fix workflwotest * fix ruff * ruff ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: test coverage in CI (#152) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> history cleanup for dev --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fixing CI test: test coverage in CI (#152) (#154) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fix CI test coverage (#155) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> backtack: the CI issues are bad (#156) * fix CI test coverage * fix aciton * backtrack on code coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: adjusting pytest coverage in CI (#159) * fix CI test coverage * fix aciton * backtrack on code coverage * fixing format of pipeline * fixing format of pipeline * fixing format of pipeline --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: test coverage in CI (#152) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fix test to temp dir declutter temp dir handling in main (#166) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> add sr2silo in namespace for workflow setup (#165) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> centralize workflow config (#167) feature / fix: temporary directory handling for amino acid aligment (#179) * Introduces TMPDIR support in both main and translate_align modules * Refactors enrich_read_with_metadata into curry_read_with_metadata and updates related tests * Updates documentation and configuration files for resource requirements and CI environment handling --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> test: validate one/zero based coords (#181) * WUP * WIP tests * support for skipped char * add support for soft clipping * add support for hardclipping * adding Padding support * fix test read 02 * fix last read * move to script * note on coord system * fix black * cleanup test to tmp_path * remove trash * bam_to_fasta_query - support for soft clippings + validation * add test for create index and move test_bam_to_fasta_query to temp * update parse_translate_align text fixture after corrections * reformat ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> resolve changing base issues ram ref nextclade change ref to same as V-Pipe fix: Insertions JSON format (#183) * change insertion strings * add validator to pydantic model * update front to end tests * Update src/sr2silo/silo_read_schema.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * satisfy ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> reformat black nextclade test cannot work skip * remove nextclade add tests for insertions Update tests/conftest.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * format black * double code bug * accidental removal * accidental removal --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update resources/sars-cov-2/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * remove notebooks * add option to skip paired-end read merging `skip_merge` and tests (#189) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * Add conda packaging and GitHub Actions workflows (#191) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * remove brnach protection (#188) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * Update version to 0.0.4 and prepare for conda packaging * Fix version handling in GitHub release workflow --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
gordonkoehn
added a commit
that referenced
this pull request
Apr 14, 2025
* feature: add paired read merging on alignments (#125) * copy over the script * update by manual copy * WIP * ignore .bai index files - should be freshly generated * add smallgenomutils conda package * add read pair merger * add sam * WIP * WIP: adding checks * move to temp * move to temp * move constants * Reset wokrflows * fix test * add proper test with check fo sorting * add test coverage * up test coverage * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * lower the test coverage * lower the test coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * test: fix in CI for test coverage (#153) * test: test coverage in CI (#152) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fixing CI --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fixing CI * test: test coverage in CI (#152) (#154) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix CI test coverage (#155) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * backtack: the CI issues are bad (#156) * fix CI test coverage * fix aciton * backtrack on code coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * test: adjusting pytest coverage in CI (#159) * fix CI test coverage * fix aciton * backtrack on code coverage * fixing format of pipeline * fixing format of pipeline * fixing format of pipeline --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * feature: integrate paired-end read merging into CLI (#160) * adding sam_to_bam * add conversion tests * remove comments * specify sorting * rought workflow * add sorting by coordinate * WIP move aux functions to convert * working start-to-end with merging * black it * fix workflwotest * fix workflwotest * fix ruff * ruff ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * history cleanup for dev --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fixing CI test: test coverage in CI (#152) (#154) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fix CI test coverage (#155) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> backtack: the CI issues are bad (#156) * fix CI test coverage * fix aciton * backtrack on code coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: adjusting pytest coverage in CI (#159) * fix CI test coverage * fix aciton * backtrack on code coverage * fixing format of pipeline * fixing format of pipeline * fixing format of pipeline --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * test: test coverage in CI (#152) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fix test to temp dir * declutter temp dir handling in main (#166) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * add sr2silo in namespace for workflow setup (#165) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * centralize workflow config (#167) * feature / fix: temporary directory handling for amino acid aligment (#179) * Introduces TMPDIR support in both main and translate_align modules * Refactors enrich_read_with_metadata into curry_read_with_metadata and updates related tests * Updates documentation and configuration files for resource requirements and CI environment handling --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * test: validate one/zero based coords (#181) * WUP * WIP tests * support for skipped char * add support for soft clipping * add support for hardclipping * adding Padding support * fix test read 02 * fix last read * move to script * note on coord system * fix black * cleanup test to tmp_path * remove trash * bam_to_fasta_query - support for soft clippings + validation * add test for create index and move test_bam_to_fasta_query to temp * update parse_translate_align text fixture after corrections * reformat ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * fix: Insertions JSON format (#183) * change insertion strings * add validator to pydantic model * update front to end tests * Update src/sr2silo/silo_read_schema.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * satisfy ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * refactor/feature: remove nextclade | add eqn to reads (#187) * layout orthogonal test * Salvage Nextclade Orth Test Changes get insertions form Nextclade get insertions form Nextclade get insertions form Nextclade WIP get nuc sequence WIP: parsesd nextclade to read objects remove trash WIP: failing tests Fix: indexing of bam black formating WIP - got the results to compare WIP all __eq__ adding Nuc comparison WIP making Nextclade use the reference from sr2silo WIP changed sr2silo ref to nextclade - still not workign add sorting by coordinate config feature: add paired read merging on alignments (#125) * copy over the script * update by manual copy * WIP * ignore .bai index files - should be freshly generated * add smallgenomutils conda package * add read pair merger * add sam * WIP * WIP: adding checks * move to temp * move to temp * move constants * Reset wokrflows * fix test * add proper test with check fo sorting * add test coverage * up test coverage * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * lower the test coverage * lower the test coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: fix in CI for test coverage (#153) * test: test coverage in CI (#152) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fixing CI --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fixing CI test: test coverage in CI (#152) (#154) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fix CI test coverage (#155) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> backtack: the CI issues are bad (#156) * fix CI test coverage * fix aciton * backtrack on code coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: adjusting pytest coverage in CI (#159) * fix CI test coverage * fix aciton * backtrack on code coverage * fixing format of pipeline * fixing format of pipeline * fixing format of pipeline --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> feature: integrate paired-end read merging into CLI (#160) * adding sam_to_bam * add conversion tests * remove comments * specify sorting * rought workflow * add sorting by coordinate * WIP move aux functions to convert * working start-to-end with merging * black it * fix workflwotest * fix workflwotest * fix ruff * ruff ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: test coverage in CI (#152) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> history cleanup for dev --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fixing CI test: test coverage in CI (#152) (#154) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge * Update .github/workflows/test.yml praying this is correct. --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fix CI test coverage (#155) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> backtack: the CI issues are bad (#156) * fix CI test coverage * fix aciton * backtrack on code coverage --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: adjusting pytest coverage in CI (#159) * fix CI test coverage * fix aciton * backtrack on code coverage * fixing format of pipeline * fixing format of pipeline * fixing format of pipeline --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> test: test coverage in CI (#152) * Update workflow to conditionally run coverage tests based on PR or push to main/dev branch * Add check for changed files before running coverage * remove template / badge Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> fix test to temp dir declutter temp dir handling in main (#166) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> add sr2silo in namespace for workflow setup (#165) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> centralize workflow config (#167) feature / fix: temporary directory handling for amino acid aligment (#179) * Introduces TMPDIR support in both main and translate_align modules * Refactors enrich_read_with_metadata into curry_read_with_metadata and updates related tests * Updates documentation and configuration files for resource requirements and CI environment handling --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> test: validate one/zero based coords (#181) * WUP * WIP tests * support for skipped char * add support for soft clipping * add support for hardclipping * adding Padding support * fix test read 02 * fix last read * move to script * note on coord system * fix black * cleanup test to tmp_path * remove trash * bam_to_fasta_query - support for soft clippings + validation * add test for create index and move test_bam_to_fasta_query to temp * update parse_translate_align text fixture after corrections * reformat ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> resolve changing base issues ram ref nextclade change ref to same as V-Pipe fix: Insertions JSON format (#183) * change insertion strings * add validator to pydantic model * update front to end tests * Update src/sr2silo/silo_read_schema.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * satisfy ruff --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> reformat black nextclade test cannot work skip * remove nextclade add tests for insertions Update tests/conftest.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * format black * double code bug * accidental removal * accidental removal --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update resources/sars-cov-2/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * remove notebooks * add option to skip paired-end read merging `skip_merge` and tests (#189) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * Add conda packaging and GitHub Actions workflows (#191) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * remove brnach protection (#188) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * Update version to 0.0.4 and prepare for conda packaging * Fix version handling in GitHub release workflow * GitHub Actions(deps): Bump actions/setup-python from 4 to 5 (#193) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * GitHub Actions(deps): Bump actions/checkout from 3 to 4 (#192) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * docs: specify usage of import-to-silo (#198) * validate silo insertions pydantic schema * add note to docs not to use untrimmed * add tests and allow N in insertiosn --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * fix: snakemake workflow / multiple batch ids clash (#202) * fix combination bug * fix multiple batch issue * fix config path * change path --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * fix: correct unknown / skipped char for amino acid padding (#200) * validate silo insertions pydantic schema * add note to docs not to use untrimmed * add tests and allow N in insertiosn * chang the Amio Acid Validation * changing unknown char for amino acid paddings * update expexte aligned reads fixture * fix test fixture aa insertions * Update src/sr2silo/silo_read_schema.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * update snakemake fixture --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * workflows: adding conda envs to rule (#194) * Adding conda rule support - (not tested) * remove redundant dependncies * adapt submission script --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * changing AA skipped chars in Readme (#205) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * update Readme with conda install (#206) Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> * feature : Add version tracking to 'import-to-loculus' metadata (#195) * WIP * make version info mandatory * adding version to all of test suits * update database config file * missing version * remove expected comparrions from smk rule * adjust test data * remove unused * generate index sr2silo_version * poetry dynamic versioning * adding transient dependency * install self-verisoning * adjust python version * remove brotli * adjsut * update fallback * simpligf --------- Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Gordon J. Köhn <gordon.koehn@dbsse.ethz.ch> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR salvages some good changes made in
as it was realized that nextclade is no good means of comparing against as the alignment is indeed not meant for short-reads. So there is not way to compares this actually.