Skip to content

2022.11 #3223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Nov 18, 2022
Merged

2022.11 #3223

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# Qiita changelog

Version 2022.11
---------------

* Adding new Woltka v0.1.4 and WoLr2 - WoLr1 and RS210 (updated version of rep200) are available but not default.
* Update qp-woltka so it runs faster, contribution with @wasade. Add newest improvements for woltka processing so it runs faster.
* Fix qp-qiime2, "filter features" bug that expects a "feature metadata" value from the dropdown.
* Avoid having [multiple git version](https://github.com/qiita-spots/qiita/issues/3216) commands running in the system - this in theory should fix some of the slow downs.
* Fix sample_name in per preparation sample information files.
* Now the sequencing artifacts (like per-sample-FASTQ, FASTA, etc) accept log files.
* SortMeRNA via qp-meta will generate processing logs and will be added to their artifacts - note that this will only be available in newer runs AKA you can delete the outputs of past runs and rerun it.
* Added new "Admin Sample Validation" to validate a list of sample names against what's available in a study, contributed by @sarayupai.

Version 2022.09
---------------

Expand Down
2 changes: 1 addition & 1 deletion qiita_core/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@
# The full license is in the file LICENSE, distributed with this software.
# -----------------------------------------------------------------------------

__version__ = "2022.09"
__version__ = "2022.11"
2 changes: 1 addition & 1 deletion qiita_db/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
from . import user
from . import processing_job

__version__ = "2022.09"
__version__ = "2022.11"

__all__ = ["analysis", "artifact", "archive", "base", "commands",
"environment_manager", "exceptions", "investigation", "logger",
Expand Down
2 changes: 1 addition & 1 deletion qiita_pet/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@
# The full license is in the file LICENSE, distributed with this software.
# -----------------------------------------------------------------------------

__version__ = "2022.09"
__version__ = "2022.11"
2 changes: 1 addition & 1 deletion qiita_pet/handlers/api_proxy/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
from .user import (user_jobs_get_req)
from .util import check_access, check_fp

__version__ = "2022.09"
__version__ = "2022.11"

__all__ = ['prep_template_summary_get_req', 'data_types_get_req',
'study_get_req', 'sample_template_filepaths_get_req',
Expand Down
1 change: 1 addition & 0 deletions qiita_pet/support_files/doc/source/dev/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ The following is a full list of the available developer tutorials
plugins
rest
resource_allocation
workflows

To request documentation on any developer use-cases not addressed here,
please add an issue `here <https://github.com/biocore/qiita/issues>`__.
40 changes: 40 additions & 0 deletions qiita_pet/support_files/doc/source/dev/workflows.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
.. _workflows:

.. index :: workflows

Qiita Workflows
===============

The Qiita Workflows allow developers to define full software pipelines of how a
give data type can be processed. A given Qiita Workflow will also define which
commands a user can see in the GUI and allow to add that "default" workflow to
a preparation via a single click on the GUI.

How to add a workflow?
----------------------

#. Add the workflow

- `INSERT INTO qiita.default_workflow (name)...;`


#. Add the data types that can use that workflow

- `INSERT INTO qiita.default_workflow_data_type (data_type_id, default_workflow_id) ...;`


#. Add nodes (commands) via the default_parameter_set of the commands (you are selecting
the command and their parameters at once). Note that order is not important as we are going to link (organize) them in the next point.

- `INSERT INTO qiita.default_workflow_node (default_parameter_set_id, default_workflow_id) ...;`


#. Link the nodes (via edges)

- `INSERT INTO qiita.default_workflow_edge (parent_id, child_id) ...;`


#. Link the specific output (as commands can have multiple) of the parent nodes/commands to their
children (inputs)

- `INSERT INTO qiita.default_workflow_edge_connections (default_workflow_edge_id, parent_output_id, child_input_id) ...;`
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Shotgun sequencing
------------------

Qiita currently has one active shotgun metagenomics data analysis pipeline: a per sample
bowtie2 alignment step with Woltka classification using either the WoLr1 or Rep200 databases.
bowtie2 alignment step with Woltka classification using either the WoLr1, WoLr2 (default) or RS210 databases.
Below you will find more information about each of these options.

.. note::
Expand All @@ -67,16 +67,16 @@ subsequent meta-analyses. Currently, the `fastp` command is set to autodetect ad
wetlab processing and we provide the following host references for your convenience:

- auto-detect adapters and artifacts + phix filtering: This is a `deblur artifacts <https://github.com/biocore/deblur/blob/master/deblur/support_files/artifacts.fa>`_ reference, mainly for debugging and testing
- auto-detect adapters and cheetah + phix filtering
- auto-detect adapters and cow + phix filtering
- auto-detect adapters and hamster + phix filtering
- auto-detect adapters and horse + phix filtering
- auto-detect adapters and `cheetah <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/003/709/585/GCF_003709585.1_Aci_jub_2/GCF_003709585.1_Aci_jub_2_genomic.fna.gz>`_ + phix filtering
- auto-detect adapters and `cow <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/003/205/GCA_000003205.6_Btau_5.0.1/GCA_000003205.6_Btau_5.0.1_genomic.fna.gz>`_ + phix filtering
- auto-detect adapters and `hamster <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/017/639/785/GCF_017639785.1_BCM_Maur_2.0/GCF_017639785.1_BCM_Maur_2.0_genomic.fna.gz>`_ + phix filtering
- auto-detect adapters and `horse <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/002/305/GCA_000002305.1_EquCab2.0/GCA_000002305.1_EquCab2.0_genomic.fna.gz>`_ + phix filtering
- auto-detect adapters and merge_genomes + phix filtering : is the combined genomes of a cheetah, cow, hamster, horse, human, mouse, pig, rabbit, and rat
- auto-detect adapters and mouse + phix filtering
- auto-detect adapters and pig + phix filtering
- auto-detect adapters and rabbit + phix filtering
- auto-detect adapters and rat + phix filtering
- auto-detect adapters only filtering [not recommended]
- auto-detect adapters and `mouse <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/001/635/GCF_000001635.27_GRCm39/GCF_000001635.27_GRCm39_genomic.fna.gz>`_ + phix filtering
- auto-detect adapters and `pig <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/003/025/GCF_000003025.6_Sscrofa11.1/GCF_000003025.6_Sscrofa11.1_genomic.fna.gz>`_ + phix filtering
- auto-detect adapters and `rabbit <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/003/625/GCF_000003625.3_OryCun2.0/GCF_000003625.3_OryCun2.0_genomic.fna.gz>`_ + phix filtering
- auto-detect adapters and `rat <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/001/895/GCA_000001895.4_Rnor_6.0/GCA_000001895.4_Rnor_6.0_genomic.fna.gz>`_ + phix filtering
- auto-detect adapters only filtering

Note that the command produces up to 6 output artifacts based on the aligner and database selected:

Expand All @@ -89,7 +89,7 @@ Note that the command produces up to 6 output artifacts based on the aligner and

.. note::
Woltka provides easy transformations for the "per gene Prediction table" to generate functional
profiles, `more information <https://github.com/qiyunzhu/woltka/blob/master/doc/wol.md#comparison>`
profiles, `more information <https://github.com/qiyunzhu/woltka/blob/master/doc/wol.md#comparison>`_.

Aligners
^^^^^^^^
Expand Down Expand Up @@ -127,8 +127,6 @@ Reference databases

Note that some of these are legacy option but not available for new processing.



#. WoLr2 (“Web of Life” release 2): A significant upgrade from WoLr1. The genome
pool is an even representation of microbial diversity, sampled from
non-redundant bacterial and archaeal genomes from NCBI (RefSeq and GenBank,
Expand All @@ -154,6 +152,21 @@ Note that some of these are legacy option but not available for new processing.
- Genera: 6,811
- Species: 12,258

#. RS210: Collection of reference microbial genomes sampled from the NCBI RefSeq
genome database, as of 2022-01-01. This time point corresponds to RefSeq
release 210.

- Genomes: 29,648
- Nucleotides: 926,894
- Basepairs: 111,767,286,504 (includes linkers)
- Numbers of taxonomic units:

- Archaea: 606
- Bacteria: 21,047
- Fungi: 409
- Protozoa: 93
- Viral: 7,493

#. WoLr1 ("Web of Life" release 1): An even representation of microbial diversity, selected using an prototype
selection algorithm based on the MinHash distance matrix among all non-redundant bacterial and archaeal genomes
from NCBI (RefSeq and GenBank, complete and draft), plus several genome quality control criteria. A
Expand Down
2 changes: 1 addition & 1 deletion qiita_ware/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@
# The full license is in the file LICENSE, distributed with this software.
# -----------------------------------------------------------------------------

__version__ = "2022.09"
__version__ = "2022.11"
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from setuptools import setup
from glob import glob

__version__ = "2022.09"
__version__ = "2022.11"


classes = """
Expand Down