Merge pull request #61 from AuReMe/fix-issue-22

Fix issue 22

.github/workflows/pythonpackage.yml

@@ -32,7 +32,7 @@ jobs:
     strategy:
       matrix:
         platform: [ubuntu-latest, macos-latest, windows-latest]
-        python-version: [3.7, 3.8]
+        python-version: [3.8, 3.9]
     runs-on: ${{ matrix.platform }}
 
     steps:

.github/workflows/pythonpublish.yml

@@ -17,16 +17,16 @@ jobs:
     - name: Set up Python
       uses: actions/setup-python@v1
       with:
-        python-version: '3.7'
+        python-version: '3.8'
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        pip install setuptools wheel twine
+        pip install setuptools wheel twine build
         pip install -r requirements.txt
     - name: Build and publish
       env:
         TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
         TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
       run: |
-        python setup.py sdist bdist_wheel
+        python -m build
         twine upload dist/*

.readthedocs.yml

@@ -5,17 +5,21 @@
 # Required
 version: 2
 
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.12"
+
 # Build documentation in the docs/ directory with Sphinx
 sphinx:
   configuration: docs/conf.py
 
-# Use conda to install biopython (requiring numpy, a C dependent package).
-conda:
-  environment: docs/environment.yml
+# Create pdf
+formats:
+   - pdf
 
 # Optionally set the version of Python and requirements required to build your docs
 python:
-  version: 3.6
   install:
     - requirements: docs/requirements.txt
     - requirements: requirements.txt

CHANGELOG.md

@@ -1,5 +1,43 @@
 # Changelog
 
+# Metage2Metabo v1.6.0 (2024-02-28)
+
+WARNING: change for individual and community scopes:
+* for individual scope: only seeds that are producible, ie associated with activated reactions are now shown in the results. Additional information on producible, non-producible and absent seeds are available in the json output. This requires MeneTools version >= `3.4.0`.
+* for community scope: use of `miscoto focus` for computation in order to retrieve the metabolite producers in community. This requires MeneTools version >= `3.2.0`.
+
+## Add
+
+* Test in m2m_analysis to check if combination of powergraph predicted by bubbletools is the same as the ones found in minimal communities.
+* Creation of boolean equation summarizing the powergraph if it is simple enough.
+* New option `--target-com-scope` to use all the community scope as targets for minimal community prediction (issue #21).
+* Add a function to modify xml created by Pathway Tools (issue #60).
+* Troubleshooting page in readthedocs (issue #24 and #25).
+* Metadata json file created when using command line (both for `m2m` and `m2m_analysis`).
+* Test for host for m2m metacom.
+
+## Fix
+
+* Fix potential error when reading taxon_id file (issue #22).
+* Fix issue with etree in reconstruction.
+* Issues with readthedocs.
+* GitHub Actions not working.
+
+## Modify
+
+* Exit m2m_analysis when there are unproducible targets (issue #23).
+* Check forbidden characters during targets file creation.
+* Do not allow abbreviation for argparse.
+* Remove unused dependency.
+* First step in replacing `pkg_resources` (which will become deprecated in the future) with `importlib.metadata` or import of \_\_version\_\_.
+* Move from `setup.py`/`setup.cfg` to `pyproject.toml`.
+* Update tutorial.
+* Update docs and readme.
+* Update license year and affiliation.
+* Move support for CI to Python 3.8 and 3.9.
+* Remvoe import in `__init__.py` to avoid loading m2m_analysis dependencies when using only m2m.
+* Avoid using os.unlink() function on Windows as it can lead to Permission Error.
+
 # Metage2Metabo v1.5.4 (2023-05-25)
 
 ## Fix

README.md

@@ -1,6 +1,7 @@
-[![PyPI version](https://img.shields.io/pypi/v/metage2metabo.svg)](https://pypi.org/project/Metage2Metabo/) [![GitHub license](https://img.shields.io/github/license/AuReMe/metage2metabo.svg)](https://github.com/AuReMe/metage2metabo/blob/master/LICENSE) [![Actions Status](https://github.com/AuReMe/metage2metabo/workflows/Python%20package/badge.svg)](https://github.com/AuReMe/metage2metabo/actions) [![Documentation Status](https://readthedocs.org/projects/metage2metabo/badge/?version=latest)](https://metage2metabo.readthedocs.io/en/latest/?badge=latest) [![](https://img.shields.io/badge/doi-10.7554/eLife.61968-blueviolet.svg)](https://doi.org/10.7554/eLife.61968)
+[![PyPI version](https://img.shields.io/pypi/v/metage2metabo.svg)](https://pypi.org/project/Metage2Metabo/) [![GitHub license](https://img.shields.io/github/license/AuReMe/metage2metabo.svg)](https://github.com/AuReMe/metage2metabo/blob/master/LICENSE) [![Actions Status](https://github.com/AuReMe/metage2metabo/actions/workflows/pythonpackage.yml/badge.svg)](https://github.com/AuReMe/metage2metabo/actions/workflows/pythonpackage.yml) [![Documentation Status](https://readthedocs.org/projects/metage2metabo/badge/?version=latest)](https://metage2metabo.readthedocs.io/en/latest/?badge=latest) [![](https://img.shields.io/badge/doi-10.7554/eLife.61968-blueviolet.svg)](https://doi.org/10.7554/eLife.61968)
+
 # M2M - metage2metabo
-Metage2metabo is a Python3 (Python >= 3.7, tested with 3.7 and 3.8) tool to perform graph-based metabolic analysis starting from annotated genomes (**reference genomes or metagenome-assembled genomes**). It uses *Pathway Tools* in a automatic and parallel way to **reconstruct metabolic networks** for a large number of genomes. The obtained metabolic networks are then **analyzed individually and collectively** in order to get the **added value of metabolic cooperation in microbiota over individual metabolism** and to **identify and screen interesting organisms** among all.
+Metage2metabo is a Python3 (Python >= 3.8, tested with 3.8 and 3.9) tool to perform graph-based metabolic analysis starting from annotated genomes (**reference genomes or metagenome-assembled genomes**). It uses *Pathway Tools* in a automatic and parallel way to **reconstruct metabolic networks** for a large number of genomes. The obtained metabolic networks are then **analyzed individually and collectively** in order to get the **added value of metabolic cooperation in microbiota over individual metabolism** and to **identify and screen interesting organisms** among all.
 
 
 m2m can be used as a whole workflow (``` m2m workflow ```, ``` m2m metacom ```) or steps can be performed individually (``` m2m recon ``` , ``` m2m iscope ``` , ``` m2m cscope ```, ``` m2m addedvalue ```, ``` m2m mincom ```, ``` m2m seeds ```).
@@ -61,7 +62,7 @@ A more detailled documentation is available at: [https://metage2metabo.readthedo
 
 ## Technologies
 
-Python 3 (Python 3.7 and 3.8 is tested). M2M uses a certain number of Python dependencies. An example of all these dependencies working for Ubuntu 18.04 is available in [requirements.txt](https://github.com/AuReMe/metage2metabo/blob/master/requirements.txt).
+Python 3 (Python 3.8 and 3.9 is tested). M2M uses a certain number of Python dependencies. An example of all these dependencies working for Ubuntu 18.04 is available in [requirements.txt](https://github.com/AuReMe/metage2metabo/blob/master/requirements.txt).
 They can be installed with:
 ````sh
 pip install -r requirements.txt --no-cache-dir
@@ -108,19 +109,22 @@ Also, m2m_analysis relies on other packages:
         * **Warning** 
 
         /!\ For all OS, Pathway Tools must be in ```$PATH```. 
-        On Linux and MacOS: ```export PATH=$PATH:your/install/directory/pathway-tools```. 
+        On Linux and MacOS: ```export PATH=$PATH:/your/install/directory/pathway-tools```. 
         Consider adding Pathway Tools in ```$PATH``` permanently by running
         ````sh
         echo 'export PATH="$PATH:your/install/directory/pathway-tools:"' >> ~/.bashrc
         ````
+        Then source the bashrc file:
+        ````sh
+        source ~/.bashrc
 
 * [Oog Power Graph Command line tool](https://github.com/AuReMe/metage2metabo/tree/master/external_dependencies/Oog_CommandLineTool2012) to create a svg file from the compressed graph at the end of m2m_analysis. This tool is a jar file (``Oog.jar``) so Java is needed to use it.
 
 ## Installation
 
-Developed and tested on Linux (Ubuntu, Fedora, Debian) and MacOs (version 10.14) with Python3.7.
+Developed and tested on Linux (Ubuntu, Fedora, Debian) and MacOs (version 10.14) with Python3.8.
 
-Continuous Integration using GitHub Actions with Python3.7 and Python3.8 on ubuntu-latest, macos-latest and windows-latest ([corresponding virtual environment](https://docs.github.com/en/free-pro-team@latest/actions/reference/specifications-for-github-hosted-runners#supported-runners-and-hardware-resources)).
+Continuous Integration using GitHub Actions with Python3.8 and Python3.9 on ubuntu-latest, macos-latest and windows-latest ([corresponding virtual environment](https://docs.github.com/en/free-pro-team@latest/actions/reference/specifications-for-github-hosted-runners#supported-runners-and-hardware-resources)).
 
 ### Installation with pip
 
@@ -274,7 +278,7 @@ Data used to create figures and tables are listed in the [article_data](https://
 - [diabetes_study](https://github.com/AuReMe/metage2metabo/tree/master/article_data/diabetes_study): scripts and tables to create the figures of the diabetes analyses in the article.
 
 ## Authors
-[Clémence Frioux](https://cfrioux.github.io/) and [Arnaud Belcour](https://arnaudbelcour.github.io/blog/), Univ Rennes, Inria, CNRS, IRISA, Rennes, France.
+[Clémence Frioux](https://cfrioux.github.io/) and [Arnaud Belcour](https://arnaudbelcour.github.io/blog/), `Univ Bordeaux, Inria, INRAE, Bordeaux, France`, `Univ Grenoble Alpes, Inria, Grenoble, France` and `Univ Rennes, Inria, CNRS, IRISA, Rennes, France`.
 
 ## Acknowledgement
 People of Pathway Tools (SRI International) for their help integrating Pathway Tools with command line and multiprocessing in the [mpwt](https://github.com/AuReMe/mpwt) package, used in M2M.

docs/conf.py

@@ -68,7 +68,7 @@
 #
 # This is also used if you do content translation via gettext catalogs.
 # Usually you set "language" from the command line for these cases.
-language = None
+language = 'en'
 
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.

docs/index.rst

@@ -37,6 +37,7 @@ Documentation of M2M
     suppinformation
     api
     m2m_analysis
+    troubleshooting
 
 
 Additional features
@@ -53,7 +54,7 @@ M2M relies on packages that can also be used independantly with more features an
 Authors
 =======
 
-`Clémence Frioux <https://cfrioux.github.io/>`__ and `Arnaud Belcour <https://arnaudbelcour.github.io/blog/>`__, Univ Bordeaux, Inria, INRAE, Bordeaux, France and Univ Rennes, Inria, CNRS, IRISA, Rennes, France.
+`Clémence Frioux <https://cfrioux.github.io/>`__ and `Arnaud Belcour <https://arnaudbelcour.github.io/blog/>`__, `Univ Bordeaux, Inria, INRAE, Bordeaux, France`, `Univ Grenoble Alpes, Inria, Grenoble, France` and `Univ Rennes, Inria, CNRS, IRISA, Rennes, France`.
 
 Acknowledgement
 ===============

docs/install.rst

@@ -49,13 +49,19 @@ m2m uses Pathway Tools for the reconstruction of draft metabolic networks from a
 
         /!\\ For all OS, Pathway-Tools must be in ``$PATH``.
 
-        On Linux and MacOS: ``export PATH=$PATH:your/install/directory/pathway-tools``.
+        On Linux and MacOS: ``export PATH=$PATH:/your/install/directory/pathway-tools``.
 
         Consider adding Pathway Tools in ``$PATH`` permanently by running
 
         .. code:: sh
 
-            echo 'export PATH="$PATH:your/install/directory/pathway-tools:"' >> ~/.bashrc
+            echo 'export PATH="$PATH:/your/install/directory/pathway-tools:"' >> ~/.bashrc
+
+        Then source bashrc file:
+
+        .. code:: sh
+
+            source ~/.bashrc
 
 m2m relies on several Python packages:
 
@@ -69,9 +75,9 @@ m2m relies on several Python packages:
 
 A list of Python dependencies is available in `requirements.txt <https://github.com/AuReMe/metage2metabo/blob/master/requirements.txt>`__
 
-Developed and tested on Linux (Ubuntu, Fedora, Debian) and MacOs (version 10.14) with Python3.6.
+Developed and tested on Linux (Ubuntu, Fedora, Debian) and MacOs (version 10.14) with Python3.8.
 
-Continuous Integration using GitHub Actions with Python3.6 and Python3.7 on ubuntu-latest, macos-latest and windows-latest (`corresponding virtual environment <https://docs.github.com/en/free-pro-team@latest/actions/reference/specifications-for-github-hosted-runners#supported-runners-and-hardware-resources>`__).
+Continuous Integration using GitHub Actions with Python3.8 and Python3.9 on ubuntu-latest, macos-latest and windows-latest (`corresponding virtual environment <https://docs.github.com/en/free-pro-team@latest/actions/reference/specifications-for-github-hosted-runners#supported-runners-and-hardware-resources>`__).
 
 Installation with pip
 =====================

docs/m2m_analysis.rst

@@ -515,7 +515,7 @@ Solution graph
 The solution graph is stored in a gml file (``gml/*.gml``). The nodes of the graph are species occuring in minimal solutions. An edge is created between two nodes when the two nodes are in the same minimal solutions.
 
 Compressed solution graph (power graph)
-++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++
 
 The previous solution graph is then compressed into a power graph using PowerGrASP. The result file is bbl file (``bbl/*.bbl``).
 

docs/troubleshooting.rst

@@ -0,0 +1,71 @@
+===============
+Troubleshooting
+===============
+
+Several potential fix for issues we encounter.
+
+`m2m_analysis`: "Can't connect to X11 window server using..."
+-------------------------------------------------------------
+
+Issue encountered when using `m2m_analysis` on a HPC:
+
+::
+    
+    <II> Create SVG ... Exception in thread "main" java.awt.AWTError: Can't connect to X11 window server using ':0' as the value of the DISPLAY variable.
+        at sun.awt.X11GraphicsEnvironment.initDisplay(Native Method)
+        at sun.awt.X11GraphicsEnvironment.access$200(X11GraphicsEnvironment.java:65)
+        at sun.awt.X11GraphicsEnvironment$1.run(X11GraphicsEnvironment.java:115)
+        at java.security.AccessController.doPrivileged(Native Method)
+        at sun.awt.X11GraphicsEnvironment.<clinit>(X11GraphicsEnvironment.java:74)
+        at java.lang.Class.forName0(Native Method)
+        at java.lang.Class.forName(Class.java:264)
+        at java.awt.GraphicsEnvironment.createGE(GraphicsEnvironment.java:103)
+        at java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment(GraphicsEnvironment.java:82)
+        at java.awt.image.BufferedImage.createGraphics(BufferedImage.java:1181)
+        at org.apache.batik.svggen.SVGGraphics2D.<init>(Unknown Source)
+        at org.apache.batik.svggen.SVGGraphics2D.<init>(Unknown Source)
+        at org.mattlab.eaglevista.ui.GraphImageFactory.getSVGGraphicsOfGraph(GraphImageFactory.java:157)
+        at org.mattlab.eaglevista.ui.GraphImageFactory.createImage(GraphImageFactory.java:80)
+        at org.mattlab.eaglevista.ui.EvCommandLineInterpreter.runImpl(EvCommandLineInterpreter.java:466)
+        at org.mattlab.eaglevista.ui.EvCommandLineInterpreter.main(EvCommandLineInterpreter.java:81)
+
+
+One potential fix is to unset DISPLAY before launching m2m:
+
+::
+
+    unset DISPLAY
+    m2m_analysys workflow...
+
+
+UnicodeDecodeError when running m2m_analysis with Singularity
+-------------------------------------------------------------
+
+Error encountered when using `m2m_analysis` in a Singularity:
+
+::
+
+    ######### Creation of the powergraph website accessible at results_m2m_analysis_scfa/html/metabolic_targets_scfa #########
+    Traceback (most recent call last):
+    File "/usr/local/bin/m2m_analysis", line 8, in <module>
+        sys.exit(main())
+    File "/usr/local/lib/python3.6/dist-packages/metage2metabo/__main_analysis__.py", line 289, in main
+        args.oog, new_arg_modelhost, args.level)
+    File "/usr/local/lib/python3.6/dist-packages/metage2metabo/__main_analysis__.py", line 303, in main_analysis_workflow
+        run_analysis_workflow(*allargs)
+    File "/usr/local/lib/python3.6/dist-packages/metage2metabo/m2m_analysis/m2m_analysis_workflow.py", line 48, in run_analysis_workflow
+        powergraph_analysis(gml_output, output_dir, oog_jar, taxon_file, taxonomy_level)
+    File "/usr/local/lib/python3.6/dist-packages/metage2metabo/m2m_analysis/graph_compression.py", line 174, in powergraph_analysis
+        merge_html_css_js(html_target +'_taxon', output_html_merged)
+    File "/usr/local/lib/python3.6/dist-packages/metage2metabo/m2m_analysis/graph_compression.py", line 473, in merge_html_css_js
+        cytoscape_min_js_str =  input_cytoscape_min_js.read()
+    File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
+        return codecs.ascii_decode(input, self.errors)[0]
+    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 234868: ordinal not in range(128)
+
+This is an issue with the encodign standard used. A solution is to set the local codec to `LC_ALL=C.UTF-8`:
+
+::
+
+    LC_ALL=C.UTF-8 && srun singularity exec -B....
+