ModuleNotFoundError: No module named 'testfixtures'

[bjpcjp]

Problem description

• running the following code in /docs/notebooks/howtos/run_doc2vec_imdb.ipynb:
  from gensim.test.test_doc2vec import ConcatenatedDoc2Vec

• A quick review of /gensim/test/test_doc2vec.py shows the offending import on line#21. I've tried replacing the line with from fixtures import log_capture, but with no improvement.

Steps/code/corpus to reproduce

run run_doc2vec_imdb.ipynb

Versions

Linux-4.15.0-91-generic-x86_64-with-Ubuntu-18.04-bionic
Python 3.6.9 (default, Jul 3 2019, 15:36:16)
[GCC 5.4.0 20160609]
NumPy 1.18.3
SciPy 1.4.1
gensim 3.8.1
FAST_VERSION 1

[gojomo]

It should be sufficient to install the testfixtures library: https://pypi.org/project/testfixtures/

[piskvorky]

@gojomo @mpenkov looks like some missing installation instructions / dependencies. How come this notebook passes our automated doc tests?

[bjpcjp]

Fix confirmed via pip install textfixtures --user. (Could have sworn that I did this previously.) TY!

[gojomo]

I see that this /docs/notebooks/howtos/run_doc2vec_imdb.ipynb file isn't in source control; I presume it's copied there from /docs/src/auto_examples/howtos/run_doc2vec_imdb.ipynb, which may just be auto-built from /docs/src/auto_examples/howtos/run_doc2vec_imdb.py (and thus maybe the .ipynb variant shouldn't be in source-control at all?).

My guess would be whatever build-test cycle that this goes through has extra unit-test-supporting packages installed that aren't general prerequisites for gensim. This advanced-demo notebook makes use of a utility class from a testing-focused module (test_doc2vec.py), hence it succeeds there but not in general user setups (unless they've previously brought in test-config only libraries).

I'm still not a fan of the extra indirection introduced by the auto-examples Sphinx-driven notebooks in a .py-format that turns Python comments into notebook cells. Both the run duration of some advanced notebooks – the original doc2vec-IMDB version took an hour or two to run – and the useful instructional technique of including some cells that are intended to error are poorly matched to fully-automatic, every-build testing.

[mpenkov]

@gojomo @mpenkov looks like some missing installation instructions / dependencies. How come this notebook passes our automated doc tests?

We install testfixtures here. So if the user does:

pip install gensim[docs]

then that should install textfixtures and any other dependencies required by the docs.

I think the root cause of the problem may be that people are running the notebooks without setting up the environment accordingly. If we make it obvious you need to run the above pip command to get all the extras installed, then it would be a solution. WDYT?

[piskvorky]

Definitely (although [docs] downloads half the internet - see #2821 (comment)).

If possible, let's show what specific libs are needed for a notebook, so people who can't or don't want to use pip install gensim[docs] (conda instead of pip, no pyemd, no nmslib, etc) are able to install the required dependencies manually.

[mpenkov]

Where would you show this information? How would you maintain it (ensure that it is correct)?

Perhaps a single separate '[tutorials]' extra would be a better idea?

[piskvorky]

Where would you show this information?

Inside the notebook – either as a comment, or as an explicit !pip install XYZ==a.b.c code cell. Comment is probably easier and more versatile (for people who don't use pip).

How would you maintain it (ensure that it is correct)?

Not sure. [tutorials] isn't fundamentally different from [docs], both seem an overkill for any one notebook. Like for the run_doc2vec_imdb.ipynb in this ticket, apparently all that was needed was pip install testfixtures.