TensorFlow Model Analysis (TFMA) is a library for evaluating TensorFlow models. It allows users to evaluate their models on large amounts of data in a distributed fashion, using the same metrics defined in their trainer. These metrics can also be computed over different slices of data, and the results can be visualised in Jupyter Notebooks.
TFMA may introduce backwards incompatible changes before version 1.0.
The easiest and recommended way to install TFMA is with the PyPI package.
pip install tensorflow-model-analysis
Currently TFMA requires that TensorFlow be installed but does not have an explicit dependency on TensorFlow as a package. See TensorFlow documentation for more information on installing TensorFlow.
To enable TFMA visualization in Jupyter Notebook, run1:
jupyter nbextension enable --py widgetsnbextension
jupyter nbextension install --py --symlink tensorflow_model_analysis
jupyter nbextension enable --py tensorflow_model_analysis
TFMA requires Apache Beam to run distributed pipelines. Apache Beam runs in local mode by default, and can also run in distributed mode using Google Cloud Dataflow. TFMA is designed to be extensible to other Apache Beam runners.
For instructions on using TFMA, see the getting started guide.
This is a table of versions known to be compatible with each other, based on our testing framework. Other combinations may also work, but are untested.
| tensorflow-model-analysis | tensorflow | apache-beam[gcp] |
|---|---|---|
| GitHub master | 1.7 | 2.4.0 |
| 0.6.0 | 1.6 | 2.4.0 |
1 If Jupyter is installed in your home directory, add --user for
all commands; if Jupyter is installed in root or virtualenv is used,
--sys-prefix might be needed.
Please direct any questions about working with TFMA to Stack Overflow using the tensorflow-model-analysis tag.