Merge branch 'master' into io-refactor

.gitignore

@@ -45,11 +45,9 @@ out/
 tmp_training_data.json
 .DS_Store
 models/
-projects/
 test_models/model_*
 test_models/test_model_keyword
 test_models/test_model_pretrained_embeddings
-test_projects/test_project_*
 .mypy_cache/
 *.tar.gz
 docs/nlu/key

CHANGELOG.rst

@@ -32,6 +32,7 @@ in ``interactive learning``
 
 Changed
 -------
+- renamed all CLI parameters containing any ``_`` to use dashes ``-`` instead (GNU standard)
 - renamed ``rasa_core`` package to ``rasa.core``
 - for interactive learning only include manually annotated and ner_crf entities in nlu export
 - made ``message_id`` an additional argument to ``interpreter.parse``
@@ -40,13 +41,17 @@ Changed
 - created a common utils package ``rasa.utils`` for nlu and core, common methods like ``read_yaml`` moved there
 - removed ``--num_threads`` from run command (server will be asyncronous but
   running in a single thread)
+- removed ``--pre_load`` from run command (Rasa NLU server will just have a maximum of one model and that model will be loaded by default)
+- changed file format of a stored trained model from the Rasa NLU server to ``tar.gz``
+- ``rasa train`` uses fallback config if an invalid config is given
 
 Removed
 -------
 - removed possibility to execute ``python -m rasa_core.train`` etc. (e.g. scripts in ``rasa.core`` and ``rasa.nlu``).
   Use the CLI for rasa instead, e.g. ``rasa train core``.
 - removed ``_sklearn_numpy_warning_fix`` from the ``SklearnIntentClassifier``
 - removed Dispatcher class from core 
+- removed projects: the Rasa NLU server now has a maximum of one model at a time loaded.
 
 Fixed
 -----
@@ -57,4 +62,5 @@ Fixed
   Store
 - ``rasa nlu test`` doesn't error anymore when a test file is passed with ``-u``
 - in interactive learning: only updates entity values if user changes annotation
-- ``rasa train core`` actually uses additional arguments, such as `augmentation`
+- ``rasa train core`` actually uses additional arguments, such as ``augmentation``
+- ``rasa test`` actually considers additional arguments, such as ``e2e`` or ``successes``

docs/core/interactive_learning.rst

@@ -187,7 +187,7 @@ keep track of where you are.
 You can view the visualization at http://localhost:5005/visualization.html
 as soon as you've started interactive learning.
 
-To skip the visualization, pass ``--skip_visualization`` to the training
+To skip the visualization, pass ``--skip-visualization`` to the training
 script.
 
 .. image:: _static/images/interactive_learning_graph.gif

docs/core/responses.rst

@@ -61,7 +61,7 @@ and you can use this file like this:
 .. code-block:: bash
 
     $ rasa run core \
-       --enable_api \
+       --enable-api \
        -m examples/babi/models \
        -o out.log \
        --endpoints endpoints.yml

docs/core/server.rst

@@ -39,13 +39,13 @@ models with:
 .. code-block:: bash
 
     $ rasa run core \
-        --enable_api \
+        --enable-api \
         -m models \
         -o out.log
 
 The different parameters are:
 
-- ``--enable_api``, enables this additional API
+- ``--enable-api``, enables this additional API
 - ``-m``, which is the path to the folder containing your Rasa Core model
   and Rasa NLU model.
 - ``-o``, which is the path to the log file.
@@ -88,14 +88,14 @@ Nevertheless, there are two authentication methods built in:
 
 **Token Based Auth:**
 
-Pass in the token using ``--auth_token thisismysecret`` when starting
+Pass in the token using ``--auth-token thisismysecret`` when starting
 the server:
 
 .. code-block:: bash
 
     $ rasa run core \
-        --enable_api \
-        --auth_token thisismysecret \
+        --enable-api \
+        --auth-token thisismysecret \
         -m models \
         -o out.log
 
@@ -108,7 +108,7 @@ as a parameter:
 
 **JWT Based Auth:**
 
-Enable JWT based authentication using ``--jwt_secret thisismysecret``.
+Enable JWT based authentication using ``--jwt-secret thisismysecret``.
 Requests to the server need to contain a valid JWT token in
 the ``Authorization`` header that is signed using this secret
 and the ``HS256`` algorithm.
@@ -121,8 +121,8 @@ if the ``sender_id`` matches the user's ``username``.
 .. code-block:: bash
 
     $ rasa run core \
-        --enable_api \
-        --jwt_secret thisismysecret \
+        --enable-api \
+        --jwt-secret thisismysecret \
         -m models \
         -o out.log
 
@@ -164,7 +164,7 @@ You can also configure the http server to fetch models from another URL:
 .. code-block:: bash
 
     $ rasa run core \
-        --enable_api \
+        --enable-api \
         -m models \
         --endpoints my_endpoints.yaml \
         -o out.log

docs/migrations.rst

@@ -32,6 +32,26 @@ General
   ``rasa.nlu.test`` -> ``rasa test nlu``
   ``rasa.nlu.train`` -> ``rasa train nlu``
 
+- All script parameter names have been unified to follow the same schema.
+  Any underscores (``_``) in arguments have been replaced with dashes (``-``).
+  Due to change the following argument names changed:
+  ``--nlu_data`` -> ``--nlu-data``
+  ``--dump_stories`` -> ``--dump-stories``
+  ``--debug_plots`` -> ``--debug-plots``
+  ``--max_history`` -> ``--max-history``
+  ``--pre_load`` -> ``--pre-load``
+  ``--max_training_processes`` -> ``--max-training-processes``
+  ``--wait_time_between_pulls`` -> ``--wait-time-between-pulls``
+  ``--response_log`` -> ``--response-log``
+  ``--fail_on_prediction_errors`` -> ``--fail-on-prediction-errors``
+  ``--max_stories`` -> ``--max-stories``
+  ``--jwt_method`` -> ``--jwt-method``
+  ``--jwt_secret`` -> ``--jwt-secret``
+  ``--log_file`` -> ``--log-file``
+  ``--enable_api`` -> ``--enable-api``
+  ``--auth_token`` -> ``--auth-token``
+  ``--skip_visualization`` -> ``--skip-visualization``
+  ``--training_fraction`` -> ``--training-fraction``
 
 Script parameters
 ~~~~~~~~~~~~~~~~~

docs/nlu/config.rst

@@ -24,83 +24,14 @@ Server Configuration
 Running the server
 ------------------
 
-You can run a simple http server that handles requests using your projects with :
+You can run a simple HTTP server that handles requests using your model with:
 
 .. code-block:: bash
 
-    $ python -m rasa.nlu.server --path projects
+    $ rasa run nlu -m models
 
-The server will look for existing projects under the folder defined by
-the ``path`` parameter. By default a project will load the latest
-trained model.
-
-.. _section_http_config:
-
-Serving Multiple Apps
----------------------
-
-Depending on your choice of backend, Rasa NLU can use quite a lot of memory.
-So if you are serving multiple models in production, you want to serve these
-from the same process & avoid duplicating the memory load.
-
-.. note::
-
-    Although this saves the backend from loading the same set of word vectors twice,
-    if you have projects in multiple languages your memory usage will still be high.
-
-
-As stated previously, Rasa NLU naturally handles serving multiple apps.
-By default the server will load all projects found
-under the ``path`` directory passed at run time.
-
-Rasa NLU naturally handles serving multiple apps, by default the server will load all projects found
-under the directory specified with ``--path`` option. unless you have provide ``--pre_load`` option
-to load a specific project.
-
-.. code-block:: console
-
-    $ # This will load all projects under projects/ directory
-    $ python -m rasa.nlu.server -c config.yaml --path projects/
-
-.. code-block:: console
-
-    $ # This will load only hotels project under projects/ directory
-    $ python -m rasa.nlu.server -c config.yaml --pre_load hotels --path projects/
-
-
-The file structure under ``path directory`` is as follows:
-
-.. code-block:: text
-
-    - <path>
-     - <project_A>
-      - <model_XXXXXX>
-      - <model_XXXXXX>
-       ...
-     - <project_B>
-      - <model_XXXXXX>
-       ...
-      ...
-
-You can specify which project to use in your ``/parse`` requests:
-
-.. code-block:: console
-
-    $ curl 'localhost:5000/parse?q=hello&project=my_restaurant_search_bot'
-
-or
-
-.. code-block:: console
-
-    $ curl -XPOST localhost:5000/parse -d '{"q":"I am looking for Chinese food", "project":"my_restaurant_search_bot"}'
-
-You can also specify the model you want to use for a given project, the default used being the latest trained:
-
-.. code-block:: console
-
-    $ curl -XPOST localhost:5000/parse -d '{"q":"I am looking for Chinese food", "project":"my_restaurant_search_bot", "model":<model_XXXXXX>}'
-
-If no project is found by the server under the ``path`` directory, a ``"default"`` one will be used, using a simple fallback model.
+The server will look for existing models under the folder defined by
+the ``-m`` parameter. By default the latest trained model will be loaded.
 
 .. _server_parameters:
 

docs/nlu/dataformat.rst

@@ -306,10 +306,9 @@ To train a model with this data, pass the path to the directory to the train scr
 
 .. code-block:: console
 
-    $ python -m rasa.nlu.train \
+    $ rasa train nlu \
         --config config.yml \
-        --data nlu_data/ \
-        --path projects
+        --data nlu_data/
 
 
 .. note::

docs/nlu/docker.rst

@@ -47,8 +47,7 @@ To train a NLU model you need to mount two directories into the Docker container
             python -m rasa.nlu.train \
                 -c /app/project/<nlu configuration>.yml \
                 -d /app/project/<nlu data> \
-                -o /app/model \
-                --project <nlu project name>
+                -o /app/model
 
 Running NLU with Rasa Core
 --------------------------

docs/nlu/endpoint_configuration.rst

@@ -15,7 +15,7 @@ For example:
 
 .. code-block:: bash
 
-    python -m rasa.nlu.server \
+    rasa run nlu \
         --path <working directory of the server> \
         --endpoints <path to endpoint configuration>.yml
 

docs/nlu/evaluation.rst

@@ -50,9 +50,9 @@ If you've done this, you can see how well your model predicts the test cases usi
 
 .. code-block:: bash
 
-    python -m rasa.nlu.test \
+    rasa test nlu \
         --data data/examples/rasa/demo-rasa.json \
-        --model projects/default/model_20180323-145833
+        --model models/model_20180323-145833.tar.gz
 
 Where the ``--data`` argument points to your test data, and ``--model`` points to your trained model.
 
@@ -64,7 +64,7 @@ To do this, run the evaluation script with the ``--mode crossvalidation`` flag:
 
 .. code-block:: bash
 
-    python -m rasa.nlu.test \
+    rasa test nlu \
         --data data/examples/rasa/demo-rasa.json \
         --config sample_configs/config_pretrained_embeddings_spacy.yml \
         --mode crossvalidation