feat: improve ReadMe.md to include all of the related notebooks (#89)

* feat: update Readme and library requirement

* Update README.md

* Update README.md

* Update travis to publish to PyPI server.

* Update README.md

* Update README.md

* Update README.md

* Update README.md

.releaserc

@@ -7,7 +7,7 @@
     [
       "@semantic-release/exec",
       {
-        "prepareCmd": "bumpversion --allow-dirty --new-version ${nextRelease.version} patch"
+        "prepareCmd": "bumpversion --allow-dirty --current-version ${lastRelease.version} --new-version ${nextRelease.version} patch"
       }
     ],
     [

.travis.yml

@@ -33,7 +33,7 @@ deploy:
 - provider: pypi
   user: "$PYPI_USER"
   password: "$PYPI_PASSWORD"
-  server: https://test.pypi.org/legacy/
+  server: https://upload.pypi.org/legacy
   skip_cleanup: true
   on:
     python: 3.8

README.md

@@ -1,70 +1,51 @@
-# Watson Assistant Recommendation Notebooks
+# Watson Assistant Improve Notebooks
 
-This repository houses Watson Assistant notebooks with a set of data operation and visualization functions.
+This repository houses Watson Assistant Improve notebooks and the underlying assistant improve toolkit library.
 
 ## Introduction
-To help improving your Watson Assistant after you have deployed it to production, we prepared the following two Jupyter notebooks. These notebooks include practical steps for measuring, analyzing, and actively improving your virtual assistant in a continuous manner. Check out [IBM Watson Assistant Continuous Improvement Best Practices](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/raw/master/notebook/IBM%20Watson%20Assistant%20Continuous%20Improvement%20Best%20Practices.pdf) for more details.
-
-### Measure Notebook 
-The Measure notebook contains a set of automated metrics that help you monitor and understand the behavior of your system. The goal is to understand where your assistant is doing well vs where it isn’t, and to focus your improvement effort to one of the problem areas identified. 
-
-### Effectiveness Notebook 
-The Effectiveness notebook helps you understand relative performance of each intent and entity as well as the confusion between your intents. This information helps you prioritize your improvement effort. 
-
-### Logs Notebook
-The Logs notebook helps you fetch logs using Watson Assistant API.  You can fetch logs with various filters, and save them as a JSON file, or export the utterances in the logs into a CSV file.  The JSON file can be loaded into the Measure notebook.  The CSV file can be updated to Watson Assistant service for intent recommendation.  
-
-Alternatively, you can run python scripts [`fetch_logs`](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/src/main/python/fetch_logs.py) and [`export_csv_for_intent_recommendation`](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/src/main/python/export_csv_for_intent_recommendation.py) to fetch logs and export them to [intent recommendation CSV](https://cloud.ibm.com/docs/assistant?topic=assistant-intent-recommendations#intent-recommendations-data-resources), respectively.  Run `python get_logs -h` and `python export_csv_for_intent_recommendation.py -h` for usage.  For example, to generate intent recommendation CSV from logs:
-
-- Fetch logs by keeping first user utterances in conversations with confidence `response.intents::confidence` between 0.1 and 0.6, and save to JSON file `OUTPUT_JSON_FILE`
-  ```angular2
-  python src/main/python/fetch_logs.py \
-  --url URL --version VERSION --apikey API_KEY --skill_id SKILL_ID --assistant_id ASSISTANT_ID \
-  --filters "language::en" "request.context.system.dialog_turn_counter::1" "response.intents:confidence<0.6" "response.intents:confidence>0.1" \
-  --output_json OUTPUT_JSON_FILE
-  ```
-- Take the fetch logs in `OUTPUT_JSON_FILE`, filter out utterances that is either too short (less than 3 tokens) or too long (more than 20 tokens), remove duplicates, and export them to a CSV file `OUTPUT_CSV_FILE` for intent recommendation
-  ```angular2
-  python src/main/python/export_csv_for_intent_recommendation.py \
-  --input_json OUTPUT_JSON_FILE --output_csv OUTPUT_CSV_FILE \
-  --deduplicate --min_length 3 --max_length 20
-  ```
-  The CSV file `OUTPUT_CSV_FILE` can be used as [intent recommendation data source](https://cloud.ibm.com/docs/assistant?topic=assistant-intent-recommendations#intent-recommendations-data-resources).
+To help improving your Watson Assistant after you have deployed it to production, we prepared the following Jupyter notebooks. These notebooks include practical steps for measuring, analyzing, and actively improving your assistant in a continuous manner. Check out [IBM Watson Assistant Continuous Improvement Best Practices](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/raw/master/notebook/IBM%20Watson%20Assistant%20Continuous%20Improvement%20Best%20Practices.pdf) for more details.
 
-## Getting Started
+- __Measure notebook__ contains a set of automated metrics that help you monitor and understand the behavior of your system. The goal is to understand where your assistant is doing well vs where it isn’t, and to focus your improvement effort to one of the problem areas identified. 
 
-We provide two versions of the above notebooks Jupyter Notebook and [Watson Studio Notebook](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/notebooks-parent.html).
+- __Effectiveness notebook__ helps you understand relative performance of each intent and entity as well as the confusion between your intents. This information helps you prioritize your improvement effort. 
 
-- _**Jupyter Notebook**_
+- __Customer Effort notebook__ helps you measure and analyze the performance improvement after enabling the [Disambiguation](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-runtime#dialog-runtime-disambiguation) and [Autolearning](https://cloud.ibm.com/docs/assistant?topic=assistant-autolearn) features
 
-    1. Install Jupyter Notebook, see [Jupyter/IPython Notebook Quick Start Guide](https://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/install.html) for more details.
+- __Logs notebook__ helps you fetch logs using Watson Assistant API.  You can fetch logs with various filters, and save them as a JSON file, or export the utterances in the logs into a CSV file.  The JSON file can be loaded into the Measure notebook.  The CSV file can be used for [intent recommendation service](https://cloud.ibm.com/docs/assistant?topic=assistant-intent-recommendations#intent-recommendations-get-intent-recommendations-task). Alternatively, you can run python scripts [`fetch_logs`](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/src/main/python/fetch_logs.py) and [`export_csv_for_intent_recommendation`](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/src/main/python/export_csv_for_intent_recommendation.py) to fetch logs and export them to [intent recommendation CSV](https://cloud.ibm.com/docs/assistant?topic=assistant-intent-recommendations#intent-recommendations-data-resources), respectively.  Run `python get_logs -h` and `python export_csv_for_intent_recommendation.py -h` for usage. 
 
-    2. Download the Jupyter notebooks available in this repository's [notebook](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/tree/master/notebook) directory. 
-    
-       - [Measure Notebook.ipynb](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/notebook/Measure%20Notebook.ipynb)
-       - [Effectiveness Notebook.ipynb](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/notebook/Effectiveness%20Notebook.ipynb)
-       - [Logs Notebook.ipynb](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/notebook/Logs%20Notebook.ipynb)
-       
-       __Note: These notebook files are not designed for Watson Studio environment__
-       
-    3. Install dependencies with `pip install -r requirements.txt`
+- __Dialog Flow Analysis notebook__ help you assess and analyze user journeys and issues related to the dialog flow of ineffective (low quality) conversations based on production logs. Check out [Dialog Flow Analysis](https://github.com/watson-developer-cloud/assistant-dialog-flow-analysis) for more details.
+
+- __Dialog Skill Analysis notebook__ help you analyze characteristics of your data such as the number of training examples for each intent or the terms which seem to be characteristic of a specific intent. Check out [Dalog Skill Analysis](https://github.com/watson-developer-cloud/assistant-dialog-skill-analysis) for more details.
+
+## Getting Started
+
+You can either run the notebooks locally or in [IBM Watson Studio](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/notebooks-parent.html).
+
+- **Run locally**
 
+    1. Install Jupyter Notebook, see [Jupyter/IPython Notebook Quick Start Guide](https://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/install.html) for more details.
+    2. Download the Jupyter notebooks available in this repository's [notebook](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/tree/master/notebook) directory. __Note: These notebook files are not designed for Watson Studio environment__
+    3. Start jupyter server `jupyter notebook`
     4. Follow the instructions in each of the notebooks. Be sure to add your Watson Assistant credentials if necessary.
 
-- _**Watson Studio Notebook**_
-
-    1. Create a Watson Studio account.
+- **Run in Watson Studio**
 
+    1. Create a Watson Studio account.    
        Sign up in [Watson Studio](https://www.ibm.com/cloud/watson-studio), or use an existing account. Lite plan is free to use.
 
-    2. Create a new project and add a Cloud Object Storage (COS) account.
-
+    2. Create a new project and add a Cloud Object Storage (COS) account.    
        For more information regarding COS plans, see [Pricing](https://www.ibm.com/cloud-computing/bluemix/pricing-object-storage).
 
     3. Copy [Measure](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/133dfc4cd1480bbe4eaa78d3f635e568) or  [Effectiveness](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/133dfc4cd1480bbe4eaa78d3f636921c) notebook from Watson Studio community into your project.
 
     4. Follow the instructions in each notebook to add project tokens and Watson Assistant credentials if necessary.
 
-## License
+## Guides
+* Learn more about our measure and effectiveness notebook on Medium: [Continuously Improve Your Watson Assistant with Jupyter Notebooks](https://medium.com/ibm-watson/continuously-improve-your-watson-assistant-with-jupiter-notebooks-60231df4f01f)
 
+## Contributing 
+See [CONTRIBUTING.md](CONTRIBUTING.md) for more details on how to contribute
+
+## License
 This library is licensed under the [Apache 2.0 license](http://www.apache.org/licenses/LICENSE-2.0).
+

notebook/README.md

@@ -1,12 +1,35 @@
-Below are descriptions for the contents in this directory.
+## Table of Contents
 
-#### Effectiveness Notebook.ipynb ####
-- A Jupytor notebook file. Effectiveness Notebook helps you understand relative performance of each intent and entity as well as the confusion between your intents. This information helps you prioritize your improvement effort.
+#### `Effectiveness Notebook.ipynb` ####
+- A Jupytor notebook file. Effectiveness notebook helps you understand relative performance of each intent and entity as well as the confusion between your intents. This information helps you prioritize your improvement effort.
 
-#### Measure Notebook.ipynb ####
-- A Jupytor notebook file. Measure Jupyter Notebook contains a set of automated metrics that help you monitor and understand the behavior of your system. The goal is to understand where your assistant is doing well vs where it isn’t, and to focus your improvement effort to one of the problem areas identified.
+#### `Measure Notebook.ipynb` ####
+- A Jupytor notebook file. Measure notebook contains a set of automated metrics that help you monitor and understand the behavior of your system. The goal is to understand where your assistant is doing well vs where it isn’t, and to focus your improvement effort to one of the problem areas identified.
 
-#### IBM Watson Assistant Continuous Improvement Best Practices.pdf ####
+#### `Logs Notebook.ipynb` ####
+- A Jupytor notebook file. Logs notebook helps you fetch logs using Watson Assistant API.  You can fetch logs with various filters, and save them as a JSON file, or export the utterances in the logs into a CSV file.  The JSON file can be loaded into the Measure notebook.  The CSV file can be updated to Watson Assistant service for intent recommendation. Alternatively, you can run python scripts [`fetch_logs`](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/src/main/python/fetch_logs.py) and [`export_csv_for_intent_recommendation`](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/src/main/python/export_csv_for_intent_recommendation.py) to fetch logs and export them to [intent recommendation CSV](https://cloud.ibm.com/docs/assistant?topic=assistant-intent-recommendations#intent-recommendations-data-resources), respectively.  Run `python get_logs -h` and `python export_csv_for_intent_recommendation.py -h` for usage.  For example, to generate intent recommendation CSV from logs:
+
+```
+ # Fetch logs by keeping first user utterances in conversations with confidence `response.intents::confidence` between 0.1 and 0.6, and save to JSON file `OUTPUT_JSON_FILE`
+
+  python src/main/python/fetch_logs.py \
+  --url URL --version VERSION --apikey API_KEY --skill_id SKILL_ID --assistant_id ASSISTANT_ID \
+  --filters "language::en" "request.context.system.dialog_turn_counter::1" "response.intents:confidence<0.6" "response.intents:confidence>0.1" \
+  --output_json OUTPUT_JSON_FILE
+  ```
+```
+# Take the fetch logs in `OUTPUT_JSON_FILE`, filter out utterances that is either too short (less than 3 tokens) or too long (more than 20 tokens), remove duplicates, and export them to a CSV file `OUTPUT_CSV_FILE` for intent recommendation
+  
+  python src/main/python/export_csv_for_intent_recommendation.py \
+  --input_json OUTPUT_JSON_FILE --output_csv OUTPUT_CSV_FILE \
+  --deduplicate --min_length 3 --max_length 20
+  ```
+
+#### `Customer Effort Notebook.ipynb` ####
+- A Jupytor notebook file. Customer Effort notebook helps you measure and analyze the performance improvement after enabling the [Disambiguation](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-runtime#dialog-runtime-disambiguation) and [Autolearning](https://cloud.ibm.com/docs/assistant?topic=assistant-autolearn) features.
+
+#### `IBM Watson Assistant Continuous Improvement Best Practices.pdf` ####
+- IBM Watson Assistant Continuous Improvement Best Practices document.
 
 #### `data` ####
 - A folder contains an example workspace with sample logs and an annotated log file for demonstration in notebooks. 

setup.py

@@ -51,14 +51,14 @@ def read_md(f):
     package_dir={'': 'src'},
     packages=setuptools.find_packages('src'),
     install_requires=[
-        'pandas==1.0.1',
+        'pandas>=1.0.3',
         'bokeh==2.0.0',
         'tqdm==4.43.0',
         'scikit-learn>=0.21.3',
         'matplotlib==3.2.1',
         'XlsxWriter==1.2.8',
         'ibm-watson>=4.3.0',
-        'numpy==1.18.2',
+        'numpy>=1.18.2',
         'requests>=2.18.4',
         'scikit-learn==0.22.2.post1',
         'xlrd==1.2.0'