AUTO docusaurus 20230817

docs/versioned_docs/version-3.x/action-server/sanic-extensions.mdx

@@ -0,0 +1,67 @@
+---
+id: sanic-extensions
+sidebar_label: Sanic Extensions
+title: Sanic Extensions
+---
+
+:::info New in 3.6
+
+You can now extend Sanic features such as middlewares, listeners, background tasks
+and additional routes.
+
+:::
+
+
+You can now create additional Sanic extensions by accessing the app object created by the action server. The hook implemented
+in the plugin package provides you access to the Sanic app object created by `rasa-sdk` when starting the action server.
+
+## Step-by-step guide on creating your own Sanic extension in rasa_sdk
+This example will show you how to create a Sanic listener using plugins.
+
+### Create the rasa_sdk_plugins package
+Create a package in your action server project which you must name `rasa_sdk_plugins`. Rasa SDK will try to instantiate this package in your project to start plugins.
+If no plugins are found, it will print a debug log that there are no plugins in your project.
+
+### Register modules containing the hooks
+Create the package `rasa_sdk_plugins` and initialize the hooks by creating an `__init__.py` file where the plugin manager will look for the module where the hooks are implemented:
+
+```
+def init_hooks(manager: pluggy.PluginManager) -> None:
+    """Initialise hooks into rasa sdk."""
+    import sys
+    import rasa_sdk_plugins.your_module
+
+    logger.info("Finding hooks")
+    manager.register(sys.modules["rasa_sdk_plugins.your_module"])
+```
+### Implement your hook
+Implement the hook `attach_sanic_app_extensions`. This hook forwards the app object created by Sanic in the `rasa_sdk` and allows you to create additional routes, middlewares, listeners and background tasks. Here's an example of this implementation that creates a listener.
+
+In your `rasa_sdk_plugins.your_module.py`:
+
+```
+from __future__ import annotations
+
+import logging
+import pluggy
+
+from asyncio import AbstractEventLoop
+from functools import partial
+
+
+logger = logging.getLogger(__name__)
+hookimpl = pluggy.HookimplMarker("rasa_sdk")
+
+
+@hookimpl  # type: ignore[misc]
+def attach_sanic_app_extensions(app: Sanic) -> None:
+    logger.info("hook called")
+    app.register_listener(
+        partial(before_server_start),
+        "before_server_start",
+    )
+
+
+async def before_server_start(app: Sanic, loop: AbstractEventLoop):
+    logger.info("BEFORE SERVER START")
+```

docs/versioned_docs/version-3.x/changelog.mdx

@@ -16,6 +16,48 @@ https://github.com/RasaHQ/rasa/tree/main/changelog/ . -->
 
 <!-- TOWNCRIER -->
 
+## [3.6.5] - 2023-08-17
+
+Rasa 3.6.5 (2023-08-17)                        
+### Improvements
+- [#12696](https://github.com/rasahq/rasa/issues/12696): Use the same session across requests in `RasaNLUHttpInterpreter`
+
+### Bugfixes
+- [#12737](https://github.com/rasahq/rasa/issues/12737): Resolve dependency incompatibility: Pin  version of `dnspython` to ==2.3.0.
+
+### Improved Documentation
+- [#12712](https://github.com/rasahq/rasa/issues/12712): Updated PII docs with new section on how to use Rasa X/Enterprise with PII management solution, and a new note on debug
+  logs being displayed after the bot message with `rasa shell`.
+
+
+## [3.6.4] - 2023-07-21
+
+Rasa 3.6.4 (2023-07-21)                        
+### Bugfixes
+- [#12575](https://github.com/rasahq/rasa/issues/12575): Extract conditional response variation and channel variation filtering logic into a separate component.
+  Enable usage of this component in the NaturalLanguageGenerator subclasses (e.g. CallbackNaturalLanguageGenerator, TemplatedNaturalLanguageGenerator).
+  Amend nlg_request_format to include a single response ID string field, instead of a list of IDs.
+
+### Improved Documentation
+- [#12663](https://github.com/rasahq/rasa/issues/12663): Updated commands with square brackets e.g (`pip install rasa[spacy]`) to use quotes (`pip install 'rasa[spacy]'`) for compatibility with zsh in docs.
+
+
+## [3.6.3] - 2023-07-20
+
+Rasa 3.6.3 (2023-07-20)                        
+### Improvements
+- [#12637](https://github.com/rasahq/rasa/issues/12637): Added a human readable component to structlog using the `event_info` key and made it the default rendered key if present.
+
+### Bugfixes
+- [#12638](https://github.com/rasahq/rasa/issues/12638): Fix the issue with the most recent model not being selected if the owner or permissions where modified on the model file.
+- [#12661](https://github.com/rasahq/rasa/issues/12661): Fixed `BlockingIOError` which occured as a result of too large data passed to strulogs.
+
+### Improved Documentation
+- [#12659](https://github.com/rasahq/rasa/issues/12659): Update action server documentation with new capability to extend Sanic features by using plugins.
+  Update rasa-sdk dependency to version 3.6.1.
+- [#12663](https://github.com/rasahq/rasa/issues/12663): Updated commands with square brackets e.g (`pip install rasa[spacy]`) to use quotes (`pip install 'rasa[spacy]'`) for compatibility with zsh in docs.
+
+
 ## [3.6.2] - 2023-07-06
 
 Rasa 3.6.2 (2023-07-06)                        

docs/versioned_docs/version-3.x/installation/environment-set-up.mdx

@@ -129,7 +129,7 @@ Rasa installations on Apple Silicon _don't_ use [Apple Metal](https://developer.
 We found that using the GPU on Apple Silicon increased training time for the
 [`DIETClassifier`](../components.mdx#dietclassifier) and [`TEDPolicy`](../policies.mdx#ted-policy) dramatically.
 You can, however, install the optional dependency to test it yourself
-or try it with other components using: `pip3 install rasa[metal]`.
+or try it with other components using: `pip3 install 'rasa[metal]'`.
 
 Currently, not all of Rasa's dependencies support Apple Silicon natively. This leads
 to the following restrictions:

docs/versioned_docs/version-3.x/installation/installing-rasa-open-source.mdx

@@ -82,7 +82,7 @@ configuration for your assistant and alert you to additional dependencies.
 If you don't mind the additional dependencies lying around, you can use
 
 ```bash
-pip3 install rasa[full]
+pip3 install 'rasa[full]'
 ```
 
 to install all needed dependencies for every configuration.
@@ -141,10 +141,21 @@ For more information on spaCy models, check out the [spaCy docs](https://spacy.i
 You can install it with the following commands:
 
 ```bash
-pip3 install rasa[spacy]
+pip3 install 'rasa[spacy]'
 python3 -m spacy download en_core_web_md
 ```
 
+:::tip Using `zsh`?
+
+In zsh, square brackets are interpreted as patterns on the command line. 
+To run commands with square brackets, you can either enclose the arguments
+with square brackets in quotes, like `pip3 install 'rasa[spacy]'`, or escape
+the square brackets using backslashes, like `pip3 install rasa\[spacy\]`.
+We recommend using the former method (`pip3 install 'rasa[spacy]'`) in our
+documentation because it works as expected across any shell
+
+:::
+
 This will install Rasa Open Source as well as spaCy and its language model
 for the English language, but many other languages are available too.
 We recommend using at least the "medium" sized models (`_md`) instead of the spaCy's
@@ -157,7 +168,7 @@ First, run
 
 ```bash
 pip3 install git+https://github.com/mit-nlp/MITIE.git
-pip3 install rasa[mitie]
+pip3 install 'rasa[mitie]'
 ```
 
 and then download the

docs/versioned_docs/version-3.x/installation/rasa-pro/installation.mdx

@@ -34,28 +34,44 @@ Installing Rasa Pro instead of Rasa Open Source will not break any existing scri
 
 ### Python Package Installation
 
-The Rasa Pro python package is named `rasa-plus`. The `rasa-plus` python packages as well as the docker containers are hosted on our GCP artifact registry.
+The Rasa Pro python package is named `rasa-plus`. The `rasa-plus` python packages as well as the Docker containers are hosted on our GCP Artifact Registry.
 As a prerequisite, you will need:
 
-- to [install](https://cloud.google.com/sdk/docs/install) and [initialize](https://cloud.google.com/sdk/docs/initializing) the Google Cloud CLI.
+- to [install](https://cloud.google.com/sdk/docs/install) the Google Cloud CLI.
 - to verify that the user or service account you are using has the required permissions to access the repository.
 
 #### Authentication Set-Up
+To authenticate you need use a service account key file provided by Rasa to authenticate with Google Cloud.
 
-In order for the package manager of choice (e.g, `pip` or `poetry`) to have the necessary credentials to authenticate with the registry, follow the steps in the [Google documentation](https://cloud.google.com/artifact-registry/docs/python/authentication#keyring).
+Authenticate with GCP using the service account key.
 
-Where required, input the following parameters:
+```sh
+gcloud auth activate-service-account --key-file=service-account.json
+```
+
+Set up keyring to allow Pip to authenticate with GCP Artifact Registry by installing keyring and then the backend that supports GCP Artifact Registry
 
 ```sh
-gcloud artifacts print-settings python \
-    --project=rasa-releases \
-    --repository=rasa-plus-py \
-    --location=europe-west3
+pip install keyring
+pip install keyrings.google-artifactregistry-auth
+```
+
+Verify that the backends have been installed correctly
+
+```sh
+keyring --list-backends
 ```
 
+The results should include `ChainerBackend` and `GooglePythonAuth`.
+
+
 #### Installing with `pip`
 
-Enter the following settings to the `.pypirc` file:
+Enter the following settings to the `.pypirc` file. This can be found:
+
+- Linux and MacOS: `$HOME/.pypirc`
+- Windows: `%USERPROFILE%\.pypirc`
+
 
 ```
 [distutils]
@@ -67,7 +83,19 @@ repository: https://europe-west3-python.pkg.dev/rasa-releases/rasa-plus-py/
 
 ```
 
-Next, add these specific settings to the pip configuration file as instructed in the [GCP authentication documentation](https://cloud.google.com/artifact-registry/docs/python/authentication#keyring-setup):
+Next, add these specific settings to the pip configuration file. The location for this depends on whether you want to update the per-user file or the file specific to a virtual environment that you are using.
+
+For the file associated with your operating system user:
+
+- Linux: `$HOME/.config/pip/pip.conf` or `$HOME/.pip/pip.conf`
+- MacOS: `/Library/Application Support/pip/pip.conf` or `$HOME/.config/pip/pip.conf`
+- Windows: `%APPDATA%\pip\pip.ini` or `%USERPROFILE%\pip\pip.ini`
+
+For virtual environments:
+
+- Linux and macOS: `$VIRTUAL_ENV/pip.conf`
+- Windows: `%VIRTUAL_ENV%\pip.ini`
+
 
 ```
 [global]
@@ -79,7 +107,7 @@ Finally, you should be able to run `pip install rasa-plus`.
 
 #### Installing with `poetry`
 
-To install `rasa-plus` with `poetry`, you will need to associate the artifact registry URL with `rasa-plus` before installing it.
+To install `rasa-plus` with `poetry`, you will need to associate the Artifact Registry URL with `rasa-plus` before installing it.
 Note that you must upgrade poetry to the latest minor (`1.2.0`) in order for `poetry` to work with the GCP authentication set-up.
 Proceed with the following steps:
 
@@ -100,20 +128,40 @@ secondary = true
 
 ### Docker Image Installation
 
-The Rasa Pro docker image is named `rasa-plus`. The docker images are hosted on our GCP artifact registry.
+The Rasa Pro Docker image is named `rasa-plus`. The Docker images are hosted on our GCP Artifact Registry.
 As a prerequisite, you will need:
 
-- to [install](https://cloud.google.com/sdk/docs/install) and [initialize](https://cloud.google.com/sdk/docs/initializing) the Google Cloud CLI.
+- to [install](https://cloud.google.com/sdk/docs/install) the Google Cloud CLI.
 - to verify that the user or service account you are using has the required permissions to access the repository.
 
-To be able to pull the docker image you need use a key file provided by Rasa to authenticate with google cloud.
+To authenticate you need use a service account key file provided by Rasa to authenticate with Google Cloud.
 
 ```bash
 gcloud auth activate-service-account --key-file=${KEYFILE}
-gcloud auth list
+gcloud auth configure-docker europe-west3-docker.pkg.dev
 docker pull europe-west3-docker.pkg.dev/rasa-releases/rasa-plus/rasa-plus
 ```
 
+### Using An Intermediate Repository
+If you are using your own intermediate repository to cache libraries or dependencies (such as Artifactory or Nexus Repository Manager), you may need to generate a set of static credentials that allow you to authenticate with GCP Artifact Registry.
+
+As a prerequisite, you will need:
+
+- to [install](https://cloud.google.com/sdk/docs/install) the Google Cloud CLI.
+- to verify that the user or service account you are using has the required permissions to access the repository.
+
+To generate your credentials, run:
+
+```sh
+gcloud artifacts print-settings python \
+    --project=rasa-releases \
+    --repository=rasa-plus-py \
+    --location=europe-west3 \
+    --json-key=service-account.json
+```
+
+Your credentials can be found in the output. The username will be `_json_key_base64` and the password will be a long, base64 encoded string.
+
 ### Runtime Configuration
 
 Rasa Pro will look for your license in the env var `RASA_PRO_LICENSE`.

docs/versioned_docs/version-3.x/installation/rasa-pro/rasa-pro-artifacts.mdx

@@ -12,7 +12,7 @@ import RasaProBanner from "@theme/RasaProBanner";
 
 <RasaProBanner />
 
-- **Rasa Pro**, a drop-in replacement for Rasa Open Source enterprise
+- **Rasa Pro**, a drop-in replacement for Rasa Open Source
 - **Rasa Pro Services**, flexible infrastructure and APIs on top of Rasa
   Open Source. Rasa Pro Services should be deployed alongside, but separately
   from your production assistant.

docs/versioned_docs/version-3.x/markers.mdx

@@ -14,7 +14,7 @@ This feature is currently experimental and might change or be removed in the fut
 
 :::
 
-:::danger
+:::danger Deprecated
 
 In the upcoming release version 3.7 of Rasa Open Source, we’re removing this experimental feature. For documentation on the markers feature in Rasa Pro, please [click here](./monitoring/analytics/realtime-markers.mdx)
 

docs/versioned_docs/version-3.x/nlg.mdx

@@ -27,12 +27,9 @@ The body of the `POST` request sent to your NLG endpoint will be structured
 like this:
 
 :::info New in 3.6
-We have added an `ids` field to the request body.
-This field contains a list of response variation IDs.
+We have added an `id` field to the request body.
+This field contains the ID of the response variation.
 You can use this information to compose/select a proper response variation on your NLG server.
-In case you are using responses with Conditional Response Variations,
-on the NLG server side, you must implement the logic which will use content of the slots from
-`tracker['slots']` along side the IDs from `ids` field to select the proper response variation.
 
 :::
 
@@ -42,7 +39,7 @@ on the NLG server side, you must implement the logic which will use content of t
   "arguments":{
 
   },
-  "ids": ["<response_variation_id_1>", "<response_variation_id_2>"],
+  "id": "<response_variation_id>",
   "tracker":{
     "sender_id":"user_0",
     "slots":{
@@ -182,7 +179,7 @@ Here is an overview of the high-level keys in the post request:
 Key | Description
 ---|---
 `response` | The name of the response predicted by Rasa.
-`ids` | A list of response variation IDs.
+`id` | An optional string representing the response variation ID, can be null.
 `arguments` | Optional keyword arguments that can be provided by custom actions.
 `tracker` | A dictionary containing the entire conversation history.
 `channel` | The output channel this message will be sent to.

docs/versioned_docs/version-3.x/pii-management.mdx

@@ -240,6 +240,34 @@ The `anonymization_topics` section contains a list of Kafka topics to which the
 Each Kafka topic must have a `name` field and an `anonymization_rules` field. The `name` field specifies the name of the
 Kafka topic. The `anonymization_rules` field specifies the `id` of the anonymization rule list to be used for the Kafka topic.
 
+### Streaming anonymized events to Rasa X/Enterprise with Kafka
+
+Streaming anonymized events to Rasa X/Enterprise is only supported for Rasa X/Enterprise versions `1.3.0` and above.
+In addition, you must use the Kafka event broker, other event broker types are not supported.
+
+You can stream anonymized events to Rasa X/Enterprise via Kafka by adding the `rasa_x_consumer: true` key-value pair to
+the `anonymization_topics` section:
+
+```yaml
+event_broker:
+  type: kafka
+  partition_by_sender: True
+  url: localhost
+  anonymization_topics:
+    - name: topic_1
+      anonymization_rules: rules_1
+      rasa_x_consumer: true
+    - name: topic_2
+      anonymization_rules: rules_2
+```
+
+If multiple Kafka anonymization topics contain the `rasa_x_consumer` key-value pair, the anonymized events will be streamed
+to the Kafka topic that is mapped to the first topic in the `anonymization_topics` list that contains the `rasa_x_consumer`
+key-value pair.
+
+Note that the `rasa_x_consumer` key-value pair is optional. If it is not specified, the anonymized events will be published
+to the Kafka topic, but they will not be streamed to Rasa X/Enterprise.
+
 ## How to enable anonymization of PII in logs
 
 You can enable anonymization of PII in logs by filling the `logger` section in the `endpoints.yml` file.
@@ -257,3 +285,8 @@ The `anonymization_rules` field specifies the `id` of the anonymization rule lis
 We strongly recommend to run with log level INFO in production.
 Running with log level DEBUG will increase the assistant's response latency because of processing delays.
 :::
+
+Note that running `rasa shell` in debug mode with a Kafka event broker might result in logs related to the event publishing
+to be printed to console **after** the bot message. This behaviour is expected because the event anonymization and publishing
+is done asynchronously as a background task, so it will complete after the assistant has already predicted and executed the
+bot response.