Skip to content

Commit

Permalink
Merge pull request #365 from lucianomartin/feauture/asrc_docs
Browse files Browse the repository at this point in the history
Updates for ASRC demo guide
  • Loading branch information
lucianomartin authored Oct 8, 2024
2 parents 06bf254 + 402cf71 commit f677508
Show file tree
Hide file tree
Showing 11 changed files with 68 additions and 34 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ XCORE-VOICE change log
* REMOVED: flash settings in .xn files, as they are not required by XMOS
Tools 15.2.x.
* ADDED: Support for reading registers over I2C slave in FFD examples.
* ADDED: Note in ASRC demo documentation about large latency in ASRC
processing. References to alternative application notes have been provided.
* CHANGED: Updated submodule fwk_rtos to version 3.2.0 from 3.0.5.
* ADDED: lib_sw_pll submodule v1.1.0.

Expand Down
2 changes: 1 addition & 1 deletion doc/programming_guide/asr/deploying/linux_macos.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Run the following commands in the root folder to build the host application usin
The host application, ``xscope_host_endpoint``, will be installed at ``/opt/xmos/bin``, and may be moved if desired. You may wish to add this directory to your ``PATH`` variable.

Before running the host application, you may need to add the location of ``xscope_endpoint.so`` to your ``LD_LIBRARY_PATH`` environment variable. This environment variable will be set if you run the host application in the XTC Tools command-line environment. For more information see `Configuring the command-line environment <https://www.xmos.ai/documentation/XM-014363-PC-LATEST/html/tools-guide/install-configure/getting-started.html>`__.
Before running the host application, you may need to add the location of ``xscope_endpoint.so`` to your ``LD_LIBRARY_PATH`` environment variable. This environment variable will be set if you run the host application in the XTC Tools command-line environment. For more information see `Configuring the command-line environment <https://xmos.com/xtc-install-guide>`__.

Building the Firmware
=====================
Expand Down
2 changes: 1 addition & 1 deletion doc/programming_guide/asr/deploying/native_windows.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Then build the host application:
The host application, ``xscope_host_endpoint.exe``, will install at ``<USERPROFILE>\.xmos\bin``, and may be moved if desired. You may wish to add this directory to your ``PATH`` variable.

Before running the host application, you may need to add the location of ``xscope_endpoint.dll`` to your ``PATH``. This environment variable will be set if you run the host application in the XTC Tools command-line environment. For more information see `Configuring the command-line environment <https://www.xmos.ai/documentation/XM-014363-PC-LATEST/html/tools-guide/install-configure/getting-started.html>`__.
Before running the host application, you may need to add the location of ``xscope_endpoint.dll`` to your ``PATH``. This environment variable will be set if you run the host application in the XTC Tools command-line environment. For more information see `Configuring the command-line environment <https://xmos.com/xtc-install-guide>`__.

Building the Firmware
=====================
Expand Down
32 changes: 26 additions & 6 deletions doc/programming_guide/asrc/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,33 @@
Overview
********

.. warning::

This example is based on the RTOS framework and drivers. This choice simplifies the example design, but it leads to high latency in the system.
The main sources of latency are:

- Large block size used for ASRC processing: this is necessary to minimise latency associated with the intertile context and thread switching overhead.
- Large size of the buffer to which the ASRC output samples are written: a stable level (half full) must be reached before the start of streaming out over USB.
- RTOS task scheduling overhead between the tasks.
- bInterval of USB in the RTOS drivers is set to 4, i.e. one frame every 1 ms.
- Block based implementation of the USB and |I2S| RTOS drivers.

The expected latencies for USB at 48 kHz are as follows:

- USB -> ASRC -> |I2S|: from 8 ms at |I2S| at 192 kHz to 22 ms at 44.1 kHz
- |I2S| -> ASRC -> USB: from 13 ms at |I2S| at 192 kHz to 19 ms at 44.1 kHz

For a proposed implementation with lower latency, please refer to the bare-metal examples below:

- `AN02002: Adding an additional I2S bus to USB Audio via SRC <https://www.xmos.com/file/AN02002>`__
- `AN02003: SPDIF/ADAT/I2S Slave Receive to I2S Slave Bridge with ASRC <https://www.xmos.com/file/AN02003>`__


This is the |SOFTWARE_URL| Asynchronous Sampling Rate Converter (ASRC) example design.

The example system implements a stereo |I2S| Slave and a stereo Adaptive UAC2.0 interface and exchanges data between the two interfaces.
Since the two interfaces are operating in different clock domains, there is an ASRC block between them that converts from the input to the output sampling rate.
There are two ASRC blocks, one each in the |I2S| ASRC USB and USB ASRC |I2S| path, as illustrated in the :ref:`fig-asrc-top-level-label`.
There are two ASRC blocks, one each in the |I2S| -> ASRC -> USB and USB -> ASRC -> |I2S| path, as illustrated in the :ref:`fig-asrc-top-level-label`.
The diagram also shows the rate calculation path, which monitors and computes the instantaneous ratio between the ASRC input and output sampling rate.
The rate ratio is used by the ASRC task to dynamically adapt filter coefficients using spline interpolation in its filtering stage.

Expand All @@ -25,7 +47,7 @@ The |I2S| Slave interface is a stereo 32 bit interface supporting sampling rates
The USB interface is a stereo, 32 bit, 48 kHz, High-Speed, USB Audio Class 2, Adaptive interface.

The ASRC algorithm implemented in the `lib_src <https://github.com/xmos/lib_src/>`_ library is used for the ASRC processing.
The ASRC processing is block based and works on a block size of 244 samples per channel in the |I2S| ASRC USB path and 96 samples per channel in the USB ASRC |I2S| path.
The ASRC processing is block based and works on a block size of 244 samples per channel in the |I2S| -> ASRC -> USB path and 96 samples per channel in the USB -> ASRC -> |I2S| path.

Supported Hardware
==================
Expand All @@ -39,7 +61,7 @@ The table :ref:`table-pin-connections-label` lists the pins on the XK-VOICE-L71
.. _table-pin-connections-label:

.. list-table:: XK-VOICE-L71 RPI host interface header (J4) connections
:widths: 30 50
:widths: 50 50
:header-rows: 1
:align: left

Expand All @@ -56,8 +78,6 @@ The table :ref:`table-pin-connections-label` lists the pins on the XK-VOICE-L71
* - One of the GND pins (6, 14, 20, 30, 34, 9, 25 or 39)
- GND on the |I2S| Master board



Obtaining the app files
=======================

Expand All @@ -72,7 +92,7 @@ Download the main repo and submodules using:
Building the app
================

First install and source the XTC version: 15.2.1 tools. The output should be
First install and source the XTC version: |TOOLS_VERSION| tools. For example with version 15.2.1, the output should be
something like this:

::
Expand Down
28 changes: 14 additions & 14 deletions doc/programming_guide/asrc/software_architecture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ The :ref:`fig-ASRC-task-diagram-label` shows the RTOS tasks and other components


The tasks can roughly be categorised as belonging to the USB driver, |I2S| driver or the application code categories.
The actual ASRC processing happens in four tasks across the two tiles; the **usb_audio_out_asrc task**, **i2s_audio_recv_asrc** task, and two instances of **asrc_one_channel_task**, one on each tile.
The actual ASRC processing happens in four tasks across the two tiles; the **usb_audio_out_asrc task**, **i2s_audio_recv_asrc task**, and two instances of **asrc_one_channel task**, one on each tile.
This is described in more detail in the :ref:`application-components-label` section below.

Most of the tasks are involved in the ASRC processing data path, while a few are involved in monitoring the input and output data rates
Expand All @@ -53,9 +53,9 @@ It interfaces with the USB app level thread (**usb_task**) via shared memory and

**usb_task** implements the app level USB driver functionality. The app level USB driver is based on `TinyUSB <https://docs.tinyusb.org/en/latest/>`_ which hooks into the application by means of callback functions.
The **usb_isr** task is triggered by the interrupt and parses the data transferred from XUD and places it on a queue that the **usb_task** blocks on for further processing.
For example, on completion of an EP1 OUT transfer, the transfer completion gets notified on the **usb_xud_thread usb_isr usb_task** path,
For example, on completion of an EP1 OUT transfer, the transfer completion gets notified on the **usb_xud_thread -> usb_isr -> usb_task** path,
and the **usb_task** calls the ``tud_audio_rx_done_post_read_cb()`` function to have the application process the data received from the host.
On completion of an EP1 IN transfer, the transfer completion again follows the **usb_xud_thread usb_isr usb_task** path, and **usb_task** calls the ``tud_audio_tx_done_pre_load_cb()``
On completion of an EP1 IN transfer, the transfer completion again follows the **usb_xud_thread -> usb_isr -> usb_task** path, and **usb_task** calls the ``tud_audio_tx_done_pre_load_cb()``
callback function to have the application load the EP1 IN data for the next transfer.

**samples_to_host_stream_buf** and **samples_from_host_stream_buf** are circular buffers shared between the application and the USB driver and allow for decoupling one from the other.
Expand Down Expand Up @@ -111,34 +111,34 @@ It has other rate-monitoring related responsibilities that are described in the
**i2s_to_usb_intertile** task receives the ASRC output data generated by **i2s_audio_recv_asrc** over the inter-tile context onto the USB tile and writes it to the USB ``samples_to_host_stream_buf``.
It has other rate-monitoring related responsibilities that are described in the :ref:`rate-server-label` section.

The :ref:`asrc_i2s_to_usb_data_path-label` diagram shows the application tasks involved in the |I2S| ASRC USB path processing and their interaction with each other.
The :ref:`asrc_i2s_to_usb_data_path-label` diagram shows the application tasks involved in the |I2S| -> ASRC -> USB path processing and their interaction with each other.

.. _asrc_i2s_to_usb_data_path-label:

.. figure:: diagrams/asrc_i2s_to_usb_data_path.png
:align: center
:width: 100%
:alt: ASRC |I2S| ASRC USB data path
:alt: ASRC |I2S| -> ASRC -> USB data path

|I2S| ASRC USB data path
|I2S| -> ASRC -> USB data path

The :ref:`asrc_usb_to_i2s_data_path-label` diagram shows the application tasks involved in the USB ASRC |I2S| path processing and their interaction with each other.
The :ref:`asrc_usb_to_i2s_data_path-label` diagram shows the application tasks involved in the USB -> ASRC -> |I2S| path processing and their interaction with each other.

.. _asrc_usb_to_i2s_data_path-label:

.. figure:: diagrams/asrc_usb_to_i2s_data_path.png
:align: center
:width: 100%
:alt: USB ASRC |I2S| data path
:alt: USB -> ASRC -> |I2S| data path

USB ASRC |I2S| data path
USB -> ASRC -> |I2S| data path


.. _rate-server-label:

**rate_server**
---------------
The ASRC ``process_frame`` API requires the caller to calculate and send the instantaneous ratio between the ASRC input and output rate. The **rate_server** is responsible for calculating these rate ratios for both USB ASRC |I2S| and |I2S| ASRC USB directions.
The ASRC ``process_frame`` API requires the caller to calculate and send the instantaneous ratio between the ASRC input and output rate. The **rate_server** is responsible for calculating these rate ratios for both USB -> ASRC -> |I2S| and |I2S| -> ASRC -> USB directions.

Additionally, the application also monitors the average buffer fill levels of the buffers holding ASRC output to prevent any overflows or underflows of the respective buffer. A gradual drift in the buffer fill level indicates that the rate ratio is being under or over calculated by the **rate_server**.
This could happen either due to jitter in the actual rates or precision limitations when calculating the rates.
Expand Down Expand Up @@ -179,8 +179,8 @@ either calculating it or getting it through shared memory from other USB tasks o
The |I2S| related information (1 and 4 above) is calculated in the **rate_server** itself with information available for calculating these available through shared memory from other tasks on this tile.

After calculating the rates, the **rate_server** sends the rate ratio for the USB ASRC |I2S| side to the **usb_to_i2s_intertile** task over the inter-tile context and it is made available to the
**usb_audio_out_asrc** task through shared memory. The |I2S| ASRC USB side rate ratio is also made available to the **i2s_audio_recv_asrc** task through shared memory since it runs on the same tile as the rate server.
After calculating the rates, the **rate_server** sends the rate ratio for the USB -> ASRC -> |I2S| side to the **usb_to_i2s_intertile** task over the inter-tile context and it is made available to the
**usb_audio_out_asrc** task through shared memory. The |I2S| -> ASRC -> USB side rate ratio is also made available to the **i2s_audio_recv_asrc** task through shared memory since it runs on the same tile as the rate server.

The :ref:`fig-rate-server-label` diagram shows the code flow during the rate ratio calculation process, focussing on the **usb_to_intertile** task that triggers the **rate_server** and the **rate_server** task where the rate ratios are calculated.

Expand Down Expand Up @@ -211,14 +211,14 @@ Handling USB speaker interface close -> open events
===================================================

When the USB host stops streaming to the device and then starts again, this event is detected through calls to the ``tud_audio_set_itf_close_EP_cb`` and ``tud_audio_set_itf_cb`` functions.
The ASRC output buffer in the USB ASRC |I2S| path (|I2S| ``send_buffer``) is reset.
The ASRC output buffer in the USB -> ASRC -> |I2S| path (|I2S| ``send_buffer``) is reset.
Zeroes are then sent over |I2S| until the buffer fills to a stable level, when we resume streaming out of this buffer to send samples over |I2S|.
The average buffer calculation state for the |I2S| ``send_buffer`` is also reset and a new stable average is calculated against which the average buffer levels are corrected.

Handling USB mic interface close -> open events
===============================================

If the USB host stops streaming from the device and then starts again, this event is detected through calls to the ``tud_audio_set_itf_close_EP_cb`` and ``tud_audio_set_itf_cb`` functions.
The ASRC output buffer in the |I2S| ASRC USB is reset (USB ``samples_to_host_stream_buf``).
The ASRC output buffer in the |I2S| -> ASRC -> USB is reset (USB ``samples_to_host_stream_buf``).
Zeroes are streamed to the host until the buffer fills to a stable level, when we resume streaming out of this buffer to send samples over USB.
The average buffer calculation state for the USB ``samples_to_host_stream_buf`` is also reset and a new stable average is calculated against which the average buffer levels are corrected.
2 changes: 1 addition & 1 deletion doc/programming_guide/prerequisites.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
Prerequisites
#############

It is recommended that you download and install the latest release of the `XTC Tools <https://www.xmos.com/software/tools/>`__. XTC Tools 15.2.1 or newer are required for building, running, flashing and debugging the example applications.
It is recommended that you download and install the latest release of the `XTC Tools <https://www.xmos.com/software/tools/>`__. XTC Tools |TOOLS_VERSION| or newer are required for building, running, flashing and debugging the example applications.

`CMake 3.21 <https://cmake.org/download/>`_ or newer and `Git <https://git-scm.com/>`_ are also required for building the example applications.

Expand Down
6 changes: 3 additions & 3 deletions doc/shared/introduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -72,11 +72,11 @@ Obtaining the Hardware

The XK-VOICE-L71 DevKit and Hardware Manual can be obtained from the |HARDWARE_URL| product information page.

The XK-VOICE-L71 is based on the: `XU316-1024-QF60A <https://www.xmos.ai/file/xu316-1024-qf60b-xcore_ai-datasheet?version=latest>`_
The XK-VOICE-L71 is based on the: `XU316-1024-QF60A <https://www.xmos.com/file/xu316-1024-qf60b-xcore_ai-datasheet?version=latest>`_

The XCORE-AI-EXPLORER DevKit and Hardware Manual used in the :ref:`Microphone Aggregation <sln_voice_mic_aggregator_programming_guide>` example can be obtained from the |HARDWARE_URL| product information page.

Learn more about the `The XMOS XS3 Architecture <https://www.xmos.ai/download/The-XMOS-XS3-Architecture.pdf>`_
Learn more about the `The XMOS XS3 Architecture <https://www.xmos.com/download/The-XMOS-XS3-Architecture.pdf>`_

######################
Obtaining the Software
Expand All @@ -86,7 +86,7 @@ Obtaining the Software
Development Tools
*****************

It is recommended that you download and install the latest release of the `XTC Tools <https://www.xmos.com/software/tools/>`__. XTC Tools 15.2.1 or newer are required. If you already have the XTC Toolchain installed, you can check the version with the following command:
It is recommended that you download and install the latest release of the `XTC Tools <https://www.xmos.com/software/tools/>`__. XTC Tools |TOOLS_VERSION| or newer are required. If you already have the XTC Toolchain installed, you can check the version with the following command:

.. code-block:: console
Expand Down
6 changes: 3 additions & 3 deletions doc/substitutions.inc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.. |HARDWARE_URL| replace:: `XK-VOICE-L71 <https://www.xmos.ai/xk-voice-l71>`__
.. |SOFTWARE_URL| replace:: `XCORE-VOICE <https://www.xmos.ai/xcore-voice>`__

.. |HARDWARE_URL| replace:: `XK-VOICE-L71 <https://www.xmos.com/xk-voice-l71>`__
.. |SOFTWARE_URL| replace:: `XCORE-VOICE <https://www.xmos.com/xcore-voice>`__
.. |TOOLS_VERSION| replace:: 15.2.1
.. |newpage| raw:: latex

\clearpage
15 changes: 13 additions & 2 deletions examples/asrc_demo/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,22 @@
ASRC Demo Application
*********************

.. warning::

This example is based on the RTOS framework and drivers, and it can lead to latency of up to 20 ms in the system.
More information can be found in the Overview section of the ASRC example in the Programming Guide.

For a proposed implementation with lower latency, please refer to the bare-metal examples below:

- `AN02002: Adding an additional I2S bus to USB Audio via SRC <https://www.xmos.com/file/AN02002>`__
- `AN02003: SPDIF/ADAT/I2S Slave Receive to I2S Slave Bridge with ASRC <https://www.xmos.com/file/AN02003>`__


This is the XCORE-VOICE Asynchronous Sampling Rate Converter (ASRC) example design.

The example system implements a stereo I2S slave and a stereo adaptive UAC2.0 interface and exchanges data between the two interfaces.
Since the two interfaces are operating in different clock domains, there is an ASRC block between them that converts from the input to the output sampling rate.
There are two ASRC blocks, one in the I2S ASRC USB path and the other in the USB ASRC I2S path.
There are two ASRC blocks, one in the I2S -> ASRC -> USB path and the other in the USB -> ASRC -> I2S path.

The application also monitors and computes the instantaneous ratio between the ASRC input and output sampling rate. The rate ratio is used by the ASRC task to dynamically adapt filter coefficients using spline interpolation in its filtering stage.

Expand All @@ -15,7 +26,7 @@ The I2S slave interface is a stereo 32 bit interface supporting sampling rates b

The USB interface is a stereo, 32 bit, 48 kHz, High-Speed, USB Audio Class 2, Adaptive interface.

The ASRC algorithm in the `lib_src <https://github.com/xmos/lib_src/>`_ library is used for the ASRC processing. The ASRC processing is block based and works on a block size of 244 samples per channel in the I2S ASRC USB path and 96 samples per channel in the USB ASRC I2S path.
The ASRC algorithm in the `lib_src <https://github.com/xmos/lib_src/>`_ library is used for the ASRC processing. The ASRC processing is block based and works on a block size of 244 samples per channel in the I2S -> ASRC -> USB path and 96 samples per channel in the USB -> ASRC -> I2S path.

Supported Hardware
==================
Expand Down
Loading

0 comments on commit f677508

Please sign in to comment.