Remote Port

Remote Port

Sources

Remote Port is a technology developed by Xilinx at https://github.com/Xilinx/qemu.git, https://github.com/Xilinx/libsystemctlm-soc and https://github.com/Xilinx/systemctlm-cosim-demo/

Xilinx board/kernel based examples are present at https://github.com/Xilinx/systemctlm-cosim-demo/tree/master/docs

Information on the actual protocol can be found here

What it is

It allows information exchange between two simulators, such as memory transactions and synchronization (among others).

Its' main goal is to make available an SOC environment capable of emulating a CPU that can communicate with any simulated fabric.

Based on what the creator mentions here, this should enable companies that produce SoCs to reliably and accurately simulate the RTL of the SoC while offloading the usually very expensive simulation of the CPU into a much faster emulation.

How to use it with ARC

By default, Remote Port is integrated in Xilinx boards and a RISCV one, in the Xilinx QEMU repository.

As of now (16/11/2023), it isn't upstream so support for CPUs not present in their repo can only be achieved by handpicking code into this repository directly.

As long as the simulations are coherent with what each provides, nothing on the SystemC side needs to change.

Status

Branch remote_port currently supports memory based accesses (reads/writes for memory masters/slaves) into the SystemC simulation.

It can be used together with Xilinxs zynq_demo. In this demo, Reads/Writes made in QEMU to the 0x40000000 memory space are redirected to the debugdev.

How it works

Overall, Xilinx made two changes to QEMU:

Devices

The most relevant one is the addition of devices that allow communication between QEMU and SystemC.

These devices also require some minor alterations to QEMU, which means they aren't "drag and drop".

These changes are (besides the ones to include the devices):

• include/chardev/char-fe.h
• include/chardev/char.h
• include/exec/memory.h
• include/sysemu/dma.h
• softmmu/memory.c
• softmmu/vl.c

See more details in the first commit of the remote_port branch, which contains solely these adaptations.

They aren't very invasive, and a few seem to be 'temporary', but should still be taken into consideration.

Generic devices

The other interesting change is the addition of generic FDT devices. This is currently not present in the remote_port branch

This change enables QEMU to generate and configure devices dynamically by inspecting a device tree from a file provided via command-line argument

This means that instead of being limited to hardcoded machines and their specific device configurability, we can adapt any machine that supports dynamic devices, at execution time.

The way devices are brought online starts with various attempts at identifying the device in question. These are, in order:

1. Try and match the nodes' name in the device tree to the name of an existing component;
2. Looking into the compatibility list for a device that is registered as an fdt_generic
3. Try and initialize a QOM object with the name of the node

Inner workings

Commands available (for more information on their exact usage see how the protocol works):

• hello
• sync
• interrupt
• ats_inv
• ats_req
• read
• write

QEMU side

Main loop

The main loop is in rp_protocol_thread(void *arg) and runs in a separate thread This infinite loop does:

1. read packet from "handle" socket
2. If hello or sync, hello/sync back if necessary
3. Otherwise leave the packet in the queue for a separate thread to pick it up

These packets are picked up by rp_event_read which spins up when:

• A device is waiting for a response
• Remote Port itself is waiting for a response
• The qemu_set_fd_handler conditions are met (aka there is something to be read)

Memory master

Memory master is a SYS_BUS device that can be bound to a certain memory subregion.

Unlike with current MMIO, instead of 'read/write' operations there is an access 'operation' which essentially is the same thing, separating into read/write when analyzing the command inside the packet in question.

Memory slave

Memory slave is a simple device that takes care of the writes to memory on the side of QEMU.

It is triggered by rp_process, which can be spin up by memory master or rp_event_read

SystemC

TLM

The main object is the remoteport_tlm which contains the main loop: rp_process(bool can_sync)

Everything in SystemC runs in a single thread by default

This infinite loop does:

1. read packet from "handle" socket
2. Get the target device (if none, dev_null is used)
3. If its a response packet:
   1. Run dev->pre_any_cmd()
   2. Look for a response slot (something's waiting for a response)
   3. Copy the packet over and notify the response slot
4. Otherwise, run dev->pre_any_cmd()
5. Depending on the received packet, run dev->[rp_cmd_hello | cmd_write | cmd_read | cmd_interrupt | cmd_ats_inv | rp_cmd_sync]
6. run dev->post_any_cmd()

tlm_dev

Every specific remote port class derives from tlm_dev which has the cmd_X virtual functions used above.

Each remote port class then specifies that object.

The only bare instance of tlm_dev is dev_null

The default virtual functions are:

virtual void cmd_write(struct rp_pkt &pkt, bool can_sync, unsigned char *data, size_t len);

Proxy call to remoteport_tlm_memory_master::cmd_write_null

virtual void cmd_read(struct rp_pkt &pkt, bool can_sync);

Proxy call to remoteport_tlm_memory_master::cmd_read_null

virtual void cmd_interrupt(struct rp_pkt &pkt, bool can_sync);

Proxy call to remoteport_tlm_wires::cmd_interrupt_null

virtual void cmd_ats_inv(struct rp_pkt &pkt, bool can_sync);

Proxy call to remoteport_tlm_ats::cmd_ats_inv_null

iconnect

This is a device meant for routing packets between devices.

Accesses available memory mappings and performs address conversion when appropriate.

Has input/output sockets with numbers matching the connected masters/slaves.

These connections are usually done in the definition of the running boards (i.e. zynq_demo)

Memory master

Responsible for managing the memory operations

Is the one targeted by QEMU when writing to an MMIO address space.

Has an output/initiator socket that if not bound to anything, simply prints the received command and returns a generic error response.

This socket is usually bound to a target/input socket of the iconnect to redirect memory requests to the appropriate memory slaves

Memory slave

Responsible for fulfilling memory operations

Has one input/target socket which should be connected to an iconnect

Tips

1. In rp_process, use the following to print packet types arriving at each simulator:

printf("cmd=%s dev=%d\n", rp_cmd_to_string((rp_cmd)(pkt_rx.pkt->hdr.cmd)), pkt_rx.pkt->hdr.dev);

2. The remote-port devices present in QEMU should have a match in ID and Type (Slave/Master/ATS/...) in SystemC

TODOs

1. Add support to other remote-port devices
2. Add support for generic fdt devices
3. Add documentation on above topics as necessary