Live tracing of events from the board¶

This chapter covers the usage of Zephelin Trace Viewer for live visualization of traces being gathered during application execution.

Real-time visualization requires running a local Python server responsible for ingesting CTF traces, parsing them into TEF, and forwarding them to the visualizer.

Real-time tracing server structure¶

Compared to the diagram from Zephelin trace collection, instead of saving and converting complete CTF traces to TEF and uploading them to Zephelin Trace Viewer, we continuously deliver traces over TCP to a Zephelin server, convert them chunk by chunk to TEF and deliver to Zephelin Trace Viewer over Remote Procedure Calls (RPC).

The Zephelin server consists of:

• Trace Handler - configures Parser Proxy, the libbtrace live plugin and the RPC Dispatcher and controls the delivery of traces and commands to the Zephelin Trace Viewer.

• RPC Dispatcher - sends and receives messages from Zephelin Trace Viewer.

• Stream Parser Proxy - collects data from west zpl-<backend>-capture over TCP and subdivides it to the commands’ stream and traces. Traces are delivered to libbtrace live plugin.

• libbtrace live plugin - receives parts of CTF traces and converts them to parts of TEF messages that are later delivered to Trace Handler.

Prerequisites¶

Before running the server, you need to install required backend dependencies and compile the frontend visualizer.

Install server dependencies¶

Apart from installing Zephelin dependencies (see Initializing the workspace), server dependencies need to be installed as well with the following command:

pip install -r server/requirements.txt

Build the frontend for Trace Viewer¶

The server requires a compiled version of Zephelin Trace Viewer. First, clone the repository:

git clone --recursive https://github.com/antmicro/zephelin-trace-viewer.git

After cloning the repository, install corepack and install the necessary dependencies with:

cd zephelin-trace-viewer
corepack enable
yarn

Lastly, build the backend with:

yarn build
cd ..

Built frontend will be available under the ./zephelin-trace-viewer/dist directory.

Running the server¶

To start the Zephelin server and host the visualizer, execute the server/run_backend.py script.

Basic usage¶

To run the server with default settings and serve the frontend, point it to the dist directory with the compiled frontend:

python server/run_backend.py --frontend-directory <path_to_dist>

Configuration Options¶

You can customize the server’s networking interfaces, and specify trace parsing options that will be used in the CTF to TEF conversion.

• --tcp-server-host - Address of the Zephelin TCP socket for CTF trace ingestion (Default: 127.0.0.1).

• --tcp-server-port- Port of the Zephelin TCP socket for CTF trace ingestion (Default: 5000).

• --backend-host - Address where the backend API and visualizer are hosted (Default: 127.0.0.1).

• --backend-port - Port where the backend API and visualizer are hosted (Default: 8000).

• --bt-port - Port used by libbtrace live plugin to collect pure CTF traces (Default: 42674).

• --frontend-directory - Path to compiled frontend directory.

• --build-dir - Path to the traced application build directory.

• --tflm-model-paths - Paths to the TFLM models.

• --tvm-model-paths - Paths to the TVM models.

• --tvm-model-metadata-paths - Paths to the TVM model metadata files.

• --tvm-model-op-remove-prefix - Regex pattern used for removing TVM operator prefixes.

• --tvm-model-op-remove-suffix - Regex pattern used for removing TVM operator suffixes.

• --verbosity - Set the logging verbosity level (DEBUG, INFO, WARNING, ERROR, CRITICAL).

Additional configuration options for tests¶

There is a possibility to run the server in mock mode, where TEF traces are delivered as inputs and sent to the website at specified speed:

• --mock-trace-file - Path to the TEF/JSON trace file to use for the mock.

• --mock-playback-speed - Playback speed multiplier for the mock.

Providing traces to the server¶

Once the server is running, it listens for CTF traces on the TCP socket. To route the traces read from one of the available backends (described in Trace Collection section), the --send-to-remote argument can be provided with the TCP socket address specified.

Example:

west zpl-uart-capture /dev/pts/12 115200 --send-to-remote 127.0.0.1:5000

Live-tracing controls¶

If the Visualizer is connected to the server, the live-tracing controls are available on the top-bar.

• Start streaming - enables continuous rendering of received traces.

• Stop streaming - disables continuous rendering.

• Collect - visualizes all the traces gathered so far.

• Stop Tailing - Stops the default behavior in which viewport follows the live edge of the visualized trace.

• Resume Tailing - Snaps the viewport back to the live edge.

Sample collection of traces¶

This sample demonstrates how to run an application running profiling for a TensorFlow Lite Micro model.

First, build a sample application running two TensorFlow Lite Micro models, with an increased number of iterations:

west build -p -b max32690fthr/max32690/m4 samples/profiling/tflm_multi_model -- -DCONFIG_ZPL_TRACE_FORMAT_CTF=y -DCONFIG_TRACING_BUFFER_SIZE=10000 -DCONFIG_BOOT_BANNER=n -DCONFIG_PRINTK=n -DCONFIG_LOG=n -DCONFIG_ZPL_SAMPLE_TFLM_NUM_ITERS=200

After this, run the server for live tracing, providing the paths to the models:

python server/run_backend.py --frontend-directory ./zephelin-trace-viewer/dist --tflm-model-paths ./samples/common/tflm/model/magic-wand.tflite ./samples/common/tflm/model/sine.tflite

Then, run the collection of traces from:

• Renode:

  python scripts/run_renode.py --trace-output test.ctf --send-to-remote 127.0.0.1:5000
  

• From physical hardware (after flash), e.g.:

  west zpl-uart-capture <path-to-uart> 115200 ./trace-hw.ctf --send-to-remote 127.0.0.1:5000
  

Live tracing with instrumentation¶

Traces produced by the instrumentation subsystem can be visualized live as well, as long as they are delivered over the same channel as regular Zephelin traces. This is the case for the tracing subsystem backend (CONFIG_INSTRUMENTATION_BACKEND_TRACING_CORE=y), which packs instrumentation events into a separate CTF stream of the regular trace. With that, no additional capture command is needed - a single west zpl-<backend>-capture with --send-to-remote feeds both streams to the server.

We will use the samples/profiling/tflm_instrumentation sample, which runs a TFLM model with both Zephelin profilers and the instrumentation subsystem enabled.

First, build the sample with the instrumentation tracing backend configuration:

west build -p -b max32690fthr/max32690/m4 samples/profiling/tflm_instrumentation -- \
  -DEXTRA_CONF_FILE="zpl.conf;instrumentation_tracing.conf"

Then, start the server, pointing it to the model used by the sample and to the build directory:

python server/run_backend.py --frontend-directory ./zephelin-trace-viewer/dist \
  --tflm-model-paths ./samples/common/tflm/model/sine.tflite \
  --build-dir ./build

Next, run the collection of traces from:

• Renode:

  python scripts/run_renode.py --send-to-remote 127.0.0.1:5000 --pause
  

• From physical hardware (after flash), e.g.:

  west zpl-uart-capture <path-to-uart> 115200 ./trace-hw.ctf --send-to-remote 127.0.0.1:5000
  

Once the simulation is connected to the backend, open the visualizer at the configured address (default: http://127.0.0.1:8000), press Start streaming and unpause the simulation.

The recording below shows the sample being visualized live.

Communication flow¶

The server implementation in server/run_backend.py consists of the following access points:

• <tcp_server_host>:<tcp_server_port> - allows for communication between west capture subcommands and the server.

• <backend_host>:<backend_port> - hosts the website loaded from the --frontend-directory directory and configures the communication between website and server.

• 127.0.0.1:<bt_port> - allows for delivery of pure CTF events to the libbtrace live plugin (it is a one-way communication).

Like in the file-based Zephelin flow, the traces are first collected and sent by the board to the host using a selected backend (e.g. UART). After this, the west zpl-<backend>-capture command with the --send-to-remote flag configured to e.g. 127.0.0.1:5000 will send CTF traces with additional data (such as _zpl_ctf_start__ start tag) to a given address.

This address should match the --tcp-server-host and --tcp-server-port settings in the server/run_backend.py script. Stream Parser Proxy will receive packets sent by the west subcommand and subdivide them into commands (such as restart the viewer based on start tag) that should be sent as RPC commands to the website, and pure CTF events.

Pure CTF events are sent to the libbtrace live plugin over the --bt-port port on loopback (e.g. 127.0.0.1:42674), for fast conversion to TEF events in C++. The plugin communicates with the libbtrace module running in the server, delivering bt2 messages with parts of parsed TEF events - it is not socket-based communication.

Those bt2 messages are delivered to the Trace Handler. Trace Handler performs final adjustments of the TEF events and sends them to the website.

In the meantime, RPC Dispatcher allows for sending calls between server and website, and controls the state of Zephelin Trace Viewer using procedures described in Server API reference.

Server API reference¶

The Zephelin real-time tracing server supports the following JSON-RPC requests sent as rpc_request WebSocket event:

• trace.connect

  async handlers.trace_handler.TraceHandler.connect(self) → dict['status' | 'message', str]¶

  Starts the TCP server to listen for incoming trace streams from capture scripts.

  Returns¶

  dict[Literal[“status”, “message”], str]

  Status message.

• trace.disconnect

  async handlers.trace_handler.TraceHandler.disconnect(self) → dict['status' | 'message', str]¶

  Terminates the background read task and tears down transportation backend.

  Returns¶

  dict[Literal[“status”, “message”], str]

  Status message.

• trace.stream_start

  async handlers.trace_handler.TraceHandler.stream_start(self) → dict['status', str]¶

  Enables continuous trace streaming to the frontend.

  Returns¶

  dict[Literal[“status”], str]

  Status message.

• trace.stream_stop

  async handlers.trace_handler.TraceHandler.stream_stop(self) → dict['status', str]¶

  Disables continuous trace streaming to the frontend.

  Returns¶

  dict[Literal[“status”], str]

  Status message.

• trace.metadata

  async handlers.trace_handler.TraceHandler.metadata(self) → dict['status' | 'message' | 'data', str | dict]¶

  Provides model metadata and memory symbols for the trace.

  Runs the heavy collection work in a background thread so the async event loop (and thus the streaming pipeline) is not blocked. Once the work finishes any events that accumulated in the emit queue during the collection are flushed immediately.

  Returns¶

  dict[Literal[“status”, “message”, “data”], Union[str, dict]]

  Message with metadata events or error.

• trace.reset

  async handlers.trace_handler.TraceHandler.reset(self)¶

  Resets the trace buffer.

• trace.collect

  async handlers.trace_handler.TraceHandler.collect(self) → dict['status' | 'message' | 'data', str | dict]¶

  Provides the increment of the trace buffer not yet sent.

  Returns¶

  dict[Literal[“status”, “message”, “data”], Union[str, dict]]

  Message with all events present in the buffer or error.