Instrumentation¶
The instrumentation subsystem leverages the GCC feature to insert custom code right before and after a function calls for profiling and tracing applications.
Capturing trace¶
Currently instrumentation subsystem supports one tracing backend - UART.
Zephelin provides a zpl-instrumentation-uart-capture West subcommand to collect instrumentation traces:
west zpl-instrumentation-uart-capture [-h] \
serial_port serial_baudrate output_path
It requests the trace by sending dump_trace message and waits for a response.
To make sure the received data is not contaminated by e.g. log messages, it is sent wrapped with -*-#...-*-!.
Base on that, the command finds the beginning and the end of message and saves trace in CTF format.
In order to visualize the trace with Zephelin Trace Viewer, it has to be converted using west zpl-prepare-trace -i {INSTRUMENTATION_CTF_TRACE} {CONVERTED_TRACE}.
For more details, check out CTF to TEF conversion.
Instrumentation with Zephelin’s features¶
Instrumentation subsystem can also be used together with the rest of the Zephelin tracing systems.
The traces from the instrumentation subsystem are collected separately from the remaining tracing data. The requirements for collecting all data are following:
A different backend or connection type for instrumentation subsystem and the rest of traces - can be e.g. UART backend for instrumentation and RAM backend for other traces.
A CTF format for the remaining traces (configurable with
CONFIG_ZPL_TRACE_FORMAT_CTF=y)
With this, a following West subcommand can be used to capture both instrumentation and Zephelin traces at the same time:
west zpl-instrumentation-uart-gdb-capture [-h] \
serial_port serial_baudrate instr_output_path output_path
The command uses UART to capture instrumentation trace (with the same logic as zpl-instrumentation-uart-capture) and GDB is used to extract remaining traces from RAM (as in zpl-gdb-capture).
The produced files have to be converted to TEF and combined into one trace with west zpl-prepare-trace -i {INSTRUMENTATION_TRACE} {ZEPHELIN_TRACE} {TEF_TRACE}.
The merging mechanism also ensures that traces do not collide and can be visualized.
Interactive version of instrumentation examples:
Additional options¶
Running without retained memory¶
By default, the instrumentation subsystem uses retained memory in order to configure trigger and stopper functions via UART, and persistently store them as long as the device is powered.
Instrumentation subsystem can also be run without retained memory by disabling CONFIG_INSTRUMENTATION_DYNAMIC_TRIGGER.
Then, trigger and stopper functions have to be selected with CONFIG_INSTRUMENTATION_TRIGGER_FUNCTION and CONFIG_INSTRUMENTATION_STOPPER_FUNCTION specifying mangled names of the functions.
Enabling and disabling instrumentation for Zephelin internals¶
A function can be excluded from the instrumentation in a few ways:
__no_instrumentation__attribute added to the function definition,function name (not mangled) added to the
CONFIG_INSTRUMENTATION_EXCLUDE_FUNCTION_LIST,file path containing the function code added to the
CONFIG_INSTRUMENTATION_EXCLUDE_FILE_LIST.
Zephelin disables instrumentation of its internal functions by default to not pollute traces with events unrelated to the profiled application. It is achieved by adding source files to the exclude file list and using:
__no_zpl_instrumentation__- extension of__no_instrumentation__attribute that can be disabled with a custom config,ZPL_DISABLE_INSTRUMENTATION- macro disabling instrumentation for a given code scope, it is not compatible with mechanisms jumping out of the given code scope.
In order to enable instrumentation for Zephelin’s internal functions the CONFIG_ZPL_INTERNALS_INSTRUMENTATION can be used.
It prevents source file from being added in the exclude list and disables both __no_zpl_instrumentation__ and ZPL_DISABLE_INSTRUMENTATION.
Dumping events when instrumentation buffer is full¶
Instead of disabling instrumentation subsystem when buffer is filled or overwriting the existing data, the subsystem can also dump all events to make space in the buffer.
This option can be enabled with CONFIG_INSTRUMENTATION_MODE_CALLGRAPH_DUMP_ON_FULL=y.
The zpl-instrumentation-uart-capture and zpl-instrumentation-uart-gdb-capture commands can automatically detect whether DUMP_ON_FULL was used and adjust capturing mechanism accordingly.
Similarly to Zephelin, instrumentation subsystem sends init tag at the start of the application.
It is detected by the West commands, to save traces into separate file, therefore it is advised to start zpl-instrumentation-uart-capture command before flashing or restarting the board.
Next, the commands wait for binary messages until user stops it or there is no new message received in specified timeout.
Also, to make sure all events are captured, after the timeout, the commands send dump_trace and gather remaining data from the buffer.
The example of this mechanism can be found in samples/profiling/tflm_instrumentation.