Skip to main content
BlogZephyr Developer Summit

Developer Tools for Debugging

By October 19, 2023No Comments

The Zephyr Developer Summit, hosted under the first-ever Embedded Open Source Summit in Prague, Czech Republic, on June 27-30 included presentations, BoFs, and training designed for real time problem solving and deep discussions. More than 1,300 people registered for the EOSS conference – representing 375 organizations across 56 countries around the globe. Zephyr had 75+ technical sessions (in-person and on-demand) for 3 tracks focused on users of Zephyr, developers contributing upstream, and maintainer-specific topics.

All of the videos from the Zephyr Developer Summit can be found on the Zephyr Youtube Channel. Each week, we’ll highlight a few videos in a blog for easy access. Today, we’re featuring a few sessions focused on developer tools for debugging including, “Overview of Logging,” “Tokenized Logging with Pigweed,” “Adding Coredumps to Your Debugging Toolkit,” and “Using Customerized QEMU Co-simulation for Development and Test of Zephyr Applications.”

Overview of Logging – Aastha Grover, Software Engineer at Intel Corporation

The logging subsystem has various capabilities that we can use to our advantage in integrating embedded software into the underlying hardware. Loggers can process both messages issued by developers and trace date, both of which can be used for analysis and debugging. This presentation gives an overview of the logging subsystem in general and talks about different logging formats (MIPI SyS-t, dictionary, text, etc.), their usage, and how they provide better ways to debug and log. It will also show users how to enable/disable logging and switch formats at runtime.

YouTube player

Tokenized Logging with Pigweed – Al Semjonovs, Software Engineer at Google

For space saving, Zephyr’s logging subsystem can be enabled to provide a dictionary based logging system. This creates a JSON database file in the build directory which contains information to correctly parse log data.  This database file only works with the build it’s generated with.  Other builds cannot use the same database file as strings are mapped to their address.  At Google, we’re opting to use Pigweeds (PW) tokenizer logging module.  Instead of mapping strings to addresses, PW tokenizer maps strings to a hash.  When log format strings stay the same between builds, the hash stays the same. Any new strings are appended to the hash database. Addresses are almost entirely unpredictable between builds.  This reduces friction for developers in viewing logs when making changes while also reducing image size. We’re working on making pigweed as an additional Zephyr module which can be linked in with KConfig options. The hash database will be updated and stored as an artifact of each build.

YouTube player


Adding Coredumps to Your Debugging Toolkit – Eric Johnson,Firmware Solutions Engineer at Memfault

Investigating root causes of device faults is difficult, particularly for deployed devices. Typically local debugging and logging subsystems are used to determine issue causes. However, these tools have limited utility in many complex scenarios. Local debugging requires physical access, lacks a similar operating environment, and is invasive by modifying operation. Logging’s unstructured nature makes efficient and effect analysis difficult. Coredumps capture detailed insight into device operation at the time of the fault in a non-invasive manner. Coredumps can be used defensively to catch existing issues. This talk introduces coredumps and how to use them in practice with Zephyr. Coredump data a valueable but often underutilized debugging resource. Coredumps contain highly structured data that can be easily parsed using GDB. Coredumps are flexible regarding what data can be captured. Let your imagination run wild! Got a memory slab that needs usage monitoring? Add it to your coredump! Sensor data that causes a unique stack overflow? Add it to your coredump! Topics covered include a brief overview of Zephyr’s coredump subsystem, how to configure the subsystem for different scenarios, how to use GDB with a coredump, and some examples of coredumps in practice.
YouTube player
QEMU (Quick EMULATOR) is a free and open-source hosted hypervisor that can be used for co-simulation. It enables virtual machines to be run on multiple different architectures, and provides a range of features such as snapshot support, pause and resume, and support for different operating systems. QEMU is a great way to test and debug applications, develop software, or run multiple operating systems without impacting your regular system. It can also be used to test virtualized applications and environments. This presentation will explain how to enable QEMU for co-simulation of Arm® Cortex®-M33 and Cadence® Tensilica® Xtensa® DSP and how to use this simulator environment to develop and test Zephyr applications.
YouTube player
Watch the rest of the Zephyr Developer Summit videos here. The schedule and links to the PPT presentations can be found here. Photos from the EOSS can be found here.

For more information about the 2024 event, stay tuned by subscribing to the Zephyr quarterly newsletter or connect with us on @ZephyrIoTZephyr Project LinkedIn or the Zephyr Discord Channel to talk with community and TSC members.

Zephyr Project