The Embedded Open Source Summit took place from April 16-18 in Seattle, Washington, alongside the Open Source Summit North America. The Zephyr Developer Summit was part of the EOSS, aimed at developers using or considering Zephyr in embedded products. This year, we concentrated on supporting topics relevant to Zephyr users, upstream contributors, and maintainers.
More than 860 individuals from 721 organizations across 46 countries registered for the EOSS conference. The Zephyr track featured over 50 technical sessions, both in-person and on-demand, focusing on users, upstream developers, and maintainers.
Videos from the Zephyr Developer Summit are available on the Zephyr Project YouTube channel. We will highlight a few videos each week in a blog for easy access.
Today we are featuring:
- Zephyr Application Development Using TDD: Getting Started – Luis Ubieda, Croxel, Inc.
- Pytest Tests in Twister – Maciej Perkowski, Nordic Semiconductor
- Streamlining Testing with Generic Tests and Emulators – Tristan Honscheid, Google
- Unit Testing Memory Mapped Device Drivers – Jeremy Bettis, Google, LLC
Zephyr Application Development Using TDD: Getting Started – Luis Ubieda, Croxel, Inc.
The presentation by Luis Ubieda at the Zephyr Developer Summit focused on introducing and advocating for Test-Driven Development (TDD) in the context of firmware application development using the Zephyr RTOS. Luis detailed the TDD methodology which involves writing tests prior to code to incrementally guide development, which can significantly impact and enhance the progress of the development cycle, reducing time-to-market for products.
Luis explained that TDD has not been widely adopted in firmware development, possibly due to misconceptions about its practicality. By demonstrating TDD’s effectiveness, especially when integrated with complex systems like Zephyr (which involves elements such as Device Trees, Kconfigs, and modules), he aimed to illustrate how developers can overcome initial fears and complexities.
The talk included practical examples and a live demo, showcasing how to implement TDD with a Zephyr-based device featuring an accelerometer and Bluetooth Low Energy (BLE).
The talk included basic concepts of TDD, the frameworks available within the Zephyr ecosystem, and strategies for mocking dependencies, such as device-tree devices, drivers, and subsystems.
Furthermore, the presentation emphasizes the importance of TDD in fostering a development culture that is responsive and adaptive to evolving project requirements. Through continuous testing and refactoring, developers can ensure their code meets the high standards required for production while minimizing the risk of post-deployment issues.
Luis’ presentation was aimed at both newcomers to TDD and those who have some experience but face challenges in its application, particularly in the context of Zephyr RTOS.
Pytest Tests in Twister – Maciej Perkowski, Nordic Semiconductor
Maciej Perkowski, a software test developer at Nordic Semiconductor, presented a session on the integration of pytest with Twister at the Zephyr Developer Summit. The session, titled “Pytest Tests in Twister,” highlighted the rationale behind incorporating pytest into Twister to enhance testing capabilities within the Zephyr ecosystem. Pytest is recognized for its powerful testing features such as fixtures and plugins which Twister traditionally lacked, enabling more dynamic and interactive testing scenarios across various environments, including hardware and QEMU.
The integration allows for the use of pytest’s robust plugin system and fixtures to create more flexible and modular tests that can handle complex testing scenarios, including interactions with the device under test during runtime. This capability is significant for advancing the quality of software testing in embedded systems by allowing for more comprehensive and interactive test cases.
Perkowski detailed how pytest has been incorporated into Twister through a dedicated plugin that functions as an executor within the Twister framework. This setup enables Twister to leverage pytest’s features while maintaining a seamless experience for users familiar with Twister’s existing workflow.
Examples were given showing the practical benefits of this integration, such as the ability to perform complex end-to-end tests, interact with hardware through shell commands, and utilize external tools like MCU manager for more detailed device interaction during tests.
Moreover, the talk addresses future potentials and improvements in test scripting and execution within the Zephyr ecosystem. The ability to handle complex testing scenarios, such as hardware interactions and network communications with minimal setup, promises a lower barrier to entry for robust testing practices. This integration not only advances the capabilities of existing testing frameworks but also broadens the scope for future developments in testing methodologies across different hardware and simulation environments.
Streamlining Testing with Generic Tests and Emulators – Tristan Honscheid, Google
The presentation by Tristan Honscheid at the Zephyr Developer Summit focused on enhancing testing efficiency in the Zephyr Project through the use of generic tests coupled with peripheral emulators. This approach, centered on a backend emulator API, facilitates the accelerated development of driver tests, reducing code duplication and leveraging existing frameworks. Tristan outlined the current methodology for Zephyr driver testing, which involves developing an emulator that replicates the behavior of the target peripheral. This is achieved by inputting expected values into the emulator, retrieving them via the driver, and comparing them to ensure accuracy.
Tristan introduced the concept of a standardized backend emulator API that allows tests to interact with emulators in a uniform way, thereby enabling the creation of “generic tests.” These tests can broadly cover many device drivers without needing specific details about each device, focusing particularly on sensor drivers. He discussed the implementation details, demonstrating how this testing framework integrates with Zephyr’s existing emulation systems, primarily using sensors as examples. The backend emulator API simplifies test creation by handling device-specific data internally and providing a standardized interface for setting and querying device states.
The discussion also touched on the potential issues and common bugs encountered in sensor drivers, highlighting the importance of extensive testing to ensure stability and reliability. He shared insights into the structure and functionality of emulators within Zephyr, which mimic hardware components like GPIOs and I2C buses, facilitating software testing in absence of physical hardware.
Furthermore, Tristan walked through the implementation of the testing framework using a digital compass sensor as a case study, explaining the necessary steps to integrate an emulator with the generic testing system. He emphasized the benefits of this approach, such as the ability to automatically validate the functionality of various sensor drivers across different operational parameters.
Tristan encouraged developers to adopt this streamlined testing methodology by contributing to emulator and API development for different sensors within the Zephyr ecosystem. This collective effort would enhance the project’s capability to deliver robust and error-free software, thus supporting a wide range of applications.
Unit Testing Memory Mapped Device Drivers – Jeremy Bettis, Google, LLC
In his presentation Jeremy Bettis from Google LLC discussed innovative strategies for unit testing memory-mapped device drivers, specifically focusing on the IT GPIO driver used in Chrome OS team projects. He emphasized the importance of increasing the amount of unit tests relative to integration and manual tests to ensure code stability and efficiency, supported by a testing pyramid model.
Jeremy outlined the challenges typically associated with testing device drivers that interact directly with hardware through memory-mapped registers, noting the common excuse that intercepting register accesses for unit tests is unfeasible. He introduced a method involving the EC_REGE macro, which was adapted to wrap register accesses and enable unit testing by simulating register manipulations. This approach allowed for the verification of the driver’s behavior in a controlled environment without direct hardware interaction.
The presentation detailed the technical setup for the test environment using Native Sim on a POSIX system, incorporating specific configurations and adaptations in the CMake build system to integrate IT-specific headers and simulate the device driver’s operating conditions. This included creating a pseudo environment where the memory addresses accessed by the driver were mapped to simulated registers, thereby avoiding segmentation faults typically caused by direct memory accesses in a standard testing environment.
Jeremy demonstrated the practical application of this testing framework by implementing a new feature-emulating level-based interrupts in a driver that originally supported only edge-based interrupts. He achieved full unit test coverage before integrating the new feature, ensuring robustness and functionality.
Furthermore, Jeremy highlighted the importance of high-level function testing over direct driver calls to include all logical layers and stressed the necessity of aligning tests with hardware specifications rather than just the driver code to avoid replicating bugs. He also discussed the potential pitfalls of testing, such as the misalignment between simulated and actual hardware behaviors, and suggested maintaining a comprehensive testing approach across different system levels to ensure coverage and system integrity.
The session concluded with a discussion on potential alternatives for capturing memory accesses, the capabilities of Native Sim for input testing, and the feasibility of manipulating linker behavior to control memory address allocations for peripherals.
Watch the rest of the Zephyr Developer Summit videos here. The schedule and links to the PPT presentations can be found here. Photos from the EOSS can be found here.
For more information about the 2024 event, stay tuned by subscribing to the Zephyr quarterly newsletter or connect with us on @ZephyrIoT, Zephyr Project LinkedIn or the Zephyr Discord Channel to talk with community and TSC members.