The Embedded Open Source Summit took place from April 16-18 in Seattle, Washington, alongside the Open Source Summit North America. The Zephyr Developer Summit was part of the EOSS, aimed at developers using or considering Zephyr in embedded products. This year, we concentrated on supporting topics relevant to Zephyr users, upstream contributors, and maintainers.
More than 860 individuals from 721 organizations across 46 countries registered for the EOSS conference. The Zephyr track featured over 50 technical sessions, both in-person and on-demand, focusing on users, upstream developers, and maintainers.
Videos from the Zephyr Developer Summit are available on the Zephyr Project YouTube channel. We will highlight a few videos each week in a blog for easy access.
- Graphing Zephyr Development and Project Health and How Visualizing Data Help with Growth and Scaling – Anas Nashif, Intel
- Generate, Develop and Visualize Zephyr Platforms and Apps with Visual System Designer and Renode – Michael Gielda, Antmicro
- From a Weekend Hack, to a Future-Proof IoT Device – Benjamin Cabé, The Linux Foundation
- Flexboard: Developing Zephyr on a Keyboard That Runs It – Daniel DeGrasse, NXP
Graphing Zephyr Development and Project Health and How Visualizing Data Help with Growth and Scaling – Anas Nashif, Intel
The Zephyr project has been systematically tracking development metrics since its inception eight years ago, with the initial focus on basic indicators like the number of commits, boards, and drivers added monthly. This tracking has allowed the project to monitor its growth and health over time, establishing a foundation of consistent development and expansion in hardware support, community contributions, and the codebase itself. As the project evolved, so did the scope of metrics tracked, expanding beyond basic statistics to include more comprehensive key performance indicators (KPIs) that are critical for assessing the project’s health and performance.
These KPIs include metrics related to project health such as review times, issue resolution times, and overall performance metrics like footprint and kernel context switching times. By tracking these indicators, the Zephyr project aims to maintain a high level of transparency and accountability towards its community, ensuring that the project remains true to its initial goals, including support for constrained devices, minimal footprint, and compatibility across different architectures.
The presentation highlights several case studies and examples of how these metrics are tracked and utilized to guide project decisions. For instance, the analysis of contributor activity over time reveals a steady influx of new contributors alongside a stable core group that has been involved for several years. This data is crucial for understanding the dynamics of the project’s contributor base and ensuring that the project remains open to new ideas and contributions while retaining a strong foundation of experienced developers.
Performance tracking has been another critical area of focus, with the project using benchmarks like Latency Measure to track key performance metrics over time. These benchmarks help identify both positive and negative trends in the project’s performance, allowing developers to pinpoint the causes of any regressions or improvements. For example, the transition to the new memory allocator Biny had a significant positive impact on the project’s footprint and performance metrics, as reflected in the data.
In addition to performance metrics, the project has also focused on ensuring that code changes are reviewed and merged in a consistent and fair manner. By tracking the adherence to merge criteria, including the requirement for multiple approvals and a minimum review period, the project has been able to improve the quality and consistency of its codebase. The introduction of dashboards to monitor merge criteria compliance has led to a significant reduction in the number of pull requests merged without sufficient review, further enhancing the project’s overall quality.
The presentation also discusses ongoing efforts to consolidate all these disparate metrics into a unified dashboard using tools like Kibana and Elastic Search. This consolidation aims to make it easier for stakeholders to access and analyze the data, as well as to establish automated alerts for significant changes in performance or other key metrics. The goal is to identify and address issues as early as possible, ideally before they are merged into the main codebase.
Finally, the Anas touches on future plans for expanding the scope of metrics tracked, including the use of real hardware for performance testing and the addition of more detailed metrics for specific subsystems like Bluetooth and networking. These efforts are part of the broader goal of continuously improving the Zephyr project’s quality, scalability, and adherence to its core principles. The key takeaway is that tracking and visualizing project data not only helps enforce guidelines and principles but also provides an early warning system that is transparent and accessible to all stakeholders, ultimately contributing to the project’s long-term success and growth.
Generate, Develop and Visualize Zephyr Platforms and Apps with Visual System Designer and Renode – Michael Gielda, Antmicro
This talk, presented by Michael Gielda from Antmicro, introduces an open source, end-to-end flow designed to facilitate the design, generation, building, running, and visualization of Zephyr applications on custom platforms using the Renode simulator and Visual System Designer (VSD).
The presentation begins with an overview of the complexities involved in building real-world systems, particularly when dealing with varied and evolving hardware configurations in large-scale, multi-node environments. Michael emphasizes the need for structured methodologies and tools to manage these complexities, highlighting Zephyr’s inherent capabilities, such as its configuration-based approach and device tree system, which support structured development.
Michael discusses the challenges posed by the rapidly growing and diversifying hardware landscape, with increasing numbers of SoCs, boards, and modules being developed by various vendors. He underscores the importance of open source collaboration in managing this complexity and points to Zephyr as a prime example of a successful community-driven project that enables efficient and scalable product development.
The core of the presentation focuses on VSD, a tool that allows developers to graphically design custom hardware platforms by creating parameterized block diagrams from a library of real-world components. These diagrams enable the combination of SoCs with external sensors, actuators, and I/O interfaces, automatically generating the corresponding Zephyr platform descriptions and basic application code. The integration of VSD with Renode further enhances this process by allowing the generated software to be seamlessly run on the newly designed platform and visualized in a live simulation.
The talk includes a demonstration of VSD in action, showing how a system can be designed, generated, and simulated in Renode, highlighting the tool’s capability to dynamically generate and run binaries based on the structured data from Zephyr’s device trees. This demonstration illustrates how VSD and Renode can streamline the development process, enabling rapid iteration and testing of custom hardware designs.
Additionally, Michael touches on the broader implications of this work, including its potential application in silicon development and the integration of proprietary and open-source IP blocks. He discusses the ongoing efforts to expand the functionality of VSD, including plans to enhance graphical configuration capabilities and support for complex multi-node systems. The session concludes with a discussion on the future of VSD, including its upcoming public release and the goal of making it a comprehensive tool for both hardware and software design in the Zephyr ecosystem.
Throughout the presentation, Michael emphasizes the importance of automation, consistency, and scalability in modern embedded system development, advocating for a more structured and data-driven approach to managing the increasing complexity of hardware and software interactions in embedded systems. The talk showcases how VSD and Renode, in conjunction with Zephyr, offer a powerful, integrated environment for developing, testing, and visualizing embedded systems in a more efficient and scalable way.
From a Weekend Hack, to a Future-Proof IoT Device – Benjamin Cabé, The Linux Foundation
In this talk, Benjamin Cabé from The Linux Foundation discusses his journey from a weekend hack to developing a future-proof IoT device using Zephyr. The project began during a long weekend in May 2020, when, amidst the pandemic, Benjamin sought to perfect his bread recipe. This led to the creation of an Arduino-based “artificial nose” that used a gas sensor and TinyML (small-scale machine learning) to detect scents, including the smell of a perfectly fermented sourdough starter.
Benjamin walks through the key features of the artificial nose, starting with its initial development using Arduino, and then explains how he refactored the original codebase to leverage Zephyr. This transition made it easier to extend the project and run it on various hardware platforms. He highlights several important concepts in this process, including:
- The shift from a complex “super loop” in Arduino to a well-architected system using threads and event-based programming in Zephyr.
- The integration of TinyML models (e.g., TensorFlow Lite) into the system without compromising the overall performance of the embedded system.
- The creation of an efficient and easy-to-maintain graphical user interface using Zephyr’s capabilities.
- The use of Zephyr’s hardware abstraction layer to make the system adaptable to different sensors and hardware configurations.
Benjamin also discusses the sensor technology used in the project, which includes a gas sensor capable of detecting various volatile organic compounds, alcohol, carbon monoxide, and nitrogen dioxide. He elaborates on the learning process, explaining how tools like Edge Impulse helped him bridge the gap between theoretical machine learning concepts and practical application in his project.
The talk emphasizes the challenges and benefits of using Zephyr for such projects, including better task management through threading, easier sensor data handling, and the ability to simulate and test the system efficiently. He demonstrates the artificial nose in action, showing how the device can detect different smells, such as lemon, and visualize the data in real-time.
Benjamin concludes by discussing potential real-world applications for the artificial nose, such as monitoring air quality in restrooms and other facilities to trigger cleaning operations. He touches on the future possibilities of the project, including the potential to detect more complex scents, such as COVID-19 markers, and the use of Zephyr’s advanced features like dynamic extensions (Lex) to update the system remotely.
The entire project is open source, and Benjamin invites the community to replicate, improve, and build upon his work, with all resources available on GitHub. The talk not only showcases a fun and educational project but also illustrates the broader implications and potential of combining IoT, AI, and open-source software in innovative ways.
Flexboard: Developing Zephyr on a Keyboard That Runs It – Daniel DeGrasse, NXP
In this talk, Daniel DeGrasse from NXP shares his experience in developing the Flexboard, a mechanical keyboard powered by Zephyr. The Flexboard project was born out of a passion for mechanical keyboards and a desire to explore product design in a low-pressure environment. The keyboard, featuring 145 keys, includes per-LED backlighting and the ability to dynamically reload keymaps. Daniel explains the process of selecting components, including the NXP K22 MCU, and the challenges faced during the chip shortage.
Daniel discusses the initial bring-up process, where he chose to develop directly with Zephyr instead of using the manufacturer’s SDK. He created an out-of-tree board definition and a manifest repository with West to manage dependencies efficiently. During bring-up, Daniel encountered issues with existing support in Zephyr for the K22 MCU, leading to the decision to fork Zephyr to accommodate the unique register definitions and clock control registers required for the project.
He then delves into driver development, specifically for a temperature sensor and an LED controller. While upstreaming the temperature sensor driver was straightforward, the LED controller posed challenges due to its unique Auto Breath mode, which Zephyr’s LED API did not support. This led to a forked version of the driver being used downstream while a simpler version was upstreamed.
The talk then transitions to Zephyr Mechanical Keyboard (ZMK) firmware, which Daniel used to power the Flexboard. ZMK is built on top of Zephyr and provides essential keyboard functionalities like key matrix scanning and special key behaviors. Daniel highlights the limitations of ZMK, such as the need to recompile firmware for any keymap changes, which prompted him to develop a dynamic keymap configuration feature using USB HID feature reports. This allows for reconfiguring the keymap without needing to reflash the firmware, making the keyboard more user-friendly, especially for non-technical users.
Daniel also reflects on the challenges of maintaining a downstream fork and the realities of product development versus upstream contributions. He acknowledges the difficulty of staying up-to-date with Zephyr releases and the technical debt that comes with maintaining a fork. He concludes with insights on potential improvements to Zephyr, such as better support for out-of-tree SoCs and the need for a more robust event subsystem like zbus, which could benefit projects like ZMK.
Overall, the talk provides a comprehensive look at the process of developing a Zephyr-powered product, from initial bring-up to dealing with the practicalities of upstream and downstream development. Daniel’s experience highlights both the power and the challenges of working with open-source firmware in a real-world product design context.
Watch the rest of the Zephyr Developer Summit videos here. The schedule and links to the slides can be found here. Photos from the EOSS can be found here.
For more information about the 2024 event, stay tuned by subscribing to the Zephyr quarterly newsletter or connect with us on @ZephyrIoT, Zephyr Project LinkedIn or the Zephyr Discord Channel to talk with community and TSC members.