Zephyr @ Linux Plumbers 2022 (Video)

Written by Christopher Friedt, Embedded Software Engineer at Meta and a member of the Zephyr Technical Steering Committee

At Linux Plumbers 2022, hosted on September 14 in Dublin, Ireland, Zephyr was once again a major theme at the IoT Microconference. This IoT MC was the largest ever to date with 7 speakers delivering great talks both in person and virtually. Watch the recording of the live stream or read the summaries below.

Putting firmware on the device: a Zephyr+Yocto+Mender hike

We began the day with a guided tour by Mr. Josef Holzmayr (a.k.a. The Yocto Jester). In his talk (slides, video), the Jester detailed the process of using Yocto to package a custom embedded solution with both Linux and Zephyr powering a NXP i.MX7 system. In this AMP-scenario, Linux runs on the Cortex-A application core, Zephyr runs on the Cortex-M4 core, and the two communicate with one another via RPMsg.

A focus of the talk highlighted a few common pitfalls in coordinating product deployment between firmware engineers, software engineers, and product engineers. These pitfalls can typically be broken down into the areas below.

• Composability – build for several architectures, pull in middleware, link to infrastucture
• Reproducibility – Yocto assists in ensuring builds are reproducible
• Compliance – SBOM
• Security – CVE fixes included to close security gaps

The additional area where Mender really can help is in

• Deployability – is there a simple, conventional way update products in the field?

The Jester finished with a live demo showcasing how Mender is able to orchestrate all of the above using a .kas file. After a few questions, he had to depart for ELCE just a few blocks away to give another talk! We debated following him out the door and into the taxi Letterman-style, but bandwidth and signal quality was a concern. In any case, fantastic talk.

Open source FPGA NVMe accelerator platform for BPF driven ML processing with Linux/Zephyr

Karol Gugala from AntMicro presented a platform developed in collaboration with Western Digital for hardware Machine Learning accelerators over NVMe (slides, video). This is again an AMP-scenario where the Linux host, running on a Cortex-A applications processor, offloads complex AI operations to Zephyr, running on the Cortex-R NVMe controller, to be executed in custom FPGA fabric with direct access to NVMe storage. The TensorFlow-Lite Machine Learning pipelines are designed using Apache TVM-VTA and are themselves run in a uBPF virtual machine. In this capacity, Zephyr operates as a custom NVMe command processor mainly to orchestrate DMA operations and kick off custom ML instructions.

While the system was initially designed to operate in the datacenter, it can also be scaled-down to operate directly in accelerated systems e.g. Smart Cameras. Additionally, while prototyping used an FPGA, it is entirely feasible that one could synthesize the ML accelerator gateware and a custom RISC-V SoC.

This talk covered so much technical ground in a such a brief window. For those interested in the details, please refer to links and repositories provided in the slides.

Abusing zephyr and meta-zephyr

Eilís Ní Fhlannagáin revamped the OpenEmbedded meta-zephyr layer (slides, video). This was done as a part of the Oniro project.

Eilís began the meta-zephyr layer and returned to it after a hiatus to learn that it had not scaled-up with upstream Zephyr. The main issues at the time were:

• inaccurate / duplicate configurations for machine, project, and kernel configs
• lack of flexibility for adding ZEPHYR_MODULES
• incomplete SRC_URI (critical for bitbake to fetch sources)
• only 8 machines supported (out of ~400)
• not Yocto-compatible (because BSP layers were mixed with the OS recipe)

Going back and forth to the drawing board a few times, coming up with several solutions that were really not easily maintainable, Eilís finally realized that the best way for her to integrate Zephyr’s metadata was to embrace west, which is Zephyr’s self-described “meta-tool”.

With west, Eilís was able to

• generate 291 machine definitions using west list
• use -DZEPHYR_MODULES (with west / cmake) flexibly add modules

The original meta-zephyr recipe was then split into meta-zephyr-bsp and meta-zephyr-core, which meant that it would be Yocto-compatible going forward.

Additionally, the meta-zephyr layer supports projects based on both the 2.7.1 branch as well as the 3.0.0 branch.

Eilís closed with a demo of the new and improved DJ in a Box: a Bluetooth MIDI device running Zephyr (the glove) and the synthesizer box running Linux.

Linux IEEE 802.15.4 MLME improvements

Miquèl Raynal’s talk focused on adding long-awaited IEEE 802.15.4 MLME features to the Linux kernel (slides, video).

IEEE 802.15.4 is of course the wireless standard that specifies the physical and MAC layers for Zigbee, Thread, Matter, Smart Home and Smart Metering devices. So it is very much at the center of IoT.

As you may have guessed, Miquèl chose to test all of his Linux kernel changes against the Zephyr 15.4 stack using an Arduino Nano 33 BLE. On the Linux side, of course, the prolific ATUSB adapter was used to prototype the new MLME features.

What is an MLME, you ask? Well, the MLME (MAC Sublayer Management Entity) features are what allow our wireless devices to

• scan beacons from other nearby devices
• transmit beacons to other nearby devices
• associate and disassociate with networks
• keep devices synchronized
• handle faulty scenarios (link loss, address conflicts, etc)

In plain terms, these are the same type of features that let us interactively view and join WiFi networks with our laptops or smart-phones – but for IoT devices.

It would be an understatement to say that these features should have been added to Linux 10 years ago, but with the majority of IoT outfits at the time focusing mainly on Zigbee and dealing with proprietary vendors, there was a distinct lack of resource. Previously, in IoT under Linux, we have mainly been hard-coding network address parameters for test purposes, or have simply been offloading those tasks to a network coprocessor.

A big thank-you to Miquèl who did the entire community a service by adding these long-outstanding features to Linux!

Libre Silicon in IoT

Our last speaker for 2022 was Michael Welling of QWERTY Embedded Design (slides, video).

In the last couple of years, Michael has worked with an all-star lineup of independent electrical, analog, and digital design engineers to create two amazing community-driven products:

• PyFive: a custom RISC-V SoC built entirely with free tools and synthesized on MPW-1
• ICE-V Wireless: a new dev board featuring an ICE40UP5K FPGA and an ESP32C3

The above projects would not have been possible (in a completely free and open way) without:

• Yosys+nextpnr arxiv: the Open Source Synthesis toolchain by Claire Wolfe
• KiCAD: The Open Source EDA suite
• eFabless, Google, and the SkyWater PDK
• A network of brilliant engineers (Greg Davill, Sylvain Munat, Eric Brombaugh, …)

The list could probably go on for a long time.

In any case, the idea is that the community has mobilized to commoditize not only EDA products like KiCAD, but also more recently FPGA synthesis tooling, and even more recently than that, the silicon PDK (Physical Design Kit). The mature reader might reminisce to the days when we paid (sometimes quite a bit of money) for a C compiler. Even 20 years ago, a license for a single seat for FPGA synthesis tools was prohibitively expensive for aspiring digital designers. Mature digital engineers probably remember OpenCores.org, OpenRISC, and so on. The community was there, they wanted to share their designs, but 20 years ago, FPGA synthesis tooling was a somewhat expensive bar-to-entry.

With the rise of open source tooling, we are now seeing all of the various IP blocks (some digital some analog), for end-users to create a completely free SoC.

The ICE-V Wireless does have support in Zephyr, for those interested and additional work is ongoing on GitHub. Driver support for flashing the ice40 from Zephyr is on its way into main.

Note: The ICE-V Wireless project has been so successful that a second crowdfunding campaign has begun on GroupGets.

Additionally, there is a first-of-its kind campaign to crowd-fund production of the PyFive ASIC as well on GroupGets.

Conclusion

The 2022 Linux Plumbers Conference was a huge success again for those who attended in-person or virtually. In fact, the hybrid aspect of the conference has actually been somewhat popular. It was great to see familiar faces in the kernel community again as well.

As usual, each presenter was able to engage in invaluable discussion with their respective communities either face-to-face or via Matrix chat, and we all have new action items and exciting new projects for the year ahead.

There were two additional talks not covered here from the IoT Microconference:

• libgpiod V2: New Major Release with a Ton of New Features by Bartosz Golaszewski
• All types of wireless in Linux are terrible and why the vendors should feel bad by Peter Robinson

Actually, Bartosz is the maintainer of the LVGL module in Zephyr, and Peter Robinson gave kudos to Nordic for releasing regression-free wireless firmware (based on Zephyr). So it would really be impossible to say that Zephyr was not a part of every presentation in some way.

In any case, all of our presenters have really done a tremendous job enabling others to create better Linux and Zephyr based products.

Thanks to all of our presenters this year, and we hope to see you in 2023!