Written by Stephen Berard, CTO of Atym
Resource-constrained devices such as microcontrollers (MCUs) and lightweight CPUs power the majority of the edge footprint, but developing software for these devices is historically time-consuming and rigid to innovation. At Atym, our mission is to enable containerization and cloud-native development principles for resource-constrained edge devices that are unable to support technologies such as Docker and Linux.
Our device edge orchestration solution enables developers to build, deploy, manage, and secure applications on fleets of constrained edge devices at scale. With Atym, developers can code in familiar languages (e.g. C, Rust, GoLang, Python) and package their apps on devices as containerized services using modern DevOps principles. Engineers knowledgeable in building distributed applications and services in the cloud will find the Atym experience to be familiar.
The Atym solution consists of the centralized Atym Hub and on-device Atym Runtime (Figure 1). The Atym Hub is available as a SaaS offering or for private cloud and on-prem deployments. Atym is purely in the infrastructure plane – focused on management, orchestration and security pf their devices and apps – we do not build apps, nor are we in the data path. Customers can keep their data on-prem and/or transmit it to the cloud based on need.
Figure 1. Atym Solution Architecture
The Atym Runtime is based on the Ocre project in LF Edge which Atym seeded with code this past August. Leveraging WebAssembly and Zephyr, Ocre supports OCI-like application containers in a footprint over 1,000x lighter than Linux-based container runtimes such as Docker and Podman.
Ocre provides a platform abstraction layer and container runtime for devices that work in tandem with their choice of orchestration backend, such as the Atym Hub. This enables developers to focus on building their edge solutions, rather than spending time on low-level device details.
With Ocre, developers do not require deep knowledge of the underlying hardware, Real-time Operating System (RTOS), or board support package (BSP). Developing applications for constrained edge devices with Ocre should be as easy as building cloud-native applications. You can find the Ocre code on the project GitHub.
In this article, I’ll dive into more details around the Ocre technology, discuss our migration from Free RTOS to Zephyr, and detail how we leverage Zephyr as part of our solution.
Ocre Architecture
The Ocre runtime consists of a base RTOS, board support package (BSP), and “glue code” that abstracts functionality into an open API. Optionally, native libraries for 3rd-party drivers can be linked in as well. When the device boots, it is the RTOS that starts and executes the Ocre runtime. The RTOS also provides the underlying OS services, hardware, and driver support. Ocre implements the container runtime and hardware abstraction layer and leverages the underlying RTOS implementation. This can be seen in Figure 2.
Figure 2 – High-Level Ocre Runtime Architecture
More technical detail on Ocre can be found in this blog on the Zephyr site.
Making the Switch
Initially, we leveraged FreeRTOS as the underlying operating system for what is now Ocre in LF Edge. While FreeRTOS is a popular choice for embedded applications, we found it to have limited library support and lacked overall flexibility. In contrast, the Zephyr RTOS has a modular, configurable design and offers a rich set of subsystems and libraries. It has broad hardware support with nearly 1,000 boards supported. Further, Zephyr is a truly open-source project governed by the Linux Foundation with a diverse and active community of contributors and sponsors.
We decided to move from FreeRTOS to Zephyr for several reasons. First, we wanted to have a consistent and integrated platform for our devices, without having to manage multiple libraries and middleware. Zephyr has a comprehensive set of tools for managing dependencies, toolchains, and builds.
Second, we wanted to leverage the benefits of Zephyr’s modern architecture, such as its scalability, flexibility, and reliability. Zephyr has a robust and secure kernel with support for multiple scheduling algorithms, memory protection, and fault handling. Third, we wanted to take advantage of Zephyr’s community and support. Zephyr is an open-source project, which means that we can contribute to its development and improvement.
Moving from FreeRTOS to Zephyr wasn’t something we considered lightly. That said, the benefits of this move have been well worth our effort. In the sections that follow, I will outline some of the key aspects of Zephyr and how we make use of them with our solution.
Key Advantages of Zephyr
The following is a summary of the key Zephyr features that Ocre leverages:
Board Support: Zephyr supports common architectures and instruction sets including ARM, x86, and RISC-V. There are currently nearly 1,000 community-supported boards across multiple architectures and vendors. Unlike FreeRTOS, it leverages concepts from Linux such as Devicetree to make supporting new boards quick and easy.
Security and reliability: It has a robust and secure kernel, which supports multiple scheduling algorithms, memory protection, and fault handling. Zephyr also provides security features such as cryptography, secure boot, and firmware updates.
Driver libraries: Zephyr includes built-in driver interfaces and libraries to support a plethora of sensors and devices. This includes IP networking, wireless communications such as Wi-Fi, Bluetooth® and LoRaWAN®, as well as power management functionality to ensure long battery life.
Modularity and configurability: Zephyr is highly configurable and modular, allowing developers to customize it according to their needs and preferences. The Zephyr RTOS can run on devices with as little as 8 KB of RAM, and it can scale up to support complex applications and hardware.
Open source and community: Zephyr is an open source project backed by the Linux Foundation, which ensures its long-term sustainability and vendor-neutral governance. The project has a vibrant and active community, which provides documentation, tutorials, forums, and mailing lists.
Board Support
Zephyr’s board support is one of its key strengths, as it gives developers and product manufacturers multiple options to solve their embedded RTOS challenges. Currently, the Zephyr community supports nearly 1,000 boards across architectures and vendors. Zephyr also provides drivers and samples for many sensors, radios, and peripherals that can be used with these boards. The Zephyr Project website has a complete list of supported boards and hardware devices.
Adding support for new boards is simple and generally requires creating a few configuration files. Existing board support files can be leveraged as a reference; something particularly helpful as devices evolve over time. The ability to quickly onboard a specific edge device is key for us to be able to support our customers.
One of the most exciting features of Zephyr is Devicetree. Devicetree has its root in the U-Boot and Linux projects. It is a powerful and flexible way to describe and configure the hardware system for use in Zephyr, enabling support for a wide range of devices and adapting easily to changes. It features a hierarchical data structure that provides a unified description of the hardware system.
Zephyr provides a set of Devicetree source files for each supported board, which describe the board-specific hardware and configuration. It also allows developers to add, remove, or change the properties of the devices in the Devicetree by using overlay (.overlay) or fragment files (.dtsi). These are helpful for supporting boards with one or more variants.
Security and Reliability
Security is a major tenet of the Atym solution and within the Ocre community. At boot, Ocre leverages a secure boot process through MCUboot for hardware equipped with this capability. This ensures that only trusted and validated firmware images get launched.
Once running, Ocre ensures that only trusted applications are allowed to execute. Our runtime has a fine-grained permissions model which only grants access to resources if the application has the required permissions. For example, a container that attempts to access the network will be denied if it doesn’t have the right permissions; this provides a robust defense-in-depth approach to edge application security. This functionality is built upon the security foundation of Zephyr and benefits from the hard work of the Zephyr community.
From a reliability perspective, Atym provides several advantages over the Ocre baseline. First, the Atym Runtime is commercially supported and provides a remotely manageable endpoint. Even if an application completely fails, the device can still be diagnosed and remediated remotely through its connection to the Atym Hub. Our commercial solution also provides a number of diagnostic and troubleshooting capabilities, including remote monitoring, log centralization, and remote restarts. We also make use of Zephyr’s watchdog timer to reboot/restart a device in the case it becomes unresponsive.
Internally, our team makes use of the Zephyr developer tools and features for monitoring, debugging, and optimizing code. This includes Tracealyzer, SystemView, Zephyr Shell, and Zephyr Tracing.
Drivers and Device Power Management
The Ocre runtime offers edge applications a consistent hardware abstraction layer, enabling containers and applications to run across a wide variety of edge devices with little or no change. Ocre accomplishes this by leveraging Zephyr’s Device Driver Model which provides a consistent approach for configuring the drivers that are part of a system. Each driver type (e.g., UART, SPI, I2C, etc.) is supported by a device class API and Ocre leverages these APIs to support hardware platform abstraction. This allows Ocre containers to call a consistent device API regardless of the underlying physical device or communication bus.
The following is a list of device classes supported by Zephyr:
- ADC: Analog-to-digital converter
- AIO: Analog input/output
- CAN: Controller area network
- COUNTER: Hardware counter
- DMA: Direct memory access
- EEPROM: Electrically erasable programmable read-only memory
- ENTROPY: Random number generator
- FLASH: Flash memory
- GPIO: General-purpose input/output
- I2C: Inter-integrated circuit
- I2S: Inter-IC sound
- IPM: Inter-processor mailbox
- LED: Light-emitting diode
- PINMUX: Pin multiplexer
- PWM: Pulse-width modulation
- RTC: Real-time clock
- SENSOR: Sensor device
- SPI: Serial peripheral interface
- UART: Universal asynchronous receiver/transmitter
- USB: Universal serial bus
- WATCHDOG: Watchdog timer
- WIFI: IEEE 802.11 wireless networking
Zephyr leverages Devicetree in two main ways: to describe hardware to the Device Driver Model, and to provide that hardware’s initial configuration. The Device Driver Model is a framework that allows Zephyr to automatically instantiate and configure devices based on the information in the Devicetree. The initial configuration of the hardware includes properties such as GPIO pin numbers, I2C addresses, clock frequencies, and so forth.
The Zephyr RTOS also supports advanced device power management, allowing the Ocre runtime to automatically manage device power and enter low-power states when not in use. This frees Ocre containers from having to manage device power management directly.
Modularity
Simplifying development of modular edge applications for constrained devices is one of the key benefits of the Ocre runtime. Engineers can leverage discrete containers to build microservice-based applications following a similar paradigm as they do in the cloud. Applications composed using a set of containers are more flexible, easier to test, and can take advantage of reusable components. Zephyr itself is designed in a modular fashion and we leverage this modularity in our runtime. This includes devices using Devicetree as described above, as well as the overall configuration using Zephyr’s Kconfig. This enables Ocre to support and adapt to new boards, devices, and application use cases.
Ocre is built as a Zephyr module, enabling us to easily integrate our solution with existing Zephyr firmware projects. In general, applications are containerized and deployed on the Ocre hardware abstraction and developers don’t have to be concerned with anything below. However, in advanced cases, embedded engineers can customize the underlying RTOS, BSP, and native libraries.
Open Source and Community
The Zephyr community is a vibrant and diverse group of developers, engineers, and users who collaborate to produce an open source RTOS that solves real-world problems. The community unites upstream code developers and product development engineers in an open, collaborative environment to produce an RTOS that is easy to deploy, secure, connect, and manage. The community also ensures balanced collaboration and feedback to evolve and meet the needs of its members.
The community is growing rapidly, with more than 2,400 contributors and nearly 1,000 supported boards as of October 2024. The community also hosts various events, such as webinars, workshops, hackathons, and conferences, to share knowledge, showcase projects, and foster connections. It also provides various resources, such as Discord, mailing lists, GitHub, documentation, and blogs, to communicate, collaborate, and learn.
We have found the Zephyr community to be a welcoming and inclusive place for anyone who is interested in using, developing, or contributing to the project. As is typical of Linux Foundation projects, the community values diversity, respect, and openness, and follows a code of conduct. It also offers mentorship, guidance, and support for newcomers and experienced members alike. The Zephyr community is a great place to be if you want to be part of an innovative and impactful open source project and Atym is proud to be involved.
Challenges
There’s no doubt about Zephyr’s capacity to accelerate the development of embedded software. After all, it’s an RTOS with batteries included. Migrating to Zephyr has helped us focus on building our core product, rather than spending cycles on all of its underlying dependencies. Still, low-level embedded software development is challenging and we need to keep our expectations realistic as we work to abstract this complexity from our customers.
Despite Zephyr’s extensive offerings, not all of its drivers or development board configurations receive the same attention or usage. In the course of our development we have uncovered some issues related to driver quality, which has taken time and attention away from our core objectives. It can be frustrating to take a dependency on a new driver that’s offered out of the box, only to find it isn’t operating the way you expect it should.
But this is a minor challenge in the greater scheme of things since we still spend far less time focusing on drivers. We’ve also been able to easily engage with the Zephyr community and its maintainers, which we have found to be open and responsive. This has enabled us to solicit feedback and contribute back improvements that we’ve made to the broader community. We’re confident that the community will continue to make Zephyr better and better, further minimizing these types of challenges.
Conclusion
Embedded development is challenging due to the constraints imposed by the hardware. At Atym, we’re working to abstract embedded development complexity to deliver a cloud-native experience on MCU-based devices that can’t run Linux. Ocre also serves as an alternative to Docker with Linux to free up memory space on more capable CPU-based devices. In turn, the Atym solution provides customers with a central orchestration hub that includes enhancements for security and on-device performance, in addition to commercial support for Ocre.
Our mission is to empower organizations with modern tools that simplify building, deploying, managing, and securing applications on constrained edge devices at scale. We selected Zephyr as our baseline RTOS due to its modularity, extensive board and driver support, security, reliability and vendor-neutral open source community. Zephyr is a key enabler for our mission and we’re excited to be part of this vibrant community!
For more information about the 2024 event, stay tuned by subscribing to the Zephyr quarterly newsletter or connect with us on @ZephyrIoT, Zephyr Project LinkedIn or the Zephyr Discord Channel to talk with community and TSC members.