Almost 700 people registered for the first-ever Zephyr Developer Summit, which took place virtually on June 8-10, to learn more about the RTOS. We had 3 tracks, 5 mini-conferences, 28 sessions and 51 speakers who presented engaging technical content, best practices, use cases and more. We’ll be adding event videos each week to the Zephyr Youtube Channel. Stay tuned here for more videos.
Today, we’re featuring all of the presentations that were featured under the “Testing Mini-Conference.”
On-target testing with Twister and the process of results publishing – Maciej Perkowski, Nordic Semiconductor
The twister framework is constantly evolving to better serve the testing needs of Zephyr-based projects. During my presentation, I will bring the developers up to date with the current state of testing with Twister. I will present a short introduction on how to use Twister for on-target tests and will show how to use some new or less common features of the framework (e.g. quarantine). Validation of Zephyr performance on real hardware becomes an important part of QA. Contributors’ reports allow us to have a wider test scope of actual hardware running Zephyr applications. Therefore, during the presentation, I will also guide the contributors through the process of on-target results publishing.
Twister, a Powerful Test Runner for Zephyr OS Automation Testing – Peng Chen, Intel
As a powerful test runner, Twister is a critical part of zephyr automation testing framework. The Intel Zephyr team is using Twister to do daily testing upon 10+ hardware platforms with multiple architectures, collect test results and build CI testing. This presentation introduces the work process and fancy features of twister script, with the following perspectives:
- Twister work process, including finding and building test cases, and get test results.
- How twister judges a test case failure or success, and what the result “passed” “failures” “errors” means.
- How to run twister command on a qemu or hardware platform.
- How to enable twister automation testing on a new platform which doesn’t support common “west flash” command to transfer test images.
- How to add a new test case and build the test case with twister script.
- Future improvements
Visualizing Zephyr’s Health – Shihao Shen, Intel
Performance of the Zephyr OS on different platforms is always what we are concerned with. There are multiple indicators of Zephyr’s “health”, so it’s critical to present them in a coherent, visually immersive way. I would like to introduce two tools, Power BI and Grafana, that Intel Zephyr team have been using to better visualize different data of the Zephyr project.
The most useful aspect that comes with Power BI is its ability to integrate multiple tables from our database or local excel sheets into one single but elegant visualization. The usage of Power BI reduces manual input and hence largely improves the efficiency of a team on Zephyr. Before the introduction of Power BI, we spent hours manually filling out tables in PowerPoint, which was now automated by using Power BI to self-generate the same reports every time when data are updated on the database. The time spent has been significantly reduced from 5+ hours to 10 minutes per weekly report.
Grafana is another visualization platform. Compared to Power BI, Grafana is more light-weighted, user-friendly, and portable. Using Grafana to present data compensates some drawbacks of Power BI. We have added one more stage in the pipeline of our daily test server which sends the immediate test results to the database that Grafana uses for visualization. In this way, Grafana serves as a dynamic way to monitor the daily test results and parties interested can access it at any time, unlike Power BI as a more static presentation refreshed on a weekly basis.
Both visualization tools combined will benefit Zephyr in that it largely improves the team’s efficiency as well as successfully visualizes Zephyr’s test data.
Functional Safety Verification & Validation Test Case Development and the Challenges – Steven Wang, Intel
Functional safety (FuSa) certification for IEC61508 standard is critical for such as automotive industry, industrial section, etc. According to IEC61508, verification and validation test cases are for verifying module/architecture design and validating safety requirements. Intel is working with TuV to get FuSa kernel parts of Zephyr certified.
This presentation shows how we develop the test cases for FuSa(Functional Safety) certification with examples. To meet FuSa specification, the test cases are designed and implemented with the techniques and measures defined in IEC61508. It turns out that the test cases help us to identify useless code and potential bugs. Moreover, we are seeing several challenges, such as how to define module and integration test cases, how to improve code coverage, and how traceability is set up between the test cases and the documents. This presentation illustrates the solutions to tackle the challenges.
If you have questions or would like to chat with any of our Zephyr speakers, ambassadors or members of the Technical Steering Committee (TSC), please join us on the Zephyr Slack.
