Open Source Summit North America 2022

Currently, Real-time Linux is evaluated using a black-box approach. While the black-box method provides an overview of the system, it fails to provide a root cause analysis for unexpected values. Developers have to use kernel trace features to debug these cases, requiring extensive knowledge about the system and fastidious tracing setup and breakdown. Such analysis will be even more impactful after the PREEMPT_RT merge. To support these cases, since version 5.17, the Linux kernel includes a new tool named rtla, which stands for Real-time Linux Analysis. The rtla is a meta-tool that consists of a set of commands that aims to analyze the real-time properties of Linux. Instead of testing Linux as a black box, rtla leverages kernel tracing capabilities to provide precise information about latencies and root causes of unexpected results. In this talk, Daniel will present two tools provided by rtla. The timerlat tool to measure IRQ and thread latency for interrupt-driven applications and the osnoise tool to evaluate the ability of Linux to isolate workload from the interferences from the rest of the system. The presentation includes examples of how to use the tool to find the root cause analysis and collect extra tracing information directly from the tool.