Rust in Linux

A fascinating aspect of Linux kernel development has been the attempts to switch away from C and assembly language to something else like C++. Linus Torvalds is notorious for caring a lot about the beauty of the kernel code as well as important features such as CPU and RAM usage and security. For him to accept any other language, it would have to be a pretty amazing language. For comparison, the developers tried adding C++ support to the kernel in the late 1990s, but it didn't pass muster.

The Rust programming language is the first serious attempt by kernel developers to add support to the kernel since then. Linus allowed a rudimentary first attempt to go into kernel version 6.1 in late 2022. Since then, many more Rust patches have been submitted. Unlike the case with C++, Rust doesn't seem to be leaving any time soon. Instead, it seems as though Linus is fairly committed to giving it a real try.

It's a surprising decision because Rust is so new. Rust was invented in 2006 while C++ was invented in 1979. Even today, Rust has hardly taken over the world. And yet it has been given this amazing pride of place in the Linux kernel. I'm not denigrating! It's fascinating to consider Rust's characteristics that would place it even above C++ as being welcome into the kernel development process. However, it's not at all surprising that Linus would evaluate a new language based on its merits rather than its popularity.

Incorporating Rust into the kernel takes a twisting and turning path. The portions of the kernel written in C and assembly need to connect with the portions written in Rust in such a way that there's no loss of speed or security, and in such a way that both Rust and C developers can write good and readable code.

For example, Wedson Almeida Filho recently posted a patch to introduce a mutex structure for Rust that, as he put it, "allows Rust code to use the kernel mutex idiomatically."

Wedson's patch modifies both C and Rust files in the kernel source to add this support. If you're curious what the Rust code would look like for a mutex initialization routine, here is a snippet of Wedson's patch:

macro_rules! new_mutex {
  ($inner:expr $(, $name:literal)? $(,)?) => {
  $crate::sync::Mutex::new(
  $inner, $crate::optional_name!($($name)?),$crate::static_lock_class!())
  };
}

Rust is a strange and fascinating language that is seemingly so different from C, but with features that make it a welcome addition to the kernel project.

Peter Zijlstra replied, "I despise all these stupid helpers … but I suppose it's the best they could come up with to interface the languages :/ The only hope is that the thing can do cross-language LTO [Link Time Optimization] or something to re-inline stuff."

Wedson replied:

"One thing we could [do to] improve the situation is to convert some of the existing macros into inline functions on the header files.

"We can't do it for all cases (e.g., cases like mutex_init that declare a new static variable when lockdep is enabled) but mutex_lock is just a function when lockdep is disabled, and just calls mutex_lock_nested() when it is enabled.

"How do you feel about this?"

In reply, Peter asked, "Can rust actually parse C headers and inline C functions ? I thought the whole problem was that it can only call C ABI symbols (which inline functions are not)."

Wedson replied:

"Rust can't. We use a tool called bindgen to read C header files and generate equivalent Rust (extern 'C') declarations for functions. The tool has been enhanced recently (https://github.com/rust-lang/rust-bindgen/pull/2335) to handle static inline functions by automatically generating helpers like the one above (in addition to the Rust decls).

"So the situation is improved in that we don't need to manually write (and commit) the helpers. It may improve further in the future if we get better integration of the languages."

Peter accepted this, saying that in that case, "feel free to convert macros to inline functions where the difference is moot. There is indeed no real reason for mutex_lock() to not be an inline function in that case." In response to Wedson saying that Rust could not actually parse C headers and inline functions, Peter added, "that's sad, I was hoping it would write the equivalent inline function in rust."

David Laight chimed in at this point, regarding converting macros to inline functions. He remarked, "mutex_lock() is probably ok. But there are cases where gcc generates much better code for #defines than for inline functions. Almost certainly because the front end gets to optimise #defines, but inlines are done much later on."

Marco Elver suggested using "static __always_inline" in C files for macros that would be better converted to inline functions. He asked, "Can bindgen deal with 'static __always_inline' functions?" And Miguel Ojeda replied, "If you mean the new feature where 'bindgen' generates wrappers automatically, it seems to handle them given '__always_inline' => 'inline' which is what I imagine it looks for [...], though I haven't tried to use the feature within the kernel yet."

Miguel, speaking for the Rust community, also said, "We initially minimized changes on the C side on purpose, but if maintainers are OK with that (modulo exceptions), it would be great."

The conversation ended there, though this thread is absolutely part of a much larger and ongoing discussion between the C and Rust communities of kernel developers. There are many Rust patches flying around these days that are being taken very seriously by both groups.

Rust is a highly impressive language. Its designers seem to take the position that they will build protection against memory leaks and segmentation violations into the language itself, at the cost of including a few relatively convoluted language features. They also seem to make it relatively easy to avoid these convolutions at the cost of writing slower code. So for someone who only wants the higher level language features, such as easy-to-use complex data structures in a fast compiled language, you can have that without too much language complexity if you don't mind taking a small speed hit. For people who want code that is fully speed-competitive with the C language, you can have that if you take advantage of Rust's more complex memory protection features.

I don't know why Linus chose to accept Rust into the kernel, but I speculate that it was precisely because of those memory protection features. There seem to be very few powerful low-level languages that incorporate such features, and it seems like a tough needle to thread. I believe Linus probably feels that the compromise of Rust's additional complexity as a language is a very fair trade for the protections it provides, which are also very difficult to find elsewhere. I'm fascinated to watch the further progression of Rust development in the Linux kernel.

Compiler and Kernel Frenemies

Theodore Ts'o posted a patch to update the ext4 filesystem code, in the course of which he mentioned that another patch by Matthew Wilcox needed to be applied on top of his, making a particular function return an error pointer instead of simply the null value.

The details of Matthew's patch (and Ted's patch) are not necessary to know, and both patches were accepted into the kernel. However, Linus Torvalds commented:

"[I]t would be wonderful if we could mark the places that return an error pointer as returning 'nonnull', and catch things like this automatically at build time where people compare an error pointer to NULL.

"However, it sadly turns out that compilers have gotten this completely wrong.

"gcc apparently completely screwed things up, and 'nonnull' is not a warning aid, it's a 'you can remove tests against NULL silently'.

"And clang does seem to have taken the same approach with 'returns_nonnull', which is really really sad, considering that apparently they got it right for '_Nonnull' for function arguments (where it's documented to cause a warning if you pass in a NULL argument, rather than cause the compiler to generate sh*t buggy code).

"Compiler people who think that 'undefined behavior is a good way to implement optimizations' are a menace, and should be shunned. They are paste-eaters of the worst kind.

"Is there any chance that somebody could hit compiler people with a big clue-bat, and say 'undefined behavior is not a feature, it's a bug', and try to make them grow up?

"Adding some clang people to the participants, since they at least seem to have *almost* gotten it right."

Nick Desaulniers replied to Linus, saying, "Good. I can feel your anger. Strike me down with all of your hatred, and your journey to the dark side will be complete. Your hate has made you powerful. Let the hate flow through you!"

Nick added that Clang's _Nonnull attribute did catch comparisons between null pointers and error pointers at build time, but that "it's not toolchain portable, at the moment. Would require changes to clang to use the GNU C __attribute__ syntax, too (which I'm not against adding support for)."

He also remarked, regarding Linus's statement that Clang got nonnull right in one spot but not in another:

"I just had this conversation maybe within the past month with Bionic (Android's libc) developers.

"Yeah, the nonnull attributes != _Nonnull 'attributes'. (Quotes because IIUC _Nonnull doesn't use the __attribute__ GNU C extension syntax). My understanding (which may be wrong) is that nonnull is implemented for compatibility with GCC, while _Nonnull was likely implemented by Apple (my guess; did not check) (so compatibility with GNU C __attribute__ syntax probably wasn't considered in code review)."

Nick added, "The Bionic developers are deploying _Nonnull throughout the codebase and intentionally not using nonnull which is dangerous (a teammate used the term 'Developer Hostile Behavior'). nonnull has implications on codegen, _Nonnull only affects diagnostics."

Linus responded to Nick's statement that Clang's _Nonnull attribute did indeed catch comparisons between null pointers and error pointers. He said:

"No, that's a warning about using it, not a warning about testing for NULL when it's there.

"I actually tested _Nonnull. It seems to work for arguments. But it does not work for return values.

"Of course, maybe there's some other magic needed, but it does seem to be sadly not working for us."

In terms of toolchain portability, and specifically the need to modify Clang to use the GNU C __attribute__ syntax, Linus said, "No need for using the __attribute__ syntax at all, that would _not_ be a show-stopper." He added, by way of letting the Clang developers know what would be workable:

"While it's true that it's the common syntax, and we sometimes use it explicitly because of that, it's by no means the only syntax, and we actually tend to try to have more legible wrappers around it.

"So, for example, we prefer using '__weak' instead of writing '__attribute__((__weak__))'.

"And no, it very much doesn't have to use __attribute__ at all. We already have things like

#define __diag(s) _Pragma(__diag_str(GCC diagnostic s))

"so we already use other syntaxes.

"End result: if it actually worked, I'd happily do something like

#define __return_nonnull _Nonnull

"in <linux/compiler-clang.h>, with then <linux/compiler-gcc.h> then just having

#define __return_nonnull

"along with a big comment about how __attribute__((nonnull)) is horrible garbage that should never every be used."

Nick replied, "I see your point; it would be nice to flag that the comparison against NULL seems suspicious." He acknowledged the rest of Linus's explanation as well.

The discussion began to meander here. At one point, Linus offered the following perspective:

"The advantage of compiler warnings really is that they get caught quicker and developers will react to them much better. They might cause the code to be properly re-organized or rewritten to be much nicer, for example.

"The 'trivial tree' kind of fixups for random other issues that get noticed separately tend to be much more about 'work around issue'. It might be the proper fix, of course, but it didn't end up being taken into account when writing the code, so it often ends up being just a 'papering over the warning' kind of fix. Particularly since the person trying to fix the problem generally isn't the main developer of that code."

In fact, at this point, the conversation began to get very technical – Nick posted some raw code as a sample of what he thought might satisfy Linus. And they (plus Theodore Ts'o) started getting into the real nitty gritty of things before the thread finally petered out.

I'm fascinated by the strange relationship between the kernel project and the various compiler projects. It's such a chicken-and-egg type problem when these projects have disagreements over compiler features. I personally feel that the Linux kernel is one of the most important software projects (or the actual most important software project) in the world, and that therefore the compiler exists almost as a service tool for Linux. It makes perfect sense to me that any compiler would want to adopt the features desired by the Linux kernel project.

However, there is an absolutely legitimate point on the other side: The compiler is truly the fundamental software, and all other software projects are the beneficiaries that need to have some faith that the compiler people will make good decisions for everybody.

The validity of both of these perspectives creates an amazing situation when kernel and compiler projects find themselves at odds. As an aside, this same situation arises when hardware projects and compiler projects find themselves at odds. In such situations, the manner in which the debate proceeds, and in which the conflicts and disagreements are ultimately resolved, seem to me like they contain the secrets of the universe.