Zack's Kernel News

Cleaning Up Build Warnings

Linus Torvalds recently upgraded his home system to use the GNU Compiler Collection (GCC) 10, which is not as important as it might sound. It doesn't mean everyone should use version 10 or anything like that. However, Linus did notice that after the upgrade, compiling the Linux kernel would spit out vast and copious quantities of warning messages.

In fact, he said, "I let them go for a while, in the belief that I could deal with it, but then yesterday I did a pull and didn't initially even notice that the end result didn't compile for me, because the build error was hidden by the hundreds of lines of warnings."

He added, "a lot of them were good warnings where gcc warns about things it really should warn about – if you have modern source code and actually use flexible arrays etc. Which we're moving towards, but we're not there yet, and clearly won't be for 5.7."

However, he had some problems with the gcc flag -Wno-maybe-initialized. It can produce useful warnings, Linus said, but it can also produce a ton of false positives. So he concluded, "I simply refuse to be in the situation where I might miss an _important_ warning (or, like happened yesterday, an actual failure), because the stupid warning noise is hiding things that matter. Yes, I caught the build error despite the noise, but that was partly luck (I did another pull before I pushed out, and caught the error on the second build). And yes, I've made my workflow now hopefully make sure that the actual build error will stand out more, but even hiding just other – more real – warnings is a problem, so I do not want to see the pointless noise."

He asked if anyone had ideas for how to fix this.

David Laight noted that, "gmake is very bad at stopping parallel makes when one command fails. So the kernel build carries on firing off new compilations even after one has failed. I've not looked inside gmake, but I fixed nmake so that it properly used a single job token pipe for the entire (NetBSD) build and then flushed and refilled it with 'abort' tokens when any command failed. That made the build stop almost immediately."

Linus replied:

"The GNU jobserver doesn't have anything like that, afaik.

"I think it always writes a '+' character as a token, so I guess it could be extended to write something else for the 'abort now' situation (presumably a '-' character).

"But at least for external jobserver clients (of which I am not aware of any, I think we only depend on the internal GNU make behavior), the documentation states that you should just write back the same token you read.

"I've looked at the low-level jobserver code because I was chasing a bug there not that long ago, but I've never looked at the interaction with actually running commands."

Linus also cc'd Paul Smith at the GNU project, and Paul replied that he was the one who had actually written the documentation text Linus had referenced and that extending the jobserver to handle failure cases was exactly his plan.

Paul also remarked, "the GCC project has a GSoC project approved for this summer, for GCC and/or binutils to participate in the jobserver protocol when they do multithreading in the compiler/linker. I think they are planning on creating a generic "jobserver library" but I'm not mentoring (I don't have the bandwidth for GSoC mentoring). I do hope to stay abreast of their work and perhaps toss in suggestions however."

He also remarked, "My current work on GNU make is fixing the atrocious mess it has with signal handling: don't even look and if you do, remember I inherited this code (yeah, yeah, a long time ago but still… :)). Right now it's not so hard (especially with large -j) to have make instances hanging when ^C is used due to race conditions in the signal handling."

But in general, Paul said, he felt he could improve gmake's ability to abort, as Linus had requested.

Linus, Paul, and David then embarked on a technical discussion to actually design the feature in question. Apparently, all three of them were familiar with the GNU make source code.

Eventually the thread petered out inconclusively, but it does seem very clear that Paul is motivated to address the issue and can expect help and feedback from top kernel people.

The issue of kernel build output is perennial. Regardless of what Paul comes up with now, something more will be needed later. There's always new output accruing to the kernel build system and always new efforts to clean it up. Often the solution is to change the kernel. This time the solution seems to have been to change GNU make.

Improving (?) Kernel Code Generation

GCC offers various levels of optimization in its -O series of command-line flags. And Linux has traditionally wrestled with GCC over exactly what constitutes a good optimization. Recently the wrestling match continued.

Jason A. Donenfeld reported that, "GCC 10 appears to have changed -O2 in order to make compilation time faster when using -flto, seemingly at the expense of performance, in particular with regards to how the inliner works. Since -O3 these days shouldn't have the same set of bugs as 10 years ago, this commit defaults new kernel compiles to -O3 when using gcc >= 10."

Peter Zijlstra remarked that in general, he thought -O3 wasn't as bad as in the old days. But he also said it would be good to get some input from some of the GCC developers before making such a sweeping change to the kernel build system.

Arnd Bergmann agreed, saying, "I also want to hear the feedback from the gcc developers about what the general recommendations are between O2 and O3, and how they may have changed over time." And he added, "Personally, I'm more interested in improving compile speed of the kernel and eventually supporting -Og or some variant of it for my own build testing, but of course I also want to make sure that the other optimization levels do not produce warnings, and -Og leads to more problems than -O3 at the moment."

At a certain point in the discussion, Linus Torvalds responded to Jason's original patch converting the kernel to use -O3 instead of -O2, saying:

"I'm not convinced this is sensible.

"-O3 historically does bad things with gcc. Including bad things for performance. It traditionally makes code larger and often SLOWER.

"And I don't mean slower to compile (although that's an issue). I mean actually generating slower code.

"Things like trying to unroll loops etc makes very little sense in the kernel, where we very seldom have high loop counts for pretty much anything.

"There's a reason -O3 isn't even offered as an option.

"Maybe things have changed, and maybe they've improved. But I'd like to see actual numbers for something like this.

"Not inlining as aggressively is not necessarily a bad thing. It can be, of course. But I've actually also done gcc bugreports about gcc inlining too much, and generating _worse_ code as a result (ie inlining things that were behind an 'if (unlikely())' test, and causing the likely path to grow a stack frame and stack spills as a result).

"So just 'O3 inlines more' is not a valid argument."

In a later post, he added, "Obviously, in the kernel, we can fix the obvious cases with 'noinline' and 'always_inline', but those take care of the outliers. Having a compiler that does reasonably well by default is a good thing, and that very much includes *not* inlining mindlessly."

Linus also gave a link to a bug report he'd submitted in 2011 on the subject: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=49194.

Jason heard Linus's call for performance numbers. He also remarked that, "you made a compelling argument in that old gcc bug report about not going down the finicky rabbit hole of gcc inlining switches that seem to change meaning between releases, which is persuasive."

So Jason seemed ready to drop the patch, finally. And, at the tail end of the thread in response to Jason's code, Artem S. Tashkinov said:

"It's a strong 'no' from me.

"1) Aside from rare Gentoo users no one has extensively tested -O3 with the kernel – even Gentoo defaults to -O2 for kernel compilation

"2) -O3 _always_ bloats the code by a large amount which means both vmlinux/bzImage and modules will become bigger, and slower to load from the disk

"3) -O3 does _not_ necessarily makes the code run faster

"4) If GCC10 has removed certain options for the -O2 optimization level you could just readded them as compilation flags without forcing -O3 by default on everyone

"5) If you still insist on -O3 I guess everyone would be happy if you just made two KConfig options:"

OPTIMIZE_O2 (-O2)
OPTIMIZE_O3_EVEN_MOAR (-O3)

And that was the end of that.

Working Around Missing Future Compiler Features

Linus Torvalds had some interesting comments to make about coding style recently – or maybe a better term would be coding techniques.

It came up in the midst of a long thread about which GCC version was the best GCC version (i.e., which produced the best code, the best warnings, the best errors, and whatnot). It was all in the context of trying to track down a kernel bug without experiencing agonizing pain in the process.

And of course, Linus recently mentioned that he'd started building with GCC 10. As a result, a lot of developers wanted to switch over to the same version so they could make sure they saw the same build errors and warnings that Linus was going to see when they sent him their patches.

Borislav Petkov chided those developers, saying, "Oh noo, we don't want Linus' kernel broken. ;-)"

But Borislav was also on board with getting things ironed out for Linus's build system.

At some point in the discussion, Nick Desaulniers remarked, regarding one particular issue, that GCC developers were working on a portable solution that would let users avoid hacky coding. He said, "Adding arbitrary empty asm statements to work around it? Hacks. Full memory barriers? Hacks." And he added, "Sprinkling empty asm statements or full memory barriers should be treated with the same hesitancy as adding sleep()s to 'work around' concurrency bugs. Red flag."

To which Linus replied:

"BS.

"A compiler person might call it a 'hack'. But said compiler person will be _wrong_.

"An intelligent developer knows that it will take years for compilers to give us all the infrastructure we need, and even then the compiler won't actually give us everything – and people will be using the old compilers for years anyway.

"That's why inline asm's exist. They are the escape from the excessive confines of 'let's wait for the compiler person to solve this for us' – which they'll never do completely anyway.

"It's a bit like unsafe C type casts and allowing people to write 'non-portable code'. Some compiler people will say that it's bad, and unsafe. Sure, it can be unsafe, but the point is that it allows you to do things that aren't necessarily _possible_ to do in an overly restrictive language.

"Sometimes you need to break the rules.

"There's a reason everybody writes library routines in 'unsafe' languages like C. Because you need those kinds of escapes in order to actually do something like a memory allocator etc.

"And that's also why we have inline asm – because the compiler will never know everything or be able to generate code for everything people want to do.

"And anybody that _thinks_ that the compiler will always know better and should be in complete control shouldn't be working on compilers. They should probably be repeating kindergarten, sitting in a corner eating paste with their friends.

"So no. Using inline asms to work around the compiler not understanding what is going on isn't a 'hack'. It's the _point_ of inline asm.

"Is it perfect? No. But it's not like there are many alternatives."