Hírolvasó

LWN editor Jonathan Corbet was asked to give a brief talk about kernel maintainership at the recently concluded Linux Foundation Member Summit. That talk was recorded and has now been made available on YouTube. There is little in it that will be news to regular LWN readers, but it may be instructive to folks who are less well versed in how kernel development works.

Matthew Garrett explains why ACPI exists and why it is not as bad a thing as some think.

There's an alternative universe where we decided to teach the kernel about every piece of hardware it should run on. Fortunately (or, well, unfortunately) we've seen that in the ARM world. Most device-specific simply never reaches mainline, and most users are stuck running ancient kernels as a result. Imagine every x86 device vendor shipping their own kernel optimised for their hardware, and now imagine how well that works out given the quality of their firmware. Does that really seem better to you?

Security updates have been issued by Debian (h2o, open-vm-tools, pmix, and zookeeper), Gentoo (GitPython), Oracle (firefox, java-11-openjdk, java-17-openjdk, libguestfs-winsupport, nginx:1.22, and thunderbird), Red Hat (samba), SUSE (container-suseconnect, libsndfile, and slurm), and Ubuntu (krb5, linux, linux-aws, linux-aws-5.15, linux-azure, linux-azure-5.15, linux-azure-fde, linux-azure-fde-5.15, linux-gcp, linux-gcp-5.15, linux-gkeop, linux-gkeop-5.15, linux-hwe-5.15, linux-ibm, linux-ibm-5.15, linux-kvm, linux-lowlatency, linux-lowlatency-hwe-5.15, linux-nvidia, linux-oracle, linux-oracle-5.15, linux, linux-aws, linux-aws-5.4, linux-azure, linux-azure-5.4, linux-bluefield, linux-gcp, linux-gcp-5.4, linux-gkeop, linux-hwe-5.4, linux-ibm, linux-ibm-5.4, linux-kvm, linux-oracle, linux-oracle-5.4, linux-raspi, linux-raspi-5.4, linux-xilinx-zynqmp, linux, linux-aws, linux-aws-6.2, linux-azure, linux-azure-6.2, linux-azure-fde-6.2, linux-gcp, linux-gcp-6.2, linux-hwe-6.2, linux-kvm, linux-lowlatency, linux-lowlatency-hwe-6.2, linux-oracle, linux-raspi, linux-starfive, linux-laptop, linux-nvidia-6.2, linux-oem-6.1, linux-raspi, open-vm-tools, and xorg-server).

"Why does ACPI exist" - - the greatest thread in the history of forums, locked by a moderator after 12,239 pages of heated debate, wait no let me start again.

Why does ACPI exist? In the beforetimes power management on x86 was done by jumping to an opaque BIOS entry point and hoping it would do the right thing. It frequently didn't. We called this Advanced Power Management (Advanced because before this power management involved custom drivers for every machine and everyone agreed that this was a bad idea), and it involved the firmware having to save and restore the state of every piece of hardware in the system. This meant that assumptions about hardware configuration were baked into the firmware - failed to program your graphics card exactly the way the BIOS expected? Hurrah! It's only saved and restored a subset of the state that you configured and now potential data corruption for you. The developers of ACPI made the reasonable decision that, well, maybe since the OS was the one setting state in the first place, the OS should restore it.

So far so good. But some state is fundamentally device specific, at a level that the OS generally ignores. How should this state be managed? One way to do that would be to have the OS know about the device specific details. Unfortunately that means you can't ship the computer without having OS support for it, which means having OS support for every device (exactly what we'd got away from with APM). This, uh, was not an option the PC industry seriously considered. The alternative is that you ship something that abstracts the details of the specific hardware and makes that abstraction available to the OS. This is what ACPI does, and it's also what things like Device Tree do. Both provide static information about how the platform is configured, which can then be consumed by the OS and avoid needing device-specific drivers or configuration to be built-in.

The main distinction between Device Tree and ACPI is that Device Tree is purely a description of the hardware that exists, and so still requires the OS to know what's possible - if you add a new type of power controller, for instance, you need to add a driver for that to the OS before you can express that via Device Tree. ACPI decided to include an interpreted language to allow vendors to expose functionality to the OS without the OS needing to know about the underlying hardware. So, for instance, ACPI allows you to associate a device with a function to power down that device. That function may, when executed, trigger a bunch of register accesses to a piece of hardware otherwise not exposed to the OS, and that hardware may then cut the power rail to the device to power it down entirely. And that can be done without the OS having to know anything about the control hardware.

How is this better than just calling into the firmware to do it? Because the fact that ACPI declares that it's going to access these registers means the OS can figure out that it shouldn't, because it might otherwise collide with what the firmware is doing. With APM we had no visibility into that - if the OS tried to touch the hardware at the same time APM did, boom, almost impossible to debug failures (This is why various hardware monitoring drivers refuse to load by default on Linux - the firmware declares that it's going to touch those registers itself, so Linux decides not to in order to avoid race conditions and potential hardware damage. In many cases the firmware offers a collaborative interface to obtain the same data, and a driver can be written to get that. this bug comment discusses this for a specific board)

Unfortunately ACPI doesn't entirely remove opaque firmware from the equation - ACPI methods can still trigger System Management Mode, which is basically a fancy way to say "Your computer stops running your OS, does something else for a while, and you have no idea what". This has all the same issues that APM did, in that if the hardware isn't in exactly the state the firmware expects, bad things can happen. While historically there were a bunch of ACPI-related issues because the spec didn't define every single possible scenario and also there was no conformance suite (eg, should the interpreter be multi-threaded? Not defined by spec, but influences whether a specific implementation will work or not!), these days overall compatibility is pretty solid and the vast majority of systems work just fine - but we do still have some issues that are largely associated with System Management Mode.

One example is a recent Lenovo one, where the firmware appears to try to poke the NVME drive on resume. There's some indication that this is intended to deal with transparently unlocking self-encrypting drives on resume, but it seems to do so without taking IOMMU configuration into account and so things explode. It's kind of understandable why a vendor would implement something like this, but it's also kind of understandable that doing so without OS cooperation may end badly.

This isn't something that ACPI enabled - in the absence of ACPI firmware vendors would just be doing this unilaterally with even less OS involvement and we'd probably have even more of these issues. Ideally we'd "simply" have hardware that didn't support transitioning back to opaque code, but we don't (ARM has basically the same issue with TrustZone). In the absence of the ideal world, by and large ACPI has been a net improvement in Linux compatibility on x86 systems. It certainly didn't remove the "Everything is Windows" mentality that many vendors have, but it meant we largely only needed to ensure that Linux behaved the same way as Windows in a finite number of ways (ie, the behaviour of the ACPI interpreter) rather than in every single hardware driver, and so the chances that a new machine will work out of the box are much greater than they were in the pre-ACPI period.

There's an alternative universe where we decided to teach the kernel about every piece of hardware it should run on. Fortunately (or, well, unfortunately) we've seen that in the ARM world. Most device-specific simply never reaches mainline, and most users are stuck running ancient kernels as a result. Imagine every x86 device vendor shipping their own kernel optimised for their hardware, and now imagine how well that works out given the quality of their firmware. Does that really seem better to you?

It's understandable why ACPI has a poor reputation. But it's also hard to figure out what would work better in the real world. We could have built something similar on top of Open Firmware instead but the distinction wouldn't be terribly meaningful - we'd just have Forth instead of the ACPI bytecode language. Longing for a non-ACPI world without presenting something that's better and actually stands a reasonable chance of adoption doesn't make the world a better place.

comments

A fast-moving patch set—seemingly the norm for Linux networking development—seeks to add some Rust abstractions for physical layer (PHY) drivers. Lots of review has been done, and the patch set has been reworked frequently in response to those comments. Unfortunately, the Rust-for-Linux developers are having trouble keeping up with that pace. There is, it would appear, something of a disconnect between the two communities' development practices.

Version 0.2 of Incus, an LXD fork, has been released. "This version incorporates most changes that went into LXD 5.19 as well as introduce a few additional features and improvements." Changes include NVME storage support, support for migrating clustered environments from LXD, and more.

Security updates have been issued by Debian (jetty9, node-browserify-sign, request-tracker4, and request-tracker5), Fedora (golang-github-altree-bigfloat, golang-github-seancfoley-bintree, golang-github-seancfoley-ipaddress, kitty, slurm, and thunderbird), Gentoo (ConnMan, libxslt, and Salt), Mageia (chromium-browser-stable), Red Hat (firefox, libguestfs-winsupport, and thunderbird), SUSE (clamav, gcc13, gstreamer-plugins-bad, icu73_2, java-17-openjdk, nodejs10, poppler, python-Werkzeug, redis, thunderbird, webkit2gtk3, xorg-x11-server, and xwayland), and Ubuntu (kernel, linux-aws, linux-azure, linux-gcp, linux-oracle, linux-raspi, linux-iot, linux-raspi, linux-raspi-5.4, and mysql-8.0).

The New Stack covers a conference talk by Bjarne Stroustrup on turning C++ into a safer language.

Stroustrup has arrived at his solution: profiles. (That is, a set of rules which, when followed, achieve specific safety guarantees.) They’d be defined by the ISO C++ standard, addressing common safety issues like pointers and array ranges. In response to a later question from the audience about the difficulty of adding new tooling, Stroustrup pointed out that the C++ compiler itself is now a pretty sophisticated static analyzer, and could also be tasked with meeting the profile's requirements.

The 6.6 kernel was released, right on schedule, on October 29. This development cycle saw the addition of 14,069 non-merge changesets from 1,978 developers — fairly typical numbers for recent releases. The time has come for LWN's traditional look at where the changes in this release came from, along with a look at the longer development "supercycle" that (probably) ends with 6.6.

Security updates have been issued by Debian (distro-info, distro-info-data, gst-plugins-bad1.0, node-browserify-sign, nss, openjdk-11, and thunderbird), Fedora (chromium, curl, nghttp2, and xorg-x11-server-Xwayland), Gentoo (Dovecot, Rack, rxvt-unicode, and UnZip), Mageia (apache, bind, and vim), Red Hat (varnish:6), SUSE (nodejs12, opera, python-bugzilla, python-Django, and vorbis-tools), and Ubuntu (exim4, firefox, nodejs, and slurm-llnl, slurm-wlm).

Linus has released the 6.6 kernel. "So this last week has been pretty calm, and I have absolutely no excuses to delay the v6.6 release any more, so here it is."

Headline features in 6.6 include the earliest eligible virtual deadline first (EEVDF) CPU scheduler, a number of enhancements (quota support, user extended attributes, direct I/O) to the tmpfs filesystem, the fchmodat2() system call, initial support for building a kernel without buffer-head support, the kmalloc() randomness patches, user-space shadow stacks for Intel CPUs, and quite a bit more. See the LWN merge window summaries (part 1, part 2) and the KernelNewbies 6.6 page for more information.

Suppose you want to create a pipeline with the subprocess and you want to capture the stderr. A colleague of mine upstream wrote this:

p1 = subprocess.Popen(cmd1, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True) p2 = subprocess.Popen(cmd2, stdin=p1.stdout, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True) p1.stdout.close() p1_stderr = p1.communicate() p2_stderr = p2.communicate() return p1.returncode or p2.returncode, p1_stderr, p2_stderr

Unfortunately, the above saves the p1.stdout in memory, which may come back to bite the user once the amount piped becomes large enough.

I think the right answer is this:

with tempfile.TemporaryFile() as errfile: p1 = subprocess.Popen(cmd1, stdout=subprocess.PIPE, stderr=errfile, close_fds=True) p2 = subprocess.Popen(cmd2, stdin=p1.stdout, stdout=subprocess.PIPE, stderr=errfile, close_fds=True) p1.stdout.close() p2.communicate() p1.wait() errfile.seek(0) px_stderr = errfile.read() return p1.returncode or p2.returncode, px_stderr

Stackoverflow is overflowing with noise on this topic. Just ignore it.

User-space developers working with highly threaded applications would often like to be able to use spinlocks to protect shared data structures from concurrent access. There is a fundamental problem with user-space spinlocks, though: there is no way to prevent a thread from being preempted. Various ways of working around this problem have been explored, but this patch from Steven Rostedt questions the premise on which much of that work is based: what if it were possible to prevent preemption, for a short period at least?

For a view into the OpenBSD approach to security, see this message from Theo de Raadt, where he describes a plan to remove the syscall() system call (which allows the invocation of any available system call by providing its number) from the kernel. The purpose, of course, is to make it harder for an attacker to invoke an arbitrary system call, even if they are able to run some code on the target system.

I hope I am forcing attack coders into using increasingly more complicated methods. Same time, it means fewer methods are available. Other methods make exploitation more fragile. This is pushing success rates into "low-percent statistical" success. If we teach more software stacks to "fail hard, don't try to recover", that is an improvement in security.

Security updates have been issued by Debian (chromium and firefox-esr), Fedora (firefox, redis, samba, and xen), Oracle (python39:3.9, python39-devel:3.9), Slackware (mozilla and xorg), and SUSE (libnbd, open-vm-tools, python, sox, vorbis-tools, and zchunk).

Theo de Raadt (deraadt@) posted to tech@ a message entitled disruptive amd64 snapshot coming. It reads:

There is a pretty disruptive amd64 snapshot coming, so anyone who is using snapshots for critical stuff should take a pause. (This warning about a development step is unusual, I won't make it common practice).

Of course, on non-critical amd64 systems running snapshots, this is a good opportunity to test (and report any problems).

The Pixel 8 hardware (Tensor G3) supports the ARM Memory Tagging Extension (MTE), and software support is available both in Android userspace and the Linux kernel. This feature is a powerful defense against linear buffer overflows and many types of use-after-free flaws. I’m extremely happy to see this hardware finally available in the real world.

Turning it on for userspace is already wired up the Android UI: Settings / System / Developer options / Memory Tagging Extension / Enable MTE until you turn if off. Once enabled it will internally change an Android “system property” named “arm64.memtag.bootctl” by adding the option “memtag“.

Turning it on for the kernel is slightly more involved, but not difficult at all. This requires manually setting the “arm64.memtag.bootctl” property mentioned above to include “memtag-kernel” as well:

• Plug your phone into a system that can run the adb tool
• If not already installed, install adb. For example on Debian/Ubuntu: sudo apt install adb
• Turn on “USB Debugging” in the phone’s “Developer options” menu, and accept the debugging session confirmation that will pop up when you first run adb
• Verify the current setting: adb shell getprop | grep memtag.bootctl

[arm64.memtag.bootctl]: [memtag]

• Enable kernel MTE: adb shell setprop arm64.memtag.bootctl memtag,memtag-kernel
• Check the results: adb shell getprop | grep memtag.bootctl

[arm64.memtag.bootctl]: [memtag,memtag-kernel]

• Reboot your phone

To check that MTE is enabled for the kernel (which is implemented using Kernel Address Sanitizer’s Hardware Tagging mode), you can check the kernel command line after rebooting:

$ mkdir foo && cd foo $ adb bugreport ... $ mkdir unpacked && cd unpacked $ unzip ../bugreport*.zip ... $ grep kasan= bugreport*.txt ...: Command line: ... kasan=off ... kasan=on ...

The latter “kasan=on” overrides the earlier “kasan=off“.

Enjoy!

© 2023, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 License.

The C programming language is replete with features that seemed like a good idea at the time (and perhaps even were good ideas then) that have not aged well. Most would likely agree that string handling, and the use of NUL-terminated strings, is one of those. Kernel developers have, for years, tried to improve the handling of strings in an attempt to slow the flow of bugs and vulnerabilities that result from mistakes in that area. Now there is an early discussion on the idea of moving away from NUL-terminated strings in much of the kernel.

Security updates have been issued by Debian (firefox-esr and xorg-server), Fedora (firefox, mbedtls, nodejs18, nodejs20, and xen), Gentoo (libinput, unifi, and USBView), Mageia (python-nltk), Oracle (linux-firmware), Red Hat (nginx:1.22), SUSE (chromium, firefox, java-11-openjdk, jetty-minimal, nghttp2, nodejs18, webkit2gtk3, and zlib), and Ubuntu (linux, linux-lowlatency, linux-oracle-5.15, vim, and xorg-server, xwayland).

Hot on the heels of the release of OpenBSD 7.4, Omar Polo (op@) has announced the release of OpenSMTPD 7.4.0p0. The announcement reads,

Subject: OpenSMTPD 7.4.0p0 Released From: Omar Polo <op () openbsd ! org> Date: 2023-10-25 7:33:43 OpenSMTPD is a FREE implementation of the SMTP protocol with some common extensions. It allows ordinary machines to exchange e-mails with systems speaking the SMTP protocol. It implements a fairly large part of RFC5321 and can already cover a large range of use-cases. It runs on OpenBSD, NetBSD, FreeBSD, DragonFlyBSD, Linux and OSX. The archives are now available from the main site at www.OpenSMTPD.org

Read more…