Linux Weekly News

On November 10, the Go programming language community celebrated the 12th anniversary of its release as open-source software. The post covers a number of different topics, including the consolidation of web sites at go.dev, releases and their features over the last year, as well as a look to the future: In February, the Go 1.18 release will expand the new register-based calling convention to non-x86 architectures, bringing dramatic performance improvements with it. It will include the new Go fuzzing support. And it will be the first release to include support for generics.

Generics will be one of our focuses for 2022. The initial release in Go 1.18 is only the beginning. We need to spend time using generics and learning what works and what doesn’t, so that we can write best practices and decide what should be added to the standard library and other libraries. We expect that Go 1.19 (expected in August 2022) and later releases will further refine the design and implementation of generics as well as integrating them further into the overall Go experience.

Security updates have been issued by Debian (icinga2, libxstream-java, ruby-kaminari, and salt), Fedora (awscli, cacti, cacti-spine, python-boto3, python-botocore, radeontop, and rust), Mageia (firefox, libesmtp, libzapojit, sssd, and thunderbird), openSUSE (samba and samba and ldb), SUSE (firefox, pcre, qemu, samba, and samba and ldb), and Ubuntu (firejail, linux-bluefield, linux-gke-5.4, linux-oracle, linux-oracle-5.4, linux-oem-5.10, linux-oem-5.14, and python-py).

The LWN.net Weekly Edition for November 11, 2021 is available.

Python supports default values for arguments to functions, but those defaults are evaluated at function-definition time. A proposal to add defaults that are evaluated when the function is called has been discussed at some length on the python-ideas mailing list. The idea came about, in part, due to yet another resurrection of the proposal for None-aware operators in Python. Late-bound defaults would help with one use case for those operators, but there are other, stronger reasons to consider their addition to the language.

Security updates have been issued by Debian (openjdk-8 and samba), Fedora (community-mysql, firefox, and vim), openSUSE (binutils, kernel, and tinyxml), Red Hat (annobin, autotrace, babel, bind, binutils, bluez, compat-exiv2-026, container-tools:2.0, container-tools:3.0, container-tools:rhel8, cups, curl, dnf, dnsmasq, edk2, exiv2, file, file-roller, firefox, gcc, gcc-toolset-10-annobin, gcc-toolset-10-binutils, gcc-toolset-10-gcc, gcc-toolset-11-annobin, gcc-toolset-11-binutils, gcc-toolset-11-gcc, glib2, glibc, GNOME, gnutls and nettle, go-toolset:rhel8, grafana, graphviz, grilo, httpd:2.4, jasper, java-17-openjdk, json-c, kernel, kernel-rt, kexec-tools, kpatch-patch, lasso, libgcrypt, libjpeg-turbo, libsepol, libsolv, libssh, libtiff, libwebp, libX11, linuxptp, lua, mingw-glib2, mutt, ncurses, NetworkManager, openjpeg2, openssh, openssl, pcre, pcs, php:7.4, python-jinja2, python-lxml, python-pillow, python-pip, python-psutil, python27:2.7, python3, python36:3.6, python38:3.8 and python38-devel:3.8, python39:3.9 and python39-devel:3.9, qt5, resource-agents, rpm, rust-toolset:rhel8, spamassassin, sqlite, squid:4, tcpdump, tpm2-tools, vim, virt:rhel and virt-devel:rhel, and zziplib), and SUSE (binutils and kernel).

There is a set of new Samba releases out there. They fix a long and intimidating list of security issues and seem worth upgrading to for any but the most protected of Samba servers.

The Julia programming language has its roots in high-performance scientific computing, so it is no surprise that it has facilities for concurrent processing. Those features are not well-known outside of the Julia community, though, so it is interesting to see the different types of parallel and concurrent computation that the language supports. In addition, the upcoming release of Julia version 1.7 brings an improvement to the language's concurrent-computation palette, in the form of "task migration".

Security updates have been issued by Arch Linux (firefox, grafana, jenkins, opera, and thunderbird), Debian (botan1.10 and ckeditor), openSUSE (chromium, kernel, qemu, and rubygem-activerecord-5_1), SUSE (qemu and rubygem-activerecord-5_1), and Ubuntu (docker.io, kernel, linux, linux-aws, linux-aws-5.11, linux-azure, linux-azure-5.11, linux-gcp, linux-gcp-5.11, linux-hwe-5.11, linux-kvm, linux-oem-5.13, linux-oracle, linux-oracle-5.11, linux, linux-aws, linux-aws-5.4, linux-azure, linux-azure-5.4, linux-gcp, linux-gcp-5.4, linux-gke, linux-gkeop, linux-gkeop-5.4, linux-hwe-5.4, linux-ibm, linux-kvm, and linux, linux-aws, linux-aws-hwe, linux-azure, linux-azure-4.15, linux-dell300x, linux-gcp-4.15, linux-hwe, linux-kvm, linux-oracle, linux-raspi2, linux-snapdragon).

The LXD team has announced the release of version 4.20 of the LXD system container and virtual machine manager. This is one very busy release with a lot of new features.

VM users will be happy to see the initial implementation of live migration and core scheduling support. Container users are getting new configuration keys to set sysctls.

Then the bulk of the new features are all network related with peer network relationships, network zones for auto-generated DNS and SR-IOV accelerated OVN networks.

And lastly, on the clustering front, it’s now possible to better control what servers will be receiving new workloads.

The x86 instruction set is large, but that doesn't mean it can't get bigger yet. Upcoming Intel processors will feature a new set of instructions under the name of "Advanced Matrix Extensions" (AMX) that can be used to operate on matrix data. After a somewhat bumpy development process, support for AMX has found its way into the upcoming 5.16 kernel. Using it will, naturally, require some changes by application developers.

Security updates have been issued by Debian (containerd, redis, and sqlalchemy), Fedora (kernel, radeontop, rpki-client, and webkit2gtk3), openSUSE (java-1_8_0-openj9, libvirt, mailman, transfig, and webkit2gtk3), Oracle (thunderbird), SUSE (libvirt), and Ubuntu (icu).

The 5.15.1, 5.14.17, 5.10.78, 5.4.158, and 4.19.216 stable kernel updates have all been released; each contains another set of important fixes.

Konstantin Ryabitsev introduces the "local email interface" (lei) functionality for the lore archive of kernel mailing lists.

Even though it started out as merely a list archival service, it quickly became obvious that lore could be used for a lot more. Many developers ended up using its search features to quickly locate emails of interest, which in turn raised a simple question — what if there was a way to “save a search” and have it deliver all new incoming mail matching certain parameters straight to the developers' inbox?

You can now do this with lei.

Back in September, LWN reported on a series of block-layer optimizations that enabled a suitably equipped system to sustain 3.5 million I/O operations per second (IOPS). That optimization work has continued since then, and those 3.5 million IOPS would be a deeply disappointing result now. A recent disagreement over the addition of a new feature has highlighted the potential cost of a heavily optimized block layer, though; when is a feature deemed important enough to outweigh the drive for maximum performance?

Security updates have been issued by Debian (python3.5, redis, and udisks2), Fedora (rust), openSUSE (binutils, java-1_8_0-openj9, and qemu), Oracle (firefox and httpd), Red Hat (thunderbird), Scientific Linux (thunderbird), and SUSE (binutils, qemu, and systemd).

Ariadne Conill shares some experience of working in the online advertising industry.

The cycle of patching on both sides is ongoing to this day. A friend of mine on Twitter referred to this tug-of-war as “core war,” which is an apt description: all of the involved actors are trying to patch each other out of being able to commit or detect subterfuge, and your browser gets slower and slower as more mitigations and countermeasures are layered on. If you’re not using an ad blocker yet, stop reading this, and install one: your browser will suddenly be a lot more performant.

The Record is reporting on massive exploitation of an oldish vulnerability in GitLab instances.

While the purpose of these attacks remained unclear for HN Security, yesterday, Google’s Menscher said the hacked servers were part of a botnet comprising of “thousands of compromised GitLab instances” that was launching large-scale DDoS attacks.

The vulnerability was fixed in April, but evidently a lot of sites have not updated.

Over at the SAMBA+ blog, the performance of the new ksmbd kernel SMB server and Samba in user space are compared: ksmbd claims performance improvements on a wide range of benchmarks: the graphs on this page show a doubling of performance on some tests. There was also the notion that an in-kernel server is likely an easier place to support SMB Direct, which uses RDMA to transfer data between systems.

Clearly, those number are impressive, but at the same time recent improvements in Samba's IO performance put this into perspective: by leveraging the new "io_uring" Linux API Samba is able to provide roughly 10x the throughput compared to ksmbd.

Time will tell whether it's better to reside in kernel-space like ksmbd or in user-space like Samba in order to squeeze the last bit of performance out of the available hardware.

There are two graphs that show some impressive results for Samba.

Patrick Horgan explains the process of starting a program on Linux in great detail.

Well, __libc_start_main calls __libc_init_first, who immediately uses secret inside information to find the environment variables just after the terminating null of the argument vector and then sets a global variable __environ which __libc_start_main uses thereafter whenever it needs it including when it calls main. After the envp is established, then __libc_start_main uses the same trick and surprise! Just past the terminating null at the end of the envp array, there's another vector, the ELF auxiliary vector the loader uses to pass some information to the process. An easy way to see what's in there is to set the environment variable LD_SHOW_AUXV=1 before running the program.

As of this writing, Linus Torvalds has pulled exactly 6,800 non-merge changesets into the mainline repository for the 5.16 kernel release. That is probably a little over half of what will arrive during this merge window, so this is a good time to catch up on what has been pulled so far. There are many significant changes and some large-scale restructuring of internal kernel code, but relatively few ground-breaking new features.