Hírolvasó

[$] Late-bound argument defaults for Python

3 év 9 hónap óta
Python supports default values for arguments to functions, but those defaults are evaluated at function-definition time. A proposal to add defaults that are evaluated when the function is called has been discussed at some length on the python-ideas mailing list. The idea came about, in part, due to yet another resurrection of the proposal for None-aware operators in Python. Late-bound defaults would help with one use case for those operators, but there are other, stronger reasons to consider their addition to the language.
jake

Security updates for Wednesday

3 év 9 hónap óta
Security updates have been issued by Debian (openjdk-8 and samba), Fedora (community-mysql, firefox, and vim), openSUSE (binutils, kernel, and tinyxml), Red Hat (annobin, autotrace, babel, bind, binutils, bluez, compat-exiv2-026, container-tools:2.0, container-tools:3.0, container-tools:rhel8, cups, curl, dnf, dnsmasq, edk2, exiv2, file, file-roller, firefox, gcc, gcc-toolset-10-annobin, gcc-toolset-10-binutils, gcc-toolset-10-gcc, gcc-toolset-11-annobin, gcc-toolset-11-binutils, gcc-toolset-11-gcc, glib2, glibc, GNOME, gnutls and nettle, go-toolset:rhel8, grafana, graphviz, grilo, httpd:2.4, jasper, java-17-openjdk, json-c, kernel, kernel-rt, kexec-tools, kpatch-patch, lasso, libgcrypt, libjpeg-turbo, libsepol, libsolv, libssh, libtiff, libwebp, libX11, linuxptp, lua, mingw-glib2, mutt, ncurses, NetworkManager, openjpeg2, openssh, openssl, pcre, pcs, php:7.4, python-jinja2, python-lxml, python-pillow, python-pip, python-psutil, python27:2.7, python3, python36:3.6, python38:3.8 and python38-devel:3.8, python39:3.9 and python39-devel:3.9, qt5, resource-agents, rpm, rust-toolset:rhel8, spamassassin, sqlite, squid:4, tcpdump, tpm2-tools, vim, virt:rhel and virt-devel:rhel, and zziplib), and SUSE (binutils and kernel).
ris

[$] Concurrency in Julia

3 év 9 hónap óta
The Julia programming language has its roots in high-performance scientific computing, so it is no surprise that it has facilities for concurrent processing. Those features are not well-known outside of the Julia community, though, so it is interesting to see the different types of parallel and concurrent computation that the language supports. In addition, the upcoming release of Julia version 1.7 brings an improvement to the language's concurrent-computation palette, in the form of "task migration".
jake

Security updates for Tuesday

3 év 9 hónap óta
Security updates have been issued by Arch Linux (firefox, grafana, jenkins, opera, and thunderbird), Debian (botan1.10 and ckeditor), openSUSE (chromium, kernel, qemu, and rubygem-activerecord-5_1), SUSE (qemu and rubygem-activerecord-5_1), and Ubuntu (docker.io, kernel, linux, linux-aws, linux-aws-5.11, linux-azure, linux-azure-5.11, linux-gcp, linux-gcp-5.11, linux-hwe-5.11, linux-kvm, linux-oem-5.13, linux-oracle, linux-oracle-5.11, linux, linux-aws, linux-aws-5.4, linux-azure, linux-azure-5.4, linux-gcp, linux-gcp-5.4, linux-gke, linux-gkeop, linux-gkeop-5.4, linux-hwe-5.4, linux-ibm, linux-kvm, and linux, linux-aws, linux-aws-hwe, linux-azure, linux-azure-4.15, linux-dell300x, linux-gcp-4.15, linux-hwe, linux-kvm, linux-oracle, linux-raspi2, linux-snapdragon).
ris

LXD 4.20 released

3 év 9 hónap óta
The LXD team has announced the release of version 4.20 of the LXD system container and virtual machine manager. This is one very busy release with a lot of new features.

VM users will be happy to see the initial implementation of live migration and core scheduling support. Container users are getting new configuration keys to set sysctls.

Then the bulk of the new features are all network related with peer network relationships, network zones for auto-generated DNS and SR-IOV accelerated OVN networks.

And lastly, on the clustering front, it’s now possible to better control what servers will be receiving new workloads.

ris

[$] Intel AMX support in 5.16

3 év 9 hónap óta
The x86 instruction set is large, but that doesn't mean it can't get bigger yet. Upcoming Intel processors will feature a new set of instructions under the name of "Advanced Matrix Extensions" (AMX) that can be used to operate on matrix data. After a somewhat bumpy development process, support for AMX has found its way into the upcoming 5.16 kernel. Using it will, naturally, require some changes by application developers.
corbet

Security updates for Monday

3 év 9 hónap óta
Security updates have been issued by Debian (containerd, redis, and sqlalchemy), Fedora (kernel, radeontop, rpki-client, and webkit2gtk3), openSUSE (java-1_8_0-openj9, libvirt, mailman, transfig, and webkit2gtk3), Oracle (thunderbird), SUSE (libvirt), and Ubuntu (icu).
jake

Ryabitsev: lore+lei: part 1, getting started

3 év 9 hónap óta
Konstantin Ryabitsev introduces the "local email interface" (lei) functionality for the lore archive of kernel mailing lists.

Even though it started out as merely a list archival service, it quickly became obvious that lore could be used for a lot more. Many developers ended up using its search features to quickly locate emails of interest, which in turn raised a simple question — what if there was a way to “save a search” and have it deliver all new incoming mail matching certain parameters straight to the developers' inbox?

You can now do this with lei.

corbet

[$] The balance between features and performance in the block layer

3 év 9 hónap óta
Back in September, LWN reported on a series of block-layer optimizations that enabled a suitably equipped system to sustain 3.5 million I/O operations per second (IOPS). That optimization work has continued since then, and those 3.5 million IOPS would be a deeply disappointing result now. A recent disagreement over the addition of a new feature has highlighted the potential cost of a heavily optimized block layer, though; when is a feature deemed important enough to outweigh the drive for maximum performance?
corbet

Security updates for Friday

3 év 9 hónap óta
Security updates have been issued by Debian (python3.5, redis, and udisks2), Fedora (rust), openSUSE (binutils, java-1_8_0-openj9, and qemu), Oracle (firefox and httpd), Red Hat (thunderbird), Scientific Linux (thunderbird), and SUSE (binutils, qemu, and systemd).
jake

Conill: an inside look into the illicit ad industry

3 év 9 hónap óta
Ariadne Conill shares some experience of working in the online advertising industry.

The cycle of patching on both sides is ongoing to this day. A friend of mine on Twitter referred to this tug-of-war as “core war,” which is an apt description: all of the involved actors are trying to patch each other out of being able to commit or detect subterfuge, and your browser gets slower and slower as more mitigations and countermeasures are layered on. If you’re not using an ad blocker yet, stop reading this, and install one: your browser will suddenly be a lot more performant.

corbet

Dave Airlie (blogspot): What do you know about video decoding/encoding?

3 év 9 hónap óta

A few weeks ago I watched Victor's excellent talk on Vulkan Video. This made me question my skills in this area. I'm pretty vague on video processing hardware, I really have no understanding of H264 or any of the standards. I've been loosely following the Vulkan video group inside of Khronos, but I can't say I've understood it or been useful.

radeonsi has a gallium vaapi driver, that talks to firmware driver encoder on the hardware, surely copying what it is programming can't be that hard. I got an mpv/vaapi setup running and tested some videos on that setup just to get comfortable. I looked at what sort of data was being pushed about.

The thing is the firmware is doing all the work here, the driver is mostly just responsible for taking semi-parsed h264 bitstream data structures and giving them in memory buffers to the fw API. Then the resulting decoded image should be magically in a buffer.

I then got the demo nvidia video decoder application mentioned in Victor's talk.

I ported the code to radv in a couple of days, but then began a long journey into the unknown. The firmware is quite expectant on exactly what it wants and when it wants it. After fixing some interactions with the video player, I started to dig.

Now vaapi and DXVA (Windows) are context based APIs. This means they are like OpenGL, where you create a context, do a bunch of work, and tear it down, the driver does all the hw queuing of commands internally. All the state is held in the context. Vulkan is a command buffer based API. The application records command buffers and then enqueues those command buffers to the hardware itself.

So the vaapi driver works like this for a video

create hw ctx, flush, decode, flush, decode, flush, decode, flush, decode, flush, destroy hw ctx, flush

However Vulkan wants things to be more like

Create Session, record command buffer with (begin, decode, end) send to hw, (begin, decode, end), send to hw, End Sesssion

There is no way at the Create/End session time to submit things to the hardware.

After a week or two of hair removal and insightful irc chats I stumbled over a decent enough workaround to avoid the hw dying and managed to decode a H264 video of some jellyfish.

The work is based on bunch of other stuff, and is in no way suitable for upstreaming yet, not to mention the Vulkan specification is only beta/provisional so can't be used anywhere outside of development.

The preliminary code is in my gitlab repo here[1]. It has a start on h265 decode, but it's not working at all yet, and I think the h264 code is a bit hangy randomly.

I'm not sure where this is going yet, but it was definitely an interesting experiment.

[1]: https://gitlab.freedesktop.org/airlied/mesa/-/commits/radv-vulkan-video-prelim-decode

GitLab servers are being exploited in DDoS attacks (The Record)

3 év 9 hónap óta
The Record is reporting on massive exploitation of an oldish vulnerability in GitLab instances.

While the purpose of these attacks remained unclear for HN Security, yesterday, Google’s Menscher said the hacked servers were part of a botnet comprising of “thousands of compromised GitLab instances” that was launching large-scale DDoS attacks.

The vulnerability was fixed in April, but evidently a lot of sites have not updated.

corbet

ksmbd: a new in-kernel SMB server (SAMBA+ blog)

3 év 9 hónap óta
Over at the SAMBA+ blog, the performance of the new ksmbd kernel SMB server and Samba in user space are compared: ksmbd claims performance improvements on a wide range of benchmarks: the graphs on this page show a doubling of performance on some tests. There was also the notion that an in-kernel server is likely an easier place to support SMB Direct, which uses RDMA to transfer data between systems.

Clearly, those number are impressive, but at the same time recent improvements in Samba's IO performance put this into perspective: by leveraging the new "io_uring" Linux API Samba is able to provide roughly 10x the throughput compared to ksmbd.

Time will tell whether it's better to reside in kernel-space like ksmbd or in user-space like Samba in order to squeeze the last bit of performance out of the available hardware.

There are two graphs that show some impressive results for Samba.

jake

Horgan: Linux x86 Program Start Up

3 év 9 hónap óta
Patrick Horgan explains the process of starting a program on Linux in great detail.

Well, __libc_start_main calls __libc_init_first, who immediately uses secret inside information to find the environment variables just after the terminating null of the argument vector and then sets a global variable __environ which __libc_start_main uses thereafter whenever it needs it including when it calls main. After the envp is established, then __libc_start_main uses the same trick and surprise! Just past the terminating null at the end of the envp array, there's another vector, the ELF auxiliary vector the loader uses to pass some information to the process. An easy way to see what's in there is to set the environment variable LD_SHOW_AUXV=1 before running the program.

corbet

[$] 5.16 Merge window, part 1

3 év 9 hónap óta
As of this writing, Linus Torvalds has pulled exactly 6,800 non-merge changesets into the mainline repository for the 5.16 kernel release. That is probably a little over half of what will arrive during this merge window, so this is a good time to catch up on what has been pulled so far. There are many significant changes and some large-scale restructuring of internal kernel code, but relatively few ground-breaking new features.
corbet

Security updates for Thursday

3 év 9 hónap óta
Security updates have been issued by Fedora (ansible, chromium, kernel, mupdf, python-PyMuPDF, rust, and zathura-pdf-mupdf), openSUSE (qemu and webkit2gtk3), Red Hat (firefox and kpatch-patch), Scientific Linux (firefox), SUSE (qemu, tomcat, and webkit2gtk3), and Ubuntu (firefox and thunderbird).
jake