Kernel Planet

Gustavo A. R. Silva: Influencing Software Security: The Impact of the Kernel Self-Protection Project ⚔️🛡️🐧

1 év 8 hónap óta
Compiler Options Hardening Guide

On November 29th, the Open Source Security Foundation (OpenSSF) released a comprehensive and thorough hardening guide aimed at mitigating potential vulnerabilities in C and C++ code through the use of various hardening compiler options.

This guide references some of the work we’ve accomplished over the years in the Kernel Self-Protection Project (KSPP), particularly our efforts to globally enable -Wimplicit-fallthrough and -fstrict-flex-arrays=3 in the upstream Linux kernel.

-Wimplicit-fallthrough

This warning flag warns when a fallthrough occurs unless it is specially marked as being intended. The Linux kernel project uses this flag; it led to the discovery and fixing of many bugs21.

-fstrict-flex-arrays=3

In this guide we recommend using the standard C99 flexible array notation [] instead of non-standard [0] or misleading [1], and then using -fstrict-flex-arrays=3 to improve bounds checking in such cases. In this case, code that uses [0] for a flexible array will need to be modified to use [] instead. Code that uses [1] for a flexible arrays needs to be modified to use [] and also extensively modified to eliminate off-by-one errors. Using [1] is not just misleading39, it’s error-prone; beware that existing code using [1] to indicate a flexible array may currently have off-by-one errors40.

GCC hardening features

The work of Qing Zhao is also referenced in the guide. Qing is making significant contributions to the KSPP by implementing hardening features in GCC, which we want to adopt in the Linux kernel.

Beyond the Linux kernel

In conclusion, it’s quite fulfilling to see the hardening work we undertake in the Kernel Self-Protection Project having a significant influence in the world of software security, beyond the Linux kernel.

Matthew Garrett: Why does Gnome fingerprint unlock not unlock the keyring?

1 év 8 hónap óta
There's a decent number of laptops with fingerprint readers that are supported by Linux, and Gnome has some nice integration to make use of that for authentication purposes. But if you log in with a fingerprint, the moment you start any app that wants to access stored passwords you'll get a prompt asking you to type in your password, which feels like it somewhat defeats the point. Mac users don't have this problem - authenticate with TouchID and all your passwords are available after login. Why the difference?

Fingerprint detection can be done in two primary ways. The first is that a fingerprint reader is effectively just a scanner - it passes a graphical representation of the fingerprint back to the OS and the OS decides whether or not it matches an enrolled finger. The second is for the fingerprint reader to make that determination itself, either storing a set of trusted fingerprints in its own storage or supporting being passed a set of encrypted images to compare against. Fprint supports both of these, but note that in both cases all that we get at the end of the day is a statement of "The fingerprint matched" or "The fingerprint didn't match" - we can't associate anything else with that.

Apple's solution involves wiring the fingerprint reader to a secure enclave, an independently running security chip that can store encrypted secrets or keys and only release them under pre-defined circumstances. Rather than the fingerprint reader providing information directly to the OS, it provides it to the secure enclave. If the fingerprint matches, the secure enclave can then provide some otherwise secret material to the OS. Critically, if the fingerprint doesn't match, the enclave will never release this material.

And that's the difference. When you perform TouchID authentication, the secure enclave can decide to release a secret that can be used to decrypt your keyring. We can't easily do this under Linux because we don't have an interface to store those secrets. The secret material can't just be stored on disk - that would allow anyone who had access to the disk to use that material to decrypt the keyring and get access to the passwords, defeating the object. We can't use the TPM because there's no secure communications channel between the fingerprint reader and the TPM, so we can't configure the TPM to release secrets only if an associated fingerprint is provided.

So the simple answer is that fingerprint unlock doesn't unlock the keyring because there's currently no secure way to do that. It's not intransigence on the part of the developers or a conspiracy to make life more annoying. It'd be great to fix it, but I don't see an easy way to do so at the moment.

comments

Gustavo A. R. Silva: November 2023 – Linux Kernel work

1 év 8 hónap óta

-Wstringop-overflow

Late in October I sent a patch to globally enable the -Wstringop-overflowcompiler option, which finally landed in linux-next on November 28th. It’s expected to be merged into mainline during the next merge window, likely in the last couple of weeks of December, but “We’ll see”. I plan to send a pull request for this to Linus when the time is right. 🙂

I’ll write more about the challenges of enabling this compiler option once it’s included in 6.8-rc1, early next year. In the meantime, it’s worth mentioning that several people, including Kees Cook, Arnd Bergmann, and myself, have sent patches to fix -Wstringop-overflow warnings over the past few years.

Below are the patches that address the last warnings, together with the couple of patches that enable the option in the kernel. The first of them enables the option globally for all versions of GCC. However, -Wstringop-overflow is buggy in GCC-11. Therefore, I wrote a second patch adding this option under new configuration CC_STRINGOP_OVERFLOW in init/Kconfig, which is enabled by default for all versions of GCC except GCC-11. To handle the GCC-11 case I added another configuration: GCC11_NO_STRINGOP_OVERFLOW, which will disable -Wstringop-overflowby default for GCC-11 only.

Boot crash on ARM64

Another relevant task I worked on recently was debugging and fixing a boot crash on ARM64, reported by Joey Gouly. This issue was interesting as it related to some long-term work in the Kernel Self-Protection Project (KSPP), particularly our efforts to transform “fake” flexible arrays into C99 flexible-array members. In short, there was a zero-length fake flexible array at the end of a structure annotated with the __randomize_layout attribute, which needed to be transformed into a C99 flexible-array member.

This becomes problematic due to how compilers previously treated such arrays before the introduction of -fstrict-flex-arrays=3. The randstruct GCC plugin treated these arrays as actual flexible arrays, thus leaving their memory layout untouched when the kernel is built with CONFIG_RANDSTRUCT. However, after commit 1ee60356c2dc (‘gcc-plugins: randstruct: Only warn about true flexible arrays’), this behavior changed. Fake flexible arrays were no longer treated the same as proper C99 flexible-array members, leading to randomized memory layout for these arrays in structures annotated with __randomize_layout, which was the root cause of the boot crash.

To address this, I sent two patches. The first patch is the actual bugfix, which includes the flexible-array transformation. The second patch is complementary to commit 1ee60356c2dc, updating a code comment to clarify that “we don’t randomize the layout of the last element of a struct if it’s a proper flexible array.”

diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 07022bb0d44d..0d28172193fa 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -162,7 +162,7 @@ struct neighbour { struct rcu_head rcu; struct net_device *dev; netdevice_tracker dev_tracker; - u8 primary_key[0]; + u8 primary_key[]; } __randomize_layout; struct neigh_ops { diff --git a/scripts/gcc-plugins/randomize_layout_plugin.c b/scripts/gcc-plugins/randomize_layout_plugin.c index 910bd21d08f4..746ff2d272f2 100644 --- a/scripts/gcc-plugins/randomize_layout_plugin.c +++ b/scripts/gcc-plugins/randomize_layout_plugin.c @@ -339,8 +339,7 @@ static int relayout_struct(tree type) /* * enforce that we don't randomize the layout of the last - * element of a struct if it's a 0 or 1-length array - * or a proper flexible array + * element of a struct if it's a proper flexible array */ if (is_flexible_array(newtree[num_fields - 1])) { has_flexarray = true;

These two patches will be soon backported to a couple of -stable trees.

-Wflex-array-member-not-at-end

During my last presentation at Kernel Recipes in September this year, I discussed a bit about -Wflex-array-member-not-at-end, which is a compiler option currently under development for GCC-14.

One of the highlights of the talk was a 6-year-old bug that I initially uncovered through grepping, and later, while reviewing some build logs from previous months, I realized that -Wflex-array-member-not-at-end had also detected this problem:

This bugfix was backported to 6.5.7, 6.1.57, 5.15.135, 5.10.198, 5.4.258 and 4.19.296 stable kernels.

Encouraged by this discovery, I started hunting for more similar bugs. My efforts led to fixing a couple more:

On November 28th, these two bugfixes were successfully backported to multiple stable kernel trees. The first fix was applied to the 6.6.3, 6.5.13, 6.1.64 stable kernels. The second fix was also applied to these, along with the 5.15.140 stable kernel.

I will have a lot of fun with -Wflex-array-member-not-at-end next year.

-Warray-bounds

In addition to these tasks, I continued addressing -Warray-boundsissues. Below are some of the patches I sent for this.

Patch review and ACKs.

I’ve also been involved in patch review and providing ACKs. Kees Cook, for instance, has been actively annotating flexible-array members with the__counted_byattribute, and I’ve been reviewing those patches.

Google Open Source Peer Bonus Award

In other news from November, I want to share that I’m thrilled to be the recipient of this award from Google for the first time. I feel really grateful and honored!

This comes as a result of my contributions to the Linux kernel over the years.

Honestly, I didn’t even know about the existence of this award until I received an email from someone at Google informing me about it. However, learning about it made me feel really great!

My appreciation goes out to my teammates in the Kernel Self-Protection Project, especially to Kees Cook, who has been an invaluable mentor to me over the years. Special thanks to Greg Kroah-Hartman as well, who was instrumental in setting me on my journey as a Linux kernel developer.

Acknowledgements

Special thanks to The Linux Foundation and Google for supporting my Linux kernel work.

Davidlohr Bueso: LPC 2023: CXL Microconference

1 év 8 hónap óta

The Compute Express Link (CXL) microconference was held, for a second straight time, at this year's Linux Plumbers Conference. The goals for the track were to openly discuss current on-going development efforts around the core driver, as well as experimental memory management topics which lead to accommodating kernel infrastructure for new technology and use cases.

CXL session at LPC23
(i) CXL Emulation in QEMU - Progress, status and most importantly what next? The cxl qemu maintainers presented the current state of the emulation, for which significant progress has been made, extending support beyond basic enablement. During this year, features such as volatile devices, CDAT, poison and injection infrastructure have been added upstream qemu, while several others are in the process, such as CCI/mailbox, Scan Media and dynamic capacity. There was also further highlighting of the latter, for which DCD support was presented along with extent management issues found in the 3.0 spec. Similarly, Fabric Management was another important topic, continuing the debate about qemu's role in FM development, which is still quite early. Concerns about the production (beyond testing) use cases for CCI kernel support were discussed, as well as semantics and interfaces that constrain qemu, such as host and switch coupling and differences with BMC behavior.
(ii) CXL Type-2 core support. The state and purpose of existing experimental support for type 2 (accelerators) devices was presented, for both the kernel and qemu sides. The kernel support led to preliminary abstraction improvement work being upstreamed, facilitating actual accelerator integration with the cxl core driver. However, the rest is merely guess work and the floor is open for an actual hardware backed proposal. In addition, HDM-DB support would also be welcomed as a step forward. The qemu side is very basic and designed to just exercise core checks, for which it's emulation should be limited, specially in light of cxl_test.
(iii) Plumbing challenges in Dynamic capacity device. An in-depth coverage and discussion, from a kernel side, of the state of DCD support and considerations around corner cases. Semantics of releasing DC for full partial extents (ranges) are two different beasts altogether. Releasing all the already given  memory can simply require memory being offline and be done, avoiding unnecessary complexity in the kernel. Therefore the kernel can perfectly well reject the request, and FM design should keep that into consideration. Partial extents, on the other hand, are unsupported for the sake of simplicity, at least until a solid industry use case comes along. Forced DC removal of online memory semantics were also discussed, emphasizing that such DC memory is not guaranteed to ever be given back by the kernel, mapped or not. Forcing the event, the hardware does not care and the kernel has most likely crashed anyway. Support for extent tagging was another topic, establishing the need for supporting it, coupling a device to a tag domain, being a sensible use case. For now at least, the implementation can be kept to to simply enumerate tags and the necessary attributes to leave the memory matching to userspace, instead of more complex surgeries to create DAX devices on specific extents, dealing with sparse regions.
(iv) Adding RAS Support for CXL Port Devices. Starting with a general overview of RAS, this touched on the current state for support in CXL 1.1 and 2.0.  Special handling is required for RCH: due to the RCRB implementation, the RCH downstream port does not have a BDF, needed for AER error handling; this work was merged in v6.7. As for CXL Virtual Hierarchy implementation, it is left still open, potentially things could move away from the PCIe port service driver model, which is not entirely liked. There are however, clear requirements: not-CXL specific (AER is a PCIe protocol, used by CXL.io); implement driver callback logic specific to that technology or device, giving flexibility to handle that specific need; and allow enable/disable on a per-device granularity. There were discussions around the order for which a registration handler is added in the PCI port driver, noting that it made sense to go top-down from the port and searching children, instead of written from a lower level.
(v) Shared CXL 3 memory: what will be required? Overview of the state, semantics and requirements for supporting shared fabric attached memory (FAM). A strong enablement use case is leveraging applications that already handle data sets in files. In addition appropriate workload candidates will fit the "master writer, multiple readers" read-only model for which this sort of machinery would make sense. Early results show that the benefits can out-weigh costly remote CXL memory access such as fitting larger data sets in FAM that would otherwise be possible in a single host. Similarly this avoids cache-coherency costs by simply never modifying the memory. A number of concrete data science and AI usecases were presented. Shared FAM is meant to be mmap-able, file-backed, special purpose memory, for which a FAMFS prototype is described, overcoming limitations of just using DAX device/FSDAX, such as distributing metadata in a shareable way.
(vi) CXL Memory Tiering for heterogenous computing. Discusses the pros and cons of interleaving heterogeneous (ie: DRAM and CXL) memory through hardware and/or software for bandwidth optimization. Hardware interleaving is simple to configure through the BIOS, but limited by not allowing the OS to manage allocations, otherwise hiding the NUMA topology (single node) as well as being a static configuration. The software interleaving solves these limitations with hardware and relies on weighted nodes for allocation distribution when doing the initial mapping (vma). Several interfaces have been posted, which incrementally are converging into a NUMA node based interface. The caveat is to have a single (configurable) system-wide set of weights, or to allow more flexibility, such as hierarchically through cgroups - something which has not been particularly sold yet. Combining both hardware and software models relies on within a socket, splitting channels among respective DDR and CXL NUMA nodes for which software can explicitly (numactl) set the interleaving - it is still restrained however by being static as the BIOS is in charge of setting the number of NUMA nodes. 
(vii) A move_pages() equivalent for physical memory. Through an experimental interface, this focused on the semantics of tiering and device driven page movement. There are currently various mechanisms for access detection, such as PMU-based, fault hinting for page promotion and idle bit page monitoring; each with its set of limitations, while runtime overhead is a universal concern. Hardware mechanisms could help with the burden but the problem is that devices only know physical memory and must therefore do expensive reverse mapping lookups; nor are there any interfaces for this, and it is difficult to with out hardware standardization. A good starting point would be to keep the suggested move_phys_pages as an interface, but not have it be an actual syscall.

Pete Zaitcev: Suvorov vs Zubrin

1 év 8 hónap óta

Bro, u mad?

Zubrin lies like he breathes, not bothering even calculate in Excel, not to mention take integrals!

But they continue to believe him — because it's Zubrin!

When they discussed his "engine on salt of Uranium" [NSWR — zaitcev] I wrote that the engine will not work in principle, because a reactor can only work in case of effective deceleration of neutrons, but already at the temperature of the moderator of 3000 degrees (like in KIWI), the cross-section of fission decreases 10 times, and the critical mass increases proportionally. But nobody paid attention — who am I, who is Zubrin!

The core has to be hot, and the moderator has to be cool, this is essential.

But they continued to fantasize, is it going to be 100,000 degrees in there, or only 10,000?

No matter how much I pointed out the principal contradiction — here is the cold sub-critical solution, and here is super-critical plasma, only in a meter or two away, and therefore neutrons from this plasma fly into the solution — which will inevitably capture them, decelerate, and react, and therefore the whole concept goes down the toilet.

But they disucss this salt engine over decades, without trying to check Zubrin's claims.

All "normal" nuclear reactors work only in the region between "first" (including the delayed neutrons) and "second" (with fast neutrons) criticalities. Only in this region, control of the reactor is possible. By the way, the difference in breeding ratios is only 1.000 and 1.007 for slow neutrons and 1.002 for the fast ones (in case of Plutonium, this much is the case even for slow neutrons).

And by the way, average delay for delayed neutrons is 0.1 seconds! The solution has to remain in the active zone for 100 milliseconds, in order to capture the delayed neutrons! Not even the solid phase RD-0410 reached that much.

Therefore, Zubrin's engine must be critical at the prompt neutrons. And because the moderator underperforms because it's hot, prompt neutrons become indistinguishable from fission neutrons, and therefore the density of plasma has to be the same as density of metal in order to achieve criticality — that is to say, almost 20 g/sm^3 for Uranium.

But this persuades nobody, because Zubrin is Zubrin, and who are you?

It all began as a discussion of Mars Direct among geeks, but escalated quickly.

James Morris: Linux Security Summit 2023 Videos & Slides

1 év 8 hónap óta

Videos and slides from the 2023 Linux Security summits may be found here:

Linux Security Summit North America (LSS-NA), May 10-12 2023, Vancouver, Canada.

Linux Security Summit Europe (LSS-EU), September 20-21 2023, Bilbao, Spain.

Note: if you wish to follow Linux Security Summit announcements and event updates via Mastodon, see https://social.kernel.org/LinuxSecSummit. You can follow this via the Fediverse or the RSS reader of your choice.

Linux Plumbers Conference: Live Stream URLs for Plumbers are now on the Website

1 év 8 hónap óta

As a reminder, The live stream of each main track of Linux Plumbers Conference will be available in real time on Youtube.  The Links are now live in the timetable.  To view, go to the Schedule Overview and click on the paperclip on the upper right of the track you want to watch to bring up the Live Stream URL.

Live Stream viewers may interact over chat by joining the Matrix Room of that event.  To see all our Matrix rooms for Plumbers, go to the space #lpc2023:lpc.events in matrix.  The room names should be pretty intuitive.

Linux Plumbers Conference: BBB Training Session for Virtual Presenters and Attendees

1 év 9 hónap óta

We’ll be holding a BBB Training session on Thursday (8 November) at:

7am PST, 10am EST, 3pm UTC, 4pm CET, 8:30pm IST, 12am Friday JST

This will be recorded so that you can watch it later.

What is BBB? It’s an open source video software, similar to Zoom and Google Meets, but is much better for interactions between remote attendees and a live audience.

There are several features that BBB provides, and this training session will go over the common ones that you will likely use during your presentation.

This session is highly recommend for those that are presenting remotely, and may also be useful for those that are only attending remotely, to get a feel for the platform.  In person attendees are welcome too, but we’ll have shepherds in the conference rooms on the day to help you out.

To join, you will need to log in to: https://meet.lpc.events

After logging in, to join the meeting, click the Hackroom entry in the leftnav then select the join button of Hackroom 1.

Dave Airlie (blogspot): nouveau GSP firmware support - current state

1 év 9 hónap óta

Linus has pulled the initial GSP firmware support for nouveau. This is just the first set of work to use the new GSP firmware and there are likely many challenges and improvements ahead.

To get this working you need to install the firmware which hasn't landed in linux-firmware yet.

For Fedora this copr has the firmware in the necessary places:

https://copr.fedorainfracloud.org/coprs/airlied/nouveau-gsp/build/6593115/ 

Hopefully we can upstream that in next week or so.

If you have an ADA based GPU then it should just try and work out of the box, if you have Turing or Ampere you currently need to pass nouveau.config=NvGspRm=1 on the kernel command line to attempt to use GSP.

Going forward, I've got a few fixes and stabilization bits to land, which we will concentrate on for 6.7, then going forward we have to work out how to keep it up to date and support new hardware and how to add new features.


Matthew Garrett: Why ACPI?

1 év 9 hónap óta
"Why does ACPI exist" - - the greatest thread in the history of forums, locked by a moderator after 12,239 pages of heated debate, wait no let me start again.

Why does ACPI exist? In the beforetimes power management on x86 was done by jumping to an opaque BIOS entry point and hoping it would do the right thing. It frequently didn't. We called this Advanced Power Management (Advanced because before this power management involved custom drivers for every machine and everyone agreed that this was a bad idea), and it involved the firmware having to save and restore the state of every piece of hardware in the system. This meant that assumptions about hardware configuration were baked into the firmware - failed to program your graphics card exactly the way the BIOS expected? Hurrah! It's only saved and restored a subset of the state that you configured and now potential data corruption for you. The developers of ACPI made the reasonable decision that, well, maybe since the OS was the one setting state in the first place, the OS should restore it.

So far so good. But some state is fundamentally device specific, at a level that the OS generally ignores. How should this state be managed? One way to do that would be to have the OS know about the device specific details. Unfortunately that means you can't ship the computer without having OS support for it, which means having OS support for every device (exactly what we'd got away from with APM). This, uh, was not an option the PC industry seriously considered. The alternative is that you ship something that abstracts the details of the specific hardware and makes that abstraction available to the OS. This is what ACPI does, and it's also what things like Device Tree do. Both provide static information about how the platform is configured, which can then be consumed by the OS and avoid needing device-specific drivers or configuration to be built-in.

The main distinction between Device Tree and ACPI is that Device Tree is purely a description of the hardware that exists, and so still requires the OS to know what's possible - if you add a new type of power controller, for instance, you need to add a driver for that to the OS before you can express that via Device Tree. ACPI decided to include an interpreted language to allow vendors to expose functionality to the OS without the OS needing to know about the underlying hardware. So, for instance, ACPI allows you to associate a device with a function to power down that device. That function may, when executed, trigger a bunch of register accesses to a piece of hardware otherwise not exposed to the OS, and that hardware may then cut the power rail to the device to power it down entirely. And that can be done without the OS having to know anything about the control hardware.

How is this better than just calling into the firmware to do it? Because the fact that ACPI declares that it's going to access these registers means the OS can figure out that it shouldn't, because it might otherwise collide with what the firmware is doing. With APM we had no visibility into that - if the OS tried to touch the hardware at the same time APM did, boom, almost impossible to debug failures (This is why various hardware monitoring drivers refuse to load by default on Linux - the firmware declares that it's going to touch those registers itself, so Linux decides not to in order to avoid race conditions and potential hardware damage. In many cases the firmware offers a collaborative interface to obtain the same data, and a driver can be written to get that. this bug comment discusses this for a specific board)

Unfortunately ACPI doesn't entirely remove opaque firmware from the equation - ACPI methods can still trigger System Management Mode, which is basically a fancy way to say "Your computer stops running your OS, does something else for a while, and you have no idea what". This has all the same issues that APM did, in that if the hardware isn't in exactly the state the firmware expects, bad things can happen. While historically there were a bunch of ACPI-related issues because the spec didn't define every single possible scenario and also there was no conformance suite (eg, should the interpreter be multi-threaded? Not defined by spec, but influences whether a specific implementation will work or not!), these days overall compatibility is pretty solid and the vast majority of systems work just fine - but we do still have some issues that are largely associated with System Management Mode.

One example is a recent Lenovo one, where the firmware appears to try to poke the NVME drive on resume. There's some indication that this is intended to deal with transparently unlocking self-encrypting drives on resume, but it seems to do so without taking IOMMU configuration into account and so things explode. It's kind of understandable why a vendor would implement something like this, but it's also kind of understandable that doing so without OS cooperation may end badly.

This isn't something that ACPI enabled - in the absence of ACPI firmware vendors would just be doing this unilaterally with even less OS involvement and we'd probably have even more of these issues. Ideally we'd "simply" have hardware that didn't support transitioning back to opaque code, but we don't (ARM has basically the same issue with TrustZone). In the absence of the ideal world, by and large ACPI has been a net improvement in Linux compatibility on x86 systems. It certainly didn't remove the "Everything is Windows" mentality that many vendors have, but it meant we largely only needed to ensure that Linux behaved the same way as Windows in a finite number of ways (ie, the behaviour of the ACPI interpreter) rather than in every single hardware driver, and so the chances that a new machine will work out of the box are much greater than they were in the pre-ACPI period.

There's an alternative universe where we decided to teach the kernel about every piece of hardware it should run on. Fortunately (or, well, unfortunately) we've seen that in the ARM world. Most device-specific simply never reaches mainline, and most users are stuck running ancient kernels as a result. Imagine every x86 device vendor shipping their own kernel optimised for their hardware, and now imagine how well that works out given the quality of their firmware. Does that really seem better to you?

It's understandable why ACPI has a poor reputation. But it's also hard to figure out what would work better in the real world. We could have built something similar on top of Open Firmware instead but the distinction wouldn't be terribly meaningful - we'd just have Forth instead of the ACPI bytecode language. Longing for a non-ACPI world without presenting something that's better and actually stands a reasonable chance of adoption doesn't make the world a better place.

comments

Pete Zaitcev: Python subprocess and stderr

1 év 9 hónap óta

Suppose you want to create a pipeline with the subprocess and you want to capture the stderr. A colleague of mine upstream wrote this:

p1 = subprocess.Popen(cmd1, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True) p2 = subprocess.Popen(cmd2, stdin=p1.stdout, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True) p1.stdout.close() p1_stderr = p1.communicate() p2_stderr = p2.communicate() return p1.returncode or p2.returncode, p1_stderr, p2_stderr

Unfortunately, the above saves the p1.stdout in memory, which may come back to bite the user once the amount piped becomes large enough.

I think the right answer is this:

with tempfile.TemporaryFile() as errfile: p1 = subprocess.Popen(cmd1, stdout=subprocess.PIPE, stderr=errfile, close_fds=True) p2 = subprocess.Popen(cmd2, stdin=p1.stdout, stdout=subprocess.PIPE, stderr=errfile, close_fds=True) p1.stdout.close() p2.communicate() p1.wait() errfile.seek(0) px_stderr = errfile.read() return p1.returncode or p2.returncode, px_stderr

Stackoverflow is overflowing with noise on this topic. Just ignore it.

Kees Cook: Enable MTE on Pixel 8

1 év 9 hónap óta

The Pixel 8 hardware (Tensor G3) supports the ARM Memory Tagging Extension (MTE), and software support is available both in Android userspace and the Linux kernel. This feature is a powerful defense against linear buffer overflows and many types of use-after-free flaws. I’m extremely happy to see this hardware finally available in the real world.

Turning it on for userspace is already wired up the Android UI: Settings / System / Developer options / Memory Tagging Extension / Enable MTE until you turn if off. Once enabled it will internally change an Android “system property” named “arm64.memtag.bootctl” by adding the option “memtag“.

Turning it on for the kernel is slightly more involved, but not difficult at all. This requires manually setting the “arm64.memtag.bootctl” property mentioned above to include “memtag-kernel” as well:

  • Plug your phone into a system that can run the adb tool
  • If not already installed, install adb. For example on Debian/Ubuntu: sudo apt install adb
  • Turn on “USB Debugging” in the phone’s “Developer options” menu, and accept the debugging session confirmation that will pop up when you first run adb
  • Verify the current setting: adb shell getprop | grep memtag.bootctl
  • [arm64.memtag.bootctl]: [memtag]

  • Enable kernel MTE: adb shell setprop arm64.memtag.bootctl memtag,memtag-kernel
  • Check the results: adb shell getprop | grep memtag.bootctl
  • [arm64.memtag.bootctl]: [memtag,memtag-kernel]

  • Reboot your phone

To check that MTE is enabled for the kernel (which is implemented using Kernel Address Sanitizer’s Hardware Tagging mode), you can check the kernel command line after rebooting:

$ mkdir foo && cd foo $ adb bugreport ... $ mkdir unpacked && cd unpacked $ unzip ../bugreport*.zip ... $ grep kasan= bugreport*.txt ...: Command line: ... kasan=off ... kasan=on ...

The latter “kasan=on” overrides the earlier “kasan=off“.

Enjoy!

© 2023, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 License.

Rusty Russell: Covenants: Dealing with Amounts in Bitcoin Script

1 év 9 hónap óta

Covenants are a construction to allow introspection: a transaction output can place conditions on the transaction which spends it (beyond the specific “must provide a valid signature of itself and a particular pubkey”).

I previously looked at Examining ScriptPubkeys, but another useful thing covenants want to enforce is amounts. This is easy for equality, but consider the case where you are allowed to merge inputs: perhaps the first output amount must be the sum of the first and second inputs.

The problem is that Bitcoin Script deals in signed ones-complement values, and 31 bits limits us to 21.47483648 bitcoin. However, using OP_MULTISHA256 or OP_CAT, it’s possible to deal with full amounts. I’ve written some (untested!) script code below.

The Vexing Problem of Amounts

Using OP_TXHASH, we can get SHA256(input amount) and SHA256(output amount) on the stack. Since this involves hashing, we can’t evaluate the number for anything but equality, so as in other cases where we don’t have Fully Complete Covenants we need to have the user supply the actual values on the witness stack, and we test those for the conditions we want, and then make sure they match what OP_TXHASH says is in the transaction. I usually object to this backwards form (just give me the value on the stack!), but as you’ll see, we couldn’t natively use 64 bit values from OP_TX anyway (I had proposed pushing two values, which is its own kind of ugly).

A Value Form Bitcoin Script Can Deal With

21M BTC is just under 2^51 satoshis.

We split these bits into a pair of stack values:

  • lower 24 bits
  • upper bits (27, but we allow up to 31)

I call this tuple “Script-friendly pair” (SFP) form. Note that all script numbers on stack are represented in little-endian, with a sign bit (0x80 on the last byte). This is a nasty format to work with, unfortunately.

Converting A Script-Friendly Pair to an 8-byte Little-Endian Value

Here’s the code to takes a positive CScriptNum, and produces two stack values which can be concatenated to make a 4 byte unsigned value:

# !UNTESTED CODE! # Stack (top to bottom): lower, upper OP_SWAP # Generate required prefix to append to stack value to make it 4 bytes long. OP_SIZE OP_DUP OP_NOTIF # 0 -> 00000000 OP_DROP 4 OP_PUSHDATA1 0x00 0x00 0x00 0x00 OP_ELSE OP_DUP 1 OP_EQUAL OP_IF # Single byte: prepend 0x00 0x00 0x00 OP_DROP 3 OP_PUSHDATA1 0x00 0x00 0x00 OP_ELSE OP_DUP 2 OP_EQUAL OP_IF # Two bytes: prepend 0x00 0x00 2 OP_PUSHDATA1 0x00 0x00 OP_ELSE 3 OP_EQUAL OP_IF # Three bytes: prepend 0x00 1 OP_PUSHDATA1 0x00 OP_ELSE # Prepend nothing. 0 OP_ENDIF OP_ENDIF OP_ENDIF OP_ENDIF OP_SWAP # Stack (top to bottom): upper, pad, lower

That 46 bytes handles upper. Now lower is a CScriptNum between 0 and 16777215, and we want to produce two stack values which can be concatenated to make an 3 byte unsigned value. Here we have to remove the zero-padding in the four-byte case:

# !UNTESTED CODE! # Stack (top to bottom): upper, pad, lower OP_ROT # Generate required prefix to append to stack value to make it 3 bytes long. OP_SIZE OP_DUP OP_NOTIF # 0 -> 000000 OP_DROP 3 OP_PUSHDATA1 0x00 0x00 0x00 OP_ELSE OP_DUP 1 OP_EQUAL OP_IF # Single byte: prepend 0x00 0x00 OP_DROP 2 OP_PUSHDATA1 0x00 0x00 OP_ELSE OP_DUP 2 OP_EQUAL OP_IF # Two bytes. Now maybe final byte is 0x00 simply so it doesn't # appear negative, but we don't care. 1 OP_PUSHDATA1 0x00 OP_ELSE # Three bytes: empty append below 3 OP_EQUAL OP_NOTIF # Four bytes, e.g. 0xff 0xff 0xff 0x00 # Convert to three byte version: negate and add 2^23 # => 0xff 0xff 0xff OP_NEG 4 OP_PUSHDATA1 0x00 0x00 0x80 0x00 OP_ADD OP_ENDIF # Prepend nothing. 0 OP_ENDIF OP_ENDIF OP_ENDIF OP_SWAP # Stack (top to bottom): lower, pad, upper, pad

You can optimize these 47 bytes a little, but I’ll leave that as an exercise for the reader!

Now we use OP_MULTISHA256 (or OP_CAT 3 times and OP_SHA256) to concatentate them to form an 8-byte little-endian number, for comparison against the format used by OP_TXHASH.

Basically, 95 bytes to compare our tuple to a hashed value.

Adding Two Script-Friendly Pairs

Let’s write some code to add two well-formed Script-Friendly Pairs!

# !UNTESTED CODE! # Stack (top to bottom): a_lower, a_upper, b_lower, b_upper OP_ROT OP_ADD OP_DUP 4 OP_PUSHDATA1 0x00 0x00 0x00 0x01 OP_GREATERTHANOREQUAL OP_IF # lower overflow, bump upper. # FIXME: We can OP_TUCK this constant above! 4 OP_PUSHDATA1 0x00 0x00 0x00 0x01 OP_SUB OP_SWAP OP_1ADD OP_ELSE OP_SWAP OP_ENDIF # Stack now: a_upper(w/carry), lower_sum, b_upper. OP_ROT OP_ADD OP_SWAP # Stack now: lower_sum, upper_sum

Note that these 26 bytes don’t check that upper doesn’t overflow: if we’re dealing with verified amounts, we can add 16 times before it’s even possible (and it’s never possible with distinct amounts of course). Still, we can add OP_DUP 0 OP_GREATERTHANOREQUAL OP_VERIFY before the final OP_SWAP.

Checking Script-Friendly Pairs

The code above assumes well-formed pairs, but since the pairs will come from the witness stack, we need to have a routine to check that a pair is wel-formed:

# !UNTESTED CODE! # Stack: lower, upper OP_DUP # lower must be 0 - 0xFFFFFF inclusive 0 4 OP_PUSHDATA1 0xFF 0xFF 0xFF 0x00 OP_WITHIN OP_VERIFY OP_OVER # upper must be 0 - 0x7FFFFFF inclusive 0 4 OP_PUSHDATA1 0xFF 0xFF 0xFF 0x07 OP_WITHIN OP_VERIFY

This ensures the ranges are all within spec: no negative numbers, no giant numbers.

Summary

While this shows that OP_CAT/OP_MULTISHA256 is sufficient to deal with bitcoin amounts in Script, the size (about 250 bytes to validate that two inputs equals one output) makes a fairly compelling case for optimization.

It’s worth noting that this is why Liquid chose to add the following 64-bit opcodes to bitscoin script: OP_ADD64, OP_SUB64, OP_MUL64, OP_DIV64, OP_NEG64, OP_LESSTHAN64, OP_LESSTHANOREQUAL64, OP_GREATERTHAN64, OP_GREATERTHANOREQUAL64.

(They also reenabled the bitwise opcodes (OP_XOR etc) to work just fine with these. They also implemented OP_SCRIPTNUMTOLE64, OP_LE64TOSCRIPTNUM and OP_LE32TOLE64 for conversion.)

In my previous post I proposed OP_LESS which works on arbitrary values, which doen’t work for these because the endian is wrong! As a minimum, we’d need to add OP_LESSTHAN64, OP_ADD64 and OP_NEG64 to allow 64-bit comparison, addition and subtraction.

But, with only OP_CAT or OP_MULTISHA256, it’s possible to deal with amounts. It’s just not pretty!

Thanks for reading!

Rusty Russell: Covenants: Examining ScriptPubkeys in Bitcoin Script

1 év 9 hónap óta

Covenants are a construction to allow introspection: a transaction output can place conditions on the transaction which spends it (beyond the specific “must provide a valid signature of itself and a particular pubkey”).

My preferred way of doing instrospection is for Bitcoin Script have a way of asking for various parts of the transaction onto the stack (aka OP_TX) for direct testing (Fully Complete Covenants, as opposed to using some tx hash, forcing the Script to produce a matching hash to pass (Equality Covenants). In the former case, you do something like:

# Is the nLocktime > 100? OP_TX_BIT_NLOCKTIME OP_TX 100 OP_GREATERTHAN OP_VERIFY

In the latter you do something like:

# They provide nLocktime on the stack. OP_DUP # First check it's > 100 100 OP_GREATERTHAN OP_VERIFY # Now check it's actually the right value, by comparing its hash the hash of nLocktime OP_SHA256 OP_TX_BIT_NLOCKTIME OP_TXHASH OP_EQUALVERIFY

However, when we come to examining an output’s ScriptPubkey, we’re forced into the latter mode unless we’re seeking an exact match: the ScriptPubkey is (almost always) a one-way function of the actual spending conditions.

Making a Simple Taproot, in Script

Let’s take a simple taproot case. You want to assert that the scriptPubkey pays to a known key K, or a script given by the covenent spender. This is the simplest interesting form of Taproot, with a single script path.

The steps to make this into a ScriptPubkey (following BIP 341) are:

  1. Get a tagged tapleaf hash of the script
  2. Tweak the key K by this value.
  3. Prepend two bytes “0x51 0x20”.
  4. Compare with the ScriptPubkey of this tx.
Step 1: We need OP_CAT, or OP_MULTISHA256

If we spell out the things we need to hash, it looks like:

SHA256(SHA256("TapLeaf") + SHA256("TapLeaf") + 0xC0 + CSCRIPTNUM(LEN(script)) + script)

CSCRIPTNUM(X) is (if X is in canonical form, as it will be from OP_SIZE):

  • if X is less than 253:
    • X
  • otherwise, if the length is less than 256:
    • 0xFD 0x00 X
  • otherwise, if the length is less than 65536:
    • 0xFD X
  • otherwise, we don’t care, make shorter scripts!

The obvious way to do this is to enable OP_CAT, but this was removed because it allows construction of giant stack variables. If that is an issue, we can instead use a “concatenate-and-hash” function OP_MULTISHA256, which turns out to be easiest to use if it hashes the stack from top to bottom.

OP_MULTISHA256 definition:

  1. If the stack is empty, fail.
  2. Pop N off the stack.
  3. If N is not a CScriptNum, fail.
  4. If there are fewer than N entries on the stack, fail.
  5. Initialize a SHA256 context.
  6. while N > 0:
    1. Pop the top entry off the stack.
    2. Hash it into the SHA256 context
    3. Decrement N
  7. Finish the SHA256 context, and push the resulting 32 bytes onto the stack.

The result is either:

# Script is on stack, produce tagged tapleaf hash # First, encode length OP_SIZE OP_DUP # < 253? OP_PUSHDATA1 1 253 OP_LESSTHAN OP_IF # Empty byte on stack: 0 OP_ELSE OP_DUP # > 255? OP_PUSHDATA1 1 0xFF OP_GREATERTHAN OP_IF OP_PUSHDATA1 1 0xFD OP_ELSE # Needs padding byte OP_PUSHDATA1 2 0xFD 0x00 OP_ENDIF OP_ENDIF # Push 0xC0 leaf_version on stack OP_PUSHDATA1 1 0xC0 # Push hashed tag on stack, twice. OP_PUSHDATA1 7 "TapLeaf" OP_SHA256 OP_DUP # Now, hash them together 6 OP_MULTISHA256

Or, using OP_CAT (assuming it also concatenates the top of stack to second on stack):

# Script is on stack, produce tagged tapleaf hash # First, encode length OP_SIZE OP_DUP # < 253? OP_PUSHDATA1 1 253 OP_LESSTHAN OP_NOTIF OP_DUP # > 255? OP_PUSHDATA1 1 0xFF OP_GREATERTHAN OP_IF OP_PUSHDATA1 1 0xFD OP_ELSE # Needs padding byte OP_PUSHDATA1 2 0xFD 0x00 OP_ENDIF OP_CAT OP_ENDIF # Prepend length to script OP_CAT # Prepend 0xC0 leaf_version OP_PUSHDATA1 1 0xC0 OP_CAT # Push hashed tag on stack, twice, and prepend OP_PUSHDATA1 7 "TapLeaf" OP_SHA256 OP_DUP OP_CAT OP_CAT # Hash the lot. OP_SHA256 Step 2: We need to Tweak a Key, OP_KEYADDTWEAK

Now, we need to tweak a public key, as detailed in BIP 341:

def taproot_tweak_pubkey(pubkey, h): t = int_from_bytes(tagged_hash("TapTweak", pubkey + h)) if t >= SECP256K1_ORDER: raise ValueError P = lift_x(int_from_bytes(pubkey)) if P is None: raise ValueError Q = point_add(P, point_mul(G, t)) return 0 if has_even_y(Q) else 1, bytes_from_int(x(Q))

Let’s assume OP_KEYADDTWEAK works like so:

  1. If there are less than two items on the stack, fail.
  2. Pop the tweak t off the stack. If t >= SECP256K1_ORDER, fail.
  3. Pop the key P off the stack. If it is not a valid compressed pubkey, fail. Convert to Even-Y if necessary. (i.e. lift_x()).
  4. Q = P + t*G.
  5. Push the X coordinate of Q on the stack.

So now we just need to create the tagged hash, and feed it to OP_KEYADDTWEAK:

# Key, tapscript hash are on stack. OP_OVER OP_PUSHDATA1 8 "TapTweak" OP_SHA256 OP_DUP # Stack is now: key, tapscript, key, H(TapTweak), H(TapTweak) 4 OP_MULTISHA256 OP_KEYADDTWEAK

Or with OP_CAT instead of OP_MULTISHA256:

# Key, tapscript hash are on stack. OP_OVER OP_PUSHDATA1 8 "TapTweak" OP_SHA256 OP_DUP # Stack is now: key, tapscript, key, H(TapTweak), H(TapTweak) OP_CAT OP_CAT OP_CAT OP_SHA256 OP_KEYADDTWEAK Step 3: We Need To Prepend The Taproot Bytes

This is easy with OP_CAT:

# ScriptPubkey, Taproot key is on stack. # Prepend "OP_1 32" to make Taproot v1 ScriptPubkey OP_PUSHDATA1 2 0x51 0x20 OP_CAT OP_EQUALVERIFY

With OP_MULTISHA256 we need to hash the ScriptPubkey to compare it (or, if we only have OP_TXHASH, it’s already hashed):

# ScriptPubkey, Taproot key is on stack. OP_SHA256 # Prepend "OP_1 32" to make Taproot v1 ScriptPubkey OP_PUSHDATA1 2 0x51 0x20 2 OP_MULTISHA256 # SHA256(ScriptPubkey) == SHA256(0x51 0x20 taproot) OP_EQUALVERIFY Making a More Complete Taproot, in Script

That covers the “one key, one script” case.

If we have more than one taproot leaf, we need to perform the merkle on them, rather than simply use the taproot leaf directly. Let’s assume for simplicity that we have two scripts:

  1. Produce the tagged leaf hash for scripts, call them H1 and H2.
  2. If H1 < H2, merkle is TaggedHash("TapBranch", H1 + H2), otherwise TaggedHash("TapBranch", H2 + H1)
Step 1: Tagged Hash

We’ve done this before, it’s just Step 1 as before.

Step 2: Compare and Hash: We Need OP_LESS or OP_CONDSWAP

Unfortunately, all the arithmetic functions except OP_EQUAL only take CScriptNums, so we need a new opcode to compare 32-byte blobs. Minimally, this would be OP_LESS, though OP_CONDSWAP (put lesser one on top of stack) is possible too. In our case we don’t care what happens in unequal lengths, but if we assume big-endian values are most likely, we could zero-prepend to the shorter value before comparing.

The result looks like this:

# Hash1, Hash2 are on the stack. # Put lesser hash top of stack if not already OP_LESS OP_NOTIF OP_SWAP OP_ENDIF OP_PUSHDATA1 9 "TapBranch" OP_SHA256 OP_DUP 4 OP_MULTISHA256

Or, using OP_CAT and OP_CONDSWAP:

# Hash1, Hash2 are on the stack. # Put lesser hash top of stack if not already OP_CONDSWAP OP_PUSHDATA1 9 "TapBranch" OP_SHA256 OP_DUP OP_CAT OP_CAT OP_CAT OP_SHA256

So now we can make arbitrarily complex merkle trees from parts, in Script!

Making More Useful Templates: Reducing the Power of OP_SUCCESS

Allowing the covenant spender to specify a script branch of their own is OK if we simply want a condition which is “… OR anything you want”. But that’s not generally useful: consider vaults, where you want to enforce a delay, after which they can spend. In this case, we want “… AND anything you want”.

We can, of course, insist that the script they provide starts with 1000 OP_CHECKSEQUENCEVERIFY. But because any unknown opcode causes immediate script success (without actually executing anything), they can override this test by simply inserting an invalid opcode in the remainder of the script!

There are two ways I can see to resolve this: one is delegation, where the remainder of the script is popped off the stack (OP_POPSCRIPT?). You would simply insist that the script they provide be exactly 1000 OP_CHECKSEQUENCEVERIFY OP_POPSCRIPT.

The other way is to weaken OP_SUCCESSx opcodes. This must be done carefully! In particular, we can use a separator, such as OP_SEPARATOR, and change the semantics of OP_SUCCESSx:

  • If there is an OP_SEPARATOR before OP_SUCCESSx:
    • Consider the part before the OP_SEPARATOR:
      • if (number of OP_IF) + (number of OP_NOTIF) > (number of OP_ENDIF): fail
      • Otherwise execute it as normal: if it fails, fail.
  • Succeed the script

This insulates a prefix from OP_SUCCESSx, but care has to be taken that it is a complete script fragment: a future OP_SUCCESSx definition must not turn an invalid script into a valid one (by revealing an OP_ENDIF which would make the script valid).

Summary

I’ve tried to look at what it would take to make generic convenants in Script: ones which can meaningfully interrogate spending conditions assuming some way (e.g. OP_TXHASH) of accessing an output’s script. There are reasons to believe this is desirable (beyond a completeness argument): vaulting in particular requires this.

We need three new Script opcodes: I’ve proposed OP_MULTISHA256, OP_KEYADDTWEAK and OP_LESS, and a (soft-fork) revision to treatment of OP_SUCCESSx. None of these are grossly complex.

The resulting scripts are quite long (and mine are untested and no doubt buggy!). It’s 41 bytes to hash a tapleaf, 19 to combine two tapleaves, 8 to compare the result to the scriptpubkey. That’s at least 109 witness weight to do a vault, and in addition you need to feed it the script you’re using for the output. That seems expensive, but not unreasonable: if this were to become common then new opcodes could combine several of these steps.

I haven’t thought hard about the general applicability of these opcodes, so there may be variants which are better when other uses are taken into account.

Thanks for reading!

Linux Plumbers Conference: LPC 2023 Schedule is now posted!

1 év 9 hónap óta

The schedule for when the miniconferences and tracks are going to occur is now posted at: https://lpc.events/event/17/timetable/#all

The Linux Plumbers Refereed track schedule is now available at: https://lpc.events/event/17/timetable/#all.detailed

The runners for the miniconferences and kernel summit will be adding more details to each of their schedules over the coming weeks, as will the leads for the networking and toolchain tracks.

For those that are registered as in person, you are free to continue to submit Birds of a Feather(BOF) sessions. They will be allocated space in the BOF rooms on a first come, first serve basis. Please note that these BOFs will not be recorded.

We’re looking forward to a great 3 days of presentations and discussions. We hope you can join us either in-person or virtually!

Matthew Garrett: Defending abuse does not defend free software

1 év 9 hónap óta
The Free Software Foundation Europe and the Software Freedom Conservancy recently released a statement that they would no longer work with Eben Moglen, chairman of the Software Freedom Law Center. Eben was the general counsel for the Free Software Foundation for over 20 years, and was centrally involved in the development of version 3 of the GNU General Public License. He's devoted a great deal of his life to furthering free software.

But, as described in the joint statement, he's also acted abusively towards other members of the free software community. He's acted poorly towards his own staff. In a professional context, he's used graphically violent rhetoric to describe people he dislikes. He's screamed abuse at people attempting to do their job.

And, sadly, none of this comes as a surprise to me. As I wrote in 2017, after it became clear that Eben's opinions diverged sufficiently from the FSF's that he could no longer act as general counsel, he responded by threatening an FSF board member at an FSF-run event (various members of the board were willing to tolerate this, which is what led to me quitting the board). There's over a decade's evidence of Eben engaging in abusive behaviour towards members of the free software community, be they staff, colleagues, or just volunteers trying to make the world a better place.

When we build communities that tolerate abuse, we exclude anyone unwilling to tolerate being abused[1]. Nobody in the free software community should be expected to deal with being screamed at or threatened. Nobody should be afraid that they're about to have their sexuality outed by a former boss.

But of course there are some that will defend Eben based on his past contributions. There were people who were willing to defend Hans Reiser on that basis. We need to be clear that what these people are defending is not free software - it's the right for abusers to abuse. And in the long term, that's bad for free software.

[1] "Why don't people just get better at tolerating abuse?" is a terrible response to this. Why don't abusers stop abusing? There's fewer of them, and it should be easier.

comments

Rusty Russell: The HRF Bounty On Human-Readable Offers

1 év 10 hónap óta

At least two people have contacted me concerning the 2 BTC bounty:

2 BTC for a human-readable bolt 12 offer generator feature integrated into a popular iOS or android bitcoin wallet. “Human-readable” means something that can be used on feature phone without QR or copy/paste ability. For example, something that looks like LN address.

This, of course, is asking to solve Zooko’s Triangle, so one of decentralizationm, human readability, or security needs to compromise! Fortunately, the reference to LN address gives a hint on how we might proceed.

The scenario, presumably, is Bob wants to pay Alice, where Alice shows Bob a “Human Readable Offer” and Bob types it into his phone. Each one runs Phoenix, Greenlight, or (if their phone is too low-end) uses some hosted service, but any new third party trust should be minimized.

There are three parts we need here:

  1. Bob finds Alice’s node.
  2. Bob requests Alice’s node for invoice.
  3. If she wants, Alice can easily check Bob’s going to pay the right thing.
The Imagined Scenario

Consider the normal offer case: the offer encodes Alice’s nodeid and description (and maybe other info) about what’s on offer. Bob turns this into an invoice_request, sends an onion message to Alice’s node, which returns the (signed) invoice, which Bob pays. We need to encode that nodeid and extra information as compactly as we can.

Part 1: Finding Alice’s Node from a Human Readable Offer

The issue of “finding Alice’s node” has been drafted already for BOLT12, at https://github.com/rustyrussell/bolt12address (but it needs updating!). This means that if you say “rusty@blockstream.com” you can get a valid generic offer, either by contacting the webserver at “blockstream.com” or having someone else do it for you (important for privacy!), or even downloading a public list of common receivers.

Note that it’s easier to type * than @ on feature phones, so I suggest allowing both rusty@blockstream.com and RUSTY*BLOCKSTREAM.COM.

What’s Needed On The Server
  1. The BOLT 12 Address Format needs to be updated.
  2. It needs to be implemented for some Web server.
  3. Ideally, integrate it into BTC Payserver or the like.
Part 2: Getting the Invoice

Now, presumably, we want a specific invoice: it might be some default “donate to Alice”, but it could be a specific thing “$2 hot dog”. So you really want some (short!) short-code to indicate which invoice you want. I suggest a hash, followed by some randomly chosen alphanumeric string here (case-insensitive!): an implementation may choose to restrict themselves to numbers however, as that’s faster to enter on a feature phone.

What’s Needed On The Server
  1. We can put the short-code in the invreq_payer_note field in BOLT 12 or add a new odd field.
  2. We need to implement (presumably in Core Lightning):
    • A way to specify/assign a short-code for each offer.
    • A way of serving a particular invoice based on this short-code match.
Part 3: Checking the Invoice

So, did you even get the right node id? That’s the insecure part; you’re trusting blockstream.com! Checking the nodeid is hard: someone can grind out a nodeid with the same first 16 digits in a few weeks. But I think you can provide some assurance, by creating a 4-color “flag” using the node id and the latest bitcoin blocks: this will change every new block, and is comparable between Alice and Bob at a glance:

This was made using this hacky code which turns my node id 024b9a1fa8e006f1e3937f65f66c408e6da8e1ca728ea43222a7381df1cc449605 into an RGB color (by hashing the nodeid+blockhash).

For a moment, when a new block comes in, one image might be displaced, hence the number, but it’ll only be out by one.

Putting it All Together What’s Needed On Alice’s Client
  1. Alice needs to configure her BOLT12 Address with some provider when she sets up the phone: it should check that it works!
  2. She should be able to choose an existing offer (may be a “donation” by default), or create a new one on the fly (with a new short code).
  3. Display the BOLT12-ADDRESS # SHORT-CODE, and the current nodeid flag.
What’s Needed On Bob’s Client
  1. It needs to be able to convert BOLT12-ADDRESS into a bolt12 address request:
    • Either via some service (to be implemented!), or by directly query (ideally over Tor).
  2. It needs to be able to produce an offer from the returns bolt12 address response, by putting the SHORT-CODE into the invreq_payer_note.
  3. It needs to be able to fetch an invoice for this offer.
  4. It needs to be able to display the current nodeid flag for the invoice’s node id.
  5. Allow Bob to confirm to send payment.
Is There Anything Else?

There are probably other ways of doing this, but this method has the advantage of driving maturity in several different areas which we want to see in Bitcoin:

  1. bolt12 address to support vendor field validation for offers.
  2. Simple name support for bootstrapping.
  3. Driving Bitcoin to be more accessible to everyone!

Feel free to contact me with questions!

Ellenőrizve
15 perc 36 másodperc ago
Kernel Planet - http://planet.kernel.org
Feliratkozás a következőre: Kernel Planet hírcsatorna