31 December 2022

The one where the wins come at the very end

Technically this is a blog reboot, not a new start. It's been much much more than a minute since I've done any sort of long-form, updates(?) on what I do for fun or pain, and maybe somebody wants to waste some time reading. This is also half-instigated by my friend Tiffany:

The ship has kinda sailed for the quarterly reports, but I think blogging more than makes up for that. Spoiler: all of these "wins" have something to do with my work on FreeBSD ports. Not sorry to disappoint if you were expecting anything otherwise. In no particular order:

PEP-517 framework support

This one's been kicking my arse since the second quarter of the year, mostly between having to mind real life (mostly academia), other moving parts of the open source ecosystem living in our ports tree and actually playing human (so more real life).

I'm so damn excited to land this. Can't put it into words. Not from fighting all the trials and tribulations called distractions and moving parts within the ports tree, but rather PEP-517 is just that much better than the lemon Python packaging once was.

Implementing this into the ports framework, with yuri@ starting it off, wasn't particularly difficult. I really wanted to focus more on the not-really-technical bits of proper problem scoping and consumer expectations, namely to provide a seamless as possible transition between from the old way without reinventing any wheels (pun intended). Those first couple implementation takes before I could catch wind of it (because real life) just didn't cut it.

Some more real life later, the one remaining but incredibly important snag to seamless transition was making sure autoplist behaved itself, including when multiple Python distributions (versions) are on the system at the same time. These two iterations of Python packaging have always included installed file manifests, so it only makes sense to continue using this facility rather than having porters maintain a separate manifest (called a plist) for each Python package port.

Next steps are to rework the use of setuptools in the tree. There's a lot of unnecessary, and now wrong, uses of setuptools, particularly after version 58…

Cinnamon 5.4

I had this going way earlier in the year, to the point where as of this writing 5.6 is the current version. But I never put it up in Phabricator, until now, because I thought I would get around to qualifying the reworked Wacom support. I guess that will wait for either when I sit down to work on 5.6 or xf86-input-wacom.

Firefox and Thunderbird toolchains

I'm currently typing this on a fully up-to-date Firefox 108.0.1:

I ran Firefox 107 for a little bit even after it End of Lifed, not the greatest idea of course. As alluded from the tweet, the delay in updating came squarely to toolchain selection…

Both the latest Firefox and Thunderbird not only require C, C++ and Rust toolchains, but the very beginning of the build process uses Node.js and parts of the source are now in WebAssembly. It didn't seem that long ago when it was just C, C++ and Rust; hell I still remember the days of wrangling Cygwin and Microsoft Visual C++ to play nice together to build my own Windows binaries.

For Firefox 107 I had LLVM/Clang 14, Rust 1.64 and a revision of WebAssembly System Interface (WASI) libc from early May 2022. Particularly between LLVM/Clang and Rust, the LLVM version has to match, particularly with link-time optimisation, else bizarre things happen. This worked until both Rust 1.65 and other factors had me moving to LLVM 15, which that revision of WASI libc did not support. I had tried earlier in the year to update to a revision dated early June 2022, but ended up with opcode errors I didn't know how to deal with…until actually having some time to sit down with them.

Turns out this was long a known issue, thanks to the folks at Arch Linux (which I used to use). Some time during May, WASI libc gained the use of bulk memory opcodes. This is otherwise a great thing! But the mozilla-central repository (home to Firefox sources) still contains and uses an older (and forked?) copy of wasm2c that does not support these opcodes, so when the WebAssembly bits get linked, the linker becomes sad:

error: unexpected opcode: 0xfc 0xa

I still needed to use a newer WASI libc that supported LLVM 15, which contains the bulk memory support, but that support was easily disabled in the Makefile, so I did that accordingly. And now we have the happy ending that, after some cleanup, I'll put in Phabricator.

Python cryptography library backports

I actually dedicated pretty much all of the week leading up to Christmas for this one, judging by commit logs. And almost entirely because about a fifth or sixth of my listed ports would not build without it.

But hold up, that's an ancient cryptography version. And you would be right, in a vacuum. But we don't live in a vacuum, we live in an imperfect world where we have choices in secure communication libraries (a very Good Thing) and how they are implemented. I switched to LibreSSL soon after Heartbleed happened and haven't looked back to OpenSSL since, even though there have been moments where I wanted to give up the good fight in keeping consumers compatible with both.

On the cryptography side of things, starting version 35, they integrated Rust code to replace bindings for non-cryptographic operations. Their project, their choice.

There exist certain operating system communities where it is a tradition to build your entire system, or as much as personally practicable, from source. Gentoo is probably the most (in)famous about building from source, but building from source has always been a tradition in the open source BSD family, even though all have binary repositories as a convenience. With this in mind, compiled programs should be relatively quick or pain-free to build, which ties in with Unix philosophy and C's wide proliferation among other things. Building the entire Rust toolchain from source is extremely painful and resource-intensive compared to all that. It is this pain that has some longer-time committers and other FreeBSD community members calling for de-oxidisation…

I care about that but not to that extent. Sure, it might take like five hours or so on this ThinkPad, but I'm also not afraid to tweak operational parameters such that I start the build when I crash for the night and wake up to hopefully success. The real problem for me is that Rust (and go and probably some more new-age compiled language toolchains) will not build or run under QEMU user-mode, which I use as part of cross-compiling stuff for my ARM and RISC-V single-boards.

So cherry-picking we go. As long as stuff keeps working and tests (particularly downstream) passing for the most part between OpenSSL 1.1.1 and 3.0 and LibreSSL 3.5 and later, I'm a happy man.

What's next

Time to enjoy the scenery and get lit for the new year, until the next post I guess.

at No comments:
Subscribe to: Posts (Atom)