11 March 2024

club dispatch (episode 1)

writing during yet more FreeBSD port builds must be a theme innit

I also happen to be in my local transit agency's dispatch office writing most of this. Yes, I sometimes dispatch buses, usually for night service so it's often chill enough to do other things whilst maintaining watch. Don't worry, shit does hit the fan sometimes, which means I get to be first to get remedies or corrective action moving. Yes I get monies for this, but more importantly gets me out of the house when I otherwise don't have plans for the week, particularly social or scheming for world domination.

Anyway, here is another episode of…open source despatches.


Technically my hardware setup was complete during the drawn-out writing process of the last episode, but I had no idea what I was doing. I still don't know what I'm doing, but at least I figured out how to beatmatch two easier-to-me tracks by ear, after the last episode dropped. Wait, what?!

"day zero posterity pic"

Yes, I am indeed learning how to DJ. Despite the computer monitor's presence, it is and will not be used to display audio waveforms like CDJs and controllers/software do. I think phonograph records and turntables are the purest form of this art, even if dialing up digital tracks for the timecode record to control.

So far I've started amassing my first crate of records and playing around with them. Genres for now are all 140-bpm grime and dubstep (specifically not the brostep branch). Once I set up my mixer, the last hardware to arrive, I attempted to mix two grime tracks together (as shown in the pic, if you know you know) and it was an epic fail. Then I tried one grime one dubstep, later both dubstep, even later almost indiscriminately selecting anything that would make sense, all to no avail. First attempts in learning.

I eventually realised that I needed to isolate some percussive element on the downbeat ("the one"). In many electronically-produced genres, following a 4/4 time signature, this would be a snare. With grime, percussive elements are all over the place, vocals included. In dubstep, the snare usually falls on the three, but there are less percussive elements overall so it is easier to pick something. So a few days later, I dialed up two dubstep tracks on the turntables and focused on the kicks starting from the bass drop. DJ Rob Swift, a longtime hip-hop turntablist, calls this "building a relationship". After a couple tries, I had the two tracks synced, so rinse and repeat those exact two a few more times to make sure I really had them. Luck would have it that these two tracks were at the same bpm so no pitch adjustments needed, but if I dropped the second track just a bit off, I would have to nudge the second record forward or touch the platter ever so slightly. It's fun when you start to get it in this manner, even with needed adjustments.

If this was on a controller or CDJ, the former of which would have been much easier on the wallet, I would have gotten up to speed a bit more quickly but with some bad habits in tow. Looking at waveforms to beatmatch would be like using a debugger on a problem program without fully understanding program architecture and control outside of specific code segments. Beatmatching by ear, ie "building a relationship", is exactly that, building a relationship with your program from a total lifecycle standpoint before diving into code at all.

I still need to practise matching before or without any bass drops, because sometimes I do want the intros heard in the mix. I also still haven't fired up Mixxx, the open source DJ program, to enable playing with my timecode records and thus control my digital tracks. We'll see how it behaves with my setup on FreeBSD. To this end, I'll turn it over to an old SEPTA slogan:

i don't think the gift shop has these to sell

DankPad omissions and update

I forgor to mention that the DankPad does have on-board wifi. In fact, ThinkPad X13 users get a random selection between three different M.2 wireless cards, none of which I believe work in FreeBSD as of this writing. My machine has the Qualcomm QCNFA765, which on Linux is driven by ath11k(4). Some machines get an Intel AX-series card, which is iwlwifi(4) territory. On FreeBSD, our home-grown iwm(4) driver supports Intel cards up to the AC-series, but anything newer uses iwlwifi(4). Our grand total of one (funded) FreeBSD committer working on wireless support started porting some Linux drivers over along with improving our LinuxKPI in our kernel, but some things are just incomplete:

ath11k_pci0: <ath11k_pci> mem 0xfd000000-0xfd1fffff at device 0.0 on pci3
ath11k_pci0: MSI vectors: 32
ath11k_pci0: wcn6855 hw2.0
linuxkpi_mhi_register_controller: XXX-BZ TODO

At some point I would like to help out in this effort. Funding would be nice too, since this is a hard but lucrative-outside-of-FreeBSD problem space. Let's not forget that I attempted to start porting not one but two Linux Realtek wireless drivers, one for the meme machine's built-in wifi and other for a TP-Link USB 3 dongle, but I stumbled over how to write a new Makefile with the existing C sources. I should build a relationship in that area before thinking about bigger things…

In the meantime my wireless needs are met by another TP-Link USB 2.0 dongle driven by rtwn_usb(4). Ethernet is through a USB 3.0 hub with a ure(4)-driven device at the end.

As for my suspend saga, the "Linux" sleep state, ie S3, BIOS option was the correct one. Our copy of ACPICA does not support S0ix so of course the `hw.acpi.suspend_state` sysctl came up empty when set to "Windows". With S3, the system suspends fine, but resuming is a different story: when I perform the actions in the X graphical environment, the AMD graphics hang the entire system on resume. I should be fine when I suspend in the console, which I have had to do on the other ThinkPad when I change monitor configurations, but again let's see what happens.

setuptools reduction act

I finally committed the setuptools-scm update discussed last time, after slightly tweaking my poudriere (port build automation) setup on the HP air fryer to add a separate environment for build verification runs. There is still some minor cleanup to do, because some Python packages have still not adopted PEP 517, and the latest setuptools-scm only officially works with setuptools versions that exclusively support the PEP 517 method…

…which brings me to why this section title.

tl;dr a long and winding history exists. For those ports still using the old method, direct setup.py execution, setuptools is a build dependency but also a run dependency. The latter part, in a blanket sense, is wrong. The only reason why setuptools would be a run dependency outside of a Python package build context is when the package uses the included pkg_resources, which itself is deprecated and replaced by other modules. Further, setuptools itself deprecated direct setup.py execution in version 58 and unsupported after, so all ports using this method will eventually specify setuptools 58 specifically. Two setuptools versions cannot exist in the same environment, at least not in the same Python distribution as they call it. The work is cut out twofold: an experimental build run to see how all ports using direct setup.py execution behave after removing the setuptools run dependency at the framework level, and finding individual ports that explicitly specify the setuptools run dependency.

More Will Be Revealed

26 February 2024

open source despatches, dankpad edition (episode 0)

Whilst I wait for my complete FreeBSD Ports rebuild to finishIn typical vishwin fashion where I let the post rot before I muster effort to finish writing, I introduce, hopefully, an excuse to write on this blog more regularly. And not in a New Year's resolution manner, because many of us know how well those go…

Welcome to Open Source Despatches, where I try to talk about not-the-most-public happenings in my open source world. As in, not formal releases, not public comments on bug trackers, patch submissions, etc. But maybe some of the public-facing stuff gets included for context.

Before I continue with this first episode, I think we need to address the elephant in the roomcabin: what the hell is a DankPad?

The DankPad

The DankPad is my colloquial name for my new-to-me ThinkPad X13 Gen 2 AMD. Not sorry to disappoint.

my active ThinkPad fleet as of Winter Field Day: W550s on left, X13 Gen 2 AMD (DankPad) on right

My ThinkPad W550s has been my primary machine through thick and thin since 2015. However, despite being a certified Ultrabook (ie thin-and-light, laugh out loud), the 15-inch form factor has gotten more cumbersome to carry around when I need to do so. Even with the machine this one replaced, I have always had a need for a smaller form factor secondary machine to both carry around and distribute processing loads. Thin doesn't cut it; the other dimensions matter more.

In 2022, Micro Center apparently got a boatload, spread across all their stores, of Evolve III Maestro netbooks (you read that right), initially retailing at $89 but later dropping to $59 a pop. At least within my circles, JonathanD jumped on early in the year to see how far he could go with them, particularly with running Linux and amateur radio applications. He eventually reported in two blog posts, but the initial vibes over IRC were good enough that jedijf got a fleet to play, break, learn.

Field Day comes around, he brings his entire Evolve fleet (amongst all the other stuff he brings) and invites me to get FreeBSD running on one.

trying to remember how to manually ZFS-on-root on a flash drive before Field Day ended

It's one thing to run an operating system from a removable disk (big up Damn Small Linux, my first Linux distribution I ever used), it's another to run from the internal disk, or in the Evolve's case, the 4 GB eMMC.

get in bitches, we're going to the toy store

I officially had to call this the meme laptop when I brought it around. See how far I could go with FreeBSD on it, which resulted in a wiki page. All for $59.

the early stages of getting Cinnamon set up

Fast forward to September 2023 (IRC log dated 18 October 2023):

[08:18:25] <vishwin> welp looks like my evolve is properly dead
[08:18:37] <vishwin> "incurable" boot looping...
[08:44:37] <JonathanD> :(
[08:54:25] <vishwin> jawn won't stay on past a few seconds, power
button doesn't take any input, just powers itself on and off anymore
[09:02:20] * waltman pours one out
[09:02:46] <waltman> vishwin: Fortunately you can switch over to
one of the other 5 you bought, right? :)
[09:03:28] <JonathanD> all mine are still going strong.
[09:04:36] <waltman> Having it die right before OKT POTA is
[09:04:42] <vishwin> it was actually dead last month

I sure as hell got my meme's worth of $59 outta that jawn. However, that made me down a small form factor machine, so back to just the W550s, again, cumbersome to carry around too often (and I was bringing it around a lot during the intervening period).

January 2024 comes around and I see a good deal on a refurbished X13 Gen 2 AMD. Low key, I'd been waiting for an opportunity to acquire an AMD machine, especially a Ryzen. Done.

eventually the stickers are getting transferred over

It is a little bigger at 13 inches compared to the Evolve's 11.6 inches, but this is practically a rounding error. Did I mention this is my first AMD machine in…12 years?

getting here took longer than it should have

The DankPad came with some Chinese no-name 256 GB NVMe SSD pre-loaded with Windows 10. I knew this would be insufficient capacity for the ZFS boot environments and other crap like source code I would load on here, especially considering this only takes a single internal disk, so the first order of business was to install a higher capacity SSD. But I needed to have the machine in hand to buy the correct new SSD, so that took a few days after I received the machine and built my first FreeBSD images for it. In the meantime, I attempted to put FreeBSD on the "stock" SSD, multiple times, to errors on first boot. I guess Windows stuff wasn't completely blown away from that SSD, but no harm no foul considering the new SSD. However, this did confirm that only UEFI boot on GPT works; all legacy MBR-based boot is off the table. Don't bother setting up both on the disk, you'd be wasting space.

This is also the first machine I own with a graphical BIOS. I guess nothing new for x86 machines released this decade, as all the Dell Optiplex 7010s I imaged for a now-completed contract had graphical BIOSes, but my last owned machine vintage is 2015 (the W550s). In the BIOS, there is a sleep state option, set to "Windows" by default but has a "Linux" option. Apparently these are bad labelling for selecting the warm suspend state presented to the operating system: "Windows" is S0ix whereas "Linux" is S3. Even though FreeBSD-CURRENT gets ACPICA updates into the base system fairly regularly, we still don't support S0ix. We don't even support S4 (hibernate) "unless S4BIOS is available" according to acpi(4). Need to play around with this a bit more, when I can carve some time when I don't need stuff running on it, but in the meantime no warm suspend for now. (The meme laptop never warm suspended under my tutelage.)

Built-in sound output generally works. Dolby Audio speaker system, however, Laugh Out Loud. Built-in microphone audio input is dodgy at best, even after adjusting mixer(8) and other userspace utilities from Ports. Built-in webcam doesn't appear to work with webcamd, at least derived from the Linux kernel version webcamd is on. No Discord voice/video calls with this machine anytime soon, unless I connect (supported) USB webcams and audio inputs. hselasky@ did all the work with webcamd and other USB class devices, but unfortunately he became another tragic victim of 2023; may he rest in peace.

Otherwise the DankPad is a major upgrade from the meme laptop and a great addition to my fleet. Zen 3 Ryzen has been worth every bit of money I dropped on it. I could use this to build all my ports and then some if I didn't already have an HP air fryer (HP ProLiant DL380 gen 7) for this role, but I do build the FreeBSD base system for this machine on this machine. With 12 threads, the first time building an entire world (everything but the kernel, configure files in /etc and release media/images) took 4060 seconds. In comparison, the HP air fryer took 3336 seconds with 24 threads and a few more LLVM machine targets. Only about a ten-minute difference which is impressive.

FreeBSD ports behind the scene

This is the part where I try to shed some light on why certain things I work on in FreeBSD Ports seem like they can take forever for no reason.

First, I'm actually being slowed down, erm crippled, by another large undertaking in the tree: mass realignment of man page locations. Because the package interiors have changed, we need to bump the revision number, as shown:

[00:00:11] Deleting libidn-1.38.pkg: new version: 1.38_1
[00:00:13] Deleting lynx-,1.pkg: missing dependency: libidn-1.38

The revision number bumping doesn't seem like a big deal, but when your package dependency metadata consist of all = instead of >=, everything that depends on the bumped package has to get rebuilt just to "correct" the metadata. For some heavy ports, this means the whole hog gets recompiled and such. Port Makefiles hardly have this kind of specificity; only the resulting package metadata end up like this. There is work in progress (not by me) to relax this. But in the meantime, every little bump can be costly on productivity, so I try to take every precaution to avoid them, until package metadata is relaxed.

One casualty from this and general tree churn is getting setuptools-scm updated. Not only updated, but the official name changed ever so slightly to use a hyphen rather than an underscore (both are equivalent in Python package metadata), so good to ensure we follow suit. Every so often, gotta rebase these changes on top of the tree as other stuff gets committed. Then everything has to get tested for build breakages at the very least.

Two more things are still being dogfooded: libxml2 2.12 and webkitgtk 2.42. The latter requires a change in our base system's copy and configuration of the libc++ API in order to build at all, let alone run. Otherwise, much of the work consisted of consolidating into one port three flavours depending on library combinations. Meanwhile libxml2 2.12 introduced some major API breakages even with a configure option, to be enabled in the port by default, to enable parts of the legacy API. Before getting there, still need to test an update to a consumer, libxslt, to make sure it behaves with the current libxml version in our tree (I don't see any obvious problems but this could be a jinx). After that I will offer the libxml2 update for wider testing.

any final words?

That's it for now. This post is long enough as it is, brain dumping without any coherency whatsoever. Plus this has been delayed for time. Some choice happenings, details and flames were omitted since I want to be done with this one already, but…more will be revealed.