Pro tip: Don’t run out of space on Btrfs.

@movq@www.uninformativ.de Pretty much. In my situation I was able to delete some files and snapshots and run a couple of different btrfs balance commands to move some allocations around. It looked like writes weren’t all committed properly to the disk but nothing told me that explicitly.

I did a system update in this state and I think I remember mkinitcpio throwing more warnings than usual but I was doing something else and I didn’t pay close attention to them. This coincided with a power outage and there was a lot of inconsistency, making me think it was hardware related. It was just btrfs, as far as I can tell, and I fixed it by reinstalling all the packages on the system once there was enough room. Luckily, I hadn’t done anything important with that computer after the system update.

@movq@www.uninformativ.de Yeah, it seems like that should never happen under any circumstances but that’s the best explanation I can come up with for what happened and once I fixed the space issue the other problems went away. That particular filesystem is on a LUKS device on a disk image served with NBD. The machine in question and the NBD server are both on Arch Linux so it has potentially unstable versions of all the software involved.

It’s a real house of cards and I’m not surprised something like this happened. I’m keeping lots of backups. My setup is pretty unique but I stand by my original post. Running out of space on Btrfs isn’t fun, even when it’s functioning properly.