Arch Linux for Windows: Now available from the Microsoft Store

Looks like I’m not the only one who noticed

Does anyone else sense the irony?

Yes, Arch Linux is available from the Microsoft Store, and no, I’m not kidding. Go ahead and see for yourself: https://apps.microsoft.com/detail/9mznmnksm73x?hl=en-us&gl=US

My immediate reaction when I saw this was, “woah, really?”, “that’s crazy”, and, “I never thought I’d see this day”, and I wonder if it’s as jarring to other people who haven’t grown up with Microsoft being such a behemoth in the news and in their lives.

I think my dad might have joked once, “if I had bought shares in Microsoft when you were a kid, instead of your Speak n’ Spell, we’d both be retired by now.” I remember being horrified by the suggestion, too. “No, dad! Really? My Speak n’ Spell?”

Most people saw over the years that Microsoft’s success was less of a testament to their ability to innovate, but to engage in monopolistic business practices. And it was certainly effective: Microsoft was ubiquitous, and everyday people across the country fantasized about where they might be had they bought stock in MS while they could afford it.

Having grown up hearing about them choking the competition all my life is certainly one of many reasons seeing Arch Linux in the Microsoft store is really super shocking. 😳

What’s next, Microsoft open sourcing their technologies?

😳

Back in the late ’90s / early ‘2000s, when the internet was still relatively new, Friendster was a thing, and RedHat was the only distro I’d ever heard of by name, there was a palpable sense that nothing would ever disrupt the Windows/MacOS duopoly, or cause them to reexamine their ultra-competitive business models.

The closed-intellectual property business model of Microsoft (and even more rabidly authoritarian monopolistic single-hardware vendor to walled-garden ecosystem Apple) seemed like one constant that would change about as much as death or taxes. Nobody expected Microsoft would start to give up its iron grip on source code, or start to even publish and market its own services using a Linux Operating System. (Yes, I’m looking at you, Azure hosted Kubernetes)

Bill Gates was still CEO, and they’d just had a hugely consequential precedent-setting battle in the courtroom for breaking anti-trust laws.

Microsoft bought or suffocated any decent competition, and leveraged their market proliferation into monopolization wherever it could. They eventually lost their lawsuit after several years of defending their position to achieve defacto omnipotence. It was surprising to see such a large, powerful company actually lose in court, but many saw the loss as a testament to how many bad faith business practices MS had engaged in. You have to be pretty anti-competition in this day and age to get charged by the federal government.

With a reputation like they had at the time, Anyone who knew anything about Microsoft was convinced they’d be going the locked-down software-selling, IP hoarding route forever. And while they’re certainly not perfect, they’re certainly engaging in practices I’ve never expected from them. Not in a million years.

The Microsoft I remember would have attempted to make it harder to run other company’s software. They’d never dream of providing Kubernetes on Azure with Linux containers, they’d make them all run Windows at $149 a pop, and force everyone to keep a copy of “MS Azure Browser” on their desktop.

And while I applaud MS for not pushing the boundaries of price elasticity as much as most other megacorps these days (i.e. Windows is still “only” $149) Linux’s source code is distributed for free in a way everyone can examine, so it’s hard to see how they could use their previously archetypal hoard-and-license IP business strategy.

What’s going on here? Are things actually changing?

To be clear, I know this distribution of Arch Linux for WSL has nothing to do with Microsoft really, other than it’s packaged for their proprietary VM layer on their still-$149-a-license operating system. But how jarring it is for me personally to see Microsoft change in this manner is pretty hard to overstate. I know MS is still making boatloads of money, but it’s amazing that something, or someone, could shake up the paradigm of how Microsoft Makes Money so fundamentally. And whether that was their intention, or it is simplyan artifact, there’s one person I believe we can point to for this eventually occurring (albeit, over the course of about 30-40 years):

Richard Stallman is the creator of the GNU Public License. Seen here playing your friend’s hippy dad taking you both to a cookout, he is the original stalwart advocate for free software distribution.

Stallman and his brainchild, the GPL, have been making the case that developers (and software companies) shouldn’t just release their code so people can know what it’s doing, but if they adopt any code released under GPL they are essentially forced to. Many other types of free distribution licenses allow companies to eventually close their source code and hoard their IP. There are plenty of reasons to do that, like security, competition, and market share. And if I understand correctly, it was an essential reason companies like Nintendo and Apple chose BSD for their OS platforms instead of Linux.

This is not meant to be a bash against companies for being greedy, under capitalism it’s essentially their main job. But it’s possible to be altruistic and make money. I don’t have any empirical evidence to support this suggestion, but I imagine enough developers realized hoarding IP is a barrier to improving technology, and whether for altruism, concerns about long-term self-preservation, or because their program relies heavily on a gzip library, enough people have gotten onboard with open-sourcing technology that it’s even been championed by Microsoft.

I have to pause a second to take that in. Microsoft has been the poster child of IP hoarding since their inception in 1980. By no means was adopting a license that would require developers and software companies to release their source code a fait accompli, but I suppose enough developers understood how much more rapidly we would progress collectively if we all shared information about how to do things. And for the rest of us who don’t make a habit of examining questions of existential importance on a daily basis, we probably just fell into it by proximity and convenience.

But realistically speaking, it also gives me pause to think of how much progress me might not have made if it weren’t for the open source community (regardless of license). What might not have been. Where we’d still be today. If, by 2024, open sourcing hadn’t proliferated so profoundly, we wouldn’t still be searching Alta Vista, trying to make out HTML tables on our flip phones.

And I say this gratefully, and without any sense of irony, that, for one reason or another, we all have Richard Stallman to thank for releasing Arch Linux in the Microsoft Store. Thank you, Richard!

And also probably because of all the parents who had the presence of mind to buy their kids Speak n’ Spells.

Using `grep` to focus only on that which is important

How can grep clear the clutter obstructing our goals?

This is a little beginner command-line demonstration for finding the results you need most. A lot of the time in the command line, a simple ls can return way more than any one person can reasonably deal with. Most people know how to use ls with extension flags: ls *.sh ; ls -a ; ls -a .??* , but what about situations when file extensions aren’t relevant, or when doing other things?

For those who don’t already know, grep is your spectacularly special search superhero you can use to slice and dice strings in a smattering of situations – and here’s a few instructive ways it can be used to make life easier when looking for something.

Say I’m troubleshooting kernel-install. kernel-install is an intrinsic part of making sure kernels are installed properly, but on most (if not all) distros, it’s packaged with systemd, a fairly sprawling package with tentacles in virtually everything. How can we sift through the systemd package to narrow down what we’re looking for, so our search becomes less overwhelming?

Bash
 pacman -Ql systemd | wc -l
1450

A quick analysis of the number of lines contained in the systemd package is 1,450 lines! That’s a lot of stuff to sort through in order to find a rather narrow section of it we need for our issue. That’s where some strategic use of the grep command can come in handy:

Bash
 pacman -Ql systemd | grep -v -E 'zsh|bash|man|polkit|locale' | wc -l
873

I usually start by thinking about what I don’t want in the results, since there’s almost always a lot more of what we don’t need than what we do need.

This filter with grep -v does the opposite of what grep does by default, and removes everything in a given search string. By using -E we can chain strings together, as if to instruct grep not to return results with any of these values (note the single quotes and pipe character separators)

If we use grep to omit all zsh functions, bash-completions, man files, polkit definitions, and locale settings, that can get us a little closer, but it’s still only narrowed down by 577 results. systemd is a lengthy package, indeed.

Bash
 pacman -Ql systemd | grep -v -E 'zsh|bash|man|polkit|locale' | grep kernel
systemd /etc/kernel/
systemd /etc/kernel/install.d/
systemd /usr/bin/kernel-install
systemd /usr/lib/kernel/
systemd /usr/lib/kernel/install.conf
systemd /usr/lib/kernel/install.d/
systemd /usr/lib/kernel/install.d/50-depmod.install
systemd /usr/lib/kernel/install.d/90-loaderentry.install
systemd /usr/lib/kernel/install.d/90-uki-copy.install
systemd /usr/lib/systemd/system/sockets.target.wants/systemd-udevd-kernel.socket
systemd /usr/lib/systemd/system/sys-kernel-config.mount
systemd /usr/lib/systemd/system/sys-kernel-debug.mount
systemd /usr/lib/systemd/system/sys-kernel-tracing.mount
systemd /usr/lib/systemd/system/sysinit.target.wants/sys-kernel-config.mount
systemd /usr/lib/systemd/system/sysinit.target.wants/sys-kernel-debug.mount
systemd /usr/lib/systemd/system/sysinit.target.wants/sys-kernel-tracing.mount
systemd /usr/lib/systemd/system/systemd-udevd-kernel.socket

 pacman -Ql systemd | grep -v -E 'zsh|bash|man|polkit|locale' | grep kernel | wc -l
17

From there, I realize we got lucky looking for kernel-install, since anything related to it is likely to have the word kernel in it. A quick addition of grep kernel as a subsequent filter on the end narrowed our search down to 17 results.

Now the list is fairly manageable. But what if we want it to be very accurate?

Bash
 pacman -Ql systemd | grep -v -E 'zsh|bash|man|polkit|locale|mount|socket' | grep kernel
systemd /etc/kernel/
systemd /etc/kernel/install.d/
systemd /usr/bin/kernel-install
systemd /usr/lib/kernel/
systemd /usr/lib/kernel/install.conf
systemd /usr/lib/kernel/install.d/
systemd /usr/lib/kernel/install.d/50-depmod.install
systemd /usr/lib/kernel/install.d/90-loaderentry.install
systemd /usr/lib/kernel/install.d/90-uki-copy.install

 pacman -Ql systemd | grep -v -E 'zsh|bash|man|polkit|locale|mount|socket' | grep kernel | wc -l
9

At this point, I’d go back to considering what not to include – in this case I know we don’t want any mount or socket files.

Are you noticing a pattern in these results, though? They include the files we’re looking for, related to kernel-install infrastructure, but also the folders. We know which folders they’re in through getting results for the files, so the folder locations are redundant:

Bash
 pacman -Ql systemd | grep -v -E 'zsh|bash|man|polkit|locale|mount|socket|/$' | grep kernel
systemd /usr/bin/kernel-install
systemd /usr/lib/kernel/install.conf
systemd /usr/lib/kernel/install.d/50-depmod.install
systemd /usr/lib/kernel/install.d/90-loaderentry.install
systemd /usr/lib/kernel/install.d/90-uki-copy.install

 pacman -Ql systemd | grep -v -E 'zsh|bash|man|polkit|locale|mount|socket|/$' | grep kernel | wc -l
5

This handy filter, grep -v '/$' should remove all the lines that end with a forward slash: / — and look, it ends up being only 5 related files: kernel-install, the binary, and the included .install scripts.

Since kernel is a fairly unique word, not often used in more than one context, it’s one of the easier ones to narrow down. We could have started by looking for the word kernel first:

Bash
 pacman -Ql systemd | grep 'kernel' | grep -v -E '/$'
systemd /usr/bin/kernel-install
systemd /usr/lib/kernel/install.conf
systemd /usr/lib/kernel/install.d/50-depmod.install
systemd /usr/lib/kernel/install.d/90-loaderentry.install
systemd /usr/lib/kernel/install.d/90-uki-copy.install
systemd /usr/lib/systemd/system/sockets.target.wants/systemd-udevd-kernel.socket
systemd /usr/lib/systemd/system/sys-kernel-config.mount
systemd /usr/lib/systemd/system/sys-kernel-debug.mount
systemd /usr/lib/systemd/system/sys-kernel-tracing.mount
systemd /usr/lib/systemd/system/sysinit.target.wants/sys-kernel-config.mount
systemd /usr/lib/systemd/system/sysinit.target.wants/sys-kernel-debug.mount
systemd /usr/lib/systemd/system/sysinit.target.wants/sys-kernel-tracing.mount
systemd /usr/lib/systemd/system/systemd-udevd-kernel.socket
systemd /usr/share/bash-completion/completions/kernel-install
systemd /usr/share/man/man7/kernel-command-line.7.gz
systemd /usr/share/man/man8/kernel-install.8.gz
systemd /usr/share/man/man8/systemd-udevd-kernel.socket.8.gz
systemd /usr/share/zsh/site-functions/_kernel-install

 pacman -Ql systemd | grep 'kernel' | grep -v -E '/$'  | wc -l
18

Since we put grep kernel first, and did what was most obvious (filter out all the bare folder results), this search becomes admittedly less complicated. However, I thought it might be helpful to demonstrate a more narrow, selective reduction with search filters, since there are likely to be situations where one is looking for something more difficult to narrow down, and eliminating results might make more sense than immediately looking for keywords related to it.

E.g. all executable binaries and libexec files in the libvirt package:

Bash
 pacman -Ql libvirt | grep -v -E 'zsh|bash|man|polkit|locale|mount|socket|/$|\.'
libvirt /usr/bin/libvirtd
libvirt /usr/bin/virsh
libvirt /usr/bin/virt-admin
libvirt /usr/bin/virt-host-validate
libvirt /usr/bin/virt-login-shell
libvirt /usr/bin/virt-pki-query-dn
libvirt /usr/bin/virt-pki-validate
libvirt /usr/bin/virt-qemu-qmp-proxy
libvirt /usr/bin/virt-qemu-run
libvirt /usr/bin/virt-qemu-sev-validate
libvirt /usr/bin/virt-ssh-helper
libvirt /usr/bin/virt-xml-validate
libvirt /usr/bin/virtchd
libvirt /usr/bin/virtinterfaced
libvirt /usr/bin/virtlockd
libvirt /usr/bin/virtlogd
libvirt /usr/bin/virtlxcd
libvirt /usr/bin/virtnetworkd
libvirt /usr/bin/virtnodedevd
libvirt /usr/bin/virtnwfilterd
libvirt /usr/bin/virtproxyd
libvirt /usr/bin/virtqemud
libvirt /usr/bin/virtsecretd
libvirt /usr/bin/virtstoraged
libvirt /usr/bin/virtvboxd
libvirt /usr/lib/libvirt/libvirt_iohelper
libvirt /usr/lib/libvirt/libvirt_leaseshelper
libvirt /usr/lib/libvirt/libvirt_lxc
libvirt /usr/lib/libvirt/libvirt_parthelper
libvirt /usr/lib/libvirt/virt-login-shell-helper
libvirt /usr/share/doc/libvirt/examples/sh/virt-lxc-convert

 pacman -Ql libvirt | grep -v -E 'zsh|bash|man|polkit|locale|mount|socket|/$|\.' | wc -l
31

This is a fairly good example, because how do you search for executables? Well, they don’t usually include a file extension, so that’s exactly what we filtered out with \. at the end (the dot needs to be escaped).

And, indeed, these are essentially all executable files, even the last one in /usr/share/doc (In the case of kernel-install, the .install files are executables, so it’s an awkward juxtaposition, but that’s not very common… I’m sure you get the idea…) A couple quick, last ones, along the same vein:

Bash
 pacman -Ql libvirt | grep '\.sh'
libvirt /usr/lib/libvirt/libvirt-guests.sh

What shell scripts are included in a package? That’s a good one, and from our first example:

Bash
 pacman -Ql dracut sbctl systemd systemd-ukify | grep -i -E '\.install|\.sh' | grep -ivE 'modules.d|x11' | sort -n
dracut /usr/lib/dracut/dracut-functions.sh
dracut /usr/lib/dracut/dracut-init.sh
dracut /usr/lib/dracut/dracut-logger.sh
dracut /usr/lib/dracut/dracut-version.sh
dracut /usr/lib/kernel/install.d/50-dracut.install
dracut /usr/lib/kernel/install.d/51-dracut-rescue.install
sbctl /usr/lib/kernel/install.d/91-sbctl.install
systemd-ukify /usr/lib/kernel/install.d/60-ukify.install
systemd /usr/lib/kernel/install.d/50-depmod.install
systemd /usr/lib/kernel/install.d/90-loaderentry.install
systemd /usr/lib/kernel/install.d/90-uki-copy.install

What .install or .sh scripts are included in dracut, sbctl, systemd and systemd-ukify – sorted in numeric order (which reflects the order of execution in most freedesktop.org software). There’s a little “flag-stacking” in that grep -ivE command, too.

Anyway, I hope this demo helps people. Feel free to leave comments or suggestions, especially if there’s anything you think I missed or left out. Thanks!

Outputting a list of variables to `json` or `yaml` using `column` and `goyq`

Quick little command line kung fu job with this parser called column

Bash
 column -h

Usage:
 column [options] [<file>...]

Columnate lists.

Options:
 -t, --table                      create a table
 -n, --table-name <name>          table name for JSON output
 -O, --table-order <columns>      specify order of output columns
 -C, --table-column <properties>  define column
 -N, --table-columns <names>      comma separated columns names
 -l, --table-columns-limit <num>  maximal number of input columns
 -E, --table-noextreme <columns>  don't count long text from the columns to column width
 -d, --table-noheadings           don't print header
 -m, --table-maxout               fill all available space
 -e, --table-header-repeat        repeat header for each page
 -H, --table-hide <columns>       don't print the columns
 -R, --table-right <columns>      right align text in these columns
 -T, --table-truncate <columns>   truncate text in the columns when necessary
 -W, --table-wrap <columns>       wrap text in the columns when necessary
 -L, --keep-empty-lines           don't ignore empty lines
 -J, --json                       use JSON output format for table

 -r, --tree <column>              column to use tree-like output for the table
 -i, --tree-id <column>           line ID to specify child-parent relation
 -p, --tree-parent <column>       parent to specify child-parent relation

 -c, --output-width <width>       width of output in number of characters
 -o, --output-separator <string>  columns separator for table output (default is two spaces)
 -s, --separator <string>         possible table delimiters
 -x, --fillrows                   fill rows before columns

 -h, --help                       display this help
 -V, --version                    display version

For more details see column(1).

It’s in the util-linux package, which, if I am remembering correctly, is included by default in basically any release outside of netboot and cloud images

Bash
 pacman -Qi util-linux
Name            : util-linux
Version         : 2.40.1-1
Description     : Miscellaneous system utilities for Linux
Architecture    : x86_64
URL             : https://github.com/util-linux/util-linux
Licenses        : BSD-2-Clause  BSD-3-Clause  BSD-4-Clause-UC  GPL-2.0-only
                  GPL-2.0-or-later  GPL-3.0-or-later  ISC  LGPL-2.1-or-later
                  LicenseRef-PublicDomain
Groups          : None
Provides        : rfkill  hardlink
Depends On      : util-linux-libs=2.40.1  coreutils  file  libmagic.so=1-64
                  glibc  libcap-ng  libxcrypt  libcrypt.so=2-64  ncurses
                  libncursesw.so=6-64  pam  readline  shadow  systemd-libs
                  libsystemd.so=0-64  libudev.so=1-64  zlib
Optional Deps   : words: default dictionary for look
Required By     : apr  arch-install-scripts  base  devtools  dracut  f2fs-tools
                  fakeroot  inxi  jfsutils  keycloak  libewf  nilfs-utils
                  ntfsprogs-ntfs3  nvme-cli  ostree  quickemu  reiserfsprogs
                  sbctl  systemd  zeromq
Optional For    : e2fsprogs  gzip  syslinux
Conflicts With  : rfkill  hardlink
Replaces        : rfkill  hardlink
Installed Size  : 14.47 MiB
Packager        : Christian Hesse <eworm@archlinux.org>
Build Date      : Mon 06 May 2024 12:22:23 PM PDT
Install Date    : Wed 08 May 2024 09:27:41 AM PDT
Install Reason  : Installed as a dependency for another package
Install Script  : No
Validated By    : Signature

this demonstrates why it’s included: It’s required by basically all of the setup infrastructure packages (makes sense). But I don’t really see column getting a lot of attention. Not sure why.

Here I used it to convert a list of installed flatpak containers to both JSON and YAML, using the column names from the flatpak’s output options (see flatpak list --help for available columns).

Start by making a list using the flatpak list --columns=$COLUMNS command:

Bash
# export comma-separated column headers, in desired order
export COLUMNS='name,application,version,branch,arch,origin,installation,ref'

# make a list of flatpaks with these columns
flatpak list --app --columns=$NAMES > all-flatpak-apps.list

# verify the list was created
cat all-flatpak-apps.list

# output (truncated):
Delta Chat	chat.delta.desktop	v1.44.1	stable	x86_64	flathub	user	chat.delta.desktop/x86_64/stable
Twilio Authy	com.authy.Authy	2.5.0	stable	x86_64	flathub	user	com.authy.Authy/x86_64/stable
Discord	com.discordapp.Discord	0.0.54	stable	x86_64	flathub	user	com.discordapp.Discord/x86_64/stable
GitButler	com.gitbutler.gitbutler	0.11.7	stable	x86_64	flathub	user	com.gitbutler.gitbutler/x86_64/stable
Drawing	com.github.maoschanz.drawing	1.0.2	stable	x86_64	flathub	user	com.github.maoschanz.drawing/x86_64/stable
Flatseal	com.github.tchx84.Flatseal	2.2.0	stable	x86_64	flathub	user	com.github.tchx84.Flatseal/x86_64/stable
Extension Manager	com.mattjakeman.ExtensionManager	0.5.1	stable	x86_64	flathub	user	com.mattjakeman.ExtensionManager/x86_64/stable
Yubico Authenticator	com.yubico.yubioath	7.0.0	stable	x86_64	flathub	user	com.yubico.yubioath/x86_64/stable
. . . 

It’s kinda hard to read like that, but I suppose if your font were small enough, or the terminal wide enough, the lines would stop wrapping and they’d be easier to read…

Then, take the list and convert it to json with a little column parsing:

Bash
# the flags are: 
# 1. create a table (-t), 
# 2. give it these headers (-N $comma,separated,headervals) 
# 3. limit it to the number we gave you (-l $integer)
# 4. make it json! (-J)
# 5. $INPUT_FILE > [$OUTPUT_FILE] 
column -t -d -N $NAMES -l 8 -J all-flatpak-apps.list > flatpak-apps.json

# column is a lot more forgiving of formatting than jq and yq

I haven’t had a lot of luck piping to column directly to make a parse-inception one-liner, which is why I am making two files (the flatpak.list and the .json file). I am sure there’s some people out there who could make this work, but I want to keep it on the less complicated side for demonstrative purposes, as well.

The output will be json format with the column parse by itself, but some of us like to pretty-print our json with jq , since the colors make it easier to read – especially when they’re not even pretty-printed:

Bash
column -t -d -N $NAMES -l 8 -J all-flatpak-apps.list | jq  

{
  "table": [
    {
      "name": "Delta",
      "application": "Chat",
      "version": "chat.delta.desktop",
      "branch": "v1.44.1",
      "arch": "stable",
      "origin": "x86_64",
      "installation": "flathub",
      "ref": "user\tchat.delta.desktop/x86_64/stable"
    },
    {
      "name": "Twilio",
      "application": "Authy",
      "version": "com.authy.Authy",
      "branch": "2.5.0",
      "arch": "stable",
      "origin": "x86_64",
      "installation": "flathub",
      "ref": "user\tcom.authy.Authy/x86_64/stable"
    },

    {
      "name": "UPnP",
      "application": "Router",
      "version": "Control",
      "branch": "org.upnproutercontrol.UPnPRouterControl",
      "arch": "0.3.4",
      "origin": "stable",
      "installation": "x86_64",
      "ref": "flathub\tuser\torg.upnproutercontrol.UPnPRouterControl/x86_64/stable"
    },
    {
      "name": "Wireshark",
      "application": "org.wireshark.Wireshark",
      "version": "4.2.5",
      "branch": "stable",
      "arch": "x86_64",
      "origin": "flathub",
      "installation": "user",
      "ref": "org.wireshark.Wireshark/x86_64/stable"
    },
    {
      "name": "ZAP",
      "application": "org.zaproxy.ZAP",
      "version": "2.15.0",
      "branch": "stable",
      "arch": "x86_64",
      "origin": "flathub",
      "installation": "user",
      "ref": "org.zaproxy.ZAP/x86_64/stable"
    }
  ]
}

And if you need yaml, the same thing works with yq:

Bash
# what NOT to do (if you want to be able to read it):

column -t -d -N $NAMES -l 8 -J all-flatpak-apps.list | yq
{"table": [{"name": "Delta", "application": "Chat", "version": "chat.delta.desktop", "branch": "v1.44.1", "arch": "stable", "origin": "x86_64", "installation": "flathub", "ref": "user\tchat.delta.desktop/x86_64/stable"}, {"name": "Twilio", "application": "Authy", . . . 

# sometimes yq requires being super-explicit:

# yq flags are: 
# 1. input file type (-p type)
# 2. output file type (-o type)
# 3. pretty-print (-P) 
column -t -d -N $NAMES -l 8 -J all-flatpak-apps.list | yq -p json -o yaml -P
table:
  - name: Delta
    application: Chat
    version: chat.delta.desktop
    branch: v1.44.1
    arch: stable
    origin: x86_64
    installation: flathub
    ref: "user\tchat.delta.desktop/x86_64/stable"
  - name: Twilio
    application: Authy
    version: com.authy.Authy
    branch: 2.5.0
    arch: stable
    origin: x86_64
    installation: flathub
    ref: "user\tcom.authy.Authy/x86_64/stable"
  - name: Discord
    application: com.discordapp.Discord
    version: 0.0.54
    branch: stable
    arch: x86_64
    origin: flathub
    installation: user
    ref: com.discordapp.Discord/x86_64/stable
. . . 

I’ve got the go-yq package, which has a much newer version of jq included with it, so if you’re seeing some differences in behavior or command-line arguments, that could be why – give it a shot if you’re having issues. (I do prefer the original jq, but I don’t really have a choice since it’s a dependency of devtools

Bash
# not a huge fan of gojq-bin, but go-yq is awesome...
 pacman -Qi go-yq
Name            : go-yq
Version         : 4.44.1-1
Description     : Portable command-line YAML processor
Architecture    : x86_64
URL             : https://github.com/mikefarah/yq
Licenses        : MIT
Groups          : None
Provides        : None
Depends On      : glibc
Optional Deps   : None
Required By     : None
Optional For    : None
Conflicts With  : yq
Replaces        : None
Installed Size  : 9.58 MiB
Packager        : Daniel M. Capella <polyzen@archlinux.org>
Build Date      : Sat 11 May 2024 08:14:21 PM PDT
Install Date    : Sat 11 May 2024 09:08:38 PM PDT
Install Reason  : Explicitly installed
Install Script  : No
Validated By    : Signature

And don’t forget, column should be able to format basically any time of list with repeating columns. I guess that’s all for now, enjoy!

ok nevermind, just one more thing: This has nothing to do with column, necessarily, but it’s all the output you can get when you choose json as your output. I’ll have to do another post about this, but it’s tangentially related.

Look at the output from kubernetes kubectl get pods -A :

Bash
kubectl get pods -A
NAMESPACE              NAME                                        READY   STATUS    RESTARTS       AGE
default                elasticsearch-coordinating-0                1/1     Running   0              91m
default                elasticsearch-coordinating-1                1/1     Running   0              91m
default                elasticsearch-data-0                        1/1     Running   0              91m
default                elasticsearch-data-1                        1/1     Running   0              91m
default                elasticsearch-ingest-0                      1/1     Running   0              91m
default                elasticsearch-ingest-1                      1/1     Running   0              91m
default                elasticsearch-master-0                      1/1     Running   0              91m
default                elasticsearch-master-1                      1/1     Running   0              91m
kube-system            coredns-7db6d8ff4d-x8nnb                    1/1     Running   1 (101m ago)   125m
kube-system            etcd-minikube                               1/1     Running   1 (101m ago)   125m
kube-system            kube-apiserver-minikube                     1/1     Running   1 (101m ago)   125m
kube-system            kube-controller-manager-minikube            1/1     Running   1 (101m ago)   125m
kube-system            kube-proxy-jbds4                            1/1     Running   1 (101m ago)   125m
kube-system            kube-scheduler-minikube                     1/1     Running   1 (101m ago)   125m
kube-system            metrics-server-c59844bb4-d8wnd              1/1     Running   1 (101m ago)   119m
kube-system            storage-provisioner                         1/1     Running   3 (101m ago)   125m
kubernetes-dashboard   dashboard-metrics-scraper-b5fc48f67-j5znq   1/1     Running   1 (101m ago)   121m
kubernetes-dashboard   kubernetes-dashboard-779776cb65-rh8s2       1/1     Running   2 (101m ago)   121m
portainer              portainer-7bf545c674-xnjjr                  1/1     Running   1 (101m ago)   119m
yakd-dashboard         yakd-dashboard-5ddbf7d777-mzqk2             1/1     Running   1 (101m ago)   119m

It’s not bad, but check out all the info you get when you add --output json:

Bash
# +`jq` to colorize the output (or `yq -p json -o json -P`)
kubectl get pods -A -o json | jq 
{
  "apiVersion": "v1",
  "items": [
    {
      "apiVersion": "v1",
      "kind": "Pod",
      "metadata": {
        "creationTimestamp": "2024-05-25T12:26:56Z",
        "generateName": "elasticsearch-coordinating-",
        "labels": {
          "app": "coordinating-only",
          "app.kubernetes.io/component": "coordinating-only",
          "app.kubernetes.io/instance": "elasticsearch",
          "app.kubernetes.io/managed-by": "Helm",
          "app.kubernetes.io/name": "elasticsearch",
          "app.kubernetes.io/version": "8.13.4",
          "apps.kubernetes.io/pod-index": "0",
          "controller-revision-hash": "elasticsearch-coordinating-64759546b",
          "helm.sh/chart": "elasticsearch-21.1.0",
          "statefulset.kubernetes.io/pod-name": "elasticsearch-coordinating-0"
        },
        "name": "elasticsearch-coordinating-0",
        "namespace": "default",
        "ownerReferences": [
          {
            "apiVersion": "apps/v1",
            "blockOwnerDeletion": true,
            "controller": true,
            "kind": "StatefulSet",
            "name": "elasticsearch-coordinating",
            "uid": "e403f4a1-3058-4830-8f4e-25323fa68be5"
          }
        ],
        "resourceVersion": "2737",
        "uid": "5c26ea5d-5ec7-40b1-946c-573651123552"
      },
      "spec": {
        "affinity": {},
        "automountServiceAccountToken": false,
        "containers": [
          {
            "env": [
              {
                "name": "MY_POD_NAME",
                "valueFrom": {
                  "fieldRef": {
                    "apiVersion": "v1",
                    "fieldPath": "metadata.name"
                  }
                }
              },
              {
                "name": "BITNAMI_DEBUG",
                "value": "false"
              },
              {
                "name": "ELASTICSEARCH_CLUSTER_NAME",
                "value": "elastic"
              },
              {
                "name": "ELASTICSEARCH_IS_DEDICATED_NODE",
                "value": "yes"
              },
              {
                "name": "ELASTICSEARCH_NODE_ROLES"
              },
              {
                "name": "ELASTICSEARCH_TRANSPORT_PORT_NUMBER",
                "value": "9300"
              },
              {
                "name": "ELASTICSEARCH_HTTP_PORT_NUMBER",
                "value": "9200"
              },
              {
                "name": "ELASTICSEARCH_CLUSTER_HOSTS",
                "value": "elasticsearch-master-hl.default.svc.cluster.local,elasticsearch-coordinating-hl.default.svc.cluster.local,elasticsearch-data-hl.default.svc.cluster.local,elasticsearch-ingest-hl.default.svc.cluster.local,"
              },
              {
                "name": "ELASTICSEARCH_TOTAL_NODES",
               . . . 

It’s nice that kubectl actually has a yaml output, since it’s a little easier to read than json, what with all the [{"punctuation": "on"},{"basically": "every}, {"word": true}], but most programs, if they even offer json, probably aren’t going to offer yaml – so to convert it, here’s the pipe string:

Bash
# adding `yq` colorizes output to `yaml` output, too
kubectl get pods -A -o json | yq -p json -o yaml -P  
apiVersion: v1
items:
  - apiVersion: v1
    kind: Pod
    metadata:
      creationTimestamp: "2024-05-25T12:26:56Z"
      generateName: elasticsearch-coordinating-
      labels:
        app: coordinating-only
        app.kubernetes.io/component: coordinating-only
        app.kubernetes.io/instance: elasticsearch
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/name: elasticsearch
        app.kubernetes.io/version: 8.13.4
        apps.kubernetes.io/pod-index: "0"
        controller-revision-hash: elasticsearch-coordinating-64759546b
        helm.sh/chart: elasticsearch-21.1.0
        statefulset.kubernetes.io/pod-name: elasticsearch-coordinating-0
      name: elasticsearch-coordinating-0
      namespace: default
      ownerReferences:
        - apiVersion: apps/v1
          blockOwnerDeletion: true
          controller: true
          kind: StatefulSet
          name: elasticsearch-coordinating
          uid: e403f4a1-3058-4830-8f4e-25323fa68be5
      resourceVersion: "2737"
      uid: 5c26ea5d-5ec7-40b1-946c-573651123552
    spec:
      affinity: {}
      automountServiceAccountToken: false
      containers:
        - env:
            - name: MY_POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: BITNAMI_DEBUG
              value: "false"
            - name: ELASTICSEARCH_CLUSTER_NAME
              value: elastic
            - name: ELASTICSEARCH_IS_DEDICATED_NODE
              value: "yes"
            - name: ELASTICSEARCH_NODE_ROLES
            - name: ELASTICSEARCH_TRANSPORT_PORT_NUMBER
              value: "9300"
            - name: ELASTICSEARCH_HTTP_PORT_NUMBER
              value: "9200"
            - name: ELASTICSEARCH_CLUSTER_HOSTS
              value: elasticsearch-master-hl.default.svc.cluster.local,elasticsearch-coordinating-hl.default.svc.cluster.local,elasticsearch-data-hl.default.svc.cluster.local,elasticsearch-ingest-hl.default.svc.cluster.local,
            - name: ELASTICSEARCH_TOTAL_NODES
              value: "4"
            - name: ELASTICSEARCH_CLUSTER_MASTER_HOSTS
              value: elasticsearch-master-0 elasticsearch-master-1
            - name: ELASTICSEARCH_MINIMUM_MASTER_NODES
              value: "2"
            - name: ELASTICSEARCH_ADVERTISED_HOSTNAME
              value: $(MY_POD_NAME).elasticsearch-coordinating-hl.default.svc.cluster.local
            - name: ELASTICSEARCH_HEAP_SIZE
              value: 128m
          image: docker.io/bitnami/elasticsearch:8.13.4-debian-12-r0
          imagePullPolicy: IfNotPresent
          livenessProbe:
            failureThreshold: 5
            initialDelaySeconds: 180
            periodSeconds: 10
            successThreshold: 1
            tcpSocket:
              port: rest-api
            timeoutSeconds: 5
          name: elasticsearch
          ports:
            - containerPort: 9200
              name: rest-api
              protocol: TCP
            - containerPort: 9300
              name: transport
              protocol: TCP
          readinessProbe:
            exec:
              command:
                - /opt/bitnami/scripts/elasticsearch/healthcheck.sh
            failureThreshold: 5
            initialDelaySeconds: 90
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 5
          resources:
            limits:
              cpu: 750m
              ephemeral-storage: 1Gi
              memory: 768Mi
            requests:
              cpu: 500m
              ephemeral-storage: 50Mi
              memory: 512Mi
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
            privileged: false
            readOnlyRootFilesystem: true
            runAsGroup: 1001
            runAsNonRoot: true
            runAsUser: 1001
            seLinuxOptions: {}
            seccompProfile:
              type: RuntimeDefault
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
            - mountPath: /tmp
              name: empty-dir
              subPath: tmp-dir
            - mountPath: /opt/bitnami/elasticsearch/config
              name: empty-dir
              subPath: app-conf-dir
              . . . 

It sure is nice to be able to read the stuff that comes out of the tools we’re using every day…

Using `obs` CLI controller with my new favorite mind-mapping software Obsidian – in a flatpak container

Obsidian.md mind map example

There’s always issues here or there trying to incorporate container apps with the desktop, due to their sandboxed nature and subsequent lack of out-of-the-box feature parity with their distro-packaged and supported counterparts.

This is not a post about how awesome Obsidian is, but if you haven’t heard of it, I recommend checking it out: https://obsidian.md/

Rather, it’s a response I just put on github issues for an obsidian CLI controller I have been using. It’s been pretty helpful, but the real gem is Obsidian itself, I just love it, right down to being able to export markmap files and having vim keymap settings.

I just switched to the flatpak version, though, because I noticed the package distributed through Arch’s extra repository was pulling electron on to my system as a dependency (I am trying to cordon nodejs off as much as possible to my adsf environment and away from my main system, because it’s caused the most intra-dependency issues out of any language I’ve dealt with, by far, but that’s too irritating to go into here).

The response ended up being so long, I figured I might as well make a blog post out of it. Here’s the original: https://github.com/Yakitrak/obsidian-cli/issues/20

To follow up, according to Obsidian manual on its URI interface, obs needs 3 things to interface with URI:

  1. .desktop file
  2. Exec= line pointing at executable binary
  3. %u to be a command-line argument

Separate pre-req: obs name and path should be different values

Bash
obs print-default
Default vault name:  Obsidian
Default vault path:  **$HOME/Documents/Obsidian**

The first thing I’d try is giving the included desktop file a lowercase %u

Bash
sed -i 's|%U|%u|g' $HOME/.local/share/flatpak/app/md.obsidian.Obsidian/x86_64/stable/active/export/share/applications/md.obsidian.Obsidian.desktop 

obs open your-vault-name

(md.obsidian.Obsidian should open even if completely closed)

If that doesn’t work, try these steps:

Copy the flatpak desktop file to $HOME/.local/share/applications
Name it obsidian.desktop
Point the Exec= line to flatpak executable,

Bash
Exec=$HOME/.local/share/flatpak/app/md.obsidian.Obsidian/current/active/export/bin/md.obsidian.Obsidian %u

run it with a lowercase %u which appears to be an error on both Linux package distributions I’ve tried so far

One might want to examine the wrapper included with the flatpak to see if there’s any other settings they want to incorporate for their setup if they are executing the binary directly. A lot of time these wrappers are outdated and/or unnecessary for their situation, but worth checking out: $HOME/.local/share/flatpak/app/md.obsidian.Obsidian/x86_64/stable/active/files/bin/obsidian.sh

#!/bin/sh

set -oue pipefail

EXTRA_ARGS=()

add_argument() {
    declare -i "$1"=${!1:-0}

    if [[ "${!1}" -eq 1 ]]; then
        EXTRA_ARGS+=(${@:2})
    fi
}

# Nvidia GPUs may need to disable GPU acceleration:
# flatpak override --user --env=OBSIDIAN_DISABLE_GPU=1 md.obsidian.Obsidian
add_argument OBSIDIAN_DISABLE_GPU       --disable-gpu
add_argument OBSIDIAN_ENABLE_AUTOSCROLL --enable-blink-features=MiddleClickAutoscroll

# Wayland support can be optionally enabled like so:
# flatpak override --user --socket=wayland md.obsidian.Obsidian

WL_DISPLAY="${WAYLAND_DISPLAY:-"wayland-0"}"
# Some compositors a real path a instead of a symlink for WAYLAND_DISPLAY:
# https://github.com/flathub/md.obsidian.Obsidian/issues/284
if [[ -e "${XDG_RUNTIME_DIR}/${WL_DISPLAY}" || -e "/${WL_DISPLAY}" ]]; then
    echo "Debug: Enabling Wayland backend"
    EXTRA_ARGS+=(
        --ozone-platform-hint=auto
    --enable-features=WaylandWindowDecorations
    )
    if [[ -c /dev/nvidia0 ]]; then
        echo "Debug: Detecting Nvidia GPU. disabling GPU sandbox."
        EXTRA_ARGS+=(
            --disable-gpu-sandbox
        )
    fi
fi

# The cache files created by Electron and Mesa can become incompatible when there's an upgrade to
# either and may cause Obsidian to launch with a blank screen:
# https://github.com/flathub/md.obsidian.Obsidian/issues/214
if [[ "${OBSIDIAN_CLEAN_CACHE}" -eq 1 ]]; then
    CACHE_DIRECTORIES=(
        "${XDG_CONFIG_HOME}/obsidian/GPUCache"
    )
    for CACHE_DIRECTORY in "${CACHE_DIRECTORIES[@]}"; do
        if [[ -d "${CACHE_DIRECTORY}" ]]; then
            echo "Deleting cache directory: ${CACHE_DIRECTORY}"
            rm -rf "${CACHE_DIRECTORY}"
        fi
    done
fi

echo "Debug: Will run Obsidian with the following arguments: ${EXTRA_ARGS[@]}"
echo "Debug: Additionally, user gave: $@"

export FLATPAK_ID="${FLATPAK_ID:-md.obsidian.Obsidian}"
export TMPDIR="${XDG_RUNTIME_DIR}/app/${FLATPAK_ID}"

# Discord RPC
for i in {0..9}; do
    test -S "$XDG_RUNTIME_DIR"/"discord-ipc-$i" || ln -sf {app/com.discordapp.Discord,"$XDG_RUNTIME_DIR"}/"discord-ipc-$i";
done

zypak-wrapper /app/obsidian $@ ${EXTRA_ARGS[@]}
****

Also don’t forget flatseal.

`rmw` – the trash-aware `rm` your CLI probably should’ve had by default

Hasn’t everybody deleted some stuff in the command line, wishing they had a way to get it back? Well, by default there are no do-overs. That’s where rmw comes in: rmw creates a trash can for your command line, so even after you delete some files (as long as you use it), you should be able to get them back within your specified period of time.

It’s a semi-compatible drop-in replacement for rm that should leave you feeling more at home than the alternatives – I checked out at least 7 of them, and this appeared to be the most developed and supported out of all the ones I tried. And while rmw doesn’t (yet) support viewing, deleting, and otherwise co-mingling files with your desktop trash, you can at least keep them all in the same place to avoid confusion.

For me, setup went something like this …

# Set your environment variables (ephemeral and persistent):

$ export RMW_CONF_DIR="$HOME/.config/rmw"
$ echo 'export RMW_CONF_DIR="$HOME/.config/rmw"' >> $HOME/.zshrc  # or .bashrc

# Create your configuration directory and go there:

$ mkdir -p $RMW_CONF_DIR
$ cd $RMW_CONF_DIR

# now this is a little weird - rmwrc has to be $CONFIG/rmwrc, but it keeps
two files for configuration in $RMW_CONF_DIR

Set up your config:

# auto-create the config with rmw:

$ rmw --config $RMW_CONF_DIR/rmwrc

The auto-created configuration has these defaults, but the Waste directory seemed a little redundant to me

$ cat rmwrc 

# rmw default waste directory, separate from the desktop trash
WASTE = $HOME/.local/share/Waste

# The directory used by the FreeDesktop.org Trash spec
# Note to macOS and Windows users: moving files to 'Desktop' trash
# doesn't work yet
# WASTE=$HOME/.local/share/Trash

# A folder can use the $UID variable.
# See the README or man page for details about using the 'removable' attribute
# WASTE=/mnt/flash/.Trash-$UID, removable

# How many days should items be allowed to stay in the waste
# directories before they are permanently deleted
#
# use '0' to disable purging (can be overridden by using --purge=N_DAYS)
#
expire_age = 0

# purge is allowed to run without the '-f' option. If you'd rather
# require the use of '-f', you may uncomment the line below.
#
# force_required

So I switched the commenting to use the desktop trash location instead of the new one rmw uses by default:

# Change configuration to reflect freedesktop (gnome, etc.) default trash location:

$ sed -i 's|# WASTE=$HOME/.local/share/Trash|WASTE=$HOME/.local/share/Trash|g' $RMW_CONF_DIR/rmwrc 

$ sed -i 's|WASTE = $HOME/.local/share/Waste|# WASTE = $HOME/.local/share/Waste|g' $RMW_CONF_DIR/rmwrc 

$ cat $RMW_CONF_DIR/rmwrc 
# rmw default waste directory, separate from the desktop trash
# WASTE = $HOME/.local/share/Waste

# The directory used by the FreeDesktop.org Trash spec
# Note to macOS and Windows users: moving files to 'Desktop' trash
# doesn't work yet
WASTE=$HOME/.local/share/Trash

Also, I’d like it to empty itself after a month so I can’t forget to do it myself:

# set 30 day retention policy:

$ sed -i 's|expire_age = 0|expire_age = 30|g' $RMW_CONF_DIR/rmwrc 

$ cat $RMW_CONF_DIR/rmwrc | grep expire_age

expire_age = 30

Lastly, I’d like to try it out in place of rm for a while and see how it goes…

alias rm=rmw 

Here’s the online manual: https://theimpossibleastronaut.com/rmw-website/
And the github repo: https://github.com/theimpossibleastronaut/rmw/
Hope it gets you out of a bind!