Outputting a list of variables to `json` or `yaml` using `column` and `goyq`

Quick little command line kung fu job with this parser called column

Bash
 column -h

Usage:
 column [options] [<file>...]

Columnate lists.

Options:
 -t, --table                      create a table
 -n, --table-name <name>          table name for JSON output
 -O, --table-order <columns>      specify order of output columns
 -C, --table-column <properties>  define column
 -N, --table-columns <names>      comma separated columns names
 -l, --table-columns-limit <num>  maximal number of input columns
 -E, --table-noextreme <columns>  don't count long text from the columns to column width
 -d, --table-noheadings           don't print header
 -m, --table-maxout               fill all available space
 -e, --table-header-repeat        repeat header for each page
 -H, --table-hide <columns>       don't print the columns
 -R, --table-right <columns>      right align text in these columns
 -T, --table-truncate <columns>   truncate text in the columns when necessary
 -W, --table-wrap <columns>       wrap text in the columns when necessary
 -L, --keep-empty-lines           don't ignore empty lines
 -J, --json                       use JSON output format for table

 -r, --tree <column>              column to use tree-like output for the table
 -i, --tree-id <column>           line ID to specify child-parent relation
 -p, --tree-parent <column>       parent to specify child-parent relation

 -c, --output-width <width>       width of output in number of characters
 -o, --output-separator <string>  columns separator for table output (default is two spaces)
 -s, --separator <string>         possible table delimiters
 -x, --fillrows                   fill rows before columns

 -h, --help                       display this help
 -V, --version                    display version

For more details see column(1).

It’s in the util-linux package, which, if I am remembering correctly, is included by default in basically any release outside of netboot and cloud images

Bash
 pacman -Qi util-linux
Name            : util-linux
Version         : 2.40.1-1
Description     : Miscellaneous system utilities for Linux
Architecture    : x86_64
URL             : https://github.com/util-linux/util-linux
Licenses        : BSD-2-Clause  BSD-3-Clause  BSD-4-Clause-UC  GPL-2.0-only
                  GPL-2.0-or-later  GPL-3.0-or-later  ISC  LGPL-2.1-or-later
                  LicenseRef-PublicDomain
Groups          : None
Provides        : rfkill  hardlink
Depends On      : util-linux-libs=2.40.1  coreutils  file  libmagic.so=1-64
                  glibc  libcap-ng  libxcrypt  libcrypt.so=2-64  ncurses
                  libncursesw.so=6-64  pam  readline  shadow  systemd-libs
                  libsystemd.so=0-64  libudev.so=1-64  zlib
Optional Deps   : words: default dictionary for look
Required By     : apr  arch-install-scripts  base  devtools  dracut  f2fs-tools
                  fakeroot  inxi  jfsutils  keycloak  libewf  nilfs-utils
                  ntfsprogs-ntfs3  nvme-cli  ostree  quickemu  reiserfsprogs
                  sbctl  systemd  zeromq
Optional For    : e2fsprogs  gzip  syslinux
Conflicts With  : rfkill  hardlink
Replaces        : rfkill  hardlink
Installed Size  : 14.47 MiB
Packager        : Christian Hesse <eworm@archlinux.org>
Build Date      : Mon 06 May 2024 12:22:23 PM PDT
Install Date    : Wed 08 May 2024 09:27:41 AM PDT
Install Reason  : Installed as a dependency for another package
Install Script  : No
Validated By    : Signature

this demonstrates why it’s included: It’s required by basically all of the setup infrastructure packages (makes sense). But I don’t really see column getting a lot of attention. Not sure why.

Here I used it to convert a list of installed flatpak containers to both JSON and YAML, using the column names from the flatpak’s output options (see flatpak list --help for available columns).

Start by making a list using the flatpak list --columns=$COLUMNS command:

Bash
# export comma-separated column headers, in desired order
export COLUMNS='name,application,version,branch,arch,origin,installation,ref'

# make a list of flatpaks with these columns
flatpak list --app --columns=$NAMES > all-flatpak-apps.list

# verify the list was created
cat all-flatpak-apps.list

# output (truncated):
Delta Chat	chat.delta.desktop	v1.44.1	stable	x86_64	flathub	user	chat.delta.desktop/x86_64/stable
Twilio Authy	com.authy.Authy	2.5.0	stable	x86_64	flathub	user	com.authy.Authy/x86_64/stable
Discord	com.discordapp.Discord	0.0.54	stable	x86_64	flathub	user	com.discordapp.Discord/x86_64/stable
GitButler	com.gitbutler.gitbutler	0.11.7	stable	x86_64	flathub	user	com.gitbutler.gitbutler/x86_64/stable
Drawing	com.github.maoschanz.drawing	1.0.2	stable	x86_64	flathub	user	com.github.maoschanz.drawing/x86_64/stable
Flatseal	com.github.tchx84.Flatseal	2.2.0	stable	x86_64	flathub	user	com.github.tchx84.Flatseal/x86_64/stable
Extension Manager	com.mattjakeman.ExtensionManager	0.5.1	stable	x86_64	flathub	user	com.mattjakeman.ExtensionManager/x86_64/stable
Yubico Authenticator	com.yubico.yubioath	7.0.0	stable	x86_64	flathub	user	com.yubico.yubioath/x86_64/stable
. . . 

It’s kinda hard to read like that, but I suppose if your font were small enough, or the terminal wide enough, the lines would stop wrapping and they’d be easier to read…

Then, take the list and convert it to json with a little column parsing:

Bash
# the flags are: 
# 1. create a table (-t), 
# 2. give it these headers (-N $comma,separated,headervals) 
# 3. limit it to the number we gave you (-l $integer)
# 4. make it json! (-J)
# 5. $INPUT_FILE > [$OUTPUT_FILE] 
column -t -d -N $NAMES -l 8 -J all-flatpak-apps.list > flatpak-apps.json

# column is a lot more forgiving of formatting than jq and yq

I haven’t had a lot of luck piping to column directly to make a parse-inception one-liner, which is why I am making two files (the flatpak.list and the .json file). I am sure there’s some people out there who could make this work, but I want to keep it on the less complicated side for demonstrative purposes, as well.

The output will be json format with the column parse by itself, but some of us like to pretty-print our json with jq , since the colors make it easier to read – especially when they’re not even pretty-printed:

Bash
column -t -d -N $NAMES -l 8 -J all-flatpak-apps.list | jq  

{
  "table": [
    {
      "name": "Delta",
      "application": "Chat",
      "version": "chat.delta.desktop",
      "branch": "v1.44.1",
      "arch": "stable",
      "origin": "x86_64",
      "installation": "flathub",
      "ref": "user\tchat.delta.desktop/x86_64/stable"
    },
    {
      "name": "Twilio",
      "application": "Authy",
      "version": "com.authy.Authy",
      "branch": "2.5.0",
      "arch": "stable",
      "origin": "x86_64",
      "installation": "flathub",
      "ref": "user\tcom.authy.Authy/x86_64/stable"
    },

    {
      "name": "UPnP",
      "application": "Router",
      "version": "Control",
      "branch": "org.upnproutercontrol.UPnPRouterControl",
      "arch": "0.3.4",
      "origin": "stable",
      "installation": "x86_64",
      "ref": "flathub\tuser\torg.upnproutercontrol.UPnPRouterControl/x86_64/stable"
    },
    {
      "name": "Wireshark",
      "application": "org.wireshark.Wireshark",
      "version": "4.2.5",
      "branch": "stable",
      "arch": "x86_64",
      "origin": "flathub",
      "installation": "user",
      "ref": "org.wireshark.Wireshark/x86_64/stable"
    },
    {
      "name": "ZAP",
      "application": "org.zaproxy.ZAP",
      "version": "2.15.0",
      "branch": "stable",
      "arch": "x86_64",
      "origin": "flathub",
      "installation": "user",
      "ref": "org.zaproxy.ZAP/x86_64/stable"
    }
  ]
}

And if you need yaml, the same thing works with yq:

Bash
# what NOT to do (if you want to be able to read it):

column -t -d -N $NAMES -l 8 -J all-flatpak-apps.list | yq
{"table": [{"name": "Delta", "application": "Chat", "version": "chat.delta.desktop", "branch": "v1.44.1", "arch": "stable", "origin": "x86_64", "installation": "flathub", "ref": "user\tchat.delta.desktop/x86_64/stable"}, {"name": "Twilio", "application": "Authy", . . . 

# sometimes yq requires being super-explicit:

# yq flags are: 
# 1. input file type (-p type)
# 2. output file type (-o type)
# 3. pretty-print (-P) 
column -t -d -N $NAMES -l 8 -J all-flatpak-apps.list | yq -p json -o yaml -P
table:
  - name: Delta
    application: Chat
    version: chat.delta.desktop
    branch: v1.44.1
    arch: stable
    origin: x86_64
    installation: flathub
    ref: "user\tchat.delta.desktop/x86_64/stable"
  - name: Twilio
    application: Authy
    version: com.authy.Authy
    branch: 2.5.0
    arch: stable
    origin: x86_64
    installation: flathub
    ref: "user\tcom.authy.Authy/x86_64/stable"
  - name: Discord
    application: com.discordapp.Discord
    version: 0.0.54
    branch: stable
    arch: x86_64
    origin: flathub
    installation: user
    ref: com.discordapp.Discord/x86_64/stable
. . . 

I’ve got the go-yq package, which has a much newer version of jq included with it, so if you’re seeing some differences in behavior or command-line arguments, that could be why – give it a shot if you’re having issues. (I do prefer the original jq, but I don’t really have a choice since it’s a dependency of devtools

Bash
# not a huge fan of gojq-bin, but go-yq is awesome...
 pacman -Qi go-yq
Name            : go-yq
Version         : 4.44.1-1
Description     : Portable command-line YAML processor
Architecture    : x86_64
URL             : https://github.com/mikefarah/yq
Licenses        : MIT
Groups          : None
Provides        : None
Depends On      : glibc
Optional Deps   : None
Required By     : None
Optional For    : None
Conflicts With  : yq
Replaces        : None
Installed Size  : 9.58 MiB
Packager        : Daniel M. Capella <polyzen@archlinux.org>
Build Date      : Sat 11 May 2024 08:14:21 PM PDT
Install Date    : Sat 11 May 2024 09:08:38 PM PDT
Install Reason  : Explicitly installed
Install Script  : No
Validated By    : Signature

And don’t forget, column should be able to format basically any time of list with repeating columns. I guess that’s all for now, enjoy!

ok nevermind, just one more thing: This has nothing to do with column, necessarily, but it’s all the output you can get when you choose json as your output. I’ll have to do another post about this, but it’s tangentially related.

Look at the output from kubernetes kubectl get pods -A :

Bash
kubectl get pods -A
NAMESPACE              NAME                                        READY   STATUS    RESTARTS       AGE
default                elasticsearch-coordinating-0                1/1     Running   0              91m
default                elasticsearch-coordinating-1                1/1     Running   0              91m
default                elasticsearch-data-0                        1/1     Running   0              91m
default                elasticsearch-data-1                        1/1     Running   0              91m
default                elasticsearch-ingest-0                      1/1     Running   0              91m
default                elasticsearch-ingest-1                      1/1     Running   0              91m
default                elasticsearch-master-0                      1/1     Running   0              91m
default                elasticsearch-master-1                      1/1     Running   0              91m
kube-system            coredns-7db6d8ff4d-x8nnb                    1/1     Running   1 (101m ago)   125m
kube-system            etcd-minikube                               1/1     Running   1 (101m ago)   125m
kube-system            kube-apiserver-minikube                     1/1     Running   1 (101m ago)   125m
kube-system            kube-controller-manager-minikube            1/1     Running   1 (101m ago)   125m
kube-system            kube-proxy-jbds4                            1/1     Running   1 (101m ago)   125m
kube-system            kube-scheduler-minikube                     1/1     Running   1 (101m ago)   125m
kube-system            metrics-server-c59844bb4-d8wnd              1/1     Running   1 (101m ago)   119m
kube-system            storage-provisioner                         1/1     Running   3 (101m ago)   125m
kubernetes-dashboard   dashboard-metrics-scraper-b5fc48f67-j5znq   1/1     Running   1 (101m ago)   121m
kubernetes-dashboard   kubernetes-dashboard-779776cb65-rh8s2       1/1     Running   2 (101m ago)   121m
portainer              portainer-7bf545c674-xnjjr                  1/1     Running   1 (101m ago)   119m
yakd-dashboard         yakd-dashboard-5ddbf7d777-mzqk2             1/1     Running   1 (101m ago)   119m

It’s not bad, but check out all the info you get when you add --output json:

Bash
# +`jq` to colorize the output (or `yq -p json -o json -P`)
kubectl get pods -A -o json | jq 
{
  "apiVersion": "v1",
  "items": [
    {
      "apiVersion": "v1",
      "kind": "Pod",
      "metadata": {
        "creationTimestamp": "2024-05-25T12:26:56Z",
        "generateName": "elasticsearch-coordinating-",
        "labels": {
          "app": "coordinating-only",
          "app.kubernetes.io/component": "coordinating-only",
          "app.kubernetes.io/instance": "elasticsearch",
          "app.kubernetes.io/managed-by": "Helm",
          "app.kubernetes.io/name": "elasticsearch",
          "app.kubernetes.io/version": "8.13.4",
          "apps.kubernetes.io/pod-index": "0",
          "controller-revision-hash": "elasticsearch-coordinating-64759546b",
          "helm.sh/chart": "elasticsearch-21.1.0",
          "statefulset.kubernetes.io/pod-name": "elasticsearch-coordinating-0"
        },
        "name": "elasticsearch-coordinating-0",
        "namespace": "default",
        "ownerReferences": [
          {
            "apiVersion": "apps/v1",
            "blockOwnerDeletion": true,
            "controller": true,
            "kind": "StatefulSet",
            "name": "elasticsearch-coordinating",
            "uid": "e403f4a1-3058-4830-8f4e-25323fa68be5"
          }
        ],
        "resourceVersion": "2737",
        "uid": "5c26ea5d-5ec7-40b1-946c-573651123552"
      },
      "spec": {
        "affinity": {},
        "automountServiceAccountToken": false,
        "containers": [
          {
            "env": [
              {
                "name": "MY_POD_NAME",
                "valueFrom": {
                  "fieldRef": {
                    "apiVersion": "v1",
                    "fieldPath": "metadata.name"
                  }
                }
              },
              {
                "name": "BITNAMI_DEBUG",
                "value": "false"
              },
              {
                "name": "ELASTICSEARCH_CLUSTER_NAME",
                "value": "elastic"
              },
              {
                "name": "ELASTICSEARCH_IS_DEDICATED_NODE",
                "value": "yes"
              },
              {
                "name": "ELASTICSEARCH_NODE_ROLES"
              },
              {
                "name": "ELASTICSEARCH_TRANSPORT_PORT_NUMBER",
                "value": "9300"
              },
              {
                "name": "ELASTICSEARCH_HTTP_PORT_NUMBER",
                "value": "9200"
              },
              {
                "name": "ELASTICSEARCH_CLUSTER_HOSTS",
                "value": "elasticsearch-master-hl.default.svc.cluster.local,elasticsearch-coordinating-hl.default.svc.cluster.local,elasticsearch-data-hl.default.svc.cluster.local,elasticsearch-ingest-hl.default.svc.cluster.local,"
              },
              {
                "name": "ELASTICSEARCH_TOTAL_NODES",
               . . . 

It’s nice that kubectl actually has a yaml output, since it’s a little easier to read than json, what with all the [{"punctuation": "on"},{"basically": "every}, {"word": true}], but most programs, if they even offer json, probably aren’t going to offer yaml – so to convert it, here’s the pipe string:

Bash
# adding `yq` colorizes output to `yaml` output, too
kubectl get pods -A -o json | yq -p json -o yaml -P  
apiVersion: v1
items:
  - apiVersion: v1
    kind: Pod
    metadata:
      creationTimestamp: "2024-05-25T12:26:56Z"
      generateName: elasticsearch-coordinating-
      labels:
        app: coordinating-only
        app.kubernetes.io/component: coordinating-only
        app.kubernetes.io/instance: elasticsearch
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/name: elasticsearch
        app.kubernetes.io/version: 8.13.4
        apps.kubernetes.io/pod-index: "0"
        controller-revision-hash: elasticsearch-coordinating-64759546b
        helm.sh/chart: elasticsearch-21.1.0
        statefulset.kubernetes.io/pod-name: elasticsearch-coordinating-0
      name: elasticsearch-coordinating-0
      namespace: default
      ownerReferences:
        - apiVersion: apps/v1
          blockOwnerDeletion: true
          controller: true
          kind: StatefulSet
          name: elasticsearch-coordinating
          uid: e403f4a1-3058-4830-8f4e-25323fa68be5
      resourceVersion: "2737"
      uid: 5c26ea5d-5ec7-40b1-946c-573651123552
    spec:
      affinity: {}
      automountServiceAccountToken: false
      containers:
        - env:
            - name: MY_POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: BITNAMI_DEBUG
              value: "false"
            - name: ELASTICSEARCH_CLUSTER_NAME
              value: elastic
            - name: ELASTICSEARCH_IS_DEDICATED_NODE
              value: "yes"
            - name: ELASTICSEARCH_NODE_ROLES
            - name: ELASTICSEARCH_TRANSPORT_PORT_NUMBER
              value: "9300"
            - name: ELASTICSEARCH_HTTP_PORT_NUMBER
              value: "9200"
            - name: ELASTICSEARCH_CLUSTER_HOSTS
              value: elasticsearch-master-hl.default.svc.cluster.local,elasticsearch-coordinating-hl.default.svc.cluster.local,elasticsearch-data-hl.default.svc.cluster.local,elasticsearch-ingest-hl.default.svc.cluster.local,
            - name: ELASTICSEARCH_TOTAL_NODES
              value: "4"
            - name: ELASTICSEARCH_CLUSTER_MASTER_HOSTS
              value: elasticsearch-master-0 elasticsearch-master-1
            - name: ELASTICSEARCH_MINIMUM_MASTER_NODES
              value: "2"
            - name: ELASTICSEARCH_ADVERTISED_HOSTNAME
              value: $(MY_POD_NAME).elasticsearch-coordinating-hl.default.svc.cluster.local
            - name: ELASTICSEARCH_HEAP_SIZE
              value: 128m
          image: docker.io/bitnami/elasticsearch:8.13.4-debian-12-r0
          imagePullPolicy: IfNotPresent
          livenessProbe:
            failureThreshold: 5
            initialDelaySeconds: 180
            periodSeconds: 10
            successThreshold: 1
            tcpSocket:
              port: rest-api
            timeoutSeconds: 5
          name: elasticsearch
          ports:
            - containerPort: 9200
              name: rest-api
              protocol: TCP
            - containerPort: 9300
              name: transport
              protocol: TCP
          readinessProbe:
            exec:
              command:
                - /opt/bitnami/scripts/elasticsearch/healthcheck.sh
            failureThreshold: 5
            initialDelaySeconds: 90
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 5
          resources:
            limits:
              cpu: 750m
              ephemeral-storage: 1Gi
              memory: 768Mi
            requests:
              cpu: 500m
              ephemeral-storage: 50Mi
              memory: 512Mi
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
            privileged: false
            readOnlyRootFilesystem: true
            runAsGroup: 1001
            runAsNonRoot: true
            runAsUser: 1001
            seLinuxOptions: {}
            seccompProfile:
              type: RuntimeDefault
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
            - mountPath: /tmp
              name: empty-dir
              subPath: tmp-dir
            - mountPath: /opt/bitnami/elasticsearch/config
              name: empty-dir
              subPath: app-conf-dir
              . . . 

It sure is nice to be able to read the stuff that comes out of the tools we’re using every day…

Using `obs` CLI controller with my new favorite mind-mapping software Obsidian – in a flatpak container

Obsidian.md mind map example

There’s always issues here or there trying to incorporate container apps with the desktop, due to their sandboxed nature and subsequent lack of out-of-the-box feature parity with their distro-packaged and supported counterparts.

This is not a post about how awesome Obsidian is, but if you haven’t heard of it, I recommend checking it out: https://obsidian.md/

Rather, it’s a response I just put on github issues for an obsidian CLI controller I have been using. It’s been pretty helpful, but the real gem is Obsidian itself, I just love it, right down to being able to export markmap files and having vim keymap settings.

I just switched to the flatpak version, though, because I noticed the package distributed through Arch’s extra repository was pulling electron on to my system as a dependency (I am trying to cordon nodejs off as much as possible to my adsf environment and away from my main system, because it’s caused the most intra-dependency issues out of any language I’ve dealt with, by far, but that’s too irritating to go into here).

The response ended up being so long, I figured I might as well make a blog post out of it. Here’s the original: https://github.com/Yakitrak/obsidian-cli/issues/20

To follow up, according to Obsidian manual on its URI interface, obs needs 3 things to interface with URI:

  1. .desktop file
  2. Exec= line pointing at executable binary
  3. %u to be a command-line argument

Separate pre-req: obs name and path should be different values

Bash
obs print-default
Default vault name:  Obsidian
Default vault path:  **$HOME/Documents/Obsidian**

The first thing I’d try is giving the included desktop file a lowercase %u

Bash
sed -i 's|%U|%u|g' $HOME/.local/share/flatpak/app/md.obsidian.Obsidian/x86_64/stable/active/export/share/applications/md.obsidian.Obsidian.desktop 

obs open your-vault-name

(md.obsidian.Obsidian should open even if completely closed)

If that doesn’t work, try these steps:

Copy the flatpak desktop file to $HOME/.local/share/applications
Name it obsidian.desktop
Point the Exec= line to flatpak executable,

Bash
Exec=$HOME/.local/share/flatpak/app/md.obsidian.Obsidian/current/active/export/bin/md.obsidian.Obsidian %u

run it with a lowercase %u which appears to be an error on both Linux package distributions I’ve tried so far

One might want to examine the wrapper included with the flatpak to see if there’s any other settings they want to incorporate for their setup if they are executing the binary directly. A lot of time these wrappers are outdated and/or unnecessary for their situation, but worth checking out: $HOME/.local/share/flatpak/app/md.obsidian.Obsidian/x86_64/stable/active/files/bin/obsidian.sh

#!/bin/sh

set -oue pipefail

EXTRA_ARGS=()

add_argument() {
    declare -i "$1"=${!1:-0}

    if [[ "${!1}" -eq 1 ]]; then
        EXTRA_ARGS+=(${@:2})
    fi
}

# Nvidia GPUs may need to disable GPU acceleration:
# flatpak override --user --env=OBSIDIAN_DISABLE_GPU=1 md.obsidian.Obsidian
add_argument OBSIDIAN_DISABLE_GPU       --disable-gpu
add_argument OBSIDIAN_ENABLE_AUTOSCROLL --enable-blink-features=MiddleClickAutoscroll

# Wayland support can be optionally enabled like so:
# flatpak override --user --socket=wayland md.obsidian.Obsidian

WL_DISPLAY="${WAYLAND_DISPLAY:-"wayland-0"}"
# Some compositors a real path a instead of a symlink for WAYLAND_DISPLAY:
# https://github.com/flathub/md.obsidian.Obsidian/issues/284
if [[ -e "${XDG_RUNTIME_DIR}/${WL_DISPLAY}" || -e "/${WL_DISPLAY}" ]]; then
    echo "Debug: Enabling Wayland backend"
    EXTRA_ARGS+=(
        --ozone-platform-hint=auto
    --enable-features=WaylandWindowDecorations
    )
    if [[ -c /dev/nvidia0 ]]; then
        echo "Debug: Detecting Nvidia GPU. disabling GPU sandbox."
        EXTRA_ARGS+=(
            --disable-gpu-sandbox
        )
    fi
fi

# The cache files created by Electron and Mesa can become incompatible when there's an upgrade to
# either and may cause Obsidian to launch with a blank screen:
# https://github.com/flathub/md.obsidian.Obsidian/issues/214
if [[ "${OBSIDIAN_CLEAN_CACHE}" -eq 1 ]]; then
    CACHE_DIRECTORIES=(
        "${XDG_CONFIG_HOME}/obsidian/GPUCache"
    )
    for CACHE_DIRECTORY in "${CACHE_DIRECTORIES[@]}"; do
        if [[ -d "${CACHE_DIRECTORY}" ]]; then
            echo "Deleting cache directory: ${CACHE_DIRECTORY}"
            rm -rf "${CACHE_DIRECTORY}"
        fi
    done
fi

echo "Debug: Will run Obsidian with the following arguments: ${EXTRA_ARGS[@]}"
echo "Debug: Additionally, user gave: $@"

export FLATPAK_ID="${FLATPAK_ID:-md.obsidian.Obsidian}"
export TMPDIR="${XDG_RUNTIME_DIR}/app/${FLATPAK_ID}"

# Discord RPC
for i in {0..9}; do
    test -S "$XDG_RUNTIME_DIR"/"discord-ipc-$i" || ln -sf {app/com.discordapp.Discord,"$XDG_RUNTIME_DIR"}/"discord-ipc-$i";
done

zypak-wrapper /app/obsidian $@ ${EXTRA_ARGS[@]}
****

Also don’t forget flatseal.

`rmw` – the trash-aware `rm` your CLI probably should’ve had by default

Hasn’t everybody deleted some stuff in the command line, wishing they had a way to get it back? Well, by default there are no do-overs. That’s where rmw comes in: rmw creates a trash can for your command line, so even after you delete some files (as long as you use it), you should be able to get them back within your specified period of time.

It’s a semi-compatible drop-in replacement for rm that should leave you feeling more at home than the alternatives – I checked out at least 7 of them, and this appeared to be the most developed and supported out of all the ones I tried. And while rmw doesn’t (yet) support viewing, deleting, and otherwise co-mingling files with your desktop trash, you can at least keep them all in the same place to avoid confusion.

For me, setup went something like this …

# Set your environment variables (ephemeral and persistent):

$ export RMW_CONF_DIR="$HOME/.config/rmw"
$ echo 'export RMW_CONF_DIR="$HOME/.config/rmw"' >> $HOME/.zshrc  # or .bashrc

# Create your configuration directory and go there:

$ mkdir -p $RMW_CONF_DIR
$ cd $RMW_CONF_DIR

# now this is a little weird - rmwrc has to be $CONFIG/rmwrc, but it keeps
two files for configuration in $RMW_CONF_DIR

Set up your config:

# auto-create the config with rmw:

$ rmw --config $RMW_CONF_DIR/rmwrc

The auto-created configuration has these defaults, but the Waste directory seemed a little redundant to me

$ cat rmwrc 

# rmw default waste directory, separate from the desktop trash
WASTE = $HOME/.local/share/Waste

# The directory used by the FreeDesktop.org Trash spec
# Note to macOS and Windows users: moving files to 'Desktop' trash
# doesn't work yet
# WASTE=$HOME/.local/share/Trash

# A folder can use the $UID variable.
# See the README or man page for details about using the 'removable' attribute
# WASTE=/mnt/flash/.Trash-$UID, removable

# How many days should items be allowed to stay in the waste
# directories before they are permanently deleted
#
# use '0' to disable purging (can be overridden by using --purge=N_DAYS)
#
expire_age = 0

# purge is allowed to run without the '-f' option. If you'd rather
# require the use of '-f', you may uncomment the line below.
#
# force_required

So I switched the commenting to use the desktop trash location instead of the new one rmw uses by default:

# Change configuration to reflect freedesktop (gnome, etc.) default trash location:

$ sed -i 's|# WASTE=$HOME/.local/share/Trash|WASTE=$HOME/.local/share/Trash|g' $RMW_CONF_DIR/rmwrc 

$ sed -i 's|WASTE = $HOME/.local/share/Waste|# WASTE = $HOME/.local/share/Waste|g' $RMW_CONF_DIR/rmwrc 

$ cat $RMW_CONF_DIR/rmwrc 
# rmw default waste directory, separate from the desktop trash
# WASTE = $HOME/.local/share/Waste

# The directory used by the FreeDesktop.org Trash spec
# Note to macOS and Windows users: moving files to 'Desktop' trash
# doesn't work yet
WASTE=$HOME/.local/share/Trash

Also, I’d like it to empty itself after a month so I can’t forget to do it myself:

# set 30 day retention policy:

$ sed -i 's|expire_age = 0|expire_age = 30|g' $RMW_CONF_DIR/rmwrc 

$ cat $RMW_CONF_DIR/rmwrc | grep expire_age

expire_age = 30

Lastly, I’d like to try it out in place of rm for a while and see how it goes…

alias rm=rmw 

Here’s the online manual: https://theimpossibleastronaut.com/rmw-website/
And the github repo: https://github.com/theimpossibleastronaut/rmw/
Hope it gets you out of a bind!

Bring Single-Meta-Keypress Overview Behavior to KDE6 (just like Gnome!) – and a bit on KDE configuration files…

I missed KDE… best of both worlds?

Like a lot of things in the tech world these days, it all started with a short question on Reddit:

Keyboard shortcut to search programs in overview like Gnome?
byu/AveryFreeman inkde

Teal;Deer:

I found out the meta-only modifier is one thing that needed to be set in $HOME/.config/kwinrc, but also the ExposeAll=Meta config in $HOME/.config/kglobalshortcutsrc – so there’s two places where it needs to be set. Also, you can add /KWin reconfigure to the end and it should start working immediately (reloads kwin config). But hey, thanks so much for pointing me in the right direction!

(my synoposis in the subred)

The biggest impediment to getting the Gnome-like super key behavior in KDE is getting KDE to interpret a single keypress as a keyboard shortcut. If you look at it critically, you can see there’s a lot going on in that single keypress in Gnome: In Gnome, a single keypress gives you both the open windows overview, and a search bar, and even has additional menus accessible from repeating the action (double-pressing super).

KDE isn’t really able to allow single-keypress shortcuts by default, but there’s a spot they’ve carved out where users can add the behavior if they really want to (I get the sense it’s caveat emptor). I’ve never been particularly interested in sticking to pre-defined conventions, so I set out to find out how we could make super behave like it does in Gnome. It couldn’t be that hard, right?

I wanted to know what was going on under the hood, so I did some digging. The most complete reference on the KDE configuration files I’ve found so far is here: https://userbase.kde.org/KDE_System_Administration/Configuration_Files

There’s a whole bunch of really great examples here: https://userbase.kde.org/Plasma/Tips

There’s some man files that are also helpful here (I don’t think this particular spec has changed from KDE 5 to 6):
https://linuxcommandlibrary.com/man/kreadconfig5
https://linuxcommandlibrary.com/man/kwriteconfig5

I found the shortcuts in KDE6 are located in a file named /home/$USER/.config/kglobalshortcutsrc. They’re grouped by category with the names in brackets, like [ActivityManager], [kmix], etc.

So, related to my question, when I opened this file, under [kwin] I found this:

# $HOME/.config/kglobalshortcutsrc

[kwin]
Activate Window Demanding Attention=Meta+Ctrl+A,Meta+Ctrl+A,Activate Window Demanding Attention
Cycle Overview=Meta+Tab,none,Cycle through Overview and Grid View
Cycle Overview Opposite=none,none,Cycle through Grid View and Overview
. . . 
ExposeAll=Meta,Ctrl+F10\tLaunch (C),Toggle Present Windows (All desktops)
. . .Code language: PHP (php)

And when I ran the following:

kreadconfig6 --file kglobalshortcutsrc --group kwin --key ExposeAll

I get this response in my terminal:


. . .  
Meta,Ctrl+F10   Launch (C),Toggle Present Windows (All desktops)

So in reference to the kreadconfig6 syntax, the key is the kwin action named ExposeAll. The group should be the category of actions, [kwin]. The harder part is keeping the description in there when writing from the command line, because it’s tab-separated, \t to the right of the same line. But anyway, I am assuming it would be:

kwriteconfig6 --file kglobalshortcutsrc --group kwin --key ExposeAll Meta"

Or if you needed the descriptor to be in the same line as the value (like in the text file), it would be:

kwriteconfig6 --file kglobalshortcutsrc --group kwin --key ExposeAll Meta\tLaunch (C),Toggle Present Windows (All desktops)"

I’m not sure if kwriteconfig6 can sort out the tab-separated sections on its own, but I imagine it probably can. I have shied away from editing these config files in a text editor just in case there’s some specificity they have regarding tab and whitespace I might mess up, but I imagine it could be one option – if you try it, be sure to use a tabspace v whitespace identifier plugin for vim

Thankfully normal people can add this behavior from the GUI under Ksettings -> Shortcuts -> Kwin, too – so that’s reassuring!

A bigger issue might be specifying that you can use only a modifier key as a shortcut (e.g. meta, alt, shift, etc.) which is what the answer to my reddit question addressed (although I don’t think they were aware, exactly).

Therefore, settings need to be created or modified in two separate places, once for the key behavior, ExposeAll (previous example), and another for allowing the key to behave like a shortcut. As far as I can tell, allowing Meta on its own to ‘be’ a shortcut does need to be invoked at the command line. Here’s the process:

Meta (the keypress) is the key value in the pair, analogous to the example with ExposeAll. It will end up in the config file for kwin:

# $HOME/.config/kwinrc 

[ModifierOnlyShortcuts]
Meta=org.kde.kglobalaccel,/component/kwin,org.kde.kglobalaccel.Component,invokeShortcut,OverviewCode language: PHP (php)

Coincidentally, there the syntax to add it is provided at the end of the kwriteconfig5 man file I linked above. At the very end, it gives the example:

kwriteconfig5 --file ~/.config/kwinrc --group ModifierOnlyShortcuts --key Meta "org.kde.kglobalaccel,/component/krunner_desktop,org.kde.kglobalaccel.Component,invokeShortcut,_launch"Code language: JavaScript (javascript)

… which is for launching krunner if you hit meta by itself.

If you’d want the config to be reloaded (to start working immediately) you can add /KWin reconfigure to the end.

So I’m assuming if you already have the ExposeAll behavior in your kglobalshortcutsrc and you run the above argument, it should combine krunner with ExposeAll under the Meta keypress, which is extremely Gnome-like behavior (and pretty exciting!)

TBH I was trying all sorts of stuff when I was doing this and am not sure the exact point at which I got the desired behavior, it might require re-logging in or something like that, too. But these are the two elements you’d need in which to get the Gnome-like action from smashing your Meta key.

One last thing – you may have noticed there’s a more conventional keyboard shortcut Ctrl+F10 for ExposeAll in $HOME/.config/kglobalshortcutsrc – one that actually combines two keys, like normal (gee, seems quaint!)

[kwin]
. . . 
ExposeAll=Meta,Ctrl+F10\tLaunch (C),Toggle Present Windows (All desktops)
. . . 

That’s because the userbase.kde.org site I linked at the top says a single-key shortcut (Meta) might not work if there isn’t a key combination shortcut also configured for the same action (Ctrl+F10)

Lastly, I threw together a little one-liner that dumps all the configs into a single text file for analyzing. It makes them a little easier to sift and not have to open each one individually. Just run it from the $HOME/. folder:

for i in $(ls -a1 .config/k*); do echo "File: $(pwd)/$i"; echo ' '; cat $i; echo ' '; echo '--- end ---';  echo ' '; done > ./kde-config-files.txtCode language: PHP (php)

I’ll probably put all my KDE6 configs in a git repo one of these days, but for now, I’m off to study ORM development with Python and SQL… Cheers!

Name Your Distro: Flatpaks irritatingly don’t follow desktop theme (written specific to KDE / Plasma-Desktop 6)

Brings back memories of being read the Ugly Duckling

This is not an issue with any Linux distro specifically, but with all distros in general. If anyone out there’s had this issue, I’m sure you’ve shared my pain. I’d struggled with getting flatpaks to follow the (dark) system theme I set for a while, and I finally just got it fixed, so I thought I’d run through the process real quick:

  1. make sure you’re setting a theme that has a flatpak version available (I chose adw-gtk3{,-dark})
  2. install the flatpak versions: as of writing, gtk3 flatpak themes are the only ones you need to download, but if your theme doesn’t include a gtk4 variant, YMMV
  3. install the gtk themes for your desktop environment and ensure they are in the proper location
  4. enable gtk themes in gsettings
  5. A couple last-ditch effort recommendations if 1-4 doesn’t work

Here’s the steps expanded:

Choose a theme you can get in a desktop version – a good place to see if you can find the theme you want is gnome-look.org. Make sure if you use both user and system flatpaks you have a theme for both (I personally only use userspace flatpaks from flathub to avoid complications) Here’s the current list of flatpak theme runtimes as of writing:

❯ clear && flatpak search --user --columns=app theme | grep org.gtk | sort -n
org.gtk.Gtk3theme.Adapta
org.gtk.Gtk3theme.Adapta-Brila
org.gtk.Gtk3theme.Adapta-Brila-Eta
org.gtk.Gtk3theme.Adapta-Eta
org.gtk.Gtk3theme.Adapta-Nokto
org.gtk.Gtk3theme.Adapta-Nokto-Eta
org.gtk.Gtk3theme.Adementary
org.gtk.Gtk3theme.Adwaita-dark
org.gtk.Gtk3theme.adw-gtk3
org.gtk.Gtk3theme.adw-gtk3-dark
org.gtk.Gtk3theme.Akwa org.gtk.Gtk3theme.Akwa-dark org.gtk.Gtk3theme.Akwa-light org.gtk.Gtk3theme.Ambiance org.gtk.Gtk3theme.Arc org.gtk.Gtk3theme.Arc-Dark org.gtk.Gtk3theme.Arc-Darker org.gtk.Gtk3theme.Arc-Darker-solid org.gtk.Gtk3theme.Arc-Dark-solid org.gtk.Gtk3theme.Arc-Lighter org.gtk.Gtk3theme.Arc-Lighter-solid org.gtk.Gtk3theme.Arc-solid org.gtk.Gtk3theme.Breeze org.gtk.Gtk3theme.Catppuccin-blue org.gtk.Gtk3theme.Catppuccin-green org.gtk.Gtk3theme.Catppuccin-orange org.gtk.Gtk3theme.Catppuccin-pink org.gtk.Gtk3theme.Catppuccin-purple org.gtk.Gtk3theme.Catppuccin-red org.gtk.Gtk3theme.Catppuccin-teal org.gtk.Gtk3theme.Catppuccin-yellow org.gtk.Gtk3theme.Chicago95 org.gtk.Gtk3theme.Communitheme org.gtk.Gtk3theme.CrosAdapta org.gtk.Gtk3theme.deepin org.gtk.Gtk3theme.deepin-dark org.gtk.Gtk3theme.elementary org.gtk.Gtk3theme.Flat-Remix-GTK-Blue org.gtk.Gtk3theme.Flat-Remix-GTK-Blue-Dark org.gtk.Gtk3theme.Flat-Remix-GTK-Blue-Darker org.gtk.Gtk3theme.Flat-Remix-GTK-Blue-Darker-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Blue-Darkest org.gtk.Gtk3theme.Flat-Remix-GTK-Blue-Darkest-NoBorder org.gtk.Gtk3theme.Flat-Remix-GTK-Blue-Darkest-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Blue-Darkest-Solid-NoBorder org.gtk.Gtk3theme.Flat-Remix-GTK-Blue-Dark-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Blue-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Green org.gtk.Gtk3theme.Flat-Remix-GTK-Green-Dark org.gtk.Gtk3theme.Flat-Remix-GTK-Green-Darker org.gtk.Gtk3theme.Flat-Remix-GTK-Green-Darker-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Green-Darkest org.gtk.Gtk3theme.Flat-Remix-GTK-Green-Darkest-NoBorder org.gtk.Gtk3theme.Flat-Remix-GTK-Green-Darkest-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Green-Darkest-Solid-NoBorder org.gtk.Gtk3theme.Flat-Remix-GTK-Green-Dark-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Green-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Red org.gtk.Gtk3theme.Flat-Remix-GTK-Red-Dark org.gtk.Gtk3theme.Flat-Remix-GTK-Red-Darker org.gtk.Gtk3theme.Flat-Remix-GTK-Red-Darker-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Red-Darkest org.gtk.Gtk3theme.Flat-Remix-GTK-Red-Darkest-NoBorder org.gtk.Gtk3theme.Flat-Remix-GTK-Red-Darkest-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Red-Darkest-Solid-NoBorder org.gtk.Gtk3theme.Flat-Remix-GTK-Red-Dark-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Red-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Yellow org.gtk.Gtk3theme.Flat-Remix-GTK-Yellow-Dark org.gtk.Gtk3theme.Flat-Remix-GTK-Yellow-Darker org.gtk.Gtk3theme.Flat-Remix-GTK-Yellow-Darker-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Yellow-Darkest org.gtk.Gtk3theme.Flat-Remix-GTK-Yellow-Darkest-NoBorder org.gtk.Gtk3theme.Flat-Remix-GTK-Yellow-Darkest-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Yellow-Darkest-Solid-NoBorder org.gtk.Gtk3theme.Flat-Remix-GTK-Yellow-Dark-Solid org.gtk.Gtk3theme.Flat-Remix-GTK-Yellow-Solid org.gtk.Gtk3theme.Greybird org.gtk.Gtk3theme.Greybird org.gtk.Gtk3theme.Greybird-dark org.gtk.Gtk3theme.Greybird-Geeko-Dark org.gtk.Gtk3theme.Greybird-Geeko-Light org.gtk.Gtk3theme.Helium org.gtk.Gtk3theme.Helium-dark org.gtk.Gtk3theme.High-Sierra org.gtk.Gtk3theme.High-Sierra-Dark org.gtk.Gtk3theme.Jade-1 org.gtk.Gtk3theme.Jade-1-Amber org.gtk.Gtk3theme.Jade-1-Aqua org.gtk.Gtk3theme.Jade-1-Blue org.gtk.Gtk3theme.Jade-1-Gray org.gtk.Gtk3theme.Jade-1-Green org.gtk.Gtk3theme.Jade-1-Indigo org.gtk.Gtk3theme.Jade-1-Purple org.gtk.Gtk3theme.Jade-1-Red org.gtk.Gtk3theme.Jade-1-Teal org.gtk.Gtk3theme.Lounge org.gtk.Gtk3theme.Lounge-compact org.gtk.Gtk3theme.Lounge-night org.gtk.Gtk3theme.Lounge-night-compact org.gtk.Gtk3theme.Matcha-aliz org.gtk.Gtk3theme.Matcha-azul org.gtk.Gtk3theme.Matcha-dark-aliz org.gtk.Gtk3theme.Matcha-dark-azul org.gtk.Gtk3theme.Matcha-dark-pueril org.gtk.Gtk3theme.Matcha-dark-sea org.gtk.Gtk3theme.Matcha-light-aliz org.gtk.Gtk3theme.Matcha-light-azul org.gtk.Gtk3theme.Matcha-light-pueril org.gtk.Gtk3theme.Matcha-light-sea org.gtk.Gtk3theme.Matcha-pueril org.gtk.Gtk3theme.Matcha-sea org.gtk.Gtk3theme.Materia org.gtk.Gtk3theme.Materia-compact org.gtk.Gtk3theme.Materia-dark org.gtk.Gtk3theme.Materia-dark-compact org.gtk.Gtk3theme.Materia-light org.gtk.Gtk3theme.Materia-light-compact org.gtk.Gtk3theme.Materia-nord org.gtk.Gtk3theme.Materia-nord-compact org.gtk.Gtk3theme.Mint-Y org.gtk.Gtk3theme.Mint-Y-Aqua org.gtk.Gtk3theme.Mint-Y-Blue org.gtk.Gtk3theme.Mint-Y-Brown org.gtk.Gtk3theme.Mint-Y-Dark org.gtk.Gtk3theme.Mint-Y-Dark-Aqua org.gtk.Gtk3theme.Mint-Y-Dark-Blue org.gtk.Gtk3theme.Mint-Y-Dark-Brown org.gtk.Gtk3theme.Mint-Y-Darker org.gtk.Gtk3theme.Mint-Y-Darker-Aqua org.gtk.Gtk3theme.Mint-Y-Darker-Blue org.gtk.Gtk3theme.Mint-Y-Darker-Brown org.gtk.Gtk3theme.Mint-Y-Darker-Grey org.gtk.Gtk3theme.Mint-Y-Darker-Orange org.gtk.Gtk3theme.Mint-Y-Darker-Pink org.gtk.Gtk3theme.Mint-Y-Darker-Purple org.gtk.Gtk3theme.Mint-Y-Darker-Red org.gtk.Gtk3theme.Mint-Y-Darker-Sand org.gtk.Gtk3theme.Mint-Y-Darker-Teal org.gtk.Gtk3theme.Mint-Y-Dark-Grey org.gtk.Gtk3theme.Mint-Y-Dark-Orange org.gtk.Gtk3theme.Mint-Y-Dark-Pink org.gtk.Gtk3theme.Mint-Y-Dark-Purple org.gtk.Gtk3theme.Mint-Y-Dark-Red org.gtk.Gtk3theme.Mint-Y-Dark-Sand org.gtk.Gtk3theme.Mint-Y-Dark-Teal org.gtk.Gtk3theme.Mint-Y-Grey org.gtk.Gtk3theme.Mint-Y-Orange org.gtk.Gtk3theme.Mint-Y-Pink org.gtk.Gtk3theme.Mint-Y-Purple org.gtk.Gtk3theme.Mint-Y-Red org.gtk.Gtk3theme.Mint-Y-Sand org.gtk.Gtk3theme.Mint-Y-Teal org.gtk.Gtk3theme.Mojave-light org.gtk.Gtk3theme.Numix org.gtk.Gtk3theme.Numix-Frost org.gtk.Gtk3theme.Numix-Frost-Light org.gtk.Gtk3theme.Obsidian-2 org.gtk.Gtk3theme.Obsidian-2-Amber org.gtk.Gtk3theme.Obsidian-2-Aqua org.gtk.Gtk3theme.Obsidian-2-Gray org.gtk.Gtk3theme.Obsidian-2-Green org.gtk.Gtk3theme.Obsidian-2-Indigo org.gtk.Gtk3theme.Obsidian-2-Mint org.gtk.Gtk3theme.Obsidian-2-Purple org.gtk.Gtk3theme.Obsidian-2-Red org.gtk.Gtk3theme.Obsidian-2-Teal org.gtk.Gtk3theme.Plata org.gtk.Gtk3theme.Plata org.gtk.Gtk3theme.Plata-Compact org.gtk.Gtk3theme.Plata-Compact org.gtk.Gtk3theme.Plata-Lumine org.gtk.Gtk3theme.Plata-Lumine org.gtk.Gtk3theme.Plata-Lumine-Compact org.gtk.Gtk3theme.Plata-Lumine-Compact org.gtk.Gtk3theme.Plata-Noir org.gtk.Gtk3theme.Plata-Noir org.gtk.Gtk3theme.Plata-Noir-Compact org.gtk.Gtk3theme.Plata-Noir-Compact org.gtk.Gtk3theme.Pop org.gtk.Gtk3theme.Pop-dark org.gtk.Gtk3theme.Pop-light org.gtk.Gtk3theme.Pop-slim-dark org.gtk.Gtk3theme.Pop-slim-light org.gtk.Gtk3theme.Qogir org.gtk.Gtk3theme.Qogir org.gtk.Gtk3theme.Qogir-dark org.gtk.Gtk3theme.Qogir-light org.gtk.Gtk3theme.Qogir-light org.gtk.Gtk3theme.Qogir-manjaro org.gtk.Gtk3theme.Qogir-manjaro org.gtk.Gtk3theme.Qogir-manjaro-dark org.gtk.Gtk3theme.Qogir-manjaro-dark org.gtk.Gtk3theme.Qogir-manjaro-light org.gtk.Gtk3theme.Qogir-manjaro-light org.gtk.Gtk3theme.Qogir-manjaro-win org.gtk.Gtk3theme.Qogir-manjaro-win org.gtk.Gtk3theme.Qogir-manjaro-win-dark org.gtk.Gtk3theme.Qogir-manjaro-win-dark org.gtk.Gtk3theme.Qogir-manjaro-win-light org.gtk.Gtk3theme.Qogir-manjaro-win-light org.gtk.Gtk3theme.Qogir-ubuntu org.gtk.Gtk3theme.Qogir-ubuntu org.gtk.Gtk3theme.Qogir-ubuntu-dark org.gtk.Gtk3theme.Qogir-ubuntu-dark org.gtk.Gtk3theme.Qogir-ubuntu-light org.gtk.Gtk3theme.Qogir-ubuntu-light org.gtk.Gtk3theme.Qogir-ubuntu-win org.gtk.Gtk3theme.Qogir-ubuntu-win org.gtk.Gtk3theme.Qogir-ubuntu-win-dark org.gtk.Gtk3theme.Qogir-ubuntu-win-dark org.gtk.Gtk3theme.Qogir-ubuntu-win-light org.gtk.Gtk3theme.Qogir-ubuntu-win-light org.gtk.Gtk3theme.Qogir-win org.gtk.Gtk3theme.Qogir-win org.gtk.Gtk3theme.Qogir-win-dark org.gtk.Gtk3theme.Qogir-win-dark org.gtk.Gtk3theme.Qogir-win-light org.gtk.Gtk3theme.Qogir-win-light org.gtk.Gtk3theme.Sierra-Negra org.gtk.Gtk3theme.SolArc-Dark org.gtk.Gtk3theme.Yaru org.gtk.Gtk3theme.Yaru-Amber org.gtk.Gtk3theme.Yaru-Amber org.gtk.Gtk3theme.Yaru-Amber-dark org.gtk.Gtk3theme.Yaru-Amber-dark org.gtk.Gtk3theme.Yaru-Amber-light org.gtk.Gtk3theme.Yaru-Amber-light org.gtk.Gtk3theme.Yaru-Aqua org.gtk.Gtk3theme.Yaru-Aqua org.gtk.Gtk3theme.Yaru-Aqua-dark org.gtk.Gtk3theme.Yaru-Aqua-dark org.gtk.Gtk3theme.Yaru-Aqua-light org.gtk.Gtk3theme.Yaru-Aqua-light org.gtk.Gtk3theme.Yaru-Aubergine org.gtk.Gtk3theme.Yaru-Aubergine-dark org.gtk.Gtk3theme.Yaru-Aubergine-light org.gtk.Gtk3theme.Yaru-bark org.gtk.Gtk3theme.Yaru-bark-dark org.gtk.Gtk3theme.Yaru-Blue org.gtk.Gtk3theme.Yaru-Blue org.gtk.Gtk3theme.Yaru-Blue-dark org.gtk.Gtk3theme.Yaru-Blue-dark org.gtk.Gtk3theme.Yaru-Blue-light org.gtk.Gtk3theme.Yaru-Blue-light org.gtk.Gtk3theme.Yaru-Brown org.gtk.Gtk3theme.Yaru-Brown org.gtk.Gtk3theme.Yaru-Brown-dark org.gtk.Gtk3theme.Yaru-Brown-dark org.gtk.Gtk3theme.Yaru-Brown-light org.gtk.Gtk3theme.Yaru-Brown-light org.gtk.Gtk3theme.Yaru-Cinnamon org.gtk.Gtk3theme.Yaru-Cinnamon org.gtk.Gtk3theme.Yaru-Cinnamon-dark org.gtk.Gtk3theme.Yaru-Cinnamon-dark org.gtk.Gtk3theme.Yaru-Cinnamon-light org.gtk.Gtk3theme.Yaru-Cinnamon-light org.gtk.Gtk3theme.Yaru-dark org.gtk.Gtk3theme.Yaru-Deepblue org.gtk.Gtk3theme.Yaru-Deepblue org.gtk.Gtk3theme.Yaru-Deepblue-dark org.gtk.Gtk3theme.Yaru-Deepblue-dark org.gtk.Gtk3theme.Yaru-Deepblue-light org.gtk.Gtk3theme.Yaru-Deepblue-light org.gtk.Gtk3theme.Yaru-Green org.gtk.Gtk3theme.Yaru-Green org.gtk.Gtk3theme.Yaru-Green-dark org.gtk.Gtk3theme.Yaru-Green-dark org.gtk.Gtk3theme.Yaru-Green-light org.gtk.Gtk3theme.Yaru-Green-light org.gtk.Gtk3theme.Yaru-Grey org.gtk.Gtk3theme.Yaru-Grey org.gtk.Gtk3theme.Yaru-Grey-dark org.gtk.Gtk3theme.Yaru-Grey-dark org.gtk.Gtk3theme.Yaru-Grey-light org.gtk.Gtk3theme.Yaru-Grey-light org.gtk.Gtk3theme.Yaru-Lavender org.gtk.Gtk3theme.Yaru-Lavender-dark org.gtk.Gtk3theme.Yaru-Lavender-light org.gtk.Gtk3theme.Yaru-light org.gtk.Gtk3theme.Yaru-magenta org.gtk.Gtk3theme.Yaru-magenta-dark org.gtk.Gtk3theme.Yaru-MATE org.gtk.Gtk3theme.Yaru-MATE org.gtk.Gtk3theme.Yaru-MATE-dark org.gtk.Gtk3theme.Yaru-MATE-dark org.gtk.Gtk3theme.Yaru-MATE-light org.gtk.Gtk3theme.Yaru-MATE-light org.gtk.Gtk3theme.Yaru-olive org.gtk.Gtk3theme.Yaru-olive-dark org.gtk.Gtk3theme.Yaru-Orange org.gtk.Gtk3theme.Yaru-Orange org.gtk.Gtk3theme.Yaru-Orange-dark org.gtk.Gtk3theme.Yaru-Orange-dark org.gtk.Gtk3theme.Yaru-Orange-light org.gtk.Gtk3theme.Yaru-Orange-light org.gtk.Gtk3theme.Yaru-Pink org.gtk.Gtk3theme.Yaru-Pink org.gtk.Gtk3theme.Yaru-Pink-dark org.gtk.Gtk3theme.Yaru-Pink-dark org.gtk.Gtk3theme.Yaru-Pink-light org.gtk.Gtk3theme.Yaru-Pink-light org.gtk.Gtk3theme.Yaru-prussiangreen org.gtk.Gtk3theme.Yaru-prussiangreen-dark org.gtk.Gtk3theme.Yaru-Purple org.gtk.Gtk3theme.Yaru-Purple org.gtk.Gtk3theme.Yaru-Purple-dark org.gtk.Gtk3theme.Yaru-Purple-dark org.gtk.Gtk3theme.Yaru-Purple-light org.gtk.Gtk3theme.Yaru-Purple-light org.gtk.Gtk3theme.Yaru-Red org.gtk.Gtk3theme.Yaru-Red org.gtk.Gtk3theme.Yaru-Red-dark org.gtk.Gtk3theme.Yaru-Red-dark org.gtk.Gtk3theme.Yaru-Red-light org.gtk.Gtk3theme.Yaru-Red-light org.gtk.Gtk3theme.Yaru-remix org.gtk.Gtk3theme.Yaru-remix-dark org.gtk.Gtk3theme.Yaru-remix-light org.gtk.Gtk3theme.Yaru-sage org.gtk.Gtk3theme.Yaru-sage-dark org.gtk.Gtk3theme.Yaru-Teal org.gtk.Gtk3theme.Yaru-Teal org.gtk.Gtk3theme.Yaru-Teal-dark org.gtk.Gtk3theme.Yaru-Teal-dark org.gtk.Gtk3theme.Yaru-Teal-light org.gtk.Gtk3theme.Yaru-Teal-light org.gtk.Gtk3theme.Yaru-Yellow org.gtk.Gtk3theme.Yaru-Yellow-dark org.gtk.Gtk3theme.Yaru-Yellow-dark org.gtk.Gtk3theme.Yaru-Yellow-light org.gtk.Gtk3theme.Yaru-Yellow-light org.gtk.Gtk3theme.Zukitre org.gtk.Gtk3theme.Zukitwo

You can see at the top of the list adw-gtk3 and adw-gtk3-dark are separate packages, so if you wanted to support both light and dark modes, install them both:

for T in adw-gtk3 adw-gtk3-dark; do flatpak install -uy org.gtk.Gtk3theme.$T; done Code language: Bash (bash)

See if there’s a packaged or scripted version of the same theme on your distro before installing from Gnome Look – I lucked out and there was one already in the AUR (number 3 looked good to me!):

❯ paru -Ss adw-gtk3
<strong>aur</strong>/<strong>adw-gtk35.3-1</strong> [<strong>+43 ~3.15</strong>]
   The theme from libadwaita ported to GTK-3
<strong>aur</strong>/<strong>adw-gtk3-git1.0.r2.a2a0114-1</strong> [<strong>+15 ~1.21</strong>]
   The theme from libadwaita ported to GTK-3
<strong>aur</strong>/<strong>adw-gtk-theme1.1-2</strong> [<strong>+12 ~1.00</strong>]
   LibAdwaita Theme for all GTK3 and GTK4 Apps. NOTE: This is a meta package
   which uses adw-gtk3 for GTK3 and official LibAdwaita theme for GTK4
Code language: Bash (bash)

If you do have to install a theme package manually, just remember they can go in either /usr/share/themes/. (system-wide) or $HOME/.local/share/themes (user), but not both!

Go to kcm_style in your KDE settings panel and choose “Configure GNOME/GTK Application Style” (top right)

❯ systemsettings <strong>kcm</strong>_styleCode language: Bash (bash)
KDE Settings Configuration (part 1)
KDE Settings Configuration (part 2)

Last step (hopefully):

Tell gsettings your defaults – this has to be done from the command line unless you have dconf-editor or gnome-tweaks installed for some reason (unusual and unnecessary for a KDE-centric system). Here’s the key-value pairs:

❯ gsettings set org.gnome.desktop.interface gtk-theme 'adw-gtk3-dark'; 
  gsettings set org.gnome.desktop.interface color-scheme 'prefer-dark'Code language: Bash (bash)

Replace the values at the end with your particular theme and preference, of course (again, I choose adw-gtk3-dark, mostly because I knew it would be the best supported theme). Obviously, if you don’t 'prefer-dark' that second line is optional.

OK now try starting one of the flatpaks you’ve been having issues with and see how it went!

If it’s still not loading in your preferred theme, I’d go back and check the steps to make sure you covered them properly. You can try gsettings get on org.gnome.desktop.interface gtk-theme to check that it has your proper value, or check /usr/share/themes to make sure a folder exists with the name of the theme you thought you installed (manually or with a package manager) or $HOME/.local/share/themes/ if you installed it manually to your user folder.

If those all seem legit, check out xsettingsd: It looks like it’s a dependency of kde-gtk-config on Arch Linux currently, so you might already have it:

❯ pacman -Qi xsettingsd
Name            : xsettingsd
Version         : 1.0.2-1
Description     : Provides settings to X11 applications via the XSETTINGS specification
Architecture    : x86_64
URL             : https://github.com/derat/xsettingsd
Licenses        : custom:BSD
Groups          : None
Provides        : None
Depends On      : libx11  gcc-libs
Optional Deps   : None
Required By     : kde-gtk-config
Optional For    : None
Conflicts With  : None
Replaces        : None
Installed Size  : 78.82 KiB
Packager        : Antonio Rojas <arojas@archlinux.org>
Build Date      : Mon 09 Aug 2021 04:16:10 AM PDT
Install Date    : Sat 16 Mar 2024 10:31:30 PM PDT
Install Reason  : Installed as a dependency for another package
Install Script  : No
Validated By    : Signature
Code language: Bash (bash)

Do the usual checks to make sure it’s running (it’s only visible using systemctl --user in Arch):

❯  systemctl --user status xsettingsd.service
● xsettingsd.service - XSETTINGS-protocol daemon
     Loaded: loaded (/usr/lib/systemd/user/xsettingsd.service; static)
     Active: active (running) since Sat 2024-03-23 14:35:18 PDT; 1h 52min ago
   Main PID: 469119 (xsettingsd)
      Tasks: 1 (limit: 9084)
     Memory: 156.0K (peak: 696.0K swap: 344.0K swap peak: 344.0K zswap: 55.8K)
        CPU: 4ms
     CGroup: /user.slice/user-1000.slice/user@1000.service/session.slice/xsettingsd.service
             └─469119 /usr/bin/xsettingsd

Mar 23 14:35:18 purplehippo systemd[1283]: Started XSETTINGS-protocol daemon.
Mar 23 14:35:18 purplehippo xsettingsd[469119]: xsettingsd: Loaded 14 settings from /home/avery/.config/xsettingsd/xsettingsd.conf
Mar 23 14:35:18 purplehippo xsettingsd[469119]: xsettingsd: Created window 0xc00001 on screen 0 with timestamp 5373106
Mar 23 14:35:18 purplehippo xsettingsd[469119]: xsettingsd: Selection _XSETTINGS_S0 is owned by 0x0
Mar 23 14:35:18 purplehippo xsettingsd[469119]: xsettingsd: Took ownership of selection _XSETTINGS_S0
Mar 23 14:53:07 purplehippo xsettingsd[469119]: xsettingsd: Reloading configuration
Mar 23 14:53:07 purplehippo xsettingsd[469119]: xsettingsd: Loaded 14 settings from /home/avery/.config/xsettingsd/xsettingsd.conf
Code language: JavaScript (javascript)

If it’s having an issue, try restarting it, or you can examine the configuration file it creates automatically in your $XDG_CONFIG_HOME dir and make sure it all looks reasonable:

❯ bat $XDG_CONFIG_HOME/xsettingsd/xsettingsd.conf
──────┬──────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: /home/avery/.config/xsettingsd/xsettingsd.conf
──────┼──────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ Gdk/UnscaledDPI 98304
   2   │ Gdk/WindowScalingFactor 1
   3   │ Gtk/EnableAnimations 1
   4   │ Gtk/DecorationLayout "icon:minimize,maximize,close"
   5   │ Net/ThemeName "adw-gtk3-dark"
   6   │ Gtk/PrimaryButtonWarpsSlider 0
   7   │ Gtk/ToolbarStyle 3
   8   │ Gtk/MenuImages 1
   9   │ Gtk/ButtonImages 1
  10   │ Gtk/CursorThemeSize 64
  11   │ Gtk/CursorThemeName "Posy_Cursor_125_175"
  12   │ Net/SoundThemeName "ocean"
  13   │ Net/IconThemeName "Papirus"
  14   │ Gtk/FontName "Noto Sans,  10"
  15   │ 
──────┴──────────────────────────────────────────────────────────────────────────────────────────────────────Code language: JavaScript (javascript)

if THAT all looks fine, there was one last thing on wiki.archlinux.org that looked like it might work (especially with Window Manager-type setups like minimal tiling desktop such as sway and dwm – although, if you’re into minimal WMs, not sure if you’d like flatpaks – in any event, nobody’s a monolith…). You could put something like this in $HOME/bin and call it refresh-flatpak-themes (etc.) then run it when you see one getting dodgy:

#!/usr/bin/env bash
for FLATPAK_APP in "$HOME/.var/app/*"; do
  [ -d "$HOME/.var/app/$FLATPAK_APP/config/gtk-3.0" ] ||
  /usr/bin/ln -s "$HOME/.config/gtk-3.0" "$HOME/.var/app/$FLATPAK_APP/config/"
done
Code language: JavaScript (javascript)

Which basically just loops through each flatpak app you have in your user’s .var folder, checks to see if it has a gtk-3.0 definition folder, and if not, makes a symbolic link to where one usually lives in your $XDG_CONFIG_HOME. You could try making the symlink to one manually to see if it works before setting up a script, but if it works out, hey, go for it.

Another option I saw which looks pretty elegant is outlined here on a xerolinux-specific BB, but should apply to pretty much any distro using the xdg user-dirs spec:

❯ export GTK3_THEME=(name of preferred theme);
  flatpak --user override --filesystem=$HOME/.local/share/themes;
  flatpak --user override --filesystem=$XDG_CONFIG_HOME/gtk-3.0:ro;
  for FLATPAK_APP in $HOME/.var/app/*; do flatpak --user override \
  $FLATPAK_APP --env=GTK_THEME=$GTK3_THEME; doneCode language: PHP (php)

Quick pointers for troubleshooting:

  1. The user-dirs.dirs file usually lives at $HOME/.config/user-dirs.dirs
  2. Any $VARIABLE can be echoed to the screen using echo $VARNAME (make sure they’re properly set)

Windows Server Core: Setting a New IP Address For Your DC (including IPv6 & Client DNS)

I’ve been working on moving to Windows Server Core for my DCs, and getting adjusted to PowerShell has been slightly daunting, as the only CLI I’m really familiar with is Unix-based, and, while there are similarties to be sure, they are also very different in a lot of respects. There’s a plethora of remote-based GUI programs for configuring Core servers available that are useful as “training wheels”, like Windows Admin Center (aka Webmin for Windows) and the ever-ubiquitous RSAT tools, which are desktop applications that, by using WinRM, can manage remote machines – programs like Server Manager, ADUC, PSRemoting (aka MS-SSH – hint: just use SSH), and time-tested (old) MMC snap-ins.

But what if you just want to get something essential done, especially if you don’t have remote access available? Well, there’s actually a lot of options still: The Sconfig TUI program, which is always included, but it’s extremely limited in scope, and I’ve realized through difficult experiences can be problematic (more later).

You actually can use MMC locally on a Server Core machine, but it’s more limited in snap-ins, and often these don’t cover what you need.

MMC on Server Core 2022 doesn’t even have Network Shares or Computer Management – boo.

I’ve learned through installing RocketDock for giggles (honorable mention) there actually is an instance of the control panel on Windows Server Core (I had read otherwise), but it’s pretty :sad trombone:.

All your .cpl are belong to us (all 6 of them, woah)

If none of those fit your necessities, you’re looking at using netsh, wmic, or PowerShell. Thankfully, they are all fairly comprehensive and easy to learn, even if they have completely different syntax from anything you’ve had to rely on CLI commands over the past few decades.

OK, enough rant and external links. Here’s something I discovered pawing through the MS wiki and random blog posts about configuring my DCs IP addresses.

I’ve had to re-configure the IPs on my DCs now a couple times since I’ve been migrating them between hosts to deal with hardware upgrades. Every time I clone one and start it on a new machine, it either resets the NIC to DHCP or a link-local address, so I have to re-set the static IP manually and (sometimes) re-authorize the connection to the domain (on the domain controller … irony?).

HAPPYMUFFIN has decided not to recognize he’s a DC anymore, probably because he’s barfing out this Link-Local address

Setting your IP address is obviously trivial on a Server instance with Desktop Experience. Observe:

Like any other Windows Desktop, via Control Panel -> Network Connections

Server Core should be even easier since the SConfig menu pops up whenever you log in, but Sconfig routinely fails to accept a static configuration. Observe:

boo.

Note, if you hate that SConfig pops up when you log in, you can disable it with the cmdlet Set-SConfig, ala:

Set-SConfig -AutoLaunch $falseCode language: JavaScript (javascript)

TL;DR

That was an incredibly long wind-up to say, here’s how to set the NetIPAddress and DnsClientServerAddress in Windows Server Core

Synopsis:

  1. If you’ve got a pre-existing static IP, be sure to delete it first (Remove-NetIPAddress)
  2. If you’ve got DHCP configured, disable it first (Set-NetIPInterface)
  3. Use New-NetIPAddress to create your new address (not Set-NetIPInterface)
  4. Use Set-DnsClientServerAddress to point to your DNS forwarders – which, in my case, are these very DCs I’m configuring

Basics – gathering necessary info:

Get-NetIPInterface will give you a list of network adapters inside your machine currently (including loopback):

PS C:\Users\administrator.DOMAIN> Get-NetIPInterface

ifIndex InterfaceAlias            AddressFamily NlMtu(Bytes) InterfaceMetric
------- --------------            ------------- ------------ ---------------
4       Ethernet0                       IPv6            1500              15
1       Loopback Pseudo-Interface 1     IPv6      4294967295              75
4       Ethernet0                       IPv4            1500              15
1       Loopback Pseudo-Interface 1     IPv4      4294967295              75Code language: CSS (css)

Get-NetIPConfiguration will give you more info about a given interface (you can identify it with -InterfaceIndex (e.g. 4), or -InterfaceAlias (e.g. Ethernet0)

PS C:\Users\administrator.DOMAIN> Get-NetIPConfiguration


InterfaceAlias       : Ethernet0
InterfaceIndex       : 4
InterfaceDescription : vmxnet3 Ethernet Adapter
NetProfile.Name      : webtool.space
IPv6Address          : 2601::dead
                       2601::beef
IPv4Address          : 10.0.0.33
IPv6DefaultGateway   : fe80::f00d
IPv4DefaultGateway   : 10.0.0.1
DNSServer            : ::1
                       10.0.0.3
                       10.0.0.33
Code language: CSS (css)

Removing an old configured IP address:

Remove-NetIPAddress can help you get rid of that old, pesky address that might be stopping you from employing a new one (if they’re on the same subnet, they’ll probably co-exist peacefully, but different subnets = food for gremlins)

Remove-NetIPAddress -AddressFamily IPv4 -IPAddress '192.168.21.3' -Confirm:$falseCode language: JavaScript (javascript)

You can pipe these commands to some extent with Get- and Set-NetIPInterface – here’s an example (albeit, not a very good one since it requires more typing):

Get-NetIPInterface -InterfaceIndex 4 | Set-NetIPInterface -AddressFamily IPv6 -Dhcp DisabledCode language: JavaScript (javascript)

The one you’re probably really after isn’t Set-NetIPInterface, but New-NetIPInterface. That kind of threw me off at first, but it’s reflected in how the workflow expects you to delete prior addresses (before creating a new one, right?)

PS C:\Users\administrator.DOMAIN> New-NetIPAddress -InterfaceIndex 4 -AddressFamily IPv6 -IPAddress '2601::dc01' -PrefixLength 64 -DefaultGateway '2601::beef'


IPAddress         : 2601::dc01
InterfaceIndex    : 4
InterfaceAlias    : Ethernet0
AddressFamily     : IPv6
Type              : Unicast
PrefixLength      : 64
PrefixOrigin      : Manual
SuffixOrigin      : Manual
AddressState      : Tentative
ValidLifetime     : Infinite ([TimeSpan]::MaxValue)
PreferredLifetime : Infinite ([TimeSpan]::MaxValue)
SkipAsSource      : False
PolicyStore       : ActiveStore

IPAddress         : 2601::dc01
InterfaceIndex    : 4
InterfaceAlias    : Ethernet0
AddressFamily     : IPv6
Type              : Unicast
PrefixLength      : 64
PrefixOrigin      : Manual
SuffixOrigin      : Manual
AddressState      : Invalid
ValidLifetime     : Infinite ([TimeSpan]::MaxValue)
PreferredLifetime : Infinite ([TimeSpan]::MaxValue)
SkipAsSource      : False
PolicyStore       : PersistentStoreCode language: PHP (php)

DNS Client configuration:

Now that your IP and Gateway are set, the only thing missing is your DNS settings, right? Since this is a DC, I’m going to point it towards myself and the other DC.

Set-DnsClientServerAddress -InterfaceIndex 4 -ServerAddresses ('2601::dc01','2601::dc02')


InterfaceAlias       : Ethernet0
InterfaceIndex       : 4
InterfaceDescription : vmxnet3 Ethernet Adapter
NetProfile.Name      : webtool.space
IPv6Address          : 2601::dc01
IPv4Address          : 10.0.0.33
IPv6DefaultGateway   : {2601::f00d,
                       fe80::f00d}
IPv4DefaultGateway   : 10.0.0.1
DNSServer            : 2601::dc01
                       2601::dc02
                       10.0.0.3
                       10.0.0.33Code language: PHP (php)

OK, this post ended up being way longer than I anticipated, but that’s how you get the basics of your network adapter working again if SConfig takes the ultimate dump when trying to use it to configure your Server Core network interfaces. Enjoy!

Simple HTTP Server for sending local images to Rancher’s Harvester HCI OS

Concurrent local uploads – not possible when using local file (also no ominous “don’t refresh your window” warning)

I haven’t gotten very far with Harvester yet, having taken down the first cluster I built to re-purpose resources, but I thought I’d explore it again for running some VMS (security video recording) packages at a local business so we can avoid ESXi license costs and potentially scale-out (add servers) in the future without having to get even more licenses for vCenter and use up even more resources.

Harvester’s concept is super cool – set up Kubernetes infrastructure and use it for “legacy workloads” (aka VMs). Contrary to what a lot of other bloggers have written, Harvester does not run containers, so don’t get it twisted. That’s what Rancher is for. (side note: This misinformation got me running down a rabbit hole months ago when I stood up my first Harvester cluster thinking I was going to run a container for my TV recorder. I couldn’t figure out why there was no way to access the container layer, until finally I wrote the developers and they informed me there was no way to access it because it doesn’t exist.)

“Legacy workloads” are completely what we’d planned to run at this business anyway, so that’s just fine. Most decent VMS (video management systems) run in Windows, but we’d like to have some of the features only available via hypervisors, like cloning and testing new systems, or performing updates, without having to take the VM currently in production down or buy another physical machine. Harvester brings it to another level with the easy scale-out (adding more servers), and having cluster-awareness and high-availability embedded by design.

Of course, none of that matters if you can’t get your damn VMs to run, which is the first problem I ran into when I started the thing up. How do I get these clones of existing machines into the storage layer so I can create some new VMs around them?

Harvester only offers two options for creating an image:

  1. Provide a URL that responds to an HTTP GET request for a file that’s a consumable disk-image format (currently qcow2, raw/img or iso)
  2. Upload one of the aforementioned images via a web browser

I was pretty bummed these were the only options when I started out, as I had already connected a disk with my images to the host machine, thinking I could just copy the files from the disk to a particular location on the host filesystem. I should have probably RTFM, as this option (which is the most intuitive to me, but, alas) totally does not exist.

So I turned off the host and dug the NVMe with my images out, and popped it in a USB enclosure, thinking I’d use the upload option from my laptop. I watched the progress bar for long enough to know when to walk away – the file was 23GB, so I knew it should take a while – but the whole thing left me feeling uneasy that the process wouldn’t work. Sure enough, when I returned to the upload page, there was a “context cancelled” error. I tried it two more times, but Harvester kept thinking I had cancelled the upload at 99% finished. I am fairly certain I encountered a bug, but no time to file an issue, I’ve gotta see if this thing will even function for our workloads, and this wasn’t instilling me with the utmost confidence.

It seems like the “URL” option for providing disk images is the more mature of the two options, so I thought I’d look into running a local web server that would respond to a GET request with my image files. This turned out to be super easy, as basically every computer has or can get a copy of python without any trouble. Python has a built-in web server that simply provides whatever files are in the same directory in which it’s run by responding to GET requests and serving them up.

If you’ve got python installed, try it out. On my “server” machine, which in my case was my laptop, I had to do a few things to get it ready. Namely, make sure port 80 was open in my firewall, and make sure I was using the correct zone on my WIFI connection (for the port I had just opened) – I’m on Fedora 37, so if you’re using another OS, you should probably read this instead:

# check current connection:
nmcli con show
NAME                UUID                                  TYPE      DEVICE
rabbit_hole         e130190c-0c3b-4e22-8dc3-ebedc31a7d75  wifi      wlp58s0
EastsideBigTom      a51d9139-89bf-4a4f-9cad-25aaf3a8a2b0  wifi      --
lan on the run      83afc4e0-a0bb-4b62-9437-59478a523da9  wifi      --
Wired connection 1  eadbe5cc-ad09-4e43-8ad1-b24412a8610e  ethernet  --

# check to see if current connection is configured for a zone:
nmcli con show rabbit_hole | grep zone
connection.zone:                        --

# assign a zone to the current connection since it's not configured:
sudo nmcli con mod rabbit_hole connection.zone home

# make sure it worked:
nmcli con show rabbit_hole | grep zone
connection.zone:                        home

# open the port in the firewall - you can use any number up to 65536, but I chose 80 so I wouldn't have to use the port notation:
sudo firewall-cmd --zone=home --add-port=80/tcp --permanent
sudo firewall-cmd --reloadCode language: PHP (php)

If you are OK with this hole being open in your firewall indefinitely, use the --permanent flag for firewall-cmd, otherwise leave it out, and once your firewall is restarted it should be closed.

Now I set up a very rudimentary test to make sure the web server is responding to GET requests, since the machine will respond to a ping, but that’s not very helpful, considering ICMP is a completely different protocol than HTTP:

### on "server" machine (aka laptop, etc.) ###
# navigate to the folder with the files you want to send to Harvester:
cd /path/to/disk/images/you/are/going/to/transfer

# put some gibberish in a file to be served up on a GET request:
echo 'server working' > index.html

# start the actual http server (needs root privileges to attach to socket):
sudo python -m http.server 80

### in Harvester node console ###
# make GET request to "server" for the test gibberish file you created:
curl http://10.0.0.207/index.htmlCode language: PHP (php)

You should see the words “server working” in your Harvester console. If it didn’t work, make sure you spelled everything right, etc. If you don’t know the IP address of your “server”, you can run ip a in your console and it’ll let you know. If you’re not sure which network you’re on, you should probably get a new hobby.

Anyway, now go back to the image menu in Harvester’s web UI, and provide the ip address of your “server” to create images from your files on your node. Make sure you spell everything properly, including using eXaCt SaMe CaSe. Unlike Google, Harvester won’t figure out what you meant if there’s tyops.

This method has proven far more reliable (well, in my case, actually worked) than trying to upload a local file, which is kind of hilarious given it is still uploading the files from the same machine (go figure). Providing files from a local URL also lets you queue a bunch of them to send at once, a definite time-saver. The upload function has an ominous warning not to leave the page or refresh your browser, so in this case you can navigate away from the page without fear of trashing (and painfully re-initiating) your arduously instigated multi-GB transfer process.

Give it a shot!

Install Fedora 36 from RAW disk image on WSL – for free – by using WSL

Neofetch doesn’t find much when you have no DE/WM/DM or icons, but tries its best…

Some people are charging for Fedora in the Windows store. I guess it’s good work if you can get it (?) Charging for someone else’s free OS seems kinda lame to me, but I don’t mind getting my hands dirty.

I could see this process being intimidating for some people, for them $5.99 might be a worthwhile investment to be able to not do what I explain herein. But if that’s the case, it’s probably better you just don’t use Linux to begin with. Those are the people for whom paying like 2000% more for a Macintosh is like, totally worth it. You know who you are.

Note: you do need a working installation of WSL (like the Ubuntu base image), or a linux machine to do this on to start with. Or you could try a util like imdisk in Windows, but I couldn’t figure it out, so I used WSL since I already knew how this process could be achieved in Linux (fairly easily).

Open up a command prompt in Windows and find suitable dir for downloading a large file, such as:

cd %USERPROFILE%\Downloads

download raw – e.g.:

wget https://mirror.sfo12.us.leaseweb.net/fedora/linux/releases/36/Cloud/x86_64/images/Fedora-Cloud-Base-36-1.5.x86_64.raw.xzCode language: JavaScript (javascript)

I chose Fedora Cloud, which is pretty cool, mainly just because it was available in a .raw file, while Fedora Server was only available in .iso. I don’t know much about it, but I noticed it uses systemd-networkd instead of NetworkManager, which makes sense for a cloud-centric OS. That’s a plus in my book.

Choose a distro to invoke, enter wsl from cmd, and you should be in same dir as you were previously:

wsl -l -v

  NAME             STATE           VERSION
* Ubuntu           Running         2

wsl -d ubuntu  # you can just invoke wsl unless need to specify distroCode language: PHP (php)

Once you’re in wsl, you can use bash to save an environment variable for your username and copy these commands straight across (except /dev/loop – make sure you’re using the right loop number and partitions, those are unique to your system, unlikely to be the same, esp if you use a dif distro or release).

export USERNAME=<your username on the PC>Code language: JavaScript (javascript)

Decompress the raw from any archive. E.g.:

xz -d Fedora-Cloud-Base-36-1.5.x86_64.raw.xzCode language: CSS (css)

xz deletes the original, so no need to worry about that. Now create a loop for all the partitions, and list those mounts :

losetup -f -P ./Fedora-Cloud-Base-36-1.5.x86_64.raw
losetup -l

At e.g. /dev/loop0 you should have your .raw file mounted as a device. Now you actually have to mount it as one image (as if it were booted). Create a working subdirectory:

mkdir -p /media/raw

list the partitions on your loop device:

fdisk -l /dev/loop0

Disk /dev/loop0: 5 GiB, 5368709120 bytes, 10485760 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: BF6BA5C6-0688-4AB6-B09C-942136A8C104

Device Start End Sectors Size Type
/dev/loop0p1 2048 4095 2048 1M BIOS boot
/dev/loop0p2 4096 2052095 2048000 1000M Linux filesystem
/dev/loop0p3 2052096 2256895 204800 100M EFI System
/dev/loop0p4 2256896 2265087 8192 4M PowerPC PReP boot
/dev/loop0p5 2265088 10483711 8218624 3.9G Linux filesystem

The root partition is going to go first, and then boot, and finally efi, so in that order.

mount /dev/loop0p5 /media/raw # largest partition generally indicates rootCode language: PHP (php)

Look at what you’re working with, sometimes there’s a root subdir, but usually there isn’t:

ls /media/raw
home root

This time there was a subdir named root, so base the subsequent mounts off of your new information:

mount /dev/loop0p2 /media/raw/root/boot
mount /dev/loop0p3 /media/raw/root/boot/efi

Now you can go to the root folder of the filesystem and archive the image in one go – the trick to making the syntax simple is being inside the directory but saving the .tar elsewhere (in this example, we’ll use the parent folder):

cd /media/raw/root
tar -cvf ../fedora-cloud-36.tar .  # notice these periods, they indicate $pwdCode language: PHP (php)

And in the parent directory will be your tar. Now move it out to C:

mv -v /media/raw/fedora-cloud-36.tar /mnt/c/Users/$USERNAME/Downloads/.Code language: PHP (php)

exit WSL, but stay at command prompt:

exitCode language: PHP (php)

cd your save location:

cd %USERPROFILE%\Downloads

Import .tar into WSL:

mkdir %PROGRAMDATA%\wsl
C:\Users\avery\Downloads>wsl --import Fedora-Server %PROGRAMDATA%\wsl\fedora-server .\fedora-cloud-36.tarCode language: JavaScript (javascript)

Run it to see if it works:

wsl -d fedora-server
cat /etc/os-release

NAME="Fedora Linux"
VERSION="36 (Cloud Edition)"
ID=fedora
VERSION_ID=36
VERSION_CODENAME=""
PLATFORM_ID="platform:f36"
PRETTY_NAME="Fedora Linux 36 (Cloud Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:36"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f36/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=36
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=36
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="Cloud Edition"
VARIANT_ID=cloudCode language: JavaScript (javascript)

BTW, you can safely remove the kernel, since WSL always uses the one installed through Windows:

dnf remove -y kernel-core

Enjoy Fedora …

PS: for homework, you could figure out what else you don’t need you could uninstall or replace, depending on what distro you choose. Systemd comes to mind, or whatever init system you’re using, since you can’t use that in WSL, either… fun fact: you use task scheduler in Windows to schedule stuff to run in WSL instead of cron

Rambling about stuff probably nobody else cares about

Ripped everything out of one of these cases to make a router

I’ve noticed this blog is really less about the info I’m trying to share with people, and more of a collection of me rambling on about the stuff I’ve fixed or put together.

I’m not sure what’s wrong with me. I just spent an hour going off on another blog about this case I owned I made a firewall out of. Since I put so much time and energy into it, I thought I’d bring it here and add it to the collection: http://disq.us/p/2p7i6zw

Orignal on site Baptiste Wicht Norco RPC 230 case review: https://baptiste-wicht.com/posts/2013/08/norco-rpc-230-case-review.html

I have one of those Norco RPC-230 cases, I’ve had it since 2011. It’s crossed 2 states during 4 moves, and at least 6 different configurations, so it’s seen a thing or two.

I suffered through using it longest for a TV recorder PC case, running HDHomeRun for at least a year or two, with 4x HGST 7K2000s in it, which are power-hungry, fast-spinning 7200RPM, hot, old drives. The thermal design of this case is garbage, the fans only hit 2 out of 4 of the drives, so the two coolest HDDs ran 50+ deg C 24/7 that entire time, the other two 55-60+ deg. It was a major testament to HGST’s enterprise line, as they were refurbished, so undoubtedly they were tormented mercilessly before I even got them. The case I got new, but was chipped and scraped after the first of my builds it endured. The front USB connector broke in the middle within a year after normal use. It still works, but is difficult to plug into. As far as quality, amazing drives, crap case, both still work, only one deserves praise.

I also used this case for a desktop PC for a while, with a i5-3570K 77w TDP processor. With stock CPU cooler it would idle at 65 deg C within 5 minutes, which was unacceptable. A Thermaltake Gravity cooler got it under 50 deg C, but the clearance was only a couple mm so it probably could have been more efficient had there been more room, or perhaps a shorter cooler. The Gravity is 3-pin non-PWM, though, which in this case is probably better since the case was never not too hot under any circumstances, so there’s no reason to give a PWM fan any time to decide to rev up.

I thought about drilling some holes in the side near where the CPU would usually sit, since it doesn’t have any airflow from either the side or the back to speak of, but have you ever tried to drill through sheet metal? It’s not as easy as it sounds, even with a specialized drill bit. Even the aluminum foil-like character of the RPC-230’s build quality would present a painstaking chore. I’m glad I decided not to, because I ended up putting a board in it where the CPU isn’t even in the back, it’s in the front, so it wouldn’t have made any difference for my current config, other than to look hideously abused.

Anything over 40 deg C consistently makes me nervous personally, so I was never at ease running any equipment in the Norco RPC-230 case that wasn’t meant to be low-powered, cool, designed for minimal energy consumption, etc. So I finally took a server board X9SPU-F meant for proprietary cases that are near impossible to find, and put it in the RPC with a very low-TDP CPU. I chose the RPC-230 mainly because the motherboard wouldn’t fit in anything else I owned at the time, but also because I finally had a very low-TDP processor I intended to use with it. I’d been watching prices of the E3-1220Lv2 for a couple years and they plummeted from being around $150 to around $35 at the time (and now, virtually worthless). Finally, I had the type of components that wouldn’t make you think you could fry an egg on the top of this thing.

To fit the X9SPU-F in the RPC-230, I had to rip the drive tray out of the front. There’s no use putting drives in the RPC-230 anyway, unless you want to make fried hard drive souffle. Then, I equipped this odd-shaped server board with an E3-1220Lv2 which is 17w TDP, butchered a heatsink out of a CPU cooler from a Dell SFF Optiplex 7020 and tightly zip-tied a Noctua NF-A8 to it (works great). Then, I velcroed a couple mSATA SSDs to the bottom of the case in a single 2-port mSATA to SATA adapter board for a RAID1-ish config designed to keep the system from being taken down by an IO shit-nado of network log files.

Now it’s an edge router + firewall! Has dual Intel NICs and IPMI, too, so it’s fancy. Pulls less than 40w at the wall, does up to 2.4Gbps as tested with an 82599 (possibly faster if not using FreeBSD/pfSense, which has terrible Intel NIC driver and kernel regressions, and less-than-ideal for 10GbE `sysctl` defaults).

For a network device SSD application, I’ve tended to think size and speed are secondary to being able to take a very certain and constant beating. I’ve been using these 24GB Intel 313 SLC mSATA SSDs on dual-SATA daughterboards and putting them in some kind of SDS version of RAID1 (either with mdadm or ZFS). The key is they’re SLC, which is

 hard to find and weren’t made very long for cost reasons. However, you’d be hard-pressed to find another SSD that’ll take a beating like these. They were designed to be cache devices for an old Windows 7 + RST config that never really caught on in I want to say like 2014. The plus side, is you can find a bunch of them around for cheap, and they’re pretty unlikely to die. They only get about 120MB/s read, 60MB/s write, but it doesn’t matter, and if you use a mSATA to SATA converter board like mine that has one port per drive, you can reap some benefit in throughput from running your SDS RAID1-ish mirror-type config (in this device’s case, it’s running ZFS).

And such is the tale of the long and winding life of my poor, beat-up, ill-equipped for most tasks I envisioned, but still utilized, and now humbly revered for being able to fit my bizarrely-shaped server board, Norco RPC-230.

— Good lord, I am such a nerd 🤓

Booting Fedora 36 on default btrfs partition structure from the grub prompt

One of the many logos from the many articles discussing Fedora 33’s stunning move to finally ditch XFS, after their unfortunate habit of talking much trash about btrfs for a number of years

Hey

I ran into an issue where I had an unpopulated grub menu on a Fedora 36 Workstation installation. It ended up only booting to the empty menu, but at least I could drop to a prompt to try and figure out how to get through it.

The usual booting with a live USB, mounting the affected drive to /mnt and the other partitions and bind mounts respectively, and rebuilding grub.cfg from chroot didn’t appear to be functional due to an error in the chroot environment that makes it unable to see /dev and use any of the usual tools to build the file.

TL;DR

All you really need to read in this article to boot from grub prompt is in the 2nd codeblock below (some people just here for the lols)

So booting from the grub prompt was necessary in order to be in an environment where /dev would be recognized and grub.cfg could be rebuilt – since grub.cfg isn’t anything that can be edited manually, unfortunately (it looks like it could be the way it’s named, but if you know anything about how grub works, you know it’s definitely not. Not even a little bit…)

I had written a while back about how to use the grub prompt in general here: The GRUB prompt: Demystified

The example I gave in the last article was done on Ubuntu. It looks like pretty much any other grub rescue operation using EXT4, XFS, etc. but it’s actually a homebrew ZFS initrd- I used to make my own ZFS built-in kernels regularly, and threw together a script for automating the process if you’re interested: https://github.com/averyfreeman/zfs-kernel-builder

But lately I’ve been dabbling with Fedora so this article’s related to Fedora. It’s upstream for some great software, what can I say. I had an NVMe from a Thinkpad I’ve had for a couple years I tried to swap into a newer Thinkpad I was upgrading to, because Thinkpads. For some reason grub.cfg got hosed in the process and I couldn’t boot from it.

Not wanting to take too much time troubleshooting, I fresh-installed a new copy of Fedora 36 to another NVMe in the new laptop and copied as much as I could from /home and a package list from repoquery -a --installed run using chroot from the old NVMe in an external enclosure (gee willikers, I sure love my external NVMe with Thunderbolt 3, by golly).

Since I still had the old installation all set up, I thought I’d try and rescue it eventually, so when I got a moment I booted it from the external NVMe enclosure on another laptop with Thunderbolt using VMware Workstation and the enclosure as a physical disk .vmdk (hypervisors are awesome).

After failing with the tried-and-true live USB chroot rescue method, at least I could get into the grub menu without issue. That poor, empty menu, so lonely feeling with its 0 boot loader entries. I knew the OS was still there, but how to get into it without the entries? What’s a nerd to do? Start messing around with the prompt to explore how to crack open the damn thing!

Since this was btrfs, of course, it was a little different, and since each distro uses a different subvolume layout, none of the info about any of them translate to the others very well. At the time of writing, I didn’t see any definitive info on how to boot from grub prompt on stock btrfs Fedora Workstation, which is kind of surprising since it’s been btrfs default for 3-4 years now, since version 33.

There’s a great guide here about Ubutntu using btrfs on “nixventure”, which I admit I’d never heard of before, but appears very thorough, and I saw one from Debian (I believe) that I’m not going to reference because it was less remarkable, but Fedora there were just a bunch of forums with people flailing about trying to figure out the same thing I was, and ending up being unsuccessful and presumably giving up from the look of their abruptly truncated threads. So that was concerning, to say the last.

I even tried “rescatux” automated rescue tool, if you can believe that, because I was being lazy and kind of running out of ideas. I can’t say I give it a glowing review, but it tries. That statement probably indicates how well it worked for my needs. I had to suck it up and adapt what I knew about using the grub prompt to the new partition layout and filesystem. Thankfully, it all turned out well in the end.

Here’s some notes I took while I was working through the process:

# Fedora's variant of grub has some helpful btrfs-specific commands:

grub> help

. . . commands . . . 
btrfs-get-default-subvol (hd0,gpt3)
btrfs-info
btrfs-list-subvols (hd0,gpt3)
btrfs-mount-subvol
. . . more commands . . .  

# so you get information like this:

grub> btrfs-list-subvols (hd0,gpt3)

ID 256 path home
ID 257 path root
ID 258 path var

# and this:

grub> btrfs-info (hd0,gpt3)

Label: 'fedora_treygouty' uuid: 7caff388-2bb3-434a-a927-096dac2dc892
        Total devices 1 FS bytes used 298520481792

# Intuitively, I thought this would work, but I was mistaken:

grub> linux /vmlinuz-5.17.9-300 <tab-tab works> root=UUID=7caff388-2bb3-434a-a927-096dac2dc892 ro rootflags=subvol=root

# You'd probably have to write all that out by hand, so be thankful I'm telling you ahead of time it didn't work for meCode language: PHP (php)

TL;DR #2 – you’re getting closer …

Anyway, I’ll cut to the chase and show how I did it. Note, again, for the record, this is the bare-minimum default partition structure on an Anaconda installed Fedora 36 Workstation setup. No LUKS or other encryption, no LVM.

Note: For the record, this is the first Anaconda (RedHat’s) installer where LVM hasn’t been enabled for the default partition configuration. LVM is marginally useful for btrfs, I’ve tried it before on OpenSUSE, but comparitively it’s much less of a crutch than it is for EXT4 or XFS filesystems. So if RH’s installer says don’t bother with LVM, don’t bother… It’s RH, you know how they LOVE their LVM, if they’re not recommending it it really must not be necessary. /rant

# Quick grub prompt recap --
# First, list your storage devices:

grub> ls

(proc) (hd0) (hd0,gpt3) (hd0,gpt2) (hd0,gpt1) (cd0) (cd0,msdos2) 

# If you're lucky like me and you only have one drive connected, /boot will be easy to find. /boot is almost always gpt2, and there's only one hard drive, so...:

grub> set root=(hd0,gpt2)

grub> ls /

config-5.17.9-300.fc36.x86_64
efi
extlinux
grub2
initramfs-0-rescue-334f7d6e8a0942d388bafdab5fa2a0a1.img
initramfs-5.17.9-300.fc36.x86_64.img
loader
lost+found
symvers-5.17.9-300.fc36.x86_64.gz
System.map-5.17.9-300.fc36.x86_64
vmlinuz-0-rescue-334f7d6e8a0942d388bafdab5fa2a0a1
vmlinuz-5.17.9-300.fc36.x86_64

# Set your vmlinuz and root device + partition. Here (hd0,gpt3) corresponds with /dev/sda3:

grub> linux /vmlinuz-5.17.9-300.fc36.x86_64 root=/dev/sda3 ro rootflags=subvol=root

# Then boot 'er up:

grub> boot
Code language: PHP (php)

That’s really all there is to it. It’s not too bad once you know what you’re doing. Since the process is so short, you might want to poke around some more to try and get more info, or to try and give your life meaning. There’s the (hd0,gpt2)/grub/grub.cfg file, or the /loader/entries folder – you can cat anything in either of those. Type help or something. Go wild.

Coincidentally, all the loaders were in the loader directory the entire time, but none of them would load all of the things. Such a bummer when one cannot load all of the things. So, frustrating, but glad it all worked out in the end. Hope this helps someone else, too.