avojak | blog

Installing ESXi with a USB NIC

Wed, 17 Jan 2024 00:00:00 +0000

Several years ago I wrote about how to create a custom ISO for installing ESXi with a Realtek 8111 NIC. Unfortunately that process was only valid for ESXi versions up to 6.7, which has reached its end of life.

To upgrade the host that I previously wrote about, I looked into the USB Network Native Driver for ESXi Fling. This had listed support for a handful of inexpensive USB NICs, so I purchased a cheap RTL8153 USB NIC to try out.

Because of the incompatibility of the existing NIC with ESXi 7, I was previously unable to even complete the install on this hardware. This meant I would have to create a new installer ISO with the Fling bundled in it, rather than upgrade in place or upgrade to 7 and then install the Fling later.

The process was largely the same as before, but to make things easy to follow, these are the exact steps that I performed:

1. Retrieve the ESXi 7.0 Offline Bundle

To obtain the prerequisite tools, refer to the prior blog post. At the time of writing, the current latest release of ESXi 7 is 7.0.3, so I downloaded the offline bundle with the following command:

PS C:\Users\Andrew\Downloads> .\ESXi-Customizer-PS.ps1 -v70 -ozip

We now have an archive named ESXi-7.0U3so-22348808-standard.zip (or something similar if a newer version is released).

2. Download the USB NIC Fling

Download the latest version of the Fling for ESXi 7 from this page: https://communities.vmware.com/t5/Flings/ct-p/77.

Currently the latest supported version is 1.10.

3. Build the new ISO

With both of the offline bundles (ESXi and the Realtek driver) in the same directory, we can now run some more commands:

Create the software depot:

 PS C:\Users\Andrew\Downloads> Add-EsxSoftwareDepot ".\ESXi703-VMKUSB-NIC-FLING-55634242-component-19849370.zip", `
    "ESXi-7.0U3so-22348808-standard.zip" 

Create an image profile by cloning an existing one:

 PS C:\Users\Andrew\Downloads> Get-EsxImageProfile
 PS C:\Users\Andrew\Downloads> New-EsxImageProfile -CloneProfile ESXi-7.0U3so-22348808-standard `
    -name ESXi-7.0U3so-22348808-usb-nic -Vendor Razz
 PS C:\Users\Andrew\Downloads> Set-EsxImageProfile ESXi-7.0U3so-22348808-usb-nic `
    -AcceptanceLevel CommunitySupported

Add the driver to the new image profile:

 PS C:\Users\Andrew\Downloads> Get-EsxSoftwarePackage
 PS C:\Users\Andrew\Downloads> Add-EsxSoftwarePackage -ImageProfile ESXi-7.0U3so-22348808-usb-nic `
    -SoftwarePackage vmkusb-nic-fling

Create the ISO:

 PS C:\Users\Andrew\Downloads> Export-EsxImageProfile -ImageProfile ESXi-7.0U3so-22348808-usb-nic `
    -ExportToIso -filepath .\ESXi-7.0U3so-22348808-usb-nic.iso

4. Create a Bootable USB Installer

I used Rufus as well to create the bootable USB drive from the ISO, but you can use any tool that will get the job done.

5. Install and Expect a Failure!

Boot the installer and go through the prompts until you reach an error screen at 81% installed. Never fear! This is expected!

Thankfully I found this blog post (Solution: ESXi Installation with USB NIC only fails at 81%) that explained how technically ESXi was installed at this point, but not fully confiugred. There is a simple fix to force ESXi to recognize the USB NIC for the management interface.

In case that site goes down, I’ll quote the steps below:

… Create ISO …

… Install until 81% failure …

At this point, ESXi is already installed, but not configured.

Remove the installation media and reboot the system

When ESXi is loaded, press F2 and login as “root” without password. (The password entered during the installation has not been saved because the configuration failed)

You should notice that all Network Options are greyed out. Select Network Restore Options.

Select Restore Network Settings

Log out

Log back in

Network options are no longer greyed out and the vusb0 adapter has been detected

Once the steps are followed, you’ll be up and running with ESXi 7.0 fully leveraging your new USB NIC! Even better, the default VM Network will be configured to use this NIC as well, so no extra steps are needed to hit the ground running with new VMs, or to update existing VMs.

Welcome to My Homelab!

Sat, 30 Sep 2023 00:00:00 +0000

This will be the first in an series of posts about my homelab setup. But first, what exactly is a homelab?

A homelab, in the simplest terms is a sandbox that you can learn and play with new or unfamiliar technologies. They can be as simple as a set of VM’s on an old PC or laptop… ¹

The complete source mentioned in this post can be found over in the GitHub repository:

avojak/homelab

Please note that this is not intended to be an exhaustive tutorial for any of the tools that I use, but simply an overview of how I’m using them.

Overview

If all you’re curious about is what services I’m running, I’ll put that up front!

Cloudflare DDNS - Dynamic DNS for domains which are managed by Cloudflare
Grafana - Dashboards
Homebridge - Apple HomeKit compatibility for smart home devices which don’t offer native compatibility
Homepage - Application dashboard and landing page for my homelab with bookmarks to most of my services
InfluxDB - Time series database that stores homelab metrics
Pi-hole - DNS-level ad-blocker
Pi-hole InfluxDB Monitor - Export Pi-hole metrics to InfluxDB
Plausible - Privacy-respecting analytics
Plex - Movie and TV show library
Portainer - Container management platform that I use simply for monitoring logs
Prometheus - Another time series database for storing metrics
Prometheus VMware Exporter - Export VMware stats to Prometheus (I had forked this repo from somewhere and pushed it as avojak/prometheus-vmware-exporter a long time ago, but it has since disappeared…)
Scrypted - Home video integration platform that I use for integrating my UniFi security cameras with Apple HomeKit
Speedtest - Runs periodic network speed tests and records the results in InfluxDB
Telegraf - Agent for collecting realtime metrics from clients that I use to monitor my VMs
Traefik - Application proxy that I use for routing traffic coming into my homelab, as well as within
Uptime Kuma - Simple uptime monitoring for servers, applications, and more

You can check out my Pi-hole InfluxDB Monitor over on GitHub:

avojak/pihole-influxdb-monitor

I also have a blog post about how I created a Grafana dashboard for all the metrics:

The Hardware

My setup is always evolving, but for now I am using a total of six physical servers: five Mac minis and one custom-build rack-mount server. Each server will be running VMware ESXi as the hypervisor. In a nutshell, a hypervisor is what is installed on the bare metal machine (host) to run and manage virtual machines (guests). The hypervisor provides the guests with isolated, virtual hardware on which to run.

I chose ESXi because I’m familiar with it, although I would like to give Proxmox a try at some point in the future.

Automation and Configuration-as-Code

Using virtual machines is not a necessity, however there are many benefits. At scale, virtual machines can be spun up or down as demand ebbs and flows. In a small homelab setting, using virtual machines allows me to easily move between hardware hosts, create backups, and quickly test and rollback changes.

When I first set out building my homelab a couple of years ago, my goal was to learn some industry standard tools. Sure, it might be overkill, but part of the fun of a homelab is having an environment to learn in! I have since been able to bring all of this experience into my day job to deploy applications enterprise-wide!

The three main pieces of the puzzle that I’m going to tackle in this Part 1 post are:

Creating the guest OS image using Packer
Deploying the guest VM using Terraform
Installing software VM using Ansible

There isn’t a particular reason I chose each of these tools over others that are available. I already knew about them, so I decided to give it a go!

Packer - Creating the OS Image

What is Packer?

Packer is an open source tool for creating identical machine images for multiple platforms from a single source configuration. … A machine image is a single static unit that contains a pre-configured operating system and installed software which is used to quickly create new running machines. Machine image formats change for each platform. Some examples include AMIs for EC2, VMDK/VMX files for VMware, OVF exports for VirtualBox, etc. ²

Why use Packer?

There are already images available for server operating systems, so why create your own?

Pre-built images a great, but think about what happens when you install it for the first time: you click through the installer, pick some date/time/keyboard/location options, create a user, etc. etc… Definitely not something I want to repeat every time I deploy a new VM!

With Packer, I took the Ubuntu 22.04 LTS Server ISO and created an OVA image file to deploy to ESXi. The OVA has Ubuntu Server already pre-installed, pre-configured, and an SSH key is setup for use later on with Ansible!

How?

Packer uses JSON template files to define how to build and provision images. My template file was largely influenced by this post: How to use Packer to Build an Ubuntu 18.04 Template for VMware vSphere. Even though I’m ultimatel not using vSphere, the process is nearly identical.

The builders block defines basic information about the ISO to use, the commands to run on boot to automate the installation, and where to place the output image upon completion. Note that some details, such as the memory, can be modified once the image is actual used to provision a VM in ESXi.

{
    ...
    "builders": [
        {
            "type": "vmware-iso",
            "guest_os_type": "ubuntu-64",
            "memory": 1024,
            "name": "{{user `vm_name`}}",
            "iso_url": "{{user `iso_url`}}",
            "iso_checksum": "{{user `iso_checksum`}}",
            "http_directory": "{{user `vm_name`}}/http",
            "boot_wait": "10s",
            "boot_command": [
                "c<wait>",
                "linux /casper/vmlinuz --- autoinstall ds=\"nocloud-net;seedfrom=http://{{.HTTPIP}}:{{.HTTPPort}}/\"",
                "<enter><wait>",
                "initrd /casper/initrd",
                "<enter><wait>",
                "boot",
                "<enter>"
            ],
            "shutdown_command": "echo '{{user `password`}}' | sudo -S shutdown -P now",
            "ssh_username": "{{user `username`}}",
            "ssh_password": "{{user `password`}}",
            "ssh_port": 22,
            "ssh_pty": true,
            "ssh_timeout": "1800s",
            "ssh_handshake_attempts": "20",
            "output_directory": "{{user `vm_name`}}/build"
        }
    ]
    ...
}

The user-data file provides cloud-init configuration for the language, keyboard configuration, user creation, etc.

Note: In a real-world setting you would not want to use a static password here. In my case I’m only using that password during the Packer build - when it comes time to deploy, the Ansible will prevent password-based authentication for SSH.

#cloud-config
autoinstall:
  version: 1
  packages_update: true
  packages_upgrade: true
  identity:
    hostname: ubuntu-server-22.04
    username: provisioner
    password: REDACTED
  early-commands:
    # otherwise packer tries to connect and exceed max attempts:
    - systemctl stop ssh
    - systemctl restart systemd-timesyncd
  keyboard:
    layout: us
  locale: en_US
  timezone: UTC
  ssh:
    install-server: true
    allow-pw: true
    authorized-keys:
      - "ssh-rsa ..."
  late-commands:
    - sed -i -e 's/^#\?PasswordAuthentication.*/PasswordAuthentication yes/g' /target/etc/ssh/sshd_config
    - echo 'provisioner ALL=(ALL) NOPASSWD:ALL' > /target/etc/sudoers.d/provisioner
  storage:
    layout:
      name: direct

Within the provisioners block, we can define commands to run against the installed operating system. This allows you to install software and make other changes before the template is finalized.

There are other types of provisioners, but in my case I only wanted to accmoplish two tasks:

Install software updates
Install an SSH key

{
    ...
    "provisioners": [
        {
            "type": "shell",
            "inline": [
                "sudo apt-get install -y curl cloud-init python3-distutils",
                "curl -sSL https://raw.githubusercontent.com/vmware/cloud-init-vmware-guestinfo/master/install.sh | sudo sh -"
            ]
        },
        {
            "type": "shell",
            "inline": [
                "sudo rm /etc/machine-id /var/lib/dbus/machine-id",
                "sudo touch /etc/machine-id",
                "sudo chmod 444 /etc/machine-id"
            ]
        },
        {
            "type": "file",
            "source": "{{user `vm_name`}}/files/00-installer-config.yaml",
            "destination": "/tmp/00-installer-config.yaml"
        },
        {
            "type": "shell",
            "inline": [
                "sudo mv /tmp/00-installer-config.yaml /etc/netplan/00-installer-config.yaml",
                "sudo chmod 644 /etc/netplan/00-installer-config.yaml",
                "sudo chown root:root /etc/netplan/00-installer-config.yaml",
                "sudo netplan apply"
            ]
        }
    ]
    ...
}

The SSH key is provided at build time using a variable:

{
    "variables": {
        "ssh_key": ""
    }
    ...
}

The final piece of the template file is a post-processor to create the OVA file from the .vmx file:

{
    ...
    "post-processors": [
        {
            "type": "shell-local",
            "inline": [ "ovftool {{user `vm_name`}}/build/packer-{{user `vm_name`}}.vmx output/{{user `vm_name`}}.ova" ]
        }
    ]
    ...
}

With the vmware-iso type builder we get VMDK/VMX files for free, but I wanted to deploy using an OVA image (I had better luck deploying OVAs with Terraform). To do this I simply leveraged the ovftool from VMware ³.

Terraform - Deploying the VMs

What is Terraform?

Terraform is a tool to enable Infrastructure as Code.

Provisioning infrastructure through point-and-click GUIs or custom scripts is slow, error-prone, inefficient, and doesn’t scale. … With HashiCorp Terraform, provisioning and security can be automated with declarative infrastructure and policy as code. Infrastructure and policies are codified, shared, versioned, and executed within a workflow that is consistent across all infrastructure. ⁴

Why use Terraform?

Even though I’m working with a relatively small environment, Terraform allows me to quickly spin up, tear down, and replace VMs.

How?

I created an appserver module that allowed me to reduce a lot of duplication in my code:

resource "esxi_guest" "appserver" {
  guest_name = var.guest_name
  disk_store = "datastore1"

  network_interfaces {
    virtual_network = "VM Network"
    nic_type        = "vmxnet3"
  }

  guest_startup_timeout  = 45 # Seconds
  guest_shutdown_timeout = 30 # Seconds

  boot_firmware  = "bios"
  boot_disk_size = var.boot_disk_size
  memsize        = var.memsize * 1024
  numvcpus       = var.numvcpus
  power          = "on"

  ovf_source = var.ova_file
}

Then for each VM that I want to create, I simply invoke the module with slightly different settings. For example, for one of my Pi-hole servers:

module "pihole01" {
  source = "./modules/appserver"
  providers = {
    esxi = esxi.esx_mac_03
  }
  guest_name     = "pihole01"
  boot_disk_size = 25
  memsize        = 4
  numvcpus       = 2
  ova_file       = var.ova_files["ubuntu"]["amd64"]
}

Ansible - Installing Software

What is Ansible?

Ansible is a tool that enables automation of infrastructure. You write files called “playbooks” which define a series a idempotent tasks that configure infrastructure, applications, systems, and more.

Why use Ansible?

Idempotency is a key concept within Ansible, and this means that a well-crafted playbook can be run repeatedly and always have the same result. If something starts acting strangely on a server, I can re-run my playbooks to get everything back in alignemnt. Similarly, if I need to rebuild a VM, I simply have to point the playbook to a different server and it will setup all the services and applications just like they were before.

How?

I decided early on that I wanted as many applications as possible to be installed via Docker. This has a few advantages:

Software is installed, managed, and monitored consistently
I can easily update/replace the underlying VM with little to no effect

As a result, most of the VMs that I have in my lab are really just Docker hosts, and most of my Ansible roles simply drop in Docker compose files.

End Result

After all of that, I finally spent a bit of time putting together a diagram of all the services that I’m running:

All the services and hosts in one picture!

References

DIY Built-in Bathroom Cabinet

Sat, 01 Jul 2023 00:00:00 +0000

Our primary bathroom used to have an awkard nook next to our shower, so I decided to dive in and learn about building cabinets! What was once wasted space is now efficient storage. And, true to form, it took me far longer than it should have.

The Space

This empty nook next to our shower has been empty since we moved into the house. We suspect that when the house was originally built, the builder offered an upgrade to do some sort of cabinet or built-in vanity here. But to us, it was wasted space!

This empty nook

Design

The driving factor in the design was to maximize storage space. Our linen closet is pretty small and we had the opportunity here to essentially double our linen closet space.

The appearance was inspired by a post that my wife found: DIY Fluted Cabinet Doors.

My first step was to model the project using Fusion 360:

At this point we hadn’t yet settled on a final color or what handles we wanted to use on the doors.

Materials

With the core design all settled, it was off to the hardware store! For the plywood, I opted for sandeply. This was a new type of plywood to me, but it had a major advantage over the birch: lower cost! I also really loved the grain pattern, and since the interior of the cabinets wouldn’t be painted, I really wanted that to pop.

I decided early on that I would try to use 1/2” plywood for the cabinet carcasses. Most people use 3/4” when building cabinets, but since this project wouldn’t be supporting a lot of weight, and would be secured to the wall, I felt comfortable using the 1/2” plywood. For the back panel I went with 1/4” sheets of sande plywood.

For the face frame, I picked up a bunch of 1”x2” poplar boards because poplar is great for painting. The doors are also made out of poplar, but are 1”x3” boards with a 1/4” sheet of sande ply for the back panel.

Left: All the sheets of plywood | Right: Poplar boards for the face frame

Cabinet Carcass

Time to start building! The first step was to cut the boards down to size for the cabinet carcasses. Next, I cut horizontal dadoes along the bottom and middle of the side panels where the bottom and middle shelf panels sit. I also cut a long veritcal dado for the back panel to slide into. Then I drilled a LOT of pocket holes for all the brace pieces!

To cut the dadoes I did repeated passes on my table saw. For the veritcal dadoes this was relatively easy, however for the horizontal dadoes I probably should’ve used a router, as that would’ve been a bit safer. For the dadoes on the very edge, I used a sacrificial piece of scrap would to sneak all the way up on the edge of the plywood.

Left: Side panels | Center-left: Test fit of the dado | Center-right: Dadoes on the braces | Right: Pocket holes!

Then on to assembly! I picked up some cheap corner clamps to help keep the side panels square with the shelves, although they don’t do much in the way of clamping pressure. There were multiple times in this project where I should’ve picked up a nail gun, and this was the first.

Left: Glue-up of shelves with side panels | Center-left: Back panel and top braces | Center-right: Finished front | Right: Finished rear

With the carcasses complete, I set about sanding, and sanding, and sanding… Finally I was able to stack them for the first time to get a feel for just how large this cabinet was going to be.

Left: Completed carcasses | Center: First stack | Right: Test fit in the nook

After multiple coats of water-based polyurethane alternated with a light sanding, I added all 176 shelf pin holes. Thankfully there’s a Kreg jig for that, so it took no time at all.

Left: Post-polyurethane | Right: Shelf pin holes

Around this time I also put together four adjustable shelves. These were made with the same 1/2” ply and a strip of poplar across the front that would eventually be painted to match the face frame. I added some edge-banding to the other three sides to help protect the plywood when the shelves are adjusted.

Left: Adjustable shelf glue-up | Right: Adjustable shelf after polyurethane

Face Frame

The face frame went together pretty quickly, and is held together with pocket holes on the back side.

Left: Test fit of the face frame | Right: I have some strong words for whoever decided that stickers belong on wood

To attach the face frame, the plan was to use biscuits for alignment followed by pocket holes. Rather than buy a biscuit joiner, I picked up a router bit that matched the thickness of the biscuits. With the face frame clamped to the carcasses, I used some painter tape across the joints and drew a straight line. Then I cut the tape and had perfect markers for where the biscuit joints should be cut on both the cabinet carcasses and the back of the face frame.

I forgot to take a picture of the process of lining up the biscuit joints, but here’s a picture of me showing off proper PPE with the markings in the background!

Doors

To assemble the doors, I cut down 1x3 pieces of poplar and cut a 1/4” dado for the plywood panel. Then I cut biscuit joints on the ends of the four side pieces using the same method that I used for the face frame. Finally I gingerly applied the clue, slid in the side panel and used a special clamp that would hold all four corners at once and apply even pressure.

Now you might be asking yourself why I decided to do miter joints on something that would ultimately be painted, and you would never know that it was a miter joint. And you would be 100% correct. This was not something that I thought through very well. A simple tongue and groove door would’ve been far simpler. But hey, I learned some new techniques!

Left: Dadoes on the side pieces | Center: Biscuit joints cut using the router | Right: Final assembly

To attach the doors, I picked up some Blum soft close hinges. As always, there’s a Kreg jig for that! I’ve never used these hinges before, so I made a small replica of the face frame and door to test and adjust the alignment settings on the jig.

Installation

Time to start installing! It was a little tight trying to stack cabinets that go 98% of the way to the ceiling, but once I got them aligned I clamped them together and slid them into place.

Left: Stacked in the bathroom | Right: Clamps ready to go

Attaching the face frame was a massive pain. My original plan was to use pocket holes coming in from the outside of the cabinet carcass into the face frame. I got the biscuits glued, applied some glue to the edges of the cabinet carcasses, got the face frame on and clamped, screwed in the first pocket screw, and… it broke through the face frame.

At this point I started panicking. Although I had applied some glue, I was relying on the screws to pull the face frame tight. In my panic, I grabbed some finish nails (I didn’t have a nail gun at this point) and started nailing the face frame to the cabinets. It wasn’t pretty, but I would later be able to patch over the nails and touch up the paint.

Left: Pocket hole damage | Right: face frame installed

Finally I added some trim pieces to match the curvature of the wall. I wasn’t confident enough to attemt to scribe the face frame to match the wall, so this allowed my some flexibility to bend the trim to match. For this step I went and bought a nail gun - so much easier!

Trim added and caulked

All that was left to do was add the hardware on the doors and install the doors!

Finished Product

All done!

Lessons Learned

As always, there were lots of lessons to be learned. Here’s a few of the ones that stuck with me the most:

I shouldn’t have used miter joints for the doors - I learned something new, but really I just made my life more difficult with this one!
Pocket hole fail for the face frame - This really falls under the umbrella of “measure twice”. If I had taken a little bit more time to measure where the screws would be going at that angle, it would’ve been obvious that there would be problems. Similarly, I could’ve made a small mockup of the face frame and cabinet edge to test this.
Alignment on the face frame - The biscuits worked well to align the face frame, but it wasn’t perfect. If I had some more clamps I could’ve more easily kept it in place. Ultimately it slid slightly when I was panicking and putting in the nails by hand.
Test, test, test. For miters, painting, door hinges, anything new! - Making small mockups of joints to test before doing the real thing is very helpful. I also should’ve tested the paint sprayer on a test piece of wood before jumping in and painting the doors.

Grafana Dashboard for Pi-hole Stats

Thu, 15 Dec 2022 00:00:00 +0000

One of the most important parts of my home network and homelab is my Pi-hole, because who likes internet ads? I also enjoy pretty graphs, so I recently started looking for a way to add Pi-hole stats to my Grafana dashboard. Unfortunately all the other monitoring tools that I came across had a fatal flaw in how they recorded the number of queries throughout the day: they simply plotted the “queries today” value at each poll interval. However, Pi-hole tracks queries in 10-minute rolling windows. Each value represents the number of queries for the past 23 hours and 50 minutes, plus the number of queries in the current 10-minute window. Charting this over time looks like a sawtooth, and isn’t what we want:

This plot over time is somewhat misleading

My goal was to replicate the same chart that the Pi-hole dashboard shows.

Using the REST API, I simply poll the overTimeData10mins endpoint and record those datapoints in InfluxDB. Sure, there will be a lot of duplicate data, but those datapoints will simply be overwritten in the database. Once I got this working, I made quick work of populating all the other data that the API provides.

# Ads over time
for timestamp,count in ads_over_time.items():
    points.append(Point.from_dict(
        {
            "measurement": "over_time_data",
            "tags": tags,
            "fields": {
                "ads_over_time": count
            },
            "time": int(timestamp)
        },
        WritePrecision.S
    ))

Bingo!

The last major challenge was displaying the tables of Top Queries, Top Ads, and Top Sources. For most other datapoints I was able to simply take the JSON data from the API and drop it in as the field data for the database record. However, attempting to group multiple records by time, and sort them turned into a massive headache, and failed in several edge cases. To make querying the data easier, I converted the JSON data from the REST API into a comma-separated string. This way there would always be a single, latest record in the database with the top items.

points.append(Point.from_dict(
    {
        "measurement": "top_queries",
        "tags": tags,
        "fields": {
            "top_10": self._json_to_csv(stats.pop('top_queries'))
        },
        "time": now_seconds
    },
    WritePrecision.S
))

In the end I tossed together a little Grafana dashboard to display all the data I collected:

The script is available as a Docker image on DockerHub, and can be run easily:

docker run -d \
    -e PIHOLE_ALIAS="pihole" \
    -e PIHOLE_ADDRESS="http://pi.hole" \
    -e PIHOLE_TOKEN="pihole_api_token" \
    -e INFLUXDB_ADDRESS="http://influxdb:8086" \
    -e INFLUXDB_ORG="my-org" \
    -e INFLUXDB_TOKEN="super_secret_token" \
    -e INFLUXDB_BUCKET="pihole" \
    avojak/pihole-influxdb:latest

Alternatively you can run the Python application directly from the command-line:

python3 pihole-influxdb.py \
    --pihole-alias "pihole" \
    --pihole-address "http://pi.hole" \
    --pihole-token "pihole_api_token" \
    --influxdb-address "http://influxdb:8096" \
    --influxdb-org "my-org" \
    --influxdb-token "super_secret_token" \
    --influxdb-bucket "pihole"

Check out the full source over on my GitHub:

avojak/pihole-influxdb-monitor

Updating Warble to GTK4

Fri, 02 Sep 2022 00:00:00 +0000

Warble is officially at version 2.0.0, which means it’s now using GTK4!

Andrew Vojak

@avojak

Warble v2.0.0 is here! 🥳

✨Updated to GTK4
✨Cleaner UI
✨New (barely modified) icon
✨Same fun gameplay!

Get the update soon on @elementary #AppCenter, or on #Flathub!

#vala #gtk #gtk4

It was a fun challenge to update Warble from GTK3 to GTK4, although a bit frustrating at times. I hope that by walking through some of the specific changes I made, you might not struggle quite as much as I did! Wherever possible I will include links to the diff in GitHub so you can see the exact change that I made.

You can view the full diff of changes here: avojak/warble/pull/39

Updating the Dependencies

The obvious first step is to, well, pull in GTK4 as a dependency!

dependency('gtk4', version: '>= 4.6.3')

And if you’re using an IDE, you’ll immediately be met with a giant slap in the face. Compile errors out the wazoo! If you’re like me, you had no idea that Libhandy was going away (and I was just getting used to it…). Turns out, Libadwaita is the successor. Not to worry, this is an easy substitution!

dependency('libadwaita-1', version: '>= 1.1.0'),
# I'm using Granite as well, so you need to grab the new version of it as well!
dependency('granite-7', version: '>= 7.0.0')

Now all you have to do is quickly rush through your code, take a quick guess at how to fix the compile errors, rebuild, run, and…

Andrew Vojak

@avojak

Me: *Updates Warble to GTK4 and spends a couple hours resolving all the compile and runtime errors*

Warble: 🙃

Welp. This was when I realized that it wasn’t going to be a quick update!

Helpful Resources

I’m going to pause briefly here to list some resources that I found helpful along the way:

GTK - Migrating 3 to 4 - This guide was very helpful at identifying specific API changes, and the replacements. When in doubt, Ctrl+f on this page first!
GTK Inspector - The GTK inspector application is incredibly useful in general, but specifically when you’re making UI changes. If you’re using Flatpak, simply add --env=GTK_DEBUG=interactive to your flatpak run command and the Inspector will open alongside your app!
Valadoc.org - Another probably obvious resource, but nothing beats reading the docs.

Input Handling

There are two main input methods in Warble: using the physical keyboard attached to the computer, and the on-screen keyboard.

Keyboard Input

With GTK3 and Libhandy, I used the key_press_event signal on the Hdy.Window, but GTK4 introduced a new event controller system.

var key_event_controller = new Gtk.EventControllerKey ();
key_event_controller.key_pressed.connect (on_key_pressed_event);
((Gtk.Widget) this).add_controller (key_event_controller);

A relatively small difference in this case, but worth noting.

Mouse Input

Mouse input is more significantly different. Instead of needing to use widgets such as EventBox, and using Gdk.EventMask for the various event types you care about, we now use Gtk.GestureClick (or other similar controllers depending on the desired event type):

var gesture_event_controller = new Gtk.GestureClick ();
gesture_event_controller.pressed.connect ((n_press, x, y) => {
    if (n_press != 1) {
        return;
    }
    get_style_context ().add_class ("key-pressed");
    clicked (letter);
});
gesture_event_controller.released.connect ((n_press, x, y) => {
    if (n_press != 1) {
        return;
    }
    get_style_context ().remove_class ("key-pressed");
});
this.add_controller (gesture_event_controller);

(Diff)

Final Classes

One error you will most certainly encounter is field 'parent_instance' has incomplete type. This is because many Gtk classes are now final and cannot be subclassed. I ran into this because I had subclassed Gtk.HeaderBar. In my case I simply created the header bar directly in a different class, but in other cases I had to use a different container widget.

Drawing with Cairo

Under GTK3, the method for doing custom drawing on a widget was to override the draw(Cairo.Context) method:

protected override bool draw (Cairo.Context ctx) {
    base.draw (ctx); // Draw the widget as normal
    ctx.save ();
    custom_drawing (ctx); // Do some custom drawing after the base widget is drawn
    ctx.restore ();
    return false;
}

For GTK4, there’s a different approach that I can use. To draw the squares on the game board with letters, the squares are now Gtk.DrawingArea widgets. This class exposes a set_draw_func(Gtk.DrawingArea da, Cairo.Context ctx, int width, int height) method with some new convenience parameters for width and height.

private void draw_func (Gtk.DrawingArea drawing_area, Cairo.Context ctx, int width, int height) {
    var color = Gdk.RGBA ();
    color.parse (Warble.ColorPalette.get_text_color (state));
    ctx.set_source_rgb (color.red, color.green, color.blue);

    ctx.select_font_face ("Inter", Cairo.FontSlant.NORMAL, Cairo.FontWeight.BOLD);
    ctx.set_font_size (FONT_SIZE);

    Cairo.TextExtents extents;
    ctx.text_extents (letter.to_string (), out extents);
    double x = (width / 2) - (extents.width / 2 + extents.x_bearing);
    double y = (height / 2) - (extents.height / 2 + extents.y_bearing);
    ctx.move_to (x, y);
    ctx.show_text (letter.to_string ());
}

(Diff)

Window Positioning

Gone are the days of saving the position of the application window to the GSettings - now the window manager handles that for you!

(Diff)

CSS Blur

This isn’t a change, but rather a new feature in GTK4 that I love. You can now use a blur() filter. In Warble, I combine blur with transparency on certain overlay views as a nice replacement for using a dialog:

.faded {
    transition-property: opacity, filter;
    transition-duration: 0.5s;
    filter: blur(5px);
    opacity: 0.3;
}

I’m a big fan of how this turned out:

Granite `ModeButton` No More

Not necessarily GTK4 related, but the ModeButton widget was removed from Granite. This is really a “shame on me” moment for using this widget incorrectly, but it might pose a “gotcha” to anyone else using it in a similar way.

The replacement is to use grouped Gtk.CheckButton widgets. For Warble, I created a simple wrapper widget (CheckButtonGroup) to handle the grouping.

I think the GTK method of grouping widgets in this case is rather clumsy. One common theme in GTK4 is the use of proper container widgets, yet here we create a “group” by assigning a sibling widget to the group property. Huh?? Now you end up with a widget that’s pulling double-duty as both an individual button, but also the group. In my opinion, a better design would be to create a specific group container widget that you add buttons to.

CSS Changes

While not directly related to GTK4, I took this opportunity to leverage CSS for styling the game area and on-screen keyboard instead of using custom icons for each possible color scenario.

The icons for the squares were replaced with two lines of code to apply some CSS classes:

get_style_context ().add_class (Granite.STYLE_CLASS_CARD);
get_style_context ().add_class (Granite.STYLE_CLASS_ROUNDED);

And the various states for the squares and keyboard keys were also represented with CSS classes:

.guess-correct {
    background-color: @LIME_300;
    border: 1px solid @LIME_700;
}

.guess-incorrect {
    background-color: @SILVER_500;
    border: 1px solid @SILVER_900;
}

.guess-close {
    background-color: @BANANA_300;
    border: 1px solid @BANANA_700;
}

.key-pressed {
    transform: translateY(2px);
}

.tile-active {
    border: 1px solid @BLACK_700;
}

When switching to high-contrast mode, all I had to do was swap out the stylesheet - no more changing ALL the icons!

private void load_theme_stylesheet () {
    var gtk_settings = Gtk.Settings.get_default ();
    bool dark_mode = gtk_settings.gtk_application_prefer_dark_theme;
    bool high_contrast_mode = settings.get_boolean ("high-contrast-mode");

    if (dark_mode && high_contrast_mode) {
        theme_provider.load_from_resource (Constants.APP_ID.replace (".", "/") + "/warble-dark-hicontrast.css");
    } else if (dark_mode && !high_contrast_mode) {
        theme_provider.load_from_resource (Constants.APP_ID.replace (".", "/") + "/warble-dark.css");
    } else if (!dark_mode && high_contrast_mode) {
        theme_provider.load_from_resource (Constants.APP_ID.replace (".", "/") + "/warble-light-hicontrast.css");
    } else {
        theme_provider.load_from_resource (Constants.APP_ID.replace (".", "/") + "/warble-light.css");
    }
}

As an added bonus, I was able to very easily add a subtle outline for the square on the game board that’s currently waiting for input. Previously this would’ve required a whole new icon.

End Result

Check out the end result over on GitHub!

avojak/warble

Installing VMware ESXi on a Raspberry Pi

Mon, 29 Aug 2022 00:00:00 +0000

I’ve had a pair of Raspberry Pi 4’s sitting around collecting dust and figured I could put them to use in my homelab. I currently run Pi-hole (a network-wide ad blocker) on a pair of very old Raspberry Pi 2’s. Although they’re working just fine for now, they’re not ideal for a few reasons:

They run from the SD card vs. an SSD (SD card notoriously wear out quickly)
They require both a power cable AND network cable vs. PoE
They are manually managed vs. the Infrastructure-as-Code powering the rest of my homelab

There’s more than one way to crack an egg, but since I run ESXi on other hosts as well, it seemed like a good way to have some consistency.

There are a number of guides out there for how to get ESXi (technically ESXi-Arm) running on a Raspberry Pi, but I constantly ran into boot issues. I finally came across this blog post by Mateusz Romaniuk and it worked like a charm. I’ll summarize how I did everything below, as there are a few points in the blog post that give you options I didn’t feel were necessary. Be sure to check out the original post for guided pictures that are very helpful as well!

Prerequisites

Raspberry Pi 4 Model B (4 or 8 GB RAM)
USB keyboard
Power suply (bonus points for using PoE!)
microHDMI to HDMI adapter + HDMI cable (or just a microHDMI to HDMI cable)
1x microSD card (for UEFI firmware, so it doesn’t need to have a very large capacity)
1x USB flash drive (for the ESXi installer, which is only ~140MB)
1x USB flash drive or SSD (for the full ESXi installation)
The Raspberry Pi Imager tool
The ESXi-Arm ISO
https://github.com/raspberrypi/firmware (Click “Code”, then “Download ZIP”)
https://github.com/pftf/RPi4 (Download the RPi4_UEFI_Firmware_vX.XX.zip file from the most recent release)

Preparation

Prepare the SD Card

Insert the SD card into your computer and open the Raspberry Pi Imager tool.
Select “CHOOSE OS”, and scroll down to “Misc utility images”. Select “Bootloader”, and then “SD Card Boot”.
Select “CHOOSE STORAGE”, and select the option for your SD card.
Select “WRITE”.
Wait until the imager is done, and then safely remove the SD card.
Insert the SD card into the Raspberry Pi (without the SSD and USB flash drive connected), and power on the Pi.
You should see a green screen and the green LED on the Raspberry Pi blinking. If so, power off the Pi. If not, there should be an error message displayed.

Copy the UEFI Firmware

Insert the SD card back into your computer and re-open the Raspberry Pi Image tool.
Select “CHOOSE OS”, and scroll to “Erase”.
Select “CHOOSE STORAGE”, and select the option for your SD card.
Select “WRITE”.
When done, it may automatically unmount the SD card. If it does, simply unplug it and plug it back in.
Extract the firmware-master.zip file downloaded previously, and delete all of the kernel*.img files within the firmware-master/boot/ directory (there should be 4 of them).
Copy all files from firmware-master/boot/ onto the SD card.
Extract the RPi4_UEFI_Firmware_vX.XX.zip file downloaded previously, and copy all contents onto the SD card (you can select the option to replace files in case there are matches).

If the Raspberry Pi that you’re using is the 4 GB model (meaning it has 4 GB of RAM, not 8 GB), you will need to open config.txt on the SD card and add the following line to the end of the file:
```
1
 gpu_mem=16
```
Safely eject the SD card from your computer and insert it into the Raspberry Pi.

Configure the UEFI Firmware

With the SD card inserted, power on the Raspberry Pi, and when you see the Raspberry Pi logo hit the Esc key.
Use the arrow keys to select “Device Manager”.
Select “Raspberry Pi Configuration”, and then “Advanced Configuration”.
Change “Limit RAM to 3 GB” to “Disabled”.
Press F10 to save, then Y to confirm.
Press Esc three times until you are back on the main screen, and then select Continue.
You will be prompted that the “Configuration changed. Reset to apply it Now. Press ENTER to reset”. Press Enter.
When the Raspberry Pi reboots, once again press Esc when the logo appears on the boot screen, and you should be back at the UEFI boot screen.
Leave the Raspberry Pi alone for a minute while we prepare the ESXi installer!

Prepare the ESXi Installer

Create a bootable USB flash drive from the ESXi-Arm ISO downloaded previously (I highly recommend using balenaEtcher for this).
Once the drive is done being flashed, safely eject it from your computer and plug it into the Raspberry Pi.

Installation

At this point, the Raspberry Pi should be powered on to the UEFI boot screen. If not already, power on the Raspberry Pi and hit Esc when the boot logo appears.

Make sure that the USB flash drive with the ESXi installer is inserted.
Connect the SSD (be sure to use the one of the USB 3.0 ports!).
From the UEFI boot screen, select “Boot Manager”.
Scroll down to your bootable USB flash drive from the menu but don’t select it yet! You will have a short window of time to press Shift+O once (that’s an ‘O’ as in octopus) before ESXi starts booting. If you miss it, it’s ok, you’ll soon be able to press Esc to cancel the installer and reboot your device to try again.
Now, select your device from the boot menu and press Shift+O.
At the bottom of the screen you should see a small prompt that says runweasel cdromBoot. Add autoPartitionOSDataSize=8192 to the end so that the full command reads:
```
1
 runweasel cdromBoot autoPartitionOSDataSize=8192
```
Press Enter.
The ESXi installer will start, and when prompted, press Enter to continue.
Press F11 to accept the EULA.
Select your SSD drive from the Storage Device menu.
Press Enter to confirm the disk selection.
Choose your keyboard layout.
Set a root password, and then press Enter.
Press F11 (yet again) to confirm the installation.
When the installation is complete, remove the USB flash drive with the installer, and press Enter to reboot the Raspberry Pi.

Set Default Boot Device

When the Raspberry Pi boot logo reappears, press Esc to enter the UEFI boot screen.
Select “Boot Maintenance Manager”, then “Boot Options”, and then “Change Boot Order”.
Move your SSD device to the top of the list.
Press F10 to save, then Y to confirm.
Press Esc three times until you are back on the main screen, and then select Continue.

ESXi Configuration

Before you start creating VMs, grab a free license for ESXi 7 from VMware: https://my.vmware.com/en/group/vmware/evalcenter?p=free-esxi7.

Open a web browser on your computer and go to the address shown by ESXi on the screen (be sure to use HTTPS).
Log in using the username root and the password you set previously.
Select “Manage” from the Navigator panel on the left side of the screen, and go to the “Licensing” tab. Click “Assign license”, and paste in your new ESXi 7 license. Click Assign.

Badabing bada-boom, you’ve got yourself ESXi running on a Raspberry Pi!

References:

https://vmattroman.com/esxi-on-raspberry-pi-4-how-to-install/

Adding Disk Space to an CentOS VM

Fri, 26 Aug 2022 00:00:00 +0000

This post is mostly a reminder to my future self how to do this!

Update the virtual disk

This is the easy part - shut down the VM and expand the disk size in the virtual machine settings. Easy peasy.

Install growpart

If not already installed, grab growpart.

 $ sudo yum install cloud-utils-growpart

Expand the partition

I found the name of my device using the Disks appliction. In my case the root partition was on /dev/nvme0n1p3, so the device is /dev/nvme0n1 and the partition is 3.

Using growpart (first with -N to do a dry-run):
```
1
 $ sudo growpart /dev/nvme0n1 3 -N
```
If no errors, run again without the -N option.

Expand the physical volume

 $ sudo pvresize /dev/nvme0n1p3
 Physical volume "/dev/nvme0n1p3" changed
 1 physical volume(s) resized or updated / 0 physical volume(s) not resized
 $ sudo pvs
 PV             VG Fmt  Attr PSize  PFree 
 /dev/nvme0n1p3 cs lvm2 a--  48.41g 30.00g
 $ sudo vgs
 VG #PV #LV #SN Attr   VSize  VFree 
 cs   1   2   0 wz--n- 48.41g 30.00g

Expand the logical volume

Using the output from vgs above (you can also find the name from the Disks application), we can now expand /root:

 $ sudo lvextend -r -l +100%FREE /dev/cs/root
   Size of logical volume cs/root changed from 16.41 GiB (4201 extents) to 46.41 GiB (11881 extents).
   Logical volume cs/root successfully resized.
   meta-data=/dev/mapper/cs-root    isize=512    agcount=4, agsize=1075456 blks
            =                       sectsz=512   attr=2, projid32bit=1
            =                       crc=1        finobt=1, sparse=1, rmapbt=0
            =                       reflink=1    bigtime=0 inobtcount=0
   data     =                       bsize=4096   blocks=4301824, imaxpct=25
            =                       sunit=0      swidth=0 blks
   naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
   log      =internal log           bsize=4096   blocks=2560, version=2
            =                       sectsz=512   sunit=0 blks, lazy-count=1
   realtime =none                   extsz=4096   blocks=0, rtextents=0
   data blocks changed from 4301824 to 12166144

Verify:

 $ df -h
   Filesystem           Size  Used Avail Use% Mounted on
   devtmpfs             3.8G     0  3.8G   0% /dev
   tmpfs                3.8G     0  3.8G   0% /dev/shm
   tmpfs                3.8G  9.9M  3.8G   1% /run
   tmpfs                3.8G     0  3.8G   0% /sys/fs/cgroup
   /dev/mapper/cs-root   47G   13G   34G  28% /
   /dev/nvme0n1p2      1014M  251M  764M  25% /boot
   /dev/nvme0n1p1       599M  7.3M  592M   2% /boot/efi
   tmpfs                774M     0  774M   0% /run/user/976
   tmpfs                774M   28K  774M   1% /run/user/1000

Fill up the disk and repeat!

Helpful resources:

https://www.unixarena.com/2022/01/how-to-extend-the-root-filesystem-in-rhel-8-centos-8.html/

Deploying Valadoc to GitHub Pages

Fri, 05 Aug 2022 00:00:00 +0000

I recently wrote a few libraries in Vala and was struggling to make the Valadoc easily accessible to users. I found several examples of developers generating the docs locally and simply checking them into version control in a docs/ directory. This doesn’t appeal to me, as documentation like this - in my opinion - is a product of a successful build and should be hosted separately.

After some digging I came across a GitHub Action for deploying to GitHub Pages. This seemed like a great fit because it will easily integrate with my existing CI process.

Preparation

Generate Valadoc

The first thing to do is ensure that your library will generate Valadoc documentation. I’m not going to go into the details of that here, but at the bottom of the post you will find a link to an example repository of mine that you can use for reference.

In most cases you will end up with a pair of commands that look something like this:

meson build -Ddocumentation=true
ninja -C build

Et voilà! There’s documentation in the build/doc/ directory!

Enable GitHub Pages

The next step is to create a separate empty branch on your repository named gh-pages. Simply create a new branch, delete all the files, and push.

Now you’ll want to enable GitHub Pages for your repository. For this, you can refer to GitHub’s documentation: Configuring a publishing source for your GitHub Pages site

Setup the Action

Now it’s time to create the GitHub action! If you don’t already, create a .github/workflows/ directory in the root of your repository. Then, create a YAML file (e.g. ci.yml, or valadoc.yml).

I already have a ci.yml file here with jobs to build and execute unit tests, so I’m omitting the irrelevant jobs below. I wanted to only publish the documentation if the build and tests were successful, so I put everything in a single file. You don’t need to do this if you don’t want to.

My ci.yml file looks like this:

name: CI
on: [pull_request, push]
permissions:
  contents: write
jobs:
  # ... other jobs omitted...
  Valadoc:
    name: Deploy Valadoc
    runs-on: ubuntu-latest
    needs: Test
    container: 
      image: elementary/docker:unstable
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Install dependencies
        run: apt-get update && apt-get -y install libvala-dev valac meson git
      - name: Build
        run: meson build -Ddocumentation=true && ninja -C build
      - name: Deploy
        if: ${{ github.ref == 'refs/heads/master' }}
        uses: peaceiris/actions-gh-pages@v3
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_dir: ./build/doc/catalyst-1-vala

The first three steps simply checkout the source code, install some dependencies, and build the documentation using the same command described previously.

The fourth step is the real magic. I have an if check to only publish documentation from the master branch. You can remove this or change the branch name to another branch if you’d like. Then we include the peaceiris/actions-gh-pages@v3 action. We don’t need to re-invent the wheel!

We provide two variables - github_token and publish_dir. The token will allow the build to have write access to publish to the gh-pages branch. The publish_dir points to the specific directory that will be published. In my case I have a specific subdirectory within build/doc/ that I want to publish.

And that’s it!

You can find complete documentation for the peaceiris/actions-gh-pages action on GitHub: peaceiris/actions-gh-pages.

End Result

When it’s all said and done you’ll have publically-accessible documentation hosted for free via GitHub Pages. I have a custom domain setup for my GitHub pages (the same one you’re looking at right now!) so you can view an example over at: https://avojak.com/libcatalyst/catalyst-1/.

And no project like this would be completely without a shiny new badge for your README:

[![Documentation](https://img.shields.io/badge/documentation-valadoc-a56de2)](https://avojak.com/libcatalyst/catalyst-1/)

You can see a full example project using this exact process over on my GitHub:

avojak/libcatalyst

Libsemver

Fri, 10 Jun 2022 00:00:00 +0000

As I’ve been developing apps in Vala and GTK for a few years now, there have been two questions nagging at me the entire time:

How libraries are written in Vala?
How do you properly unit test code written in Vala?

Frankly the second question is one that I should have investigated from the start, but better late than never, no time like the present to learn, and other adages.

To learn by doing, I settled on writing a library to handle Semantic Versions that I aptly named libsemver:

avojak/libsemver

How Are Libraries Written in Vala?

Well the real meat and potatoes is just like writing a library in any other language, but the project setup for a library is incredibly confusing if you’re not sure what you’re doing. Spoiler: I wasn’t sure what I was doing, and frankly I still don’t. The amount of boilerplate code that goes into a library so that it’s properly picked up by the system when you try to use it is a bit overwhelming. I can’t really write much about this process because like I said, I still don’t fully understand all the ins and outs. I used Granite heavily as a reference since I had used it before, and in the end I was able to successfully pull my library into a test project and use it.

Another great reference that I came across was Valdo, which contains a handful of templates for creating various types of projects in Vala.

How Do I Properly Write Unit Tests in Vala?

GLib Test was the natural starting place, and after poking around on how other people did testing, I landed on the following:

Directory structure

The root directory for all testing code is libsemver/tests/. That directory is included in the meson.build file in the root of the project.

In libsemver/tests/, we have all of our files containing the test cases. I chose to break it down such that there’s a 1:1 relationship between library sources files and test source files (e.g. Version.vala corresponds to VersionTest.vala). In my research I found some GNOME projects that broke out test cases into their own file, but this seemed to have the potential to get out of control very quickly.

In the tests meson.build, each test source file is listed and a loop is used to create separate test executables for each file:

unit_test_files = [
    'Specification',
    'Util',
    'Version'
]

foreach name : unit_test_files
    test(name, executable(name, name + 'Test.vala', libsemver_sources, dependencies: libsemver_deps, install: false))
endforeach

Notice that in reality just part of the test source file name is enumerated, and the full name is build later. This lets us have more user-friendly test case names when we go to run the tests.

Test Source Files

Lets take a quick look at the UtilTest.vala file which contains unit tests for a large-number arithmetic utility class I wrote to safely handle adding, subtracting, and comparison of numbers larger than what can be held in a uint64.

public class UtilTest : GLib.Object {

    /**
      * Tests large number addition.
      */
    private static void test_addition () {
        assert_true (Util.large_number_addition ("0", "0") == "0");
        assert_true (Util.large_number_addition ("1", "0") == "1");
        assert_true (Util.large_number_addition ("0", "1") == "1");
        assert_true (Util.large_number_addition ("1", "1") == "2");
        assert_true (Util.large_number_addition ("123", "58") == "181");
        assert_true (Util.large_number_addition ("999", "1") == "1000");
        assert_true (Util.large_number_addition ("1", "999") == "1000");
        assert_true (Util.large_number_addition ("99999999999999999999998", "1") == "99999999999999999999999");
        assert_true (Util.large_number_addition ("99999999999999999999999", "1") == "100000000000000000000000");
    }

    /**
      * Tests large number subtraction.
      */
    private static void test_subtraction () {
        assert_true (Util.large_number_subtraction ("0", "0") == "0");
        assert_true (Util.large_number_subtraction ("1", "0") == "1");
        assert_true (Util.large_number_subtraction ("1", "1") == "0");
        assert_true (Util.large_number_subtraction ("2", "1") == "1");
        assert_true (Util.large_number_subtraction ("181", "58") == "123");
        assert_true (Util.large_number_subtraction ("121", "58") == "63");
        assert_true (Util.large_number_subtraction ("58", "121") == "-63");
        assert_true (Util.large_number_subtraction ("0", "1") == "-1");
        assert_true (Util.large_number_subtraction ("99999999999999999999999", "1") == "99999999999999999999998");
    }

    public static void main (string[] args) {
        GLib.Test.init (ref args);
        GLib.Test.add_func ("/util/addition", test_addition);
        GLib.Test.add_func ("/util/subtraction", test_subtraction);
        GLib.Test.run ();
    }

}

In the main method we initialize GLib Test and then queue up our test functions. It’s all pretty straightforward, using assertions to verify that your code behaves as expected. However when we go to run this test, we get the following output:

$ ninja -C build test
[0/1] Running all tests.
1/1 Util                                    OK       0.01 s 

Ok:                    1
Expected Fail:         0
Fail:                  0
Unexpected Pass:       0
Skipped:               0
Timeout:               0

Well, ok, we can see everything passed, but even though we wrote 18 different assertions across two different test functions, it was all globbed into a single test entry. Not awful, but not ideal.

Now let’s see what happens if we have a failing test:

$ninja -C build test
[3/4] Running all tests.
1/1 Util                                    FAIL     0.01 s (killed by signal 6 SIGABRT)

Ok:                    0
Expected Fail:         0
Fail:                  1
Unexpected Pass:       0
Skipped:               0
Timeout:               0


The output from the failed tests:

1/1 Util                                    FAIL     0.01 s (killed by signal 6 SIGABRT)

--- command ---
16:07:52 /home/avojak/git/libsemver/build/tests/Util
--- stdout ---
# random seed: R02S9ff2e81c6ab0e5ea9e06492ec046b612
1..2
# Start of util tests
Bail out! ERROR:../tests/UtilTest.vala:14:sem_ver_util_test_test_addition: 'g_strcmp0 (_tmp1_, "1") == 0' should be TRUE
--- stderr ---
**
ERROR:../tests/UtilTest.vala:14:sem_ver_util_test_test_addition: 'g_strcmp0 (_tmp1_, "1") == 0' should be TRUE
-------

Full log written to /home/avojak/git/libsemver/build/meson-logs/testlog.txt
FAILED: meson-test 
/usr/bin/meson test --no-rebuild --print-errorlogs
ninja: build stopped: subcommand failed.
make: *** [Makefile:19: test] Error 1

Ok… we can conclude that we failed on the assertion on line 14, but the output is not exactly friendly… Our assertion was:

assert_true (Util.large_number_addition ("0", "0") == "1"); // Will fail intentionally

It’s easy to see how in a slightly more complicated test this would not be as simple to figure out. Additionally, if you use any sort of helper function in your testing to do assertions, there’s no trace. That means any tests using that helper function would show the same line number in case of a failure.

Conclusion

Writing unit tests with GLib Test feels a bit cludgy.

Poor feedback from failed assertions (need better messaging)
Poor visibility as to which test case failed (output is only on a class level, so if a class has multiple assertions they all get globbed together)

What’s Next?

Better Unit Testing

As a next step, I would really like to spend some time digging into a project that I recently discovered called Valadate, which is a testing framework built on top of GLib Test:

chebizarro/valadate

Although it is no longer maintained, I’ve forked the project and hope that it can still be of use.

I would also like to explore testing of Gtk widgets.

Deployment of Documentation

Although the library is currently configured to generate a documentation site using Valadoc, it’s not deployed anywhere. I would like to explore some options, including leveraging GitHub Pages for static hosting. One challenge here will be managing multiple versions of the documentation (e.g. 1.0.0 vs. latest).

avojak | blog

Installing ESXi with a USB NIC

1. Retrieve the ESXi 7.0 Offline Bundle

2. Download the USB NIC Fling

3. Build the new ISO

4. Create a Bootable USB Installer

5. Install and Expect a Failure!

Welcome to My Homelab!

Overview

The Hardware

Automation and Configuration-as-Code

Packer - Creating the OS Image

What is Packer?

Why use Packer?

How?

Terraform - Deploying the VMs

What is Terraform?

Why use Terraform?

How?

Ansible - Installing Software

What is Ansible?

Why use Ansible?

How?

End Result

References

DIY Built-in Bathroom Cabinet

The Space

Design

Materials

Cabinet Carcass

Face Frame

Doors

Installation

Finished Product

Lessons Learned

Grafana Dashboard for Pi-hole Stats

Updating Warble to GTK4

Updating the Dependencies

Helpful Resources

Input Handling

Keyboard Input

Mouse Input

Final Classes

Drawing with Cairo

Window Positioning

CSS Blur

Granite ModeButton No More

CSS Changes

End Result

Installing VMware ESXi on a Raspberry Pi

Prerequisites

Preparation

Prepare the SD Card

Copy the UEFI Firmware

Configure the UEFI Firmware

Prepare the ESXi Installer

Installation

Set Default Boot Device

ESXi Configuration

Adding Disk Space to an CentOS VM

Deploying Valadoc to GitHub Pages

Preparation

Generate Valadoc

Enable GitHub Pages

Setup the Action

End Result

Libsemver

How Are Libraries Written in Vala?

How Do I Properly Write Unit Tests in Vala?

Directory structure

Test Source Files

Conclusion

What’s Next?

Better Unit Testing

Deployment of Documentation

Granite `ModeButton` No More