Physical hosts with Ansible¶

Omni builds the Talos VMs and Argo CD runs everything inside Kubernetes. The machines underneath them — five Proxmox hosts and the management Pi — are plain Debian systems that Argo CD cannot reach. Their operating-system settings live in host-monitoring/ and are applied with Ansible from a workstation.

Status: current. Playbooks are applied by hand. Merging a pull request changes nothing on a host until someone runs the playbook against it.

What Ansible manages¶

Playbook	What it installs or changes	Hosts	Full guide
`playbook.yaml`	`node_exporter` (CPU, RAM, disk I/O on `:9100`) and `smartctl_exporter` (drive health on `:9633`) for Prometheus	Proxmox, Pi	README
`logging-playbook.yaml`	Persistent journal plus an OpenTelemetry Collector that ships it to Loki (Grafana Nodes / Crash logs)	Proxmox, Pi	LOGGING.md
`crash-capture-playbook.yaml`	Lockups become panics that reboot, panic text saved to EFI pstore, chipset hardware watchdog, kernel messages streamed to the Pi with netconsole	Proxmox, one host at a time	LOGGING.md § Crash capture

TrueNAS is an appliance and is not changed by Ansible. Its exporters run as a TrueNAS Custom App from truenas-compose.yaml.

Ansible does not manage the Talos VMs (the Omni Proxmox provider builds them from omni/machine-classes/), Proxmox network interfaces, USB and PCI resource mappings, or Proxmox package upgrades.

The inventory¶

inventory.yaml names every machine. Use these names with --limit.

Group	Names	SSH user
`proxmox`	`gpu` (.14), `shed` (.20), `sff` (.21), `elite` (.22)	`root`
`management`	`pi` (.15)	`vanillax` with sudo
`appliances`	`nas` (.133)	listed for reference; no playbook targets it

A setting that only one host needs is a host variable in the inventory, so it survives the next run. Changing a host by hand works only until a playbook runs again and puts the declared state back.

Prerequisites¶

Run everything from the repository root.

Ansible on your workstation: ansible-playbook --version prints a version.
The pinned collection: ansible-galaxy collection install -r host-monitoring/requirements.yaml
SSH key login to each host, then confirm Ansible can reach them:
```
ansible -i host-monitoring/inventory.yaml physical_linux -m ping
```
Expected: every host answers "ping": "pong". Fix SSH before continuing if any host is UNREACHABLE.

Run a playbook safely¶

Replace <playbook> with a file from the table and <host> with an inventory name.

Check the syntax.

ansible-playbook -i host-monitoring/inventory.yaml host-monitoring/<playbook> --syntax-check

Preview one host. --check --diff shows each file it would change without changing it.
```
ansible-playbook -i host-monitoring/inventory.yaml host-monitoring/<playbook> --limit <host> --check --diff
```
Check mode skips shell and command steps, and a step can fail because a file created earlier in the same run does not exist yet. Read those failures; they do not always mean the real run will fail.
Apply to that host.
```
ansible-playbook -i host-monitoring/inventory.yaml host-monitoring/<playbook> --limit <host>
```
Expected: PLAY RECAP shows failed=0. Running it again shows changed=0, because a playbook only changes what differs.
Verify with the checks in that playbook's guide.
Repeat for the remaining hosts, one at a time.

Playbooks never reboot a host. A kernel boot option they add takes effect at the next reboot.

Rollback¶

Each guide lists the files and services its playbook creates. To undo one, stop or remove those on the affected host, then revert the pull request so the next run does not put them back.