FUCKUP #004: The Filesystem Resize That Failed — But Was Already Done
The symptom The incident did not start with somebody watching the VM boot. It started with monitoring: Checkmk reported a problem on the VM, which triggered the investigation. The exact Checkmk service and alert message were not recorded. During that investigation, a recent storage change and an alarming boot-time error became relevant. The Linux VM in Proxmox had its virtual disk expanded from 20 GB to 25 GB. During the next boot, cloud-init attempted to grow the root filesystem. The relevant operation was effectively: ...