ZFS on Proxmox

What Is ZFS?

ZFS is a combined filesystem and volume manager originally built at Sun Microsystems for Solaris, and now maintained as OpenZFS across Linux, FreeBSD, and illumos. Proxmox VE ships it as a first-class storage option, both for the host’s own root filesystem and for VM and container disks.

Think of ZFS as a filesystem that never trusts the disk. Every block it writes gets a checksum, and every read verifies that checksum before handing data back. Traditional RAID controllers assume a disk that returns data is returning correct data; ZFS assumes nothing and catches silent corruption that a hardware RAID card would quietly pass along.

A ZFS pool is built from one or more vdevs (virtual devices), each a mirror or a parity group of physical disks. The pool stripes data across its vdevs. Inside the pool, you carve out datasets, individually mountable filesystems (or zvols, raw block volumes) that each inherit pool-wide settings like compression but can override them.

Why ZFS Instead of ext4 + LVM?

Proxmox’s default local install uses ext4 on LVM-thin, which works, but each piece solves a narrower problem on its own. ZFS folds RAID, volume management, filesystem, and snapshotting into a single layer that knows about all of it at once.

The trade-off is memory and CPU: ZFS wants RAM for its cache and, on RAIDZ, pays a parity write penalty that mirrors don’t. For a homelab node with a handful of disks and enough memory, that trade is usually worth it for the snapshot and integrity story alone.

Why Proxmox Cares About ZFS

Proxmox’s own snapshot, clone, and live-migration-with-local-storage features are built directly on ZFS’s copy-on-write model. When you create a VM on ZFS-backed storage, Proxmox provisions a zvol for its disk; a “Proxmox snapshot” on that VM is a ZFS snapshot underneath, which is why it commits in a second rather than copying gigabytes.

ZFS also removes the dependency on a hardware RAID controller. Proxmox explicitly recommends against RAID cards doing their own caching under ZFS, since ZFS needs direct, honest communication with each disk. An HBA in IT mode, or onboard SATA/NVMe presented as individual disks, is the expected setup.

Pool Layouts and ashift

Proxmox supports the full set of vdev layouts from a single-disk stripe up through triple-parity RAIDZ and the newer dRAID variants for large disk counts. The layout decides both fault tolerance and usable capacity, and it can’t be changed after the pool is created without destroying and rebuilding it.

ashift sets the pool’s block size as a power of two: ashift=12 means 4096-byte blocks, the right default for essentially all modern SSDs and HDDs. Set it too low for the underlying disk’s real sector size and every write pays an unnecessary read-modify-write penalty, permanently, for the life of the vdev.

# Create a mirrored pool (RAID1) from two disks - the everyday homelab choice
zpool create -f -o ashift=12 tank mirror /dev/sdb /dev/sdc

# Create a RAIDZ-2 pool from four disks - more usable capacity, tolerates two failures
zpool create -f -o ashift=12 tank raidz2 /dev/sdb /dev/sdc /dev/sdd /dev/sde

# Create striped mirrors (RAID10) from four disks - best random IO for VM workloads
zpool create -f -o ashift=12 tank mirror /dev/sdb /dev/sdc mirror /dev/sdd /dev/sde

Datasets and Compression

Once a pool exists, datasets are how you actually use it. Each dataset inherits pool-wide defaults but can override compression, quotas, and other properties independently, which is why a homelab typically has separate datasets for VM disks, ISOs, and backups rather than dumping everything at the pool root.

# Create a dataset for VM disk storage
zfs create tank/vm-storage

# Enable lz4 compression - low CPU overhead, good ratio, Proxmox's own recommended default
zfs set compression=lz4 tank/vm-storage

# Check current compression ratio achieved on a dataset
zfs get compressratio tank/vm-storage

# List all datasets and their space usage
zfs list

Compression only applies to blocks written after it’s enabled, not retroactively, so turn it on before loading data rather than after.

Monitoring and Maintenance Commands

# Check pool health - look for state: ONLINE and no errors
zpool status

# List pools with capacity and fragmentation
zpool list

# Verbose per-disk breakdown of a specific pool
zpool list tank -v

# Take a snapshot of a dataset before a risky change
zfs snapshot tank/vm-storage@before-upgrade

# List all snapshots
zfs list -t snapshot

# Roll back to a snapshot, destroying anything newer (-r required if not the latest)
zfs rollback -r tank/vm-storage@before-upgrade

# Start a manual scrub to verify checksums across the whole pool
zpool scrub tank

# Check scrub progress or last scrub result
zpool status tank

ARC Memory Tuning

ZFS caches recently and frequently used blocks in RAM, called the Adaptive Replacement Cache (ARC). By default it claims 10% of host memory, capped at 16 GiB, which on a memory-tight homelab node can crowd out VM RAM allocations if left unchecked.

# Check current ARC size
cat /proc/spl/kstat/zfs/arcstats | grep ^size

# Temporarily cap ARC at 8 GiB for this boot
echo "$[8 * 1024*1024*1024]" >/sys/module/zfs/parameters/zfs_arc_max

# Make the 8 GiB cap permanent
echo "options zfs zfs_arc_max=8589934592" >> /etc/modprobe.d/zfs.conf
update-initramfs -u -k all

As a starting point, Proxmox suggests roughly 2 GiB base plus 1 GiB per TiB of pool capacity for the ARC, then adjusting based on how much RAM the VMs on that node actually need.

Troubleshooting Cheat Sheet

Final Thoughts

ZFS is the reason Proxmox’s snapshot and clone workflow feels instant instead of like a multi-gigabyte copy operation, and the checksum-everything design is worth the memory cost on any homelab node that has data you’d actually miss. Pick the layout deliberately, since RAID level and ashift are permanent once a vdev exists, and give the ARC a ceiling before it competes with your VMs for RAM.

Start with a mirror for anything running VMs, keep compression on by default, and put a scrub on the calendar. That combination catches most of the failure modes that turn a homelab outage into a real data-loss event.