I’m keeping track of the write performance of various disk configurations, so I can get a kind of fuzzy benchmark about how a particular configuration is performing. I only care about fully synchronous write performance, I don’t measure read performance at all.

A few observations:

  • Optane is really insane. If you want maximum performance, this is the way to go. It’s not even close.
  • For bulk media storage, 5k4 disks perform just as well as 7k2.
  • External read and write caches provide nothing unless you’re running in an environment where you absolutely cannot lose data.
  • There are a lot of shitty SSD’s out there. Be careful.

The command I use for my benchmarks is:

iozone -i 0 -i 1 -r 8k -a -o

Here are the completely unscientific results:

# of Disks Make & Model Medium Transit Filesystem External Cache Kbytes/sec
1 Intel Optane 905p 3D XPoint NVME ext4 No 131,256
1 Samsung PM863a 240GB SSD SATA ext4 No 74,383
2 Seagate 600GB 15k SAS ZFS Mirror ZeusRAM ZLOG & L2ARC (LSI) 65,723
1 Intel SSDSC2BB800G6 SSD SATA ext4 No 65,513
1 Zeus IOPS SSD SAS ext4 No 64,358
16 Assorted 7k2 & 5k4 eSATA ZFS 4xRAIDZ1 ZeusRAM ZLOG & L2ARC 62,372
2 Seagate 600GB 15k SAS via LSI ZFS Mirror, Dedupe, LZ4 ZeusRAM ZLOG & L2ARC 58,062
6 Assorted 10k eSATA ZFS 3xMirror Mirror ZeusRAM ZLOG 54,585
15 Assorted 7k2 & 5k4 SAS ZFS 3xRAIDZ1 Mirror ZeusRAM ZLOG 52,554
2 Samsung PM863a 240GB SSD SATA ZFS Mirror, Dedupe, LZ4 43,438
1 Intel SSDSC2BB800G4 SSD SATA ext4 No 31,626
16 Assorted 7k2 & 5k4 eSATA ZFS 4xRAIDZ1 ZeusRAM ZLOG (Adaptec) 24,738
66 Assorted 7k2 NFS Netapp WAFL No 22,395
24 Assorted 400GB SSD NFS Netapp WAFL No 22,019