No, specifically multi-tenant, cgroups workloads. It works great for single-tenant workloads, esp. when you can tune the ARC to your workload. I know some folks who run it for large data workloads, and it works awesome for them. Unfortunately, our large container workload hasn't had the same luck.
Unfortunately, these problems stem from page faults occurring inside of VM that propagate to ZFS. If the fault occurs during cgroup memory pressure, ZFS / SPL may fail to allocate memory. ZFS will never get out of this case unless memory is freed up elsewhere in the hierarchy.
The other issue we had was with ZFS integration. There are a few symbols around mounts that are exported GPL only, and they interfere with volume GC, and mounting, causing some issues with standard tools like Systemd, and others.
For the most part, our container applications are stateless, or soft state, in this, we don't rely on RAID1, 5, 6, etc... but instead RAID0, or no RAID at all. If we detect a bad disk, we'd rather just evacuate the containers, and restore state later.
I'd love to hear about your experience with BtrFS. What issues have you seen with it (single disk, or RAID0 -- I know RAID1, 5, and 6 are hokey at best)? How did the project handle your issues?
Yeah. These ZFS issues look like they are deal breakers. In this use case btrfs could really be a good idea - afaik cgroup and kernel integration is better there.
> I'd love to hear about your experience with BtrFS. What issues have you seen with it (single disk, or RAID0 -- I know RAID1, 5, and 6 are hokey at best)? How did the project handle your issues?
Single disk, no RAID - lockups, broken filesystems (undeletable files), filesystem not mountable - however in recent kernels things are likely better (>4.8).
Performance and defrag are issues on btrfs, scrub and balance take all io and there are some warts... might be good enough for your use case, through. RAID-1 is not really double read-throughput, not sure how RAID-0 is implemented.
There is a mailinglist and and a bugzilla but the ZFS issue tracker was always more helpful.
The issues I've hit are the following:
https://github.com/zfsonlinux/zfs/issues/5814
https://github.com/zfsonlinux/zfs/issues/5535
Unfortunately, these problems stem from page faults occurring inside of VM that propagate to ZFS. If the fault occurs during cgroup memory pressure, ZFS / SPL may fail to allocate memory. ZFS will never get out of this case unless memory is freed up elsewhere in the hierarchy.
The other issue we had was with ZFS integration. There are a few symbols around mounts that are exported GPL only, and they interfere with volume GC, and mounting, causing some issues with standard tools like Systemd, and others.
For the most part, our container applications are stateless, or soft state, in this, we don't rely on RAID1, 5, 6, etc... but instead RAID0, or no RAID at all. If we detect a bad disk, we'd rather just evacuate the containers, and restore state later.
I'd love to hear about your experience with BtrFS. What issues have you seen with it (single disk, or RAID0 -- I know RAID1, 5, and 6 are hokey at best)? How did the project handle your issues?