Which Filesystem to use on AFS data partitions?

XFS used to be the filesystem of choice on my AFS fileservers. However - when I moved to RAID5, it turned out to be a showstopper. Especially volume clone operations (backup,release,move,remove,...) were as slow as 21 files/s (time for cloning volumes correlates with the number of files in them).

So I decided to compare some filesystems. This is, what they had to do:

• mkfs - Be created
• restore - Restore a volume from a dump file over the network
• salvage - The salvager had to check the newly restored volume instance
• fsck - The filesystem was checked offline
• backup - The volume was vos backup ed (cloned)
• remove backup - The backup was removed (the backup clone was removed)
• remove - The volume was removed (physical file deletion)

The test equipment

• AFS-Fileserver:
  • 2x Xeon 2.66 GHz , 2GByte Memory
  • Intel 82540EM Gigabit NIC
  • Storage system
    • 3Ware 39xx SATA-Raid controller
    • 8x Maxtor 7Y250M0, 233 GiByte^?
    • RAID5 over 8 disks, 1.59 TiByte^?, stripe size 16kiByte, writeback cache enabled
  • Openafs 1.4.4
  • Test volume
    • 10 x Recent linux kernel source tree
    • 193141 files
    • 1,88 GiByte^?
• AFS-Client (for Volume Restore)
  • P4 2,4 GHz, 3,0 GByte Memory
  • Intel ns83820 Gigabit NIC
  • Openafs 1.4.4

Hard facts

All results (except of the fps values) are given in seconds which means: the smaller the better. wc_clone is the worst case speed of clone operations (clone or remove clone) on a given filesystem.

Conclusion

XFS seems to be a bad choice for AFS-on-RAID. Reiserfs3 ist the best choice in most cases. Ext3 would be equally good, if it hadn't such bad fsck and salvage times (~4.5x time of reiserfs3). You have to decide, if you want to use a probably more mature filesystem which is basically on any linux computer (ext3) or if you prefer saving 77% of the time needed for a possibly necessary filesystemcheck in case of a crash. Keep in mind that ext3 sometimes thinks , it's necessary to fsck (e.g. after 1 year or 26x mounting).

-- FrankBurkhardt - 20 Jul 2007
[Zurück zum Start]