Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
Kaspars_Zibarts
Authority
Authority

Real life comparison of XFS and EXT3 file systems

To give a bit historic background - XFS was introduced in R80.20. The caveat was that if you were doing upgrades from R80.10 or earlier version using CPUSE, it would keep existing EXT3 file system and you would miss out on apparently faster XFS. The only way to change the file system to XFS would be by full fresh install from external disk, which for many of us meant a hard choice of being forced to rely on CPUSE upgrade and keeping EXT3 as boxes were in remote locations and cost of the travel and time of full box reinstall was too high.

I had a bit of lab time today and decided to test the difference between two file systems as we have many 5900 clusters that were introduced back in 2017 with R80.10 onboard = EXT3 file system. As mentioned above, we decided to perform upgrades to R80.30 and R80.40 using CPUSE, thus keeping original file system.

And the main reason for the test actually was the fact that snapshot creation on EXT3 boxes puts insane strain on CPU usage - they almost become unmanageable with multiple CPU cores running zero CPU idle time.

Results actually exceeded my expectations. In case you wanted to know 🙂

Test XFS EXT3
Snapshot creation 3 mins Minimal CPU impact 15 mins System becomes nearly unusable
HFA install 17 mins   21 mins Fair impact on CPU
Reboot time 4 mins   6 mins  
Write speed large file 100MB/s   27MB/s  
Write speed small files 0.6-1.6GB/s Varied a lot! 0.9GB/s Fairly consistent

 

Just to add if you wanted to know how to check it:

image.png

9 Replies
Daniel_
Collaborator

Creation snapshot with BTRFS: less then a second 😏

0 Kudos
genisis__
Advisor

This is awesome information!

Are these results from a gateway or a Manager?  Everything I know about the XFS filesystem suggested there was no real benefit to do a clean build on a gateway, but certainly was worth doing it at the management layer, especially on a logging server.

0 Kudos
Kaspars_Zibarts
Authority
Authority

Sorry - I should have been more clear - it's definitely gateways, not management. For management there was no question from the start - XFS was a MUST. And gateways I was doubtful as all they do is write a bit of local logging, but I was really wrong - there are very good reasons to update file system even on gateways, like taking snapshots during daytime which is impossible with EXT3 as it will affect traffic. And faster boot times. And faster HFA installations. For me it's all valid operational reasons.

0 Kudos
genisis__
Advisor

Awesome!

I think CP should be adding these figures into there documentation! 

Gives technical resources real figures to justify clean installation to the business.

0 Kudos
PhoneBoy
Admin
Admin

Just to clarify, XFS was available on anything that has a Linux 3.10 kernel.
That meant initially on the management in R80.20 but didn’t become mainstream on gateways until R80.40.
And yes, it requires a fresh install from ISO to convert to the new filesystem since we use a different partitioning scheme as well.

In any cases this is exceptionally useful information you’ve provided here.

Timothy_Hall
Champion
Champion

Wow, what a difference!  Thanks for the post.

New 2021 IPS/AV/ABOT Immersion Self-Guided Video Series
now available at http://www.maxpowerfirewalls.com
0 Kudos
Kaspars_Zibarts
Authority
Authority

I know! Totally changed my mind about having XFS on gateways! 🙂 will be making plan of upgrading them all! 🙂

0 Kudos

XFS was developed by SGI to run on their Cray mainframes and later "donated" to Linux community.  As such it is targeted to work best with large files. On the down side it uses aggressive caching and in general eats a bit more memory. I was a great fan of it until BTRFS appeared. I wonder if CheckPoint is going to stick with XFS in the future or it is planning to support other file systems as well... 

0 Kudos
Bob_Zimmerman
Advisor

Eh. BTRFS's desirable features are way too immature for my taste. Needs another few years before I'll trust it on my management servers, let alone my firewalls. On the plus side, offering it would mean moving away from RHEL (which removed all support in RHEL 8), which could delay the systemd nonsense. I am absolutely never allowing systemd anywhere near my firewalls.

ZFS, on the other hand ...

draid2:1s means never having to say you're sorry.

0 Kudos