Upgrade VM resources for MDS

[kmoc]

Hi

We are using CheckPoint MDS R80.10 running on VM. The VM has following parameters:
8 CPU Cores
32GB RAM
about 1TB Disk
We manage about 40 gateways and they generate about 200mln logs per day I guess because I see about 10 log files over 2GB size.
It works slowly especially when looking for logs.
I think that VM upgrade would help.

I have 2 questions:
1. What you would recommend in this situation?
2. Can I just add some CPUs/RAM to VM or I need to use special procedure for the upgrade like rebuild it or use different installation package?

Krzysztof

[PhoneBoy]

More RAM is definitely going to help with that many gateways.
In VM environments, disk I/O can be a bottleneck.

Also, it's probably a good idea to upgrade from R80.10 as numerous bug fixes and improvements have been made since then.
You can follow the standard upgrade process for that.

[HristoGrigorov]

If you are not already using XFS consider switching over to it.

[Timothy_Hall]

How many Domains/CMAs do you have?  The more of these you have the more RAM and CPU cores you will need. 

When running top during a busy period, what is the wio (waiting for I/O) percentage?  If it is >10% that indicates a slow/saturated disk path which will cause performance issues no matter how many cores you add.  Also before adding cores check your st (steal) percentage in top and sar as well, if it is nonzero you need to dedicate cores to your SMS VM in VMWare as it is not getting full usage of the 8 cores you already have from the hypervisor.

[Kaspars_Zibarts]

Just to give you some ideas for calculation 🙂

we get approx 250GB/day worth of logs on dedicated MLM. Our VM has 16 cores and 132GB memory.

Seems to work fairly OK with number of concurrent administrators. 

And that's a dedicated log server. So if you run mgmt and logs on the same machine you probably want to increase RAM.

Plus get the latest take, ideally upgrade as there has been many fixes regarding Java heap sizes etc. We are on R80.20 T155

[kmoc]

Thanks for the advice

I 've found that in normal situation the wa value is about 1% while when my colleague was watching on logs it increased to above 10% (maximum 16% I've seen). The st is always 0%.

But I've found load average was over 8% during that operation while normal is about 4%.

Krzysztof

[kmoc]

Thanks for all advises. All are valid and I have following plan:

1. Upgrade current version to latest hotfix to increase stability and fix problem with Java according to sk123417 recommendations.

2. Upgrade VM according to CheckPoint response:
In general a VM should have twice the HW recommended on a physical appliance for the environment to match the actual computing power required

3. Check if the VM uses SSD disks and try to change it to SSD if possible.

4. Consider upgrade to newer version R80.30 or R80.40 to be able using XFS.

What do you think about it?

Krzysztof

[Maarten_Sjouw]

I have 4 boxes running R80.30 with about 40 domains each with around 100 gateways per MDS. We run with 12 cores, 128GB mem and 6TB with XFS and I can tell you the 12 core is most of the times still not enough, but we deal with it.
If you have the luxary to use SSD's, always do.

For sure upgrade to R80.30 or even R80.40 by clean install and
advanced migration.

[Daniel_]

I also agree to use SSDs.
We are running 26 Domains on two MDS (primary,secondary). No dedicatet Logserver. We have 35 GB per day.
We run it on bare metal. We hav RAID10 SSDs with 384GB RAM. I/O wait is most often 0.0. During start we see it a litte bit over 1.

VMs are great for restore and upgrades but if you don't have a very good VM admin you are lost.....