Publish take so long time with lot of change

[Arthur_DENIS1]

Hi,

For one customer, with MGMT in R80.30 - open server, we have change a lot of rules (around 400) on one policy containing 5000 rules. And publish take few hours to complete !!!

As you can see, on the top below, server is not really overloaded.

Someone already get this issue and a way to solve this ?

Thanks for your help guys!

Arthur

top - 17:45:14 up 13 days,  7:51,  2 users,  load average: 1.68, 2.05, 2.05

Tasks: 231 total,   2 running, 229 sleeping,   0 stopped,   0 zombie

%Cpu(s): 27.1 us,  0.8 sy,  3.1 ni, 68.4 id,  0.5 wa,  0.0 hi,  0.1 si,  0.0 st

KiB Mem : 32846464 total,   615404 free, 12984148 used, 19246912 buff/cache

KiB Swap: 17840176 total, 17617344 free,   222832 used. 18440132 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND

12491 cp_post+  20   0  774048 746728 687892 R  75.1  2.3  77:10.47 postgres

5212 admin     20   0 6997584 1.143g   9704 S  72.4  3.6   1229:24 java

12497 cp_post+  20   0  770464 743240 687844 S  61.5  2.3  69:51.17 postgres

4987 admin     39  19 90.730g 6.484g 187260 S  15.9 20.7   4859:52 java

5137 admin     39  19  999044 383048   7540 S  10.0  1.2   1702:18 log_indexer

5929 admin     20   0  339524  34196  11316 S   4.3  0.1 834:04.65 lea_session

30135 admin     20   0  337056  36484  11336 S   3.0  0.1 696:04.91 lea_session

5092 admin     20   0 6899844 714356  10368 S   2.0  2.2 260:54.02 java

1665 admin     20   0  869948 489284  38084 S   1.3  1.5  34:19.38 fwm

4781 admin     20   0 1087456 313604  13292 S   1.3  1.0 312:18.45 fw_full

4672 admin     20   0   17456   1984   1756 S   0.7  0.0  10:48.43 cpwd

17680 cp_post+  20   0  708776 690520 687044 S   0.7  2.1  57:27.45 postgres

5035 admin     39  19 5805956 304128   8992 S   0.3  0.9  24:44.17 java

6381 admin     20   0   12604   3656   2960 S   0.3  0.0   0:01.60 sshd

17248 cp_post+  20   0  708812 690720 687056 S   0.3  2.1  57:32.87 postgres

    1 admin     20   0    2584    592    564 S   0.0  0.0   0:08.00 init

    2 admin     20   0       0      0      0 S   0.0  0.0   0:00.04 kthreadd

    3 admin     20   0       0      0      0 S   0.0  0.0   4:04.17 ksoftirqd/0

    5 admin      0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/0:0H

    7 admin     rt   0       0      0      0 S   0.0  0.0   0:03.98 migration/0

    8 admin     20   0       0      0      0 S   0.0  0.0   0:00.00 rcu_bh

    9 admin     20   0       0      0      0 S   0.0  0.0  17:04.23 rcu_sched

   10 admin     20   0       0      0      0 S   0.0  0.0   0:00.00 rcuob/0

   11 admin     20   0       0      0      0 S   0.0  0.0   2:02.09 rcuos/0

   12 admin     rt   0       0      0      0 S   0.0  0.0   0:03.73 watchdog/0

   13 admin     rt   0       0      0      0 S   0.0  0.0   0:03.39 watchdog/1

   14 admin     rt   0       0      0      0 S   0.0  0.0   0:03.46 migration/1

[Tomer_Noy]

The duration is extreme and it should not take so long.

Are all operations on the Management very slow, or just publish?
Are you just adding the 400 rules, or perhaps also making changes to thousands of objects?

If everything is slow, then perhaps there is some problem on the server (maybe VM on an overloaded ESX...).

In case it's not that, I suggest opening a ticket to TAC.

[HeikoAnkenbrand · Accepted solution]

Hi @Arthur_DENIS1,

Debug policy installation and related solutions:

• sk33207 - How to debug FWM daemon on Provider-1 CMA / Domain Management Server
• sk33208 - How to debug FWM daemon on Provider-1 / Multi-Domain MDS
• sk86320 - How to debug CPD daemon
• sk86321 - How to debug FWD daemon
• sk31874 - How to debug VSX Provisioning Process
• sk44338 - How to debug Policy Verification
• sk112334 - How to debug SmartConsole / SmartDashboard
• sk75200 - How to debug policy installation from Security Management / Provider-1 / Multi-Domain Serv...
• sk60745 - How to debug policy installation for UTM-1 Edge device on Security Management Server
• sk101226 - Policy installation flow process
• sk115557 - R80.x Security Management server main processes debugging

[HeikoAnkenbrand]

More read here:

R80.10 Management Performance Guide

[Arthur_DENIS1]

Hi both

Thanks a lot for your feedback!

Indeed, all action was very slow.... Solved by a reboot. However uptime was not really big...

Hope this is solved for a while, keep all the debug link ready 🙂

Nice day,

Arthur

[Jenni_Guerrica]

Was there any other resolution to this issue besides a reboot?  We have the same issue and a reboot does not help.