topic Re: Ansible Connection Error when playbook with large group member in Ansible

Ansible Connection Error when playbook with large group member

mbouri — Thu, 28 May 2020 17:45:43 GMT

Hello

I have an issue when I try to create or update group with large member (>3000 member), the playbook work fine with small group, is there any option to enable on ansible config settings or checkpoint SMS side ?

my playbook :

---
- name: Deploy Grp
hosts: check_point
connection: httpapi

tasks:

- name: include
include_vars: 'var.yaml'

- name: add-group
cp_mgmt_group:
name: "gp-name"
ignore_warnings: yes
members: "{{name}}"
- name: publish
cp_mgmt_publish:

var file :

name:

- h-name1

- h-name2

....

hosts.ini file :

[check_point]
IP_SMS
[check_point:vars]
ansible_httpapi_use_ssl=True
ansible_httpapi_validate_certs=False
ansible_user=admin
ansible_password=password
ansible_network_os=checkpoint

the error message :

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ansible.module_utils.connection.ConnectionError: The read operation timed out
fatal: [10.1.1.100]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"/var/lib/jenkins/.ansible/tmp/ansible-local-37897pe768pwq/ansible-tmp-1590687244.472336-2766174878375/AnsiballZ_cp_mgmt_group.py\", line 102, in <module>\n _ansiballz_main()\n File \"/var/lib/jenkins/.ansible/tmp/ansible-local-37897pe768pwq/ansible-tmp-1590687244.472336-2766174878375/AnsiballZ_cp_mgmt_group.py\", line 94, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/var/lib/jenkins/.ansible/tmp/ansible-local-37897pe768pwq/ansible-tmp-1590687244.472336-2766174878375/AnsiballZ_cp_mgmt_group.py\", line 40, in invoke_module\n runpy.run_module(mod_name='ansible.modules.network.checkpoint.cp_mgmt_group', init_globals=None, run_name='__main__', alter_sys=True)\n File \"/usr/lib64/python2.7/runpy.py\", line 176, in run_module\n fname, loader, pkg_name)\n File \"/usr/lib64/python2.7/runpy.py\", line 82, in _run_module_code\n mod_name, mod_fname, mod_loader, pkg_name)\n File \"/usr/lib64/python2.7/runpy.py\", line 72, in _run_code\n exec code in run_globals\n File \"/tmp/ansible_cp_mgmt_group_payload_wCTvQX/ansible_cp_mgmt_group_payload.zip/ansible/modules/network/checkpoint/cp_mgmt_group.py\", line 140, in <module>\n File \"/tmp/ansible_cp_mgmt_group_payload_wCTvQX/ansible_cp_mgmt_group_payload.zip/ansible/modules/network/checkpoint/cp_mgmt_group.py\", line 135, in main\n File \"/tmp/ansible_cp_mgmt_group_payload_wCTvQX/ansible_cp_mgmt_group_payload.zip/ansible_collections/check_point/mgmt/plugins/module_utils/checkpoint.py\", line 256, in api_call\n File \"/tmp/ansible_cp_mgmt_group_payload_wCTvQX/ansible_cp_mgmt_group_payload.zip/ansible_collections/check_point/mgmt/plugins/module_utils/checkpoint.py\", line 63, in send_request\n File \"/tmp/ansible_cp_mgmt_group_payload_wCTvQX/ansible_cp_mgmt_group_payload.zip/ansible/module_utils/connection.py\", line 185, in __rpc__\nansible.module_utils.connection.ConnectionError: The read operation timed out\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

Re: Ansible Connection Error when playbook with large group member

PhoneBoy — Thu, 28 May 2020 19:47:52 GMT

I imagine the API is timing out with this request.
Recommend getting the TAC involved.
Also tagging @Omer_Kleinstern

Re: Ansible Connection Error when playbook with large group member

Or_Soffer — Sun, 31 May 2020 09:36:38 GMT

Hey @mbouri ,

let's continue this conversation on mail.
Please contact me by mail - orso@checkpoint.com

Thanks,

Re: Ansible Connection Error when playbook with large group member

PhoneBoy — Mon, 01 Jun 2020 15:50:46 GMT

I'll provide out of band

Re: Ansible Connection Error when playbook with large group member

IgorBodrik — Tue, 12 Apr 2022 19:22:35 GMT

Hello, guys! I have the same issue. I need to update group containing ~3000 objects. Do you have a solution for this?

Re: Ansible Connection Error when playbook with large group member

Art_Zalenekas — Tue, 12 Apr 2022 19:24:23 GMT

Try to do a publish every 100 objects.

Re: Ansible Connection Error when playbook with large group member

IgorBodrik — Tue, 12 Apr 2022 19:33:15 GMT

How can i do that?

The error appears in a block:

- name: add-group
cp_mgmt_group:
name: "gp-name"
ignore_warnings: yes
members: "{{name}}"

{{ name }} - is a huge array ~3000 elements.

Re: Ansible Connection Error when playbook with large group member

Art_Zalenekas — Tue, 12 Apr 2022 20:49:23 GMT

Take a look at an example that I have posted before. This should be very similar if you are familiar with looping in Ansible.
https://community.checkpoint.com/t5/Ansible/Ansible-Improving-Performance/m-p/141062/highlight/true#M636

Re: Ansible Connection Error when playbook with large group member

IgorBodrik — Tue, 12 Apr 2022 21:14:45 GMT

@Art_Zalenekas Thanks for your response. But i don't need to loop over group members and publish each N changes. If I did this, I would end up with a group with one element.

Re: Ansible Connection Error when playbook with large group member

Art_Zalenekas — Wed, 13 Apr 2022 21:34:21 GMT

True. Through the API directly you could do add a member to a group. Through Ansible you cannot do it that way. I tested adding 3,000 members into a group, and that worked. What I had to do is increase the connection timeout to 120 (2 minutes) for the httpapi plugin of a task from the Ansible perspective.
https://docs.ansible.com/ansible/latest/collections/ansible/netcommon/httpapi_connection.html
Look at the variable persistent_command_timeout
I would highly recommend that you add/set the group membership of an object when you are adding those objects. That way you don't have to care about the group itself, but its existence/creation.

Good luck!

Re: Ansible Connection Error when playbook with large group member

IgorBodrik — Thu, 28 Apr 2022 08:28:04 GMT

Hi! I tried to tune ANSIBLE_PERSISTENT_CONNECT_TIMEOUT and ANSIBLE_PERSISTENT_COMMAND_TIMEOUT variables. It didn't help. In my case the error messaged appears after ~2 min, while timeouts are more than that.

Actually on the first run when the group is empty it worked. Sorry, I didn't mention that. Try to run such playbook second time. Maybe you will see the error.

Thank you for your advise about setting membership in object. It works great for initial objects creation. But it would be inconvenient to have infrastructure config in such format.

Re: Ansible Connection Error when playbook with large group member

Art_Zalenekas — Thu, 28 Apr 2022 12:18:50 GMT

You might be having performance issues with your management. My testing worked. Increase the timeout to 5 minutes.