Trying to add hosts in mgmt_cli using csv file

[Ohad]

Dear all,

I have created a Black List group and every week I am adding roughly between 20 up to 40(ish) new hosts...

I have read that there is a possibility to use mgmt_cli in order to create all of them in one shot....

I have created the csv file in a UTF-8 format and used sk113078 as a guideline...

name,ip-address
Black_List_408,104.248.164.231
Black_List_409,118.70.113.1

These are the top three rows of the csv file...

when trying to execute the file I get the following:

[Expert@management:0]# mgmt_cli add host --batch blacklist.csv
Username: ohadadmin
Password:
Failed to read parameters file [blacklist.csv]: Failed to open file.

I have no idea what I am doing wrong here... and the Management API Reference is less than useless... There isnt even 1 example of how it should be...

Can anyone please assist...?

Also if anyone could educate me on how to add those objects to the "Black_List" group that I created (Preferably with the same csv file) I would really appreciate it

[_Val_]

Check the file is in the right directory and has at least read-only permissions for your API user

[Ohad]

Hi Val,

Thanks for the quick reply 🙂

The file is in the right directory...

I also did a #cat Blacklist.csv and saw that the rows are as the mentioned SK should...

Furthermore I also gave the file full permissions #chmod +rwx

[Maarten_Sjouw · Accepted solution]

Make sure to add the full path to the csv filename.
Check out this post:
https://community.checkpoint.com/t5/API-CLI-Discussion-and-Samples/Creating-multiple-object-via-CSV/...
Add a column in the csv named groups, and put the name of the group there.

[Ohad]

Full path did the trick!!!
Thank you so very much!
I've been wrecking my brain and it didnt even occur to me to try the full path....

And in sk113078 they dont use the full path.. Just the name.. Which is weird...
Also .\<Filename.csv> doesnt work either...

[Maarten_Sjouw]

Most probable: the path is related to the location of the file mgmt_cli....

[abnet]

Where did you use the full path, It is not working on my side I even tried it with different dirctory???? Using sk113078

[the_rock]

Here is example TAC gave me back in 2020:

--->To add address-range via API:
mgmt_cli add address-range --batch address-ranges_full.csv

#cat address-ranges_full.csv
name,ip-address-first,ip-address-last
range1,10.0.0.0,10.0.0.100

---> To add a network via API:
mgmt_cli add network --batch networks.csv

#cat networks.csv
name,subnet,subnet-mask
network1,10.10.10.0,255.255.255.0
network2,20.20.20.0,255.255.255.0
network3,30.30.30.0,255.255.255.0

---> To add a host 
mgmt_cli add host --batch test.csv

#cat test.csv
name,ip-address
obj1,192.168.1.1

Andy

[_Val_]

Correct, I believe the full path was a miss here

[the_rock]

Hey guys,

I know Im updating this thread late, but anyone has any idea as to what full path I should use to try import stuff from .csv file that I attached? By the way, I tried flags mentioned here, but sadly it failed.

Thanks

Andy

[Maarten_Sjouw]

You store the file on the management server, ie in the /home/admin/ directory, tehn you use mgmt_cli add host --batch /home/admin/tarion.csv

However, the file you have posted is not going to be importable, you need these fields: name, ipv4-address 
in your file I see lines that have an IP block as the first field, the name always has to start with a Character, not a number. but hosts require IP addresses and I do not see any on these lines.

[the_rock]

Thanks Maarten. I consulted with Ottawa tac and guy saw what I was doing and confirmed process was definitely correct, but it looks like excel was inserting a character where it was not supposed to : ). What he did was create a new txt file and then either use that or convert it into .csv, but either one works.

Thanks again! I wish this was easy like back in R77 and before where you could simply copy and paste objects in dashboard...I do see there is an option now to export all the objects, but its sort of useless really if you can never import them.

Thank you for the response and be well!

Andy

[genisis__]

Just come across this.

I've got a requirement to add about 1000 hosts to an existing group, so I can create the file with the hosts in there, but what I'm not sure about is how to then assign these hosts the existing group; if I was doing this one at a time I would do something like this:

add host name test_host1 ip-address a.b.c.d color "red" comments "test host" groups.1 testgroup
Note:
Above assumes the group 'testgroup' already exists and contains hosts.

How would I do this via the batch process?

Do I import the hosts first then have to run something separately to then add these in? i.e.

mgmt_cli set group --batch add_hosts_to_group.csv

Note:  The add_hosts_to_group.csv would be a separate file with just the hostnames in there?

Ideally I want to create the hosts and add them into the existing group as part of a single process using the batch process.

[Maarten_Sjouw]

While creating the hosts you add them directly to the group in the same way as you pointed out, only in the csv flie the first line is this:

name,ipv4-address,color,groups
test_host1,a.b.c.d,red,testgroup

Also here the rule applies, you need to make sure the group exists.

[the_rock]

Here is the answer TAC gave me last year when I had case about this and it did actually work. We tried doing this from actual csv file, but kept failing, so engineer said this is another option, where you actually do it from text file created on the firewall or premade, just in the right format.

--->To add address-range via API:
mgmt_cli add address-range --batch address-ranges_full.csv

#cat address-ranges_full.csv
name,ip-address-first,ip-address-last
range1,10.0.0.0,10.0.0.100

---> To add a network via API:
mgmt_cli add network --batch networks.csv

#cat networks.csv
name,subnet,subnet-mask
network1,10.10.10.0,255.255.255.0
network2,20.20.20.0,255.255.255.0
network3,30.30.30.0,255.255.255.0

---> To add a host 
mgmt_cli add host --batch test.csv

#cat test.csv
name,ip-address
obj1,192.168.1.1

[genisis__]

Also figured out some excel stuff (not an excel person).

scenario - you are just given a list of IPs to block in a spreadsheet.  Well now we want to create a node name thats meaningful.

Heres what I did

Column A (formula),   Column B (IP)

="EXT_"&B2&"_NODE", 1.1.1.1

The result in Column A would be 'EXT_1.1.1.1_NODE'

You take the cell with the formula (select from bottom right  of the cell, and copy this to all rest of the cells in Column A; saves allot of time if you have a 1000 IPs!

[genisis__]

Believe I figured it out:
I created a group 'testgroup' and added a node into this, then did the following as a test:

1 - create a file on the system with the following information
name,ip-address,color,comments,groups
EXT_1.1.1.1_NODE,1.1.1.1,red,test batch,testgroup
EXT_2.2.2.2_NODE,2.2.2.2,red,test batch,testgroup

Note:
When I imported a csv file I noted that dos2unix was required.

2 - ran the following:
mgmt_cli -d "a.b.c.d" add host --batch /var/log/tmp/test.csv
Note:
In our case we have an mds and therefore the -d and IP of the DMS is required.

Dare I now push it and change my question to the following:
Is is possible to create a group and then add hosts as a single process?

[Maarten_Sjouw]

You can create the group in a mgmt_cli command first and then do the import, do keep in mind that you don't import all 1000 in one go, I would opt for 2 x 350 and 300 and put a publish in between. To my knowledge there is a limit of 500 in one go.

[genisis__]

nice to know thanks,  It would nice if sk113078 noted this limit.

[the_rock]

Thats 100% true, because I also remember seeing that pop up on the screen when I used api objects import script from Eric Beasley

https://community.checkpoint.com/t5/API-CLI-Discussion-and-Samples/CLI-API-Example-for-exporting-imp...

[genisis__]

Strange I just imported 800 objects and no messages.