topic Re: Create/Update Azure Network/Group Objects for Public IP Space (77.30-below) in API / CLI Discussion

Create/Update Azure Network/Group Objects for Public IP Space (77.30-below)

Adam_Forester — Mon, 26 Jun 2017 13:09:59 GMT

## Version 2: Updated to support Ubuntu/Linux

Overview

The purpose of this code is to generate objects based on the Azure Public IP; Download Microsoft Azure Datacenter IP Ranges from Official Microsoft Download Center

The scripts were made for users of the R77 code that have to maintain Azure objects for rules.

Description

The code is to be used on systems 77.30 and below. For the R80 and above code please go here; Create objects for Azure Data-Center IP ranges - Python script

There are 3 scripts contained in the attached ZIP file. They are all Bash scripts;

Azure-get-public.sh – This is the main script you will execute. It will automatically download the latest Public_IP list from Microsoft and output the files needed for import.
Cp-grp-maker.sh – is called by the main script. This puts all the network objects into the dbedit format for a Simple Group
Cp-net-maker.sh – is called by the main script. Puts all subnets into the dbedit format for network objects.

Requires curl, wget, awk, cat, sed, and XMLSTARLET (this is used to parse Azure’s XML format)
The script will generate 3 dbedit files per Azure region;

Regionname-net-import.txt - Will create all the network objects for that region
Regionname-group-import.txt - Will create a simple group for that region and put all network objects for that region into the group.
Regionname-group-import-update.txt - This file is to be used to update groups that have already been built using the Regionname-net-import.txt script previously.

Instructions

Download the attached zip file.

Unzip the contents into a folder.

The script requires; curl, wget, awk, cat, sed, and XMLSTARLET (this is used to parse Azure’s XML format)

Ubuntu - apt-get install xmlstarlet
Mac - Use Homebrew - 'brew install xmlstarlet'

Exectute the script (make sure you have internet access) - ./azure-get-public.sh

The script will clean up any previous files from previous imports.
The script will call out to Microsoft to download the latest Public_IPs* list. Parses the XML for regions/subnets and puts them into a named file for each subnet and translates the Mask-length into a dotted format. Lastly, it runs those region files through the other scripts to create the dbedit outputs.
Default naming convention; NETWORK objects are named azure-regionname-x.x.x.x. GROUP objects are named azure-regionname.

The output is 3 dbedit files per Azure region;

Regionname-net-import.txt - Will create all the network objects for that region
Regionname-group-import.txt - Will create a simple group for that region and put all network objects for that region into the group.
Regionname-group-import-update.txt - This file is to be used to update groups that have already been built using the Regionname-net-import.txt script previously.

Move the files for each region you wish to create over to your Managment server. Follow the instructions in sk30383; Using a dbedit script to create new network objects and network object groups

NOTE: You must always import the NETWORK file before importing the GROUP file.

You can run this NETWORK script multiple times for updates. Each time the script is run dbedit will skip over objects that are already made. The Regionname-group-import-update.txt file will be used to update group objects that are already created.

GitHub for Code: GitHub - WadesWeaponShed/Azure-Region-Objects-R77_Below: This will allow you to build network and group objects per regi…

Code Version

Code version 1.0.0

Tested on version

R77 and below DBEDIT

NOTICE: By using this sample code you agree to terms and conditions in this Terms and Conditions

...

Re: Create/Update Azure Network/Group Objects for Public IP Space (77.30-below)

Wyatt_Felger — Thu, 13 Sep 2018 22:15:50 GMT

This document and script look great, but it appears that it may no longer be working. Is there some tweaks to the script that need to be made to ensure it continues to work.

Re: Create/Update Azure Network/Group Objects for Public IP Space (77.30-below)

Adam_Forester — Thu, 13 Sep 2018 23:54:02 GMT

Wyatt, Could you give me a little more detail. I just tried this on my 77.30 gateway/mgmt and it's still working

Re: Create/Update Azure Network/Group Objects for Public IP Space (77.30-below)

Wyatt_Felger — Fri, 14 Sep 2018 03:01:45 GMT

Checkpoint Community won't let me pm you the info, because you aren't following me. But here are the details.

I am running into issues when trying to run this in Ubuntu.

Downloading latest IP list
2018-09-13 20:59:54 URL:https://download.microsoft.com/download/0/1/8/018E208D-54F8-44CD-AA26-CD7BC9524A8C/PublicIPs_20180910.xml [95329] -> "PublicIPs_20180910.xml" [1]
./azure-get-public.sh: line 10: xml: command not found
There are regions
./azure-get-public.sh: line 14: xml: command not found
Making Individual Files for Regions
./azure-get-public.sh: line 16: xml: command not found
sed: invalid option -- 'g'

Re: Create/Update Azure Network/Group Objects for Public IP Space (77.30-below)

Adam_Forester — Fri, 14 Sep 2018 14:08:37 GMT

Did you install xmlstarlet on the box?

Re: Create/Update Azure Network/Group Objects for Public IP Space (77.30-below)

Wyatt_Felger — Fri, 14 Sep 2018 22:57:20 GMT

Yes it's installed on my Ubuntu box: xmlstarlet is already the newest version (1.6.1-2).

But please note your instructions have a typo for Ubuntu.

Ubuntu - apt-get install xmstarlet

^Missing a l, should be: Ubuntu - apt-get install xmlstarlet

Ubuntu - apt-get install xmstarlet

Are you able to still run this in Ubuntu?

Re: Create/Update Azure Network/Group Objects for Public IP Space (77.30-below)

Adam_Forester — Thu, 20 Sep 2018 16:48:02 GMT

Hey sorry, it took me a couple days to respond. I fixed it... apparently, in Ubuntu I couldn't use the short name of 'XML' for xmlstarlet. You can find the updated here.

GitHub - WadesWeaponShed/Azure-Region-Objects-R77_Below: This will allow you to build network and group objects per regi…

Re: Create/Update Azure Network/Group Objects for Public IP Space (77.30-below)

Rick_Rodrix — Wed, 26 Dec 2018 14:47:37 GMT

Greetings Adam.

Apparently, everything gone well around here. But on SmartDashboard I´m not be able to see the new objects created. Could you know what went wrong?

[Expert@fw-gerencia:0]# ls
brazilsouth-group-import.txt brazilsouth-net-import.txt
[Expert@fw-gerencia:0]# sed -i 's/[[:space:]]*$//' brazilsouth-net-import.txt
[Expert@fw-gerencia:0]# sed -i 's/[[:space:]]*$//' brazilsouth-group-import.txt
[Expert@fw-gerencia:0]# pwd
/home/admin
[Expert@fw-gerencia:0]# dbedit -local -globallock -f /home/admin/brazilsouth-net -import.txt
network_objects::azure-brazilsouth-104.41.0.0 Updated Successfully
network_objects::azure-brazilsouth-191.232.160.0 Updated Successfully
network_objects::azure-brazilsouth-191.232.192.0 Updated Successfully
network_objects::azure-brazilsouth-191.232.32.0 Updated Successfully
network_objects::azure-brazilsouth-191.233.0.0 Updated Successfully
network_objects::azure-brazilsouth-191.233.128.0 Updated Successfully
network_objects::azure-brazilsouth-191.233.130.0 Updated Successfully
network_objects::azure-brazilsouth-191.233.132.0 Updated Successfully
network_objects::azure-brazilsouth-191.233.136.0 Updated Successfully
network_objects::azure-brazilsouth-191.233.192.0 Updated Successfully
network_objects::azure-brazilsouth-191.233.24.0 Updated Successfully
network_objects::azure-brazilsouth-191.234.160.0 Updated Successfully
network_objects::azure-brazilsouth-191.235.196.0 Updated Successfully
network_objects::azure-brazilsouth-191.235.200.0 Updated Successfully
network_objects::azure-brazilsouth-191.235.224.0 Updated Successfully
network_objects::azure-brazilsouth-191.235.240.0 Updated Successfully
network_objects::azure-brazilsouth-191.235.248.0 Updated Successfully
network_objects::azure-brazilsouth-191.235.32.0 Updated Successfully
network_objects::azure-brazilsouth-191.235.64.0 Updated Successfully
network_objects::azure-brazilsouth-191.237.195.0 Updated Successfully
network_objects::azure-brazilsouth-191.237.200.0 Updated Successfully
network_objects::azure-brazilsouth-191.237.248.0 Updated Successfully
network_objects::azure-brazilsouth-191.238.128.0 Updated Successfully
network_objects::azure-brazilsouth-191.238.192.0 Updated Successfully
network_objects::azure-brazilsouth-191.239.112.0 Updated Successfully
network_objects::azure-brazilsouth-191.239.204.0 Updated Successfully
network_objects::azure-brazilsouth-191.239.240.0 Updated Successfully
network_objects::azure-brazilsouth-20.190.145.0 Updated Successfully
network_objects::azure-brazilsouth-20.40.112.0 Updated Successfully
network_objects::azure-brazilsouth-20.40.16.0 Updated Successfully
network_objects::azure-brazilsouth-20.40.32.0 Updated Successfully
network_objects::azure-brazilsouth-23.97.96.0 Updated Successfully
network_objects::azure-brazilsouth-40.126.17.0 Updated Successfully
network_objects::azure-brazilsouth-40.90.133.32 Updated Successfully
network_objects::azure-brazilsouth-40.90.141.64 Updated Successfully
network_objects::azure-brazilsouth-40.90.144.224 Updated Successfully
network_objects::azure-brazilsouth-52.108.36.0 Updated Successfully
network_objects::azure-brazilsouth-52.109.108.0 Updated Successfully

Re: Create/Update Azure Network/Group Objects for Public IP Space (77.30-below)

Adam_Forester — Mon, 31 Dec 2018 18:08:18 GMT

Rick, just got back from Holiday so sorry for the delay. Did you get this figured out? My first questions would be is this an MDS environment?