topic CPViewer - visualize your cpview/cpinfo files in 5 minutes in Firewall and Security Management

CPViewer - visualize your cpview/cpinfo files in 5 minutes

Petar_Markota — Sun, 30 May 2021 17:49:14 GMT

What is CPViewer?

CPViewer is an open-source community tool which simplifies the way to get a very detailed visual insights into:

1) Exported cpview histories with charts related to CPU, memory, connections and packet rates, throughputs, RX&TX drops etc. using the visualization metrics tool called Grafana;

2) OS analysis (.info file) - healthcheck (sk121447) report for "known issues" and "things not to miss";

The tool comes within an .ova (exported VM) with everything already preconfigured.

The main benefits are:

analyzing and identifying performance bottlenecks in minutes
user-friendly: just upload it to web-site (no docker or other components needed)
Integrated OS analysis tool for richer results (healthcheck script)
available to both customers and partners

CPViewer .ova file download - HERE.

CPViewer guide - HERE.

How to use it?

Video below explains all you need to do to get CPViewer up and running - 5 simple steps:

Written step by step guide:

1. Import the VM:

a. Download and import OVA image into your VMWare environment – DOWNLOAD LINK.

b. VM’s network adapter is set to NAT, it has IPv4 – 10.8.0.15, default gw – 10.8.0.2 and DNS – 8.8.8.8 predefined already, but you can adjust this by your needs;

c. Adjust your VMWare NAT adapter;

d. Credentials:
- OS: root/vpn123
- Grafana: admin/Vpn123!

*NOTE: Your VM must have internet access if you want to use GDrive download functionality.

2. Working with CPViewer portal:

a. CPViewer portal can process two types of data:

1) CPInfo files (contains cpinfo and cpview files) -> you will get 2 reports, grafana cpview insights and cpinfo OS analysis report (in separate tab);

2) CPView (.dat or .gz – with .dat in it) files only;

*NOTE: In case you are using type 1, please be aware that you need to either configure your browser to allow pop-ups for http://10.8.0.15:80 in order to get the CPInfo healthcheck report. Other option is just to go to http://10.8.0.15/healthcheck_reports manually and select a report you need.

b. After setting up the VM, open any browser and go to CPViewer portal -> http://10.8.0.15.

Select upload method:

1) Manual/attachment upload: you can submit .dat or .gz file (which contains .dat);

2) Google link (server will automatically download file from GDrive). In this case solid upload link is highly recommended;

*NOTE: You can pick one of the two methods, not both at the same time;

c. Enter customer`s name (this will be used for name db and datasource of cpview; d. Select version from which cpview was exported – R77.30 – R80.10 or R80.20+;

e. If you did all of the above, select the submit button and wait for your reports to get created;

*NOTE: Speed of the processing will depetend on the size of the file (upload time + querying/healthcheck.sh execution through the .dat/.info and taking all the relevant info).

3. After you get redirected to Grafana you will be able to see your cpview visualized through graphs focused on different parameters. In case you uploaded CPInfo file you will also get GW`s healthcheck report in a separate tab.

Few useful GrafanaUI details:

- Top left corner – selected datasource (datasources will automatically be deleted on weekly basis);

- On menu at the far left you will be able to see possible dashboards (do not need to be changed since everything related to your cpview is automatically provisioned);

- Top right corner – time span which we are looking into (this is also automatically set from the first to the last timestmmp from your cpview);

- When clicking on different views you will be able to adjust some parameters or queries according to your needs;

*NOTE: All datasources – their dbs and healthcheck reports are being automatically deleted every Monday at midnight. If you do not want this – enter crontab using command crontab – e from CLI, erase the camm of deleteALL.php and/or delete_hc.sh script/s and save it.

*DISCLAIMER - This open source tool is provided “As Is”. No representations or warranties are provided with the use of this tool.

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

alexsp — Tue, 24 Dec 2019 00:19:30 GMT

Amazing tool and very helpful!

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

PhoneBoy — Tue, 24 Dec 2019 00:47:38 GMT

And clearly to customers if you're posting on CheckMates.
Nicely done!

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

JesusJ_Sanz — Fri, 27 Dec 2019 14:26:21 GMT

Thank you very much for sharing!

Very intesresting tool for support purposes. Nicely done! 🙂

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

_Val_ — Fri, 27 Dec 2019 14:33:08 GMT

Great job, @Petar_Markota!

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Milos_Jovovic — Fri, 27 Dec 2019 16:02:40 GMT

Well done Petar! Svaka cast. 🙂

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Danny — Fri, 27 Dec 2019 20:49:31 GMT

Thanks for reminding us that the CPview history database can be visualized with tools like Grafana, DBPlot and others.

The web visualization output of Check Points Gaia Healthcheck script is a default feature of the tool with full vendor support. Within your project one would lose the vendor support, right?

A few other questions pop up in my head:

You publish Check Points copyrighted health check script publicly on a Google drive (within an ova file) making it downloadable for everyone thus circumventing Check Points Support Center Access policy -> Where is Check Points statement that they granted permission for this?
You use Check Points official document template including copyright notes while you distribute Open Source software -> What about copyrights?
Your CPViewer guide looks like an official Check Point document but Check Point doesn't know anything about it?
Your CPViewer guide is entitled for SEs/partners, however this thread says it's for customers as well. What is correct?
Your CPViewer guide says: *NOTE: Your VM must have internet access.. Why?
Your CPViewer accepts downloads of Check Point CPinfo files from Google Drives. Why would anyone ever upload his confidential Check Point data to a Google drive?
You are a Check Point employee. Why aren't you hosting this on Check Points GitHub account? Check Point has other open source projects that are hosted there (example: sk120342). This would look way more official than the Google drive solution.

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Petar_Markota — Sat, 28 Dec 2019 00:08:37 GMT

Hi @Danny ,

Thanks for feedback and great observations.

Prior to publishing CPViewer on Check Mates (for everyone) we checked with product management and legal department after which we got an approval for the public post on here. Months before, this tool was available internally/was shared with partners and after a great feedback we decided to publish it for all since it obviously can be helpful to anyone with CP gateways and it does not contain anything that is confidential.

HC script will be updated in case of a new versions.

Answers to your questions:

1) The main idea of the tool is to have it all centralized in one place, automatically provisioned, where without much hustle people can use its capabilities and get the reports ASAP when needed. Healthcheck script is a great feature which acts as an addition to cpview chart and enriches the info we get, so it was a nice idea to add it up to the tool. Again, the purpose of the tool is only to help people solve potential problems faster and easier, so I don`t think we need to be so strict on this.
2,3) I was not notified about the word template issue which you mentioned. Once again, thanks for great observation. It is removed now.
4) I renamed it.
5) The tool has a feature where you can add a GDrive link to your cpview/cpinfo files from where they would be downloaded. This is the only reason why it needs to have internet access. If you will be attaching files only from your local machine you don`t need Internet access. I added a clarification.
6) Here, you answered your question from above. Some people do, it`s just another feature which can be used if you want.
7) We went through the process of approval for the tool`s posting on CheckMates and I was not pointed to use any of those. Thank you for mentioning it though, we might transfer it to one of them in the future.

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

HenriqueFM96 — Sat, 28 Dec 2019 03:05:29 GMT

Hey @Danny ,

Glad to hear from you!

I really think this tool is helping a lot of people so far, both employees and not.
@Petar_Markota might be able to tell us how many people already thanked him so far...
Apart from that, just have a look at the previous comments on this post, many Check Point employees are really happy with the tool.

Just trying to answer some of your questions...
- By using this tool, you don't lose your product support. As the VM doesn't need to be in your network, it can be in a lab/monitoring environment. So you just need to import the files to populate the Grafana with the data from the Gateway. The HealthCheck file on the gateway is not going to be modified.

- Check Point does know about it. It's being shared accross the whole company for months.

- Regarding the internet access requirement...
I think it's clearly stated in the full sentence. It is a must IF you want to use Google Drive feature.

@Petar_Markota may be able to explain better if I got something wrong

Please, let us know your thoughts.
Regards,
Henrique

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Danny — Sat, 28 Dec 2019 09:42:21 GMT

Thanks for the clarifications and corrections.

I'm still curious about the Google Drive thing. Petar confirmed some people (i.e. Check Point SEs/Partners) do upload Check Point CPinfo and other troubleshooting files to Google Drive which is why he added the feature to download files from there.

CPinfo files may contain valid UserCenter account data, customers' entire network topology and security policy information and many other highly confidential data and therefore should never be uploaded to any external resource.

I'd be glad if Check Point's own DiagnosticsView would allow showing Healthcheck reports and CPview database plots for the opened CPinfo/CPview files.

@Mor_Aisenberg , @Anna_Fridman : What about this idea? Obviously Check Point SE's/partner have the need for a simple utility to runs advanced health checks and CPview database plots, let's extend DiagnosticsView for this in order to have one central CP support diagnostics tool.

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Petar_Markota — Sat, 28 Dec 2019 10:07:31 GMT

GDrive functionality is there since June/July. At that time the tool was accepting only cpview .dat files, or cpview .dats within .gz or .zip, and at that time I saw couple of cases where people for easier sharing are uploading it to other sources. GDrive seemed as one of the nice ways to share stuff, so this is the reason it was added in the first place. CPInfo analysis (healthcheck) has being added in November this year. At this time GDrive functionality was already a part of the tool.

Danny, I fully agree with you that CPInfo isn`t something that should be uploaded anywhere ever, but GDrive is just a functionality which has been added as one of the options for CPInfo analysis too, since it was there for cpview .dats. Healthcheck script is basically the only thing within CPViewer which processes .info files. When I created a GDrive download option, I looked at it from the development perspective and I didn`t think if someone wants or doesn`t want to do this - it was purely to have another way to do it besides the local attachments.

Again, this tool is something that we created for purposes of troubleshooting in our region (because easy and automatized way for getting it visualized nicely did not exist) and after realizing it can really help not only us, but anyone with our gateways, we decided to spread it out.

It would be really awesome if DiagnosticsView would be doing this too. Thumbs up for that!!

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Danny — Sat, 28 Dec 2019 10:27:06 GMT

So let's sum up what CPViewer currently is:

A Linux OS Ubuntu installation within a VMware .ova file (1.1 GB)
- containing a web server to support uploading for CP diagnostic files
- containing a Grafana installation to show a database plot of CPview databases within a web browser
- containing the Gaia Health Check script to provide the HTML result of the script

Correct?

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Petar_Markota — Sat, 28 Dec 2019 11:52:25 GMT

Correct, more precisely, It is the Ubuntu (release 18.04) server running the following:

1) Apache v2.4 as a web server (+PHPv7.3, +SQLite3 used in queries);

2) Grafana (v6.5.1) for the data visualization;

3) InfluxDB (v1.7.9) where all the data is stored after extraction;

4) Number of bash scripts with queries which are used for data extraction from .dat files;

5) Python (python3) script using "GDown (v3.8.3)" utility for downloading the files from GDrive;

6) Number of .php scripts which are working in convergance with bash scripts in order to automatize the following:

- Data extraction from .dat files depending on the version (R77.30/R80.10 or R80.20 and above) since two engines differ in queries structure as the versions DBs are different;

- Automatic Grafana datasource provisioning;

- Automatic Grafana dashboard provisioning. Here we created two dashboards R77 and R80 which are used depending on the user`s input at the beginning regarding the versions;

- Automatic Grafana timestamp provisioning;

- Automatic URL provisioning (IP range checks - for cases where the VM is exposed to the access from the internet);

- Running the healthcheck script on cpinfo files and showing a result in .html to the user;

- Uncompressing from .gz, .zip, .info.tar.gz when needed;

7) Crontab lines responsible for automatically deleting all data on weekly basis;

8. Since ubuntu didn`t have preinstalled many small tools which were needed, I installed few along the way, necessary for dev fixes and the tool to run properly (examples open-vm-tools, mlocate, unzip etc. );

9) User interface (some html+css for frontend and JS for input checks on few places) which takes the relevant info from the fields (later used for naming the datasources/cpviews and version pointer);

I hope these details are clearing it more up.

/var/www/html contains all the code responsible for the stuff from above.

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Danny — Sat, 28 Dec 2019 12:45:46 GMT

Cool, I'm adding this project to my CPX 2020 presentation 'CheckMates - Best of Code Hub Contributions' in Vienna on February 5th, 2020 (14:00 - 14:30).

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Petar_Markota — Sat, 28 Dec 2019 12:47:13 GMT

Sounds awesome! Thanks a lot @Danny !!

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

_Val_ — Sat, 28 Dec 2019 13:29:56 GMT

@Petar_Markota Once more, fantastic job, Petar!

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Petar_Markota — Sat, 28 Dec 2019 13:48:32 GMT

Thank you @_Val_ !!

Always happy to contribute to the community 🙂

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

HeikoAnkenbrand — Sat, 28 Dec 2019 19:39:54 GMT

Great job:-)

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Anna_Fridman — Sun, 29 Dec 2019 08:13:30 GMT

DiagnosticsView latest version already supports CPView database chart creation (CPView DB Viewer). Please refer to the sk125092 for the detailed description and DiagnosticsView download link (https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk125092).

Regarding health checks, it is in our future features list.

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Evgeniy_Olkov — Mon, 30 Dec 2019 08:53:16 GMT

Great staff! Thanks!

Could you please share the manual how to change the IP address? I found that it's not easy 🙂

/etc/network/interfaces doesn't work.

topic CPViewer - visualize your cpview/cpinfo files in 5 minutes in Firewall and Security Management

CPViewer - visualize your cpview/cpinfo files in 5 minutes

The tool comes within an .ova (exported VM) with everything already preconfigured.

The main benefits are:

analyzing and identifying performance bottlenecks in minutes

user-friendly: just upload it to web-site (no docker or other components needed)

Integrated OS analysis tool for richer results (healthcheck script)

available to both customers and partners

CPViewer .ova file download - HERE.

CPViewer guide - HERE.

How to use it?

Video below explains all you need to do to get CPViewer up and running - 5 simple steps:

Written step by step guide:

3. After you get redirected to Grafana you will be able to see your cpview visualized through graphs focused on different parameters. In case you uploaded CPInfo file you will also get GW`s healthcheck report in a separate tab.

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes

Re: CPViewer - visualize your cpview/cpinfo files in 5 minutes