topic Re: MDS Global IP search in API / CLI Discussion

MDS Global IP search

StuartGreen — Fri, 28 Feb 2020 13:11:06 GMT

Hey all, I've been interested in trying out Smart Console Extensions for a while now and this request came up locally in a discussion so thought I'd try and smoosh the two things together.

https://github.com/sg84/MDSSearch

The main function comes from a Docker image which acts as a simple web host to serve the Smart Console Extension and also as a proxy to send API requests to the Smart Center (without it, you run into all sorts of fun CORS issues). Once installed - you get a dashboard extension pane that lets you search for an IP address across all domains. Currently, it's limited to IPv4 and hosts - but that's a fairly easy change to make depending on what's required.

I've written it to use the API key functionality in R80.40 but this could easily be ported to R80.30 to use username and password auth for the API.

Hope it works for someone, let me know if there are any issues.

Stu

Re: MDS Global IP search

Maarten_Sjouw — Thu, 27 Feb 2020 10:37:08 GMT

I wanted to add this but the URL cannot be used it has to be a github manifest, I'm not a developer, so not sure where to look for the right URL to add it directly to the SmartConsole.

Re: MDS Global IP search

StuartGreen — Thu, 27 Feb 2020 10:53:51 GMT

Hey, there's a little more to it than that as the extension requires a helper VM / container.

There are some instructions in the readme file under Github that go into more detail - but you need to clone / download all of the files from Github and then launch a Docker container based on the docker file. If you have a virtualisation environment and can build something like an Ubuntu VM - that will probably be easiest to run these commands in. I used Ubuntu in Virtualbox to build this and that worked fine. It doesn't have to be a particularly high powered VM, just something that can handle a few web connections.

Re: MDS Global IP search

Maarten_Sjouw — Thu, 27 Feb 2020 11:07:22 GMT

Ok, that makes it a bit harder to get going for me.
Thanks.

Re: MDS Global IP search

StuartGreen — Thu, 27 Feb 2020 11:14:00 GMT

it's not as scary as it sounds, you don't have to do anything with docker other than install it and run the commands. drop me a message if you need a hand.

Re: MDS Global IP search

PhoneBoy — Thu, 27 Feb 2020 12:54:24 GMT

Funny you mention CORS, it came up on the community recently.
And yes, it's a problem.
https://community.checkpoint.com/t5/API-CLI-Discussion-and-Samples/Enabling-CORS/m-p/70990#M4233

In any case, well done!

Re: MDS Global IP search

StuartGreen — Thu, 27 Feb 2020 13:36:22 GMT

CORS is a pain and a bit of a weird protection to get your head around. I had a play with the gateway and it is possible to add the required headers to the API responses from your management server, but it's super unsupported (you need to modify the httpd config files for the API and I have 0% confidence that changing this won't break something else so don't do it in or anywhere near production environments!). The headers aren't enough to satisfy CORS though in most cases because the JSON requests we use aren't classified as 'simple' requests and need the web server to respond in a particular way to an HTTP OPTIONS request. At that point it was too late at night for me to delve any further so I went down the proxy route.

Re: MDS Global IP search

Tomer_Noy — Thu, 27 Feb 2020 14:03:46 GMT

Thanks, and it's really cool that you were able to take the SmartConsole Extensions I/S with the Management APIs to develop such a feature without R&D involvement 😀

I hope that this will be useful for customers using current versions that are already GA.

I also want to share that we are planning some new capabilities for the upcoming version (after R80.40). This includes a cross-domain-search that will be native within SmartConsole. It will also come with new APIs that can perform searches across multiple domains without the need to iterate over them one-by-one, so the performance will be better for large environments.

Re: MDS Global IP search

vinceneil666 — Tue, 02 Feb 2021 07:43:47 GMT

hi, I might be way off here.. But would this be something you could run as a cotainer in r81 ? I see that the management server in r81 pops up with a docker interface ?

Re: MDS Global IP search

Tomer_Noy — Tue, 02 Feb 2021 08:03:33 GMT

You should be aware that the full "Global" cross-domain search feature is available in R81.

If you log into the system domain, you can use the object explorer to search across all domains and there is no need for an extension or running a custom container.

This discussion was started before R81 was released...

Re: MDS Global IP search

vinceneil666 — Tue, 02 Feb 2021 08:06:19 GMT

yeah, appreciated. But just as an example - it would be possible ? I am just starting to look into docker on the management, so I was just wondering if I understodd it correct.

Re: MDS Global IP search

Tomer_Noy — Tue, 02 Feb 2021 08:15:43 GMT

The docker I/S on the Management was added to run containers that are part of the product. It would probably be technically possible to run custom containers that are not part of our product, but this is not something that we officially recommend or support. It's similar to the ability to run various external binaries and cron jobs in expert mode.

These custom containers will not be carried over in upgrades and there may be other limitations around that.

Re: MDS Global IP search

StuartGreen — Tue, 02 Feb 2021 09:51:39 GMT

potentially yes - the docker image is basically running as a forwarding proxy to work around CORS issues. That said - be aware that you would be allowing your management server to run as an open proxy, so please be very careful if you do test this. The best long term solution for the MDS global search would be to upgrade to R81 and use the native feature of course 🙂

Thanks,

Stu