MDS Global IP search

[StuartGreen]

Hey all, I've been interested in trying out Smart Console Extensions for a while now and this request came up locally in a discussion so thought I'd try and smoosh the two things together. 

https://github.com/sg84/MDSSearch

The main function comes from a Docker image which acts as a simple web host to serve the Smart Console Extension and also as a proxy to send API requests to the Smart Center (without it, you run into all sorts of fun CORS issues). Once installed - you get a dashboard extension pane that lets you search for an IP address across all domains. Currently, it's limited to IPv4 and hosts - but that's a fairly easy change to make depending on what's required. 

I've written it to use the API key functionality in R80.40 but this could easily be ported to R80.30 to use username and password auth for the API. 

Hope it works for someone, let me know if there are any issues.

Stu

[Maarten_Sjouw]

I wanted to add this but the URL cannot be used it has to be a github manifest, I'm not a developer, so not sure where to look for the right URL to add it directly to the SmartConsole.

[StuartGreen]

Hey, there's a little more to it than that as the extension requires a helper VM / container.

There are some instructions in the readme file under Github that go into more detail - but you need to clone / download all of the files from Github and then launch a Docker container based on the docker file. If you have a virtualisation environment and can build something like an Ubuntu VM - that will probably be easiest to run these commands in. I used Ubuntu in Virtualbox to build this and that worked fine. It doesn't have to be a particularly high powered VM, just something that can handle a few web connections.

[Maarten_Sjouw]

Ok, that makes it a bit harder to get going for me.
Thanks.

[StuartGreen]

it's not as scary as it sounds, you don't have to do anything with docker other than install it and run the commands. drop me a message if you need a hand.

[vinceneil666]

hi, I might be way off here.. But would this be something you could run as a cotainer in r81 ? I see that the management server in r81 pops up with a docker interface ? 

[Tomer_Noy]

You should be aware that the full "Global" cross-domain search feature is available in R81. 

If you log into the system domain, you can use the object explorer to search across all domains and there is no need for an extension or running a custom container.

This discussion was started before R81 was released...

[vinceneil666]

yeah, appreciated. But just as an example - it would be possible ? I am just starting to look into docker on the management, so I was just wondering if I understodd it correct.

[Tomer_Noy]

The docker I/S on the Management was added to run containers that are part of the product. It would probably be technically possible to run custom containers that are not part of our product, but this is not something that we officially recommend or support. It's similar to the ability to run various external binaries and cron jobs in expert mode.

These custom containers will not be carried over in upgrades and there may be other limitations around that.

[StuartGreen]

potentially yes - the docker image is basically running as a forwarding proxy to work around CORS issues. That said - be aware that you would be allowing your management server to run as an open proxy, so please be very careful if you do test this. The best long term solution for the MDS global search would be to upgrade to R81 and use the native feature of course 🙂

Thanks,

Stu

[PhoneBoy]

Funny you mention CORS, it came up on the community recently.
And yes, it's a problem.
https://community.checkpoint.com/t5/API-CLI-Discussion-and-Samples/Enabling-CORS/m-p/70990#M4233

In any case, well done!

[StuartGreen]

CORS is a pain and a bit of a weird protection to get your head around. I had a play with the gateway and it is possible to add the required headers to the API responses from your management server, but it's super unsupported (you need to modify the httpd config files for the API and I have 0% confidence that changing this won't break something else so don't do it in or anywhere near production environments!). The headers aren't enough to satisfy CORS though in most cases because the JSON requests we use aren't classified as 'simple' requests and need the web server to respond in a particular way to an HTTP OPTIONS request. At that point it was too late at night for me to delve any further so I went down the proxy route. 

[Tomer_Noy]

Thanks, and it's really cool that you were able to take the SmartConsole Extensions I/S with the Management APIs to develop such a feature without R&D involvement 😀

I hope that this will be useful for customers using current versions that are already GA.

I also want to share that we are planning some new capabilities for the upcoming version (after R80.40). This includes a cross-domain-search that will be native within SmartConsole. It will also come with new APIs that can perform searches across multiple domains without the need to iterate over them one-by-one, so the performance will be better for large environments.