While monitoring the joining process of a new SGM in EXL cluster with insights utility (one can do it with cinfo -l overview, but insights outputs the stages so nice and colorful), I noticed the insights utility exiting. Checked also on a Maestro installation, the same behaviour, insights utility exited.
After exactly 10 minutes, even though I set the inactivity-timeout to max and unset TMOUT bash envar (one might remember ssh sessions dropped while performing a remote JHF install due to TMOUT).
So I checked the /opt/GaiaTools/insights/src/insights.py and I found a line:
self.tmout = 600 if "TMOUT" not in os.environ else int(os.environ.get("TMOUT"))
that explains the behaviour: especially because I unset TMOUT the default hardcoded tmout kicked in. After all, insights is not designed to be a continuous monitoring tool (there are others).
My workaround, that I'll not recommend, was to
g_all 'sed -i -e 's/600/3600/g' /opt/GaiaTools/insights/src/insights.py'
after new SGM joined the EXL cluster. I'll check if insights.py is included or not in the initial lightshot (I love the term and the simplicity and speed of the mechanism - but I do remember the elegance and robustness of using drbd to sync configuration in Crossbeam XOS) performed when a new SGM joins cluster to have a more consistent workaround, if needed.
