igmp_max_memberships value dont survive a reboot

[TINTIN8]

Hi Gurus

We have an interesting problem with our 9300 VSNext cluster. We upgraded to the latest patch.

kernel: R82 - Build 003
[SecurePlatform]
HOTFIX_GAIA_API_AUTOUPDATE
HOTFIX_R82_JUMBO_HF_MAIN Take: 12

The cluster has 2 VSs running. VS1, and VS2.

The cluster is down with "VSX " pnote. But doesn't give much information on insights.

The issue we found was.

Member 1

VS0-

[Expert@xxx-xxx-VSGroup-s01-01:0]# cat /proc/sys/net/ipv4/igmp_max_memberships
1024

VS1-

[Expert@xxx-xxx-VSGroup-s01-01:0]# vsenv 1
Context is set to Virtual Gateway CORP-XXXX (ID 1).
[Expert@xxx-xxx-VSGroup-s01-01:1]# cat /proc/sys/net/ipv4/igmp_max_memberships
1024

VS2-

[Expert@xxx-xxx-VSGroup-s01-01:1]# vsenv 2
Context is set to Virtual Gateway OFFICE-XXXX (ID 2).
[Expert@xxx-xxx-VSGroup-s01-01:2]# cat /proc/sys/net/ipv4/igmp_max_memberships
1024

So, initially the value for VS1, and VS2 was 20.

we changed the value using the following command.

echo 1024 > /proc/sys/net/ipv4/igmp_max_memberships

We changed that on Member 1 VS1, and VS2

And then we changed that on Member 2 VS1, and VS2 to the same value, 1024

The cluster came up after a cpstop, and cpstart

However, I rebooted member 2, and after that, the cluster was down again. And the membership values for VS1, and VS2 returned to 20

[Expert@xxx-xxx-VSGroup-s02-01:0]# cat /proc/sys/net/ipv4/igmp_max_memberships
1024
[Expert@xxx-xxx-VSGroup-s02-01:0]# vsenv 1
Context is set to Virtual Gateway CORP-XXXX (ID 1).
[Expert@xxx-xxx-VSGroup-s02-01:1]# cat /proc/sys/net/ipv4/igmp_max_memberships
20
[Expert@xxx-xxx-VSGroup-s02-01:1]# vsenv 2
Context is set to Virtual Gateway OFFICE-XXXX (ID 2).
[Expert@xxx-xxx-VSGroup-s02-01:2]# cat /proc/sys/net/ipv4/igmp_max_memberships
20

I've logged a TAC case, and he suggested adding a value to rc.local file

echo "echo $(cat /proc/sys/net/ipv4/igmp_max_memberships) > /proc/sys/net/ipv4/igmp_max_memberships" >> /etc/rc.d/rc.local

we added this to both member 1 and member 2. After adding this, the rc.local file, at the end , looks like this.

# Create copy_partitions flag for SW raid systems

# When the flag is turned on raid_add script will copy the paritions

# from one hard disk to the other in case the system identified a new disk was inserted

if [ -f /boot/SW_RAID ] ; then

        echo 1 > /etc/sw_raid_copy_paritions

fi

echo 1024 > /proc/sys/net/ipv4/igmp_max_memberships

This value survived the reboot, but the IGMP values didn't. I'm sure if we reboot Member 1, the IGMP values for VS1 and VS2 will return to 20.

It has been 2 weeks since we logged the case, and the issue remains the same. We plan to migrate to the newer cluster next week, but with this,  everything is on hold right now.

Any help in resolving this issue would be highly appreciated!!!

Thank you gurus!!

[AkosBakos]

Hi, 

To be honest, the community has lack experience with R82. This is my opinion. Please push the TAC, and ask them to open an internal ticket for this behaviour and investigate it. 

Meanwhile, I had a private remote session with the R&D yesterday. We had issues with ElasticXL with VSnext. I can say this is not a mature technology in this state. Like Maestro at the beginning. 🙂

If you can please share the experience with us.

And there is a take 14 for R82, maxbe it worth a try.

Akos

[TINTIN8]

@AkosBakos  Ok, finally, we had to run a script. This was provided by the CP Engineer.

save the following to a script, ex- vscript_igmp.sh

#!/bin/bash
source /etc/profile.d/vsenv.sh

vsid1=1
vsid2=2
value=1024

vsenv $vsid1
echo $value> /proc/sys/net/ipv4/igmp_max_memberships
vsenv $vsid2
echo $value> /proc/sys/net/ipv4/igmp_max_memberships

copy it to both members(ex copy to "/opt/CPsuite-R82/fw1/scripts). You can scp to the active member from the standby to download it.

run

Then run the commands below:
chmod 755 /opt/CPsuite-R82/fw1/scripts/script_igmp.sh
dos2unix /opt/CPsuite-R82/fw1/scripts/script_igmp.sh

echo "/opt/CPsuite-R82/fw1/scripts/script_igmp.sh" > /etc/rc.d/rc.local.user

Reboot the member and wait. "Wait" is the key. It takes 5-10 minutes for the member to go to Active from the down state.