Fixed in $MDSDIR/conf/mds.cpbak, just change the unpacking extension to tar
yes | ./mds_restore *.mdsbk.tgz
to:
yes | ./mds_restore *.mdsbk.tar
Thanks to @Ofer_Barzvi for speedy resolution!
But most importantly! It's been truly amazing long journey since we upgraded to R80 in early 2017 and noticed MDS backup file increasing quite noticeably. Nobody believed me at the beginning that there is a problem and my cases were promptly shut down. Our backup grew from 3 to 22GB and restore time increased from half an hour to nearly 4hrs..
Now, more than 2 years and many posts and cases later, backup is reduced to 8GB and restore time was 45mins in the lab ESX that's under-powered compare to production one.
Kudos also to @Tal_Kagan for helping over very long time period to sort it out!
And it would not have happened if it was not for the community - just because of my nagging here case was eventually escalated to the right gang (via Dorit and our Swedish user group) 🙂
So never give up hope haha