Hi Bishal,
Given that most of the traffic displayed on the graph you provided is "drop" traffic, I think it is definitely worth enabling drop optimization since when a new connection matches the Drop Template, subsequent connections are dropped without performing a rule match and therefore are accelerated. Along with the sk I have already mentioned, have a look at the "Drop Template" section on sk98722 for more information.
Once enabled, you can monitor the traffic over a period of time and see what results you get on SmartEvent.
I hope this helps.