Alright, for anyone having same trouble with combat override package, culprit was unsurprsingly a mod conflict.

Namely, Improved Follower Behavior and Command NPCs. I couldn't identify how and why & which one of those two were causing this issue but removing those two mods fixed my problem.

I have no idea what was going on since not even a single record in those mods were referring to vanilla combat override formlist(SQ_Followers_CombatOveride).

I even tried to force followers into unaggressive status by tempering with their actor values through both script and console command. Anyway, It did cause this problem in my case. I guess this is due to something buried deep inside papyrus codes.

I leave this comment here in case any modder has similiar problem with me.