Rabbi Rodal on Shabbos was asked, it says you are not allowed to judge people – what about if someone is harming you? – he answered – what if you do something wrong? – you are upset not at yourself, rather, you are upset on the action!