Can pervert neighbor sue me for slander?

My neighbor groped me (touched my and through my clothing). I have talked to people and told them what he did. Now, I am afraid that he is going to file a lawsuit against me for slander. I didn't report it because he is older and I thought the cops would take his word against mine. He is going around telling people that I am lying about him. Can I be sued for slander by him?