I'm getting harring phone calls for the person who formerly had my number, what can I do?

I've been getting haring and even threatening phone calls intended for the former owner of my phone number. I've tried telling the callers that there is no one here by the name they are asking for, but they accuse me of being that woman! I'm sick of it! I do not feel that I should have to go through the hle of changing my phone number, what else can I do?