Is it legal for a person to take money from my credit card without my authorization? i gave them the info btw.?

i gave this person the credit card information but did not authorize him to withdraw any amount until i said OK. but i didn't and the person did it anyway. is that legal for them to do?