I don't think it's reasonable to be mad at someone for being nice to you. I say let it go for now. Just be very unavailable and if he gets flirty, just openly say you're not interested. Don't be afraid to take steps for self preservation, like blocking him if it gets to it. But it seems okay so far, to me.