Like if you are hurting him and feeling his pain, why not alive that pain?
Why let this abuser have any power if you can help it? Isn't that almost like continuing the abuse? Why isn't love and communication the solution? Why decide avoiding the pain is more important than sharing love?
It sounds like you're building a wall made of reasons that no longer exist and can pretty much all be fixed by you.
The whole "living for his moments of affection and feeling like dying when I cannot get them" sounds a bit like the boogey man. You can handle having a life with love I would think? It's tolerable. This is not going to get fixed by living alone, I don't think.
It sounds like a lot of self-imposed road blocks. You might want to look carefully at that. Like write down each road block you are talking about and think about how you could be the solution to it?
Woolens? Lol Good job, Spellcheck.