This might help: buy two identical clocks, nice big clear ones, set one to the time he's allowed into your room but don't put any batteries in it, set the other properly, tell him when the hands on both clocks match thats when he can come into your room. If he mixes up the hands, colour code them with stickers.

I am sorry you go through such a struggle. There is really no need for it. Your son can sleep with you in the family bed and he will grow up confident and secure.
Being rejected by you and left to sleep alone is painful and create self-doubt and anxiety. There is no advantage in forcing children to do anything they are not ready for. They are meant to sleep with us. Just do what is easy. In your bed he will be happy and sleep calmly and feel loved. That's how it has been done through history and until 160 years ago and still in most countries.
Relax, respond naturally and enjoy the togetherness.