I would question his father about how he acts when he is with him. Get his perspective. It could be a try on but if you have any concerns that his father is mistreating him, please get help straight away as your son may be afraid. It may also be that your son is not handling the separation very well, maybe counselling would be the way to go.

Good luck and I hope things work out well for you and your son.