I sympathise with your difficult situation. You obviously know and trust the young man and he sounds like a real diamond. But, like your friend, I would personally be uncomfortable with this arrangement. There are professional babysitting agencies who can provide child-trained and police-checked people who would be pefect for baby-sitting your daughter. You can find them on google.
I have read your message and the replies you have been given and i'd say stick with him. If you trust him and you know his family what is the problem? I don't know any babysitters around my area at the moment but have had both boys and girls look after my son in the past. If your comfortable with him tell everyone to stop putting doubts in your mind. In todays society it doesn't matter whether you hire a pro babysitter or not as we have all learnt from the news even if they have been police checked the wrong ones still get through. I think you should stick with what you know and as he has given you no reason for concern i wouldn't worry. Your daughter is happy so why get rid of him just because your friend thinks a boy shouldn't look after her.