It sounds like your son regards using the bathroom as a chore which stops him doing something more interesting and should therefore be done in the quickest time possible. I think you should try to nip this in the bud before it becomes engrained.
Tell him that he cannot return to whatever he was doing until he has used the toilet properly. Take him back to the bathroom and make sure he has flushed and washed his hands (they should not be bone dry). It may be useful to do a little retraining on wiping his bottom: children can forget how to do this if they don't get enough practice!
As for the damp pants, well it seems your son has discovered how he can put off the need to use the toilet by peeing a small amount into his pants without making any puddles. I would definitely advise discouraging this before it becomes habitual. (Try to imagine him leaking into his pants while he is sitting on your lap - not a nice thought). Tell him you will treat damp pants the same as wet pants, requiring a change of clothes and wash/bath/shower. That should show him that it will be a lot quicker to use the toilet properly.