and do you mean you give him some milk, after that immediately give solid food then milk again? it could be he is full from solid food thus he cant finish the milk at later part. i think you can just give him cereal then give milk another meal when he is NOT having any solid meal lik cereal. if not you can feed abit of milk 1 hr after feeding cereal instead of immediately.
as for outside, yes distraction is one of the main reason i think. but now you hav to solve the home pro 1st, if you have not change a new teat for him, can do so alrdy.