Yes it's normal to feel sad because breastfeeding from the start helps us bond with our child.
My son at 5 mths would also get distracted during breastfeeding so I try my best to breastfeed him in a quiet place. When we're out shopping, I do bottled EBM so that he can drink faster.
Hopefully it will be possible for you to continue to latch him. Try to talk to him, sing to him, soothe him.
For me, when finally I stopped, I was sad. I guess it's a passing phase which we learn to get by with. There'll be more things to look forward to for bonding with your son. As my son grows older, there're new joys that he brings to us. dont't worry. You'll get by.