can i ask how much milk does your baby drink? cos it sounds not much form the number of pumps and direct latch feedings you have mentioned.
if you can, try to pump 3 times in office. check your latch with baby - is it positioned properly so that baby is able to help you clear your breast?
does your baby wake up in the middle of the night for milk? if not, it'll be good if you still wake up to pump out the milk to clear your breast, tiring though it may be. i had recurring blocked ducts for my first 6-8 weeks and literally woke up baby sometimes in the middle of the nite to help me clear breasts as baby's suction is stronger than any pump.
when i started pumping, i use hand compression while pumping. meaning that i massage and squeeze my breasts with one hand to help in clearing the breasts as much as possible during each pumping session, while the other hand holds on to the pump. i still wake up in the middle of the nite to pump, even if my baby does not wake up for his feed, as I find that my breasts will get lumpy by the next morning if i dont't wake up.