There is no need to feel guilty about it. The more you stress, the more your supply will be affected. Even if it's 5ml of breast milk left, it is still precious and dont't give up until the last drop!
I also felt very guilty that my supply was diminishing, thinking that I'm so lousy. I'm a stay at home mum yet I cannot bring back my supply. I pumped very frequently, almost every 2 hours, hoping to boost my supply. But end up still didn't work out and baby didn't want to latch on properly. So in the end I gave it up.
Now having 2nd baby, hoping that I can BF my elder again, and hopefully can BF the youngest until at least 1 yr old!