It is indeed important to be very careful when it comes to chinese herbs. A lot of nice seemingly mild chinese herbs (like bird's nest) we normally take suddenly become inappropriate for pregnant women according to experienced mothers and grandmas. The advice I have gathered so far said excessive bird's nest cause phlegm, even asthma later in children. It's for you to decide.
Ginseng is different from Paoshen. Take everything in moderation. That's the key for me.
I also had very pale complexion (up until now) and doc via blood test said my blood level is low, and hence prescribed iron tablet for me. I drink red date tea once in a while and increased beef intake which seem to help. I am looking less pale. Once I was super tired and felt so weak I could not walk up the flight of stairs at home. My mum opened up and warmed just chicken essence for me and astonishingly I immediately felt less weak and more vitalized. You might want to try since chicken essence is deemed ok.