Ebichan - unfortunately, if your boss is only making unpleasant remarks, but has not done anything yet, it's hard to lodge a formal complaint against him. If you trust your company's HR, can speak to them and if they are good, will speak to your boss about his behavior and upholding company values, employment act, etc, and also inform his boss about his poor attitude towards your pregnant state. This may help to shut your boss up and hopefully stop his nonsense behavior towards you. However, if your company HR is the kind who can't be bothered or ineffectual kind, they may not even step in. But bottom line is, if your boss really does try something funny like terminate your employment or refuse to let you go on maternity leave, MOM will go after your company's HR who will then go after your boss... Does this make sense?
Another method you could try is, if you are on good terms with your boss's boss or some other high-ranking person in your company, can try to let them know what you've been encountering with your boss... And see if they can do something about it.
Or, worst case scenario, write an email / set up a meeting with your company HR director with specific details of your boss's comments and remarks to you, highlight MOM's regulations wrt pregnancy, and close off with something like "I really hope xyz company is compliant to labour law and I hope HR can kindly educate abc that such comments and remarks seem to indicate that xyz company is not supportive of our nation's efforts to encourage married couples to raise the birth rate..." In other words, be vaguely threatening hahaha... But I would say take this stance only if you have tried other means to shut your boss up, have decided that you dont't see a long term future in this company and are happy just to ensure you get your maternity benefits.