if you smoke or eat the seeds yes. but that's dangerous. I can't have children anymore and I do believe it's because of my drug experiments when I was younger. You should take care of your body, and keep your faith.
It hurts the baby but it doesn't like... improve your chances or whatever. Second-hand smoke is a big issue when it comes to babies since the mother and the baby share the same oxygen. My advice better safe than sorry.