Where do you live that people can still smoke inside in public places? Your whole "woman and her baby at the table next to you" example makes no sense because I don't know of anywhere that could happen.
I've never worried about the extra breaks that smokers may take. We all do extra stuff we probably aren't supposed to while at work - such as responding to posts like this.