The newsletter email function is subbing in &apos when rendering HTML. Outlook choking on &apos is a known issue and nothing specifically to do with Ghost except the rest of the world codes around it. In fact in the js code, there are even commented references to this problem.
You can work out a partial solution now by always using markup text for paragraphs where you want to use an apostrophe, and subbing out either &-#-x2019-; or &-rsquo-; (remove the dashes) for each instance of that character. The solution obviously is for Ghost’s newsletter converter not to use &apos at all, and use one of these instead. I have been hunting through the js but haven’t found the location where this is done yet.