So unless your podcast files are huge (or your host has your file upload size throttled way down), it actually is possible to host a podcast on Ghost. Put it in an audio card, add some custom code to make an RSS feed, and you can get it picked up by the podcast providers.
You can see that in action over at The foxwizard podcast
What’s not supported natively there is having a subscribers-only feed. You can restrict your podcasts to only being shown to subscribers, but there’s not a way to have the podcast providers pick the episodes up by rss that way. That’s where you’d really need an integration of some kind. Here’s a link to one option: Private, fulltext RSS feeds for Paid Members