Speed of the blogs is not constant

The speed of all the blog post is not constant and keeps on fluctuating. And as and when the speed increases the No of Users Drop. so please help me in what all things we can do to improve the speed of our blogs which are hosted on ghost.