Does it start again on its' own or do you have to do something to initiate a restart? What, if anything, do you do?
Hot spot, home network, where does the problem occur?
More information is needed.
Overall, either the source is being slowed down or stopped for some reason or your phone is simply being overworked and pausing one app to process another app.