Depends. If it is your internet connection on the phone, then if it is through data, check with your carrier/service provider. If it is via WiFi I would check to make sure the WiFi isn't fluctuating in speed or worse dropping connection often. If that is the case you would need to contact the ISP.
Now if neither of those is the cause, then I ask if you are using the youtube app? If yes, try just restarting the phone (turn it off and back on). Sometimes that is all it needs. Should that not help, then you may want to clear the apps cache and data.
NOTE: These will remove the data that is in that app. All data for that app. So be sure you save anything you want or need first.
To do that go into the phone's "Settings" and then "Apps". Go to "All" and scroll down to the right app. Tap that app and then tap "Clear Cache" and then tap "Clear Data".
If this still doesn't resolve it. Then uninstall the app, restart (turn off and back on) the phone and then reinstall the app.