You don't just hold the power button. A hard reboot requires you to hold the power button, the volume down button, and the home button for about 10 seconds.
It is really important to hold the on off button down at the same time as holding the volume + button until the screen turns black it may take a minute or two