You will have to either wait until it is drained and turns off on its own, or pull the battery. Yes some phones batteries are hard to get to. I would not suggest opening yours, however, if it is hard to get to and still under warranty as that can possibly void that warranty.
Depending on the reason for the freezing, you may just need to restart it, or you may need to remove something from the phone.