First make sure you aren't in 'developer mode' on the phone. Also, it is possible that using low quality third party cables may cause the GearVR service to become active on its own. To avoid that you may need to use Samsung approved accessories.
To stop this service, go to your"Application Manager" and tap the "More" option and select "Show System Apps". Then locate all the gear devices listed and 'force stop' them.
Also, the app you need to turn off (Oculus) may not be listed in the main apps area. You may also have to tap the "More" option to locate it.