You have to swing the mic all the way down.
If you can't hear game audio, just chat audio...
1. Go to Settings
2. *Ignore* the "Sound and Screens" option
3. Scroll down and select Devices
4. Select Audio Devices > Output to Headphones > All Audio
5. Curse at Sony for misleading menu organization