Was having the same problem, and for me it ended up not being the headphones or windows at all!
It was my computer case.
My case had front 3.5mm jacks for headphones/mic which is what I was using. Switching the headphones to the actual motherboard jacks in the back fixed the problem.