My best guess is that electrical noise from the CPU, GPU or PSU is finding its way to the microphone input amplifier. If you are using the front panel microphone input, try switching to the rear microphone input, see if eliminating the internal wiring to the motherboard header helps.
If that does not help, you have three other options:
- use an external amplifier for your microphone and plug it in the line-in input which should be less susceptible to noise
- get an external audio adapter to isolate the ADC from noise within your PC
- get a USB headset which has its own ADC/DAC built-in
You could also try to hunt down the noise within your PC but this may require swapping out the PSU, GPU and motherboard to find which component or combination thereof is responsible for generating the noise.