The issue is the input signal of the microphone may be mixing with the windows input sound. On some audio software mixers there is something called, "What U Hear". If that's selected then your original speech gets copied and repeated depending on if the volume is above 50%.
Its kind of like putting Echo on a voice. If you put the initial volume and feedback both above 50%, it creates a loop that increases in volume each time.
Check to see if there is anything else other than just your microphone active for input signal. If you can isolate all audio inputs to just your microphone, you'll be fine.
Another thought, you speak into your microphone, it comes out your speakers, the microphone pics up the speakers and a loop is created. try turning down your speakers and see if the issue persists.