It's mostly to feel good. A higher sample rate means that it can accurately reproduce higher frequencies. The highest frequency that can be accurately reproduced by a certain sample rate is 1/2 the sample rate. So, 44.1kHz (CD quality) can reproduce up to 22.05kHz sounds, 48kHz (DVD) can do 24kHz, and 96kHz can do 48kHz sounds accurately. Of course, human hearing ends at 20kHz, and most adults' hearing actually ends at more like 16kHz.
The bit depth matters more actually - most CD quality and base computer quality audio is 16 bit, which allows for 96dB of dynamic range. The human ear can hear more than that (it's around 120dB from the threshold of hearing to the threshold of pain). This is solved if your sound card supports 24 bit, which can give 140dB+ of dynamic range (and therefore a lower noise floor as well).
Realistically, though, if you can hear a difference (blinded, where you don't know which is which), I'd be amazed. Of course, I run mine at 24 bit/96kHz anyways, just because I can.