One thing I didn't see mentioned is that the PS3 will still be compatible with PS1 games. This only affects PS2.
For me, the best thing about the backwards compatibility is the number of other hassles that it helps alleviate the number of devices I have hooked up. I use my HDTV as my computer monitor, which means that on the AV racks around it, I currently have my stereo components (speakers, tuner, receiver, etc), a tower computer, my cable box, a PS3, an xBox 360, a gamecube, an NES, a TurboGrafx-16, a printer, a flatbed scanner and various other small items (3 external HDD, a couple USB hubs, a fair number of notebooks used to keep track of whatever project I'm working on). I do have a Dreamcast and SNES sitting in boxes nearby and ready to plug in and play, but it's a pain finding a place to put them while they are in use.
I don't list this as any attempt to come across as bragging, since I find the collection pretty pathetic myself, but it is just to demonstrate why some people do care about backwards compatibility.