As a general rule of thumb, it is unwise to leave the PC on and running when replacing anything in it. Unless of course its a hot swap drive or something.
Cheers; can't wear plastic gloves either. I would have done it though because the cpu had to be removed to update BIOS and the cooler was a fit one time only type not even made anymore. I think I might have sort of glued it down to the mobo. But it was a long time ago.
How do I change the battery without resetting BIOS
Be sure u know the difference between BIOS and CMOS. Taking out the battery wipes CMOS, BIOS is in non-volatile memory and does not rely on battery. As others mentioned, better completely power down before unplugging anything.