As far as I know it only matters if you think someone is going to try to hack and steal your files. Deleting them permanently doesn't offer any kind of performance gains. Many HDD manufactures offer free software that will perform a low level format of the entire drive. This effectively does the same thing and is good if you plan to donate/sell your PC.
As for SSD I'm not sure if TRIM actually removes that data. Hopefully someone else can clarify.