First suggestion, don't take the SD card in and out of the device. That is a sure fire way to damage it, and possibly damage the port. It is better to have a card for each device you wish to use one in. And then to get data off the card, say to put on a computer, to connect the device to it via USB cord, rather than removing the card over and over.
When you put the card back in the phone, is it giving you an error message? Or only when you try to view specific files? And, if it is files that give the error, were the files put on the card when it was in a different device?