the old ipods were a spinning hdd. would not be too hard to pop it out and format it. the newer ones are all flash memory. may as well go with a flash drive if you're gonna use flash memory anyway. probably cheaper than an mp3 player as well.
rasperry pi would be a solid option as well. pretty small and there are a ton or resources for how to use them in many different projects.