I let my 3 year old nephew play with my PSPgo, he loved it. Sure, he could only do the most basic game I had (Spyro), but he still could. I also taught him some basic PC children's games.
A iphone for a child? No problem. I'd suggest going for a cheaper alternative however, as a child doesn't need a 8-64GB MP3 player or a high speed 3G connection or to make phone calls.