The main thing wrong with the PSP Go is Sony's half-hearted commitment to it. I don't care about transferring UMD games, but in most cases I can't even re-buy them, because they're not there! And chances are if a game does get released on the PSP store, it's only in one region. Why can't US gamers play the Vagrant Story Playstation Classic? Why can't EU gamers play (insert title here, there's plenty choose from)?
It's no good having nice hardware if there's no games to back it up. The stupid thing is that Sony have plenty of brilliant PSP games but just haven't bothered to get them re-released on the PSN Store.