The 'dumbing down' effect is likely very small.
Most of the people who's gaming could be dumbed down, feel like you do, and won't buy the wii in the fist place.
The wii is mostly new, simpler gamers.
There may be some real, hard core gamers who go wii and may have their gaming dumbed down like some have said. There is also a slight transfer of this to other consoles, for example the viva pinata, movie trivia, and other family games that might have otherwise not made it to x360. Many of those are there so the x360 gets a slice of the dumbing down.
I still stand by the idea though that the vast majority of gamers that are hard core enough to be dumbed down are, by the same logic, too hard core to allow that to happen.
Just look at the numbers after all. x360 has sold more than the original xbox now. That would have been hard to do if very many of those 45 million wii's were bought by original xbox owners; despite some people having both.