From what I have read, each method has its own strengths and weaknesses due to the diversity of components within a computer. Some people talk about time-to-failure for mechanical parts like fans and solid-state components like capacitors. Other people talk about thermodynamics and expansion-contraction damage done by the mixture of different materials. Both seem valid to me.
I work for a university that maintains a large 24 hour computer lab. For security/patch reasons and formerly due to hardware limits, we leave the systems on 24/7. However, as already stated the sleep state helps us save on power. Additionally, we are implementing other options.
For example, server farms could save significant amounts of power by moving to virtualization, such as provided by VMWares ESX system. It puts the hardware in a pool with a virtual interface to the software and can allocate systems on demand.
The traditional model is to have 4 servers as 4 separate computers. Each needs to have hardware that meets peak loads. With something like ESX, the 4 systems are pooled. When demand is low, all 4 virtual servers can be running on 1 system while the others are off or in a sleep state. As demand creeps up, systems are powered on and virtual servers are shifted over. This means all 4 systems are rarely all on. Even better, if demand frequently peaks and chokes all 4 systems, upgrading is easy for administrators.
However, if we want to see real energy savings in systems, we should be looking at heat dissipation. Currently server farms are in climate controlled rooms. This means AC runs most of the time, but we dump the waste heat energy into the air we just cooled - requiring us to cool that air again. Significant energy savings could be done if we could work on a better way to dump waste heat energy from areas where several always-on computers are together.
Also, maintaining more current components may save energy as long as the savings in energy exceeds the energy cost to manufacture the components. Fortunately, most computer components have the added benefit of improving performance at the same time (which defrays some of the cost).
Essentially, it is true that turning off computers will save energy - but there are many other aspects we can target that will give equal benefits.