It is normal to get warm. How hot? I don't know how hot is too much, but I play by hotter than my skin, is too hot. I find it is hottest when playing a game and charging at the same time.
What apps are running when it gets hot? Texts and calls should have little effect, web browsing depends on what you are browsing but shouldn't cause too much heat. It is mostly games (and obviously processor/GPU demanding apps) that would generate the most heat.
If your playing games or using intense apps, try to avoid charging while doing so and avoid leaving it on surfaces that don't conduct heat very well while playing (beds, sofa). I usually rest the phone after some games due to heat (Bloons TD is very intense late in matches/games due to tons of objects on screen.)
I use a Nexus 4, but your S4 is even more powerful and heat is generally accepted with powerful processors when used to its full. Soon I imagine there will be new cooling tech out as phones these days are getting more and more powerful.
If this is an issue no matter what app or state your phone is in (charging, sleeping or not), I'd try to get a replacement.
Hope this helps.
Edit: I found this:
NEC Watercooled Smartphone
Facebook
Twitter