Try a different charger to see if that is the problem. Even though you get that first red light, it still could be the charger. Also take note of if the port where the charger connects to the phone is loose or part of the problem.
If neither of these turns out to be the issue, then you likely will need to replace the battery. If the phone is under warranty, then you will need ot contact the manufacturer. If it isn't, then you can try doing it yourself, or take it to a local tech. This is assuming of course that the battery is complex to remove. If not, then you can easily do it yourself.