The new battery is it OEM or aftermarket? If it is aftermarket, that may be your problem. They don't always work correctly and they can be problematic. If this is the case, I would suggest returning the battery and getting an OEM one.
Also you didn't mention if it won't run on the battery, even if it has a charge, or if it just won't run on it when it isn't charged.