Yeah, you could consider a smaller all-in-one PC as being almost like a laptop without a battery. Typically, they have larger screens than a laptop though (starting at around 20 inches diagonal), and tend to weigh more and be less portable as a result.
You could just go with a laptop that has a lower-capacity battery though, if you don't care much about battery life. I kind of doubt the cost of the battery adds that much to the cost of a laptop though. A laptop manufacturer has the batteries manufactured and shipped to them in bulk, so it undoubtedly adds less to the cost of the laptop than what you see cheap, off-brand replacement batteries selling for online. For a feature that doesn't likely cost them more than $20 to add, and is something most people will want, it just makes sense to include it. And even if you mainly use the system plugged in to a power source, and don't carry it around a lot, you may still have occasions where you wish to carry it to another room for some purpose, and not having to bring along a power adapter might make having a battery worthwhile. Plus, that battery might be useful in the event of a power outage, or if the power cord gets snagged loose or something, so that your system doesn't just instantly power down and lose whatever you were working on.