Yeah, you could consider a smaller all-in-one PC as being almost like a laptop without a battery. Typically, they have larger screens than a laptop though (starting at around 20 inches diagonal), and tend to weigh more and be less portable as a result.
You could just go with a laptop that has a lower-capacity battery though, if you don't care much about battery life. I kind of doubt the cost of the battery adds that much to the cost of a laptop though. A laptop manufacturer has the batteries manufactured and shipped to them in bulk, so it undoubtedly adds less to the cost of the laptop than what you see cheap, off-brand replacement batteries selling for online. For a feature that doesn't likely cost them more than $20 to add, and...