They could use extra thin paper, like the one found on edible paper on cent wafers for instance.
I'd love to know more about the capacity, as it seems to me that the capacity of these batteries will probably not exceed Cd, Ni or Li-ion batteries for the same size. It'll all depend on how thin they can create this paper.