Something worth noting that doesn't seem to have been mentioned yet - not all games are developed to use resources the same way. Some games thrive on having more VRAM available to them than others.
That said, personally, I'd prefer to have MORE VRAM available at my disposal than not. There's no...