May (will) sound silly but to clean them, I always put my earplugs in the pocket of my trousers, and throw it in the washmachine to get rid of the earwax. They really sound better after drying. :p And these are high quality earplugs.
However, headphones will probably not fit into your pockets...