You should see a little upward facing triangle to the left of where the icon would be. Right click that icon and then click "Properties". Locate the volume on the list and make sure it is set to "on".
You may also want to then click "Customize notification icons" on the bottom left of that window and select the option you want next to the "Volume". The usual setting would be "Show icon and notifications".