Tap your Apps square then scroll through to Settings. Choose My Device then Calls and you should find an option to pick a notification or ringtone. If you already did that, is the volume turned right down?
Vibrate also comes with a sound and without that, it hardly rattles so it would be affected by the volume level.