This is not normal! The buzzing is coming from the transformer inside the AC Adapter. This happens when the transformer laminations are not assembled tightly. In addition to the buzzing, it will also create increased heat.
Replace the AC Adapter; do not continue to use the one that is buzzing.