The worst that can happen if impedance is too low is the receiver overheats and temporarily shuts down because halving the impedance doubles the power output supplied by the amp, to possibly beyond its continuous rating.
I don't know about you but my receivers seldom are driven higher than a few watts as my speakers are all >90dB per watt (and most of the power consumption is from powered subs). They would work fine with 4ohm speakers although perhaps with a bit more THD.