HDMI will handle lossless audio codecs like Dolby TrueHD. Optical can't.
If there is any actual difference to you will depend on whether your audio system can reproduce those codecs and if your content has them.
There is no difference if you compare using HDMI-ARC to optical when you are connecting to a TV for a soundbar or surround sound system. This will change as eARC is rolled out. That can handle lossless audio.