Connecting an audio only output of the PC to the TV may not work because some TVs won't allow you to select an input without video and audio. If you have another source with composite video out that you aren't using then connect that.
Since most TV speakers are pretty bad you may not enjoy listening to music on them. Maybe get a soundbar that you can use with the TV and the PC?
Is this a "standard" USB sound adapter, or something specifically provided with your TV?
Does it have brand / model number?
Does your PC register that an USB device has been plugged in, and complain about drivers?
Connecting an audio only output of the PC to the TV may not work because some TVs won't allow you to select an input without video and audio. If you have another source with composite video out that you aren't using then connect that.
Since most TV speakers are pretty bad you may not enjoy listening to music on them. Maybe get a soundbar that you can use with the TV and the PC?