Since the Yamaha receiver doesn't have a subwoofer output your options are limited.
You can't connect a powered sub to the line level output because this is a fixed output and isn't controlled by the volume control of the receiver. If the sub is passive it won't produce any sound at all.
If the sub is passive you can cut the RCA off and connect it to one pair of the speaker B terminal and select A+B on the receiver. No way to match the level with the A speakers so the result is iffy.
If the sub is self powered you will need a speaker to line level convert so you can connect it as above.
Best option is a powered subwoofer with speaker level inputs.