The RX 480 is currently out on the market, and is starting at a retail of $200 USD.
However, i'm sorry to say that the cheapest VR headset for PC at the moment is the Oculus Rift going for 400 pounds at the time of writing.
The HTC Vive has more market demand due to its better features, so that is very expensive at the moment coming in at around 700.
Create another thread for a PC build and I can put one together for you.
Edit: In regards to greens' comment, the RX480 is perfectly adequate for VR, hitting above the 80fps mark in some, and a solid 90 in others. Although I will admit it has its limitations, getting poor frame rates in a few select titles, it is in fact a VR ready device, and should give a great experience with a headset...