Do you mean you have to mechanically adjust your monitor via the menu and settings it may have or do you mean physically?
If you mean by physically moving it around to get a good color, you more than likely have to replace the cable. Try adjusting that first the next time it happens to make sure that is the problem. Or if you have a spare or know someone with an extra one try that. Also this just to eliminate the possibility of it being the monitor or your video card or controller.
I had this problem with an old Compaq PC I had years ago. So again just moving the cable around that is more than likely the culprit. You have to change the cable.