Not sure why the TV being off changes how you "program" a remote.
If it's the most common type of universal you select the remote codes with a number that represents the location of those IR codes in the memory of that particular remote. The remote manual lists those locations and there are usually not that many codes for one manufacturers TVs that it's too difficult to try them one at a time. The on command is perfect to try the codes.
If you get a Logitech Harmony remote those are programmed from their online database with the MyHarmony app. Again the TV doesn't have to be on or even in the room while you do this.