So, two solutions. As mentioned the virtualizers can be a bit hacky (I use Cameyo myself) and time consuming, since they basically work by taking a pre-install image and a post-install image and find the differences to figure out what they have to bring along for the right.
A better option, provided it covers the apps you want, is to go to PortableApps.com. These are all designed to work only within their directory & therefore on a USB stick. Some applications (often smaller utilities) will also offer the "use own directory" or a portable installation download option you can use to the same effect.