Yes if you delete them the other stuff could possibly be gone. If it isn't downloading correctly, through the app, and you are certain the settings are set to download files to the SD card directly, the app is probably the issue, not the phone. If this started being a problem right after an update to the app, I would try rolling it back to before the update. Also, if you haven't updated it in a while, I would try that now to see if it resolves the problem.
If the files are not somewhere you can download them from again, then your best bet is to try and move them all over to the SD card yourself.
The quickest way is to copy move all the files to a folder on a computer and then copy them over to the SD card. You would do it just by connecting your phone to a computer via USB.
Also, if the problem with the app persists, you should probably contact the apps creators about it.