If any folders (or the SD card itself) are secure or encoded in any way, the new phone won't be able to just 'read' them. You would need to back up the files from the current phone to another device, say a cloud account or a computer (done via USB connection) and then, once the files are all on the computer (or in the cloud account) you can move the SD card over to the other device. Then you would download them to the card from the cloud account, or move them back over from the computer.
You may want to also leave a copy in the cloud account or on the computer so you have backups.
Now, if any of these files on the card are apps, those you can't transfer. You would have to remove them and then once the card is in the new device, redownload the apps.
If, the files you want to move are not secured and/or not encoded, and there are no apps involved, then you should be able to just move the card from one device to the other. That is if they use the same size card.