Time of day, and what day it is, can have a great effect. Why you may not notice that this is a factor. Also, if they are doing less intensive things and then start a game, movie, etc., that will do it. Also phones, not always would they be accessing via WiFi. Depends on what is being done.
You may also want to try securing your network. Change the password and name if need be. You could have someone in the area that is accessing it which would only be effected when they are accessing it.
If yours is the only device having issues, you may also want to try the following (as computers tend to have more sensitive connections).
Try this....
First, make sure your laptop is 'not' connected to the modem/router via an Ethernet cable and...