Maybe not isp related, unless your isp are throttling the upload speeds. This i have never heard of an isp doing for uploads, with the poor bandwidth uploads have to start with.
Do a test at a bandwidth speed testing site, of three worldwide servers. Three repeat tests for each server at following times of the day morning, afternoon, evening and after midnight. Do these tests then show dramatic speed changes for both uploading and downloading, between all the tests you do.