Why is my internet so laggy at night time?

For some reason around the 3 o clock in the afternoon, my net becomes really laggy. My ping is around maybe 100-300 in Australian servers at pingtest.net. In Australian game servers is aroind 90-200 and it is very very laggy. In the morning my ping is around 20-30 but it still lags every now and then. I do not know what is causing this problem. Anyone have ideas how i can fix this? I asked my ISP, Optus but they said i was too far for them to do anything.