New deadlink-checking bot/Fight against server lag
i want to annonce that ive made a script that checks deadlinks from certain hosts (at the moment megauplaod, 2hsared and youtube, soon more..).
also i have been experiencing some annoying server lag lately. i am not yet 100% sure of the cause but i am investigating. i have a few *suspect causes which i have been working on fixing, if anything that should help speed up things.
eh in the end im pretty sure that not only this new server lag has started to happen about when i started my auto-deadlink-checker BOT but it was the auto-deadlink-checker BOT that was causing the lag... logical...
cos in the end i went and changed things so that the heavy query is done about 100 times less often and it reduced all that lag so much that now i was able to go up to 20 simultaneous running auto-deadlink-checker BOT and i havent experienced the lag problem ever since.
what helped me a lot in figuring out this one was to start using htop on my server instead of top. htop gives much more precise info about whats running on the server and how much juice it takes. everytime the weird lag happened top was always saying that nothing was happening, nothing was hogging the CPU, thats why i was confused. but htop strangely was showing me the CPU beeing "full" when it happened and i could see it was mysql that was tying it up...
no i dont think this has anything to do with this. everything should be working at the moment. if you want to see, example, movies that start with F click on the "videos" link on the top left of the page and click the letter "F".
ah i see! thats a mysql error, thats totally an error on my side. and thanks to you showing me this error message i did look in list.php on line 782 and i did find the error there that did cause this and i fixed this.
thing was, this error was only showing up to people who are not logged in. thats why i didnt see it
let me know if you have any more problems and thanks!