That's probably because FanFiction.Net doesn't want something that looks like an entirely automated bot network (spanning hundreds of IPs) crawling their website -- It currently looks like that, because the same exact non-default user-agent header is used by a wide range of users, thus they blocked it, just like they did with the Python urllib default user-agent.
An easy fix for this would be to simply allow users to customize the user-agent in their personal.ini configuration file, and set it to something commonly used.
Last edited by cryzed; 09-08-2013 at 07:49 PM.
|