[GUI Plugin] FanFictionDownLoader

[JimmXinu]

Originally Posted by redscarlett

I have a lot of ebooks in calibre that are works-in-progress and have been downloaded via FFDL, so I use that to update them. The thing is it often wants to update fics that don't need to be updated - the log states that it wants to re-download all the chapters, and I have no idea why. Is this just a quirk?

Without more context about what you're doing, I'm not sure what you mean.

If you download epubs and use the 'update epub if newer chapters' setting, FFDL will only update a story when there are new chapters. The only time it should re-download all the chapters is when you use 'overwrite always'.

If what are you seeing in the job log is all the "Writing chapter text for: XXX" lines, those are not downloading again--that's writing the chapters to the new, updated epub from the old existing epub.

FYI: IMO, you should use the email notifications from the sites to inform you when your stories are updated. Using FFDL to poll them for updates is rather wasteful and inflates the stories' hit counts. FFDL makes it easy by allowing you to drag and drop urls (and even whole email messages from Thunderbird) onto the FFDL button on the toolbar and the 'Add New from URL' dialog box.

[redscarlett]

Originally Posted by JimmXinu

Without more context about what you're doing, I'm not sure what you mean.

If you download epubs and use the 'update epub if newer chapters' setting, FFDL will only update a story when there are new chapters. The only time it should re-download all the chapters is when you use 'overwrite always'.

If what are you seeing in the job log is all the "Writing chapter text for: XXX" lines, those are not downloading again--that's writing the chapters to the new, updated epub from the old existing epub.

FYI: IMO, you should use the email notifications from the sites to inform you when your stories are updated. Using FFDL to poll them for updates is rather wasteful and inflates the stories' hit counts. FFDL makes it easy by allowing you to drag and drop urls (and even whole email messages from Thunderbird) onto the FFDL button on the toolbar and the 'Add New from URL' dialog box.

I've checked my configuration of the plugin, and I have it set to 'update if new chapters' not to always overwrite; and yes, I'm downloading epubs. I have site notifications for when a fic has been updated, and then I update those. I add the odd one in accidentally sometimes if I'm not paying attention, and that's the fics I mean. I press update on the plugin, and the log will say 'update epub completed, added 1 chapter for 1 total'. But that fic has been in my calibre library for months - does this make it any clearer?

[SallyK]

Originally Posted by JimmXinu

Sorry, I missed seeing this post entirely until seabream replied to it.

I'm sorry, I didn't get a notification that you'd replied either, must have been a forum glitch.

Thank you very much, that worked perfectly - and in case you didn't see my other post about the Stories of Arda stories, the testing version worked absolutely fine for that as well.

[JimmXinu]

Originally Posted by redscarlett

I've checked my configuration of the plugin, and I have it set to 'update if new chapters' not to always overwrite; and yes, I'm downloading epubs. I have site notifications for when a fic has been updated, and then I update those. I add the odd one in accidentally sometimes if I'm not paying attention, and that's the fics I mean. I press update on the plugin, and the log will say 'update epub completed, added 1 chapter for 1 total'. But that fic has been in my calibre library for months - does this make it any clearer?

Not really. From what you've said, I agree that shouldn't happen.

Can you post (or PM to me) the full text of the log in question? Including the job log and debug log (run calibre from command line with calibre-debug -g and copy the command line output) would also help a lot.

[SallyK]

Also, thank you for trying, seabream - that's useful information for the future, even if not necessary now.

[JimmXinu]

Originally Posted by Jade Aislin

I tested it under defaults, test site, and the original site (FFN, with an actual storyurl (http://www.fanfiction.net/s/7461968/1/)). The only one it worked under was FFN, where both the character (Alice K) and (Alice in Wonderland, 2010) converted to the way I wanted without extra code. Through some testing with the test site, it seems that the &&category is preventing the regex from working in any but active sites. Is this how it is supposed to work?

Conditional regex (with a && clause) works fine for me in [test1.com] and [defaults]--as long as the conditional clause is met, of course. This works for me:

[teststory:1002]
category_list:Alice in Wonderland\, 2010
characters_list:Bob W

[defaults]
replace_metadata:
 category=>Alice in Wonderland, 2010=>Alice in Wonderland (2010)
 characters=>Bob W=>Robert While&&category=>Alice in Wonderland \(2010\)

What are you putting in category_list in the teststory section? You don't want \(2010\), and if you put Alice in Wonderland, 2010 that will be come two entries "Alice in Wonderland" and "2010", you have to use Alice in Wonderland\, 2010.

[Jade Aislin]

Originally Posted by JimmXinu

Conditional regex (with a && clause) works fine for me in [test1.com] and [defaults]--as long as the conditional clause is met, of course.

What are you putting in category_list in the teststory section? You don't want \(2010\), and if you put Alice in Wonderland, 2010 that will be come two entries "Alice in Wonderland" and "2010", you have to use Alice in Wonderland\, 2010.

I wasn't sure how to put it in the category_list so I hadn't placed the \,. I now realize that I made a large error when I was testing the different character regex. I used the characters from Gundam Wing. I made sure that I had the code for category regex of Gundam Wing, as well as that the conditional regex matched. However, I forgot to make sure that Gundam Wing was in the category_list.

I swear that I was having problems with other character regexes before though, but they seem to be working now.

I'm starting to think that I should take a break from fixing my code. Or at least wait until, I'm no longer pulling 9 hours days.

Sorry to be such a bother lately.

[s_stabeler]

I've juts discovered a problem- it will no longer download from HPFanficarchive, coming up with a 403 forbidden error. (regardless of the exact link)

[JimmXinu]

Originally Posted by s_stabeler

I've juts discovered a problem- it will no longer download from HPFanficarchive, coming up with a 403 forbidden error. (regardless of the exact link)

Already fixed. Update to the latest version of the plugin.

[riothamus]

Originally Posted by JimmXinu

Already fixed. Update to the latest version of the plugin.

After updating I am still getting the 403 Forbidden error for hpfanficarchive.com when I try to get the stories from webpage option. However, the ability to copy and paste individual story links for download now works.

[JimmXinu]

Originally Posted by riothamus

After updating I am still getting the 403 Forbidden error for hpfanficarchive.com when I try to get the stories from webpage option. However, the ability to copy and paste individual story links for download now works.

Ah. That's different then. The 'story urls from web page' uses the same code for all sites so it didn't get the latest change for hpfanficarchive.com. I'll look into it.

[JimmXinu]

Originally Posted by JimmXinu

Ah. That's different then. The 'story urls from web page' uses the same code for all sites so it didn't get the latest change for hpfanficarchive.com. I'll look into it.

Here's a new version that works with url lists from hpfanficarchive.com and now works correctly when pulling url lists from pages that aren't on FFDL supported sites.

This version changes some internals that may possible effect other things. So the more testers I can get, the better.

UPDATE Aug 6 2013 - Remove obsolete beta versions

[s_stabeler]

the new version works ( I thought I had the latest version of the plugin though, since Calibre hadn't mentioned a newer version...)

[Jade Aislin]

Originally Posted by s_stabeler

( I thought I had the latest version of the plugin though, since Calibre hadn't mentioned a newer version...)

I haven't been getting told about updates either. When I checked the plugin updater, under installed plugins, I found that FFDL is not listing the available version. In addition, the release date says 01 Jan 0101 (see attached pic). I'm not sure why or if anyone else is having this problem. The plugin still works. I'm just not being told of updates.

[Firedancer885]

Originally Posted by Jade Aislin

the release date says 01 Jan 0101 (see attached pic).

Mine is the same.