[GUI Plugin] FanFictionDownLoader

[watani]

Alright, thanks for the info. I look forward to the update which includes the new parser.

[JimmXinu]

In honor of it being three years since the first upload of the FFDL plugin, I'm posting a test version with some significant new features:

• Find in personal.ini/defaults.ini
• Color coding in personal.ini/defaults.ini -- automatic color code highlighting
• Error checking when saving personal.ini -- it's not complete yet, but will catch bad section names and bad regular expressions
• Using BS4/html5lib more internally even for older adapters

(That last change is bit scary just because it effects all stories from all sites.)

Some other changes also included:

• Add universe & crossoverfandom site-specific to ksarchive, clean up ini files
• FaceDeer's BS4 changes for FimF

Color Code Key:

• Red: Errors -- a bad section header will cause all settings under it to also be red
• Bold Dark Blue: recognized section headers
• Blue: recognized keywords
• Dark Green: true/false and recognized metadata entry names
• Dark Magenta: unrecognized, but not necessarily bad keywords--could be site specific or customized entry names.
• Dark Yellow: Comments -- note comments must start with '#' in the first column
• Dark Cyan: teststory sections

[cryzed]

That looks great !

[firefoxxy]

Happy New Year to you all, especially to the FDL-team.

And thank you for your wonderful work and for the tireless help you provide!

[JimmXinu]

(Thanks, firefoxxy. )

Attached is another test version, this time with ini error messages you can click on.

[JimmXinu]

Another test version now with another fix for the BS4/html5lib changes.

[mehetabelo]

I want to thank everyone that helped me a couple of weeks ago. I've been very happy with my modifications, and was able to make several more after reading through the entire thread.

That led me to two additional questions.

The first, I don't think is possible, based on one of the posts I read, but I want to verify.

Some of the sites (in this case storiesonline) do not have a word-count associated with the story, and I'm trying to get the word count placed in the title page entry by inserting information from a custom column into the entry. Generally even having the last updated information (generally 4-6k words different depending on author) is ok with me, so I don't mind if it's pulling older WC's, as long as it can pull them. The post I read mentioned that using a custom column in the calibre title page was not possible… I wanted to verify this is as I read it.

I have found a sort of workaround but inserting the metadata at the beginning, and making sure that information (wordcount) is displayed there, but this leaves me with two metadata pages/title pages. I can accept that, if it's the only choice, but I'd like to verify it. I did consider not using the title-page at all, (not sure if I can turn it off) but I like the look of it, and the way it keeps the author/title as links. Whereas calibre doesn't… though I think it may be able to with enough customization.

The second question came up while I was thinking about the above question, and trying to find an answer. I understand that the plugin can call the Count Pages plugin, but most sites don't need it. Is there a way to specify which sites call this, or does it have to be an all or nothing? I did look through the ini file extensively to see if there was any sort of option for this (and the custom column addition) but if there was anything, I either could not understand it at all, or missed it completely.

Thanks again for the help everyone! And Happy New Years!

[JimmXinu]

Attached is (yet another) test version with a new option on the FFDL Count Pages config tab to only call Word Count if the FFDL metadata is missing it. This is in response to mehetabelo's 2nd request--it seems a better solution to me than specifying by site.

So the Count Pages plugin can potentially be called twice--once with a list of books that need Word Count and again with the full list of downloaded/updated books to update Page Count, etc. If you only use Word Count (or only Page Count, or leaving the Missing option off), it will still only be called once.

As for mehetabelo's 1st request--injecting Word Count into the FFDL Title Page when the site doesn't provide it--that's considerably trickier. I don't see a way to do it that I like.

For one thing, the Count Pages plugin isn't run until after FFDL has finished. Outdated info is worse than missing info, IMO. Series can already be injected if it's set in calibre (but not available in FFDL metadata) because that scratched a personal itch I had about ffnet not having a series mechanism--and series doesn't change with every update.

If Word Count is that big a deal to you, in addition to the solution you already have, you could also consider the solution I use, which is a calibre Metadata Plugboard setting to put the tags and word count in as part of the title when books are transferred to my Nook and Kindle.

title:

{#title_prefix:|| }{series:|| }{series_index:0>2s|[|] - }{title}{#wordcount:| (|)}{tags:| [|]}{#extranotes:| [|]}

Generate Cover with word count in it is only an option if you run it yourself--FFDL calls GC before it calls Count Pages.

[Firedancer885]

Hi Jimm,

is there a way to use slow_down_sleep_time to slow down AO3 downloads? I'm asking because of the downloader part of this: http://archiveofourown.org/admin_posts/2183

[JimmXinu]

Fix for 'Only if Word Count is Missing'.

[JimmXinu]

Originally Posted by Firedancer885

is there a way to use slow_down_sleep_time to slow down AO3 downloads? I'm asking because of the downloader part of this: http://archiveofourown.org/admin_posts/2183

You may use slow_down_sleep_time with any site by putting it in personal.ini in that site's section. Or for all sites by putting it in defaults.

If you determine a good sleep time for AO3, post it and I'll make it part of the built-in default settings.

[mehetabelo]

Uhm. Thanks for updating the plugin. ... I updated it, and it didn't work on storiesonline. I tried new stories (No existing WC), updated stories with and without WC, and stories that I was just 'updating' to make sure.

Next, I restarted Calibre twice before trying anything further, to verify that everything had been working correctly. I did make sure the checkbox was checked, tried variations of it (like checking then turn the first box off... which obviously didn't work, as it disabled the new checkbox and it was unchecked the next time I went back in).

I went through my ini file and deleted the word call feature under storiesonline, because I'd left it there in case they ever added that feature. Then, to make sure it wasn't something in my ini file, I completely disabled everything under the storiesonline options, with the exception of logon information.

And... nothing, never once did it call WC. I did check, and it still calls it if every time I keep only call WC checked (including on other sites like ff.net). I did this just to verify nothing got somehow crossed in the program modification. I'm not sure what else I can do. I thought I was doing everything right. Perhaps not.

[JimmXinu]

Originally Posted by mehetabelo

Uhm. Thanks for updating the plugin. ... I updated it, and it didn't work on storiesonline. I tried new stories (No existing WC), updated stories with and without WC, and stories that I was just 'updating' to make sure.
...

I think this version covers all the WC/WCMissing cases correctly, give it a try.

[mehetabelo]

Works exactly as it's supposed to now with storiesonline, though I don't know of any other sites that don't provide word counts, so I can't speak to that. I may try the idea of using wordcount in the title, see if I can get used to it. Thanks again.

[coolshades]

i have not got the FFDL to work on my Windows PC. when i enter the link (from one of the supported sites) in the application and press check, or check and download, nothing happens at all.

i want a Mobi output.

any help appreciated