[GUI Plugin] FanFictionDownLoader

[Robin42069]

sorry been busy but damn jimm your newest update seemed to fixed it. thank you very much i was honestly expecting it to be my problem and thus not fixable. all my problem updates fine now so once again thank you very much

[AmberDragon]

Originally Posted by JimmXinu

The comment field for anthologies is only set on initial create, not on update. This is a deliberate design decision so you can fill in the series description without worrying about FFDL overwriting it.

This was from way back in post #3184, but I was wondering if, after all the hubbub of moving and renaming is over, we could maybe get an option to update the comment field when we update an anthology? And maybe one to pull the generated comment from the series summary/description? It would help a lot, especially since Calibre pastes not only the copied text but the formatting into the comment box and you have to go through the HTML Source tab to clean everything up.

[Firedancer885]

Jimm, please check this story: Not Your Usual Veela Mate by Janara
I got the following error:
Error downloading Chapter: https://www.fanfiction.net/s/2571676/61/! Missing required element!
https://www.fanfiction.net/s/2571676/1/

[JimmXinu]

Originally Posted by Firedancer885

Jimm, please check this story: Not Your Usual Veela Mate by Janara
I got the following error:
Error downloading Chapter: https://www.fanfiction.net/s/2571676/61/! Missing required element!
https://www.fanfiction.net/s/2571676/1/

Downloaded successfully for me. That error can occur when ffnet has internal problems gives an error page. That's probably what happened.

[Firedancer885]

Originally Posted by Firedancer885

Jimm, please check this story: Not Your Usual Veela Mate by Janara
I got the following error:
Error downloading Chapter: https://www.fanfiction.net/s/2571676/61/! Missing required element!
https://www.fanfiction.net/s/2571676/1/

Sorry, I managed to download it on the second try.

[mcb123]

On the latest version of FFDL when I use "Update Anthology Epub" it goes through the usual process and then on add to library gives me this error:

calibre, version 2.22.0
ERROR: Unhandled exception: <b>ValueError</b>:substring not found

calibre 2.22 isfrozen: True is64bit: True
Darwin-14.1.0-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '14.1.0', 'Darwin Kernel Version 14.1.0: Thu Feb 26 19:26:47 PST 2015; root:xnu-2782.10.73~1/RELEASE_X86_64')
Python 2.7.9
OSX: ('10.10.2', ('', '', ''), 'x86_64')
Successfully initialized third party plugins: DeDRM && Kindle and Mobipocket DeDRM && Inept PDF DeDRM && eReader PDB 2 PML && all recipes Recipe Reader && EpubSplit && FanFictionDownLoader && Open With && EpubMerge && Generate Cover && Kobo Utilities && Goodreads && Reading List
Traceback (most recent call last):
File "site-packages/calibre/gui2/proceed.py", line 205, in do_ask_question
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 1467, in do_download_merge_update
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 2070, in merge_meta_books
ValueError: substring not found

[JimmXinu]

Originally Posted by mcb123

On the latest version of FFDL when I use "Update Anthology Epub" it goes through the usual process and then on add to library gives me this error:

calibre, version 2.22.0
ERROR: Unhandled exception: <b>ValueError</b>:substring not found

calibre 2.22 isfrozen: True is64bit: True
Darwin-14.1.0-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '14.1.0', 'Darwin Kernel Version 14.1.0: Thu Feb 26 19:26:47 PST 2015; root:xnu-2782.10.73~1/RELEASE_X86_64')
Python 2.7.9
OSX: ('10.10.2', ('', '', ''), 'x86_64')
Successfully initialized third party plugins: DeDRM && Kindle and Mobipocket DeDRM && Inept PDF DeDRM && eReader PDB 2 PML && all recipes Recipe Reader && EpubSplit && FanFictionDownLoader && Open With && EpubMerge && Generate Cover && Kobo Utilities && Goodreads && Reading List
Traceback (most recent call last):
File "site-packages/calibre/gui2/proceed.py", line 205, in do_ask_question
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 1467, in do_download_merge_update
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 2070, in merge_meta_books
ValueError: substring not found

What is the series URL?

[mcb123]

Originally Posted by JimmXinu

What is the series URL?

For example: http://archiveofourown.org/series/212528

However, on the dialogue box it displays each of the individual story URLs. I originally created it by using the "Get Story URLS From Webpage [http://archiveofourown.org/series/212528] and Make Anthology Epub" option

This is true not just for this anthology, but 3 other Ao3 and one fandom specific website anthology I have tried, as well.

[JimmXinu]

Originally Posted by mcb123

For example: http://archiveofourown.org/series/212528

Really? That one works fine for me.

Originally Posted by mcb123

However, on the dialogue box it displays each of the individual story URLs. I originally created it by using the "Get Story URLS From Webpage [http://archiveofourown.org/series/212528] and Make Anthology Epub" option

That's the way the anthology works. Other sites, like ffnet, don't have a series feature, so it's built so you can give it the story URLs yourself.

Originally Posted by mcb123

This is true not just for this anthology, but 3 other Ao3 and one fandom specific website anthology I have tried, as well.

Again, it works fine for me with that series URL. The only thing that occurs to me is if you're using a different skin than the standard AO3 skin.

However, there is a bug in that line in the PI that will cause problems trying to add/update a series on a site that FFDL supports using the eFiction Base code, like devianthearts.com.

Attached is a test version that fixes the problem with eFiction Base site anthologies. It may help with your AO3 problem, too.

This version also has some translation updates.

[Amalthia]

Hi,

I've noticed that for stories that don't have a lot of chapter breaks that the epub doesn't split the files into smaller pieces.

Is there a way to turn this on in FFDL?

I ask because my reader seems to cut off bits of the story when it reaches a certain size limit for the html, anything larger than 250kb seems affected. In my regular ebook epub conversions I have my setting "split files larger than" set to 160kb.

I'd appreciate any and all help.
Thanks,
Amalthia

[JimmXinu]

Originally Posted by Amalthia

I've noticed that for stories that don't have a lot of chapter breaks that the epub doesn't split the files into smaller pieces.

Is there a way to turn this on in FFDL?

I ask because my reader seems to cut off bits of the story when it reaches a certain size limit for the html, anything larger than 250kb seems affected. In my regular ebook epub conversions I have my setting "split files larger than" set to 160kb.

How often are you bumping into this? Can you give few example stories? What reader(s) specifically? And is this a problem for anyone else?

This is one of those things that sounds simple, but is actually more complex than it seems.

First, splitting an HTML file at an arbitrary byte count without breaking the structure can be pretty challenging (eg, what if it falls in the middle of a large table?). Calibre has code to do it, but FFDL also runs without calibre.

Second, FFDL reads (and counts) the files named file0001.xhtml, file0002.xhtml, etc as one chapter apiece in an existing epub to do epub updates. Split chapter files would have to be taken into account.

Using calibre's Convert or Edit Book (then save) on the epub will invoke the split function. But if you later try to do an FFDL epub update on the epub with split chapters, it won't come out correctly--some chapters may be lost and others duplicated depending on which ones were split.

So right now, your options are somewhat limited:

Manually force a convert or edit to do the split on the (hopefully few) books that need it and remember to do an over write instead of update on those books next time.

Or default to using 'Overwrite when Newer' instead of 'Update Epub' and automatically converting epub->epub every time if it happens a lot.

I'll consider options for splitting in FFDL, but unless I get some brilliant idea (or there are a lot of people with the same problem), I doubt it will happen.

[Amalthia]

Hi,

I noticed it on this story in the first part: http://archiveofourown.org/series/148518

Each part is one chapter and about 45k words.

The odd part is I've read stories from months ago that are structured the same but I don't recall parts of the story dropping off mid-sentence.

I may need to go back and see when this began to happen. I haven't changed readers. I'm still using the Sony PRS-T3.

I'll consider the option to manually convert the stories. I don't think the short stories will need it or the ones that have short chapters.

Thanks,
Amalthia

[JimmXinu]

The new name is going to be FanFicFare. We even have data to support it:



And attached is a test version of the new FanFicFare plugin you can try out.

While mostly the same as FFDL, there are some complications to changing the name:

• Because it's a different PI, your saved window sizes, hot keys and 'show this dialog again' settings will not carry over from FFDL to FanFicFare.
• Your FFDL config settings (personal.ini, etc) will be copied into FanFicFare, but once you change them in FanFicFare, they will be separate.
• Many of the translated strings will need to be updated--I'll need to ask Kovid to add/change the Transifex resource.
• We'll need a new MobileRead thread when FanFicFare is ready for release (I assume). But that may be a good thing--it's been suggested that we should split this thread into more than one topic anyway.

I've also taken this opportunity to change a few things to make them clearer and easier to find (I hope). For example, there's an additional menu entry for "Make Anthology Epub from Web Page". Previously, you had to know to use "Get Story URLs from Web Page"--which you still can.

If you have suggestions for changing menu/dialog layout or texts, now is the time to make them.

[thatblondelady]

Covers aren't working for me from FFN anymore, and I've been crawling through the ini files for hours trying to figure out why.

Settings:

## include images from img tags in the body and summary of
## stories.  Images will be converted to jpg for size if possible.
include_images:true

## If not set, the summary will have all html stripped for safety.
## Both this and include_images must be true to get images in the
## summary.
keep_summary_html:true

## If set, the first image found will be made the cover image.  If
## keep_summary_html is true, any images in summary will be before any
## in chapters.
make_firstimage_cover:true

I haven't changed my personal.ini in months, so I can't figure out what the heck I might have broken. Is there another setting somewhere I am missing?

Example URL: https://www.fanfiction.net/s/1002759...t-of-the-Ashes

Edit: I tried the new test version, and it doesn't work there either. I'm totally at a loss to figure out what is broken where.

Edit 2: This is the bit about covers in plugindefaults.ini:

## fanfiction.net's 'cover' images are really just tiny thumbnails.
## Set this to true to never use them.
#never_make_cover: false

[JimmXinu]

Originally Posted by thatblondelady

Covers aren't working for me from FFN anymore, and I've been crawling through the ini files for hours trying to figure out why.
...
I haven't changed my personal.ini in months, so I can't figure out what the heck I might have broken. Is there another setting somewhere I am missing?

Example URL: https://www.fanfiction.net/s/1002759...t-of-the-Ashes

In this case, it's nothing to do with your settings--ffnet changed the HTML for the story images a little bit--but enough that FFDL can't find them anymore with the code it's been using.

It's not a horrible problem, I've attached a test version of FFDL that should fix it.

But I'm not in a great position to release a new version of FFDL for everyone right now--I'm ~90% down the path to FanFicFare instead.