[GUI Plugin] FanFictionDownLoader

[Jester11]

I have started getting this error from the hpfanficarchive:

time data '' does not match format '%m %d, %Y'

[JimmXinu]

Originally Posted by chrnno

...
Error
XSGCOM: Goa'uld Defence
Hotpoint
'utf8' codec can't decode byte 0xe3 in position 51: invalid continuation byte
http://www.tthfanfic.org/Story-16092
...
Error
Escaping Fate
Arashi Leonhart
'utf8' codec can't decode byte 0xe3 in position 51: invalid continuation byte
https://www.fanfiction.net/s/6960576/1/

Also hpfanficarchive has made some major changes to the site so while I have no tested it myself problems are likely to arise there.

Those are working fine for me.

The fact you're getting the same error in the same place for two different sites suggests that perhaps you're getting some kind of error/auth page injected by your provider. Like when you're using a hotel's wi-fi and they want you to click through their web page first.

hpfanficarchive.com has changed and I have fixes for it. Posted separately.

[JimmXinu]

New test version with fixes for hpfanficarchive.net changes.

UPDATE Jan 13, 2014 - Remove obsolete beta versions

[chrnno]

Originally Posted by JimmXinu

Those are working fine for me.

The fact you're getting the same error in the same place for two different sites suggests that perhaps you're getting some kind of error/auth page injected by your provider. Like when you're using a hotel's wi-fi and they want you to click through their web page first.

hpfanficarchive.com has changed and I have fixes for it. Posted separately.

Hum, from home here and I am not aware of the connection here having anything like that. Another weird part is that I have also downloaded a dozen fics from tth and several dozen from ff.net without any problem. Will give a look at the other sites I frequent to see if any comes up. For the time being any suggestions?

Also tested an update and a new fic from hpfanfic and it worked correctly.

[JimmXinu]

Version 1.8.07 - 13 Jan 2014

• Change default encoding for finestories.com.
• Change default encoding and improve metadata for storiesonline.net, thanks davidfor.
• Fixes for hpfanficarchive.com changes.
• Fix for 'update' with no updates and 'mark' enabled.
• Now with French translation, thanks Ptitprince.

[citac]

Originally Posted by JimmXinu

It downloaded fine for me.

Make sure you're using the most recent version of the plugin.

If it still doesn't work, it may be ffnet blocking you. They've taken to doing that to people who do 'too many' requests to their servers too quickly. How many is 'too many' is unknown.

You can create a text custom column in calibre and at the bottom of the Custom Columns tab of the FFDL config, you can set FFDL to populate error messages there. This only works for updates/overwrites of existing stories, not new downloads--there's no book to write the error too for them.

Thank you, for the suggestion - it seems that it is ffnet blocking me, since I had updated both calibre and the plugin before trying to download the fic. I'll try it again now that I have some more time on my hands. And thanks for the clarification of the error column, I'll do that right now!

[Amalthia]

I was looking at the default settings in the personal.ini tab and I have "margin: 1%%" as my margin setting, but I have no clue what the percentages mean.

This is what is pasted in the personal.ini file as a guide for users but I'm not sure why the margin needs to escape and I've seen the percentages used in other areas??

## Make sure to keep at least one space
## at the start of each line and to escape % to %%

And to be clear margins aren't the same as padding in ebooks? I tend to get these two confused.

[JimmXinu]

Originally Posted by Amalthia

I was looking at the default settings in the personal.ini tab and I have "margin: 1%%" as my margin setting, but I have no clue what the percentages mean.

The defaults.ini file specifies for epubs:

 body { ...
        margin: 2%%;
        ...
}

When I first started working on FFDL, the original author included 5% margins all the way around. I later thought that was too much wasted space on my 1st generation Nook and reduced it to 2%. That was honestly the last time I really thought about how big they should be.

Originally Posted by Amalthia

This is what is pasted in the personal.ini file as a guide for users but I'm not sure why the margin needs to escape and I've seen the percentages used in other areas??

The % sign is doubled because the ini file parser included in Python uses % to indicate special characters and doubling it escapes past that behavior, leaving one % in the output.

Originally Posted by Amalthia

And to be clear margins aren't the same as padding in ebooks? I tend to get these two confused.

EPUB format books are made using a subset of (X)HTML and CSS. The difference between padding and margin is that padding is space inside the border (none in this case) and margin is outside. So for this case, there's no real difference.

W3schools has a good page about it.

[Amalthia]

Originally Posted by JimmXinu

The defaults.ini file specifies for epubs:

 body { ...
        margin: 2%%;
        ...
}

The % sign is doubled because the ini file parser included in Python uses % to indicate special characters and doubling it escapes past that behavior, leaving one % in the output.

EPUB format books are made using a subset of (X)HTML and CSS. The difference between padding and margin is that padding is space inside the border (none in this case) and margin is outside. So for this case, there's no real difference.

W3schools has a good page about it.

Thank you for the explanation. It sounds like I may not need to use padding in my css code at all. I'll try adjusting my margins and remove the padding. I got a new ebook reader and it seems like my previous settings don't quite work the same way on the new device. I figured if I'm going to change the settings I'd like to do it right.

Thank you for the link! I'll read it and maybe print a copy.

[Amalthia]

I had another question, does FFDL use any part of the Calibre conversion settings when it converts the ebooks? Like margins, font-size, css override, line height, and etc...

I ask because as I change my settings I want to make sure I don't confuse the two areas if they are separate.

[JimmXinu]

Originally Posted by Amalthia

I had another question, does FFDL use any part of the Calibre conversion settings when it converts the ebooks? Like margins, font-size, css override, line height, and etc...

No. Because FFDL also has CLI and web service versions, it uses it's own code to generate epub, mobi, html and text output.

However, I also acknowledge that FFDL's mobi output is marginal and recommend that Kindle users download to epubs and use calibre to convert to azw3 or mobi for their devices. It's better output and can use the epub update feature. That's what I do for my Kindle.

In that case, yes, the calibre conversion settings matter.

And in case you didn't notice it, calibre remembers conversion settings per book. So changing settings for one book will be used and remembered for that book, but not others.

[Amalthia]

Originally Posted by JimmXinu

No. Because FFDL also has CLI and web service versions, it uses it's own code to generate epub, mobi, html and text output.

Thanks for the explanation. I use .ePub so my conversions so far have been great.

I tend to try not to change the conversion settings book-by-book but instead use the default conversion settings I've selected.

[zraith]

Not sure if this is an error or not, but I have recently encountered problems with downloading new stories off fanfiction.net (tried it with several other sites, including fictionpress and no problems). The strange thing is that I can still update existing stories that I have had previously downloaded, the only issue is with new ones.

When trying to download a new story, the add-on gets to this part and just stays there instead of continuing:

Download FanFiction Book
FFDL:INFO:calibre_plugins.fanfictiondownloader_plu gin.jobs(44):FanFictionDownLoader v1.8.7
FFDL:INFO:calibre_plugins.fanfictiondownloader_plu gin.jobs(48):Adding jobs for URLs:
FFDL:INFO:calibre_plugins.fanfictiondownloader_plu gin.jobs(50):https://www.fanfiction.net/s/8601250/1/

I tried it with different length stories and still get the same problem, it doesn't matter if the story is 100k+ words over 40 chapters or a 1 chapter clip with <1k words.

[JimmXinu]

Originally Posted by zraith

...
When trying to download a new story, the add-on gets to this part and just stays there instead of continuing:

I tried it with different length stories and still get the same problem, it doesn't matter if the story is 100k+ words over 40 chapters or a 1 chapter clip with <1k words.

Evidence continues to mount that ffnet has instituted some sort of throttling system that significantly slows down some requests once they decide that you've made too many. We don't know what their criteria are, but it does seem to be slow downs rather than outright blocking.

The only advice I can give is don't do 'too many' downloads or update at once, or in a day. The value of 'too many' is unknown, but I'd guess it's lower than 100.

[riothamus]

I'm trying to figure out how to get FFDL to populate a custom column. Specifically, I want to populate the "Collection" column for Kindle Collections to read. I tried doing it through Reading List, but without knowing how FFDL can call it, I've had no luck. Any help would be appreciated. Otherwise I love FFDL and it works great.