[GUI Plugin] FanFictionDownLoader

[thatblondelady]

Forgive me if this is documented somewhere, but I can't figure out for the life of me how to get the error column set up. I am familiar with the process of configuring a custom column in calibre, but I don't see an Error option at all in the custom tags screen. What am I missing?

https://www.evernote.com/shard/s10/s...88f5cf1dbf1275 << this is the stuff I see under the custom tags option.

[JimmXinu]

Originally Posted by thatblondelady

Forgive me if this is documented somewhere, but I can't figure out for the life of me how to get the error column set up. I am familiar with the process of configuring a custom column in calibre, but I don't see an Error option at all in the custom tags screen. What am I missing?

Scroll down to the bottom of the tab. The minimum size is dictated by the largest tab, so even though that tab doesn't need more space, it gets scrolled anyway.

[thatblondelady]

I feel so dumb. I've been looking at this for hours and never noticed the freeking scroll bars. I thought it was in that area I linked the picture of.

I'm all sorted, thanks.

[Firedancer885]

Jimm,

please ask the AO3 mods about the latest bot issue. I'd like to use FFDL to download AO3 stories in the future and I'm not sure whether the downloader is considered a bot by them.

[BookJunkieLI]

I don't know if FFDL is considered a bot by AO3 but, based on the error message that was going out sitewide last night when they had problems, it looks like they have a number of downloads per minute limit which appears to be 6 since they requested waiting at least 10 seconds between each download.

I typically download 10-15 fics at a time, 25 if I'm feeling impatient, using a 15 second slow_down_sleep_time setting in my personal.ini and I rarely have an issue outside when AO3 is having overall site issues.

[JimmXinu]

Originally Posted by BookJunkieLI

I don't know if FFDL is considered a bot by AO3 but, based on the error message that was going out sitewide last night when they had problems, it looks like they have a number of downloads per minute limit which appears to be 6 since they requested waiting at least 10 seconds between each download.

I don't use AO3 regularly, so I will be guided by the reports of the users that do.

Is there consensus that I should set a longer default slow_down_sleep_time for AO3? And how long?

I've had no problems in the last couple days downloading 1-2 stories of modest length (<20 chapters) with slow_down_sleep_time:2.

[Firedancer885]

I have no idea how long a sleep_time to use. I'm currently using 1.5 but I check different number of stories each day. I only check the ones I get an alert for. That's 3 to 15 stories.

[Starbeam]

I don't know if it's just me, but with v2.1.2, I've been getting SSL errors on Fictionpress and Fanfiction.net, e.g.:

Bad
An Unlikely Magical Girl and an Unlikely World War
Kilopi505
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)>
https://www.fictionpress.com/s/3189505/1/

In my browser I can access the https versions of the sites and log in without issue, so there doesn't seem to be a problem with their certificates.

[JimmXinu]

Originally Posted by Starbeam

I don't know if it's just me, but with v2.1.2, I've been getting SSL errors on Fictionpress and Fanfiction.net, e.g.:

Bad
An Unlikely Magical Girl and an Unlikely World War
Kilopi505
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)>
https://www.fictionpress.com/s/3189505/1/

In my browser I can access the https versions of the sites and log in without issue, so there doesn't seem to be a problem with their certificates.

What is your OS version? Is it up to date? And do you have the latest calibre version?

That error suggests that your computer isn't able to verify the SSL cert from the site.

Typically, your browser and OS come with sets of top level SSL certs that are used to verify individual site certs. I don't know where calibre gets its initial cert store from, but it could be from the OS.

I'm not seeing this issue with Win8 and the calibre 2.16.

[Starbeam]

Originally Posted by JimmXinu

What is your OS version? Is it up to date? And do you have the latest calibre version?

That error suggests that your computer isn't able to verify the SSL cert from the site.

Typically, your browser and OS come with sets of top level SSL certs that are used to verify individual site certs. I don't know where calibre gets its initial cert store from, but it could be from the OS.

I'm not seeing this issue with Win8 and the calibre 2.16.

Mageia Linux 4.1 (latest version), calibre 2.16 (also latest version)
Thank you for your response, of course.

[BookJunkieLI]

@JimmXinu: I'm a bit OCD so I tend to download whatever has been posted/updated each day in the pairings I follow. In most fandoms that's only a few fics a day or even a week. In my big fandoms like Arrow, Hobbit, and Harry Potter that's between 20 to 30 fics a day. I average about 200-250 stories a day.

I run it with slow_down_sleep_time:15.0
A light user would probably be fine with a 5.0 setting.

Honestly though it's hard to say what the best setting is right now. I was doing my usual downloads, 10 at a time and had no problems then about 4pm EST suddenly it was taking forever. When I looked at the Job List Calibre/FFDL was on fic 1 of 10 at 1% for over 40 minutes. I finally cancelled it and tried again but cancelled after 10 minutes of the same thing. Finally I set it to download 1 story after checking that AO3 was up and functioning. It took 13 minutes to download. Same happened with the next 5 stories before it suddenly sped back up again. My thought is, at least until AO3 has things sorted out slow_down_sleep_time should probably be set at 5.0 as a base. The slowdown isn't really noticeable, I think, and it eases the load on AO3. Power users, like me, should probably consider setting their personal.ini to at least 10.0, even 15.0. The slowdown is a little noticeable at that point but I always had FFDL running in the background anyway while I did other stuff so it just means I might actually finish my solitaire game before Calibre/FFDL is done instead of being interrupted halfway through it.

[JimmXinu]

New test version with:

- Add slow_down_sleep_time:2 to AO3 defaults.

- Fix setDescription for adapter_tolkienfanfiction.

[JimmXinu]

FYI, Twisting the Hellmouth (one of my personal favorite fanfic sites) is in the process of moving servers and domains.

As of this writing, it isn't working well enough to download stories (author links are broken), so I haven't updated FFDL for it yet.

[QueenFrigga]

Hello there,

I've been using the plugin for a while now and I find it quite useful. However, I was wondering where exactly the ebooks downloaded are stored once the conversion is done. Is there a way to simply grab the converted ebooks from whichever folder or must you add the books to calibre when prompted in order to have access the ebooks converted by the plugin?

Thanks!

[JimmXinu]

Originally Posted by QueenFrigga

I've been using the plugin for a while now and I find it quite useful. However, I was wondering where exactly the ebooks downloaded are stored once the conversion is done. Is there a way to simply grab the converted ebooks from whichever folder or must you add the books to calibre when prompted in order to have access the ebooks converted by the plugin?

You can find the book files in the calibre library directory. If you select a book in calibre and hit the 'o' key (in windows at least) it will open the directory for you.

The plugin is integrated with calibre, but FFDL also has a command line Python version and a web service (in fact, those pre-date the plugin version by a couple years). You can download stories using those without using calibre at all.

The FFDL plugin can also be called from command line, same as the Python CLI. Look at the first post in the thread and read the 'Command Line Interface(CLI)' section.

(FYI, the FFDL plugin doesn't do 'conversion' as such. It downloads HTML from the sites and generates epub (or other output formats) directly. It can then launch calibre's conversion process for you.)