[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by ILB

Is it intentional that the uppercase column headers come first and then all the lowercase?

In so far as that's the way Python naturally sorts, yes.

Before the list was just random.

[JimmXinu]

New Test Version Posted

2020-01-09
- Make sure storyUrl has &->& and then change it back for Calibre. Matches how authorUrl is handled(in all_metadata). Closes #460
- adapter_sugarquillnet change chapter text td to div. Issue #460

I don't think this will break anything, but it does change some code that is used on every download.

[ampws]

Hi,
I am a total newbie at this - I recently downloaded the latest FanFic plugin and downloaded several stories off of Sufficient Velocity including one HUGE one called Taylor Varga so I was very happy with it until I ran into a problem.

I did as I did before and pasted the url into the plugin but this time the url failed to work it only downloaded the first post. I fiddled and was able to get it to download the visible page (25 posts) as seperate entries but there it stopped.

Unlike the Taylor Varga with nearly 400 posts it hung and didn't merge the posts.

What am I doing wrong?

The url in question:
https://forums.sufficientvelocity.co...ry-only.25054/

Yours

AMPWS

[JimmXinu]

Originally Posted by ampws

...Unlike the Taylor Varga with nearly 400 posts it hung and didn't merge the posts.

What am I doing wrong?

The url in question:
https://forums.sufficientvelocity.co...ry-only.25054/

That thread doesn't have threadmarks or a 'chapter list' of post URLs in the first post. FanFicFare's paradigm requires some kind of chapter list to be effective. This has been discussed before.

So short answer, you're not wrong, but the thread isn't compatible with FFF.

If it actually hung--that is, stopped processing and you eventually had to kill Calibre, that's a problem I'd like to hear more about.

[JimmXinu]

New Test Version Posted

2020-01-13
- Remove site nfacommunity.com, parked domain, last worked Dec2018.
- Remove site fannation.shades-of-moonlight.com, parked domain, last worked Dec2018.
- Kludge fix for incorrect HTTP response 500 code from trekfanfiction.net.
- trekfanfiction.net uses utf8 now

Last test version before next release. Probably.

[ampws]

To: JimmXinu

Many thanks for replying to my post, I was actually pulling my hair trying to figure what I was doing wrong and I don't have that much left.

It is a relief that it is not my fault, so thanks again for replying.

Yours
AMPWS

[shinykari]

I just downloaded this amazing plug in, and am hoping to update my library to make it easier to browse my fic collection on my Kindle. I already have a large number of story files (mostly downloaded from archiveofourown.org) and none of them have a URL associated with them, which is making updating the metadata almost impossible. I tried the "Get story URLs from selected books," but even though there is a link at the top and bottom of each file, FFF tells me that there are no story URLs there.

Am I doing something wrong here?

[JimmXinu]

@shinykari - In FFF's config, on the Basic tab, left side, bottom option under "Updating Calibre Options", there's a checkbox for "Search inside ebooks for Story URL?"

Checking that on should help you.

[shinykari]

Thank you! That does help a lot!

[JimmXinu]

New Version Released

Version 3.15.0 - 15 Jan 2020
- Remove site nfacommunity.com, parked domain, last worked Dec2018.
- Remove site fannation.shades-of-moonlight.com, parked domain, last worked Dec2018.
- Kludge fix for incorrect HTTP response 500 code from trekfanfiction.net.
- trekfanfiction.net uses utf8 now
- Make sure storyUrl has &->& and then change it back for Calibre. Matches how authorUrl is handled(in all_metadata). Closes #460 for adapter_sugarquillnet .
- adapter_sugarquillnet change chapter text td to div. Issue #460
- Update adapter_alternatehistorycom for XenForo2. Issue #457
- Order custom columns in config page.
- Add -U/--update-epub-always option to CLI.
- Don't do URL quoting on file: URLs. Fix for spaces->+ breaking default cover.
- Catch ',' in front of 'Thread' in forum titles. XenForo sites.
- Set minimum_calibre_version to v2.85.1, last of 2 series.
- Changes for Python3 compatibility, thanks eschwartz!

[JimmXinu]

New Test Version Posted

2020-01-16
- Optional EPUB 3.0 output

A new option virtually no-one was asking for!

You have to turn it on (see below) if you want epub 3.0 output. It works fine in my testing and checking with epubcheck (the web site version) for output, updates and unnew.

Note, however, that FFF anthologies are created by EpubMerge which doesn't write epub 3.0 yet. But it can use FFF's epub 3.0 output due to including the toc.ncx still. So I think it all works, but the final anthology will be epub 2.0 still.

Setting:

[epub]
## As of Jan 2020, FanFicFare can output either epub 2.0 or 3.0 (with
## backward compatibility toc.ncx file).  2.0 will still be the
## default for now.
epub_version:3.0

[Inanis]

Originally Posted by JimmXinu

Adding/increasing slow_down_sleep_time setting is about the only FFF change that might help. However, if you've already done enough downloads to trip a throttling mechanism, it might not help for a few minutes/hours/days/???

The other most likely option is a problem at AO3. A quick look at the front page of AO3 shows this as the second news item: Intermittent site slowness.

So I would suggest patience.

erm the issue is still happening? like it been 2 months and Ao3 staff tells me they no longer have issues but im still getting "HTTP Error 429: Too Many Requests"



PS: does slow_down_sleep_time actually does something noticeable? i started at 1.. but then 3 , 5 , 10 and started doubling it.. till i reached 50000 and i didnt see any increase in the time it took to download stuff? (increased it to 1500000 to see if it did anything.. was thinking it would take like 4 hrs to download.. but it took the same time as 5? and still had the same error)

been thinking they might have a cap per day?maybe i should try using a VPN.

[Tanjamuse]

Originally Posted by Inanis

erm the issue is still happening? like it been 2 months and Ao3 staff tells me they no longer have issues but im still getting "HTTP Error 429: Too Many Requests"

PS: does slow_down_sleep_time actually does something noticeable? i started at 1.. but then 3 , 5 , 10 and started doubling it.. till i reached 50000 and i didnt see any increase in the time it took to download stuff? (increased it to 1500000 to see if it did anything.. was thinking it would take like 4 hrs to download.. but it took the same time as 5? and still had the same error)

been thinking they might have a cap per day?maybe i should try using a VPN.

Let me know what you find out ... I haven't heard anything about solving the server problems.

[Inanis]

Originally Posted by Tanjamuse

Let me know what you find out ... I haven't heard anything about solving the server problems.

they posted on dec 19 that it got fixed and i didnt have any issues unless downloading 82+ links for an anthology

PS: using a VPN actually allowed me to download w/o any errors.. so my guess is that they have either a daily/weekly/something-else cap per IP..

[JimmXinu]

What you can do in FFF is limited. All of that, given the network errors, is really outside of FFF's direct control.

Given the reports, I agree that it is possible, even probable, that AO3 is throttling users in some way--which is honestly their right.

slow_down_sleep_time as previously discussed, adds a sleep between network requests. Inanis, I suspect you're putting slow_down_sleep_time in the wrong section or something.

Another key setting for AO3 specifically that is on by default is use_view_full_work:

[archiveofourown.org]
## AO3 offers a page to download the entire work at once.  This can
## save a lot of network traffic if there are many chapters.
## True by built-in default.
use_view_full_work:true

Turning use_view_full_work off will cause more network requests--one per chapter, but fewer database hits per network request. If AO3 is throttling by network hits, turning this off will only make it worse, but it is something to be aware of. IE, this is why total download time isn't at least sleep time X number of chapters for AO3 downloads.

With AO3 you also have the option of downloading an EPUB directly from the site. So if you have troubles downloading with FFF, that's always an option.