[GUI Plugin] FanFicFare

[Depletion]

Hello, just posting to say the new test version for fimfiction fix is working well for me.

[baggins41]

Originally Posted by FaceDeer

Learning to use Git while having had 3 hours of sleep in the past 48 is beyond me, if this fix is good I assume Jimm will take it.

Git's commands can be hard enough to grok with a perfectly clear head let alone while sleep deprived.

Thanks for figuring out the fix and putting out the patched version so quickly, Facedeer and Jimm. It's working great so far.

[ij26]

Originally Posted by JimmXinu

New test version posted (direct link).

2016-02-23
- Fix for fimfiction.net site update -- thanks Facedeer

Works for me; thanks.

[megme]

Never mind. I had a problem with fictionpad downloads, but it's working now.

[JimmXinu]

New test version posted (direct link)

2016-02-25
- New site: www.artemis-fowl.com - Thanks gcomyn!
- New site: www.naiceanilme.net - Thanks gcomyn!
- New site: fanfiction.lucifael.com - Thanks gcomyn!
- New site: buffygiles.velocitygrass. - Thanks gcomyn!
- New site: www.andromeda-web.com - Thanks gcomyn!
- New site: www.therabidreader.com - Thanks gcomyn!

[iakovl]

have some strange issue with FFF not being able to download all fics i want from fimfic

i want to update 7 fics, end up with only 3

FFF:INFO:calibre_plugins.fanficfare_plugin.jobs(45):FanFicFare v2.2.18
FFF:INFO:calibre_plugins.fanficfare_plugin.jobs(49):Adding jobs for URLs:
FFF:INFO:calibre_plugins.fanficfare_plugin.jobs(51):http://www.fimfiction.net/story//
FFF:INFO:calibre_plugins.fanficfare_plugin.jobs(51):http://www.fimfiction.net/story//
FFF:INFO:calibre_plugins.fanficfare_plugin.jobs(51):http://www.fimfiction.net/story//
FFF:INFO:calibre_plugins.fanficfare_plugin.jobs(51):http://www.fimfiction.net/story//
FFF:INFO:calibre_plugins.fanficfare_plugin.jobs(51):http://www.fimfiction.net/story//
FFF:INFO:calibre_plugins.fanficfare_plugin.jobs(51):http://www.fimfiction.net/story//
FFF:INFO:calibre_plugins.fanficfare_plugin.jobs(51):http://www.fimfiction.net/story//

thats the job it gets (removed fic id)

then fails with 404 but it does work on the first 3 so...?

Download Results:
http://www.fimfiction.net/story// Download epub completed, 9 chapters.
http://www.fimfiction.net/story// Download epub completed, 16 chapters.
http://www.fimfiction.net/story// Download epub completed, 55 chapters.
http://www.fimfiction.net/story// HTTP Error 404: 
http://www.fimfiction.net/story// HTTP Error 404: 
http://www.fimfiction.net/story// HTTP Error 404: 
http://www.fimfiction.net/story// HTTP Error 404: 

FFF:INFO:calibre_plugins.fanficfare_plugin.jobs(99):
Successful:
http://www.fimfiction.net/story//
http://www.fimfiction.net/story//
http://www.fimfiction.net/story//

FFF:INFO:calibre_plugins.fanficfare_plugin.jobs(100):
Unsuccessful:
http://www.fimfiction.net/story//
http://www.fimfiction.net/story//
http://www.fimfiction.net/story//
http://www.fimfiction.net/story//

[JimmXinu]

Originally Posted by iakovl

have some strange issue with FFF not being able to download all fics i want from fimfic

Since you removed the story ID numbers, I cannot test any of the stories you report having problems with.

However, your problem appears to be identical to the one already reported (see the previous page of the thread) and has been fixed in the current test version, see second post at the beginning of the thread.

[iakovl]

Originally Posted by JimmXinu

has been fixed in the current test version, see second post at the beginning of the thread.

works now

[edeniz]

Just a quick check:

Is it normal that the "download_metadata" part of the download process alone takes up to 5+ minutes? I mean, I probably can ascribe that to there being far too many "exclude_metadata_pre", right? Or should I look for faulty/inefficient lines?

[JimmXinu]

Originally Posted by edeniz

Is it normal that the "download_metadata" part of the download process alone takes up to 5+ minutes? I mean, I probably can ascribe that to there being far too many "exclude_metadata_pre", right? Or should I look for faulty/inefficient lines?

How many stories you're downloading/updating and which sites they are on is usually the biggest factor.

The second biggest factor in my experience is the number of story URLs in your Reject List.

Both of those will be further slowed by large numbers of replace_metadata or in/exclude_metadata lines.

I have some new changes I'm currently testing that speed up the Reject List checking considerably as well as adding some new ini functionality. I plan to post a new version for others to test early this week.

[edeniz]

Originally Posted by JimmXinu

How many stories you're downloading/updating and which sites they are on is usually the biggest factor.

The second biggest factor in my experience is the number of story URLs in your Reject List.

Both of those will be further slowed by large numbers of replace_metadata or in/exclude_metadata lines.

I have some new changes I'm currently testing that speed up the Reject List checking considerably as well as adding some new ini functionality. I plan to post a new version for others to test early this week.

Hm. Thank you for clearing that up for me. Now I'm curious about this new test version.

But it's interesting. I have nothing in the Reject List, as it has been to date irrelevant to me. The archive in question is AO3. And I meant the long time thing actually for a single story only. For example, this fic

http://archiveofourown.org/works/4327644

took almost 10 minutes to finish the "download_metadata" part and I wasn't trying to download more than one fic at a time. (I try to avoid that for the sake of the servers' limits.) On the other hand, this fic

http://archiveofourown.org/works/1292134

has proved to be impossible to download for me with my ini, as it caused calibre to freeze up. (That is, I waited well over an hour to see if it would go past the "download metadata" stage, and when it didn't I gave up and forced calibre shut, because it really had frozen.)

Since a download without the replace/exclude metadata lines took only a few minutes, I could only assume it was either a bad line or simply too many lines. (This happens only with ao3 fics, since it's the archive where I have the single most replace/exclude lines. My ini is 123kb big--so yeah, a lot of lines.)

[kapaka]

Am I the only one that cannot download from storiesonline?

[JimmXinu]

Originally Posted by kapaka

Am I the only one that cannot download from storiesonline?

Do you have the latest released version of FanFicFare? That site made changes recently.

Second, using the latest test version, I was able to download at least one story from that site. So it looks fine to me.

When reporting problems, always include at least one story URL that exhibits the problem and the failure message you receive. Ideally, you'd also include the log from running calibre in debug mode while trying. (Under Calibre Preferences drop down.)

Without knowing some of the story URLs that failed for you, I cannot help any further.

[JimmXinu]

New test version posted (direct link)

2016-02-29

- Performance improvements (extra caching) for complex metadata manipulation.

- Changes to all plugin logging output (debug) to add datetime and reduce inadvertent smilies. ("FFFEBUG")

- Additional ini options for base_xenforoforum adapters.

Spoiler:

CODE]## When given a thread URL, use threadmarks as chapter links when
## there are at least this many threadmarks. A number of older
## threads have a single threadmark to an 'index' post. Set to 1 to
## use threadmarks whenever they exist.
minimum_threadmarks:2

## When 'first post' (or post URL) is being added as a chapter, give
## the chapter this title.
first_post_title:First Post

## In normal operation, if given a post URL or a thread URL with less
## than minimum_threadmarks, the given post or the first post of the
## thread will be included as the first chapter (with chapter title
## from first_post_title) unless that post is explicitly linked to in
## the collected chapter list. First post is not included when using
## thread marks.
##
## If always_include_first_post:true, then the given or first post
## will be included as above even if it is a link in the post or even
## if threadmarks are used. Can result in a duplicated chapter.
always_include_first_post:false

## In normal operation, forumtags will only be populated when
## threadmarks are used for chapters (see minimum_threadmarks above).
## When always_use_forumtags:true, always populate forumtags.
always_use_forumtags:false[/CODE]

- Add storyUrl ini section feature. Add custom configuration for individual stories by adding personal.ini sections specifically for them. Example:

Spoiler:

[https://forums.spacebattles.com/threads/346640/]
always_include_first_post:true
first_post_title:Prologue and Chapter 1
extratags: Original, Sci-Fi, AI

[Estrecca]

Hi.

I seem to be having some issues with questionablequesting in regards to its nsfw content and would appreciate any assistance that could be provided.

I am pretty sure that I've added a valid ID and password (with the relevant permissions in regards to the desired content) to personal.ini, but when I try to download what I want, I get 403 errors.

Now, I am pretty damn terrible in regards to tinkering under the hoods of programs or, generally, anything involving code, but the modifications seem straightforward enough that even I shouldn't have been able to fuck this up.

Any suggestions?