[GUI Plugin] FanFicFare

[JimmXinu]

New Test Version Posted

2019-11-02
- At least one efiction site said Completed: Completed instead of Yes.

@AndersW -

I've fixed the In-Progress vs Completed. The other metadata is collected in different entries, from defaults.ini:

[tgstorytime.com]
## ...
## This site has a number of additional site specific metadata
## entries.  This is the first test case of base_efiction 'Auto
## metadata' automatically including unrecognized metadata.  Still
## requires entries in extra_valid_entries to be used.
extra_valid_entries:turnedinto,featureditems,locale,motivationforchange,sexualorientation,storytheme,bodymodification,personality,storytype,typeofchange

You can add all those to your tags field and/or title page in personal.ini:

[tgstorytime.com]
add_to_include_subject_tags:,turnedinto,featureditems,locale,motivationforchange,sexualorientation,storytheme,bodymodification,personality,storytype,typeofchange
add_to_titlepage_entries:,turnedinto,featureditems,locale,motivationforchange,sexualorientation,storytheme,bodymodification,personality,storytype,typeofchange

FYI, mobileread forums technically forbid NSFW links. I suggest you remove the link above and send by PM next time.

[DavidBrookes]

Are we having any problems with the plugin? I noticed when I ran my monthly check of fan fiction sites to update my libraries that it will run through the full list of URLS but only find a couple of updates instead of the full amount. Initially I thought it was only FF.Net but it did on two other sites before I gave up.

[JimmXinu]

Originally Posted by DavidBrookes

Are we having any problems with the plugin? I noticed when I ran my monthly check of fan fiction sites to update my libraries that it will run through the full list of URLS but only find a couple of updates instead of the full amount. Initially I thought it was only FF.Net but it did on two other sites before I gave up.

First, I would direct you to this FAQ:
Why shouldn't I just select my entire library and use FanFicFare plugin to check for updates?

I am not aware of any such issues. But without story URLs that exhibit the problem, an error message, a debug log showing the issue (or ideally all of those), there's nothing for me to investigate.

[DavidBrookes]

Originally Posted by JimmXinu

First, I would direct you to this FAQ:
Why shouldn't I just select my entire library and use FanFicFare plugin to check for updates?

I am not aware of any such issues. But without story URLs that exhibit the problem, an error message, a debug log showing the issue (or ideally all of those), there's nothing for me to investigate.

Sorry about that - I forgot to attach my log files from one of my attempts. Have attached it this time. One of the stories that I know that has updated is https://archiveofourown.org/works/2812049 and I have attached the exported ePub from my library as well. I tried reverting back to Calibre 3.48 as a test to no effect. Do you have the version of the plugin from around the beginning of October so I can revert back to that version as a test?

[JimmXinu]

Originally Posted by DavidBrookes

Sorry about that - I forgot to attach my log files from one of my attempts. Have attached it this time. One of the stories that I know that has updated is https://archiveofourown.org/works/2812049 and I have attached the exported ePub from my library as well. I tried reverting back to Calibre 3.48 as a test to no effect. Do you have the version of the plugin from around the beginning of October so I can revert back to that version as a test?

First, again, that's waaay more updates at once than I recommend. We have seen sites start blocking users' IP addresses before. I usually draw the line at ~30 at a time. To the point that I'm not going to try to duplicate it as shown. I have done some testing with a smaller set, though.

(You might reduce traffic by excluding completed works--21 of the first 30 URLs in your list are already status Completed.)

Second, that's the background job output.

What I see in the file attached is consistent with 'Background Metadata' option OFF and most of those files being newer--having been touched by something to make their filesystem date newer than the site. That something does not have to have been FFF.

In earlier versions of Calibre, the reader saving it's place would update the epub. The current version still has options for saving bookmarks/etc in the epub that would do the same.

That's one of the major disadvantages of 'Overwrite if Newer' mode.

Have you tried running Calibre in debug mode? (Calibre Preferences drop down.) That will give more info in the debug log, including file timestamps for the overwrite if newer check.

Third, past released versions of FFF are available in the GitHub project.

[Tanjamuse]

Hi JimmXinu.

Is it possible to add the option for fixing case in custom collumns as well?

I really like the option for fixing it in Title and Author.

And is there somewhere I can add which options shouldn't be tweaked like The?

[SallyK]

Hi - I haven't used FanFicFare for a couple of months, and I'm finding something that used to work isn't any more. I've been using

include_subject_tags:
extratags:New

in my personal.ini to remove all the tags and add just one that says New, until I've read the story. Downloading from AO3 and fanfiction.net today, the tags are removed, but the one saying New isn't added. It's not the end of the world, but I was hoping it might be easy to fix.

[JimmXinu]

Originally Posted by SallyK

Hi - I haven't used FanFicFare for a couple of months, and I'm finding something that used to work isn't any more. I've been using

include_subject_tags:
extratags:New

in my personal.ini to remove all the tags and add just one that says New, until I've read the story. Downloading from AO3 and fanfiction.net today, the tags are removed, but the one saying New isn't added. It's not the end of the world, but I was hoping it might be easy to fix.

Back in May 2019 it was changed so extratags is included/excluded by include_subject_tags, which is more logically correct. Use:

include_subject_tags: extratags
extratags:New

[SallyK]

Ah, thank you, I hoped it was something as simple as that. Help much appreciated.

[vargr]

I'd like to have a field for total chapters as visible on AO3 (so I can have template results like "Chapters: 3/6" as on the site), but I can only seem to find current number of chapters and In-Progress/Complete status to work with. Is there a way for FFF to access the total chapter count?

(In comparison, when I use AO3's built-in download button to download a fic, it does include the "Chapters: 3/6" info, so it's coming from somewhere, just don't know how to tell FFF to make that part of the metadata?)

Thanks/sorry if this is unclear at all.

[JimmXinu]

Originally Posted by vargr

I'd like to have a field for total chapters as visible on AO3 ...

Hmmm. I'm a bit surprised that's not something currently collected. adapter_archiveofourownorg determines In-Progress/Completed by comparing Chapter: X/Y after all.

I can add it, but I wonder if it should be as "5/33" or as a separate "33". I'm thinking as the original string, since it can also be "2/?".

I'm also at a loss what to name it. chapterratio?

[JimmXinu]

Originally Posted by Tanjamuse

Is it possible to add the option for fixing case in custom columns as well?

I really like the option for fixing it in Title and Author.

And is there somewhere I can add which options shouldn't be tweaked like The?

How about a more generic 'call python's title()' solution as a personal.ini setting?

Something like a titlecase_entries setting that you could give a list of metadata entries to. I'd apply it before include/replace/exclude so you could address words like 'The' and 'A' in replace_metadata yourself if so inclined. (I have included something similar once before in the base_xenforo specific capitalize_forumtags setting which also does English contractions properly.)

Is this something anyone would use?

FYI, the plugin options for fixing title and author call Calibre routines specifically for that (from calibre.ebooks.metadata.sources.base fixcase and fixauthors). The 'small words' that aren't capitalized by those routines are hard-coded in Calibre. And the Calibre routines rely on enough other code that I judge it impractical to pull them into FFF.

[Tanjamuse]

Originally Posted by JimmXinu

How about a more generic 'call python's title()' solution as a personal.ini setting?

Something like a titlecase_entries setting that you could give a list of metadata entries to. I'd apply it before include/replace/exclude so you could address words like 'The' and 'A' in replace_metadata yourself if so inclined. (I have included something similar once before in the base_xenforo specific capitalize_forumtags setting which also does English contractions properly.)

Is this something anyone would use?

FYI, the plugin options for fixing title and author call Calibre routines specifically for that (from calibre.ebooks.metadata.sources.base fixcase and fixauthors). The 'small words' that aren't capitalized by those routines are hard-coded in Calibre. And the Calibre routines rely on enough other code that I judge it impractical to pull them into FFF.

Thanks for the explanation, and the consideration.

I'll look forward to see if you figure something out.

Thanks again for a great plugin.

[JimmXinu]

New Test Version Posted

2019-11-05
- Add user/pass to adapter_fictionhuntcom, required to see chapter text now.
- Discard cookie cache on library change--might use different users/settings.

[JimmXinu]

New Test Version Posted

2019-11-07
- Fixes for adapter_webnovelcom site changes to JSON.
- Add parentforums site specific metadata for base_xenforo, include partial list in category.

This includes these changes in defaults.ini to include a small subset of parent forums names in category in the xenforo sites (SB,SV,AH,QQ).

## Copy title to tagsfromtitle for parsing tags.
add_to_extra_valid_entries:,tagsfromtitledetect,tagsfromtitle,forumtags,parentforums

parentforums:Parent Forums
keep_in_order_parentforums:true

## What forum a thread is in can be an indicator of it's category, but
## most are too general to be useful; only keep the more specific
## ones.  You can replace add to this list in personal.ini
## [base_xenforoforum], or add to it in the sites' [section].
add_to_include_metadata_pre:
 parentforums==Worm
 parentforums==NSFW
 parentforums==Original Fiction

include_in_category:category,tagsfromtitle,parentforums