[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by sios00

How do I set Fanficfare to auto set a column value for all downloads? My Calibre library has a custom column called "Type" with tag #type, which is a dropdown list of values "Fiction", "Fanfiction", and "Non-fiction". How do I get Fanficfare to set the #type column value of all downloads and updates as "Fanfiction"? Currently, I am doing it manually whenever I download a new story. Thanks for any assistance!

Turn on the check box towards the bottom of the Custom Columns settings Tab to allow custom_columns_settings, then set it in personal.ini:

[defaults]
# ...
custom_columns_settings:
 "Fanfiction"=>#type

You may have to match the allowed value perfectly, IE Fanfiction vs FanFiction?

It's even one of the examples in (plugin) defaults.ini:

## (Plugin Only) - You can also populate calibre custom columns with
## the site specific metadata using custom_columns_settings (but only
## if 'Allow custom_columns_settings from personal.ini' is checked in
## the plugin GUI config.)  There are three parts, the entry name,
## then the label of the calibre custom column, then (optionally) a
## 'mode'.  'r' to Replace any existing values, 'a' to Add to existing
## value (use with tag-like columns), and 'n' for setting on New books
## only.  (Default is 'r'.)

## Literal strings can be set into custom columns using double quotes.
## Each metadata=>column mapping must be on a separate line and each
## needs to have one space at the start of each line.

## 'r_anthaver' and 'n_anthaver' can be used to indicate the same as
## 'r' and 'n' for normal downloads, but to average the metadata for
## the differents story in an anthology before setting in integer and
## float type custom columns.  This can be useful for a averrating
## column, for example.  Default is to sum the values of all stories,
## and numChapters and numWords are always summed.

#custom_columns_settings:
# cliches=>#acolumn
# themes=>#bcolumn,a
# timeline=>#ccolumn,n
# "FanFiction"=>#collection
# averrating=>#averrating,r_anthaver

[sios00]

Thanks! It didn't work for me though, is it because I have "custom_columns_settings" twice? Once under [defaults] and another under [archiveofourown.org]?

For reference, here is my personal.ini file.

Spoiler:

[defaults]
custom_columns_settings:
 "Fanfiction"=>#type,r


<.......unrelated>


## This section will override anything in the system defaults or other
## sections here.
[overrides]
## default varies by site.  Set true here to force all sites to
## collect series.
#collect_series: true

[fanfiction.net]
#use_nsapa_proxy:true
#use_cloudscraper:false
use_flaresolverr_proxy:true

[archiveofourown.org]
extra_valid_entries:fandoms, freeformtags, freefromtags,  ao3categories, comments, chapterslashtotal, chapterstotal, kudos,  hits, bookmarks, collections, byline, bookmarked, bookmarktags,  bookmarksummary, bookmarkprivate, bookmarkrec, restricted, series00,  series01, series02, series03, series04, series05, series06, series07,  series08, series09, series00Url, series01Url, series02Url, series03Url,   series04Url, series05Url, series06Url, series07Url, series08Url, series90Url,  series00HTML, series01HTML, series02HTML, series03HTML, series04HTML,  series05HTML, series06HTML, series07HTML, series08HTML, series09HTML 
fandoms_label:Fandoms 
freeformtags_label:Freeform Tags 
freefromtags_label:Freeform Tags 
ao3categories_label:AO3 Categories 
comments_label:Comments 
chapterslashtotal_label:Chapters/Total Chapters 
chapterstotal_label:Total Chapters 
kudos_label:Kudos 
hits_label:Hits 
collections_label:Collections 
## Count of bookmarks on story by all users 
bookmarks_label:Bookmarks 
## Tags & Summary from *your* bookmark on the story.  Only collected 
## if always_login:true 
bookmarked_label:I Bookmarked Story 
bookmarktags_label:My Bookmark Tags 
bookmarksummary_label:My Bookmark Summary 
bookmarkprivate_label:My Bookmark Private 
bookmarkrec_label:My Bookmark Rec 
restricted_label:Restricted to Registered Users 
series00HTML_label:Series 
series01HTML_label:Additional Series 
series02HTML_label:Additional Series 
series03HTML_label:Additional Series 
series04HTML_label:Additional Series 
series05HTML_label:Additional Series 
series06HTML_label:Additional Series 
series07HTML_label:Additional Series 
series08HTML_label:Additional Series 
series09HTML_label:Additional Series  
make_linkhtml_entries:series00,series01,series02,series03,series04,series05,series06,series07,series08,series09  
titlepage_entries: seriesHTML,category,genre,language,characters,ships,status,datePublished,dateUpdated,dateCreated,rating,warnings,numChapters,numWords,site,description 
extra_titlepage_entries: fandoms,freeformtags,ao3categories,comments,chapterslashtotal,chapterstotal,kudos,hits,bookmarks,bookmarked,bookmarktags,bookmarksummary,series01HTML,series02HTML,series03HTML,series04HTML,series05HTML,series06HTML,series07HTML,series08HTML,series09HTML,byline  
custom_columns_settings: 

 series=>#multiseries,a
 series00=>#multiseries,a
 series01=>#multiseries,a
 series02=>#multiseries,a
 series03=>#multiseries,a
 series04=>#multiseries,a
 series05=>#multiseries,a
 series06=>#multiseries,a
 series07=>#multiseries,a
 series08=>#multiseries,a
 series09=>#multiseries,a


include_logpage: true

#include_logpage: smart

logpage_at_end: true


logpage_entries: dateCreated,<br />,datePublished,<br />,dateUpdated,<br />,numChapters,numWords,status,<br />,series,<br />,title,<br />,author,<br />,description,<br />,category,<br />,genre,<br />,rating,<br />,warnings,<br />,fandoms,<br />,freeformtags,<br />,freefromtags,
 ao3categories,<br />,comments,<br />,chapterstotal,<br />,kudos,
 hits,<br />,bookmarks,<br />,collections,<br />,byline,<br />,bookmarked,<br />,bookmarktags,
 bookmarksummary,<br />,bookmarkprivate,<br />,bookmarkrec,<br />,restricted,<br />,series00,
 series01,<br />,series02,<br />,series03

[JimmXinu]

Originally Posted by sios00

Thanks! It didn't work for me though, is it because I have "custom_columns_settings" twice? Once under [defaults] and another under [archiveofourown.org]?

Yes. You'll want to use add_to_custom_columns_settings. See the wiki for all the details.

[sios00]

Originally Posted by JimmXinu

Yes. You'll want to use add_to_custom_columns_settings. See the wiki for all the details.

Thank you for this!

[dysphere]

Originally Posted by JimmXinu

I experimentally downloaded the first 90 chapters, then updated a couple times, first to 91 then to 94 comparing between each--no problems found.

Is it just a change in size, or can you identify text that was lost?

An epub is a zip file. Zip file size depends on how compressed it is, which depends on the contents. Pretty sure I've seen a zip file get smaller before when containing more text if the text happens to be more compressible.

Calibre's Edit book has a "Compare to another book" feature (under Files) that's very useful. If you open an epub in Edit book before updating with FFF, you can then Compare to the updated file on disk (find the epub on disk from the Calibre library with the 'o' hotkey) to see exactly what's changed.

Or if you have the before and after epubs you could post somewhere for me to examine, it may help.

Hmm, it looks like it's removing more divs than necessary, starting on the 22nd file, which causes the location number to change on the kindle, which makes it more difficult for me to find where I was reading last. Epubs attached so you can compare.

[watani]

As far as I can tell, when the update mode is set to "Update EPUB if New Chapters" for most sources any manual edits you make to previous chapters get kept and only the new chapters get downloaded and appended. I seem to be encountering an exception to this when updating a story from AO3, however.

There is a specific story on AO3 that has an in-text higher resolution image available than the one that FFF downloads, so I replace it in the EPUB manually (tried with both the original filename and with the FFF filename and format). The next time I update the fic the image gets reverted to the lower resolution FFF downloaded version. Why is this occurring and can I prevent it from happening?

[JimmXinu]

Originally Posted by watani

There is a specific story on AO3 that has an in-text higher resolution image available than the one that FFF downloads, so I replace it in the EPUB manually (tried with both the original filename and with the FFF filename and format). The next time I update the fic the image gets reverted to the lower resolution FFF downloaded version. Why is this occurring and can I prevent it from happening?

The difference you are seeing is text(html really) vs image, not AO3 vs other sites.

Image processing happens and, by default, ensures that images are jpg and small enough to be visible in ebook readers. There are several settings you can tweak, see plugin-defaults.ini starting here.

You likely want to either increase image_max_size or set no_image_processing:true.

[JimmXinu]

Originally Posted by dysphere

Hmm, it looks like it's removing more divs than necessary, starting on the 22nd file, which causes the location number to change on the kindle, which makes it more difficult for me to find where I was reading last. Epubs attached so you can compare.

The only change I see is removing empty div tags (containing only whitespace).

It looks like the author/poster is using p tags in some chapters and div tags--nested in other div tags with nested div tags containing   in-between--in many (but not all) chapters between chapters 22 and 83 to indicate paragraphs.

It's the difference between this:

<p>The first thought in my mind was: <em>Is he dead?</em></p>

And this:

<div>
<div>"So let me get this straight," Ran said, lowering her brow.</div>
</div>
<div>
<div>&nbsp;</div>
</div>

p is obviously preferred. Using div tags can work, but it requires some CSS to appear the same--div doesn't have the same spacing by default.

FFF has code to automatically remove most empty(whitespace only) tags because, generally speaking, that's how HTML is meant to work. For most tags, whitespace doesn't matter and an empty tag shouldn't do anything.

By default, the exceptions are p, th, and td tags, but that list can be changed with FFF setting keep_empty_tags.

So as far as I'm concerned, the question isn't why is this happening on update?, the question is why isn't this happening on the initial download?

And the answer is that FFF is removing the whitespace only tags (ie <div>&nbsp;</div>), but that removal leaves it's parent tag, which is now empty(except for whitespace),

This:

<div>
<div>&nbsp;</div>
</div>

...becomes this:

<div>

</div>

...and, on update, is itself removed.

So, for dysphere's immediate problem (locations in book reader changing between updates), the solution that's available today is to add div to keep_empty_tags, something like this:

[https://www.royalroad.com/fiction/28806]
add_to_keep_empty_tags:,div

Change the section to royalroad.com to apply it to all stories on the site.

This is less than ideal as it just keeps the 'empty' tags, but it will stop FFF changing the chapter text across updates.

I'm looking at ways to loop the remove-empty-tags code to also remove newly emptied tags. If I figure out something I'm happy with, there will be test version posted.

[ownedbycats]

Question: I seem to recall that there was an option to set a custom URL (or filepath?) of an image for the cover. I'm having trouble finding it in defaults.ini. Am I mistaken?

[labradors]

Hey. First time posting, hope this is the right place to ask this.
I'm trying to add something in the replace_metadata: part of the personal.ini so that for example in the relationship category: Background Person 1/Person 2 becomes Person 1/Person 2 (minor) so removing/replacing the Background part of the tag with (minor) at the end.

[labradors]

Originally Posted by labradors

Hey. First time posting, hope this is the right place to ask this.
I'm trying to add something in the replace_metadata: part of the personal.ini so that for example in the relationship category: Background Person 1/Person 2 becomes Person 1/Person 2 (minor) so removing/replacing the Background part of the tag with (minor) at the end.

Sorry, I forgot to put everything I wanted to in the first post and don't see any option to edit so replying to my own post. So yeah, I'm hoping someone can tell me how to do what I detailed in the post above. Thanks a ton

[JimmXinu]

Originally Posted by ownedbycats

Question: I seem to recall that there was an option to set a custom URL (or filepath?) of an image for the cover. I'm having trouble finding it in defaults.ini. Am I mistaken?

Search in defaults.ini. The setting is default_cover_image. That's for setting a default when there isn't a specific cover. To override story's cover, also set cover_exclusion_regexp to exclude the specific cover.

Originally Posted by labradors

I'm trying to add something in the replace_metadata: part of the personal.ini so that for example in the relationship category: Background Person 1/Person 2 becomes Person 1/Person 2 (minor) so removing/replacing the Background part of the tag with (minor) at the end.

Have you seen the wiki page?

Something like this should work:

add_to_replace_metadata:
 ships=>^Background (.*)$=>\1 (minor)

When asking for help, it works better to give specific examples. Giving a story URL and the exact metadata you want changed to what.

[JimmXinu]

New Test Version Posted

2022-06-25
- Refactor code to remove empty tags to also remove now-empty parents by making another pass.
- Collect rating for adapter_libraryofmoriacom, refactor rating from TOC in base_efiction. Closes #859

[penumbria]

When trying to update a story from FFN, I am getting the message: browser cache failed to load with error ‘list index out of range’. I’ve tried it about a dozen times and get the same error message.
Is there another new workaround I am missing to get FFN to work? I don’t have stories get updates from FFN very often. Most have migrated to/cross post to AO3.
I went through and made sure every chapter of the story was in my cache by clicking on each one. And there were 77. And I have the older workarounds about browser cache and whatever in my personal ini.
I’m really not very tech savvy with this kind of stuff but I can follow directions on what to do, I just don’t really grasp the why I am doing it or what it means exactly.

[JimmXinu]

There's suggestion a couple pages back, found searching thread for 'cache'.

I've also made it into an FAQ now:
I've followed all the instructions in Browser Cache Feature and it isn't working / I'm getting errors--help?