[GUI Plugin] FanFicFare

[JimmXinu]

New Test Version Posted

2025-10-29
- Preserve original titles for Reader Post blocks from fiction.live #1269, thanks MacaroonRemarkable!
- adapter_ficbooknet: Collect additional metadata #1270, thanks dbhmw!

[ThreeLizards]

Hi, this is probably an easy answer...I'm just a little brain dead after a long day. How do I get published date to be the updated date? Please and thank you!

[slimcat]

I recently realized how much you can do to customize FFF title pages and I’ve set that up in my personal.ini. Now I’m trying to update my existing fics so the title pages match my new format and I don’t have to edit them manually. These fics have injected covers that I want to keep. I’m using “Update Epub Always” and it’s all working as expected except I can’t figure out how to keep my cover within the epub itself.

I’ve read through the Cover page instructions on the wiki and tested out a bunch of different configurations in the cover settings (and added use_old_cover:true both in the defaults and epub sections of my personal.ini), but no matter what I’ve tried, updating wipes out the cover page xhtml file and cover image completely. I still have my cover in Calibre, so I can inject it again, but I’d like to skip that extra step if possible. From the documentation, it sounds like there’s a way to preserve the cover page in the epub itself when updating, so either I’m doing something wrong or I’m misunderstanding and you can’t preserve injected covers.

[JimmXinu]

Multi-reply

Originally Posted by ThreeLizards

Hi, this is probably an easy answer...I'm just a little brain dead after a long day. How do I get published date to be the updated date? Please and thank you!

Short answer: The accepted solution is to create and populate a custom column for dateUpdated.

FFF records three dates for most stories: datePublished, dateUpdated, (both from the story metadata on site) and dateCreated (when FFF ran to create the download. These are labeled Published, Updated, and Packaged respectively.

Calibre's standard metadata has two settable date fields:
- Date (internally 'timestamp'), FFF sets it to dateCreated on any update.
- Published (internally 'pubdate'), FFF sets it to datePublished on any update.

There is not a simple way to get FFF to populate Calibre's Published field with FFF's dateUpdated.

Originally Posted by slimcat

I recently realized how much you can do to customize FFF title pages and I’ve set that up in my personal.ini. Now I’m trying to update my existing fics so the title pages match my new format and I don’t have to edit them manually. These fics have injected covers that I want to keep. I’m using “Update Epub Always” and it’s all working as expected except I can’t figure out how to keep my cover within the epub itself.

[defaults]
## If set, on EPUB update, the existing EPUB cover will be used,
## taking precedence over specific, first image, and
## default_cover_image, but not force_cover_image
use_old_cover:true

I would also suggest you setup an FFF Saved Metadata Column. It won't help with updating title pages inside books, but will with updating customized Calibre metadata.

[wildfire070]

I don't normally download from wattpad, but I've got a few authors who only post to that site. When I download from there, it gives me a "Reads" entry on the titlepage even though I do NOT have this in the wattpad section of my personal.ini

extra_titlepage_entries: reads

Is there a way to exclude this from showing in my titlepage?

This is all I have in my standard titlepage entries:

titlepage_entries:<div class="inline">,seriesHTML,rating,datePublished,dateUpdated,numChapters,numWords,genre,</div>

[JimmXinu]

Originally Posted by wildfire070

I don't normally download from wattpad, but I've got a few authors who only post to that site. When I download from there, it gives me a "Reads" entry on the titlepage even though I do NOT have this in the wattpad section of my personal.ini

extra_titlepage_entries: reads

Is there a way to exclude this from showing in my titlepage?

This is all I have in my standard titlepage entries:

titlepage_entries:<div class="inline">,seriesHTML,rating,datePublished,dateUpdated,numChapters,numWords,genre,</div>

Some devs, when adding site specific metadata, also add it to the title page by default.

In this case, you should be able to clear it in personal.ini with:

[wattpad.com]
extra_titlepage_entries:

[slimcat]

Originally Posted by JimmXinu

Multi-reply

[defaults]
## If set, on EPUB update, the existing EPUB cover will be used,
## taking precedence over specific, first image, and
## default_cover_image, but not force_cover_image
use_old_cover:true

I must be missing something here or misunderstanding how this is supposed to work, because I already have use_old_cover:true in my defaults.
I've also tried adding it to the epub section since the default.ini talks about it there, but my injected cover pages & images are still getting deleted from inside the epub on update.

I’ve set up a new library to test, simplified the epub customizations in my personal.ini in the test library, and tried every plugin cover setting configuration I can think of, but I still can’t find a way to keep the existing covers within the epub itself.

To be clear, I'm having no problem keeping my Calibre covers, just the injected ones.

## Mel's personal configuration (test library).
## Uncomment options by removing the '#' in front of them.

[defaults]

is_adult:true
collect_series: true
use_old_cover:true

## My customized titlepage_entries and the label used for them:
## <entryname>_label:<label>
characters_label:Additional Characters
description_label:Description
dateRange_label:Published

datePublished_format:%%B %%d, %%Y
dateUpdated_format:%%B %%d, %%Y
add_to_extra_valid_entries:,dateRange
include_in_dateRange:datePublished,dateUpdated
keep_in_order_dateRange:true

add_to_replace_metadata:
# same year      1          2            3        4          5
 dateRange_LIST=>([^\s]+)\s+(\d{1,2}),\s+(\d{4}), ([^\s]+)\s+(\d{1,2}),\s+\3=>\1 \2–\4 \5, \3
# same month     1          2              3            4
 dateRange_LIST=>([^\s]+)\s+(\d{1,2})[-–]\1\s+(\d{1,2}),\s+(\d{4})=>\1 \2–\3, \4
# different years
 dateRange_LIST=>([^\s]+)\s+(\d{1,2}),\s+(\d{4}), ([^\s]+)\s+(\d{1,2}),\s+(\d{4})=>\1 \2, \3 – \4 \5, \6

[epub]

## (Note that Anthology epubs will still be epub 2.0 only until
## EpubMerge learns to write epub 3.0.)
epub_version:3.0

use_old_cover:true

## If set to 'smart', logpage will only be included if the story is
## status:In-Progress or already had a logpage.
include_logpage: smart
logpage_entries: dateCreated,datePublished,dateUpdated,numWords,words_added,chapterslashtotal,status

## metadata ${title}, etc.  Make sure to keep at least one space
## at the start of each line and to escape % to %%.
titlepage_start:<?xml version='1.0' encoding='utf-8'?>
 <!DOCTYPE html>
 <html xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops" xml:lang="en" lang="en">
 <head>
    <meta charset="utf-8"/>
 <title>${title} by ${author}</title>
 <link href="stylesheet.css" type="text/css" rel="stylesheet"/>
 </head>
 <body class="summary">
 <h2>Summary</h2>
 <div class="metadata">

## ${id}, ${label}, ${value}
titlepage_entry:
 <p><b>${label}:</b> ${value}</p>

## metadata
titlepage_end:
 </div>
 <h3>Description:</h3>
 ${description}
 </body>
 </html>

[epub]

[archiveofourown.org]

use_basic_cache:true
##use_archiveofourown_gay:true
username:XXXXXXXX
password:XXXXXXXX

## Summary page metadata and order
titlepage_entries:title,author,seriesHTML,fandoms,storyUrlHTML,dateRange,status,rating,warnings,numWords,chapterslashtotal,ships,characters,freeformtags

include_subject_tags:fandoms,ships,status

add_to_extra_valid_entries:,storyUrlHTML
include_in_storyUrlHTML:storyUrl

## Add extra metadata fields:
add_to_replace_metadata:
 seriesHTML=><a[^>]*>([^<]+?)\s*\[(\d+)\]\s*</a>=>Part \2 of <a href="%%series00Url%%">\1</a>
 storyUrlHTML=>^https?://(archiveofourown\.org/works/\d+)$=><a href="https://\1">\1</a>
 characters=>\s*\([^)]*\)(?=\s*(?:,|$))=>
 chapterslashtotal=>^\s*(\d+)\s*/\s*\1\s*$=>\1
 description=>^<div class="userstuff">\s*=> 
 description=>\s*</div>\s*$=>
 
keep_in_order_fandoms:true
keep_in_order_ships:true
keep_in_order_characters:true
keep_in_order_freeformtags:true

fandoms_label:Category
chapterslashtotal_label:Chapters
freeformtags_label:Tags
storyUrlHTML_label:Source

exclude_metadata_pre:
 warnings==Creator Chose Not To Use Archive Warnings
 warnings==No Archive Warnings Apply
 status==Completed

strip_chapter_numbers:true
## Remove the site's leading number ("1.", "2 -", "10:") **only if**
## the remainder is exactly "Chapter <num>", "Prologue", or "Epilogue".
## Keep the number when there's a real title.
chapter_title_strip_pattern:(?i)^[0-9]+[\.: -]+\s*(?=chapter\s*\d+\s*$|prologue\s*$|epilogue\s*$)

## Uncomment out for Anthologies where works operate like chapters and
## don't need their own individual summary page:
## include_titlepage:false

exclude_notes:authorheadnotes,chaptersummary,chapterheadnotes,chapterfootnotes,authorfootnotes
## inspiredlinks

[overrides]
## default varies by site.

Here are some of the settings I’ve tried that retain the Calibre cover but delete the cover page file and image from within the epub:

Spoiler:

1)
- Update Calibre Cover (from EPUB): Yes, if EPUB has a cover image
- Set Calibre Cover Only for New Books: On
- Generate Calibre Cover: No

2)
- Update Calibre Cover (from EPUB): No
- Generate Calibre Cover: No

3)
- Update Calibre Cover (from EPUB): Yes, if EPUB has a cover image
- Set Calibre Cover Only for New Books: On
- Generate Calibre Cover: Yes, unless FanFicFare found a cover image
- Generate Covers Only for New Books: On
- Inject/update the generated cover inside EPUB: On

4)*
- Update Calibre Cover (from EPUB): No
- Generate Calibre Cover: Yes, unless FanFicFare found a cover image
- Generate Covers Only for New Books: Off
- Inject/update the generated cover inside EPUB: On

5)*
- Update Calibre Cover (from EPUB): Yes, if EPUB has a cover image
- Set Calibre Cover Only for New Books: On
- Generate Calibre Cover: Yes, unless FanFicFare found a cover image
- Generate Covers Only for New Books: Off
- Inject/update the generated cover inside EPUB: On

*If "Update Calibre metadata" is checked during Update Always, settings 4 & 5 generate a new cover, update the Calibre cover, and inject the cover in the epub. Even if Update Calibre Cover is set to No. (see screenshots)

I feel like I must be missing something obvious here. Like I said before, I can always inject the Calibre covers into the epubs again, but I'd prefer to skip that step if it's possible, especially because of all the ORIGINAL_EPUB backups this creates.

[JimmXinu]

FFF's cover system is an evolved, convoluted mess that I regret ever providing. Or at least, I should have said 'no' much more often when asked for additional options and features over the years.

In this particular case, however, the issue is less to do with covers per se, but using epub_version:3.0 and generated cover injection via Calibre Polish book and FFF epub update.

FFF, when it generates an epub3, puts both the epub2 and epub3 metadata files in for maximum back compatibility. Calibre's Polish Book, when updating an epub3, removes the epub2 standard <reference type="cover"> tag that FFF is looking for to find an existing cover image on update.

I will look more closely at what might be done to detect 'pure' epub3 covers this weekend. It looks straightforward, but I've thought that before.

FYI, Re: "Update Calibre Metadata" checkbox -- My first thought is that the "Update Calibre Metadata" checkbox shouldn't make any difference to the epub. That's literally one of the oldest settings in FFF.

But you have "Inject/update the generated cover inside EPUB" checked--And cover generation only happens if "Update Calibre Metadata" is checked. And cover can't be injected if it isn't generated.

[slimcat]

I understand your regrets, but truly you are doing the Lord's work.

I added the <reference type="cover"> tag back in to a few books using the Set Semantics tool in Calibre’s ebook editor and did more testing. With that one change, I can update my epub 3 fics and keep the injected covers (Hooray!). Interestingly, that also fixes the weird “Update Calibre Metadata” behavior (it no longer appears to impact the epub).

Originally Posted by JimmXinu

I will look more closely at what might be done to detect 'pure' epub3 covers this weekend. It looks straightforward, but I've thought that before.

If possible and straightforward (*knock repeatedly on wood*), that would be amazing. Right now, the issue with updating even after manually adding a cover entry to the OPF guide is that it deletes the cover-properties attribute from the manifest that's required for epub 3 (see screenshots).

Thanks again for the plugin and all your help. I sincerely hope I don't add to your regrets!

[JimmXinu]

Originally Posted by JimmXinu

I will look more closely at what might be done to detect 'pure' epub3 covers this weekend. It looks straightforward, but I've thought that before.

Yes, it is more complicated.

• Epub2 metadata marks a cover page--an (x)html file--that displays an image.
  
• Epub3 metadata marks a cover image directly, with no special marking of the page that contains it.
  
• To preserve presentation--because it does matter--FFF wants the cover page, not just the image.
  
• Calibre marks the cover page <item ... properties="calibre:title-page svg">, both with Book polish inject and Edit book's Update Internals. But only on epub3. (No idea what the svg is about--it's a jpeg file.)

So now the "straightforward" solution appears to be to start looking for that "calibre:title-page" property in epub3 as well as adding it to FFF's epub3 output.

But I don't know if the svg is needed or is junk. Or why I've seen both "calibre:title-page svg" and "svg calibre:title-page".

[Pavan7]

Is there a way to find duplicate URLs/fanfics? I merged 2 libraries together and now there are a lot of duplicates floating around.
I tried the calibre plugin but its filtering them by names (of which there are a lot). Not letting me sort by date either so I could handle the recent ones.

[JimmXinu]

Originally Posted by JimmXinu

But I don't know if the svg is needed or is junk. Or why I've seen both "calibre:title-page svg" and "svg calibre:title-page".

FYI, the svg apparently refers to whether the cover page wraps the real cover image with a <svg> image to improve scaling. Apparently this workaround has been around for a long time. Link 1 Link 2

It looks like adding the <svg> wrapper to FFF via cover_content setting is trivial. (Does anyone already do that? Looks like this person does.)

Ideally, FFF should be looking for <svg> tags in all (x)html files and marking them with properties='svg' when writing epub3. Reference

Neither Calibre Edit book or Polish add that property when a <svg> is in a random file, but Covert does.

[JimmXinu]

New Test Version Posted

2025-11-01
- adapter_storiesonlinenet.py - tolerate contest stories with "(Hidden)" author, #1272, thanks bpothier
- Find and use epub3 cover on update--relies on Calibre's calibre:title-page property.
- writer_epub: Also write the calibre:title-page property for epub3.
- writer_epub: Rearrange to detect and flag files containing svg tags for epub3.

This should address slimcat's cover issues.

In theory, rearranging the order writer_epub writes files shouldn't matter. (Yes, I have still kept the 'mimetype' file first and uncompressed as required.)

In practice, it would not hugely surprise me if some readers care.

[KJS]

Hey, my normal method of keeping track of AO3 fics I need to add to calibre is to add them to my "mark for later" list, and then use the "get story URLs from webpage" option to collect them quickly. I just ran the latest update, and although I can add individual fics just fine, or use the buttons to get several URLs from a series or author page, the mark for later page gives me this error:


calibre, version 8.13.0
ERROR: Unhandled exception: <b>KeyError</b>:'href'

calibre 8.13 Portable embedded-python: True
Windows-10-10.0.26200-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.26200')
Python 3.11.14
Windows: ('10', '10.0.26200', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Users\username\OneDrive\Calibre Portable\Calibre\calibre.exe
Successfully initialized third party plugins: Diaps Editing Toolbag (0, 5, 0) && EpubMerge (3, 2, 0) && EpubSplit (3, 12, 0) && FanFicFare (4, 50, 0) && Find Duplicates (1, 10, 9) && Library Splitter (1, 0, 7) && Reading List (1, 15, 6)
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 712, in get_urls_from_page_menu
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 735, in get_urls_from_page
File "C:\Users\username\OneDrive\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\geturls .py", line 44, in get_urls_from_page
File "C:\Users\username\OneDrive\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\adapter s\base_adapter.py", line 504, in get_urls_from_page
File "C:\Users\username\OneDrive\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\adapter s\base_otw_adapter.py", line 688, in get_series_from_page
File "C:\Users\username\OneDrive\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\adapter s\base_otw_adapter.py", line 688, in <listcomp>
File "bs4\element.py", line 1573, in __getitem__
KeyError: 'href'






Blind googling has gotten me nowhere, any suggestions?

[slimcat]

Your update did the trick. Thank you so much!!!

Originally Posted by JimmXinu

Ideally, FFF should be looking for <svg> tags in all (x)html files and marking them with properties='svg' when writing epub3. Reference

Neither Calibre Edit book or Polish add that property when a <svg> is in a random file, but Covert does.

FYI: You can opt out of SVG covers in Calibre's Output settings (which I did, since it's my understanding that using svg wrappers is more of a legacy technique), and if you do you don't get the svg tag in epub3, just "calibre:title-page."