[GUI Plugin] FanFicFare

[JimmXinu]

New Test Version Posted

2025-03-05
- BrowserCache Chrome Block: Treat entry missing headers same as not found. #1167 #1169

[ThreeLizards]

Okay, so for the cover thing, it's every single story I try to download.

Here is a URL for AO3 (I choose a G rated fic): https://archiveofourown.org/works/60...ters/155465734

Also I included a copy of the safe version of my personal.ini

Spoiler:

Safe Personal.Ini



## This is an example of what your personal configuration might look
## like.  Uncomment options by removing the '#' in front of them.

[defaults]
## [defaults] section applies to all formats and sites but may be
## overridden at several levels.  See
## https://github.com/JimmXinu/FanFicFare/wiki/INI-File for more
## details.

## Some sites also require the user to confirm they are adult for
## adult content.  Uncomment by removing '#' in front of is_adult.
is_adult:true

## Don't like the numbers at the start of chapter titles on some
## sites?  You can use strip_chapter_numbers to strip them off.  Just
## want to make them all look the same?  Strip them off, then add them
## back on with add_chapter_numbers.  Don't like the way it strips
## numbers or adds them back?  See chapter_title_strip_pattern and
## chapter_title_add_pattern in defaults.ini.
#strip_chapter_numbers:true
#add_chapter_numbers:true

always_overwrite: true

datePublished_format:%%B %%d, %%Y
dateUpdated_format:%%B %%d, %%Y
chapterslashtotal_label:Chapters

keep_in_order_freeformtags:true
keep_in_order_ships:true
keep_in_order_category:true
keep_in_order_characters:true
keep_in_order_fandoms:true
keep_in_order_genre:true
keep_in_order_ships:true

add_to_extra_valid_entries:,prefix_ships,ships_modified,category_modified,tags_modified,chapterslashtotal

include_in_prefix_ships:ships
include_in_ships_modified:ships
include_in_category_modified:category
include_in_tags_modified:freeformtags,characters,ships,warnings

replace_metadata: 

## Fandoms
 category_modified=>^Arcane: League of Legends \(Cartoon 2021\)$=>Arcane

# separate lines for ease of maintenance 
 category_modified=>^(Dragon Age).*$=>\1
 category_modified=>^(Mass Effect).*$=>\1
 category_modified=>^(Star Wars).*$=>\1
 category_modified=>^(Harry Potter).*$=>\1
 category_modified=>^(Buzzfeed Unsolved).*$=>\1

## Characters
 characters=>\(mentions of\) =>
 characters=> \([Ch]haracter\)=>
 characters=> ?\((cameo|briefly|if you squint|sort of)\)=>
 characters=> - Character=>
 characters=>^Other mentions$=>

 ships=> - Relationship=>

## Status
 status=>^In-Progress$=>WIP

## Removes images from summaries
 description=><img[^>]+>=>

## Removes fluff from categories
 category_modified=> \(?(- All Media Types|- Fandom|Movies|Movie [0-9]+|Comics|TV|Video Games?)\)?$=>

## Publisher
 site=>^archiveofourown.org$=>Archive of Our Own

## Ratings
 rating=>^General Audiences$=>G
 rating=>^Teen And Up Audiences$=>T
 rating=>^Mature$=>M
 rating=>^Explicit$=>E
 rating=>^Not Rated$=>U


## Standardize names for consistency
 ships,prefix_ships=>(OFC\/OFC)|(Original Female Character\/Original Female Character)=>Original Female Character(s)/Original Female Character(s)

exclude_metadata_pre:

## Removes "CharacterName" if Male or Female CharacterName is already present.
 characters=~^CharacterName&&characters=~^(Male|Female) CharacterName
## Exclude all entries that don't have / or &
 ships,prefix_ships!~(.*)(\/| &amp; )(.*)
## Exclude friendships in the ship tag
## remove this if you still want to see them
 ships,prefix_ships=~(.*) &amp; (.*)
## Exclude ship tags that are "implied/mentioned"
 prefix_ships=~^(.*)\(?(implied|mentioned)\)?
## Tags to be added removed
 additional_characters=~(Original|Various) Character\(?s?\)?
 freeformtags,genre==Other Additional Tags to Be Added


add_to_custom_columns_settings:
 ships_modified=>#pairings
 category_modified=>#fandom
 tags_modified=>#mytags
 kudos=>#ao3kudos_count
 hits=>#ao3hit_count
 bookmarks=>#ao3bookmark_count
 warnings=>#tags

# Add HTML to format the title page

titlepage_entries: series,rating,warnings,ao3categories,fandoms,ships,characters,freeformtags,language,collections,numWords,chapterslashtotal,description

titlepage_start:
 <?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
 <html xmlns="http://www.w3.org/1999/xhtml">
 <head>
 <title>${title} by ${author}</title>
 <link href="stylesheet.css" type="text/css" charset="UTF-8" rel="stylesheet"/>
 <meta name="calibre-html-render-data"/>
 </head>
 <body>
 <div class="title">
 <a href="${storyUrl}">${title}</a> by ${authorHTML}
 </div>
 <dl class="titlepagelist">

titlepage_entry:
 <dt><b>${label}:</b></dt> <dd>${value}</dd>

## When using tables, make these span both columns.
wide_titlepage_entries: 

## ${id}, ${label}, ${value}
## Used with entries listed in wide_titlepage_entries, typically description, storyUrl, authorUrl
titlepage_wide_entry:
 <span><b>${label}:</b> ${value}</span>&nbsp;&nbsp;

## metadata
titlepage_end:
## <dl class="stats">
## </dl>
 </dl>
 <hr style="border: .75px solid #999; width: 90%%; margin: 10px auto;" />
 <br />
 <div style="text-align: left; font-weight: bold; margin-bottom: 5px;">Stats:</div>
 <div style="display: flex; justify-content: space-between; margin-bottom: 8px; width: 100%%;">
     <span style="flex: 1; text-align: left;">Published: ${datePublished}</span>
     <span style="flex: 1; text-align: left;">Updated: ${dateUpdated}</span>
     <span style="flex: 1; text-align: left;">Status: ${status}</span>
 </div>
 <div style="display: flex; justify-content: space-between; margin-bottom: 8px; width: 100%%;">
     <span style="flex: 1; text-align: left;">Kudos: ${kudos}</span>
     <span style="flex: 1; text-align: left;">Hits: ${hits}</span>
     <span style="flex: 1; text-align: left;">Bookmarks: ${bookmarks}</span>
   </div>
 </body>
 </html>


[epub]
## Include images from img tags in the body and summary of stories.
## Images will be converted to jpg for size if possible.  Images work
## in epub format only.  To get mobi or other format with images,
## download as epub and use Calibre to convert.
## true by default, uncomment and set false to not include images.
#include_images:true

## If set false, the summary will have all html stripped for safety.
## Both this and include_images must be true to get images in the
## summary.
## true by default, uncomment and set false to not keep summary html.
keep_summary_html:true

## If set true, and there isn't a specific cover image, the first
## image found in the story will be made the cover image.  If
## keep_summary_html is true, images in the summary will be before any
## in chapters.
## true by default, uncomment and set false to turn off
make_firstimage_cover:false


## Most common, I expect will be using this to save username/passwords
## for different sites.  Here are a few examples.  See defaults.ini
## for the full list.



output_css:
 body { background-color: #%(background_color)s;
        text-align: left;
        margin: 2%%;
        adobe-text-layout: optimizeSpeed; }
 pre { font-size: x-small; }
 sml { font-size: small; }
 h1 { text-align: center; }
 h2 { text-align: center; }
 h3 { text-align: center; }
 h4 { text-align: center; }
 h5 { text-align: center; }
 h6 { text-align: center; }
 .CI {
     text-align:center;
     margin-top:0px;
     margin-bottom:0px;
     padding:0px;
     }
 .center   {text-align: center;}
 .cover    {text-align: center;}
 .full     {width: 100%%; }
 .quarter  {width: 25%%; }
 .smcap    {font-variant: small-caps;}
 .u        {text-decoration: underline;}
 .bold     {font-weight: bold;}
 p { text-indent: 2em; }
 blockquote.userstuff {
    margin-bottom: 0.5em; /* Reduce spacing */
    padding-bottom: 0.5em;
    border-bottom: 1px solid #888;
 }

##/* But leave the summary's .userstuff alone (or style differently) */
 dd .userstuff {
    border-bottom: none;
    margin-bottom: 0;
    padding-bottom: 0;
 }


replace_xbr_with_hr:3
replace_hr:false
replace_br_with_p:false


[archiveofourown.org:epub]

add_to_output_css:
 dl.titlepagelist dd { margin: 0 0 1em 1em; }
 div.title {
  font-weight: bold;
  text-align: center;
 }

[archiveofourown.org]
## Some sites require login (or login for some rated stories) The
## program can prompt you, or you can save it in config.  In
## commandline version, this should go in your personal.ini, not
## defaults.ini.

use_workskin:false


freeformtags_label:Additional Tags
freefromtags_label:Additional Tags
ao3categories_label:Category
chapterslashtotal_label:Chapters
description_label: Summary

keep_in_order_ao3categories:true
keep_in_order_freeformtags:true


## This section will override anything in the system defaults or other
## sections here.

[overrides]
## default varies by site.  Set true here to force all sites to
## collect series.
#collect_series: true

So yeah, any help would be appreciated. It happens to every story. I don't know why.

[JimmXinu]

Originally Posted by ThreeLizards

Okay, so for the cover thing, it's every single story I try to download.
....

Yeah, that's not Generate Cover having a problem.

That's Calibre's "Set cover from ebook format" code called by having FFF's "Update Calibre Cover (from EPUB)" setting turned on, in combination with your custom titlepage_start setting.

You can see this happen manually if you Edit metadata in Calibre, choose the epub format and click the cover from format button:


Technically, the very start of an xhtml file has to be "<?xml ...?>". You have an extra new line.

# don't do this:
titlepage_start:
 <?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
 ...

# do this:
titlepage_start:<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
 ...

The next question is: Why didn't the GC cover run and immediately hide that problem?

The answer to that is that the GC GUI selection uses FFF's site metadata entry--and you've changed it in replace_metadata.

There are certain metadata entries FFF uses internally that, technically can be changed by the user, but will break certain features. Especially, storyUrl, but also site, & status.

So: You can either not change site, use Default GUI GC instead of archiveofourown.org, or use generate_cover_settings to specify the GC setting to use.

FYI, publisher is a copy of site that gets put in Calibre's Publisher field. That, you can safely change.

From defaults.ini:

## The metadata entry 'site' predates Calibre integration, and was
## later labeled Publisher and used to fill Calibre's Publisher field.
## Because users keep expecting it to be 'publisher' instead,
## 'publisher' is now a copy of 'site' and the Calibre plugin now uses
## 'publisher' instead.  Default titlepage_entries are also changed.
include_in_publisher:site
publisher_label:Publisher

[JimmXinu]

New Test Version Posted

2025-03-06
- adapter_fimfictionnet: Fetch stories only found in bookshelf. #1174 thanks, dbhmw
- adapter_syosetucom: site update #1173 thanks, praschke

[ownedbycats]

AO3: Noticed it pretty consistent fail (525) when the work is adult. Is this noticed by anyone else? I look last few pages, but different errors than I see.

[culytera]

Originally Posted by ownedbycats

AO3: Noticed it pretty consistent fail (525) when the work is adult. Is this noticed by anyone else? I look last few pages, but different errors than I see.

Just tried, getting that too.

HTTP Error in FFF '525 Server Error:  for url: https://archiveofourown.org/token_dispenser.json'(525)

[steltek]

Originally Posted by ownedbycats

AO3: Noticed it pretty consistent fail (525) when the work is adult. Is this noticed by anyone else? I look last few pages, but different errors than I see.

Yeah, I've noticed it too.

I've been playing around with FFF settings in personal.ini, but haven't found anything that definitively helps with it yet though.

Sometimes, commenting out the "is_adult:true" setting for A03 in personal.ini lets it work. But, it isn't consistent.

[DNSB]

That seems a bit odd since the only time I've seen a 525 error was an SSL handshake failure. Cloudflare seems very fond of producing them.

[ownedbycats]

Originally Posted by DNSB

That seems a bit odd since the only time I've seen a 525 error was an SSL handshake failure. Cloudflare seems very fond of producing them.

AO3 is having issues until April: https://www.mobileread.com/forums/sh...ostcount=10390

[JimmXinu]

New Test Version Posted

2025-03-09
- adapter_storiesonlinenet: Change parsing for when the description/details contains extraneous /div tag #1176 thanks, bpothier

[Lys]

Originally Posted by ownedbycats

AO3: Noticed it pretty consistent fail (525) when the work is adult. Is this noticed by anyone else? I look last few pages, but different errors than I see.

I found the 525 error being totally random. Sometimes I get it a lot, sometimes I don't. Sometimes I get it, and if I re-run the update half a hour later, everything is okay.

The only constant is that downloading from AO3 is much slower than it was a couple of months ago, but 525 error seems to be a coin toss.

[ownedbycats]

Originally Posted by Lys

I found the 525 error being totally random. Sometimes I get it a lot, sometimes I don't. Sometimes I get it, and if I re-run the update half a hour later, everything is okay.

The only constant is that downloading from AO3 is much slower than it was a couple of months ago, but 525 error seems to be a coin toss.

Yeah, when I get it I usually try again a few minutes later and it'll work.

[mijkala]

Originally Posted by ownedbycats

Yeah, when I get it I usually try again a few minutes later and it'll work.

I don't know why, but I almost rarely DON'T get the error. I haven't been able to add any new fics or update any in a few weeks. I'm sometimes able to do maaaybe one a day but that's about it (and with me changing slow down time, if I'm logged in, etc)

[watani]

How does one get FFF to read in a local image file when the website requires you to use the browser cache to get its current chapter list?

The fic in particular I'm trying to get this work for is this one: https://archiveofourown.org/works/32014516

On chapter 6 of this story, there were originally two images that have since been removed from the linked source, however I happen to have the two images stored locally. I have been unable to get FFF to insert these images into the story using the normal means:
- When setting FFF to use the browser cache, but not always (i.e., use_browser_cache:true and use_browser_cache_only:false), I get the 525 error due to Ao3's current issues. This occurs even if the navigate page and first chapter are already cached in the browser cache.
- When setting FFF to always use the browser cache, FFF just sets the images to "failedtoload"
- When setting FFF to also open the pages in the browser, FFF opens the images in the default application for images and still sets the images to "failedtoload". Even opening the images in the browser by dragging them into the browser doesn't cache them.

Past attempts to just manually add the images into the story result in the images getting overwritten the next time they're updated, but as I've explained above I can't do the usual means to get FFF to load the images into the story (iirc, Option 1).

[JimmXinu]

Originally Posted by watani

How does one get FFF to read in a local image file when the website requires you to use the browser cache to get its current chapter list?
...

This is getting well into 'off label' use, as it were. But I'll try to explain.

First, I'm approaching this purely as "I want to insert an image I have into a chapter of a book I will update, and have it remain after update." Since the image as linked in the story isn't available anymore in this case, trying to force it in via browser cache isn't going to work for reasons.

longdesc

FFF uses the 'longdesc' attribute to record the original URL for included images so it can reuse images that appear more than once. Images that failed to download are given 'failedtoload'. 'longdesc' is used because most tools & book readers leave it alone.

On update, FFF looks in chapter text files for images tags and uses the 'longdesc' attribute to index them. So if 'longdesc' isn't present, they don't get indexed and are instead dropped.

Edit book

So, if you open the epub in Calibre's Edit book, you should be able to add your images to the file and manually add / update the <img> tags with appropriate src attributes AND add an longdesc attribute to each with a different value.

For example, I added couple of random pictures to an epub in Edit book (File > Import files into book), then added <img> tags (type/edit <img src=" and Edit book will offer the images in the epub):

<img src="images/IMG_0227.JPG" longdesc="IMG_0227.JPG"><br>
<img src="images/IMG_0228.JPG" longdesc="IMG_0228.JPG"><br>

Use the original URL from the story if you want to hunt it down--it really just needs to be unique within the file.

Now, the next time the epub is updated by FFF, those images will be kept--BUT, they will have FFF's image processing applied (such as being reduced in size, converted to jpg depending on settings), and renamed such as ffdl-9.jpg.

No promises

As I said, this is an 'off label' use. I've tried it briefly in a simple case and if it works for you, great. But I'm not going to be surprised if there are cases it doesn't work.

Honestly, expecting FFF to fix images that are broken in the story is really asking a bit much.