[GUI Plugin] FanFicFare

[JimmXinu]

FYI, AO3 is now showing a banner:

We're aware that many users are experiencing frequent forced logouts and errors. Our apologies for the inconvenience! Our volunteers are working on the problem - it isn't necessary to report it to our Support team. For some suggestions that might help reduce how often you are logged out or receive errors, please visit our Known Issues page.

Where is says:

If you are receiving a forced logout error, "session expired" or "auth error", please log back into your account from the Login page rather than using the login form that pops up in the top corner. Once you have logged back in, try opening the page you were viewing again, rather than refreshing the same tab where you received the error. We are aware that this issue has been happening frequently since Sept. 19, 2024, and are continuing to work on a solution.

This is what's causing the AO3 login issues with FFF.

Removing always_login:true or setting it false is about all we can do in FFF.

[kamille3]

Facing this error - invalid literal for int() with base 10: '' when trying to download certain literotica series.

/authors/SirBigBadWolf/works/stories/all
/series/se/493934889
/series/se/494046255

[JimmXinu]

Originally Posted by kamille3

Facing this error - invalid literal for int() with base 10: '' when trying to download certain literotica series.

/authors/SirBigBadWolf/works/stories/all
/series/se/493934889
/series/se/494046255

It looks like that author is broken, or their works aren't linked correctly or something. For all 'series' under that author, either there aren't any chapters listed on the series page, or it flat out says "Error 404 This work is not available". So broken site, basically.

FFF, perhaps optimistically, assumes that any story there's a URL for will list at least one chapter. And I'm not interested in parsing the author page for the list. There are issues.

[kamille3]

Originally Posted by JimmXinu

It looks like that author is broken, or their works aren't linked correctly or something. For all 'series' under that author, either there aren't any chapters listed on the series page, or it flat out says "Error 404 This work is not available". So broken site, basically.

FFF, perhaps optimistically, assumes that any story there's a URL for will list at least one chapter. And I'm not interested in parsing the author page for the list. There are issues.

Okay thanks. I'll check if site/author page gets fixed.

[Clem2605]

So, I've got a weird error when trying to use FFF on AO3 today:

Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.jobs", line 318, in do_download_for_worker
  File "C:\Users\clem\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\writers\base_writer.py", line 234, in writeStory
  File "C:\Users\clem\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\writers\writer_epub.py", line 613, in writeStoryImpl
  File "C:\Users\clem\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\story.py", line 1483, in getChapters
  File "string.py", line 121, in substitute
  File "string.py", line 118, in convert
  File "string.py", line 101, in _invalid
ValueError: Invalid placeholder in string: line 1, col 18

I'm trying new things in my personal.ini right now, but the only thing I've changed (on purpose) since the last time it worked is the title page display code, and I've tried to download without this part, without any change.

It's the same error with all the URL I've tried.

Oh and now that I think about it, I've updated Calibre this morning.

[JimmXinu]

Originally Posted by Clem2605

So, I've got a weird error when trying to use FFF on AO3 today:

I'm not seeing that happen.

That error is coming when it's processing the chapter title page for TOC. What's your personal.ini look like?

[Clem2605]

Originally Posted by JimmXinu

I'm not seeing that happen.

That error is coming when it's processing the chapter title page for TOC. What's your personal.ini look like?

[defaults]
is_adult:true
collect_series: true

authorUrl_label:Author link
category_label:Fandom
dateCreated_label:Created
site_label:Site
storyUrl_label:Story link


#dateRange_label:Dates
#include_in_dateRange:datePublished,dateUpdated

datePublished_format:%%Y-%%m-%%d %%H:%%M:%%S
dateUpdated_format:%%Y-%%m-%%d %%H:%%M:%%S


add_to_output_css:
 body {
  text-align: left;
 }
 body.fff_titlepage {
  font-size: 2.5vmax;
 }
 hr {
  display: block;
  border: 0;
  border-top: 1px solid;
  padding: 0;
 }
## https://stackoverflow.com/a/50191135
 .flex-parent {
  display: flex;
  align-items: center;
 }
 .flex-child-edge {
  flex-grow: 2;
  border-bottom: 1px solid;
 }
 .flex-child-text {
  flex-basis: auto;
  flex-grow: 0;
  text-align: center;
  margin: 0 5px 0 5px;
 }
 .flex-child-text.stats span { font-weight: bold; }
 .flex-child-text.category { font-weight: bold; }
 .fff_titlepage .title h1 {
  font-size: 3vmax;
  margin: 0;
 }
 .fff_titlepage .title h1 { font-weight: normal; }
 .fff_titlepage .title h1 a { font-weight: bold; } 
 .fff_titlepage .title a,
 .fff_titlepage .tags a,
 .fff_titlepage .tag-summary-content a {
   color: #1C70C8;
 }
 .fff_titlepage .tag-summary-content a {
  font-style: italic;
 }
 .fff_titlepage .tag-summary-content p {
  margin-top: 0;
 }
 .fff_titlepage .tags {
  display: block;
  padding: 0;
 }
 dl .inline dt:after { content:'\00a0'; }
 dl .inline dt { float: left; }
 .fff_titlepage dd { margin-left: 0.75em; }
 .fff_titlepage dl, .fff_titlepage dt { margin: 0; }


titlepage_start:<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
 <html xmlns="http://www.w3.org/1999/xhtml">
 <head>
 <title>${title} by ${author}</title>
 <link href="stylesheet.css" type="text/css" charset="UTF-8" rel="stylesheet"/>
 <meta name="calibre-html-render-data"/>
 </head>
 <body class="fff_titlepage">
 <div class="title">
 <h1><a href="${storyUrl}">${title}</a> by ${authorHTML}</h1>
 </div>
  <div class="flex-child-edge"></div>
  <div class="flex-child-text category">${category} ◈ ${status}</div>
  <div class="flex-child-edge"></div>
 <div class="tag-summary-content">${description}</div>
  <div class="flex-child-edge"></div>
  <div class="flex-child-text stats"> ${numWords} ◈ ${chapterslashtotal}</div>
  <div class="flex-child-edge"></div>
 <dl class="tags">

titlepage_entries:<div class="inline">,seriesHTML,<br />,rating,characters,ships,<br />,datePublished,dateUpdated,dateCreated,</div>


## ${id}, ${label}, ${value}
titlepage_entry:
 <dt><b>${label}:</b></dt> <dd>${value}</dd>

## ${id}, ${label}, ${value}
## Used with entries listed in wide_titlepage_entries, typically description, storyUrl, authorUrl
titlepage_wide_entry:

## When using tables, make these span both columns.
wide_titlepage_entries:

## ${id}, ${value}
## For example, if you've set description_label:
## (that is, nothing)
titlepage_no_title_entry:${value}<br />

## metadata
titlepage_end:
 </dl>
  <div class="flex-child-edge"></div>
  <div class="flex-child-text"> ${language} ◈ ${siteabbrev}</div>
  <div class="flex-child-edge"></div>
 </body>
 </html>


include_subject_tags: extratags, genre

keep_summary_html:true

sort_ships:true

extratags:

replace_metadata:
 status=>Completed=>Complete
 freeformtags,characters,ships=>"|’|”|“=>'
 freeformtags,characters,ships=>(?i)Other .* to be Added|.*to be added.*=>
 freeformtags,characters,ships=>(?i).*background.*=>


## Categories
 category=> \(?(- All Media Types|- Fandom|Movies?|Movie [0-9]+|Comics|TV [0-9]+|TV|Video Games?|\d\d\d\d)\)?$=>
 category=>Trilogy=>
 category=> - J. K. Rowling| - C. S. Pacat| - Casey McQuiston=>
 category=>.*Shadowhunter.*|.*Mortal Instrument.*=>Shadowhunters
 category=>.*Star Wars.*=>Star Wars
 category=>No Fandom=>Original Work
 category,ships=> - Fandom=>
 category=> - Neil Gaiman &amp; Terry Pratchett=>
 category=>^A Song of Ice and Fire.*=>Game of Thrones
 category=>^Avatar.*=>Avatar
 category=> Cinematic Universe=>
 category=>The Avengers.*=>Marvel
 category=>.*The Witcher=>Wiedźmin | The Witcher

## *************************************************************************************************************************************************************************************
## *************************************************************************************************************************************************************************************
## *************************************************************************************************************************************************************************************
## *************************************************************************************************************************************************************************************
## Put tags here




## *************************************************************************************************************************************************************************************
## *************************************************************************************************************************************************************************************
## *************************************************************************************************************************************************************************************
## *************************************************************************************************************************************************************************************
## Put Characters here





browser_cache_path:C:\Users\cleme\AppData\Local\Google\Chrome\User Data\Default\Cache\Cache_Data
browser_cache_age_limit:-1

strip_chapter_numbers:true
add_chapter_numbers:true
chapter_title_strip_pattern:^([0-9]+[\.: -]+)?(Chapter *[0-9]+[\.:, -]*)?
chapter_title_add_pattern:Chapter ${number}${title:|: |}
chapter_title_addnew_pattern:Chapter ${number}${title:|: |} | New <3
mark_new_chapters:true

anthology_title_pattern:${title}

[epub]

replace_hr: true

include_logpage: false

cover_min_size: 170,230

image_max_size: 2000, 2000

force_img_self_referer_regexp:gelbooru.com|photobucket.com



[fanfiction.net]
slow_down_sleep_time:6

use_browser_cache:true
use_browser_cache_only:true
open_pages_in_browser:true

add_to_include_subject_tags:,category,rating
skip_author_cover:false
## Extra metadata that this adapter knows about.  See [archiveofourown.org]
## for examples of how to use them.
extra_valid_entries:reviews,favs,follows

titlepage_entries: seriesHTML,description,<br />,category,genre,characters,ships,rating,<br />,favs,reviews,numChapters,numWords,language,<br />,status,datePublished,dateUpdated,dateCreated,siteabbrev,storyUrl

add_to_custom_columns_settings:
 favs=>#popularity
 reviews=>#reviews

## ffnet uses 'Pairings', not 'Relationship', stating they don't have
## to be romantic pairings.
ships_label:Relationship



[archiveofourown.org]
use_basic_cache:true
username:XXXXXXXX
password:XXXXXXXX
add_to_include_subject_tags:,warnings
use_view_full_work:true
use_workskin:true
exclude_notes:inspiredlinks
remove_authorfootnotes_on_update:true
use_archived_author:true

datechapter_format:%%d/%%m/%%y
chapter_title_add_pattern:Chapter ${number}${title:|: |} ◈ ${date}
chapter_title_addnew_pattern:Chapter ${number}${title:|: |} ◈ ${date} | New <3

include_in_description:description,freeformtags.NOREPL

keep_in_order_description:true
keep_in_order_freeformtags:true

join_string_description:\s|\s


ao3categories_label:Sexual Orientation
chapterslashtotal_label:Chapters
titlepage_entries:series00HTML,series01HTML,series02HTML,series03HTML,description,<br />,category,characters,ships,rating,<br />,bookmarked,bookmarks,comments,kudos,chapterslashtotal,numWords,language,<br />,status,datePublished,dateUpdated,dateCreated,siteabbrev,storyUrl

add_to_replace_metadata:
 authorUrl=>(/users/orphan_account)/pseuds/.*$=>\1

add_to_custom_columns_settings:
 bookmarks=>#popularity,r_anthmax
 comments=>#reviews
#Put tags in the test column
# genre,warnings=>#test_tag_column,r

[https://archiveofourown.org/works/33085891/chapters/148727137:epub]
no_image_processing: true

[www.starslibrary.net]
username:XXXXXXXX
password:XXXXXXXX
collect_series: false

[www.twilighted.net]
username:XXXXXXXX
password:XXXXXXXX
## default is false
collect_series: true

Her it is.
I've removed a big part of my 'replace_metadata' because it's very long and doesn't seems to be the problem here.

[culytera]

Hey that looks familiar

If it's about the chapter title page for TOC, does your chapter_start look like this?

chapter_start:<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
 <html xmlns="http://www.w3.org/1999/xhtml">
 <head>
 <title>${chapter}</title>
 <link href="stylesheet.css" type="text/css" charset="UTF-8" rel="stylesheet"/>
 <meta name="chapterurl" content="${url}"></meta>
 <meta name="chapterorigtitle" content="${origchapter}"></meta>
 <meta name="chaptertoctitle" content="${tocchapter}"></meta>
 <meta name="chaptertitle" content="${chapter}"></meta>
 </head>
 <body class="fff_chapter">
 <h3 class="fff_chapter_title">${origchapter}<hr><small>${date}</small></h3>

[Clem2605]

Originally Posted by culytera

Hey that looks familiar

If it's about the chapter title page for TOC, does your chapter_start look like this?

chapter_start:<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
 <html xmlns="http://www.w3.org/1999/xhtml">
 <head>
 <title>${chapter}</title>
 <link href="stylesheet.css" type="text/css" charset="UTF-8" rel="stylesheet"/>
 <meta name="chapterurl" content="${url}"></meta>
 <meta name="chapterorigtitle" content="${origchapter}"></meta>
 <meta name="chaptertoctitle" content="${tocchapter}"></meta>
 <meta name="chaptertitle" content="${chapter}"></meta>
 </head>
 <body class="fff_chapter">
 <h3 class="fff_chapter_title">${origchapter}<hr><small>${date}</small></h3>

I didn't reset the chapter start, the only only toc-related changes I make, that I can think about, are my chapter_title_strip/add/addnew code:

strip_chapter_numbers:true
add_chapter_numbers:true
chapter_title_strip_pattern:^([0-9]+[\.: -]+)?(Chapter *[0-9]+[\.:, -]*)?
chapter_title_add_pattern:Chapter ${number}${title:|: |}
chapter_title_addnew_pattern:Chapter ${number}${title:|: |} | New <3
mark_new_chapters:true

And I modify it in the AO3 section with:

chapter_title_add_pattern:Chapter ${number}${title:|: |} ◈ ${date}
chapter_title_addnew_pattern:Chapter ${number}${title:|: |} ◈ ${date} | New <3

Otherwise, there is my 'titlepage_start' code, but I didn't modify the <head> part of the HTML, so nothing that should impact the toc.

[JimmXinu]

Originally Posted by Clem2605

I didn't reset the chapter start, the only only toc-related changes I make, that I can think about, are my chapter_title_strip/add/addnew code:

strip_chapter_numbers:true
add_chapter_numbers:true
chapter_title_strip_pattern:^([0-9]+[\.: -]+)?(Chapter *[0-9]+[\.:, -]*)?
chapter_title_add_pattern:Chapter ${number}${title:|: |}
chapter_title_addnew_pattern:Chapter ${number}${title:|: |} | New <3
mark_new_chapters:true

And I modify it in the AO3 section with:

chapter_title_add_pattern:Chapter ${number}${title:|: |} ◈ ${date}
chapter_title_addnew_pattern:Chapter ${number}${title:|: |} ◈ ${date} | New <3

Otherwise, there is my 'titlepage_start' code, but I didn't modify the <head> part of the HTML, so nothing that should impact the toc.

The problem is the ${title:|: |}.

The chapter title pattern uses Python's Template object from it's string module.

The :|: | part of that is, I assume, copied from Calibre's templating language which has additional features.

This works:

chapter_title_add_pattern:Chapter ${number} ${title} ◈ ${date}
chapter_title_addnew_pattern:Chapter ${number} ${title} ◈ ${date} | New <3

[Clem2605]

Originally Posted by JimmXinu

The problem is the ${title:|: |}.

The chapter title pattern uses Python's Template object from it's string module.

The :|: | part of that is, I assume, copied from Calibre's templating language which has additional features.

This works:

chapter_title_add_pattern:Chapter ${number} ${title} ◈ ${date}
chapter_title_addnew_pattern:Chapter ${number} ${title} ◈ ${date} | New <3

Thank you!!
But then, there is no way to display the ': ' only if there is a chapter title available?

[Jager956]

Hello,

I am trying to download from SOL with calibre 7.19 and FFF plugin 4.38, but I always get now:

module 'fanficfare.adapters.base_adapter' has no attribute 'get_url_search'

With previous versions it was working fine, so configuration should be OK, just with this latest update of Calibre and FFF?

Thanks.
J.

[JimmXinu]

Originally Posted by Jager956

I am trying to download from SOL with calibre 7.19 and FFF plugin 4.38, but I always get now:

module 'fanficfare.adapters.base_adapter' has no attribute 'get_url_search'

With previous versions it was working fine, so configuration should be OK, just with this latest update of Calibre and FFF?

Probably this same problem with OS packaged Calibre conflicting with pypi installed FanFicFare.

Originally Posted by JimmXinu

I assume you are using Calibre installed from a Fedora repo of some sort. Which has been modified to use python and dependencies installed on the system. The official Calibre download has it's own versions of python and dependencies bundled in precisely because of this sort of issue.

I am reasonably confident installing the official release of Calibre instead would correct this.

Another option would be to not install FFF CLI and install call the FFF plugin as CLI:

calibre-debug --run-plugin FanFicFare -- --help

It's mostly identical to FFF CLI from pypi except for using your personal.ini from either the library specified by new arg --library-path or your last used library (if I recall correctly, it's late). It does not automatically add or update to the library like the GUI plugin does.

[Jager956]

Originally Posted by JimmXinu

Probably this same problem with OS packaged Calibre conflicting with pypi installed FanFicFare.

Forgot to specify. Using Windows, Calibre downloaded from official location. No CLI FFF, no pypi installed FFF.

[JimmXinu]

In that case, all I can think of is that the plugin zip file download was incomplete or something.

Try downloading(the plugin)/installing/restarting again. If you still have the problem, restart Calibre in debug and post a debug log of it happening.