[GUI Plugin] FanFicFare

[chrisric]

Originally Posted by JimmXinu

Honestly, FFF plugin CLI was added more because it possible than because it was useful.

Understood. Thanks for the quick reply. I appreciate the CLI as it works great when creating and testing new adapters.

[WinterPhoenix]

Hello! Not sure if this is the right place to ask about these things, but I think it is.

I've recently started using the function that fetches story URLs from email, and I've noticed that the process slows down significantly whenever one of the URLs is from Royal Road. My first thought was that it might be related to Royal Road’s email-click redirect URLs, but I'm not certain, so I wanted to ask here to see if that's expected behavior.

I also feel like fetching metadata has been noticeably slower since a recent update. I’m not sure whether that’s just my imagination, but I can’t think of anything specific I changed that would cause it—although I *have* been modifying my `personal.ini` quite a bit. As far as I can remember, the only changes I made are listed below. Could any of these cause slowdown?

I also added some URLs to my previously empty reject list, and may have enabled the setting that splits URLs into separate jobs by site. I’m about 95% sure I already had that turned on previously.

Here are the relevant parts of my configuration:

[defaults]
# Commented out
#open_pages_in_browser:true

# Added:
skip_threadmarks_categories:Media,Apocrypha,Informational,Staff Post,Extras,Voting,Index
order_threadmarks_by_date:true
mark_new_chapters:true
use_reader_mode:true

custom_columns_settings:
 "FanFicFare"=>#downloaded_by,n
add_to_include_metadata_pre:
 parentforums==Quests

strip_chapter_numbers:true
add_chapter_numbers:toconly

[epub]
# Changed from true to false
include_images:false

# Added:
[www.scribblehub.com]
username: <not included here>
password: <not included here>
#use_flaresolverr_proxy:true

[https://forums.spacebattles.com/thre...i-oc.1211852/]
skip_threadmarks_categories:Media,Apocrypha,Informational,Staff Post,Extras,Voting,Index,Sidestory

[https://forums.spacebattles.com/thre...elow.1225985/]
order_threadmarks_by_date:false

[JimmXinu]

Originally Posted by WinterPhoenix

I've recently started using the function that fetches story URLs from email, and I've noticed that the process slows down significantly whenever one of the URLs is from Royal Road. ...

I don't remember anything that changed recently that would have that effect. And except for logging in (and slow_down_sleep_time of course), most personal.ini settings don't change anything during the metadata fetch process.

Honestly, I'm inclined to blame it on the ever increasing levels of screening and blocking aimed at the AI training bots vacuuming the entire web over and over--especially written prose.

Considering the issues we have with other sites, I count myself lucky that FFF still works at all.

[WinterPhoenix]

Originally Posted by JimmXinu

Considering the issues we have with other sites, I count myself lucky that FFF still works at all.

Well that is unfortunate, but it does make sense. Hopefully its just some arcane issue on my end that will resolve itself, rather than a permanent issue.

[taratears]

When fetching email updates, I get this error.

calibre, version 8.14.0
ERROR: Error Fetching Email: <p><b>An error has occurred while FanFicFare was fetching email.</b></p><p>If it was an authentication error, be aware:<ul><li>Microsoft mail servers such as live.com, outlook.com and hotmail.com no longer allow third party apps like FanFicFare.</li><li>Gmail requires Two Factor Authentication and <a href='https://support.google.com/accounts/answer/185833#app-passwords'>Google App Passwords</a>.</li><li>Usernames or passwords in personal.ini that contain the percent sign(%) must have it escaped as two percent signs(%%).</li></ul>

Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 594, in get_urls_from_imap_menu
File "C:\Users\tarat\AppData\Roaming\calibre\plugins\Fa nFicFare.zip\fanficfare\geturls.py", line 195, in get_urls_from_imap
File "imaplib.py", line 1332, in __init__
File "imaplib.py", line 205, in __init__
File "imaplib.py", line 1345, in open
File "imaplib.py", line 315, in open
File "imaplib.py", line 1335, in _create_socket
File "imaplib.py", line 305, in _create_socket
File "socket.py", line 839, in create_connection
File "socket.py", line 974, in getaddrinfo
socket.gaierror: [Errno 11001] getaddrinfo failed

[JimmXinu]

Originally Posted by taratears

When fetching email updates, I get this error.
...
File "socket.py", line 974, in getaddrinfo
socket.gaierror: [Errno 11001] getaddrinfo failed

That error indicates a failure to get an IP address from DNS for the name given.

That suggests that you have a mistake in the imap server name (That's what it was last time this came up), or that there's a problem with getting it from DNS. Look for advice on how to flush DNS cache on your type of computer if you think it's the DNS.

[gthesob]

I've noticed in recent weeks that occasionally on QQ when I update in FFF the story on QQ gets marked as read and loses my place so that after using "jump to new" I have to backtrack to figure out where I actually had been. Sometimes "jump to new" takes me to the next post after the threadmark, sometimes to several posts or even pages later.

I have not noticed any obvious patterns that would let me predict when it will happen, such as a specific story, or where "jump to new" will take me. Sorry, I can't be any less vague, it just doesn't happen very often and I have not kept notes on its occurrences.

Not a big deal, just thought I'd let you know as I have not seen this happen on AO3, SV or SB.

[JimmXinu]

AO3 isn't going to be the same, but QQ, SV and SB all share code in FFF.

If QQ is behaving differently, it's going to be because the site is different in some way, or you use QQ differently.

I don't think I've ever paid any attention to how FFF affects 'reading location' on xenforo sites. As a general rule, I've only cared about 'read' status on sites that won't notify you for new updates unless you're 'caught up', eg TtH. And that's to ensure it's been marked read, which seems to be the opposite of what you want.

I assume it's only affected when downloading while logged in. In fact, I suspect the difference is that you always have FFF log in to QQ. So the site assumes that anything FFF has seen, you have seen.

By default, the adapter uses xenforo's reader mode to get the most 'chapter' posts per network request. So I don't have an explanation why QQ would mark posts past the thread marked ones 'read'--other than maybe the site marks all posts that would be on the same page 'read'.

OR "jump to new" on QQ doesn't mean what you think it means. IE, new to you. Because I just went and looked at the first two pages of thread I'd never looked at before (in browser), the clicked 'jump to new' expecting to get page 3 and got page 79.

That suggest pretty strongly it's actually new=='recent', not new=='I haven't see'.

[chrisric]

Hi - I noticed books downloaded via CLI do not get a cover from royalroad. (haven't tried other sites)

I added some debug logging.

fanficfare -b 1 -e 2 -d "https://www.royalroad.com/fiction/137415/unclassed-a-skill-merging-litrpg"

I tried:

fanficfare --update-epub --update-cover -d "Unclassed_ A Skill Merging LitRPG (Ch 1-2)-rylrdl_137415.epub"

Same results.

FFF: DEBUG: 2025-11-17 12:01:59,603: adapter_royalroadcom.py(292): --- Cover Processing START ---
FFF: DEBUG: 2025-11-17 12:01:59,603: adapter_royalroadcom.py(294): Found cover image URL in page: https://www.royalroadcdn.com/public/covers-large/137415-unclassed-a-skill-merging-litrpg.jpg?time=1762674414
FFF: DEBUG: 2025-11-17 12:01:59,604: adapter_royalroadcom.py(298): Attempting to fetch conjectured large version: https://www.royalroadcdn.com/public/covers-large/137415-unclassed-a-skill-merging-litrpg.jpg?time=1762674414
FFF: DEBUG: 2025-11-17 12:01:59,604: adapter_royalroadcom.py(305): Call to setCoverImage for large cover returned: (None, None)
FFF: DEBUG: 2025-11-17 12:01:59,604: adapter_royalroadcom.py(317): --- Cover Processing END ---

From GUI:

https://www.royalroad.com/fiction/137415
FFF: DEBUG: 2025-11-17 13:14:09,725: requestable.py(55): Encoding:utf8
FFF: DEBUG: 2025-11-17 13:14:10,205: adapter_royalroadcom.py(279): stars:(4.62958)
FFF: DEBUG: 2025-11-17 13:14:10,208: base_adapter.py(637): setCoverImage(https://www.royalroad.com/fiction/137415,https://www.royalroadcdn.com/public/covers-large/137415-unclassed-a-skill-merging-litrpg.jpg?time=1762674414)
FFF: DEBUG: 2025-11-17 13:14:10,209: story.py(1560): addImgUrl(parenturl=https://www.royalroad.com/fiction/137415,url=https://www.royalroadcdn.com/public/covers-large/137415-unclassed-a-skill-merging-litrpg.jpg?time=1762674414,cover=specific,coverexclusion=(imgur.com/dzOACJf.png|/forum/images/smilies/)

My .ini files are the same for "pip installed" FFF and Calibre FFF plugin.

curl can download the cover just fine so I know the file exists.

curl -LO "https://www.royalroadcdn.com/public/covers-large/137415-unclassed-a-skill-merging-litrpg.jpg"

I must be missing something obvious. I checked the .ini files for answers but didn't see anything.

Thanks.

-Chris

Edit:

You have to set "include_images:true" in personal.ini, it now works.

## include images from img tags in the body and summary of stories.
## Images will be converted to jpg for size if possible.  Images work
## in epub format only.  To get mobi or other format with images,
## download as epub and use Calibre to convert.
include_images:true

[chrisric]

How can I update cover images for a bunch of existing books in calibre? I was planning on using "Update Calibre Metadata from Web Site" but this does not seem to download covers.

Thanks.

-Chris

Edit:

Is there a way to keep tags sourced from RR up to date and keep my locally add tags too? I can enable "Keep Existing Tags when Updating Metadata" but that means any tags changes from RR are not synced.

Eg: Book 1, tags from RR: hiatus, inactive - Local Tags: Favs, Lib1, Tag1

If "Keep Existing Tags when Updating Metadata" in enabled, I keep my local tags but when a book becomes active again or not on hiatus anymore the tags are not removed.

[JimmXinu]

Originally Posted by chrisric

Hi - I noticed books downloaded via CLI do not get a cover from royalroad. (haven't tried other sites)
...
You have to set "include_images:true" in personal.ini, it now works.

FYI, this thread is specifically for the Calibre plugin version of FFF.

Please use github or the FanFicFare maillist for CLI specific questions.

Originally Posted by chrisric

How can I update cover images for a bunch of existing books in calibre? I was planning on using "Update Calibre Metadata from Web Site" but this does not seem to download covers.

Use 'Update EPUB Always'.

Originally Posted by chrisric

Is there a way to keep tags sourced from RR up to date and keep my locally add tags too? I can enable "Keep Existing Tags when Updating Metadata" but that means any tags changes from RR are not synced.

Eg: Book 1, tags from RR: hiatus, inactive - Local Tags: Favs, Lib1, Tag1

If "Keep Existing Tags when Updating Metadata" in enabled, I keep my local tags but when a book becomes active again or not on hiatus anymore the tags are not removed.

FFF automatically handles 'In-Progress' and 'Completed' in Calibre's Tags that way. Which is why I make it a point to use those strings for all sites, those are the only two that are considered 'standard' values of status. And it only happens on Tags, not custom columns.

Site specific status values, ie hiatus and inactive, do not get special handling.

I suppose I could make that list of values a configurable plugin setting if it would be useful to more users. Anyone else want this?

How I'd do it:

Spoiler:

There would be an edit box on the plugin config Standard Columns tab where you could enter/change the list of 'status' values that should be exclusive. Only one of them should exist in Tags when FFF is done.

It would function pretty simply, by removing all the configured values from Tags if any of them are in the updated book tags before adding the tags from the updated book.

Only applied when Keep Existing tags is checked and New Only on the standard tab is not checked.