[GUI Plugin] FanFicFare

[ownedbycats]

Is there a way, maybe in longtext metadata column, to dfferentiate newly-downloaded fic from updated one after download?

[JimmXinu]

If you use log page and save numWords, you could look at words_added. That's my general indicator for new download vs update--my words_added column.

I also I use this, but the only place it's useful is in logpage_entries because logpage automatically leaves out unchanged values except on overwrite, so this indicates overwrite always vs 'lots of fields changed':

add_to_extra_valid_entries:,fulldownload
include_in_fulldownload:title
fulldownload_label:New Download

add_to_replace_metadata:
 fulldownload=>^.*$=>Yes

[Dudemanguy]

Been having problems with FFF for a few days now that i didn't have before, every time i try to download fics from ao3 using URL's from webpage feature i get the following error:
calibre, version 8.13.0
ERROR: Unhandled exception: <b>KeyError</b>:'href'

calibre 8.13 embedded-python: True
macOS-26.0.1-arm64-arm-64bit Darwin ('64bit', '')
('Darwin', '25.0.0', 'Darwin Kernel Version 25.0.0: Wed Sep 17 21:41:23 PDT 2025; root:xnu-12377.1.9~141/RELEASE_ARM64_T8122')
Python 3.11.14
OSX: ('26.0.1', ('', '', ''), 'arm64')
Interface language: None
EXE path: /Applications/calibre.app/Contents/MacOS/calibre
Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (2, 27, 0) && DeDRM (10, 0, 9) && Package KFX (from KFX Input) (2, 27, 0) && FanFicFare (4, 50, 0) && KFX metadata reader (from KFX Input) (2, 27, 0) && From KFX (2, 27, 0) && KFX Input (2, 27, 0) && Obok DeDRM (10, 0, 9)
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 712, in get_urls_from_page_menu
frompage = self.get_urls_from_page(url)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 735, in get_urls_from_page
return get_urls_from_page(url,configuration)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/NAME/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/geturls.py", line 44, in get_urls_from_page
return adapter.get_urls_from_page(url,normalize)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/NAME/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/adapters/base_adapter.py", line 504, in get_urls_from_page
series = self.get_series_from_page(url,data,normalize)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/NAME/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/adapters/base_otw_adapter.py", line 688, in get_series_from_page
pageurls = [ a['href'] for a in pageas ]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/NAME/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/adapters/base_otw_adapter.py", line 688, in <listcomp>
pageurls = [ a['href'] for a in pageas ]
~^^^^^^^^
File "bs4/element.py", line 1573, in __getitem__
KeyError: 'href'

[JimmXinu]

I believe that is already addressed in the posted test version.

[Elaine T]

Originally Posted by JimmXinu

I believe that is already addressed in the posted test version.

FWIW, I just did a download from webpage and it behaved properly.

[HowWouldIKnow]

Hello there.
Is it possible to get Ratings, SCore, Favorites from
Literotica?

On a side note: Can I see somewhere what additional fields there are per website?

[JimmXinu]

Originally Posted by HowWouldIKnow

Hello there.
Is it possible to get Ratings, SCore, Favorites from
Literotica?

The adapter for literotica collects these site specific entries:

extra_valid_entries:eroticatags,averrating
eroticatags_label:Erotica Tags
averrating_label:Average Rating

If you open an issue about adding additional metadata entries (with details about what and where they appear), I can call it to the addition of the dev who works on the adapter for literotica lately. (No promises they would be interested in doing it, tho.)

Originally Posted by HowWouldIKnow

On a side note: Can I see somewhere what additional fields there are per website?

Look in defaults.ini and search for the site's section. Look at the extra_valid_entries setting.

[Mandabar]

I just noticed something today. If I'm following a story, say on royalroad, that like many others likes to stub itself. Deleting chapters to make them exclusively paid for, on Amazon KU is the usual place.

Used to be, I would just get an error on update, due to having more chapters in my book copy than are available online. I would remove in-progress from the story, and make a new book with the same url to continue grabbing the newer content.

I've been updating less frequently to prevent as many hits to the site and story, so more stubbing and updates since stubbing must've happened in a few stories. What I'm referring to is... hmm..
An example.

A story has chapters 1-100. Chapters 5-40 get removed. My copy has chapters 1-100. The author keeps releasing chapters. I attempt to update the story, which is now 1-5:41-200, and it updates fine with no error message; however, it deletes from my copy chapters 6-40. Since my copy had less than the online version.

Is this a change to how it used to work, never removing content from local copy and forcing manual edits? Not sure how many stories I might've lost chapters from instead of just making a new book for. Or is this how it always worked, that I just avoided because of more frequent update scraping?

(Noticed today, was a single update and checked before updating and clearly saw the TOC lost the first book and was able to recover from recently deleted formats)

I know this has always been an issue, that had to be delt with manually by making new books.

[HowWouldIKnow]

Originally Posted by JimmXinu

The adapter for literotica collects these site specific entries....
Look in defaults.ini and search for the site's section. Look at the extra_valid_entries setting.

Thank you!
So, extra_valid_entries are just those that are additionally available for this Website? Ahh and the same counts for extra_titlepage_entries, I see.
Does the order matter? ie.. Can I use the averrating in the custom_columns_settings, before the definition of the extra_valid_entires?

[JimmXinu]

Multi-reply

Originally Posted by Mandabar

An example.

A story has chapters 1-100. Chapters 5-40 get removed. My copy has chapters 1-100. The author keeps releasing chapters. I attempt to update the story, which is now 1-5:41-200, and it updates fine with no error message; however, it deletes from my copy chapters 6-40. Since my copy had less than the online version.

Update EPUB updates or not based just on the number of chapters. If there's more chapters on the site than in the epub, it assumes there are new chapters. It's always worked this way.

This whole business of authors deliberately removing chapters to encourage purchase is outside FanFicFare's paradigm. I am not interested in trying to work around it better for both technical and moral reasons. This has come up before.

Originally Posted by HowWouldIKnow

Thank you!
So, extra_valid_entries are just those that are additionally available for this Website? Ahh and the same counts for extra_titlepage_entries, I see.
Does the order matter? ie.. Can I use the averrating in the custom_columns_settings, before the definition of the extra_valid_entires?

As I recall, order of settings in the INI file doesn't matter. Order within a setting, like extra_titlepage_entries or replace_metadata matters a great deal.

[Ryune]

Getting the weirdest error recently. Currently a problem on both the test build and official release. I've been getting this and on occasion and not sure what is going on with it.


Error
The Citadel Expedition
40thousandninjas
File is not a zip file
https://forums.spacebattles.com/thre...rgate.1097162/

[JimmXinu]

Originally Posted by Ryune

Getting the weirdest error recently. Currently a problem on both the test build and official release. I've been getting this and on occasion and not sure what is going on with it.

Error
The Citadel Expedition
40thousandninjas
File is not a zip file
https://forums.spacebattles.com/thre...rgate.1097162/

Can you provide a debug log?

The only reason I can think of for that error is trying to update an epub that's been corrupted or truncated.

[Ryune]

Originally Posted by JimmXinu

Can you provide a debug log?

The only reason I can think of for that error is trying to update an epub that's been corrupted or truncated.

Unfortunately, the default settings would not produce a log at all. I had to set it to "always overwrite" to get the job log.

Download 1 FanFiction Book(s) for forums.spacebattles.com
FFF: INFO: 2025-10-28 00:00:14,522: calibre_plugins.fanficfare_plugin.jobs(39): FanFicFare v4.50.1
calibre 8.13  embedded-python: True
Windows-10-10.0.22631-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.22631')
Python 3.11.14
Windows: ('10', '10.0.22631', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre2\calibre-parallel.exe
Successfully initialized third party plugins: EpubMerge (3, 2, 0) && FanFicFare (4, 50, 1) && Generate Cover (2, 3, 6) && KOReader Sync (0, 7, 2) && Manage Sony x50 Reader Book List (1, 2, 11) && Reading List (1, 15, 6) && SmartEject (2, 6, 0) && TTS to MP3 (0, 6, 0)
FFF: INFO: 2025-10-28 00:00:22,885: calibre_plugins.fanficfare_plugin.jobs(146): 

-------------------------------------------------------------------------------- https://forums.spacebattles.com/threads/the-citadel-expedition-mass-effect-stargate.1097162/
FFF: DEBUG: 2025-10-28 00:00:22,885: configurable.py(643): config site:forums.spacebattles.com
FFF: DEBUG: 2025-10-28 00:00:22,900: configurable.py(1110): use_browser_cache:
FFF: DEBUG: 2025-10-28 00:00:22,900: configurable.py(1131): use_basic_cache:true
FFF: INFO: 2025-10-28 00:00:22,901: base_xenforo2forum_adapter.py(721): url: https://forums.spacebattles.com/threads/the-citadel-expedition-mass-effect-stargate.1097162/
FFF: DEBUG: 2025-10-28 00:00:22,901: cache_basic.py(117): 
========== HIT (GET) BasicCache
https://forums.spacebattles.com/threads/the-citadel-expedition-mass-effect-stargate.1097162/
FFF: DEBUG: 2025-10-28 00:00:22,901: requestable.py(55): Encoding:utf8
FFF: DEBUG: 2025-10-28 00:00:22,902: requestable.py(89): Applying max_zalgo:3
FFF: INFO: 2025-10-28 00:00:22,934: base_xenforo2forum_adapter.py(725): use useurl: https://forums.spacebattles.com/threads/the-citadel-expedition-mass-effect-stargate.1097162/
FFF: DEBUG: 2025-10-28 00:00:23,291: cache_basic.py(117): 
========== HIT (GET) BasicCache
https://forums.spacebattles.com/threads/the-citadel-expedition-mass-effect-stargate.1097162/threadmarks?per_page=200
FFF: DEBUG: 2025-10-28 00:00:23,291: requestable.py(55): Encoding:utf8
FFF: DEBUG: 2025-10-28 00:00:23,291: requestable.py(89): Applying max_zalgo:3
FFF: DEBUG: 2025-10-28 00:00:23,576: cache_basic.py(117): 
========== HIT (GET) BasicCache
https://forums.spacebattles.com/threads/the-citadel-expedition-mass-effect-stargate.1097162/threadmarks?threadmark_category=7&per_page=200
FFF: DEBUG: 2025-10-28 00:00:23,577: requestable.py(55): Encoding:utf8
FFF: DEBUG: 2025-10-28 00:00:23,577: requestable.py(89): Applying max_zalgo:3
FFF: DEBUG: 2025-10-28 00:00:23,782: calibre_plugins.fanficfare_plugin.jobs(199): metadata newforanthology:
FFF: INFO: 2025-10-28 00:00:23,787: calibre_plugins.fanficfare_plugin.jobs(381): Exception: {'title': 'The Citadel Expedition', 'author_sort': ['40thousandninjas'], 'author': ['40thousandninjas'], 'comments': '<div class="bbWrapper">Codex Entry: The Ring of the Protheans.<br/>\n<br/>\nFound on most garden worlds, the Ring of the Protheans is a tall ring like structure theorized by anthropologists to demarcate their empire fifty thousand standard galactic years ago. The architecture is distinct from prothean ruins of any era but commonly located in the center of what are thought to be prothean plazas, trading posts, cities and other community areas. The symbols of the ring are not found elsewhere in prothean culture, but each ring shares common symbols with others across the galaxy. The exact meaning and purpose of these symbols is unknown but postulated to be religious or cultural in nature...</div>', 'good': False, 'status': 'Error', 'showerror': True, 'calibre_id': 12530, 'begin': None, 'end': None, 'comment': 'File is not a zip file', 'url': 'https://forums.spacebattles.com/threads/the-citadel-expedition-mass-effect-stargate.1097162/', 'site': 'forums.spacebattles.com', 'series': '', 'added': False, 'pubdate': datetime.datetime(2023, 6, 10, 10, 39, 2, tzinfo=datetime.timezone(datetime.timedelta(days=-1, seconds=72000), 'Eastern Daylight Time')), 'publisher': 'forums.spacebattles.com', 'uniqueurl': 'https://forums.spacebattles.com/threads/the-citadel-expedition-mass-effect-stargate.1097162/[None-None]', 'listorder': 0, 'collision': 'Overwrite Always', 'is_adult': False, 'username': 'NoneGiven', 'password': '', 'totp': None, 'icon': 'dialog_error.png', 'all_metadata': {'category': 'Mass Effect, Stargate', 'genre': 'Crossover, Discuss I Demand It, Discussion Makes The Author Happy, Mass Effect, Stargate', 'characters': '', 'ships': '', 'warnings': '', 'extratags': 'FanFiction', 'author': '40thousandninjas', 'authorId': '40thousandninjas.309222', 'authorUrl': 'https://forums.spacebattles.com/members/40thousandninjas.309222/', 'lastupdate': 'Last Update Year/Month: 2025/10, Last Update: 2025/10/25', 'series': '', 'seriesUrl': '', 'language': '', 'status': 'In-Progress', 'datePublished': '2023-06-10 10:39', 'dateUpdated': '2025-10-25 00:14', 'dateCreated': '2025-10-28 00:00:07', 'rating': '', 'numChapters': '110', 'numWords': '', 'words_added': '', 'site': 'forums.spacebattles.com', 'publisher': 'forums.spacebattles.com', 'storyId': '1097162', 'title': 'The Citadel Expedition', 'titleHTML': "<a class='titlelink' href='https://forums.spacebattles.com/threads/the-citadel-expedition-mass-effect-stargate.1097162/'>The Citadel Expedition</a>", 'storyUrl': 'https://forums.spacebattles.com/threads/the-citadel-expedition-mass-effect-stargate.1097162/', 'sectionUrl': 'https://forums.spacebattles.com/threads/1097162/', 'description': '<div class="bbWrapper">Codex Entry: The Ring of the Protheans.<br/>\n<br/>\nFound on most garden worlds, the Ring of the Protheans is a tall ring like structure theorized by anthropologists to demarcate their empire fifty thousand standard galactic years ago. The architecture is distinct from prothean ruins of any era but commonly located in the center of what are thought to be prothean plazas, trading posts, cities and other community areas. The symbols of the ring are not found elsewhere in prothean culture, but each ring shares common symbols with others across the galaxy. The exact meaning and purpose of these symbols is unknown but postulated to be religious or cultural in nature...</div>', 'formatname': '', 'formatext': '', 'siteabbrev': 'fsb', 'version': 'FanFicFare v4.50.1', 'authorHTML': "<a class='authorlink' href='https://forums.spacebattles.com/members/40thousandninjas.309222/'>40thousandninjas</a>", 'seriesHTML': '', 'langcode': '', 'output_css': '', 'cover_image': '', 'newforanthology': '', 'tagsfromtitledetect': 'The Citadel Expedition (Mass Effect/ Stargate)', 'tagsfromtitle': 'Mass Effect, Stargate', 'forumtags': 'Crossover, Discuss I Demand It, Discussion Makes The Author Happy, Mass Effect, Stargate', 'prefixtags': '', 'contenttags': 'Crossover', 'formattags': '', 'timeperiodtags': '', 'parentforums': '', 'threadmarks_title': '', 'threadmarks_description': '<div class="bbWrapper">Codex Entry: The Ring of the Protheans.<br/>\n<br/>\nFound on most garden worlds, the Ring of the Protheans is a tall ring like structure theorized by anthropologists to demarcate their empire fifty thousand standard galactic years ago. The architecture is distinct from prothean ruins of any era but commonly located in the center of what are thought to be prothean plazas, trading posts, cities and other community areas. The symbols of the ring are not found elsewhere in prothean culture, but each ring shares common symbols with others across the galaxy. The exact meaning and purpose of these symbols is unknown but postulated to be religious or cultural in nature...</div>', 'threadmarks_status': 'Ongoing', 'estimatedWords': '130k'}, 'tags': ['FanFiction', 'Crossover', 'Discuss I Demand It', 'Discussion Makes The Author Happy', 'Mass Effect', 'Stargate', 'In-Progress'], 'updatedate': datetime.datetime(2025, 10, 25, 0, 14, 46, tzinfo=datetime.timezone(datetime.timedelta(days=-1, seconds=72000), 'Eastern Daylight Time')), 'timestamp': datetime.datetime(2025, 10, 28, 0, 0, 22, 894920, tzinfo=datetime.timezone(datetime.timedelta(days=-1, seconds=72000), 'Eastern Daylight Time')), 'epub_for_update': 'C:\\Users\\Josh\\AppData\\Local\\Temp\\calibre-622w2wia\\fanficfare_pin722cn\\old-12530-ty25i7lw.epub', 'calibre_columns': {}, 'outfile': 'C:\\Users\\Josh\\AppData\\Local\\Temp\\calibre-622w2wia\\fanficfare_pin722cn\\The Citadel Expedition-40thousandninjas-xlbnjq48.epub'}:File is not a zip file
Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.jobs", line 230, in do_download_for_worker
  File "C:\Users\----\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\epubutils.py", line 39, in get_update_data
  File "zipfile.py", line 1329, in __init__
  File "zipfile.py", line 1396, in _RealGetContents
zipfile.BadZipFile: File is not a zip file
FFF: INFO: 2025-10-28 00:00:23,787: calibre_plugins.fanficfare_plugin.jobs(77): 
Download Results:
Error https://forums.spacebattles.com/threads/the-citadel-expedition-mass-effect-stargate.1097162/ File is not a zip file

FFF: INFO: 2025-10-28 00:00:23,788: calibre_plugins.fanficfare_plugin.jobs(102): 
Error
https://forums.spacebattles.com/threads/the-citadel-expedition-mass-effect-stargate.1097162/