[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by kimboosan

Randomly, I commented out "always_login:true" and it worked, but only for unlocked fic (natch).

I mean, it's working, sorta. My question is, does this suggest that my user profile is blocked? Obviously not my IP since it worked without logging in. Is this something I should put in a ticket with AO3 for?

Where you previously trying to download a large batch at once? Because if it's a traffic block, you can usually wait a few days (or weeks) and it will go back to normal.

I wouldn't bug AO3 about it. I'm always afraid that drawing attention to a downloader will result in more blocking, not less. No idea if it really would.

[JimmXinu]

New Version Released

Version 4.50.0 - 2025-10-19
- OTW(AO3): Fix for site change breaking logged in detection. Closes #1263
- adapter_literotica: Site change, regex outdated. PR1261, thanks dbhmw!
- adapter_storiesonlinenet: Add check for SOL accounts in renewal warning period to verbosely explain to users why their downloads don't work. PR1262, thanks bpothier!
- base_otw_adapter: Fix for markedforlater site change
- OTW(AO3): Accept /chapter/999 URLs without /works/999 for #1258
- Fix tuple vs grouping vs list, closes #1254 (royalroad "'NoneType' object does not support item assignment")
- Spiritfanfiction login, closes #1247, thanks kilandra
- New site: althistory.com (NOT alternatehistory.com) for #1252

Released early due to AO3 site change.

[indecisivefan]

Originally Posted by JimmXinu

I'm not seeing this. Can you provide example URL and a debug log?

Sure. It's the "URLs from Webpage" itself that's not working, as I've tried it on many different pages (my bookmarks, a collection, a series, etc.). Here is one URL that I confirmed did not work:
https://archiveofourown.org/tags/Original%20Work/works

And here's the error and debug log:

calibre, version 8.13.0
ERROR: Unhandled exception: <b>KeyError</b>:'href'

calibre 8.13 embedded-python: True
Windows-10-10.0.19044-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19044')
Python 3.11.14
Windows: ('10', '10.0.19044', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre2\calibre.exe
Successfully initialized third party plugins: DeDRM (10, 0, 3) && Count Pages (1, 14, 1) && EpubMerge (3, 2, 0) && FanFicFare (4, 49, 6) && Generate Cover (2, 3, 6)
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 712, in get_urls_from_page_menu
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 735, in get_urls_from_page
File "C:\Users\NAME\AppData\Roaming\calibre\plugins\Fan FicFare.zip\fanficfare\geturls.py", line 44, in get_urls_from_page
File "C:\Users\NAME\AppData\Roaming\calibre\plugins\Fan FicFare.zip\fanficfare\adapters\base_adapter.py", line 504, in get_urls_from_page
File "C:\Users\NAME\AppData\Roaming\calibre\plugins\Fan FicFare.zip\fanficfare\adapters\base_otw_adapter.p y", line 688, in get_series_from_page
File "C:\Users\NAME\AppData\Roaming\calibre\plugins\Fan FicFare.zip\fanficfare\adapters\base_otw_adapter.p y", line 688, in <listcomp>
File "bs4\element.py", line 1573, in __getitem__
KeyError: 'href'

[JimmXinu]

New Test Version Posted

2025-10-18
- OTW/AO3: Don't apply series page handling to non-series pages

For indecisivefan's issue.

I'm honestly not sure if AO3 changed, or this just never came up before. Given the logout tag changed...

[kimboosan]

Originally Posted by JimmXinu

Where you previously trying to download a large batch at once? Because if it's a traffic block, you can usually wait a few days (or weeks) and it will go back to normal.

I wouldn't bug AO3 about it. I'm always afraid that drawing attention to a downloader will result in more blocking, not less. No idea if it really would.

Good point. That said, I remembered I had an old account there so used that log-in info, got the same result, so it's not personal! LOL!

For documentation's sake: I tried to update 10 fic, and six of them worked fine while four, which are locked, did not (expected behavior). Then I tried 20 fic, and it failed completely even with unlocked fic (same 403 error). I went back and tried to update 1 unlocked fic and it failed (403 error).

Which makes me think this is server side with AO3 kicking out requests after a set number. However I don't see anyone else having this problem????

I'll try the browser cache approach and see what happens. Oh, for the good old days before the AO3 dos attacks and server changes...life was simpler then!!!

[taratears]

I'm using it to grab ao3 updates thru yahoo email, and it works, but requires login every time despite login info being in personal.ini with adult set to true.
Safe copy of personal.ini below.

Spoiler:

## This is an example of what your personal configuration might look
## like. Uncomment options by removing the '#' in front of them.

[defaults]
## [defaults] section applies to all formats and sites but may be
## overridden at several levels. See
## https://github.com/JimmXinu/FanFicFare/wiki/INI-File for more
## details.

## Some sites also require the user to confirm they are adult for
## adult content. Uncomment by removing '#' in front of is_adult.
#is_adult:true

## Don't like the numbers at the start of chapter titles on some
## sites? You can use strip_chapter_numbers to strip them off. Just
## want to make them all look the same? Strip them off, then add them
## back on with add_chapter_numbers. Don't like the way it strips
## numbers or adds them back? See chapter_title_strip_pattern and
## chapter_title_add_pattern in defaults.ini.
#strip_chapter_numbers:true
#add_chapter_numbers:true


[epub]
## Include images from img tags in the body and summary of stories.
## Images will be converted to jpg for size if possible. Images work
## in epub format only. To get mobi or other format with images,
## download as epub and use Calibre to convert.
## true by default, uncomment and set false to not include images.
#include_images:true

## If set false, the summary will have all html stripped for safety.
## Both this and include_images must be true to get images in the
## summary.
## true by default, uncomment and set false to not keep summary html.
#keep_summary_html:true

## If set true, and there isn't a specific cover image, the first
## image found in the story will be made the cover image. If
## keep_summary_html is true, images in the summary will be before any
## in chapters.
## true by default, uncomment and set false to turn off
#make_firstimage_cover:true


## Most common, I expect will be using this to save username/passwords
## for different sites. Here are a few examples. See defaults.ini
## for the full list.

[www.twilighted.net]
#username:XXXXXXXX
#password:XXXXXXXX
## default is false
#collect_series: true

[www.fimfiction.net]
#is_adult:true
#fail_on_password:XXXXXXXX

[www.tthfanfic.org]
#is_adult:true
## tth is a little unusual--it doesn't require user/pass, but the site
## keeps track of which chapters you've read and won't send another
## update until it thinks you're up to date. If you set
## username/password, FFF will login to download. Then the site
## thinks you're up to date.
#username:XXXXXXXX
#password:XXXXXXXX

[www.archiveofourown.org]
## This is a OTW-archive site.
## Some sites require login (or login for some rated stories) The
## program can prompt you, or you can save it in config. In
## commandline version, this should go in your personal.ini, not
## defaults.ini.
#username:XXXXXXXX
#password:XXXXXXXX
#is_adult:true

## This section will override anything in the system defaults or other
## sections here.
[overrides]
## default varies by site. Set true here to force all sites to
## collect series.
#collect_series: true

#slow_down_sleep_time:4

[ownedbycats]

Originally Posted by JimmXinu

New Test Version Posted

2025-10-18
- OTW/AO3: Don't apply series page handling to non-series pages

For indecisivefan's issue.

I'm honestly not sure if AO3 changed, or this just never came up before. Given the logout tag changed...

otwarchive might've changed something. Since beginning of month, I see changes AO3-7113, AO3-6471, AO3-5668.

[JimmXinu]

Originally Posted by taratears

I'm using it to grab ao3 updates thru yahoo email, and it works, but requires login every time despite login info being in personal.ini with adult set to true.
Safe copy of personal.ini below.

[www.archiveofourown.org]
## This is a OTW-archive site.
## Some sites require login (or login for some rated stories) The
## program can prompt you, or you can save it in config.  In
## commandline version, this should go in your personal.ini, not
## defaults.ini.
#username:XXXXXXXX
#password:XXXXXXXX
#is_adult:true

You have to remove the leading '#' from those lines to uncomment them. Note the color highlighting.

[texasnightowl]

ok...never mind me.

i updated from 4.48 to 4.50 and it was fine.

[taratears]

May I ask what the colors mean? Red is obvious, but I didn't see a list of what the colors mean, and I'm not certain what to search for to look it up.

[wildfire070]

I'm seeing an issue with "Get Story URLs from Web Page" as well when trying to download from an author's works page on AO3. I haven't used "get story from web page" in a couple months though, so not sure when AO3 changed things on their end.

[JimmXinu]

Originally Posted by taratears

May I ask what the colors mean? Red is obvious, but I didn't see a list of what the colors mean, and I'm not certain what to search for to look it up.

It's not documented as such. From https://github.com/JimmXinu/FanFicFa...ighter.py#L41:

        colors = {
            'knownentries':Qt.darkGreen,
            'errors':Qt.red,
            'allkeywords':Qt.darkMagenta,
            'knownkeywords':Qt.blue,
            'knownsections':Qt.darkBlue,
            'teststories':Qt.darkCyan,
            'storyUrls':Qt.darkMagenta,
            'comments':Qt.darkYellow
            }
        try:
            if( hasattr(QApplication.instance(),'is_dark_theme')
                and QApplication.instance().is_dark_theme ):
                colors = {
                    'knownentries':Qt.green,
                    'errors':Qt.red,
                    'allkeywords':Qt.magenta,
                    'knownkeywords':QColor(Qt.blue).lighter(150),
                    'knownsections':Qt.darkCyan,
                    'teststories':Qt.cyan,
                    'storyUrls':QColor(Qt.magenta).lighter(150),
                    'comments':Qt.yellow
                    }
        except Exception as e:
            logger.error("Failed to set dark theme highlight colors: %s"%e)

[ownedbycats]

Is there a way, maybe in longtext metadata column, to dfferentiate newly-downloaded fic from updated one after download?

[JimmXinu]

If you use log page and save numWords, you could look at words_added. That's my general indicator for new download vs update--my words_added column.

I also I use this, but the only place it's useful is in logpage_entries because logpage automatically leaves out unchanged values except on overwrite, so this indicates overwrite always vs 'lots of fields changed':

add_to_extra_valid_entries:,fulldownload
include_in_fulldownload:title
fulldownload_label:New Download

add_to_replace_metadata:
 fulldownload=>^.*$=>Yes

[Dudemanguy]

Been having problems with FFF for a few days now that i didn't have before, every time i try to download fics from ao3 using URL's from webpage feature i get the following error:
calibre, version 8.13.0
ERROR: Unhandled exception: <b>KeyError</b>:'href'

calibre 8.13 embedded-python: True
macOS-26.0.1-arm64-arm-64bit Darwin ('64bit', '')
('Darwin', '25.0.0', 'Darwin Kernel Version 25.0.0: Wed Sep 17 21:41:23 PDT 2025; root:xnu-12377.1.9~141/RELEASE_ARM64_T8122')
Python 3.11.14
OSX: ('26.0.1', ('', '', ''), 'arm64')
Interface language: None
EXE path: /Applications/calibre.app/Contents/MacOS/calibre
Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (2, 27, 0) && DeDRM (10, 0, 9) && Package KFX (from KFX Input) (2, 27, 0) && FanFicFare (4, 50, 0) && KFX metadata reader (from KFX Input) (2, 27, 0) && From KFX (2, 27, 0) && KFX Input (2, 27, 0) && Obok DeDRM (10, 0, 9)
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 712, in get_urls_from_page_menu
frompage = self.get_urls_from_page(url)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 735, in get_urls_from_page
return get_urls_from_page(url,configuration)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/NAME/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/geturls.py", line 44, in get_urls_from_page
return adapter.get_urls_from_page(url,normalize)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/NAME/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/adapters/base_adapter.py", line 504, in get_urls_from_page
series = self.get_series_from_page(url,data,normalize)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/NAME/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/adapters/base_otw_adapter.py", line 688, in get_series_from_page
pageurls = [ a['href'] for a in pageas ]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/NAME/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/adapters/base_otw_adapter.py", line 688, in <listcomp>
pageurls = [ a['href'] for a in pageas ]
~^^^^^^^^
File "bs4/element.py", line 1573, in __getitem__
KeyError: 'href'