[GUI Plugin] FanFicFare

[taratears]

Thank you for the prompt response. I'm currently pulling errors, despite login creds being in place, and that I'm pretty sure I'm not spamming it. I've tried one link several times over at least 5 hours.

[chicleeblair]

Same, I'm getting errors, and I've tried turning on only cache, logging in, logging out, cutting out all is_adult:true, putting them back in, everything I can think of.

Never had this much trouble with AO3

[wildfire070]

I've had pretty consistent, successful downloads using

use_archiveofourown_gay:true

This is under the AO3 section in my personal.ini:

always_login:true
username:XXXXXX
password:XXXXXX
use_view_full_work:true
use_archiveofourown_gay:true
use_browser_cache:false
use_browser_cache_only:false
open_pages_in_browser:false

[JimmXinu]

New Version Released

Version 4.46.0 - 2025-06-06
- OTW(AO3): Don't attempt login with use_archive_transformativeworks_org or open_pages_in_browser #1210
- Ignore use_browser_cache_only when URL scheme is file
- Update translations.
- adapter_ficbooknet: Site changes, thanks dbhmw
- adapter_literotica: Fix date parsing. See #1208, thanks dbhmw
- Add Ctrl-Enter to AddDialog, consolidating code with INIEdit
- Split BG: Fixes for error column & showing meta collection errors
- adapter_literotica: Update for site change
- adapter_literotica: Update for site change
- Single proc bg processing, optionally split by site & accumulate results (See [this post](https://www.mobileread.com/forums/sh...ostcount=10766))
- Fix remove_empty_p regexp to work with nested <br> tags and whitespace.
- AO3 make use_(domain) options not replace media.archiveofourown.org
- Fix 'Add New Book' dialog when multiple existing found on update.
- Include 'Add New Book' dialog when multiple existing found on update.
- adapter_spiritfanfictioncom: Fix for author collection after site change. Thanks, kilandra

[ownedbycats]

What are the default single-proc settings, for those who didn't install the test version?

[JimmXinu]

Originally Posted by ownedbycats

What are the default single-proc settings, for those who didn't install the test version?

All three options on. New processing, split by site, reconsolidate on finish.

[ownedbycats]

BTW, how does the split/reconsolidate affect the marked books? My post-download action chain is based on the marked:fff search. i remember some talk about it but not sure.

[JimmXinu]

As I recall, you will get marks based on the Proceed dialog.

If you leave them split, you'll get more than one Proceed, each of which reset marks. If you reconsolidate, you'll get all the updates marked at once.

[Ealdwulf]

I'm running into some issues with AO3 that are a tad bit different.

I updated to the latest FFF plugin yesterday when it became available for me to do so in Calibre, which is also at the latest version.

Here is the log for just one story (I'd actually create a table for it if I knew how to do so here, manually formatting it is a pain):

Status     Title                Author              Comment                                             URL
Error      Ravens and Wolves    Amberpoconuts       'NoneType' object has no attribute 'get_text'       https://archiveofourown.org/works/55867468

Issue: The story DOES exist and it was updated on 6/4/2025 and it does require login. It is occurring with all stories that require login, not just this one.


Here is the relevant part of my personal.ini:

[archiveofourown.org]
username:XXXXXXXX
password:XXXXXXXX
#always_login:false
always_login:true
is_adult:true
#use_archive_transformativeworks_org:true
use_archiveofourown_gay:true

## AO3 adapter defines a few extra metadata entries.
extra_valid_entries:fandoms,freeformtags,freefromtags,ao3categories,comments,kudos,hits,bookmarks,collections
fandoms_label:Fandoms
freeformtags_label:Freeform Tags
freefromtags_label:Freeform Tags
ao3categories_label:AO3 Categories
comments_label:Comments
kudos_label:Kudos
hits_label:Hits
collections_label:Collections
bookmarks_label:Bookmarks

## freeformtags was previously typo'ed as freefromtags.  This way,
## freefromtags will still work for people who've used it.
include_in_freefromtags:freeformtags

## adds to titlepage_entries instead of replacing it.
#extra_titlepage_entries: fandoms,freeformtags,ao3categories,comments,kudos,hits,bookmarks

## adds to include_subject_tags instead of replacing it.
#extra_subject_tags:fandoms,freeformtags,ao3categories

Below those are exclude metadata entries as well as replace metadata entries, other settings for AO3 are just the defaults. All that should be relevant is the first section, but...

• If I comment out use_archiveofourown_gay:true then I get the 525 error because it will use the actual site URL.
• If I use always_login:false and comment out always_login:true then it does of course give the same error as to be expected.
• I have not tried use_archive_transformativeworks_org:true since, as stated, the story does require login
• I have also been logged in on the .gay site and on the first chapter and it still reports the same error
• I have not tried using cached pages
• I have tried with both 'Overwrite if Newer' and 'Overwrite Always'.

[JimmXinu]

Originally Posted by Ealdwulf

I'm running into some issues with AO3 that are a tad bit different.

I updated to the latest FFF plugin yesterday when it became available for me to do so in Calibre, which is also at the latest version.

Here is the log for just one story (I'd actually create a table for it if I knew how to do so here, manually formatting it is a pain):

Status     Title                Author              Comment                                             URL
Error      Ravens and Wolves    Amberpoconuts       'NoneType' object has no attribute 'get_text'       https://archiveofourown.org/works/55867468

Downloaded fine for me as a new download and as an Overwrite Always with use_archiveofourown_gay:true.

If you can post a debug log, it will help pin point exactly where it's failing.

But I kind of expect that it's a matter of archiveofourown.gay starting to block now, too.

[ownedbycats]

For what it matters, i sent AO3 support a question regarding the 525 errors and asking when the new servers will be installed (if they're not already). I will post response when I get one.

[Ealdwulf]

Originally Posted by ownedbycats

I will post response when I get one.

I look forward to reading that post; and I would imagine I am not alone in saying that.

[Ealdwulf]

Originally Posted by JimmXinu

Downloaded fine for me as a new download and as an Overwrite Always with use_archiveofourown_gay:true.

If you can post a debug log, it will help pin point exactly where it's failing.

But I kind of expect that it's a matter of archiveofourown.gay starting to block now, too.

Debug Log:

Spoiler:

ApplicationPaletteChange event ignored
Using calibre Qt style: True
ApplicationPaletteChange event received
Traceback (most recent call last):
  File "calibre_plugins.wikidata_gui.common_utils", line 14, in <module>
ImportError: cannot import name 'QRegExpValidator' from 'qt.core' (C:\Program Files\Calibre2\app\bin\python-lib.bypy.frozen\qt\core.pyc)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre\gui2\ui.py", line 148, in __init__
  File "calibre\gui2\ui.py", line 166, in init_iaction
  File "calibre\customize\__init__.py", line 658, in load_actual_plugin
  File "importlib\__init__.py", line 126, in import_module
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "calibre\customize\zipplugin.py", line 210, in exec_module
  File "calibre_plugins.wikidata_gui.ui", line 22, in <module>
  File "calibre\customize\zipplugin.py", line 210, in exec_module
  File "calibre_plugins.wikidata_gui.importbooks", line 11, in <module>
  File "calibre\customize\zipplugin.py", line 210, in exec_module
  File "calibre_plugins.wikidata_gui.dictdicttable", line 12, in <module>
  File "calibre\customize\zipplugin.py", line 210, in exec_module
  File "calibre_plugins.wikidata_gui.common_utils", line 20, in <module>
ModuleNotFoundError: No module named 'PyQt4'
EpubMerge: DEBUG: 2025-06-07 12:22:57,845: calibre_plugins.epubmerge.epubmerge_plugin(156): macmenuhack file_path:C:\Users\bryan\AppData\Roaming\calibre\plugins\fanficfare_macmenuhack.txt
FFF: DEBUG: 2025-06-07 12:22:57,873: calibre_plugins.fanficfare_plugin.fff_plugin(218): Plugin FanFicFare macmenuhack file_path:C:\Users\bryan\AppData\Roaming\calibre\plugins\fanficfare_macmenuhack.txt
calibre Debug log
calibre 8.4  embedded-python: True
Windows-10-10.0.26100-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.26100')
Python 3.11.12
Windows: ('10', '10.0.26100', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre2\calibre-debug.exe
Integration status: True
Initialized urlfixer
Successfully initialized third party plugins: DeACSM (0, 0, 16) && DeDRM (7, 2, 1) && ACE (1, 1, 6) && Amazon.com Multiple Countries (1, 0, 0) && Baen (1, 1, 0) && Barnes & Noble (1, 5, 7) && CalibreSpy (1, 0, 92) && Clean Metadata (0, 0, 6) && Comicvine (0, 14, 2) && Consolidate All Library Metadata (2, 0, 46) && Count Pages (1, 13, 6) && Embed Comic Metadata (1, 6, 6) && EpubCheck (0, 2, 6) && EpubMerge (3, 2, 0) && EpubSplit (3, 11, 0) && Extract ISBN (1, 6, 5) && FanFicFare (4, 46, 0) && Fantastic Fiction (1, 7, 2) && Fantastic Fiction Adults (1, 3, 1) && Favourites Menu (1, 3, 1) && FictionDB (1, 4, 2) && Find Duplicates (1, 10, 9) && Generate Cover (2, 3, 5) && Goodreads (1, 8, 3) && Goodreads More Tags (1, 2, 1, 2) && ISFDB3 (1, 4, 2) && Job Spy (1, 0, 239) && Kindle hi-res covers (0, 5, 1) && KindleUnpack - The Plugin (0, 83, 7) && Kobo Metadata (1, 11, 0) && Last Modified (0, 8, 5) && Manage Series (1, 5, 2) && Modify ePub (1, 8, 6) && MultiColumnSearch (1, 0, 96) && Open With (1, 8, 4) && Overdrive Link (2, 58, 0) && Prettify Cover (1, 6, 0) && Quality Check (1, 14, 2) && Quick Preferences (1, 7, 1) && Reading List (1, 15, 4) && Resize Cover (1, 2, 1) && Search The Internet (1, 11, 3) && Skoob Books (1, 5, 5) && Smashwords Metadata (1, 0, 2) && Standard Ebooks (1, 0, 0) && User Category (1, 6, 1) && View Manager (1, 10, 3) && Walk Search History (1, 5, 4) && Wikidata (2, 0, 0) && Wikidata gui (2, 0, 0) && ePub Extended Metadata {Writer} (0, 11, 4) && ePub Extended Metadata {Reader} (0, 11, 4) && ePub Extended Metadata (0, 11, 4)
calibre 8.4  embedded-python: True
Windows-10-10.0.26100-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.26100')
Python 3.11.12
Windows: ('10', '10.0.26100', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre2\calibre-debug.exe
Successfully initialized third party plugins: DeACSM (0, 0, 16) && DeDRM (7, 2, 1) && ACE (1, 1, 6) && Amazon.com Multiple Countries (1, 0, 0) && Baen (1, 1, 0) && Barnes & Noble (1, 5, 7) && CalibreSpy (1, 0, 92) && Clean Metadata (0, 0, 6) && Comicvine (0, 14, 2) && Consolidate All Library Metadata (2, 0, 46) && Count Pages (1, 13, 6) && Embed Comic Metadata (1, 6, 6) && EpubCheck (0, 2, 6) && EpubMerge (3, 2, 0) && EpubSplit (3, 11, 0) && Extract ISBN (1, 6, 5) && FanFicFare (4, 46, 0) && Fantastic Fiction (1, 7, 2) && Fantastic Fiction Adults (1, 3, 1) && Favourites Menu (1, 3, 1) && FictionDB (1, 4, 2) && Find Duplicates (1, 10, 9) && Generate Cover (2, 3, 5) && Goodreads (1, 8, 3) && Goodreads More Tags (1, 2, 1, 2) && ISFDB3 (1, 4, 2) && Job Spy (1, 0, 239) && Kindle hi-res covers (0, 5, 1) && KindleUnpack - The Plugin (0, 83, 7) && Kobo Metadata (1, 11, 0) && Last Modified (0, 8, 5) && Manage Series (1, 5, 2) && Modify ePub (1, 8, 6) && MultiColumnSearch (1, 0, 96) && Open With (1, 8, 4) && Overdrive Link (2, 58, 0) && Prettify Cover (1, 6, 0) && Quality Check (1, 14, 2) && Quick Preferences (1, 7, 1) && Reading List (1, 15, 4) && Resize Cover (1, 2, 1) && Search The Internet (1, 11, 3) && Skoob Books (1, 5, 5) && Smashwords Metadata (1, 0, 2) && Standard Ebooks (1, 0, 0) && User Category (1, 6, 1) && View Manager (1, 10, 3) && Walk Search History (1, 5, 4) && Wikidata (2, 0, 0) && Wikidata gui (2, 0, 0) && ePub Extended Metadata {Writer} (0, 11, 4) && ePub Extended Metadata {Reader} (0, 11, 4) && ePub Extended Metadata (0, 11, 4)
QPA platform: windows
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 81.55183946488295 x 81.64285714285714
[0.00] Starting up...
[0.00] Showing splash screen...
[1.17] splash screen shown
[1.17] Initializing db...
[1.53] db initialized
[1.53] Constructing main UI...
Initializing wikidata gui preferences
[5.26] GUI main window shown
CalibreSpy: initialization_complete
CALM: initialization_complete............
Job Spy has begun initialization...
Calibre, and hence Job Spy, was gracefully shut down last time?  True
Last time daemon started:  never
Last time daemon failed:  never
Total daemon starts inception_to_date:  0
Total daemon failures inception-to-date:  0
JS: ===>>> change_gui_alternating_row_colors()
Job Spy has finished initialization...
[0] un-optimized number of undesirable characters accumulated:  756
[1] optimized number of undesirable characters accumulated:  596
MCS has finished initialization...
Key 'Alt+Left' for shortcut Goto previous is already used by Previous result, ignoring
Key 'Alt+Right' for shortcut Goto next is already used by Next result, ignoring
Key 'Alt+Left' for shortcut Goto previous is already used by Previous result, ignoring
Key 'Alt+Right' for shortcut Goto next is already used by Next result, ignoring
[5.69] main UI initialized...
[5.69] Hiding splash screen
[5.69] splash screen hidden
[5.69] Started up in 5.69 seconds with 3710 books
FFF: DEBUG: 2025-06-07 12:23:19,058: configurable.py(639): config site:test1.com
FFF: DEBUG: 2025-06-07 12:23:24,043: calibre_plugins.fanficfare_plugin.fff_plugin(1163): FanFicFare v4.46.0
FFF: INFO: 2025-06-07 12:23:24,239: calibre_plugins.fanficfare_plugin.prefs(220): Using default settings
FFF: DEBUG: 2025-06-07 12:23:24,240: configurable.py(639): config site:archiveofourown.org
FFF: DEBUG: 2025-06-07 12:23:24,300: configurable.py(1106): use_browser_cache:
FFF: DEBUG: 2025-06-07 12:23:24,300: configurable.py(1127): use_basic_cache:true
FFF: INFO: 2025-06-07 12:23:24,300: base_otw_adapter.py(149): url: https://archiveofourown.org/works/55867468/navigate?view_adult=true
FFF: INFO: 2025-06-07 12:23:24,300: base_otw_adapter.py(150): metaurl: https://archiveofourown.org/works/55867468?view_adult=true
FFF: DEBUG: 2025-06-07 12:23:24,301: cache_basic.py(116): 
========== MISS (GET) BasicCache
https://archiveofourown.gay/works/55867468/navigate?view_adult=true
FFF: DEBUG: 2025-06-07 12:23:24,301: fetcher_requests.py(114): 
---------- REQ (GET) RequestsFetcher
https://archiveofourown.gay/works/55867468/navigate?view_adult=true
FFF: DEBUG: 2025-06-07 12:23:24,644: fetcher_requests.py(127): response code:200
FFF: DEBUG: 2025-06-07 12:23:24,644: decorators.py(112): fromcache:False
FFF: DEBUG: 2025-06-07 12:23:24,645: decorators.py(123): random sleep(2.00-6.00):3.98
FFF: DEBUG: 2025-06-07 12:23:28,624: requestable.py(55): Encoding:utf8
FFF: DEBUG: 2025-06-07 12:23:28,624: cache_basic.py(116): 
========== MISS (GET) BasicCache
https://archiveofourown.gay/works/55867468?view_adult=true
FFF: DEBUG: 2025-06-07 12:23:28,625: fetcher_requests.py(114): 
---------- REQ (GET) RequestsFetcher
https://archiveofourown.gay/works/55867468?view_adult=true
FFF: DEBUG: 2025-06-07 12:23:28,916: fetcher_requests.py(127): response code:200
FFF: DEBUG: 2025-06-07 12:23:28,916: decorators.py(112): fromcache:False
FFF: DEBUG: 2025-06-07 12:23:28,916: decorators.py(123): random sleep(2.00-6.00):4.82
FFF: DEBUG: 2025-06-07 12:23:33,734: requestable.py(55): Encoding:utf8
FFF: WARNING: 2025-06-07 12:23:33,734: base_otw_adapter.py(107): Not doing OTW(AO3) login -- doesn't work with open_pages_in_browser
FFF: DEBUG: 2025-06-07 12:23:33,735: cache_basic.py(116): 
========== MISS (GET) BasicCache
https://archiveofourown.gay/works/55867468/navigate?view_adult=true
FFF: DEBUG: 2025-06-07 12:23:33,735: fetcher_requests.py(114): 
---------- REQ (GET) RequestsFetcher
https://archiveofourown.gay/works/55867468/navigate?view_adult=true
FFF: DEBUG: 2025-06-07 12:23:34,042: fetcher_requests.py(127): response code:200
FFF: DEBUG: 2025-06-07 12:23:34,042: decorators.py(112): fromcache:False
FFF: DEBUG: 2025-06-07 12:23:34,042: decorators.py(123): random sleep(2.00-6.00):5.13
FFF: DEBUG: 2025-06-07 12:23:39,169: requestable.py(55): Encoding:utf8
FFF: DEBUG: 2025-06-07 12:23:39,169: cache_basic.py(116): 
========== MISS (GET) BasicCache
https://archiveofourown.gay/works/55867468?view_adult=true
FFF: DEBUG: 2025-06-07 12:23:39,170: fetcher_requests.py(114): 
---------- REQ (GET) RequestsFetcher
https://archiveofourown.gay/works/55867468?view_adult=true
FFF: DEBUG: 2025-06-07 12:23:39,399: fetcher_requests.py(127): response code:200
FFF: DEBUG: 2025-06-07 12:23:39,400: decorators.py(112): fromcache:False
FFF: DEBUG: 2025-06-07 12:23:39,400: decorators.py(123): random sleep(2.00-6.00):4.99
FFF: DEBUG: 2025-06-07 12:23:44,392: requestable.py(55): Encoding:utf8
FFF: ERROR: 2025-06-07 12:23:44,449: calibre_plugins.fanficfare_plugin.dialogs(791): Exception: {'title': 'Ravens and Wolves', 'author_sort': 'Amberpoconuts', 'author': ['Amberpoconuts'], 'comments': '', 'good': False, 'status': 'Error', 'showerror': True, 'calibre_id': 9032, 'begin': None, 'end': None, 'comment': "'NoneType' object has no attribute 'get_text'", 'url': 'https://archiveofourown.org/works/55867468', 'site': 'archiveofourown.org', 'series': '', 'added': False, 'pubdate': None, 'publisher': 'archiveofourown.org', 'listorder': 0, 'collision': 'Overwrite if Newer'}:'NoneType' object has no attribute 'get_text'
Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.dialogs", line 778, in do_loop
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1376, in prep_download_loop
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1249, in get_story_metadata_only
  File "C:\Users\bryan\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\base_adapter.py", line 343, in getStoryMetadataOnly
  File "C:\Users\bryan\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\base_adapter.py", line 447, in doExtractChapterUrlsAndMetadata
  File "C:\Users\bryan\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\base_otw_adapter.py", line 209, in extractChapterUrlsAndMetadata
  File "C:\Users\bryan\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\htmlcleanup.py", line 77, in stripHTML
AttributeError: 'NoneType' object has no attribute 'get_text'
FFF: DEBUG: 2025-06-07 12:23:59,523: calibre_plugins.fanficfare_plugin.fff_plugin(2285): update_error_column_loop bad Ravens and Wolves https://archiveofourown.org/works/55867468 'NoneType' object has no attribute 'get_text'
FFF: DEBUG: 2025-06-07 12:24:00,142: calibre_plugins.fanficfare_plugin.fff_plugin(1985): Finished Adding/Updating 0 books.
FFF: DEBUG: 2025-06-07 12:24:00,142: calibre_plugins.fanficfare_plugin.fff_plugin(1991): removed tdir(C:\Users\bryan\AppData\Local\Temp\calibre-jd87lmfg\fanficfare_gj6wp2ge)
[77.33] Shutdown starting...
[79.66] Shutdown complete, quitting...

[JimmXinu]

New Test Version Posted

2025-06-07
- Add use_flaresolverr_session and flaresolverr_session settings for #1211
- Include Accept:image/* header when requesting an image url, thanks bellisk

[JimmXinu]

Originally Posted by Ealdwulf

Debug Log:

  FFF: WARNING: 2025-06-07 12:23:33,734: base_otw_adapter.py(107): Not doing OTW(AO3) login -- doesn't work with open_pages_in_browser

Log indicates you have open_pages_in_browser:true. Go turn off open_pages_in_browser and try it again.

When I added to code to skip login when open_pages_in_browser:true, I should have anticipated users leaving that on and only turning off use_browser_cache.