[GUI Plugin] FanFicFare

[unclebob]

Originally Posted by JimmXinu

403 errors are blocking or login failures. I've done all I can about it.

Percent done of BG processing is by chapters, not stories. I would guess you have some super-long stories towards the end.

BTW, 200 stories at once is about 10x more than I would recommend. Not surprised you're running into blocking.

I don't often run 200 stories, I usually run 10 or so. This is more of a test. Anyway, thank you for your hard work! We all appreciate it!

[JimmXinu]

New Version Released

Version 4.63.0 - 2026-11-02
- Update translations.
- Plugin only: use_camoufox option using Calibre recipes.py Browser. #1444
- Add do_camoufox_warmup optional feature
- Fix continue_on_chapter_error with fff_chapter_data
- adapter_royalroadcom: Take Legacy dateUpdated/datePublished from oldest/newest chapter dates. See also #1445
- Update bundled certifi to 2026.10.23
- adapter_royalroadcom: Fix for dateUpdated in new UI, thanks VictorVow(#1445)
- adapter_royalroadcom: Increase default sleep time.
- Plugin: Show book title in BG job title when only one.
- Add use_series_page_metadata option, defaults false.
- Add QuitDecorator to fetches to allow plugin prog bar quit to work better.
- adapter_storiesonlinenet: Fix for site login change.
- Change how chapter metadata is stored in chapter files
- Fix mark new/unnew works with preserved chapters
- Add first_chapter/last_chapter to valid settings list.
- Fix numChapters for update_preserve_deleted_chapters
- Don't skip equal-count chapter swaps when preserving deleted chapters - #1428, thanks kitoma!
- Fixing Unnew for #1434.
- RoyalRoad: Handle email-click URLs in geturls, not adapter. For reasons. #1429
- adapter_fanficsme: Fix word count for site change. Closes #1433
- Plugin: avoid blocking the event loop #1430, thanks, logologist
- Plugin: Add modal dialog for imap/urls from page/series page fetches to allow safe threading re:#1430
- Plugin: Fix Anthology update for ad-hoc(no series URL) anthologies(broken by ini_snippets)
- adapter_sofurrycom: Fix for login, closes #1426, thanks dbhmw
- adapter_royalroadcom: isUnlocked flag isn't accurate, but null or empty subscriptionTiers is?
- adapter_royalroadcom: Fix for older py3 versions, reported in #1428

[culytera]

The AO3 downtime got me redoing my titlepage layout again . It was fun figuring out how to clean up stuff using Action Chains and Editor Chains so I didn't have to use media queries as a workaround for how I make the covers and the actual titlepage. Wish I could recreate the EPUBs with the current data so I could test it on actual fics now.

[JimmXinu]

Updates made to wiki pages, if anyone is interested:

https://github.com/JimmXinu/FanFicFa...i/CustomOutput
https://github.com/JimmXinu/FanFicFa.../ProxyFeatures
https://github.com/JimmXinu/FanFicFa...oblematicSites

[cranetoad]

Just to be clear, definitely not asking for the plugin author to wonder the weirdness of nixos for me, but is there any user here who has gotten Calibre's camoufox to work on nixos? Mine complains that there is no browserforge, which is probably some nix store weirdness, but wanted to see if anyone has resolved this already.

[serpi]

Decided to clean up my ini with AO3 being down and can anyone help me with the generate cover section of the code.

I have three different covers I use for fic, a completed fic, one in progress, and one connected to a series

Do these templates look right to achieve this, I'm not too sure what to put for the series one.

Also is there anyway for me to test if this work with AO3 being down?

${status} => Completed => completedcover
${status} => In-Progress => inprogresscover
${series} => (unsure) => seriescover

Any help would be much appreciated

[Pavan7]

Hey guys, coming back to FFF after a gap of 4-5 months.
Found out its not working on Ao3 (apart from the downtime), or Spacebattles anymore either. The Browser cache is also failing for FFN.

Can someone please help with the workaround and how to update my fics now

[Serkadion]

New poster here, made this just to give additional information on my RR experience. (Tried earlier, but it posted empty, wouldn't let me edit, and was quite rightly removed)

Linux Mint (current), Calibre 9.17, FFF Plugin 4.63, Camoufox installed.

Options enabled -
[www.royalroad.com]
username:<redacted>
password:<redacted>
use_browser_cache:false
use_camoufox:true
use_camoufox:directimages

Issues:

Initial use was promising. There was a long delay for the first download of camoufox, and then the story finally updated with 4 chapters (haven't verified yet, but that looked right).

Tried to update another story. Constant requests for username/password, which then errored out.

Reset calibre, then updated again - worked.

Reset again, then told it to download 8 stories, completely, not bothering to attempt to do updates. (With RR, I end up with multiple versions as authors go on a stubbing fetish. I merge when complete, if they ever do)

Here's the problem. Downloading 1 story, after 54 minutes, was only at 19%. I left the laptop to stay running and finish, and apparently the power save mode kicked in even with Calibre running actively (I'll just disable power save next time). So, when I brought it back up, it continued to go, but at this point it's somewhere on the order of three hours of downloading, and it's done 6 out of 8. Now, there was one very large story in there, but there still seems to be something sticking partially.

I am seeing a random sleep function (2.50 to 7.50 seconds), but can't seem to adjust that; even at 7 seconds per chapter, that should still be able to do 1,000 chapters in two hours. So, it's interesting.

Even better is the weirdness about having to close and open calibre, but oh well.

Suggestion:

Could pulling images be turned off? While nice to have, they're not absolutely essential to the story, and I don't recall seeing an option.

Thanks for the efforts!

[JimmXinu]

Originally Posted by serpi

I have three different covers I use for fic, a completed fic, one in progress, and one connected to a series

Do these templates look right to achieve this, I'm not too sure what to put for the series one.

For anthology downloads use:

generate_cover_settings:
 ${anthology}=>true=>Anthology Cover

See https://github.com/JimmXinu/FanFicFa...thologies-only

Originally Posted by Pavan7

Found out its not working on Ao3 (apart from the downtime), or Spacebattles anymore either. The Browser cache is also failing for FFN.

My advise is on https://github.com/JimmXinu/FanFicFa...oblematicSites

I'm not having problems with SB, and use_camoufox:true is working for me on ffnet.

Originally Posted by Serkadion

Suggestion:

Could pulling images be turned off? While nice to have, they're not absolutely essential to the story, and I don't recall seeing an option.

You would be surprised how many people disagree. I know I was.

Use setting:

[epub]
include_images:false

#Or just for RR
[www.royalroad.com:epub]
include_images:false

FYI, Serkadion, your first 10 or so posts on MR require moderator approval before they appear. Give it half a day before posting again.

[culytera]

Is it possible to add chapter data (number, title, date, dummy text) to custom test stories?

[cranetoad]

Just in case someone looks here, this is what I did in Nixos:

A system installation of camoufox and injecting browserforge to Calibre PYTHON_PATH does the trick, and is easily achievable by using an overlay. For Camoufox, I used the flake from here and installed system-wide: https://github.com/maximoffua/camoufox-nix. Just pick and choose what you need from there.

Then I created the overlay:

Code:

(final: prev: {
calibre = prev.calibre.overrideAttrs (old:
{
buildInputs = old.buildInputs ++ [final.camoufox final.python314Packages.browserforge];

postPatch =
(old.postPatch or "")
+ ''
unwrapped=$(grep -o '/nix/store/[^/"]*camoufox-unwrapped[^/"]*' ${final.camoufox}/bin/camoufox | head -n1)
test -n "$unwrapped" || { echo "could not find unwrapped camoufox"; exit 1; }

substituteInPlace setup/resources.py \
--replace-fail "if not opts.system_camoufox:" "if False:" \
--replace-fail "parts = opts.system_camoufox.split(os.pathsep)" "parts = '$unwrapped/bin/camoufox:$unwrapped/lib/camoufox'.split(os.pathsep)"
'';
preFixup =
''
qtWrapperArgs+=(
--set-default FONTCONFIG_FILE ${final.camoufox}/bin/fonts.conf
--prefix PYTHONPATH : ${final.python314Packages.makePythonPath [final.python314Packages.browserforge]}
--prefix LD_LIBRARY_PATH : ${final.lib.makeLibraryPath [
final.gtk3
final.glib
final.dbus-glib
final.libGL
final.libxkbcommon
final.alsa-lib
final.libpulseaudio
final.pciutils
final.xorg.libX11
final.xorg.libXtst
final.xorg.libXrandr
final.xorg.libXcomposite
final.xorg.libXdamage
final.xorg.libXfixes
final.xorg.libXcursor
final.xorg.libXi
final.xorg.libxcb
final.wayland
]}
)
''
+ old.preFixup;
});
})

I'm not entirely sure if Calibre needs camoufox and browserforge as buildInputs, but I started by adding them, and never got to testing without, they are harmless if not used.

The hacky grep-based substitution is because I had problems with python whitespace inside nix and my formatter destroying the whitespace, so instead of elegant, you get hacky. It works ��.

qtWrapper adds the run-time dependencies to calibre. e.g. browserforge and camoufox fonts, and also the libraries that camoufox might need. The libraries probably could be whittled down, if you are inclined to test.

If someone has done something prettier, feel free to share!

 --set-default FONTCONFIG_FILE ${final.camoufox}/bin/fonts.conf

is actually unnecessary, and messes up calibre fonts for no gain. I'm not sure whether omitting camoufox fonts from the environment causes problems with fingerprinting resistance, but my eyes are not bleeding anymore, so I take what I can get 😅

[Pavan7]

Setting up camoufox now. Could somebody help if my .ini is correct or if I need to edit anything?

[www.fanfiction.net]
use_camoufox:true
use_flaresolverr_proxy:false
use_browser_cache:false
use_browser_cache_only:false
open_pages_in_browser:true
meta_from_last_chapter:true
try_shortened_title_urls:true

[forum.questionablequesting.com]
is_adult:true
username:<redacted>
password:<redacted>
slow_down_sleep_time:60
[forums.sufficientvelocity.com]
slow_down_sleep_time:60
[forums.spacebattles.com]
slow_down_sleep_time:60

[archiveofourown.org]
is_adult:true
username:<redacted>
password:<redacted>
always_login:true
slow_down_sleep_time:20
use_camoufox:true

[www.archiveofourown.org]
is_adult:true
username:<redacted>
password:<redacted>
##use_archive_transformativeworks_org:true
##use_archiveofourown_gay:true
always_login:true
slow_down_sleep_time:20
use_camoufox:true

[www.royalroad.com]
use_camoufox:true

[www.hentai-foundry.com]
use_camoufox:true

[JimmXinu]

Originally Posted by culytera

Is it possible to add chapter data (number, title, date, dummy text) to custom test stories?

Somewhat. I assume you're talking about [teststory:1010] type test stories.

You can give chapter titles using adapter_test1's usual chapter text using chaptertitles.

Example from defaults.ini:

[teststory:1000]
## Add additional sections with different numbers to get different
## parameters for different story urls.
## test1.com?sid=1000
# note the leading commas when doing add_to_ with valid_entries and *_list
add_to_valid_entries:,favs
title:Testing New Feature {{storyId}}
author_list:Bob Smith
authorId_list:45
authorUrl_list:http://test1.com?authid=45
datePublished:2013-03-15
dateUpdated:2013-04-15
numWords:1456
favs:56
series:The Great Test [4]
seriesUrl:http://test1.com?seriesid=1
rating:Tweenie
category_list:Harry Potter,Furbie,Crossover,Puella Magi Madoka Magica/魔法少女まどか★マギカ,Magical Girl Lyrical Nanoha
genre_list:Fantasy,Comedy,Sci-Fi,Noir
warnings_list:Swearing,Violence
characters_list:Bob Smith,George Johnson,Fred Smythe

chaptertitles:Prologue,Chapter 1\, Xenos on Cinnabar,Chapter 2\, Sinmay on Kintikin,3. Chapter 3

There's also chapter_urls you can use instead of chaptertitles to give both a chapter URL and chapter title. Example:

chapter_urls:
 http://test1.com?sid=1003&chapterid=asdf1,1 - Prologue
 http://test1.com?sid=1003&chapterid=asdf2,2 - Chapter 1 Xenos on Cinnabar
 http://test1.com?sid=1003&chapterid=asdf3,3 - Chapter 2 Sinmays 'Kintikin'  "Whoa"
 http://test1.com?sid=1003&chapterid=asdf4,4 - Chapter 3

If the URL given is in test1.com, adapter_test1's usual chapter text is used, but you can make the URL anything you want. Chapter metadata timestamp is given, and the usual index, but not date.

If you what to get really fancy, you give real URLs (or file:// URLs) and adapter_test1 will attempt to use them.

Here's an example I was testing with recently:

[teststory:1003]
#cover_url:http://localhost:8000/cover.jpeg
chapter_urls:
 http://localhost:8000/storyredir.html,Fake Story
 file:///C:/Users/youruser/Desktop/FanFicFare/t/story.html,Fake File Story

Originally Posted by Pavan7

Setting up camoufox now. Could somebody help if my .ini is correct or if I need to edit anything?

We're more about fixing problem than prechecking around here. But I'll point out what I see immediately.

First, wrap ini sections in [code] tags when posting here for best results.

Your slow_down_sleep_time settings are quite high. Especially for the forum sites that can easily have a dozen network requests just getting all the threadmarks for chapter list. I'm fine with 4 on those sites.

You do not need both [archiveofourown.org] and [www.archiveofourown.org] See [www.domain] Sections.

I don't know if use_camoufox will work with hentai-foundry.com. It's not a site I read or have tested recently. My notes don't show needing anything special last time I did test it.

[Pavan7]

Originally Posted by JimmXinu

We're more about fixing problem than prechecking around here. But I'll point out what I see immediately.

First, wrap ini sections in [code] tags when posting here for best results.

Your slow_down_sleep_time settings are quite high. Especially for the forum sites that can easily have a dozen network requests just getting all the threadmarks for chapter list. I'm fine with 4 on those sites.

You do not need both [archiveofourown.org] and [www.archiveofourown.org] See [www.domain] Sections.

I don't know if use_camoufox will work with hentai-foundry.com. It's not a site I read or have tested recently. My notes don't show needing anything special last time I did test it.

Thanks for the advice. I tried it and it seems to be working.
One issue I ran across though was Email Update. It threw an error after 5-6 hours. I generated a new app password and tried again but that also threw an error after 3-4 hours. It's a gmail account, 2FA enabled but I've been using app passwords for around 2 years on that.
The 1st error I got was

calibre, version 9.17.0
ERROR: Unhandled exception: <b>abort</b>:command: UID =&gt; socket error: EOF

calibre 9.17  embedded-python: True
Windows-11-10.0.26200-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '11', '10.0.26200')
Python 3.14.8
Windows: ('11', '10.0.26200', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre2\calibre.exe
Successfully initialized third party plugins: FanFicFare (4, 63, 0) && Find Duplicates (1, 10, 11)
Traceback (most recent call last):
  File "imaplib.py", line 1233, in _command_complete
  File "imaplib.py", line 1399, in _get_tagged_response
  File "imaplib.py", line 1289, in _get_response
  File "imaplib.py", line 1411, in _get_line
imaplib.IMAP4.abort: socket error: EOF

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.dialogs", line 990, in do_it
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 3388, in run_off_gui_thread
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 3374, in target
  File "C:\Users\redacted\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\geturls.py", line 285, in get_urls_from_imap
  File "imaplib.py", line 1052, in uid
  File "imaplib.py", line 1496, in _simple_command
  File "imaplib.py", line 1235, in _command_complete
imaplib.IMAP4.abort: command: UID => socket error: EOF

I then changed the app password but got a similar (but slightly different) error

calibre, version 9.17.0
ERROR: Unhandled exception: <b>abort</b>:command: UID =&gt; System Error

calibre 9.17  embedded-python: True
Windows-11-10.0.26200-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '11', '10.0.26200')
Python 3.14.8
Windows: ('11', '10.0.26200', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre2\calibre.exe
Successfully initialized third party plugins: FanFicFare (4, 63, 0) && Find Duplicates (1, 10, 11)
Traceback (most recent call last):
  File "imaplib.py", line 1233, in _command_complete
  File "imaplib.py", line 1391, in _get_tagged_response
  File "imaplib.py", line 1141, in _check_bye
imaplib.IMAP4.abort: System Error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.dialogs", line 990, in do_it
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 3388, in run_off_gui_thread
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 3374, in target
  File "C:\Users\redacted\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\geturls.py", line 285, in get_urls_from_imap
  File "imaplib.py", line 1052, in uid
  File "imaplib.py", line 1496, in _simple_command
  File "imaplib.py", line 1235, in _command_complete
imaplib.IMAP4.abort: command: UID => System Error

[JimmXinu]

Originally Posted by Pavan7

One issue I ran across though was Email Update. It threw an error after 5-6 hours. I generated a new app password and tried again but that also threw an error after 3-4 hours. It's a gmail account, 2FA enabled but I've been using app passwords for around 2 years on that.
The 1st error I got was
...

What do you mean by "error after x hours"? How often were you hitting it?

The expected use pattern is watching for notification emails in a regular email client, then hitting FFF's 'get from email'.

If you are instead polling frequently in FFF, I expect you're getting blocked by google.

Another possibility is that your public facing IP is on a block list. Changing it or using a VPN might help.