[GUI Plugin] FanFicFare

[Clem2605]

Hello,

I've got a problem with the Browser Cache feature.

I've just changed my computer, importing my Calibre Data from the old one via Calibre's import/export tool. I don't know if it's relevant, but I also updated from windows 10 to 11.

Everything seems to work, but when I try to update my works with the Browser Cache, it looks like FFF doesn't find the cached files. It tries to open the stories in my browser two times and after some time, it crashes, telling me it can't find any of the cached files.
The thing is, the pages are opening fine in my browser, no cloudfare, nothing.

My architecture shouldn't have changed, but I've still tried to re-copy/paste the path to the cache in my personal.ini, without any change.

Another thing I've noticed is that, even if I kill the job, the pages keep opening in my browser for 5-10mins, don't know if it's normal behavior.

When I try to download the normal way, everything works as expected.

Here is the job's details, I tried a ff at random and just realized it was a NSFW one, so the link is censored.

Download 1 FanFiction Book(s)
FFF: INFO: 2025-03-17 01:26:17,906: calibre_plugins.fanficfare_plugin.jobs(48): CPUs:5
FFF: INFO: 2025-03-17 01:26:17,906: calibre_plugins.fanficfare_plugin.jobs(51): FanFicFare v4.43.0
calibre 7.26  embedded-python: True
Windows-10-10.0.22631-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.22631')
Python 3.11.5
Windows: ('10', '10.0.22631', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre2\calibre-parallel.exe
Successfully initialized third party plugins: Action Chains (1, 20, 9) && Count Pages (1, 13, 6) && EpubMerge (3, 2, 0) && FanFicFare (4, 43, 0) && Generate Cover (2, 3, 5) && Kobo Utilities (2, 19, 0) && KoboTouchExtended (3, 7, 3) && Manage Series (1, 5, 2) && Reading Goal (1, 9, 0) && Reading List (1, 15, 4) && View Manager (1, 10, 3)
FFF: INFO: 2025-03-17 01:26:17,955: calibre_plugins.fanficfare_plugin.jobs(72): Launch background process for site archiveofourown.org:
https://archiveofourown.org/works/
FFF: INFO: 2025-03-17 01:27:36,630: calibre_plugins.fanficfare_plugin.jobs(134): 

================================================================================ site:(archiveofourown.org)
FFF: INFO: 2025-03-17 01:26:19,205: calibre_plugins.fanficfare_plugin.jobs(215): 

-------------------------------------------------------------------------------- https://archiveofourown.org/works/
FFF: DEBUG: 2025-03-17 01:26:19,205: configurable.py(620): config site:archiveofourown.org
FFF: DEBUG: 2025-03-17 01:26:19,329: configurable.py(1087): use_browser_cache:true
FFF: DEBUG: 2025-03-17 01:26:19,330: configurable.py(1089): browser_cache_path:C:\Users\cleme\AppData\Local\Google\Chrome\User Data\Default\Cache\Cache_Data
FFF: DEBUG: 2025-03-17 01:26:19,332: browsercache_blockfile.py(53): Using BlockfileCache
FFF: DEBUG: 2025-03-17 01:26:19,332: configurable.py(1108): use_basic_cache:true
FFF: INFO: 2025-03-17 01:26:19,333: base_otw_adapter.py(142): url: https://archiveofourown.org/works//navigate?view_adult=true
FFF: INFO: 2025-03-17 01:26:19,333: base_otw_adapter.py(143): metaurl: https://archiveofourown.org/works/?view_adult=true
FFF: DEBUG: 2025-03-17 01:26:19,333: cache_basic.py(116): 
========== MISS (GET) BasicCache
https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:26:19,333: base_browsercache.py(98): Cache Key:1/0/_dk_https://archiveofourown.org https://archiveofourown.org https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:26:19,334: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:26:19,334: base_browsercache.py(98): Cache Key:1/0/_dk_chrome-extension://akiljllkbielkidmammnifcnibaigelm chrome-extension://akiljllkbielkidmammnifcnibaigelm https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:26:19,334: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:26:19,335: cache_browser.py(72): 

open page in browser: https://archiveofourown.org/works//navigate?view_adult=true
tries:None

FFF: DEBUG: 2025-03-17 01:26:21,544: cache_browser.py(82): Checking for cache...
FFF: DEBUG: 2025-03-17 01:26:21,544: base_browsercache.py(98): Cache Key:1/0/_dk_https://archiveofourown.org https://archiveofourown.org https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:26:21,545: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:26:21,545: base_browsercache.py(98): Cache Key:1/0/_dk_chrome-extension://akiljllkbielkidmammnifcnibaigelm chrome-extension://akiljllkbielkidmammnifcnibaigelm https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:26:21,546: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:26:23,547: cache_browser.py(82): Checking for cache...
FFF: DEBUG: 2025-03-17 01:26:23,548: base_browsercache.py(98): Cache Key:1/0/_dk_https://archiveofourown.org https://archiveofourown.org https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:26:23,626: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:26:23,626: base_browsercache.py(98): Cache Key:1/0/_dk_chrome-extension://akiljllkbielkidmammnifcnibaigelm chrome-extension://akiljllkbielkidmammnifcnibaigelm https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:26:23,627: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:26:27,756: cache_browser.py(82): Checking for cache...
FFF: DEBUG: 2025-03-17 01:26:27,756: base_browsercache.py(98): Cache Key:1/0/_dk_https://archiveofourown.org https://archiveofourown.org https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:26:27,891: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:26:27,891: base_browsercache.py(98): Cache Key:1/0/_dk_chrome-extension://akiljllkbielkidmammnifcnibaigelm chrome-extension://akiljllkbielkidmammnifcnibaigelm https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:26:27,892: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:26:37,947: cache_browser.py(82): Checking for cache...
FFF: DEBUG: 2025-03-17 01:26:37,948: base_browsercache.py(98): Cache Key:1/0/_dk_https://archiveofourown.org https://archiveofourown.org https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:26:38,130: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:26:38,130: base_browsercache.py(98): Cache Key:1/0/_dk_chrome-extension://akiljllkbielkidmammnifcnibaigelm chrome-extension://akiljllkbielkidmammnifcnibaigelm https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:26:38,131: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:26:58,131: cache_browser.py(82): Checking for cache...
FFF: DEBUG: 2025-03-17 01:26:58,131: base_browsercache.py(98): Cache Key:1/0/_dk_https://archiveofourown.org https://archiveofourown.org https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:26:58,144: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:26:58,144: base_browsercache.py(98): Cache Key:1/0/_dk_chrome-extension://akiljllkbielkidmammnifcnibaigelm chrome-extension://akiljllkbielkidmammnifcnibaigelm https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:26:58,145: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:26:58,154: cache_browser.py(72): 

open page in browser: https://archiveofourown.org/works//navigate?view_adult=true
tries:1

FFF: DEBUG: 2025-03-17 01:27:00,226: cache_browser.py(82): Checking for cache...
FFF: DEBUG: 2025-03-17 01:27:00,226: base_browsercache.py(98): Cache Key:1/0/_dk_https://archiveofourown.org https://archiveofourown.org https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:27:00,227: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:27:00,227: base_browsercache.py(98): Cache Key:1/0/_dk_chrome-extension://akiljllkbielkidmammnifcnibaigelm chrome-extension://akiljllkbielkidmammnifcnibaigelm https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:27:00,227: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:27:02,230: cache_browser.py(82): Checking for cache...
FFF: DEBUG: 2025-03-17 01:27:02,230: base_browsercache.py(98): Cache Key:1/0/_dk_https://archiveofourown.org https://archiveofourown.org https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:27:02,243: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:27:02,243: base_browsercache.py(98): Cache Key:1/0/_dk_chrome-extension://akiljllkbielkidmammnifcnibaigelm chrome-extension://akiljllkbielkidmammnifcnibaigelm https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:27:02,243: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:27:06,244: cache_browser.py(82): Checking for cache...
FFF: DEBUG: 2025-03-17 01:27:06,244: base_browsercache.py(98): Cache Key:1/0/_dk_https://archiveofourown.org https://archiveofourown.org https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:27:06,245: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:27:06,245: base_browsercache.py(98): Cache Key:1/0/_dk_chrome-extension://akiljllkbielkidmammnifcnibaigelm chrome-extension://akiljllkbielkidmammnifcnibaigelm https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:27:06,246: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:27:16,248: cache_browser.py(82): Checking for cache...
FFF: DEBUG: 2025-03-17 01:27:16,248: base_browsercache.py(98): Cache Key:1/0/_dk_https://archiveofourown.org https://archiveofourown.org https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:27:16,249: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:27:16,249: base_browsercache.py(98): Cache Key:1/0/_dk_chrome-extension://akiljllkbielkidmammnifcnibaigelm chrome-extension://akiljllkbielkidmammnifcnibaigelm https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:27:16,250: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:27:36,251: cache_browser.py(82): Checking for cache...
FFF: DEBUG: 2025-03-17 01:27:36,251: base_browsercache.py(98): Cache Key:1/0/_dk_https://archiveofourown.org https://archiveofourown.org https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:27:36,252: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:27:36,252: base_browsercache.py(98): Cache Key:1/0/_dk_chrome-extension://akiljllkbielkidmammnifcnibaigelm chrome-extension://akiljllkbielkidmammnifcnibaigelm https://archiveofourown.org/works//navigate?view_adult=true
FFF: DEBUG: 2025-03-17 01:27:36,252: browsercache_blockfile.py(111): []
FFF: DEBUG: 2025-03-17 01:27:36,255: cache_browser.py(101): 
========== MISS (GET) BrowserCache
https://archiveofourown.org/works//navigate?view_adult=true
FFF: INFO: 2025-03-17 01:27:36,257: calibre_plugins.fanficfare_plugin.jobs(449): Exception: {'title': 'Magnus Is Quite... Magical', 'author_sort': 'AceOnIce', 'author': ['AceOnIce'], 'comments': '', 'good': False, 'status': 'Error', 'showerror': True, 'calibre_id': 5970, 'begin': None, 'end': None, 'comment': "HTTP Error in FFF 'Page not found or expired in Browser Cache (see FFF setting browser_cache_age_limit)'(428) URL:'https://archiveofourown.org/works//navigate?view_adult=true'", 'url': 'https://archiveofourown.org/works/', 'site': 'archiveofourown.org', 'series': '', 'added': False, 'pubdate': None, 'publisher': 'archiveofourown.org', 'listorder': 0, 'collision': 'Update EPUB if New Chapters', 'uniqueurl': 'https://archiveofourown.org/works/[None-None]', 'is_adult': False, 'username': 'NoneGiven', 'password': '', 'icon': 'dialog_error.png', 'epub_for_update': 'C:\\Users\\cleme\\AppData\\Local\\Temp\\calibre_nt_r423w\\fanficfare_kbk787tz\\old-5970-agydg8dp.epub', 'calibre_columns': {}, 'outfile': 'C:\\Users\\cleme\\AppData\\Local\\Temp\\calibre_nt_r423w\\fanficfare_kbk787tz\\bgmeta-1oynpx09.epub'}:HTTP Error in FFF 'Page not found or expired in Browser Cache (see FFF setting browser_cache_age_limit)'(428) URL:'https://archiveofourown.org/works//navigate?view_adult=true'
Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.jobs", line 254, in do_download_for_worker
  File "C:\Users\cleme\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\base_adapter.py", line 342, in getStoryMetadataOnly
  File "C:\Users\cleme\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\base_adapter.py", line 446, in doExtractChapterUrlsAndMetadata
  File "C:\Users\cleme\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\base_otw_adapter.py", line 145, in extractChapterUrlsAndMetadata
  File "C:\Users\cleme\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\requestable.py", line 119, in get_request
  File "C:\Users\cleme\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\requestable.py", line 111, in get_request_redirected
  File "C:\Users\cleme\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\fetchers\base_fetcher.py", line 133, in get_request_redirected
  File "C:\Users\cleme\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\fetchers\cache_basic.py", line 122, in fetcher_do_request
  File "C:\Users\cleme\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\fetchers\decorators.py", line 102, in fetcher_do_request
  File "C:\Users\cleme\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\fetchers\cache_browser.py", line 110, in fetcher_do_request
fanficfare.exceptions.HTTPErrorFFF: HTTP Error in FFF 'Page not found or expired in Browser Cache (see FFF setting browser_cache_age_limit)'(428) URL:'https://archiveofourown.org/works//navigate?view_adult=true'

FFF: INFO: 2025-03-17 01:27:36,630: calibre_plugins.fanficfare_plugin.jobs(146): 
Download Results:
Error https://archiveofourown.org/works/ HTTP Error in FFF 'Page not found or expired in Browser Cache (see FFF setting browser_cache_age_limit)'(428) URL:'https://archiveofourown.org/works//navigate?view_adult=true'

FFF: INFO: 2025-03-17 01:27:36,631: calibre_plugins.fanficfare_plugin.jobs(170): 
Error
https://archiveofourown.org/works/

Thanks in advance for any help!

[JimmXinu]

Originally Posted by Clem2605

I've got a problem with the Browser Cache feature.

I've just changed my computer, importing my Calibre Data from the old one via Calibre's import/export tool. I don't know if it's relevant, but I also updated from windows 10 to 11.

Most common reason is browser cache dir has changed. Admittedly, it's less likely with Chrome, since it doesn't include a random string like Firefox.

Or your browser_cache_path setting isn't pointed to the cache dir for your default browser. Like maybe Edge is now your default browser, but it copied your Chrome settings?

I've avoided win11 so far, but also haven't heard anything about it causing FFF problems.

[Clem2605]

Chrome is my default browser, I'm sure, and the pages are opening in Chrome.

When I try to put something else than the path I have, FFF is able to figure out that this isn't a cache directory.
When I go in the directory, I can see that files were created/modified in it, corresponding to when I was last using Chrome.

It's like Chrome isn't caching AO3's site for some reason. Or like FFF isn't able to read them.

[JimmXinu]

Both of those things are possible.

For example, there was an issue with a Firefox 'Security Container' issue that changed how files were cached.

And I have sometimes seen pages marked 'no-cache' by the server for no apparent reason.

[Clem2605]

When I try with ff.net, it doesn't work either, so it seems to be a problem between FFF and the cache files...

[ThreeLizards]

Hi,

So I've got a NSFW fic that I've been trying to download, and I know I'm not supposed to post an NSFW link. I've tried multiple times on multiple days to download this fic.

It's on AO3. It does NOT have the little lock on it. It is orphan'ed.

Could I possibly DM a link?

Am I just SOL?

What should I do?

[ownedbycats]

Save it and try again later (if its orphaned, the author won't be around to suddenly delete it). The issues are expected to be over by April I think.

[ThreeLizards]

Originally Posted by ownedbycats

Save it and try again later (if its orphaned, the author won't be around to suddenly delete it). The issues are expected to be over by April I think.

Can do!

I'll just keep the tab then

[JimmXinu]

You can send me NSFW links in PM. Or give just the story ID number. Until somebody tells me otherwise, I'm prepared to assume that's sufficient obfuscation.

And remember, AO3 has it's own epub download button on each story.

[JimmXinu]

New Test Version Posted

2025-03-18
- ffnet only: try_shortened_title_urls option #1166
- Revert "adapter_fanfictionnet: Attempt chapter from m. (vs www) when chapter not found"

If you set try_shortened_title_urls:true under [fanfiction.net], FFF will, on 'Page not found or expired', shorten the title in the URL and try again.

Only useful with Browser Cache, and while you could do it manually, intended for use with open_pages_in_browser:true

Discussion on github #1166.

No, this isn't useful for AO3, there is no equivalent.

Also removed the similar attempt to get m. (vs www.) on chapter not found--failure page isn't cached anymore so it didn't work.

[Firewolf88]

After dealing with it manually for years, I finally decided to try and use the metadata replace function to help me out with ao3 characters with multiple names. I finally ran into a fandom with too many to want to do manually. I mean the kind of canonized tag thats "Adrien Agreste | Chat Noir"

I did figure out how to do it from the canonized tag, but is there a way I can use to also use each half on its own?

ie right now my line looks like
characters,ships,ships_CHARS=>(?i)Adrien Agreste \| Chat Noir$=>Adrien

is there a way to make it also catch just Adrien Agreste and just Chat Noir? Or do I need to just add more lines/do those ones manually?


(Also just a side note, but I do have to do freeformtags separately and drop the $, as someone who doesn't really understand anything I'm doing here I just find that potentially helpful to mention for anyone else trying to do this in the future.)

[ChaoticAdventure]

Originally Posted by Firewolf88

ie right now my line looks like
characters,ships,ships_CHARS=>(?i)Adrien Agreste \| Chat Noir$=>Adrien

is there a way to make it also catch just Adrien Agreste and just Chat Noir? Or do I need to just add more lines/do those ones manually?

For just a character, you could do this in two lines: one that catches any name with "Adrien Agreste" and one that catches any name with "Chat Noir". Or even in one line, with an "or" function, which is definitely possible but I haven't tried it. With a ship you can't do that, because if you change the whole line it will get rid of the other person in the ship.

For ships, I use three lines, like this:

characters,ships=>Adrien Agreste \| Chat Noir=>Adrien
characters,ships=>Chat Noir=>Adrien
characters,ships=>Adrien Agreste=>Adrien

I don't know if there is a better way to do this, but this is a solution, at the very least.

Also I have no idea what (?!) does, but if you want the function to take the literal, you don't need any extra markings. You can just leave it plain like my example.

[ChaoticAdventure]

Sorry to post twice, but I also came here with a question/issue.

It seems like one of my fics is having this download issue. When I update it, the beginning notes of the first new chapter are put into the body of the text, even though I have them excluded. The fic in question is marked mature, although I don't think the actual story is just yet. But just to be safe, here is the ID instead of a link: 40831656

This is what I did, exactly:

1. The fic had 43 chapters that I had previously downloaded.
2. I updated it, so it added chapters 44 and 45. BOTH chapters have starting notes in Ao3, thanking the reader for reviews.
3. In Calibre, ONLY chapter 44 had the starting notes before the chapter. Chapter 45 did not. It didn't say "notes" before it, either: It was as if the notes were just part of the chapter text. I have since edited it out so I can't take a screenshot, unfortunately.

I have no idea why this could be happening, or even if it's replicable? It's also not a big deal, just very strange. But I'm worried about this happening with other fics and I just don't realize because I'm not reading them right away.

Extra info: I'm not downloading from cache, I never did get that to work, but downloading normally seems to be going okay most of the time.

[JimmXinu]

Multi-reply

Originally Posted by Firewolf88

I did figure out how to do it from the canonized tag, but is there a way I can use to also use each half on its own?

ie right now my line looks like
characters,ships,ships_CHARS=>(?i)Adrien Agreste \| Chat Noir$=>Adrien

is there a way to make it also catch just Adrien Agreste and just Chat Noir? Or do I need to just add more lines/do those ones manually?

It can probably be done it one line, but regex has to be very precise, a single misplaced character can break it. And there can be confusion of characters--regex uses the pipe (|), so to match it as part of the text, it has to be escaped to \|.

I haven't tested it, but something like this maybe?

 characters,ships,ships_CHARS=>(?i)(Adrien Agreste)?( \| )?(Chat Noir)?=>Adrien

If you don't understand regular expressions, it's probably easier to do multiple lines.

FYI, (?i) in python regex just means 'ignore case'.

Originally Posted by ChaoticAdventure

3. In Calibre, ONLY chapter 44 had the starting notes before the chapter. Chapter 45 did not. It didn't say "notes" before it, either: It was as if the notes were just part of the chapter text. I have since edited it out so I can't take a screenshot, unfortunately.

That's literally what it is. If you go look at chapter 44 on AO3, it has "Thank you again for all the reviews!" in both chapter notes AND at the beginning of the chapter text.

[JimmXinu]

New Test Version Posted

2025-03-21
- adapter_literotica: Fixed incorrect parsing for get url from webpage option, thanks dbhmw! #1182 & #1183
- Note on open_pages_in_browser for MacOS users linking to #1142
- adapter_asianfanficscom: Add inject_chapter_image option. Closes #1143