[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by Courge

I gather the "category" metadata in a custom column and I would like to simplify the metadata while avoiding multiple identical values.

For example, some fanfics are tagged as:

• The Silmarillion and other histories of Middle-Earth - J. R. R. Tolkien
• The Lord of the Rings - All Media Types
• The Hobbit - All Media Types
• TOLKIEN J. R. R. - Works & Related Fandoms

I set the personal.ini file to:

add_to_replace_metadata:
 category=>The Silmarillion and other histories of Middle-Earth - J. R. R. Tolkien=>Tolkien
 category=>The Lord of the Rings - All Media Types=>Tolkien
 category=>The Hobbit - All Media Types=>Tolkien
 category=>TOLKIEN J. R. R. - Works \& Related Fandoms=>Tolkien

Which gives me in the "Category" custom column: "Tolkien, Tolkien, Tolkien, Tolkien".

Is it possible to either pick up (and replace) only the first tag in "category" or remove identical replacements?

Both FFF and Calibre should be de-duping (and ordering)--assuming you are using a 'Like Tags' type column.

If you have keep_in_order_category:true that will disable both de-duping and ordering in FFF.

[Courge]

Originally Posted by JimmXinu

If you have keep_in_order_category:true that will disable both de-duping and ordering in FFF.

That was indeed the problem. Thank you!

[leckadams]

Originally Posted by steltek

What browser and OS are you trying to get it working with?

The easiest browser to get it working on is Firefox or one of its variants (like Waterfox). Go to about:cache in the browser URL bar and it will give you a page that contains the exact path to the browser cache that you can copy/paste into personal.ini directly after browser_cache_path: with no spaces in between.

The line you want to copy is the entire path shown under the storage disk location in the disk section.

You my dear sweet person are a freaking GODSEND! Thank you sooo much!

[steltek]

Originally Posted by JimmXinu

Corrupted zip file download perhaps?

If you restart Calibre in debug mode and post the log, it may tell us something.

I've posted the debug log in the code tags below. I tried to re-download and re-install the test plugin, but it still exhibits this behavior (i.e. no FFF button on menu). If I download and install the most recent official version again, the FFF button again returns to the menu after I restart. Strange, as I don't recall ever having this particular problem before.

calibre Debug log
calibre 4.23 [64bit]  embedded-python: True is64bit: True
Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.16+
Windows: ('7', '6.1.7601', 'SP1', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: DeDRM (6, 6, 3) && Obok DeDRM (6, 5, 4) && EpubMerge (2, 11, 0) && FanFicFare (4, 3, 10) && EpubSplit (3, 1, 0) && Libri_hu (1, 0, 6)
calibre 4.23 [64bit]  embedded-python: True is64bit: True
Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.16+
Windows: ('7', '6.1.7601', 'SP1', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: DeDRM (6, 6, 3) && Obok DeDRM (6, 5, 4) && EpubMerge (2, 11, 0) && FanFicFare (4, 3, 10) && EpubSplit (3, 1, 0) && Libri_hu (1, 0, 6)
Turning on automatic hidpi scaling
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 72.035450517 x 72.0567375887
Using calibre Qt style: True
[0.00] Starting up...
[0.00] Showing splash screen...
[0.64] splash screen shown
[0.64] Initializing db...
[0.83] db initialized
[0.83] Constructing main UI...
DEBUG:    0.0 obok::utilities.py - loading translations
DEBUG:    0.0 obok::dialogs.py - loading translations
DEBUG:    0.0 obok::config.py - loading translations
DEBUG:    0.0 obok::action_err.py - loading translations
DEPRECATION: The OpenSSL being used by this python install (OpenSSL 1.0.2s  28 May 2019) does not meet the minimum supported version (>= OpenSSL 1.1.1) in order to support TLS 1.3 required by Cloudflare, You may encounter an unexpected Captcha or cloudflare 1020 blocks.
Traceback (most recent call last):
  File "site-packages\calibre\gui2\ui.py", line 157, in __init__
  File "site-packages\calibre\gui2\ui.py", line 171, in init_iaction
  File "calibre_plugins.fanficfare_plugin.__init__", line 121, in load_actual_plugin
  File "site-packages\calibre\customize\__init__.py", line 613, in load_actual_plugin
  File "importlib\__init__.py", line 37, in import_module
  File "site-packages\calibre\customize\zipplugin.py", line 185, in load_module
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 83, in <module>
  File "C:\Users\EFXRHNSXN\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\__init__.py", line 29, in <module>
  File "C:\Users\EFXRHNSXN\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\configurable.py", line 44, in <module>
  File "C:\Users\EFXRHNSXN\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\nsapa_proxy.py", line 170
SyntaxError: 'continue' not supported inside 'finally' clause
EpubMerge: DEBUG: 2021-07-02 01:08:00,401: calibre_plugins.epubmerge.epubmerge_plugin(158): macmenuhack file_path:C:\Users\EFXRHNSXN\AppData\Roaming\calibre\plugins\fanficfare_macmenuhack.txt
[2.72] main UI initialized...
[2.72] Hiding splash screen
Failed to write backup metadata for id: 469 once
Traceback (most recent call last):
  File "site-packages\calibre\db\backup.py", line 103, in do_one
  File "site-packages\calibre\db\cache.py", line 62, in call_func_with_lock
  File "site-packages\calibre\db\cache.py", line 1209, in write_backup
  File "site-packages\calibre\db\backend.py", line 1662, in write_backup
  File "site-packages\calibre\db\backend.py", line 1653, in write_backup
  File "site-packages\calibre\startup.py", line 169, in local_open
IOError: [Errno 13] Permission denied: u'M:\\Misc Backup\\Calibre\\Salix15\\So Damn Domestic (469)\\metadata.opf'
Failed to write backup metadata for id: 469 again, giving up
Traceback (most recent call last):
  File "site-packages\calibre\db\backup.py", line 109, in do_one
  File "site-packages\calibre\db\cache.py", line 62, in call_func_with_lock
  File "site-packages\calibre\db\cache.py", line 1209, in write_backup
  File "site-packages\calibre\db\backend.py", line 1662, in write_backup
  File "site-packages\calibre\db\backend.py", line 1653, in write_backup
  File "site-packages\calibre\startup.py", line 169, in local_open
IOError: [Errno 13] Permission denied: u'M:\\Misc Backup\\Calibre\\Salix15\\So Damn Domestic (469)\\metadata.opf'
[6.44] splash screen hidden
[6.44] Started up in 6.44 seconds with 2063 books
Failed to write backup metadata for id: 14 once
Traceback (most recent call last):
  File "site-packages\calibre\db\backup.py", line 103, in do_one
  File "site-packages\calibre\db\cache.py", line 62, in call_func_with_lock
  File "site-packages\calibre\db\cache.py", line 1209, in write_backup
  File "site-packages\calibre\db\backend.py", line 1662, in write_backup
  File "site-packages\calibre\db\backend.py", line 1653, in write_backup
  File "site-packages\calibre\startup.py", line 169, in local_open
IOError: [Errno 13] Permission denied: u'M:\\Misc Backup\\Calibre\\Lisette\\Godless Provenance (14)\\metadata.opf'
Traceback (most recent call last):
  File "site-packages\calibre\db\backend.py", line 157, in write_serialized
IOError: [Errno 13] Permission denied: u'M:\\Misc Backup\\Calibre\\metadata_db_prefs_backup.json'

[steltek]

Originally Posted by leckadams

You my dear sweet person are a freaking GODSEND! Thank you sooo much!

Any time, glad to help.

The hard part is always finding the path to the cache file, which is often confusing if you don't normally deal with that kind of stuff. Now that you have one working for Firefox, you can easily find the proper cache path for Chrome and its derivatives (i.e. Edge, Vivaldi, Opera, etc) if you prefer to use a Chromium-based browser. Just keep in mind that only one browser cache can be active at a time in FFF (you can have multiple browser_cache_path statements for different browsers in Personal.ini, but all but the one for the browser you are actually using needs to be commented out with a # in front of that line -- doing this does make it easy to switch between browsers with a quick edit of Personal.ini, though).

If you look at the path statement you found for Firefox, they will be the same for both Firefox-based and Chrome-based browsers in Windows up through the \AppData\Local\ part.

For each of the following browsers, to get its complete cache path simply replace the part of the working path statement directly after \AppData\Local\ with the following:

Google Chrome:
Chrome\User Data\Default\Cache
Microsoft Edge:
Edge\User Data\Default\Cache
Vivaldi:
Vivaldi\User Data\Default\Cache

You see the pattern here for Chromium-based browsers, I'm sure. I don't have Opera installed, but I'm sure it follows the same pattern.

For all Firefox-based browsers, you can just continue to use about:cache in the browser URL bar to directly copy its cache path.

[SilentShadow123]

Is there a way I can update a fanfiction regardless of whether it has fewer or the same number of chapters on the website than in calibre?

For example, one of the fanfics I read had 2 chapters and I added it into calibre with 2 chapters. However, the author removed the 2nd chapter and added a different 2nd chapter, and it said "Already existing" and the update failed. Is there a way I could update it in calibre so that it has 3 chapters in total with both the new one and the old one that was removed? Is Overwrite the only option?

[Philosophize]

Originally Posted by Philosophize

Success! I got it to work in Firefox.

...and then it stopped.

It's now completely failed in both Firefox and Chrome.

I've cleared the cache first and I've not done it. I click through each chapter manually and I've used WebToEpub. Settings are set to always use browser cache. The age limit has been set to 4 and -1. Every time, I get the same error:

HTTP Error in FFF 'Page not found or expired in Browser Cache (see FFF setting browser_cache_age_limit)'(428)

There's no doubt that I have the right folders because I can actually see all the new cache files appear when I load a story up. There's clearly a lot there to dig through, but FFF is not finding the fics, even when a few pages from FF.net are the only pages that have been loaded.

I've made sure I'm using the latest test version of FFF, naturally, but I haven't updated my system or my browsers. I haven't changed anything at all, in fact, so I have to wonder of FF.net has done something with how their pages are cached. How or why else would something that was working fine stop completely a few days later?

Right now I'm thinking that my only option is to manually download everything via WebToEpub, drag it all into Calibre, then manually find and delete the old versions.

[steltek]

Originally Posted by Philosophize

...and then it stopped.

It's now completely failed in both Firefox and Chrome.

I've cleared the cache first and I've not done it. I click through each chapter manually and I've used WebToEpub. Settings are set to always use browser cache. The age limit has been set to 4 and -1. Every time, I get the same error:

HTTP Error in FFF 'Page not found or expired in Browser Cache (see FFF setting browser_cache_age_limit)'(428)

There's no doubt that I have the right folders because I can actually see all the new cache files appear when I load a story up. There's clearly a lot there to dig through, but FFF is not finding the fics, even when a few pages from FF.net are the only pages that have been loaded.

I've made sure I'm using the latest test version of FFF, naturally, but I haven't updated my system or my browsers. I haven't changed anything at all, in fact, so I have to wonder of FF.net has done something with how their pages are cached. How or why else would something that was working fine stop completely a few days later?

Right now I'm thinking that my only option is to manually download everything via WebToEpub, drag it all into Calibre, then manually find and delete the old versions.

Are you getting this error when trying to update a fic soon after receiving an email notification from FFN?

If so, sometimes it can take literally hours to be able to actually update a fic between the time that the email comes and the site servers finally catch up to reflect the existence of the updated new chapter. I have this problem ALL the time and finally had to resort to using an old workaround for it.

You might try to see if adding the following statement in the [overrides] section of Personal.ini under [www.fanfiction.net] helps with your problem:

check_next_chapter:true

Also, you aren't running a tool like CCleaner that is set to automatically clear your browser caches are you?

[Philosophize]

Originally Posted by steltek

Are you getting this error when trying to update a fic soon after receiving an email notification from FFN?

No such luck, I'm afraid. My test fic was one that updated days and days ago. The copy in Calibre was 4 chapters and the one online was 5. The fifth chapter was definitely there - I could click to it manually in the browser and WebToEpub would gather it up.

I suppose that's one advantage of only trying to do this once a week or so, rather than immediately whenever a fic is updated.

And no, I'm not running any sort of cache cleaner. As part of ensuring that I was using the correct folder, I'd open it up to see what there, refresh after clearing the cache, then refresh again after loading the chapters. I wanted to ensure that the folder was emptying and then getting new files.

I tried your suggestion about "check next chapter" and... it's made me very confused.

So I'm testing with four different fics now: two that aren't in Calibre at all and two that are in Calibre, but have at least a couple of new chapters online.

With totally new fics, I get the same error as I mentioned before: can't find the page in cache. This happens whether checking next chapter is true or false.

With existing fics, I'm now getting a different error: I'm told that the fic already has X chapters, where X is the number of chapters in the copy in Calibre. It's not the number of chapters that are online (and which I've loaded). I'm getting this error now whether "check next chapter" is true or false.

These aren't compatible errors! In order to determine that the fic already has a certain number of chapters, FFF must have found the pages in cache, right? Granted, it's not finding all the pages because it's got the number wrong, but this strongly implies that it's locating some chapter pages. But when it comes to completely new fics, it can't find anything?

Ergo, I'm very confused. I'm hopeful that the "already has x chapters" error can be linked back to something I've done wrong and can fix.

[JimmXinu]

New Test Version Posted

2021-07-02
- Pre-v5 Calibre fix for nsapa_proxy.

Originally Posted by steltek

I've posted the debug log in the code tags ...

calibre Debug log
calibre 4.23 [64bit]  embedded-python: True is64bit: True
Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.16+
...
  File "C:\Users\EFXRHNSXN\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\nsapa_proxy.py", line 170
SyntaxError: 'continue' not supported inside 'finally' clause
...

FYI, I assume everybody's using the most recent version of Calibre unless told otherwise. nsapa introduced some code that wasn't python2 (AKA pre-v5 Calibre) compatible. I've posted a new test version that's py2 compatible again.

Originally Posted by SilentShadow123

Is there a way I can update a fanfiction regardless of whether it has fewer or the same number of chapters on the website than in calibre?

For example, one of the fanfics I read had 2 chapters and I added it into calibre with 2 chapters. However, the author removed the 2nd chapter and added a different 2nd chapter, and it said "Already existing" and the update failed. Is there a way I could update it in calibre so that it has 3 chapters in total with both the new one and the old one that was removed? Is Overwrite the only option?

FFF primarily just looks at the number of chapters. Some sites have unique chapter URLs, so if a chapter is replaced the chapter URL would change and FFF would replace it on next update (Update Always needed if no additional chapters to trigger update).

However, for sites that just number chapter URLs, you'll need to either manually delete the changed chapter from the EPUB or do Overwrite to get the new one.

FFF does not have a way to keep deleted chapters. Best option would be to rename the old one and remove the story URL in Calibre, then keep a different copy as the 'proper' version.

Originally Posted by Philosophize

No such luck, I'm afraid. My test fic was one that updated days and days ago. The copy in Calibre was 4 chapters and the one online was 5. The fifth chapter was definitely there - I could click to it manually in the browser and WebToEpub would gather it up.
...

Two things come to mind:

1. When doing tons of tests with multiple setups, I personally find it entirely too easy to get out of sync. If you have FFF pointed at one brower's cache while you're updating the other one by accident, you'd get this behavior, right?
2. We (somewhat) recently added code to skip cache entries that FFF can't read instead of erroring and crashing the whole download. It's possible you're bumping into that. Seeing a debug log would help.

[PeterT]

For folks having cache issues o wonder if some of the utilities from NirSoft might be of use.

See https://www.nirsoft.net/utils/chrome_cache_view.html for tools to display what is in cache for FireFox and Chrome browsers.

Sent from my Pixel 4a using Tapatalk

[Philosophize]

Originally Posted by JimmXinu

When doing tons of tests with multiple setups, I personally find it entirely too easy to get out of sync. If you have FFF pointed at one brower's cache while you're updating the other one by accident, you'd get this behavior, right?

That's a good point, but to be fair I only occasionally check Chrome. I haven't gotten Chrome to work at all, only Firefox, so that's where my focus is. I'll then go and try Chrome occasionally to see if I'm getting consistent results.

But just to be sure, I went back and tried again with two fics, double-checking that I was working with the Firefox cache in the settings. I was, and the results didn't change.

Here's a debug log of the same two fics: the first is a fic that isn't in Calibre at all yet, the second is a fic that I have a few chapters of, but there are a couple of new chapters online. I can do other tests as well, if they would prove informative. Nothing jumps out at me as an obvious issue or problem, but you'll understand it better than I could.

FFF: DEBUG: 2021-07-02 13:18:00,350: calibre_plugins.fanficfare_plugin.fff_plugin(1109): FanFicFare v4.3.10
FFF: DEBUG: 2021-07-02 13:18:00,491: calibre_plugins.fanficfare_plugin.fff_plugin(1230): prep_download_loop url:https://www.fanfiction.net/s/13903544/1/Unintended-Consequences
FFF: INFO: 2021-07-02 13:18:00,491: calibre_plugins.fanficfare_plugin.prefs(217): Attempting to read settings from predecessor--FFDL
FFF: INFO: 2021-07-02 13:18:00,491: calibre_plugins.fanficfare_plugin.prefs(222): Using default settings
FFF: DEBUG: 2021-07-02 13:18:00,502: configurable.py(1052): 1configuration.get_browser_cache:None
FFF: DEBUG: 2021-07-02 13:18:00,502: configurable.py(990): use_cloudscraper:true
FFF: DEBUG: 2021-07-02 13:18:00,503: configurable.py(1008): use_browser_cache:true
FFF: DEBUG: 2021-07-02 13:18:00,503: configurable.py(1010): browser_cache_path:/Users/XXX/Library/Caches/Firefox/Profiles/x0sr0051.default-release/cache2
FFF: DEBUG: 2021-07-02 13:18:00,572: firefoxcache2.py(50): Using FirefoxCache2
FFF: DEBUG: 2021-07-02 13:18:00,572: configurable.py(1022): use_basic_cache:true
FFF: DEBUG: 2021-07-02 13:18:00,572: configurable.py(1056): 2configuration.get_browser_cache:<fanficfare.browsercache.BrowserCache object at 0x137a5a490>
FFF: DEBUG: 2021-07-02 13:18:00,572: calibre_plugins.fanficfare_plugin.fff_plugin(1320): tweak_fg_sleep: Set FG sleep override time 4.0
FFF: DEBUG: 2021-07-02 13:18:00,572: adapter_fanfictionnet.py(112): URL: https://www.fanfiction.net/s/13903544/1/Unintended-Consequences
FFF: DEBUG: 2021-07-02 13:18:00,573: fetcher.py(234): 
========== MISS (GET) BasicCache
https://www.fanfiction.net/s/13903544/1/Unintended-Consequences
FFF: DEBUG: 2021-07-02 13:18:00,573: basebrowsercache.py(119): do_map_cache_keys()
FFF: DEBUG: 2021-07-02 13:18:00,573: firefoxcache2.py(81): using scandir
FFF: DEBUG: 2021-07-02 13:18:04,845: firefoxcache2.py(89): Read 9897 entries
FFF: DEBUG: 2021-07-02 13:18:04,845: basebrowsercache.py(122): Cached 4019 entries
FFF: DEBUG: 2021-07-02 13:18:04,845: basebrowsercache.py(55): do_cprofile time:4.272258758544922
FFF: DEBUG: 2021-07-02 13:18:04,845: fetcher.py(273): 
========== MISS (GET) BrowserCache
https://www.fanfiction.net/s/13903544/1/Unintended-Consequences
FFF: ERROR: 2021-07-02 13:18:04,845: calibre_plugins.fanficfare_plugin.dialogs(720): Exception: {'title': 'Unknown', 'author_sort': ['Unknown'], 'author': ['Unknown'], 'comments': '', 'good': False, 'status': 'Error', 'showerror': True, 'calibre_id': None, 'begin': None, 'end': None, 'comment': "HTTP Error in FFF 'Page not found or expired in Browser Cache (see FFF setting browser_cache_age_limit)'(428) URL:'https://www.fanfiction.net/s/13903544/1/Unintended-Consequences'", 'url': 'https://www.fanfiction.net/s/13903544/1/Unintended-Consequences', 'site': 'www.fanfiction.net', 'series': '', 'added': False, 'pubdate': None, 'publisher': None, 'uniqueurl': 'https://www.fanfiction.net/s/13903544/1/Unintended-Consequences[None-None]', 'listorder': 0, 'collision': 'Update EPUB if New Chapters'}:HTTP Error in FFF 'Page not found or expired in Browser Cache (see FFF setting browser_cache_age_limit)'(428) URL:'https://www.fanfiction.net/s/13903544/1/Unintended-Consequences'
Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.dialogs", line 707, in do_loop
    self.foreach_function(book)
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1323, in prep_download_loop
    story = self.get_story_metadata_only(adapter)
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1195, in get_story_metadata_only
    adapter.getStoryMetadataOnly(get_cover=False)
  File "/Users/XXX/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/adapters/base_adapter.py", line 308, in getStoryMetadataOnly
    self.doExtractChapterUrlsAndMetadata(get_cover=get_cover)
  File "/Users/XXX/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/adapters/adapter_fanfictionnet.py", line 114, in doExtractChapterUrlsAndMetadata
    data = self.get_request(url)
  File "/Users/XXX/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/requestable.py", line 114, in get_request
    return self.get_request_redirected(url,
  File "/Users/XXXLibrary/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/requestable.py", line 106, in get_request_redirected
    (data,rurl) = self.configuration.get_fetcher().get_request_redirected(
  File "/Users/XXX/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/fetcher.py", line 390, in get_request_redirected
    fetchresp = self.do_request('GET',
  File "/Users/XXX/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/fetcher.py", line 240, in fetcher_do_request
    fetchresp = chainfn(
  File "/Users/XXX/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/fetcher.py", line 278, in fetcher_do_request
    raise exceptions.HTTPErrorFFF(
fanficfare.exceptions.HTTPErrorFFF: HTTP Error in FFF 'Page not found or expired in Browser Cache (see FFF setting browser_cache_age_limit)'(428) URL:'https://www.fanfiction.net/s/13903544/1/Unintended-Consequences'
qt.qpa.fonts: Populating font family aliases took 366 ms. Replace uses of missing font family "Monospace" with one that exists to avoid this cost. 
FFF: DEBUG: 2021-07-02 13:18:33,145: calibre_plugins.fanficfare_plugin.fff_plugin(1109): FanFicFare v4.3.10
FFF: DEBUG: 2021-07-02 13:18:33,293: calibre_plugins.fanficfare_plugin.fff_plugin(1230): prep_download_loop url:https://www.fanfiction.net/s/13829522/1/Fate-s-Twist-from-Beyond-the-Veil
FFF: DEBUG: 2021-07-02 13:18:33,306: configurable.py(1052): 1configuration.get_browser_cache:None
FFF: DEBUG: 2021-07-02 13:18:33,306: configurable.py(990): use_cloudscraper:true
FFF: DEBUG: 2021-07-02 13:18:33,306: configurable.py(1008): use_browser_cache:true
FFF: DEBUG: 2021-07-02 13:18:33,306: configurable.py(1010): browser_cache_path:/Users/XXX/Library/Caches/Firefox/Profiles/x0sr0051.default-release/cache2
FFF: DEBUG: 2021-07-02 13:18:33,324: firefoxcache2.py(50): Using FirefoxCache2
FFF: DEBUG: 2021-07-02 13:18:33,325: configurable.py(1022): use_basic_cache:true
FFF: DEBUG: 2021-07-02 13:18:33,325: configurable.py(1056): 2configuration.get_browser_cache:<fanficfare.browsercache.BrowserCache object at 0x137a078e0>
FFF: DEBUG: 2021-07-02 13:18:33,325: calibre_plugins.fanficfare_plugin.fff_plugin(1320): tweak_fg_sleep: Set FG sleep override time 4.0
FFF: DEBUG: 2021-07-02 13:18:33,325: adapter_fanfictionnet.py(112): URL: https://www.fanfiction.net/s/13829522/1/Fate-s-Twist-from-Beyond-the-Veil
FFF: DEBUG: 2021-07-02 13:18:33,325: fetcher.py(234): 
========== MISS (GET) BasicCache
https://www.fanfiction.net/s/13829522/1/Fate-s-Twist-from-Beyond-the-Veil
FFF: DEBUG: 2021-07-02 13:18:33,326: basebrowsercache.py(119): do_map_cache_keys()
FFF: DEBUG: 2021-07-02 13:18:33,326: firefoxcache2.py(81): using scandir
FFF: DEBUG: 2021-07-02 13:18:37,482: firefoxcache2.py(89): Read 9897 entries
FFF: DEBUG: 2021-07-02 13:18:37,482: basebrowsercache.py(122): Cached 4019 entries
FFF: DEBUG: 2021-07-02 13:18:37,482: basebrowsercache.py(55): do_cprofile time:4.156722068786621
FFF: DEBUG: 2021-07-02 13:18:37,560: fetcher.py(273): 
========== HIT (GET) BrowserCache
https://www.fanfiction.net/s/13829522/1/Fate-s-Twist-from-Beyond-the-Veil
FFF: DEBUG: 2021-07-02 13:18:37,560: requestable.py(50): Encoding:utf8
FFF: DEBUG: 2021-07-02 13:18:37,705: calibre_plugins.fanficfare_plugin.fff_plugin(1432): from URL(https://www.fanfiction.net/s/13829522/1/Fate-s-Twist-from-Beyond-the-Veil)
FFF: DEBUG: 2021-07-02 13:18:37,869: calibre_plugins.fanficfare_plugin.fff_plugin(1445): existing found by identifier URL

[JimmXinu]

I've no idea what's going on with your setup.

If FFF failed to read any cache files, it would report them ("Cache file %s failed to load, skipping."%path) right before "Read 9897 entries".

I used FFF with Firefox on Unintended-Consequences myself just this morning.

OS differences maybe? Are you using Calibre from a non-official (ie, not calibre-ebook.com) source? Or FFF from somewhere other than this thread?

[steltek]

Originally Posted by JimmXinu

New Test Version Posted

2021-07-02
- Pre-v5 Calibre fix for nsapa_proxy.



FYI, I assume everybody's using the most recent version of Calibre unless told otherwise. nsapa introduced some code that wasn't python2 (AKA pre-v5 Calibre) compatible. I've posted a new test version that's py2 compatible again.

Didn't think about that. It is running on an older Win7 machine that I apparently can't bring myself to take the time to upgrade to Win10. I guess I'll have to do something about that soon.

I appreciate the effort.

As far as Philosophize's issue was concerned, I was also able to add both of those fics to my library with no problem on first try (one I was already following but it hadn't gotten long enough for me to add yet). Of course, I am on Windows and not running Apple stuff, so it could also be some sort of privacy-related thing they are doing and just haven't announced yet (especially given their recent privacy bent and the fact that web browser cache files in general are a really bad weakness in privacy protection).

[Philosophize]

Originally Posted by JimmXinu

OS differences maybe? Are you using Calibre from a non-official (ie, not calibre-ebook.com) source? Or FFF from somewhere other than this thread?

Perhaps it's OS - but this worked in Firefox a couple of weeks ago, no problem. I walked away for a few days, came back, and it didn't work (I didn't post here immediately - I tried off-and-on over a few days to get it to work myself). There weren't any software changes that I can think of. I'm not even sure that I restarted.

It's not a non-standard source for Calibre. There might be a new version, but that shouldn't cause your plugin to spontaneously stop working. And I did get the plugin from this thread. In fact, before posting today, I re-downloaded the test/beta version to see if that helped.

Originally Posted by JimmXinu

If FFF failed to read any cache files, it would report them ("Cache file %s failed to load, skipping."%path) right before "Read 9897 entries".

I noticed that. It certainly seems like it's seeing the cache files. And in this instance, the only cache files should be that and another story or two. Is there any chance that it's seeing the cache files, but throwing that error by mistake? That there's some other error but it's not reporting it correctly?

In the case of the other fic, I get the impression that it's seeing the cache files, but somehow reading them incorrectly because it's not seeing the new chapters. These two errors are coming up one right after the other, and seem to be happening consistently based on whether I have the fic started in Calibre or not. I doubt they are completely separate and independent - but what would connect them?