Quote:
Originally Posted by JimmXinu
When doing tons of tests with multiple setups, I personally find it entirely too easy to get out of sync. If you have FFF pointed at one brower's cache while you're updating the other one by accident, you'd get this behavior, right?
|
That's a good point, but to be fair I only occasionally check Chrome. I haven't gotten Chrome to work at all, only Firefox, so that's where my focus is. I'll then go and try Chrome occasionally to see if I'm getting consistent results.
But just to be sure, I went back and tried again with two fics, double-checking that I was working with the Firefox cache in the settings. I was, and the results didn't change.
Here's a debug log of the same two fics: the first is a fic that isn't in Calibre at all yet, the second is a fic that I have a few chapters of, but there are a couple of new chapters online. I can do other tests as well, if they would prove informative. Nothing jumps out at me as an obvious issue or problem, but you'll understand it better than I could.
Code:
FFF: DEBUG: 2021-07-02 13:18:00,350: calibre_plugins.fanficfare_plugin.fff_plugin(1109): FanFicFare v4.3.10
FFF: DEBUG: 2021-07-02 13:18:00,491: calibre_plugins.fanficfare_plugin.fff_plugin(1230): prep_download_loop url:https://www.fanfiction.net/s/13903544/1/Unintended-Consequences
FFF: INFO: 2021-07-02 13:18:00,491: calibre_plugins.fanficfare_plugin.prefs(217): Attempting to read settings from predecessor--FFDL
FFF: INFO: 2021-07-02 13:18:00,491: calibre_plugins.fanficfare_plugin.prefs(222): Using default settings
FFF: DEBUG: 2021-07-02 13:18:00,502: configurable.py(1052): 1configuration.get_browser_cache:None
FFF: DEBUG: 2021-07-02 13:18:00,502: configurable.py(990): use_cloudscraper:true
FFF: DEBUG: 2021-07-02 13:18:00,503: configurable.py(1008): use_browser_cache:true
FFF: DEBUG: 2021-07-02 13:18:00,503: configurable.py(1010): browser_cache_path:/Users/XXX/Library/Caches/Firefox/Profiles/x0sr0051.default-release/cache2
FFF: DEBUG: 2021-07-02 13:18:00,572: firefoxcache2.py(50): Using FirefoxCache2
FFF: DEBUG: 2021-07-02 13:18:00,572: configurable.py(1022): use_basic_cache:true
FFF: DEBUG: 2021-07-02 13:18:00,572: configurable.py(1056): 2configuration.get_browser_cache:<fanficfare.browsercache.BrowserCache object at 0x137a5a490>
FFF: DEBUG: 2021-07-02 13:18:00,572: calibre_plugins.fanficfare_plugin.fff_plugin(1320): tweak_fg_sleep: Set FG sleep override time 4.0
FFF: DEBUG: 2021-07-02 13:18:00,572: adapter_fanfictionnet.py(112): URL: https://www.fanfiction.net/s/13903544/1/Unintended-Consequences
FFF: DEBUG: 2021-07-02 13:18:00,573: fetcher.py(234):
========== MISS (GET) BasicCache
https://www.fanfiction.net/s/13903544/1/Unintended-Consequences
FFF: DEBUG: 2021-07-02 13:18:00,573: basebrowsercache.py(119): do_map_cache_keys()
FFF: DEBUG: 2021-07-02 13:18:00,573: firefoxcache2.py(81): using scandir
FFF: DEBUG: 2021-07-02 13:18:04,845: firefoxcache2.py(89): Read 9897 entries
FFF: DEBUG: 2021-07-02 13:18:04,845: basebrowsercache.py(122): Cached 4019 entries
FFF: DEBUG: 2021-07-02 13:18:04,845: basebrowsercache.py(55): do_cprofile time:4.272258758544922
FFF: DEBUG: 2021-07-02 13:18:04,845: fetcher.py(273):
========== MISS (GET) BrowserCache
https://www.fanfiction.net/s/13903544/1/Unintended-Consequences
FFF: ERROR: 2021-07-02 13:18:04,845: calibre_plugins.fanficfare_plugin.dialogs(720): Exception: {'title': 'Unknown', 'author_sort': ['Unknown'], 'author': ['Unknown'], 'comments': '', 'good': False, 'status': 'Error', 'showerror': True, 'calibre_id': None, 'begin': None, 'end': None, 'comment': "HTTP Error in FFF 'Page not found or expired in Browser Cache (see FFF setting browser_cache_age_limit)'(428) URL:'https://www.fanfiction.net/s/13903544/1/Unintended-Consequences'", 'url': 'https://www.fanfiction.net/s/13903544/1/Unintended-Consequences', 'site': 'www.fanfiction.net', 'series': '', 'added': False, 'pubdate': None, 'publisher': None, 'uniqueurl': 'https://www.fanfiction.net/s/13903544/1/Unintended-Consequences[None-None]', 'listorder': 0, 'collision': 'Update EPUB if New Chapters'}:HTTP Error in FFF 'Page not found or expired in Browser Cache (see FFF setting browser_cache_age_limit)'(428) URL:'https://www.fanfiction.net/s/13903544/1/Unintended-Consequences'
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.dialogs", line 707, in do_loop
self.foreach_function(book)
File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1323, in prep_download_loop
story = self.get_story_metadata_only(adapter)
File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1195, in get_story_metadata_only
adapter.getStoryMetadataOnly(get_cover=False)
File "/Users/XXX/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/adapters/base_adapter.py", line 308, in getStoryMetadataOnly
self.doExtractChapterUrlsAndMetadata(get_cover=get_cover)
File "/Users/XXX/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/adapters/adapter_fanfictionnet.py", line 114, in doExtractChapterUrlsAndMetadata
data = self.get_request(url)
File "/Users/XXX/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/requestable.py", line 114, in get_request
return self.get_request_redirected(url,
File "/Users/XXXLibrary/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/requestable.py", line 106, in get_request_redirected
(data,rurl) = self.configuration.get_fetcher().get_request_redirected(
File "/Users/XXX/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/fetcher.py", line 390, in get_request_redirected
fetchresp = self.do_request('GET',
File "/Users/XXX/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/fetcher.py", line 240, in fetcher_do_request
fetchresp = chainfn(
File "/Users/XXX/Library/Preferences/calibre/plugins/FanFicFare.zip/fanficfare/fetcher.py", line 278, in fetcher_do_request
raise exceptions.HTTPErrorFFF(
fanficfare.exceptions.HTTPErrorFFF: HTTP Error in FFF 'Page not found or expired in Browser Cache (see FFF setting browser_cache_age_limit)'(428) URL:'https://www.fanfiction.net/s/13903544/1/Unintended-Consequences'
qt.qpa.fonts: Populating font family aliases took 366 ms. Replace uses of missing font family "Monospace" with one that exists to avoid this cost.
FFF: DEBUG: 2021-07-02 13:18:33,145: calibre_plugins.fanficfare_plugin.fff_plugin(1109): FanFicFare v4.3.10
FFF: DEBUG: 2021-07-02 13:18:33,293: calibre_plugins.fanficfare_plugin.fff_plugin(1230): prep_download_loop url:https://www.fanfiction.net/s/13829522/1/Fate-s-Twist-from-Beyond-the-Veil
FFF: DEBUG: 2021-07-02 13:18:33,306: configurable.py(1052): 1configuration.get_browser_cache:None
FFF: DEBUG: 2021-07-02 13:18:33,306: configurable.py(990): use_cloudscraper:true
FFF: DEBUG: 2021-07-02 13:18:33,306: configurable.py(1008): use_browser_cache:true
FFF: DEBUG: 2021-07-02 13:18:33,306: configurable.py(1010): browser_cache_path:/Users/XXX/Library/Caches/Firefox/Profiles/x0sr0051.default-release/cache2
FFF: DEBUG: 2021-07-02 13:18:33,324: firefoxcache2.py(50): Using FirefoxCache2
FFF: DEBUG: 2021-07-02 13:18:33,325: configurable.py(1022): use_basic_cache:true
FFF: DEBUG: 2021-07-02 13:18:33,325: configurable.py(1056): 2configuration.get_browser_cache:<fanficfare.browsercache.BrowserCache object at 0x137a078e0>
FFF: DEBUG: 2021-07-02 13:18:33,325: calibre_plugins.fanficfare_plugin.fff_plugin(1320): tweak_fg_sleep: Set FG sleep override time 4.0
FFF: DEBUG: 2021-07-02 13:18:33,325: adapter_fanfictionnet.py(112): URL: https://www.fanfiction.net/s/13829522/1/Fate-s-Twist-from-Beyond-the-Veil
FFF: DEBUG: 2021-07-02 13:18:33,325: fetcher.py(234):
========== MISS (GET) BasicCache
https://www.fanfiction.net/s/13829522/1/Fate-s-Twist-from-Beyond-the-Veil
FFF: DEBUG: 2021-07-02 13:18:33,326: basebrowsercache.py(119): do_map_cache_keys()
FFF: DEBUG: 2021-07-02 13:18:33,326: firefoxcache2.py(81): using scandir
FFF: DEBUG: 2021-07-02 13:18:37,482: firefoxcache2.py(89): Read 9897 entries
FFF: DEBUG: 2021-07-02 13:18:37,482: basebrowsercache.py(122): Cached 4019 entries
FFF: DEBUG: 2021-07-02 13:18:37,482: basebrowsercache.py(55): do_cprofile time:4.156722068786621
FFF: DEBUG: 2021-07-02 13:18:37,560: fetcher.py(273):
========== HIT (GET) BrowserCache
https://www.fanfiction.net/s/13829522/1/Fate-s-Twist-from-Beyond-the-Veil
FFF: DEBUG: 2021-07-02 13:18:37,560: requestable.py(50): Encoding:utf8
FFF: DEBUG: 2021-07-02 13:18:37,705: calibre_plugins.fanficfare_plugin.fff_plugin(1432): from URL(https://www.fanfiction.net/s/13829522/1/Fate-s-Twist-from-Beyond-the-Veil)
FFF: DEBUG: 2021-07-02 13:18:37,869: calibre_plugins.fanficfare_plugin.fff_plugin(1445): existing found by identifier URL