[GUI Plugin] FanFicFare

[Philosophize]

Originally Posted by steltek

As far as Philosophize's issue was concerned, I was also able to add both of those fics to my library with no problem on first try (one I was already following but it hadn't gotten long enough for me to add yet). Of course, I am on Windows and not running Apple stuff, so it could also be some sort of privacy-related thing they are doing and just haven't announced yet (especially given their recent privacy bent and the fact that web browser cache files in general are a really bad weakness in privacy protection).

That's a good point about security, but this Mac is running Catalina, which was released back in 2019. Even the most recent updates haven't introduced all the privacy measures we're reading about. Now that you've mentioned it, I suspect it will eventually become a problem, but I don't think it's my problem here.

I also went ahead and updated Calibre to the latest and the plugin to the version posted a little bit ago today. Errors are exactly the same.

Blaming the OS would be the obvious solution, if it weren't for the fact that it did work for one session already. But I don't have any better ideas - just the speculation that there's an error which isn't being reported correctly.

[JimmXinu]

Originally Posted by Philosophize

It's not a non-standard source for Calibre. There might be a new version, but that shouldn't cause your plugin to spontaneously stop working. And I did get the plugin from this thread. In fact, before posting today, I re-downloaded the test/beta version to see if that helped.

We have seen non-standard versions of Calibre treat plugins differently enough to break things, albeit in Calibre v5.

If you haven't, I would try the stock-standard Calibre that Kovid builds. Officially, I don't support FFF plugin on anything but Kovid's builds.

I have no specific reason to believe that will help, but I also have no idea why you're having problems.

[steltek]

Out of curiosity, Philosophize, what do you get if you enter about:profiles in the Firefox URL bar and hit enter? Does it show multiple user profiles? And, which one does it currently show as being in use?

The only other thing I can think of is that you have a corrupt Firefox user profile, or that your machine has more than one user profile and the default profile has changed somehow. It might even be worth creating a new user profile and switching to it for testing purposes (just remember if you do this you have to change the user profile browser cache path in personal.ini to match the new profile).

[Ealdwulf]

I seem to be having a slightly different error in regards to using browser cache. This is what I get in the FanFicFare log:

Directory does not contain a known browser cache type: 'C:\Users\bryan\AppData\Local\Mozilla\Firefox\Prof iles\4vb4brdx.default-release'

what I have in the personal.ini is:

browser_cache_path:C:\Users\bryan\AppData\Local\Mo zilla\Firefox\Profiles\4vb4brdx.default-release\cache2

I am using FireFox (and only use FireFox unless a website requires IE which is very rare these days). It is up to date and is 89.0.2 (64-bit)

Could this error be due to being the 64-bit version of FireFox instead of 32-bit?

[steltek]

Originally Posted by Ealdwulf

I seem to be having a slightly different error in regards to using browser cache. This is what I get in the FanFicFare log:

Directory does not contain a known browser cache type: 'C:\Users\bryan\AppData\Local\Mozilla\Firefox\Prof iles\4vb4brdx.default-release'

what I have in the personal.ini is:

browser_cache_path:C:\Users\bryan\AppData\Local\Mo zilla\Firefox\Profiles\4vb4brdx.default-release\cache2

I am using FireFox (and only use FireFox unless a website requires IE which is very rare these days). It is up to date and is 89.0.2 (64-bit)

Could this error be due to being the 64-bit version of FireFox instead of 32-bit?

It shouldn't make a difference, as both use the same cache format. I'm running the 64-bit versions of both Firefox and Waterfox 3rd gen, and both work just fine with FFF. As a test, I just downloaded a random fic containing 4 chapters and it worked just fine reading from the cache in the most recent versions of both Firefox and Waterfox.

If you type about:cache in the browser URL bar, what path does it show under Storage disk location in the disk section of the resulting page?

[Ealdwulf]

Originally Posted by steltek

It shouldn't make a difference, as both use the same cache format. I'm running the 64-bit versions of both Firefox and Waterfox 3rd gen, and both work just fine with FFF. As a test, I just downloaded a random fic containing 4 chapters and it worked just fine reading from the cache in the most recent versions of both Firefox and Waterfox.

If you type about:cache in the browser URL bar, what path does it show under Storage disk location in the disk section of the resulting page?

The location displayed is:

C:\Users\bryan\AppData\Local\Mozilla\Firefox\Profi les\4vb4brdx.default-release\cache2


not sure why it is throwing extra spaces in there once it saves the post


that's actually where I got the path to put in place for the browser_cache_path setting

now I did have to reinstall my O/S and applications a couple of months ago since I built a new PC, but if anything, new installs of O/S and applications would normally help and not hinder, right?

[steltek]

Originally Posted by Ealdwulf

The location displayed is:

C:\Users\bryan\AppData\Local\Mozilla\Firefox\Profi les\4vb4brdx.default-release\cache2


not sure why it is throwing extra spaces in there once it saves the post


that's actually where I got the path to put in place for the browser_cache_path setting

now I did have to reinstall my O/S and applications a couple of months ago since I built a new PC, but if anything, new installs of O/S and applications would normally help and not hinder, right?

Yeah, a clean OS install is always a good thing when you can manage it. To avoid the board sticking extra spaces in, you can just stick the line in a code block like I did below and it won't do that.

By any chance have you tried to go into the Firefox privacy settings and clear the contents of the cache, close Firefox, open it again, then tried to page through the fic again before running FFF to see if it makes a difference?

BTW, as a test, I had a clean Win10 virtual machine that has not had either Firefox or Calibre ever installed on it. I went ahead and just installed the newest versions of both Firefox and Calibre, added the FFF plugin to Calibre, and configured FFF to read from the Firefox cache (I also set the use_browser_cache_only:true option to force it to read the cache only or fail) . Then, I paged through a short 4 page incomplete fic link on FFN.net and tried to use FFF to pull the fic out of the cache. Everything worked as it should have and the fic is in the new Calibre library. As a result, at least so far as Windows is concerned, there appears to have been no change in the Firefox cache structure.

Just to be sure, you have the Defaults and Overrides sections of personal.ini in FFF set up as follows (the line in red is optional and can be ignored - it just serves to force exclusive use of the browser cache or the operation to fail without trying to download the pages if they aren't in the browser cache. Do note everything below [overrides] has to be manually keyed or copy/pasted)?

[defaults]
browser_cache_path:C:\Users\bryan\AppData\Local\Mozilla\Firefox\Profiles\4vb4brdx.default-release\cache2


[overrides]

[www.fanfiction.net]
use_browser_cache:true
browser_cache_age_limit:4.0
use_browser_cache_only:true

[kamille3]

I'm trying to download this story https://storiesonline.net/s/22301/hope-and-betrayal but I keep getting this error in the log -


Status
Title
Author
Comment
URL
Error
Unknown
Unknown
HTTPSConnectionPool(host='storiesonline.net', port=443): Max retries exceeded with url: /s/22301:i (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1123)')))
https://storiesonline.net/s/22301

I'm a newbie with this software so dunno what I'm doing wrong. Please kindly help.

[steltek]

Originally Posted by kamille3

I'm trying to download this story https://storiesonline.net/s/22301/hope-and-betrayal but I keep getting this error in the log -


Status
Title
Author
Comment
URL
Error
Unknown
Unknown
HTTPSConnectionPool(host='storiesonline.net', port=443): Max retries exceeded with url: /s/22301:i (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1123)')))
https://storiesonline.net/s/22301

I'm a newbie with this software so dunno what I'm doing wrong. Please kindly help.

I don't use that site, but I recall a conversation from December of last year on the FFF Github site that you have to log into your account on the site in a web browser, go to your account settings, and change the site theme to use the classic theme (it used to be the default, so the FFF adapter is coded to use it, but doesn't get set as the default anymore).

Also, depending upon how much you've downloaded, the site also has a limit on the number of stories you can download in 24 hours (it resets every 24 hours) - I presume this limit is void if you are a paid member.

[kamille3]

Originally Posted by steltek

I don't use that site, but I recall a conversation from December of last year on the FFF Github site that you have to log into your account on the site in a web browser, go to your account settings, and change the site theme to use the classic theme (it used to be the default, so the FFF adapter is coded to use it, but doesn't get set as the default anymore).

Also, depending upon how much you've downloaded, the site also has a limit on the number of stories you can download in 24 hours (it resets every 24 hours) - I presume this limit is void if you are a paid member.

I do have classic theme set in my account. And as for the limit I've barely downloaded anything in a while and couldn't even download a single story through FFF today. Although I do see my account showing 6 downloads today which I presume is just due to viewing a story in my browser.

[steltek]

Originally Posted by kamille3

I do have classic theme set in my account. And as for the limit I've barely downloaded anything in a while and couldn't even download a single story through FFF today. Although I do see my account showing 6 downloads today which I presume is just due to viewing a story in my browser.

I presume you gave FFF your login id and password if/when prompted for it?
I was able to download that story URL using the current release of FFF with no problems in both versions 4.23 and 5.22 of Calibre after creating an account.

What version of Calibre are you using, and did you download it from the official Calibre site at https://calibre-ebook.com ?

It might be worth restarting your machine and waiting a day in case it is an Internet glitch. Then, if it still doesn't work, you might post the FFF log in a code box here for those more learned than me to peruse.

[davidfor]

Originally Posted by kamille3

I'm trying to download this story https://storiesonline.net/s/22301/hope-and-betrayal but I keep getting this error in the log -


Status
Title
Author
Comment
URL
Error
Unknown
Unknown
HTTPSConnectionPool(host='storiesonline.net', port=443): Max retries exceeded with url: /s/22301:i (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1123)')))
https://storiesonline.net/s/22301

I'm a newbie with this software so dunno what I'm doing wrong. Please kindly help.

Certificate problems are frequently issues with something not being updated on your machine. If you are using Windows, open the site in the Microsoft browser (IE or Edge). This should trigger an update for the CA in use.

And, I have been able to download that story and a couple of other using FFF and calibre 5.22.1 on a Windows 10 machine.

[JimmXinu]

Originally Posted by kamille3

I'm trying to download this story https://storiesonline.net/s/22301/hope-and-betrayal but I keep getting this error in the log -

HTTPSConnectionPool(host='storiesonline.net', port=443): Max retries exceeded with url: /s/22301:i (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1123)')))
https://storiesonline.net/s/22301

That site/story works fine for me.

The error is clearly an SSL issue of some kind. Those are frequently due to mismatched SSL versions. Are you using the latest official Calibre release?

You also might try this setting in personal.ini:

[storiesonline.net]
use_ssl_unverified_context:true

[kamille3]

Originally Posted by JimmXinu

That site/story works fine for me.

The error is clearly an SSL issue of some kind. Those are frequently due to mismatched SSL versions. Are you using the latest official Calibre release?

You also might try this setting in personal.ini:

[storiesonline.net]
use_ssl_unverified_context:true

Yes I'm using the latest version 5.22.1 [64bit]. Thankfully downloads have starting working again without me doing anything. Dunno what was causing the problem in the first place.

[Ealdwulf]

Originally Posted by steltek

Yeah, a clean OS install is always a good thing when you can manage it. To avoid the board sticking extra spaces in, you can just stick the line in a code block like I did below and it won't do that.

By any chance have you tried to go into the Firefox privacy settings and clear the contents of the cache, close Firefox, open it again, then tried to page through the fic again before running FFF to see if it makes a difference?

BTW, as a test, I had a clean Win10 virtual machine that has not had either Firefox or Calibre ever installed on it. I went ahead and just installed the newest versions of both Firefox and Calibre, added the FFF plugin to Calibre, and configured FFF to read from the Firefox cache (I also set the use_browser_cache_only:true option to force it to read the cache only or fail) . Then, I paged through a short 4 page incomplete fic link on FFN.net and tried to use FFF to pull the fic out of the cache. Everything worked as it should have and the fic is in the new Calibre library. As a result, at least so far as Windows is concerned, there appears to have been no change in the Firefox cache structure.

Just to be sure, you have the Defaults and Overrides sections of personal.ini in FFF set up as follows (the line in red is optional and can be ignored - it just serves to force exclusive use of the browser cache or the operation to fail without trying to download the pages if they aren't in the browser cache. Do note everything below [overrides] has to be manually keyed or copy/pasted)?

[defaults]
browser_cache_path:C:\Users\bryan\AppData\Local\Mozilla\Firefox\Profiles\4vb4brdx.default-release\cache2


[overrides]

[www.fanfiction.net]
use_browser_cache:true
browser_cache_age_limit:4.0
use_browser_cache_only:true

OK. Finally got a chance to verify this. Cleaned the cache, verified FireFox up to date, restarted the browser.

Current Calibre version: 5.22 [64bit]
Current FanFicFare version: 4.3.11
Current FireFox version: 89.0.2

Opened two stories on FF.net and paged through each chapter, clicked the author URL, and clicked the stories images for each.

confirmed the relevant sections of personal.ini are:

[defaults]
browser_cache_path:C:\Users\bryan\AppData\Local\Mozilla\Firefox\Profiles\4vb4brdx.default-release\cache2

[www.fanfiction.net]
use_cloudscraper:true
use_browser_cache:true
browser_cache_age_limit:4.0
#use_browser_cache_only:false

I have tried both to overwrite if newer (the stories are, one with a single new chapter and the other with four new chapters), and also just updating. Both give same error message in the comment column of the log file for the job:

Directory does not contain a known browser cache type: 'C:\Users\bryan\AppData\Local\Mozilla\Firefox\Profiles\4vb4brdx.default-release'

And looking at about:cache once again confirms the entry:

Storage disk location: 	C:\Users\bryan\AppData\Local\Mozilla\Firefox\Profiles\4vb4brdx.default-release\cache2

I'm not sure what else to do on this, I haven't tried to use WebtoEpub for this but considering the error message I doubt it would allow this to work.