[GUI Plugin] FanFicFare

[tricklem]

Anyone seen this error yet?

cloudscraper reports: "Cloudflare IUAM possibility malformed, issue extracing delay value."

I scanned back a few pages in the thread but didn't see it.

[ownedbycats]

It's something to do with Cloudflare's under-attack mode - I suspect ffnet's implemented a higher CF security level for certain countries/IP ranges (or more likely just misconfigured something). The test version lets you grab the chapters from browser cache (Chromium-based browsers) to bypass it entirely.

Jim: Just curious, were there any plans/discussions to also implement page caching for Firefox-based browsers?

[JimmXinu]

Barring unforeseen difficulties, Firefox cache reading is coming. That's my current project.

[Ser4nb2LUY6e]

Originally Posted by JimmXinu

I did consider that, but they you would get as many 'Update your library?' dialogs as there were sites.

I'm aware and ok with that. That's how I would expect it to behave. (Except for not showing the dialog at all for any site which has 0 updates, and if the metadata is retrieved before the job(s) is/are created.)

If there was an option to turn this one right now, I would happily use it.

Originally Posted by JimmXinu

And a user can do that right now manually if they want.

I know, and I sometimes do that when I know that some of them will take a long time. But this is way too fiddly, having to carefully select different books for starting the jobs, for the "daily update" run.


Btw, the download changes now result in something like:
Job: Download 6 FanFiction Book(s)
Status: 23 of 63 stories finished downloading

That is, books that have already been determined before the job was created not to require updating (otherwise it couldn't say "Download x FanFiction Book(s)" are still included in the actual job progress.

[ownedbycats]

Originally Posted by Ser4nb2LUY6e

I'm aware and ok with that. That's how I would expect it to behave. (Except for not showing the dialog at all for any site which has 0 updates, and if the metadata is retrieved before the job(s) is/are created.)

If there was an option to turn this one right now, I would happily use it..

If it does become an option at some point in the future, i would make it a user choice in personal.ini - separate_process_per_site maybe.

[ownedbycats]

Originally Posted by Ser4nb2LUY6e

I know, and I sometimes do that when I know that some of them will take a long time. But this is way too fiddly, having to carefully select different books for starting the jobs, for the "daily update" run..

Assuming I am not misunderstanding this:

I used to do a daily update of all my in-progresses, but after reaching a certain critical mass of them I decided to switch over to emails. (I use drag & drop from Thunderbird instead of FFF's email fetch.) It's a bit nicer and I suggest it if you have an account or don't mind setting them up.

Also I have a custom bool column, a custom column setting for "N"=>#fanficsubbed,n, and a column icon so I actually remember to subscribe.

[JimmXinu]

New Test Version Posted

2021-02-07
- Don't count continue_on_chapter_error chapters when checking vs 'new chapters'.
- More properly get msg payload fromemail drag and drop. Closes #645
- Count stories downloading in BG better.
- First version of Firefox cache2 reader.

It's much easier to find your cache dir in Firefox: enter about:cache in the address bar and it will tell you. Ends in 'cache2', like:

C:\Users\youruser\AppData\Local\Mozilla\Firefox\Pr ofiles\ZjwI7Fo4.default\cache2

The Firefox cache reader has some performance improvements I may also apply to the Chrome Simple cache reader.

[steltek]

I can verify that it works fine when downloading new fics utilizing the browser cache in the most recent versions of Firefox, Waterfox 3rd gen, and Waterfox Classic.

Whatever you did performance improvement-wise causes it to run far faster for me than it did in either Chrome or Vivaldi.

[tricklem]

I have my FireFox browser cache location, but I can't seem to find where I would put that info on the new plug in. Can anyone point me in the right direction?

[JimmXinu]

New Test Version Posted

2021-02-08
- Restore multi-process background jobs but as one process per site.


FFF BG processes still limited by Calibre's max BG jobs setting (Preferences->Misc) because I don't want to bother making my own setting.

Multi-threading was fast, but something about Calibre's word count (which launches yet another process) would fail once in awhile--even when protected by a thread lock.

[ownedbycats]

I can confirm the fix to #645 works.

[JimmXinu]

New Test Version Posted (again)

2021-02-08
- Restore multi-process background jobs as one process per site.
- Add scandir & stat checking for simplecache for performance.

Went back and added the same performance improvements for Chrome Simple Cache. Chrome Blockfile Cache can't benefit from the same changes.

I'll be curious to know if this improves performance for people.

[ownedbycats]

I use BG metadata, and I noticed the little lag I got on the initial phase (while queueing metadata, from it access the cache) is gone now.

[steltek]

Originally Posted by tricklem

I have my FireFox browser cache location, but I can't seem to find where I would put that info on the new plug in. Can anyone point me in the right direction?

Go in to configure FFF. Go to the Personal.ini tab, then hit the "Edit Personal.ini" button. When Personal.ini opens up, look for the [defaults] section right near the top. Right below [defaults], enter a new line beginning with browser_cache_path: followed by your cache path (no spaces between the : and your browser cache path).

Then, go to the [overrides] section at the bottom of Personal.ini. Paste the following text below [overrides]:

[www.fanfiction.net]

##the following line tells FFF to use the browser cache when set to true

use_browser_cache:true

##the following line limits FFF to use cache entries no older than the
##number of hours shown (in this case, cached items no 4 hours or less old).

browser_cache_age_limit:4.0

Then, save the changes to Personal.ini using the OK button. Once you have done this, you should be able to browse a fic in Firefox, then run FFF to collect the pages out of the browser cache.

[tricklem]

Originally Posted by steltek

Go in to configure FFF. Go to the Personal.ini tab, then hit the "Edit Personal.ini" button. When Personal.ini opens up, look for the [defaults] section right near the top. Right below [defaults], enter a new line beginning with browser_cache_path: followed by your cache path (no spaces between the : and your browser cache path).

Then, go to the [overrides] section at the bottom of Personal.ini. Paste the following text below [overrides]:

[www.fanfiction.net]

##the following line tells FFF to use the browser cache when set to true

use_browser_cache:true

##the following line limits FFF to use cache entries no older than the
##number of hours shown (in this case, cached items no 4 hours or less old).

browser_cache_age_limit:4.0

Then, save the changes to Personal.ini using the OK button. Once you have done this, you should be able to browse a fic in Firefox, then run FFF to collect the pages out of the browser cache.

Thanks! Hopefully I did that right.