Register Guidelines E-Books Today's Posts Search

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 02-06-2021, 04:37 PM   #5401
tricklem
Connoisseur
tricklem began at the beginning.
 
tricklem's Avatar
 
Posts: 60
Karma: 10
Join Date: Feb 2015
Device: none
Anyone seen this error yet?

cloudscraper reports: "Cloudflare IUAM possibility malformed, issue extracing delay value."

I scanned back a few pages in the thread but didn't see it.
tricklem is offline   Reply With Quote
Old 02-06-2021, 05:33 PM   #5402
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 10,994
Karma: 75337983
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
It's something to do with Cloudflare's under-attack mode - I suspect ffnet's implemented a higher CF security level for certain countries/IP ranges (or more likely just misconfigured something). The test version lets you grab the chapters from browser cache (Chromium-based browsers) to bypass it entirely.

Jim: Just curious, were there any plans/discussions to also implement page caching for Firefox-based browsers?

Last edited by ownedbycats; 02-06-2021 at 09:26 PM.
ownedbycats is offline   Reply With Quote
Old 02-06-2021, 07:39 PM   #5403
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,974
Karma: 4604635
Join Date: Dec 2011
Location: Midwest USA
Device: Kobo Clara Colour running KOReader
Barring unforeseen difficulties, Firefox cache reading is coming. That's my current project.
JimmXinu is offline   Reply With Quote
Old 02-06-2021, 07:46 PM   #5404
Ser4nb2LUY6e
Connoisseur
Ser4nb2LUY6e began at the beginning.
 
Posts: 60
Karma: 10
Join Date: Jun 2017
Device: Moon+ Reader Pro on Android
Quote:
Originally Posted by JimmXinu View Post
I did consider that, but they you would get as many 'Update your library?' dialogs as there were sites.
I'm aware and ok with that. That's how I would expect it to behave. (Except for not showing the dialog at all for any site which has 0 updates, and if the metadata is retrieved before the job(s) is/are created.)

If there was an option to turn this one right now, I would happily use it.

Quote:
Originally Posted by JimmXinu View Post
And a user can do that right now manually if they want.
I know, and I sometimes do that when I know that some of them will take a long time. But this is way too fiddly, having to carefully select different books for starting the jobs, for the "daily update" run.


Btw, the download changes now result in something like:
Job: Download 6 FanFiction Book(s)
Status: 23 of 63 stories finished downloading

That is, books that have already been determined before the job was created not to require updating (otherwise it couldn't say "Download x FanFiction Book(s)" are still included in the actual job progress.
Ser4nb2LUY6e is offline   Reply With Quote
Old 02-06-2021, 09:24 PM   #5405
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 10,994
Karma: 75337983
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
Quote:
Originally Posted by Ser4nb2LUY6e View Post
I'm aware and ok with that. That's how I would expect it to behave. (Except for not showing the dialog at all for any site which has 0 updates, and if the metadata is retrieved before the job(s) is/are created.)

If there was an option to turn this one right now, I would happily use it..
If it does become an option at some point in the future, i would make it a user choice in personal.ini - separate_process_per_site maybe.
ownedbycats is offline   Reply With Quote
Old 02-06-2021, 11:19 PM   #5406
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 10,994
Karma: 75337983
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
Quote:
Originally Posted by Ser4nb2LUY6e View Post
I know, and I sometimes do that when I know that some of them will take a long time. But this is way too fiddly, having to carefully select different books for starting the jobs, for the "daily update" run..
Assuming I am not misunderstanding this:

I used to do a daily update of all my in-progresses, but after reaching a certain critical mass of them I decided to switch over to emails. (I use drag & drop from Thunderbird instead of FFF's email fetch.) It's a bit nicer and I suggest it if you have an account or don't mind setting them up.

Also I have a custom bool column, a custom column setting for "N"=>#fanficsubbed,n, and a column icon so I actually remember to subscribe.

Last edited by ownedbycats; 02-06-2021 at 11:22 PM.
ownedbycats is offline   Reply With Quote
Old 02-07-2021, 04:39 PM   #5407
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,974
Karma: 4604635
Join Date: Dec 2011
Location: Midwest USA
Device: Kobo Clara Colour running KOReader
New Test Version Posted

2021-02-07
- Don't count continue_on_chapter_error chapters when checking vs 'new chapters'.
- More properly get msg payload fromemail drag and drop. Closes #645
- Count stories downloading in BG better.
- First version of Firefox cache2 reader.

It's much easier to find your cache dir in Firefox: enter about:cache in the address bar and it will tell you. Ends in 'cache2', like:

C:\Users\youruser\AppData\Local\Mozilla\Firefox\Pr ofiles\ZjwI7Fo4.default\cache2

The Firefox cache reader has some performance improvements I may also apply to the Chrome Simple cache reader.
JimmXinu is offline   Reply With Quote
Old 02-08-2021, 01:59 PM   #5408
steltek
Connoisseur
steltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura about
 
Posts: 69
Karma: 4484
Join Date: Jan 2016
Device: none
I can verify that it works fine when downloading new fics utilizing the browser cache in the most recent versions of Firefox, Waterfox 3rd gen, and Waterfox Classic.

Whatever you did performance improvement-wise causes it to run far faster for me than it did in either Chrome or Vivaldi.
steltek is offline   Reply With Quote
Old 02-08-2021, 03:00 PM   #5409
tricklem
Connoisseur
tricklem began at the beginning.
 
tricklem's Avatar
 
Posts: 60
Karma: 10
Join Date: Feb 2015
Device: none
I have my FireFox browser cache location, but I can't seem to find where I would put that info on the new plug in. Can anyone point me in the right direction?
tricklem is offline   Reply With Quote
Old 02-08-2021, 03:37 PM   #5410
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,974
Karma: 4604635
Join Date: Dec 2011
Location: Midwest USA
Device: Kobo Clara Colour running KOReader
New Test Version Posted

2021-02-08
- Restore multi-process background jobs but as one process per site.


FFF BG processes still limited by Calibre's max BG jobs setting (Preferences->Misc) because I don't want to bother making my own setting.

Multi-threading was fast, but something about Calibre's word count (which launches yet another process) would fail once in awhile--even when protected by a thread lock.
JimmXinu is offline   Reply With Quote
Old 02-08-2021, 03:44 PM   #5411
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 10,994
Karma: 75337983
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
I can confirm the fix to #645 works.
ownedbycats is offline   Reply With Quote
Old 02-08-2021, 05:05 PM   #5412
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,974
Karma: 4604635
Join Date: Dec 2011
Location: Midwest USA
Device: Kobo Clara Colour running KOReader
New Test Version Posted (again)

2021-02-08
- Restore multi-process background jobs as one process per site.
- Add scandir & stat checking for simplecache for performance.

Went back and added the same performance improvements for Chrome Simple Cache. Chrome Blockfile Cache can't benefit from the same changes.

I'll be curious to know if this improves performance for people.
JimmXinu is offline   Reply With Quote
Old 02-08-2021, 05:21 PM   #5413
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 10,994
Karma: 75337983
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
I use BG metadata, and I noticed the little lag I got on the initial phase (while queueing metadata, from it access the cache) is gone now.
ownedbycats is offline   Reply With Quote
Old 02-08-2021, 06:24 PM   #5414
steltek
Connoisseur
steltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura aboutsteltek has a spectacular aura about
 
Posts: 69
Karma: 4484
Join Date: Jan 2016
Device: none
Quote:
Originally Posted by tricklem View Post
I have my FireFox browser cache location, but I can't seem to find where I would put that info on the new plug in. Can anyone point me in the right direction?
Go in to configure FFF. Go to the Personal.ini tab, then hit the "Edit Personal.ini" button. When Personal.ini opens up, look for the [defaults] section right near the top. Right below [defaults], enter a new line beginning with browser_cache_path: followed by your cache path (no spaces between the : and your browser cache path).

Then, go to the [overrides] section at the bottom of Personal.ini. Paste the following text below [overrides]:

[www.fanfiction.net]

##the following line tells FFF to use the browser cache when set to true

use_browser_cache:true

##the following line limits FFF to use cache entries no older than the
##number of hours shown (in this case, cached items no 4 hours or less old).

browser_cache_age_limit:4.0

Then, save the changes to Personal.ini using the OK button. Once you have done this, you should be able to browse a fic in Firefox, then run FFF to collect the pages out of the browser cache.
steltek is offline   Reply With Quote
Old 02-08-2021, 06:37 PM   #5415
tricklem
Connoisseur
tricklem began at the beginning.
 
tricklem's Avatar
 
Posts: 60
Karma: 10
Join Date: Feb 2015
Device: none
Quote:
Originally Posted by steltek View Post
Go in to configure FFF. Go to the Personal.ini tab, then hit the "Edit Personal.ini" button. When Personal.ini opens up, look for the [defaults] section right near the top. Right below [defaults], enter a new line beginning with browser_cache_path: followed by your cache path (no spaces between the : and your browser cache path).

Then, go to the [overrides] section at the bottom of Personal.ini. Paste the following text below [overrides]:

[www.fanfiction.net]

##the following line tells FFF to use the browser cache when set to true

use_browser_cache:true

##the following line limits FFF to use cache entries no older than the
##number of hours shown (in this case, cached items no 4 hours or less old).

browser_cache_age_limit:4.0

Then, save the changes to Personal.ini using the OK button. Once you have done this, you should be able to browse a fic in Firefox, then run FFF to collect the pages out of the browser cache.
Thanks! Hopefully I did that right.
tricklem is offline   Reply With Quote
Reply

Tags
fanfiction


Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] KindleUnpack - The Plugin DiapDealer Plugins 523 07-15-2025 06:45 PM
[GUI Plugin] Open With kiwidude Plugins 404 02-21-2025 05:42 AM
[GUI Plugin] Marvin XD Philantrop Plugins 126 01-29-2017 12:48 PM
[GUI Plugin] KiNotes -axel- Plugins 0 07-14-2013 06:39 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 07:41 PM.


MobileRead.com is a privately owned, operated and funded community.