Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 11-14-2022, 02:08 AM   #7741
bigwoof
Enthusiast
bigwoof began at the beginning.
 
Posts: 42
Karma: 10
Join Date: Jun 2020
Device: Kobo Libra H2O & Kindle Oasis v1
Quote:
Originally Posted by ownedbycats View Post
While on the Flaresolverr thing anyways, is is possible to check cache first and then 'fall back' to Flaresolverr? Currently I think it's exclusive.
related to this, I found out that if have Flaresolverr 2.x+ enabled, it will not add cover images from the cache -- even if the entire document is in the cache and being served from there.

Took me a while to debug and it maybe working as intended.. just something to note if you leave both (Flaresolverr and Proxy cache) enabled at the same time..
bigwoof is offline   Reply With Quote
Old 11-14-2022, 02:12 AM   #7742
ownedbycats
Grand Sorcerer
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 5,524
Karma: 28562994
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
Quote:
Originally Posted by bigwoof View Post
related to this, I found out that if have Flaresolverr 2.x+ enabled, it will not add cover images from the cache -- even if the entire document is in the cache and being served from there.

Took me a while to debug and it maybe working as intended.. just something to note if you leave both (Flaresolverr and Proxy cache) enabled at the same time..
Yeah, even without both enabled I remember having trouble with cover images -- sometimes even if I deliberately clicked on them to load them into cache, they failed to download and I ended up needing to manually insert them.

Or thanks to ffnet being broken, the cover images broke on the site and it ended up updating to a gen'd cover (thankfully I keep backups, and the recent removal of 'update epub cover?' seems to have stopped this).

Last edited by ownedbycats; 11-14-2022 at 07:43 AM.
ownedbycats is offline   Reply With Quote
Advert
Old 11-14-2022, 01:50 PM   #7743
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 5,708
Karma: 1776370
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by ownedbycats View Post
EDIT: Replacing the %20s with actual spaces seems to work. Perhaps this could be mentioned in the default.ini for future people?
I guess I expected users to just try putting the space in. %%20 should also work.

But I will revisit that comment because allow_unsafe_filename isn't included in plugin-defaults.ini.

Quote:
Originally Posted by bigwoof View Post
related to this, I found out that if have Flaresolverr 2.x+ enabled, it will not add cover images from the cache -- even if the entire document is in the cache and being served from there.
ffnet covers come in two sizes--small and smaller. Smaller is what you see on the page, when you click it, you see a slightly larger version. That's the one FFF downloads. So to get cover via Browser Cache, you also need to click the cover image. And, BTW, FFF, by default, doesn't download a cover image if it's the same as the author image.

Flaresolverr stopped allowing download of images in FSv2, see Issue #290.
JimmXinu is offline   Reply With Quote
Old 11-14-2022, 02:32 PM   #7744
ownedbycats
Grand Sorcerer
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 5,524
Karma: 28562994
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
Quote:
Originally Posted by JimmXinu View Post
I guess I expected users to just try putting the space in. %%20 should also work.

But I will revisit that comment because allow_unsafe_filename isn't included in plugin-defaults.ini.
Yes, I was a bit confused about that because the only other mention of allow_unsafe_filename was in the comments for the default cover.

%20 - that came from copying the file:// url from my browser. Perhaps an additional example showing the actual spaces would show that its possible.

Last edited by ownedbycats; 11-14-2022 at 06:07 PM.
ownedbycats is offline   Reply With Quote
Old 11-14-2022, 02:42 PM   #7745
ownedbycats
Grand Sorcerer
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 5,524
Karma: 28562994
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
New batch of fanfic updates appeared in my inbox so I got a chance to test this again with both downloaded and gen'd covers.

This setup seems to be working reasonably well. Removing 'old' seems to keep it from overwriting any pre-existing 'Generated' in #fanficcover. I also removed 'default' because it doesn't seem to be used unless default_cover_image is active (is this correct?).

Code:
[defaults]

custom_columns_settings: 
 covertype=>#fanficcover
 
 replace_metadata: 
 covertype=>^(specific|first)$=>Downloaded
 covertype=>^force$=>Manual
 
 add_to_extra_valid_entries:,covertype
 
 include_in_covertype:cover_image

Last edited by ownedbycats; 11-14-2022 at 06:05 PM.
ownedbycats is offline   Reply With Quote
Advert
Old 11-14-2022, 09:11 PM   #7746
bigwoof
Enthusiast
bigwoof began at the beginning.
 
Posts: 42
Karma: 10
Join Date: Jun 2020
Device: Kobo Libra H2O & Kindle Oasis v1
Quote:
Originally Posted by JimmXinu View Post

ffnet covers come in two sizes--small and smaller. Smaller is what you see on the page, when you click it, you see a slightly larger version. That's the one FFF downloads. So to get cover via Browser Cache, you also need to click the cover image. And, BTW, FFF, by default, doesn't download a cover image if it's the same as the author image.

Flaresolverr stopped allowing download of images in FSv2, see Issue #290.
yeah, this is the issue. if FSv2 is enabled, it immediately disables cover images. Even if the document is fully fetched from the cache (that is checked before FSv2 is called) and the full cover image is there (because you clicked on it as needed).

just pointing it out to hopefully stop someone else from having to go through every cover image setting to figure out why the cover image is not appearing in the epub even though it is in the cache.. :-)

is it possible to use cover images from the cache if the document is being served from there even if FSv2 is enabled? right now, the disabling happens right at the start of the entire download process. Thanks!
bigwoof is offline   Reply With Quote
Old 11-14-2022, 09:19 PM   #7747
ownedbycats
Grand Sorcerer
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 5,524
Karma: 28562994
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
Question: I have include_in_series:series,collections, and a include_metadata_pre on collections so that it only captures a few specific ones for fic swaps and the like.

I believe that pre means it captures before doing the replace_metadatas. So would it be better practice to do replace_metadata on collections itself?

EDIT: Another question: does include_metadata_pre or exclude_metadata_pre run first?

Example: Suppose I have this, so that selected collections are included if there's no series.

Code:
include_metadata_pre:
 collections=~(Collection 1|Collection 2)&&series!~^.*$
But then I also have this, to exclude "Some Series"

Code:
exclude_metadata_pre:
 series=~Some Series
If a book has both "Collection 1" and "Some Series," what should be the intended result?

Last edited by ownedbycats; 11-15-2022 at 02:26 AM.
ownedbycats is offline   Reply With Quote
Old 11-14-2022, 10:03 PM   #7748
ownedbycats
Grand Sorcerer
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 5,524
Karma: 28562994
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
EDIT: I figured this out

Last edited by ownedbycats; 11-14-2022 at 10:24 PM.
ownedbycats is offline   Reply With Quote
Old 11-15-2022, 10:52 AM   #7749
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 5,708
Karma: 1776370
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
New Test Version Posted

2022-11-15
- Update metadata caching with dependency invalidating
- Still allow images with use_flaresolverr_proxy if use_browser_cache

This version rewrites the way metadata caching works to track dependencies and invalidated derived values when a metadata entry is updated. It potentially sacrifices speed for improved accuracy in corner cases such as found earlier with cover_image.

Quote:
Originally Posted by bigwoof View Post
is it possible to use cover images from the cache if the document is being served from there even if FSv2 is enabled? right now, the disabling happens right at the start of the entire download process. Thanks!
I forgot that was actually coded, sorry.
JimmXinu is offline   Reply With Quote
Old 11-15-2022, 09:14 PM   #7750
bigwoof
Enthusiast
bigwoof began at the beginning.
 
Posts: 42
Karma: 10
Join Date: Jun 2020
Device: Kobo Libra H2O & Kindle Oasis v1
Quote:
Originally Posted by JimmXinu View Post

I forgot that was actually coded, sorry.
Thanks!! :-)
bigwoof is offline   Reply With Quote
Old 11-16-2022, 07:27 AM   #7751
ownedbycats
Grand Sorcerer
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 5,524
Karma: 28562994
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
I found a glitch that may be partially the result of FFF. As I'm not entirely sure and didn't want to crosspost it, I posted here:

https://www.mobileread.com/forums/sh....php?p=4274079

Thanks
ownedbycats is offline   Reply With Quote
Old 11-17-2022, 08:07 AM   #7752
oenwolf
Junior Member
oenwolf began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Jul 2020
Device: none
Quote:
Originally Posted by JimmXinu View Post
New Test Version Posted

2022-09-27
- adapter_adultfanfictionorg: Fixes for site changes.
Still having trouble downloading from adult-fanfiction. I'm running v4.17.0 which has the changes you made in September. Sample link: redacted

Moderator Notice

Please don't include links to NSFW sites in public posts. You can send story links via PM.

Last edited by JimmXinu; 11-17-2022 at 10:26 AM. Reason: redact NSFW link
oenwolf is offline   Reply With Quote
Old 11-17-2022, 10:29 AM   #7753
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 5,708
Karma: 1776370
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Removed NSFW link for you.

What misbehavior are you seeing? Looks like that story downloaded fine for me.
JimmXinu is offline   Reply With Quote
Old 11-17-2022, 05:03 PM   #7754
apm
Junior Member
apm began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Feb 2016
Device: none
I'm getting the same.

Code:
Traceback (most recent call last):
  File "/Users/x/.pyenv/versions/3.10.4/bin/fanficfare", line 33, in <module>
    sys.exit(load_entry_point('FanFicFare==4.16.0', 'console_scripts', 'fanficfare')())
  File "/Users/x/.pyenv/versions/3.10.4/lib/python3.10/site-packages/fanficfare/cli.py", line 344, in main
    dispatch(options, urls, passed_defaultsini, passed_personalini, warn, fail)
  File "/Users/x/.pyenv/versions/3.10.4/lib/python3.10/site-packages/fanficfare/cli.py", line 320, in dispatch
    do_download(url,
  File "/Users/x/.pyenv/versions/3.10.4/lib/python3.10/site-packages/fanficfare/cli.py", line 435, in do_download
    adapter.getStoryMetadataOnly()
  File "/Users/x/.pyenv/versions/3.10.4/lib/python3.10/site-packages/fanficfare/adapters/base_adapter.py", line 320, in getStoryMetadataOnly
    self.doExtractChapterUrlsAndMetadata(get_cover=get_cover)
  File "/Users/x/.pyenv/versions/3.10.4/lib/python3.10/site-packages/fanficfare/adapters/adapter_adultfanfictionorg.py", line 221, in doExtractChapterUrlsAndMetadata
    for i, chapter in enumerate(chapters.findAll('a')):
AttributeError: 'NoneType' object has no attribute 'findAll'

Last edited by apm; 11-17-2022 at 05:08 PM.
apm is offline   Reply With Quote
Old 11-17-2022, 06:12 PM   #7755
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 5,708
Karma: 1776370
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by apm View Post
I'm getting the same.

Code:
...
    sys.exit(load_entry_point('FanFicFare==4.16.0', 'console_scripts', 'fanficfare')())
...
First, it looks like you are using FFF 4.16.0.

Second, it looks like you are using the python CLI. This forum is technically for Calibre plugin version.

Third, it looks like I failed to upload FFF 4.17.0 to pypi last month. I've uploaded it now. Update and try again.
JimmXinu is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Open With kiwidude Plugins 401 01-03-2023 09:00 AM
[GUI Plugin] KindleUnpack - The Plugin DiapDealer Plugins 492 10-25-2022 09:13 AM
[GUI Plugin] Marvin XD Philantrop Plugins 126 01-29-2017 01:48 PM
[GUI Plugin] KiNotes -axel- Plugins 0 07-14-2013 07:39 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 01:27 PM


All times are GMT -4. The time now is 09:39 AM.


MobileRead.com is a privately owned, operated and funded community.