[GUI Plugin] FanFicFare

[bigwoof]

Originally Posted by ownedbycats

While on the Flaresolverr thing anyways, is is possible to check cache first and then 'fall back' to Flaresolverr? Currently I think it's exclusive.

related to this, I found out that if have Flaresolverr 2.x+ enabled, it will not add cover images from the cache -- even if the entire document is in the cache and being served from there.

Took me a while to debug and it maybe working as intended.. just something to note if you leave both (Flaresolverr and Proxy cache) enabled at the same time..

[ownedbycats]

Originally Posted by bigwoof

related to this, I found out that if have Flaresolverr 2.x+ enabled, it will not add cover images from the cache -- even if the entire document is in the cache and being served from there.

Took me a while to debug and it maybe working as intended.. just something to note if you leave both (Flaresolverr and Proxy cache) enabled at the same time..

Yeah, even without both enabled I remember having trouble with cover images -- sometimes even if I deliberately clicked on them to load them into cache, they failed to download and I ended up needing to manually insert them.

Or thanks to ffnet being broken, the cover images broke on the site and it ended up updating to a gen'd cover (thankfully I keep backups, and the recent removal of 'update epub cover?' seems to have stopped this).

[JimmXinu]

Originally Posted by ownedbycats

EDIT: Replacing the %20s with actual spaces seems to work. Perhaps this could be mentioned in the default.ini for future people?

I guess I expected users to just try putting the space in. %%20 should also work.

But I will revisit that comment because allow_unsafe_filename isn't included in plugin-defaults.ini.

Originally Posted by bigwoof

related to this, I found out that if have Flaresolverr 2.x+ enabled, it will not add cover images from the cache -- even if the entire document is in the cache and being served from there.

ffnet covers come in two sizes--small and smaller. Smaller is what you see on the page, when you click it, you see a slightly larger version. That's the one FFF downloads. So to get cover via Browser Cache, you also need to click the cover image. And, BTW, FFF, by default, doesn't download a cover image if it's the same as the author image.

Flaresolverr stopped allowing download of images in FSv2, see Issue #290.

[ownedbycats]

Originally Posted by JimmXinu

I guess I expected users to just try putting the space in. %%20 should also work.

But I will revisit that comment because allow_unsafe_filename isn't included in plugin-defaults.ini.

Yes, I was a bit confused about that because the only other mention of allow_unsafe_filename was in the comments for the default cover.

%20 - that came from copying the file:// url from my browser. Perhaps an additional example showing the actual spaces would show that its possible.

[ownedbycats]

New batch of fanfic updates appeared in my inbox so I got a chance to test this again with both downloaded and gen'd covers.

This setup seems to be working reasonably well. Removing 'old' seems to keep it from overwriting any pre-existing 'Generated' in #fanficcover. I also removed 'default' because it doesn't seem to be used unless default_cover_image is active (is this correct?).

[defaults]

custom_columns_settings: 
 covertype=>#fanficcover
 
 replace_metadata: 
 covertype=>^(specific|first)$=>Downloaded
 covertype=>^force$=>Manual
 
 add_to_extra_valid_entries:,covertype
 
 include_in_covertype:cover_image

[bigwoof]

Originally Posted by JimmXinu

ffnet covers come in two sizes--small and smaller. Smaller is what you see on the page, when you click it, you see a slightly larger version. That's the one FFF downloads. So to get cover via Browser Cache, you also need to click the cover image. And, BTW, FFF, by default, doesn't download a cover image if it's the same as the author image.

Flaresolverr stopped allowing download of images in FSv2, see Issue #290.

yeah, this is the issue. if FSv2 is enabled, it immediately disables cover images. Even if the document is fully fetched from the cache (that is checked before FSv2 is called) and the full cover image is there (because you clicked on it as needed).

just pointing it out to hopefully stop someone else from having to go through every cover image setting to figure out why the cover image is not appearing in the epub even though it is in the cache.. :-)

is it possible to use cover images from the cache if the document is being served from there even if FSv2 is enabled? right now, the disabling happens right at the start of the entire download process. Thanks!

[ownedbycats]

Question: I have include_in_series:series,collections, and a include_metadata_pre on collections so that it only captures a few specific ones for fic swaps and the like.

I believe that pre means it captures before doing the replace_metadatas. So would it be better practice to do replace_metadata on collections itself?

EDIT: Another question: does include_metadata_pre or exclude_metadata_pre run first?

Example: Suppose I have this, so that selected collections are included if there's no series.

include_metadata_pre:
 collections=~(Collection 1|Collection 2)&&series!~^.*$

But then I also have this, to exclude "Some Series"

exclude_metadata_pre:
 series=~Some Series

If a book has both "Collection 1" and "Some Series," what should be the intended result?

[ownedbycats]

EDIT: I figured this out

[JimmXinu]

New Test Version Posted

2022-11-15
- Update metadata caching with dependency invalidating
- Still allow images with use_flaresolverr_proxy if use_browser_cache

This version rewrites the way metadata caching works to track dependencies and invalidated derived values when a metadata entry is updated. It potentially sacrifices speed for improved accuracy in corner cases such as found earlier with cover_image.

Originally Posted by bigwoof

is it possible to use cover images from the cache if the document is being served from there even if FSv2 is enabled? right now, the disabling happens right at the start of the entire download process. Thanks!

I forgot that was actually coded, sorry.

[bigwoof]

Originally Posted by JimmXinu

I forgot that was actually coded, sorry.

Thanks!! :-)

[ownedbycats]

I found a glitch that may be partially the result of FFF. As I'm not entirely sure and didn't want to crosspost it, I posted here:

https://www.mobileread.com/forums/sh....php?p=4274079

Thanks

[oenwolf]

Originally Posted by JimmXinu

New Test Version Posted

2022-09-27
- adapter_adultfanfictionorg: Fixes for site changes.

Still having trouble downloading from adult-fanfiction. I'm running v4.17.0 which has the changes you made in September. Sample link: redacted

Moderator Notice

Please don't include links to NSFW sites in public posts. You can send story links via PM.

[JimmXinu]

Removed NSFW link for you.

What misbehavior are you seeing? Looks like that story downloaded fine for me.

[apm]

I'm getting the same.

Traceback (most recent call last):
  File "/Users/x/.pyenv/versions/3.10.4/bin/fanficfare", line 33, in <module>
    sys.exit(load_entry_point('FanFicFare==4.16.0', 'console_scripts', 'fanficfare')())
  File "/Users/x/.pyenv/versions/3.10.4/lib/python3.10/site-packages/fanficfare/cli.py", line 344, in main
    dispatch(options, urls, passed_defaultsini, passed_personalini, warn, fail)
  File "/Users/x/.pyenv/versions/3.10.4/lib/python3.10/site-packages/fanficfare/cli.py", line 320, in dispatch
    do_download(url,
  File "/Users/x/.pyenv/versions/3.10.4/lib/python3.10/site-packages/fanficfare/cli.py", line 435, in do_download
    adapter.getStoryMetadataOnly()
  File "/Users/x/.pyenv/versions/3.10.4/lib/python3.10/site-packages/fanficfare/adapters/base_adapter.py", line 320, in getStoryMetadataOnly
    self.doExtractChapterUrlsAndMetadata(get_cover=get_cover)
  File "/Users/x/.pyenv/versions/3.10.4/lib/python3.10/site-packages/fanficfare/adapters/adapter_adultfanfictionorg.py", line 221, in doExtractChapterUrlsAndMetadata
    for i, chapter in enumerate(chapters.findAll('a')):
AttributeError: 'NoneType' object has no attribute 'findAll'

[JimmXinu]

Originally Posted by apm

I'm getting the same.

...
    sys.exit(load_entry_point('FanFicFare==4.16.0', 'console_scripts', 'fanficfare')())
...

First, it looks like you are using FFF 4.16.0.

Second, it looks like you are using the python CLI. This forum is technically for Calibre plugin version.

Third, it looks like I failed to upload FFF 4.17.0 to pypi last month. I've uploaded it now. Update and try again.