[GUI Plugin] FanFicFare

[ownedbycats]

I'll ask in the library management subforum.

[ownedbycats]

Another question about metadata: are multiple conditionals possible? I'm guessing not as I don't see anything in the documentation about it.

[ownedbycats]

Originally Posted by steltek

Looking at Twitter their own apps and web services weren't working for some folks today, so who knows what they managed to mess up before they fixed it.

This is what I suspect the ffnet server room looks like:

[davidfor]

Originally Posted by ownedbycats

This is what I suspect the ffnet server room looks like:

Attachment 185194

Honestly, that's relatively neat.

[ownedbycats]

OK, how about this?

[JimmXinu]

Originally Posted by ownedbycats

Another question about metadata: are multiple conditionals possible? I'm guessing not as I don't see anything in the documentation about it.

Not directly.

If you want, you can experiment with combining more than one entry into a new extra entry (using include_in_entryname) and then use that for the conditional (with entryname_LIST).

[ownedbycats]

That makes sense. Thank you.

Good news is that Import List can be used to mark books by their identifiers, in case I ever need to do that in the future.

[JimmXinu]

New Test Version Posted

2021-02-03
- use_browser_cache feature added -- Read fanfiction.net stories from your Chrome browser cache instead of the network. Many thanks to Sidney Markowitz for the stellar help with this.

FanFicFare now has the ability to read your Chrome browser cache for fanfiction.net stories instead of (or in addition to) going out to the ffnet servers for them.

Obviously, the reason to do this is for when FFF is failing due to CloudFlare, but you can still see the story pages manually in your browser.

To use this:

1. Figure out where your Chrome cache directory is and set browser_cache_path under [defaults]. If the notes below aren't sufficient, try google--that's what I'll be doing if you ask me about your OS.
2. Set use_browser_cache:true under [www.fanfiction.net].
3. Navigate to the story you want to download in Chrome.
4. Click the cover image thumbnail to see the bigger version once if you care about covers.
5. Click the author link once if you care about not getting the author image as cover.
6. Click through all the chapters of the story. Or, if you are updating an existing story, click the first and last chapters (or all new chapters).
7. The WebToEpub can be used to spin through the chapters for you when it generates it's epub (the button is 'Pack EPUB').
8. Download story in FanFicFare as normal. There will be a delay as FFF starts to read your browser cache.

Example ini settings:

Spoiler:

[defaults]

## browser_cache_path needs to be set to the location of YOUR Chrome
## browser cache.  Here are a few examples for different OS.  Note
## that all have YOUR user name in them somewhere as a Chrome Profile
## name, frequently Default.  Make sure you have set (and uncommented)
## appropriately.

## Note also that browser_cache_path goes under [defaults] because
## browser cache is read once and shared between all downloads for
## performance.

## Further note that there is a time cost to first reading the browser
## cache on each download session (not Calibre session)

## windows:
#browser_cache_path:C:\Users\YourUser\AppData\Local\Google\Chrome\User Data\Default\Cache
#browser_cache_path:C:\Users\YourUser\AppData\Local\Google\Chrome\User Data\Profile 1\Cache

## mac:
#browser_cache_path:/Users/your.user/Library/Caches/Google/Chrome/Default/Cache 
#browser_cache_path:/Users/your.user/Library/Caches/Google/Chrome/Profile 2/Cache 

## linux:
#browser_cache_path:/home/youruser/.cache/google-chrome/Default/Cache
#browser_cache_path:/home/youruser/.cache/google-chrome/Profile 1/Cache

[www.fanfiction.net]

## browser cache will only be used if use_browser_cache:true and ONLY
## for fanfiction.net right now.  Requires a browser_cache_path set in
## [defaults].
use_browser_cache:true

## use_browser_cache_only:true prevents FFF from falling through to
## request from the server when a page isn't in the browser cache.
## You can turn this on to be sure you're only using the browser
## cache.
use_browser_cache_only:false

## Pre-existing ffnet INI settings you might want to think about
## changing with use_browser_cache:

## If you want covers to work correctly with use_browser_cache, you'll
## need to click the cover image at least once for each story.  Or
## turn cover off.
never_make_cover:true

## If you want skip_author_cover to work correctly with
## use_browser_cache, you'll need to click the author page at least
## once for each story.  Or turn skip_author_cover off.
##
## From defaults.ini:
## April 2015, ffnet changed their story and user pictures urls to be
## the same.  Now the only way to know if the story image is really
## the author image is to go get the author image and check.
skip_author_cover:false

## If you turned this on before, you probably want it off while using
## browser cache.
##
## From defaults.ini:
## ffnet used to have a tendency to send out update notices in email
## before all their servers were showing the update on the first
## chapter.  It generates another server request and doesn't seem to
## be needed lately, so now default it to off.
check_next_chapter:false

Again this only works with Chrome browsers and fanfiction.net (so far). I have, however, tested on Windows, Mac and Linux.

If you want to see what's happening in more detail, or have problems, run Caliber in debug mode to see the debug output.

FYI, I ran out of time this afternoon, so pre-Cal5 versions haven't been tested yet--they worked last time I tried, but code has changed since then.
UPDATE: Just did a quick test in Calibre versions 2, 3 & 4 (in windows) successfully.

[ownedbycats]

Originally Posted by JimmXinu

Figure out where your Chrome cache directory is and set browser_cache_path under [defaults]. If the notes below aren't sufficient, try google--that's what I'll be doing if you ask me about your OS.

Vivaldi (Chromium-based) has its profile path listed in About. Is Chrome not do that?

I am going to test the page cache with Vivaldi though I need to set it up first. Will post results later.

EDIT:

Also, I'm assuming this is another "ran out of time" thing, but I noticed the commented ini code doesn't seem to be present in defaults.ini in 3.99.16.

ANOTHER EDIT:

I tested a fresh download of six-chapter story cached in Vivaldi 3.6.2165.34 using use_browser_cache_only. Worked perfectly.

I then deleted the cache and tried again. Got a "Story does not exist." If possible for FFF to detect that, perhaps a unique error e.g. "this fic is not cached" would work better, otherwise I predict at least one "FFF says it doesn't exist??? help!!!" post.

Also perhaps check for mismatches between the listed # of chapters and how many actually found? Might be more effort than worth it though, and might cause issues if ffnet's still having that metadata lag issue.

[JimmXinu]

Originally Posted by JimmXinu

FYI, I ran out of time this afternoon, so pre-Cal5 versions haven't been tested yet--they worked last time I tried, but code has changed since then.

Newer FYI: Just did a quick test in Calibre versions 2, 3 & 4 (in windows) successfully.

Originally Posted by ownedbycats

Also, I'm assuming this is another "ran out of time" thing, but I noticed the commented ini code doesn't seem to be present in defaults.ini in 3.99.16.

Test version.

Originally Posted by ownedbycats

I then deleted the cache and tried again. Got a "Story does not exist." If possible for FFF to detect that, perhaps a unique error e.g. "this fic is not cached" would work better, otherwise I predict at least one "FFF says it doesn't exist??? help!!!" post.

I'll change that to another status code in the next version.

Originally Posted by ownedbycats

Also perhaps check for mismatches between the listed # of chapters and how many actually found? Might be more effort than worth it though, and might cause issues if ffnet's still having that metadata lag issue.

continue_on_chapter_error:true

[ownedbycats]

Originally Posted by JimmXinu

Test version.

Or that too. Just wanted to make sure you were aware that the absence.

Thanks for all your work!

[steltek]

@JimmXinu,

Just for reference I tried the new browser cache feature and it works perfectly in the both the Vivaldi web browser and Microsoft Edge (both of which are Chromium-based web browsers) for downloading new fics. Next time I get a couple of update notifications, I'll try to use it in updating existing fics.

[JimmXinu]

New Test Version Posted

2021-02-03
- use_browser_cache feature added -- Read fanfiction.net stories from your Chrome browser cache instead of the network. Many thanks to Sidney Markowitz for the stellar help with this.

Added since last post:
- Give 428 code for not found in cache with use_browser_cache_only so doesn't come up StoryDoesNotExist
- Update brotlidecpy for another corner case. Thanks, Sidney!

[ownedbycats]

Originally Posted by steltek

@JimmXinu,

Just for reference I tried the new browser cache feature and it works perfectly in the both the Vivaldi web browser and Microsoft Edge (both of which are Chromium-based web browsers) for downloading new fics. Next time I get a couple of update notifications, I'll try to use it in updating existing fics.

Curious: I've not tested, but the way the example INI looks like it's possible to add multiple paths. What happens if you put paths from two different OSes in? It seems like a plausible use case (e.g. dual-booters, though I'm not sure how many of them use FFF so it may be another corner).

Actually I'll test what happens if you put in the wrong path type.

EDIT:

Commenting out my existing cache, adding a Linux-type path and using the cache override resulted in a Directory does not contain a known browser cache type: 'C:\home\youruser\.cache\google-chrome\Default\Cache.

Uncommenting my existing cache unfortunately still tried to use the Linux one (Windows one was first in the list, if needed for replication) and failed since it didn't exist—unsure whether it only supports one path, or just gave up after seeing one path didn't contain a cache.

Also, in the zip file I found a \fanficfare\browsercache\basebrowsercache.py-compress-fail-save that's around a day older than the basebrowsercache.py. From the extension I suspect it might be a leftover.

[ownedbycats]

Edit: Merged above