[GUI Plugin] FanFicFare

[Ytierian]

Originally Posted by Ytierian

I'm having the same issue despite updating both Calibre and FFF to the lstest versions....AO3's new security is the problem. Hope it doesn't go the way of FFN...

Whatever it was seems to have been fixed....FFF downloading AO3 books properly now.

Now if I could get someone to succinctly explain how to get this Browser Cache function working so I can directly download FFN stories

Windows 11 Home
Calibre 7.25
FireFox 134.0.2
FanFicFare 4.42.0

[booster-t]

Originally Posted by Ytierian

Whatever it was seems to have been fixed....FFF downloading AO3 books properly now.

Yes ... weird.

Stephen

[JimmXinu]

CloudFlare (or something like it) is very common on virtually all large web sites. And the difference between an automated downloader like FFF and an automated attacker can be subtle.

We've been dealing with it for several years now on various sites--I just don't have the energy to get very excited about it anymore.

Originally Posted by booster-t

UPDATE: I can still download from AO3 by selecting "entire story" and then downloading from AO3, but that is a pain in the butt to have to rename them and then move them into Calibre's library.

Or instead use their built-in download epub. That should have title/author in it and not need renaming of anything.

Originally Posted by Ytierian

Now if I could get someone to succinctly explain how to get this Browser Cache function working so I can directly download FFN stories

It's been discussed here several times, and there's a wiki page. I wouldn't call it "directly download", though.

[bpothier]

Currently on Calibre 7.24 on Windows.
Not entirely sure what the issue is, but I have had this issue sporadically for a while across multiple Calibre and FFF versions(since Calibre 5.x?).

After clicking "Yes" to the "FFF download complete / Proceed with updating your library?" dialog, the "Updated X/X" dialog will appear and progress, but sometimes the entire Calibre UI will become unresponsive - to the point of force quitting app - even after waiting for an hour.
When Calibre loads again, the library/database updates are actually complete.
I run a database check, and no issues found.
I have tried running Calibre with debug and nothing jumped out.

But... I think I found a "workaround" to avoid it - which may also point to the actual issue?
After clicking the "Yes" / update library dialog for FFF...
*DO NOT CLICK ON ANY CALIBRE UI WINDOW!*
... until progress dialog window completes. Do not minimize it, restore it or anything... It seems it may be related to the progress dialog window and main UI window getting "stuck" somehow?

[ownedbycats]

That happens to me on occasion. I hear error noises when clicking like its behind a window or somethijg.

If you had Calibre in system tray you can restart from there. Not much else.

[bpothier]

Exactly.. it happens sometimes.. no apparent rhyme or reason.. but.. since (ok, only a few days/week) I have completely avoided clicking on any Calibre UI window (I now use "hover" over side library filter items as a way to check for response) it has not happened...

[ownedbycats]

I dont touch anything and it happens though.

[JimmXinu]

Originally Posted by bpothier

After clicking "Yes" to the "FFF download complete / Proceed with updating your library?" dialog, the "Updated X/X" dialog will appear and progress, but sometimes the entire Calibre UI will become unresponsive - to the point of force quitting app - even after waiting for an hour.

I've seen something like that happen once in great while, but never with any useful information or replication--and I run Calibre in debug mode 99% of the time.

I have vague recollections of discussing this or something similar with Kovid once and we thought it was Qt issue?

Originally Posted by bpothier

But... I think I found a "workaround" to avoid it - which may also point to the actual issue?
After clicking the "Yes" / update library dialog for FFF...
*DO NOT CLICK ON ANY CALIBRE UI WINDOW!*
... until progress dialog window completes. Do not minimize it, restore it or anything... It seems it may be related to the progress dialog window and main UI window getting "stuck" somehow?

If you (or anyone else) have this happen often enough to be able to test it, I could make some test versions that either dump even more debug to trace where it stops, or a version that doesn't use a progress dialog and see if that helps.

[bpothier]

Heh... if you're extra motivated for adding debug options... every once in a great while I run across a page that fails to process for unknown reasons. I have to go in and add in a bunch of print/debug lines and repackage the plugin to figure out what broke where... Would be nice to have a VERBOSE=YES or VERBOSE=9 in FFF options to enable those super-verbose messages... The last one I think I had was an SOL one where the site had leftover extraneous HTML (extra close DIV or something? I forget exactly..) in that specific story's description paragraph that I sent feedback there to cleanup and it worked after that... I think I still have one that was producing an error but I have not looked into it deeper yet..

[ThreeLizards]

Hi! So I'm modifying stuff in my personal.ini file. And I'm trying to set it up to replace relationship names.

This is what I have below

Spoiler:

replace_metadata:
## Ships
ships, relationships,ships_LIST,prefix_ships_LIST=^Din Djarin/Boba Fett$=BobaDin

And none of them work. How do I get it to not have the full long name? It'd be great if I don't have to do a find and replace every time I'm downloading fanfics.

[JimmXinu]

Originally Posted by ThreeLizards

Hi! So I'm modifying stuff in my personal.ini file. And I'm trying to set it up to replace relationship names.

This is what I have below

replace_metadata: 
## Ships
 ships, relationships,ships_LIST,prefix_ships_LIST=^Din Djarin/Boba Fett$=BobaDin

And none of them work. How do I get it to not have the full long name? It'd be great if I don't have to do a find and replace every time I'm downloading fanfics.

For one thing, you're missing the '>' characters. Should be more like:

ships=>^Din Djarin/Boba Fett$=>BobaDin

If we assume that's just a typo from copying it here (since I wrote all this before noticing the '>'s):

relationships and prefix_ships are not FFF defined entries. Unless you added them, they are unnecessary. You also shouldn't need the _LIST entries if you change it in ships.

So, either your pattern doesn't match your stories (perhaps you have sort_ships:true?), you have other replacements interfering, or you have it in the wrong part of personal.ini.

I'd need to see a few story URL examples and a "Safe" copy of your personal.ini (there's a button for that) to help any further.

[ThreeLizards]

OMG! Thank you! That worked. I swear I had the > in at one point but the code didn't work so I took them out.

Also, I was looking at someone else's code... culterya? And they had something about Dragon Age and only showing the specific dragon age game. I was wondering if there was like a reverse of that? If I could just have it show me only Dragon Age if it finds Dragon Age.

Kinda like this one that you had in the wiki

Spoiler:

category=> \(?(- All Media Types|- Fandom|Movies|Movie [0-9]+|Comics|TV|Video Games?)\)?$=>

Could I tell the program to ignore anything after Dragon Age? or Mass Effect for that matter?

[JimmXinu]

Originally Posted by ThreeLizards

Also, I was looking at someone else's code... culterya? And they had something about Dragon Age and only showing the specific dragon age game. I was wondering if there was like a reverse of that? If I could just have it show me only Dragon Age if it finds Dragon Age.

...

Could I tell the program to ignore anything after Dragon Age? or Mass Effect for that matter?

Sure, that's a common usage. Something like:

# separate lines for ease of maintenance 
 category=>^(Dragon Age).*$=>\1
 category=>^(Mass Effect).*$=>\1
# or single line for both
 category=>^(Dragon Age|Mass Effect).*$=>\1

It probably takes a bit of searching, but many such patterns have been discussed here over the years.

[zeusosphere]

I've been having an issue with FFN for a while (within the last several weeks) where the fic is updated but every single new chapter except the first fails. I've updated Calibre to 7.24, Fanficfare to the latest version, restarted Calibre and Firefox (my default browser), cleared Firefox's FFN cache, and set the browser cache age limit to -1. The only change has been for the worse, because now FFF doesn't open the webpages automatically on Firefox anymore and just keeps giving me 'Page not found or expired' errors despite me manually opening the page in Firefox anew. Please help!

Below are my browser cache and FFN-related settings:

Spoiler:

browser_cache_path:/Users/redacted/Library/Caches/Firefox/Profiles/y4x4faz1.default-1671202703436/cache2

# It's common for browser cached files to be kept for several hours
## or more. You can limit the age of cached files FFF will use from
## browser_cache_path with browser_cache_age_limit. Only cached files
## that were downloaded within 'browser_cache_age_limit' hours will be
## used. If set to -1, all cached files will be used.
browser_cache_age_limit:-1

## If browser_cache_path is set *and* use_browser_cache:true *and*
## use_browser_cache_only:true, then you can also set
## open_pages_in_browser:true then FFF to attempt to open each page it
## can't already find in browser cache in your default browser, then
## check for it in the cache again. Note that your browser_cache_path
## setting *must* use your default browser for this to work.
open_pages_in_browser:true

## Because some adapters can pull chapter URLs from human posts, the
## odds of errors in the chapter URLs can be higher for some
## sites/stories. You can set continue_on_chapter_error:true to
## continue on after failing to download a chapter and instead record
## an error message in the ebook for that chapter.
continue_on_chapter_error:true


## Append this to chapter titles that errored. Only used with
## continue_on_chapter_error:true
## Set empty to not mark failed chapters.
chapter_title_error_markCHAPTER ERROR)

[www.fanfiction.net]
## use_basic_cache:true
## Using cloudscraper can satisfy the first couple levels of
## Cloudflare bot-proofing, but not all levels. Older versions of
## OpenSSL will also raise problems, so versions of Calibre older than
## v5 will probably fail. Only a few sites are configured with
## use_cloudscraper:true by default, but it can be applied in other
## sites' ini sections. user_agent setting is ignored when
## use_cloudscraper:true

## Errors on more common during long downloads. This way, at least
## the chapters the worked are saved. Often running an update will
## fix the error chapters.
continue_on_chapter_error:true

## browser cache will only be used if use_browser_cache:true and ONLY
## for a few sites. Requires a browser_cache_path set in
## [defaults].
use_browser_cache:true

## use_browser_cache_only:true prevents FFF from falling through to
## request from the server when a page isn't in the browser cache.
## You can turn this on to be sure you're only using the browser
## cache.
use_browser_cache_only:true

## If browser_cache_path is set *and* use_browser_cache:true *and*
## use_browser_cache_only:true, then you can also set
## open_pages_in_browser:true then FFF to attempt to open each page it
## can't already find in browser cache in your default browser, then
## check for it in the cache again. Note that your browser_cache_path
## setting *must* use your default browser for this to work.
open_pages_in_browser:true

[ThreeLizards]

Thank you for your help!

I did a google search for anything in containing personal.ini. Do you have a suggestion on words to use to search to find more information that y'all might've discussed over the years?

Also, another question for you, I typically only care about one pairing in the story and the other side pairings don't matter and basically could clutter up stuff. Is there a way that I could exclude these "extra" pairings?

(Like if they aren't 'in' this list, don't include them?)

Spoiler:

add_to_extra_valid_entries:,prefix_ships,ships_mod ified

include_in_prefix_ships:ships

include_in_ships_modified:ships


replace_metadata:

## Ships
ships_modified=>^Din Djarin/Boba Fett$=>BobaDin
ships_modified=>^Alexsandr Kallus/Garazeb "Zeb" Orrelios$=>Kalluzeb
ships_modified=>^Bilbo Baggins/Thorin Oakenshield$=>Bagginshield
ships_modified=>^Castiel/Dean Winchester$=>Destiel
ships_modified=>^Cullen Rutherford/Female Trevelyan$=>CullenInqy
ships_modified=>^Dorian Pavus/Cullen Rutherford$=>Cullrian
ships_modified=>^Derek Hale/Stiles Stilinski$=>Sterek
ships_modified=>^Female Shepard/Garrus Vakarian$=>Shakarian
ships_modified=>^Fenris/Female Hawke \(Dragon Age\)$=>FenHawke
ships_modified=>^Frodo Baggins/Sam Gamgee$=>FroSam
ships_modified=>^Geralt z Rivii \| Geralt of Rivia/Jaskier \| Dandelion$=>Geraskier
ships_modified=>^James T. Kirk/Spock$=>Spirk
ships_modified=>^Jango Fett/Obi-Wan Kenobi$=>JangObi
ships_modified=>^CC-2224 \| Cody/Obi-Wan Kenobi$=>Codywan
ships_modified=>^Jayce/Viktor \(League of Legends\)$=>JayVik
ships_modified=>^John "Soap" MacTavish/Simon "Ghost" Riley$=>GhostSoap
ships_modified=>^Peter Parker/Wade Wilson$=>SpideyPool
ships_modified=>^Steve McGarrett/Danny "Danno" Williams$=>McDanno
ships_modified=>^Steve Rogers/Tony Stark$=>Stony
ships_modified=>^Caitlyn/Vi \(League of Legends\)$=>CaitVi
ships_modified=>^Jayce \(League of Legends\)/Mel Medarda$=>MelJay
add_to_custom_columns_settings:
ships_modified=>#pairings

Also thank you again for all your help so far! I've edited the personal.ini file a lot!

With your help, stuff that culytera has posted, and stuff in the github, I edited it so that only the ship name is changed in the custom column and appears as the full name in the actual fic document!