[GUI Plugin] FanFicFare

[JimmXinu]

You're welcome.

Originally Posted by w4tchdoge

I've done it like this as:
1) FanFiction being in the extratags was not behaviour I wanted
2) I would like to keep open the option of adding site-based extratags

Just be aware that there are some 15 supported sites that clear or change extratags in defaults.ini and those site section settings take precedence over [defaults] in personal.ini. There's an INI File Wiki page that documents the precedence order. You can search defaults.ini for extratags for which sites.

So you have removed FanFiction, but not guaranteed Web Fiction will always be there, your original stated intent.

[JimmXinu]

Sorry, I should have included the solution, too:

[defaults]
extratags:
add_to_extratags:,Web Fiction

This both clears the [defaults] value and adds Web Fiction.

Or rather, A solution. Due to how add_to_X works, you couldn't easily remove 'Web Fiction' for certain sites.

[w4tchdoge]

Originally Posted by JimmXinu

You're welcome.

Just be aware that there are some 15 supported sites that clear or change extratags in defaults.ini and those site section settings take precedence over [defaults] in personal.ini. There's an INI File Wiki page that documents the precedence order. You can search defaults.ini for extratags for which sites.

So you have removed FanFiction, but not guaranteed Web Fiction will always be there, your original stated intent.

Thank you for bringing this to my attention.

Would I be covering all my bases (that is, both always removing "FanFiction" from extratags and always adding "Web Fiction" to extratags) if I were to implement the following code in my personal.ini?

[defaults]
extratags:

[overrides]
add_to_extratags:,Web Fiction

[JimmXinu]

Originally Posted by w4tchdoge

Would I be covering all my bases (that is, both always removing "FanFiction" from extratags and always adding "Web Fiction" to extratags) if I were to implement the following code in my personal.ini?

Yes.

(Due to how add_to_X works, that is functionally equivalent to what I showed with both lines under [defaults]. The only difference would be order which doesn't matter for Calibre tags--they all get alphabetized.)

You can add additional tags for specific sites the same way, with add_to_extratags under the section for that site.

[ipaqi]

Hey, Jimm, something weird with FFF + Flaresolverr happening lately to me, if I try to use FFF to get stories from FF.net and AO3 in the same update, it looks like you're passing the AO3 cookies to Flaresolverr, or at least passing them with the AO3 domain, which causes FF.net to reject the cookie.

I'm attaching the Flaresolverr log, because I'm not sure what else to send you. If you need something from FFF, I can do that, too.

[JimmXinu]

New Test Version Posted

2023-07-22
- AO3 fix for protected email addr in chapter name breaking chapter datetime

Reported over on the FFF google list.

https://archiveofourown.org/works/46176928

What an obscure little error.

AO3 doesn't allow authors to put HTML tags in chapter titles (they get escaped automatically), so I thought it was safe to assume the only <span> in chapter title would always be the chapter date.

But apparently the email address protect code can also appear there.

Fixed in posted test version.

FYI, the chapter title causing the issue will appear as "The Gunner's Dream 1.z (supplemental;[email protected],[email protected])". There's nothing FFF can do about that.

[JimmXinu]

Originally Posted by ipaqi

Hey, Jimm, something weird with FFF + Flaresolverr happening lately to me, if I try to use FFF to get stories from FF.net and AO3 in the same update, it looks like you're passing the AO3 cookies to Flaresolverr, or at least passing them with the AO3 domain, which causes FF.net to reject the cookie.

FFF saves cookies between requests (because cookies) and passes them to flaresolverr because it's the browser / requester in this case.

FFF doesn't isolate cookies between domains because the HTTP library handles it.

Flaresolverr, apparently, does not. I would have expected that to come 'for free' with the headless browser instance.

I'll look into this.

[JimmXinu]

Attached is a test (test) version that filters cookies for flaresolverr.

I haven't been able to test it fully myself because Flaresolverr "v3.2.2 (Windows-only hotfix)" trips a trojan warning in Windows defender for me so I'm not using it. And v3.2.1 doesn't appear to work anymore with ffnet.

This can be tested by trying to download/update stories from other sites (AO3, TtH, SB/SV, whatever--everybody uses cookies) before a ffnet story (with flaresolverr configured) in the same download/update.

Debug output would be useful.

EDIT: Nobody said it exploded, so rolled into the usual test version.

[ownedbycats]

Originally Posted by JimmXinu

New Test Version Posted

2023-07-22
- AO3 fix for protected email addr in chapter name breaking chapter datetime

Reported over on the FFF google list.

What does 'breaking chapter datetime' mean? The last-updated date would fail?

[JimmXinu]

Complete failure to capture metadata leading to download failure.

Status
Title
Author
Comment
URL
Error
Unknown
Unknown
time data 'email\xa0protected' does not match format '%Y-%m-%d'
https://archiveofourown.org/works/46176928

https://groups.google.com/g/fanfic-d...m/emm9HNYoCgAJ

[gand3r]

no idea if this has been mentioned before, but requests_toolbelt has a problem with incompatibility to >urllib3-2.0.0 which requires an up-to-date version of kfp to fix. after downgrading urllib3 it works again. with the new version fff simply vanished and refused to configure and in debug-mode I get:

Traceback (most recent call last):
File "/home/p/.config/calibre/plugins/FanFicFare.zip/requests_toolbelt/_compat.py", line 48, in <module>
from requests.packages.urllib3.contrib import appengine as gaecontrib
ImportError: cannot import name 'appengine' from 'requests.packages.urllib3.contrib' (/usr/lib/python3.11/site-packages/urllib3/contrib/__init__.py)

During handling of the above exception, another exception occurred:

...

from .. import configurable as configurable
File "/home/p/.config/calibre/plugins/FanFicFare.zip/fanficfare/configurable.py", line 38, in <module>
from . import fetchers
File "/home/p/.config/calibre/plugins/FanFicFare.zip/fanficfare/fetchers/__ini
t__.py", line 19, in <module>
from .fetcher_cloudscraper import CloudScraperFetcher
File "/home/p/.config/calibre/plugins/FanFicFare.zip/fanficfare/fetchers/fetch
er_cloudscraper.py", line 22, in <module>
import cloudscraper
File "/home/p/.config/calibre/plugins/FanFicFare.zip/cloudscraper/__init__.py", line 14, in <module>
from requests_toolbelt.utils import dump
File "/home/p/.config/calibre/plugins/FanFicFare.zip/requests_toolbelt/__init__.py", line 12, in <module>
from .adapters import SSLAdapter, SourceAddressAdapter
File "/home/p/.config/calibre/plugins/FanFicFare.zip/requests_toolbelt/adapters/__init__.py", line 12, in <module>
from .ssl import SSLAdapter
File "/home/p/.config/calibre/plugins/FanFicFare.zip/requests_toolbelt/adapters/ssl.py", line 16, in <module>
from .._compat import poolmanager
File "/home/p/.config/calibre/plugins/FanFicFare.zip/requests_toolbelt/_compat
.py", line 50, in <module>
from urllib3.contrib import appengine as gaecontrib
ImportError: cannot import name 'appengine' from 'urllib3.contrib' (/usr/lib/python3.11/site-packages/urllib3/contrib/__init__.py)

[julo]

'Shiloh' from SciFiStories does not download with this error

Status Error
Title Unknown
Author Unknown
Comment list index out of range
URL https://scifistories.com/s/1029/shiloh

also #8 on:
https://scifistories.com/universe/3/damsels-in-distress

Could it be related since this has two authors they left the author field blank?

[JimmXinu]

Originally Posted by julo

'Shiloh' from SciFiStories does not download with this error

also #8 on:
https://scifistories.com/universe/3/damsels-in-distress

Could it be related since this has two authors they left the author field blank?

Yeah, that's not going to work with no author link. The adapter gets metadata from the author page rather than the story page because the story page doesn't have all the details.

Unless you can point to several stories like this, I'm going to declare this story a site bug that FFF isn't going to address.

[JimmXinu]

Originally Posted by gand3r

no idea if this has been mentioned before, but requests_toolbelt has a problem with incompatibility to >urllib3-2.0.0 which requires an up-to-date version of kfp to fix. after downgrading urllib3 it works again. with the new version fff simply vanished and refused to configure and in debug-mode I get:

I admit I haven't updated the bundled versions of dependencies included in the FFF zip for a while, but as far as I know everything is still working with the versions included in Calibre.

This looks very much like you are using some vendor packaged version of Calibre and possibly FFF plugin as well. I don't support those, sorry.

[chicleeblair]

Okay, so, I’ve standardized all Ons to appear as Original Character. I’ve also got all the characters I’d usually come across in

characters=~^(CHARACTER)&&ships=~^(CHARACTER)$ tags to keep them from duplicating in my tags. I’ve also got it set to only include / ships

What I’d like to be able to do is have Canon/OC ships appear as characters instead—unless the canon character is already in a ship.

So, basically, if I have a fic that’s Spike/Buffy Summers, Buffy Summers/Original Character, I’d want the Original Character to appear as a character. If the only ship was Spike/Original character, then I’d want them both to show up as characters. (the work would also be tagged Canon/OC Ship, I’m just trying to Lower the number of tags.)

It might not be possible, but I’m only starting to get how regexs really work, so thought I’d put it out there.