[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by w4tchdoge

I'm sorry if I'm about to ask an obvious question which has already been answered (apologies if it has been answered, but I haven't been able to find it) but I'm currently trying to append "Web Fiction" as a default tag to the tags the FanFicFare plugin writes to the metadata of the file it outputs.

There's an easier way using extratags. The default (for most sites) is extratags: FanFiction, you could:

# Keep FanFiction from defaults:
[defaults]
add_to_extratags:,Web Fiction

# or set for all sites, replacing both default FanFiction and 
# sites with no extratags set.
[overrides]
extratags:Web Fiction

Originally Posted by culytera

You forgot to retrieve the group that matched your regex. It should be:

 web_fic_tag=>^(.*)=>Web Fiction \1

That would append to every tag. I believe w4tchdoge only wants one additional tag.

[culytera]

Yes I misunderstood their issue.

[w4tchdoge]

Originally Posted by JimmXinu

There's an easier way using extratags. The default (for most sites) is extratags: FanFiction, you could:

# Keep FanFiction from defaults:
[defaults]
add_to_extratags:,Web Fiction

# or set for all sites, replacing both default FanFiction and 
# sites with no extratags set.
[overrides]
extratags:Web Fiction

Thank you very much for your input. I have now gotten my desired behaviour by adding extratags to the defaults in personal.ini like so:

[defaults]
extratags:Web Fiction

I've done it like this as:
1) FanFiction being in the extratags was not behaviour I wanted
2) I would like to keep open the option of adding site-based extratags

Once again, thank you very much for your help. I would also like to thank you for making this plugin as it is going to make the archival of the works I read into one place where I can easily sort them much easier.

[JimmXinu]

You're welcome.

Originally Posted by w4tchdoge

I've done it like this as:
1) FanFiction being in the extratags was not behaviour I wanted
2) I would like to keep open the option of adding site-based extratags

Just be aware that there are some 15 supported sites that clear or change extratags in defaults.ini and those site section settings take precedence over [defaults] in personal.ini. There's an INI File Wiki page that documents the precedence order. You can search defaults.ini for extratags for which sites.

So you have removed FanFiction, but not guaranteed Web Fiction will always be there, your original stated intent.

[JimmXinu]

Sorry, I should have included the solution, too:

[defaults]
extratags:
add_to_extratags:,Web Fiction

This both clears the [defaults] value and adds Web Fiction.

Or rather, A solution. Due to how add_to_X works, you couldn't easily remove 'Web Fiction' for certain sites.

[w4tchdoge]

Originally Posted by JimmXinu

You're welcome.

Just be aware that there are some 15 supported sites that clear or change extratags in defaults.ini and those site section settings take precedence over [defaults] in personal.ini. There's an INI File Wiki page that documents the precedence order. You can search defaults.ini for extratags for which sites.

So you have removed FanFiction, but not guaranteed Web Fiction will always be there, your original stated intent.

Thank you for bringing this to my attention.

Would I be covering all my bases (that is, both always removing "FanFiction" from extratags and always adding "Web Fiction" to extratags) if I were to implement the following code in my personal.ini?

[defaults]
extratags:

[overrides]
add_to_extratags:,Web Fiction

[JimmXinu]

Originally Posted by w4tchdoge

Would I be covering all my bases (that is, both always removing "FanFiction" from extratags and always adding "Web Fiction" to extratags) if I were to implement the following code in my personal.ini?

Yes.

(Due to how add_to_X works, that is functionally equivalent to what I showed with both lines under [defaults]. The only difference would be order which doesn't matter for Calibre tags--they all get alphabetized.)

You can add additional tags for specific sites the same way, with add_to_extratags under the section for that site.

[ipaqi]

Hey, Jimm, something weird with FFF + Flaresolverr happening lately to me, if I try to use FFF to get stories from FF.net and AO3 in the same update, it looks like you're passing the AO3 cookies to Flaresolverr, or at least passing them with the AO3 domain, which causes FF.net to reject the cookie.

I'm attaching the Flaresolverr log, because I'm not sure what else to send you. If you need something from FFF, I can do that, too.

[JimmXinu]

New Test Version Posted

2023-07-22
- AO3 fix for protected email addr in chapter name breaking chapter datetime

Reported over on the FFF google list.

https://archiveofourown.org/works/46176928

What an obscure little error.

AO3 doesn't allow authors to put HTML tags in chapter titles (they get escaped automatically), so I thought it was safe to assume the only <span> in chapter title would always be the chapter date.

But apparently the email address protect code can also appear there.

Fixed in posted test version.

FYI, the chapter title causing the issue will appear as "The Gunner's Dream 1.z (supplemental;[email protected],[email protected])". There's nothing FFF can do about that.

[JimmXinu]

Originally Posted by ipaqi

Hey, Jimm, something weird with FFF + Flaresolverr happening lately to me, if I try to use FFF to get stories from FF.net and AO3 in the same update, it looks like you're passing the AO3 cookies to Flaresolverr, or at least passing them with the AO3 domain, which causes FF.net to reject the cookie.

FFF saves cookies between requests (because cookies) and passes them to flaresolverr because it's the browser / requester in this case.

FFF doesn't isolate cookies between domains because the HTTP library handles it.

Flaresolverr, apparently, does not. I would have expected that to come 'for free' with the headless browser instance.

I'll look into this.

[JimmXinu]

Attached is a test (test) version that filters cookies for flaresolverr.

I haven't been able to test it fully myself because Flaresolverr "v3.2.2 (Windows-only hotfix)" trips a trojan warning in Windows defender for me so I'm not using it. And v3.2.1 doesn't appear to work anymore with ffnet.

This can be tested by trying to download/update stories from other sites (AO3, TtH, SB/SV, whatever--everybody uses cookies) before a ffnet story (with flaresolverr configured) in the same download/update.

Debug output would be useful.

EDIT: Nobody said it exploded, so rolled into the usual test version.

[ownedbycats]

Originally Posted by JimmXinu

New Test Version Posted

2023-07-22
- AO3 fix for protected email addr in chapter name breaking chapter datetime

Reported over on the FFF google list.

What does 'breaking chapter datetime' mean? The last-updated date would fail?

[JimmXinu]

Complete failure to capture metadata leading to download failure.

Status
Title
Author
Comment
URL
Error
Unknown
Unknown
time data 'email\xa0protected' does not match format '%Y-%m-%d'
https://archiveofourown.org/works/46176928

https://groups.google.com/g/fanfic-d...m/emm9HNYoCgAJ

[gand3r]

no idea if this has been mentioned before, but requests_toolbelt has a problem with incompatibility to >urllib3-2.0.0 which requires an up-to-date version of kfp to fix. after downgrading urllib3 it works again. with the new version fff simply vanished and refused to configure and in debug-mode I get:

Traceback (most recent call last):
File "/home/p/.config/calibre/plugins/FanFicFare.zip/requests_toolbelt/_compat.py", line 48, in <module>
from requests.packages.urllib3.contrib import appengine as gaecontrib
ImportError: cannot import name 'appengine' from 'requests.packages.urllib3.contrib' (/usr/lib/python3.11/site-packages/urllib3/contrib/__init__.py)

During handling of the above exception, another exception occurred:

...

from .. import configurable as configurable
File "/home/p/.config/calibre/plugins/FanFicFare.zip/fanficfare/configurable.py", line 38, in <module>
from . import fetchers
File "/home/p/.config/calibre/plugins/FanFicFare.zip/fanficfare/fetchers/__ini
t__.py", line 19, in <module>
from .fetcher_cloudscraper import CloudScraperFetcher
File "/home/p/.config/calibre/plugins/FanFicFare.zip/fanficfare/fetchers/fetch
er_cloudscraper.py", line 22, in <module>
import cloudscraper
File "/home/p/.config/calibre/plugins/FanFicFare.zip/cloudscraper/__init__.py", line 14, in <module>
from requests_toolbelt.utils import dump
File "/home/p/.config/calibre/plugins/FanFicFare.zip/requests_toolbelt/__init__.py", line 12, in <module>
from .adapters import SSLAdapter, SourceAddressAdapter
File "/home/p/.config/calibre/plugins/FanFicFare.zip/requests_toolbelt/adapters/__init__.py", line 12, in <module>
from .ssl import SSLAdapter
File "/home/p/.config/calibre/plugins/FanFicFare.zip/requests_toolbelt/adapters/ssl.py", line 16, in <module>
from .._compat import poolmanager
File "/home/p/.config/calibre/plugins/FanFicFare.zip/requests_toolbelt/_compat
.py", line 50, in <module>
from urllib3.contrib import appengine as gaecontrib
ImportError: cannot import name 'appengine' from 'urllib3.contrib' (/usr/lib/python3.11/site-packages/urllib3/contrib/__init__.py)

[julo]

'Shiloh' from SciFiStories does not download with this error

Status Error
Title Unknown
Author Unknown
Comment list index out of range
URL https://scifistories.com/s/1029/shiloh

also #8 on:
https://scifistories.com/universe/3/damsels-in-distress

Could it be related since this has two authors they left the author field blank?