[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by dailyalice

Edited to say: Oh, I could just change publisher instead of site and all would be well, I bet.

[ownedbycats]

Originally Posted by dailyalice

Though I just realized that I have a line in my personal ini right now under replace metadata to change site "archiveofourown.org" to "Archive of Our Own" thinking it looks nicer under Publisher in my calibre library.

I made that mistake for a long time (at least three years!): https://www.mobileread.com/forums/sh...postcount=7178

[dailyalice]

Originally Posted by ownedbycats

I made that mistake for a long time (at least three years!): https://www.mobileread.com/forums/sh...postcount=7178

Glad to see I'm not alone! And I appreciate the link.

I had it like that in an old personal.ini I dug up and thought it seemed like a reasonable thing to do, didn't realize it would make a difference. Now I know better (and am glad I learned this before recommending it in my guide).

It's been interesting looking at old personal.ini files and trying to figure out what I was doing and why I did it that way.

[icecold]

So I'm not expecting anything to be done about this because there's are always going to be edge cases for SB/SV/QQ title and category/fandom parsing. Just wanted to report this so you knew what one of those edge cases were.

But anyways FFF mangles the title and fandom for this story, https://forums.sufficientvelocity.co...y-quest.99223/

[JimmXinu]

Originally Posted by icecold

So I'm not expecting anything to be done about this because there's are always going to be edge cases for SB/SV/QQ title and category/fandom parsing. Just wanted to report this so you knew what one of those edge cases were.

But anyways FFF mangles the title and fandom for this story, https://forums.sufficientvelocity.co...y-quest.99223/

Yep, that's a thing.

You can correct the title and clear tagsfromtitle by putting in story specific replacements:

[https://forums.sufficientvelocity.com/threads/the-best-and-worst-time-of-our-lives-a-rwby-quest.99223/]
add_to_replace_metadata:
 title=>^.*$=>Correct Title(with) stuff in (parans) in
 tagsfromtitle=>^.*$=>

[storyUrl] section and add_to_ will make sure these are the last replacement patterns.

[ownedbycats]

Question: Are there any improvements to this regex? It's intended to remove "Day ##:" from various October challenge fics.

## Removing 'Day ##:' from titles
 title=>^Day \d+: =>

I suppose I might need to change it in the future if I see dashes or some other punctuation used in place of colons.

[JimmXinu]

Without a sample set to work from, who knows?

You could use ^Day ?\d+[ \.;:-]* and add more separator chars as you may find them. Remember to keep - last so it doesn't indicate a character range.

[ipaqi]

JimmXinu, I'm following the flaresolverr progress regarding the new way Cloudflare broke it.

Here's the github thread if you care: https://github.com/FlareSolverr/FlareSolverr/issues/536

Anyway, issue is, the only version of flaresolverr that has any workaround right now is the new v3 beta (written in python). It seems to work fine, at least with single page requests, but FFF doesn't seem to be able to read the output properly.

Here's the two relevant bits from the output from FRv3beta2 with the workaround from the linked thread above:

Moderator Notice

Excessively long tech data removed by power of mod -- Jim

Hopefully you can do something with this, if not for right now, then for support once v3 is out of beta

[ipaqi]

Hi, JimmXinu, I've been following the flaresolverr developments regarding the new way cloudflare broke it.

Here's the github thread if it interests you:
https://github.com/FlareSolverr/FlareSolverr/issues/536

Anyway, it looks like with the workaround posted in the thread, the code returns the output properly, but for some reason FFF isn't reading it properly -- the log is this:
Status: Error
Title: Unknown
Author: Unknown
Comment: 'expires'
URL: https://www.fanfiction.net/s/3145203/1/The-End

I'm attaching the jsons I pulled from fiddler for this request so you can look into it. If there's anything I can do on my own as a workaround or a way to change the response structure on my local clone of the code, I'd appreciate you telling me, although I won't be able to change anything actually committed to the github.

Also, on a related note -- since the workaround just skips the internal challenge resolution and changes it to a time.sleep(5), that probably will only work for the initial few requests - is there a way to get FFF to wait, say 120 seconds every 10 chapters?

[JimmXinu]

ipaqi - Please don't post large logs inline like your first post. I edited it for you.

I don't have a current linux install handy to test the flaresolverr beta docker with.

If you run Calibre & FFF in debug mode (under the Calibre Prefs drop down) and post the debug log from that, I might be able to figure something out.

[ipaqi]

Originally Posted by JimmXinu

ipaqi - Please don't post large logs inline like your first post. I edited it for you.

I don't have a current linux install handy to test the flaresolverr beta docker with.

If you run Calibre & FFF in debug mode (under the Calibre Prefs drop down) and post the debug log from that, I might be able to figure something out.

Here, attached. Note that I have also posted just after the message you had to edit (sorry about that, btw) with the request and response bodies as I cought them in Telerik Fiddler. This log is from today, but it's the same request and should be roughly the same body of the response (barring, obviously, date fields).

If you really need the request, response and the debug log from the same call, I'll be able to provide those, too.

Note that the log has two FFF calls, the first one is without flaresolverr activated in FFF (not on purpose, I was just messing around a bit more yesterday), and the second is with flaresolverr activated.

[JimmXinu]

New Test Version Posted

2022-10-14
- Fix site name fanfiction.tenhawkpresents.ink
- Flaresolverr v3 beta using 'expiry' cookie key, was 'expires'.

ipaqi - It looks like your flaresolverr beta has changed the expiration time key for cookies from 'expires' to 'expiry'. This test version will accept either.

Searching the flaresolverr repo doesn't find any mention of such a change, so it may be something underlying that changed.

[ipaqi]

Originally Posted by JimmXinu

New Test Version Posted

2022-10-14
- Fix site name fanfiction.tenhawkpresents.ink
- Flaresolverr v3 beta using 'expiry' cookie key, was 'expires'.

ipaqi - It looks like your flaresolverr beta has changed the expiration time key for cookies from 'expires' to 'expiry'. This test version will accept either.

Searching the flaresolverr repo doesn't find any mention of such a change, so it may be something underlying that changed.

Well, the v3 is a completely new codebase written in python vs the previous node.js code, so it might be missing documentation, or you didn't switch to the v3 beta branch in the repo view.

Regardless, thank you very much.

[ipaqi]

Regrettably, it looks like the fix didn't fix enough. The job now fails with Error message "content-type", which I have to imagine you were pulling from the headers field in the return object. Request and response objects haven't changed.

Apparently the headers and userAgent fields have been deprecated. Can you work with that or do you need any of that info to be passed through some way to FFF?

From the new V1RequestBase constructor:

headers: list = None # deprecated v2.0.0, not used
userAgent: str = None # deprecated v2.0.0, not used

[JimmXinu]

New Test Version Posted

2022-10-14
- Fix site name fanfiction.tenhawkpresents.ink
- Flaresolverr v3 beta using 'expiry' cookie key, was 'expires'.
- Flaresolverr v3 beta doesn't have 'headers'??


FFF looks in resp.json['solution']['headers']['content-type'] for type 'image' for dealing with FSv1 supporting images vs v2 not supporting images.

At the time FSv2 came out, users wanted continued support in FFF for FSv1 specifically for images. I don't know if anyone still does.

This change by-passes that check if 'headers' or 'content-type' don't exist. Hopefully that won't be a problem.