[GUI Plugin] FanFicFare

[Snomad]

Hi Jimm, I'm running into an error when downloading a fic from scribblehub. I thought that it was because I was getting IP banned again, but I proxied into the connection that flaresolver is using and it can still access scribblehub without issue. I've attached the logs and these are my current flaresolver settings:
[www.scribblehub.com]
use_basic_cache:false
extra_valid_entries:views, averageWords, fandoms
views_label:Views
averageWords_label:Average Words (Chapter)
add_to_titlepage_entries:,views, averageWords, fandoms

## for site is rate limiting
slow_down_sleep_time:120

use_flaresolverr_proxy:directimages
## option settings, with default values commented out:
flaresolverr_proxy_address:localhost
flaresolverr_proxy_port:8191
flaresolverr_proxy_protocol:http
flaresolverr_proxy_timeout:1800000

Hopefully you'll be able to make some sense of it

[JimmXinu]

Originally Posted by Snomad

Hi Jimm, I'm running into an error when downloading a fic from scribblehub. I thought that it was because I was getting IP banned again, but I proxied into the connection that flaresolver is using and it can still access scribblehub without issue.

There are so many network blocking issues -- and so little we can do about them -- that recommending flaresolverr or FFF's browser cache feature are our go-to answers.

If flaresolverr is working for you, that's your solution.

[Snomad]

It's only sorta working. I think that the issue might be more the story size tbh as it's over 70 chapters. The logs from the most recent attempt show that it got 33 chapters successfully before failing. Is there anything I can add to the ini snippet section of the download to limit the download request to say 15-20 chapters before considering it successful and then creating or updating the book?

[JimmXinu]

Originally Posted by Snomad

It's only sorta working. I think that the issue might be more the story size tbh as it's over 70 chapters. The logs from the most recent attempt show that it got 33 chapters successfully before failing. Is there anything I can add to the ini snippet section of the download to limit the download request to say 15-20 chapters before considering it successful and then creating or updating the book?

I don't have the energy to type it all again, but here's an earlier post:

Originally Posted by JimmXinu

You can:

• Put a chapter range on end of a story URL, example: https://www.royalroad.com/fiction/104434/[45-65] You can also do [5] (single chapter 5) [-5] (all before and chapter 5) and [5-] (chapter 5 and all after). The down side is it isn't remembered if you want to update the book.
• Use the 'undocumented' test/dev first_chapter and/or last_chapter in personal.ini in that story's [section]. This is equivalent to the [x-y] range, but saved so it is applied on updates.

[Atherton25]

Speaking of Scribblehub... is there a way to get flaresolverr to get the TOC and then open the rest in a browser?

I know if I open everything in the browser, I can get a Scribblehub fic, but when I try and use flaresolverr it starts giving me issues after a dozen chapters or so, even with slow down set to 7. Then I have to open up all the chapters manually or try and use the download through flaresolverr a couple hours later - which only works sometimes.

I've done a search, and just browsed through anything I could think of and can't seem to find an example of using both.

[Snomad]

Originally Posted by JimmXinu

I don't have the energy to type it all again, but here's an earlier post:

No worries, that's already heaps helpful. Thank you!

[Snomad]

I finally managed to download a single chapter of that fic and I just want to double-check if this sounds right as the downloading from scribbehub was working fine only a few days ago. The logs show that fanficfare sent 78 requests through flaresolver to scribblehub to download a chapter of 2,985 words.

I originally thought it was sending a single request per chapter and that's the most I've seen it manage without hitting a "Error: Error solving the challenge. Message: unknown error: net::ERR_CONNECTION_CLOSED\n" or "Error solving the challenge. Message: invalid cookie domain\n" issue.

If that is actually normal that's all good, I'm sure I can figure something else out

[JimmXinu]

I don't actually know anything about what appears in the flaresolverr log. There are retries that happen from FFF, depending on the error type.

Honestly, I can't do anything more about sites blocking us. Thank the AI overlords for ruining automated scraping for everyone.

You could look into the unsupported idea of overriding no-cache with BrowserCache. But I'm not going to help any further than linking to it.