[GUI Plugin] FanFicFare

[Lys]

Originally Posted by JimmXinu

You could try use_archiveofourown_gay:true or Browser Cache feature. See wiki page for ProblematicSites

And worst case, AO3 has their own epub download feature you can use.

I tried different combinations with use_archiveofourown_gay:true or with the browser cache feature, but the result was always the same (with a slight different long message error, but always a 403)

That's the reason I asked if someone managed to download locked AO3 fics through FFF in the last few days, and if yes, if they could share their own setting to see if I could replicate that and solve the issue

[ownedbycats]

Originally Posted by Lys

Hi all

I'm having some issues updating any locked story from AO3. Not locked stories are updating correctly, instead.
I tried playing around with my personal ini (taking out always_login + credential, adding the open in browser one), but no matter what I did, I always got a 403 error.
Are you also experiencing this issue?
If yes, and you managed to solve it, can you please share your personal ini setting?

Originally Posted by Lys

I tried different combinations with use_archiveofourown_gay:true or with the browser cache feature, but the result was always the same (with a slight different long message error, but always a 403)

That's the reason I asked if someone managed to download locked AO3 fics through FFF in the last few days, and if yes, if they could share their own setting to see if I could replicate that and solve the issue

Restricted stories work as expected for me.

How long a period are you testing over? I get occasional 525 errors, but they clear up the next day. I suspect related to repeated requests when they're getting higher traffic.

Might also be worth rebooting your modem in case you got assigned an IP used by a previous bad actor.

[Lys]

Originally Posted by ownedbycats

Restricted stories work as expected for me.

How long a period are you testing over? I get occasional 525 errors, but they clear up the next day. I suspect related to repeated requests when they're getting higher traffic.

Might also be worth rebooting your modem in case you got assigned an IP used by a previous bad actor.

I came back from holiday after a couple of weeks, so... since yesterday. Two weeks ago, everything was fine

Initially, I had a 403 error for everything (locked and not locked stories), then I commented always_login:true and my username, and not locked stories could be downloaded again.
For locked stories, instead, no luck so far, no matter which instruction I try to comment or to add: I always get a 403 or a Failed to Login for URL

I'll try rebooting my modem and see how it goes!

[JimmXinu]

It's not just a matter of settings. Cloudflare can and does give different treatment to different people on different networks.

I'm currently able to download 'locked' AO3 stories using FFF's automatic login.

[wildfire070]

Originally Posted by Lys

I tried different combinations with use_archiveofourown_gay:true or with the browser cache feature, but the result was always the same (with a slight different long message error, but always a 403)

That's the reason I asked if someone managed to download locked AO3 fics through FFF in the last few days, and if yes, if they could share their own setting to see if I could replicate that and solve the issue

I'm not having any issues downloading locked/unlocked fics. I have this in my personal.ini under the AO3 section:

use_archiveofourown_gay:true
always_login:true
username:
password:
slow_down_sleep_time:4
use_view_full_work:true

[DevilsReject97]

Received this error with FFF today, never seen this one before:

AttributeError: 'NoneType' object has no attribute 'find'

Basic settings aside from adding the setup for flaresolverr

[JimmXinu]

Originally Posted by DevilsReject97

Received this error with FFF today, never seen this one before:

AttributeError: 'NoneType' object has no attribute 'find'

Basic settings aside from adding the setup for flaresolverr

I can't even look at this without a story URL and/or debug log.

[DevilsReject97]

Originally Posted by JimmXinu

I can't even look at this without a story URL and/or debug log.

My apologies.

The link was https://novelfull.com/the-genius-sys...out-equal.html

I also got it on a second story, but it was the same site.

I got it twice on the same story, but now the story will run some and then cap out as a timeout on Flaresolverr.... which seems odd because it runs to about 7%

[DevilsReject97]

I honestly am suspecting its the site being difficult and not anything to do with FFF - I retried for a 3rd time and it's already doubled what it previously did running.

As a sidenote, if I get a failed challenge response for flaresolverr within FFF, is there a setting I should change?

[DevilsReject97]

Originally Posted by DevilsReject97

I honestly am suspecting its the site being difficult and not anything to do with FFF - I retried for a 3rd time and it's already doubled what it previously did running.

As a sidenote, if I get a failed challenge response for flaresolverr within FFF, is there a setting I should change?

Spoke too soon...

I just added the last bits of the log as it was working up until Ch. 285.

[JimmXinu]

If you're downloading hundreds of chapters, and it's not failing at the same place, I strongly suspect you are running into some form of blocking from cloudflare or the site.

(Geez, a story with 850 chapters and 18 pages just of ToC... I downloaded the first 5 chapters without issue.)

You can try:

• Increasing slow_down_sleep_time: (default is 5 for novelfull.com)
• Only downloading a hundred or so chapters at a time using story URL with chapter range. Ie, https://novelfull.com/the-genius-sys...out-equal.html[1-100], then when that finishes, wait a while, then update by story URL with [101-200], etc.
• Set continue_on_chapter_error:true -- not necessary a good idea if you don't also do ranges, because FFF will continue trying for the next 600 chapters of failures if you give it the whole thing and it starts failing.

The advantage of continue_on_chapter_error:true is that FFF will keep all the chapters already downloaded. The downside is that continuing to try while failing may get you blocked longer.

Maybe I will investigate adding an intermediate setting that keeps chapters that have succeeded, but still stops on first failure.

[DevilsReject97]

Originally Posted by JimmXinu

If you're downloading hundreds of chapters, and it's not failing at the same place, I strongly suspect you are running into some form of blocking from cloudflare or the site.

(Geez, a story with 850 chapters and 18 pages just of ToC... I downloaded the first 5 chapters without issue.)

You can try:

• Increasing slow_down_sleep_time: (default is 5 for novelfull.com)
• Only downloading a hundred or so chapters at a time using story URL with chapter range. Ie, https://novelfull.com/the-genius-sys...out-equal.html[1-100], then when that finishes, wait a while, then update by story URL with [101-200], etc.
• Set continue_on_chapter_error:true -- not necessary a good idea if you don't also do ranges, because FFF will continue trying for the next 600 chapters of failures if you give it the whole thing and it starts failing.

The advantage of continue_on_chapter_error:true is that FFF will keep all the chapters already downloaded. The downside is that continuing to try while failing may get you blocked longer.

Maybe I will investigate adding an intermediate setting that keeps chapters that have succeeded, but still stops on first failure.

This is what I have for my settings:

Default ini for FFF and this:
[novelfull.com]
use_flaresolverr_proxy:true
slow_down_sleep_time:7
## flaresolverr_proxy_timeout is in integer milliseconds.
## Keep flaresolverr_proxy_timeout set smaller than connect_timeout
## otherwise the connection *to* flaresolverr will timeout first.
flaresolverr_proxy_timeout:59000

If 5 is better, or 10 is better?


Thank you for looking into it. I very much appreciate the help as I really only read books from this site because most of the others that are similar don't work with FFF. I can certainly try to do that, as updates may be easier. I've actually wondered the same, as all 3 attempts that downloaded all stopped at different spots with the same error.

Ironically, the site itself has been acting weird even with updates, as I have books with over 3k chapters but FFF will say the site only has 1100 chapters available - yet the site shows otherwise.

Most likely, it is as you suspect and the site itself will probably become more unusable in this manner.