[GUI Plugin] FanFicFare

[applegaa]

I had a story stub again on ScribbleHub, so I scrubbed and saved the current epub as book 5 and created a new book from the URL. As tends to happen with ScribbleHub's use of CloudFlare, it failed to download all of the chapters on the first try. Instead I got:

Download epub completed, 5 failed chapters, 129 total chapters.

I waited a while, then did another Update EPUB if New Chapters, figuring I could grab the last five with no problem. However, it failed again, and I got this message:

Update epub completed, added 114 chapters, 5 failed chapters, for 129 total.

That surprised me, as (I thought) that meant it had downloaded less than 15 chapters in the first attempt. I looked and verified there were more than 5 chapters that 403 errored, but I didn't actually count them. Still, (I thought) it meant there were less than 15 remaining, so I waited a while and tried again. This time, I got:

Update epub completed, added 85 chapters, 5 failed chapters, for 129 total.

Now, 114+85 is quite a bit more than 129, so I took a closer look. Strangely, the epub appeared to have 88 chapters.

So, first of all, it appears the failed chapter count is not working. I don't know if it works for counts less than 5 or if it always shows 5. I'm pretty sure this is a bug rather than a configuration error, but I'm open to being corrected.

Second, rather than just adding the missing chapters, it's apparently starting over and downloading the whole thing again rather than just the missing chapters. This kind of defeats a big reason for saving the part downloaded successfully, and it keeps the URL for the missing chapters, so it's not like it's not sure what to replace them with.

Is there something I missed that would tell it to only download the errored out chapters instead of everything? None of the other update modes seem likely to work better, but CloudFlare kind of puts a damper on extensive testing.

EDIT: I just had another one show 2 chapter errors, so it does give counts other than 5.

I'm still afraid to try and pull those, for fear it will start downloading the whole story and remove chapters currently included.

[JimmXinu]

Originally Posted by applegaa

I had a story stub again on ScribbleHub, so I scrubbed and saved the current epub as book 5 and created a new book from the URL. As tends to happen with ScribbleHub's use of CloudFlare, it failed to download all of the chapters on the first try. Instead I got:

As it is, I don't have much to work from, since you've not given even a story URL.

Run Calibre in debug and post the logs while you are seeing these issues. PM if NSFW story.

I would speculate, that if the author 'stubbed' the story by replacing each chapter, the chapters URLs all changed. In which case, discarding the old chapters is technically the correct course of action for FFF.

I am also concerned what you mean by you "scrubbed" the epub. Running other tools on FFF epubs can break update.

Originally Posted by applegaa

I'm still afraid to try and pull those, for fear it will start downloading the whole story and remove chapters currently included.

I recommend backups. Especially if this is something your authors do regularly, I would backup before every update.

[applegaa]

Originally Posted by JimmXinu

As it is, I don't have much to work from, since you've not given even a story URL.

Sorry that I didn't include a specific URL, but the specific story is not really relevant to the issue. Any story where CloudFlare throws 403 errors on some chapters will suffice. I can provide a couple of stories it's happened to me on, but I can't guarantee that CloudFlare will trigger for you.

Run Calibre in debug and post the logs while you are seeing these issues. PM if NSFW story.

I don't think it's NSFW, but I'll PM to be safe.

I would speculate, that if the author 'stubbed' the story by replacing each chapter, the chapters URLs all changed. In which case, discarding the old chapters is technically the correct course of action for FFF.

I am also concerned what you mean by you "scrubbed" the epub. Running other tools on FFF epubs can break update.

What I meant was that I removed anything from the old epub and its metadata that would cause FFF to link it to any new download attempts with that URL/Title. I included that information to make it clear that the previous epub and the removed chapters were not part of the issue. In hindsight, I probably created more confusion than I cleared up.

I recommend backups. Especially if this is something your authors do regularly, I would backup before every update.

I use Update EPUB if New Chapters, then View Log before accepting changes as a precaution. I did not realize that it would replace already downloaded chapters with error messages rather than just downloading new and missing chapters, or I would have proceeded differently.



So, to try again to more clearly explain the issues I'm concerned about, as I clearly failed in my first attempt…

1 - When some chapters fail during a download, it counts them, but for some reason the count always stops at 5. If there are 5 or less, it displays an accurate count. However, if there were 6 or more chapters with errors, it still says there were 5.

2 - Despite using Update EPUB if New Chapters, it is attempting to overwrite the EPUB instead. It is downloading and replacing existing chapters instead of just new or missing ones. FFF stores the URL of each missing chapter in the epub file for that chapter, so it should not need to download the entire story to figure out which chapter it needs to grab.

[JimmXinu]

Originally Posted by applegaa

1 - When some chapters fail during a download, it counts them, but for some reason the count always stops at 5. If there are 5 or less, it displays an accurate count. However, if there were 6 or more chapters with errors, it still says there were 5.

That sounds like continue_on_chapter_error_try_limit.

From defaults.ini:

## Because some adapters can pull chapter URLs from human posts, the
## odds of errors in the chapter URLs can be higher for some
## sites/stories.  You can set continue_on_chapter_error:true to
## continue on after failing to download a chapter and instead record
## an error message in the ebook for that chapter.
continue_on_chapter_error:false

## When continue_on_chapter_error:true, after
## continue_on_chapter_error_try_limit chapters have failed, continue
## processing, but stop trying to download chapters.  Mark all such
## chapters with chapter_title_error_mark, but chapter text will
## explain that no download attempt was made because
## continue_on_chapter_error_try_limit was exceeded.  Set to -1 for
## infinite chapter errors.
continue_on_chapter_error_try_limit:5

I suppose I could count skipped chapters as errors still. I'll need to look at the logic more to see if that would cause other problems.

If you are getting blocks like this regularly, I would definitely set slow_down_sleep_time considerably higher. Your log shows you have it set lower than the default 5sec.

Originally Posted by applegaa

2 - Despite using Update EPUB if New Chapters, it is attempting to overwrite the EPUB instead. It is downloading and replacing existing chapters instead of just new or missing ones. FFF stores the URL of each missing chapter in the epub file for that chapter, so it should not need to download the entire story to figure out which chapter it needs to grab.

The code shows that old chapters will be honored, even when continue_on_chapter_error_try_limit is exceeded. I will need to contrive a test case.

[JimmXinu]

Originally Posted by JimmXinu

That sounds like continue_on_chapter_error_try_limit.

...

The code shows that old chapters will be honored, even when continue_on_chapter_error_try_limit is exceeded. I will need to contrive a test case.

My (admittedly contrived) test does show FFF reusing existing chapters: trying new 5 chapters, then auto-failing the rest.

However, from FFF's status log output, I can see why you would assume the other 10 chapters were there:

Update epub completed, added 10 chapters, 5 failed chapters, for 25 total

In point of fact, all 10 added chapters are failures, but the later 5 are 'assumed failed'.

I will look into improving reporting when continue_on_chapter_error_try_limit trips.

[JimmXinu]

New Test Version Posted

2026-05-10
- Improve continue_on_chapter_error_try_limit exceeded reporting
- Chrome browsercache_sqldb added, but cannot read cache while browser is open. See #1341

continue_on_chapter_error_try_limit exceeded chapters counted as failed now.

I do not consider the browsercache_sqldb particularly useable.

But I did learn you can go into Chrome's Experiments flags and change the cache type to 'simple' or 'blockfile'. Enter chrome://flags/#http-cache-custom-backend into the address bar.

[applegaa]

Originally Posted by JimmXinu

That sounds like continue_on_chapter_error_try_limit.

I suppose I could count skipped chapters as errors still. I'll need to look at the logic more to see if that would cause other problems.

Yeah, that is what I was expecting. It appears to continue creating the files for all of the missing chapters, so I expected it was counting them. In hindsight, I should have figured out that it wasn't…

If you are getting blocks like this regularly, I would definitely set slow_down_sleep_time considerably higher. Your log shows you have it set lower than the default 5sec.

I normally don't, but I had a bunch of stories stub since the last update, so I was downloading more chapters at a time than usual. I thought I had my sleep time higher than the default, but maybe that was a different library or you raised it without me noticing.

The code shows that old chapters will be honored, even when continue_on_chapter_error_try_limit is exceeded. I will need to contrive a test case.

However, from FFF's status log output, I can see why you would assume the other 10 chapters were there:

In point of fact, all 10 added chapters are failures, but the later 5 are 'assumed failed'.

After more careful testing with this in mind, I think the issue was coincidental numbers adding to a misunderstanding of what the results actually meant. After testing several of them, I can confirm that Total Chapters minus Added Chapters equals Prior Good Chapters.

It's counting the placeholder error chapters as being added, even though they exist to say the chapter wasn't added. It also doesn't matter if they were already there from a previous update, so the number of "added" chapters is kind of meaningless, and it definitely doesn't mean what I thought it meant.


Anyway, sorry for bugging you on a Sunday evening over a misunderstanding, but thanks for helping me through it! I appreciate it!

[Atherton25]

@applegaa As far as ScribbleHub goes, the best way to do it is use a combination of cache and flaresolverr. I've found if I 'download' the story or view all the chapters - using webtoepub and then use FFF, it will go through flaresolverr to grab the chapter list, and maybe some metadata, and then grab everything else from cache (if you have it setup). If I remember right, there's something in Scribblehubs method of making chapters that means you have to grab it live.

And yes, before the update yesterday if you were using flaresolverr and continue on error it would stop on 5 chapters and then say only 5 chapters were 'bad' but essentially everything after that was just a blank chapter. I never thought to mention it here, because I didn't realize it wasn't working as intended.

[JimmXinu]

New Test Version Posted

2026-05-14
- Add OTW story note labels config #1347

## Customizable headers for the different OTW story notes:
## See exclude_notes.
#notelabel_authorheadnotes:Author's Note:
#notelabel_chaptersummary:Summary for the Chapter:
#notelabel_chapterheadnotes:Notes for the Chapter:
#notelabel_chapterfootnotes:Notes for the Chapter:
#notelabel_authorfootnotes:Author's Note:

[ownedbycats]

Nice

[MidwestJen]

Hi there!

In the last couple of days I've found a weird response when trying to download FFnet story updates. Even though I can see that all metadata on the pages are correct (meaning the story shows the correct number of chapters on the 1st chapter and you can navigate to all the chapters) it says it doesn't need an update and returns the error saying it has x chapters, etc. Which isn't true, since I just opened the story in my browser. The odd thing is that the next day, it seems to go through fine.

I use the browser cache method, and I download any updates on ffnet one at a time (luckily most of my updates are A03 these days). I still have scars from the days when you'd get locked out (hahaha), so I just keep doing them one at a time using the download story option.

As an example, yesterday I tried to download an update for https://www.fanfiction.net/s/14550534/23/The-Finding and got the error "none of the stories need updated, the epub contains 22 chapters, use force overwrite ..." - today I ran that same story through and it went through fine.

Today I've got a few to choose from, but I received a notification at 10:20pm last night for https://www.fanfiction.net/s/1450728...mborn-Sorcerer and the website is showing that it has 9 chapters on the first chapter page - you can navigate to all the chapters from any chapter and they all indicate there are 9 chapters. However, this is the error I'm getting:

Skipped
The Stormborn Sorcerer
gitar002
Existing epub contains 8 chapters, web site only has 7. Use Overwrite or force_update_epub_always to force update.
https://www.fanfiction.net/s/14507282/1/The-Stormborn-Sorcerer

But tomorrow it'll most likely go through fine. I'll do nothing different on my end.

Waiting until tomorrow is more of a pain than anything and not something that NEEDS fixed, but I will admit that I'm confused. I don't pretend to be an expert, but I thought that when using browser cache to update stories that it used the info showing in the browser instead of querying the website. I'm using Firefox and it's up to date if that helps. I updated my calibre and plugins yesterday and initially thought that was the reason for this, but I rolled my plugin back to what it was in my last backup and it's still erroring out.

Thoughts (and thank you for taking the time!)??

PS - I was going to give you another URL to try in case the one I provided that's erroring out is now "ready", and I had another to give you that came in at 5:39am that I assumed would also error out (https://www.fanfiction.net/s/1392967...nfinity-Stones) - and it updated fine. *throws hands in the air* I have no idea. Help?

[JimmXinu]

Originally Posted by MidwestJen

In the last couple of days I've found a weird response when trying to download FFnet story updates.
...

ffnet goes through periods where it just doesn't work 100%. What you are probably seeing is the metadata chapter count showing the full number, but the chapter list on the first chapter not showing the last one.

I saw that recently, too.

FFF has two settings specifically for ffnet's problems:

## ffnet used to have a tendency to send out update notices in email
## before all their servers were showing the update on the first
## chapter.  It generates another server request and doesn't seem to
## be needed lately, so now default it to off.
check_next_chapter:false

## ffnet sometimes doesn't update the metadata on the first chapter
## immediately.  meta_from_last_chapter:true will instruct the adapter
## to take the metadata from the last chapter instead of chapter 1.
## It can generate another server request and usually isn't necessary,
## so it defaults to off.
meta_from_last_chapter:false

I don't leave check_next_chapter especially on by default because it will generate a bunch of failed lookups when there isn't a missing new chapter.

[MidwestJen]

That fixed it, thank you so much! I'm not sure how I didn't see those settings - then again, I rolled back my plugin and haven't updated it again, so perhaps I missed them. I appreciate your time helping me with this!

[Elaine T]

My turn with an Ao3 issue. Anthology update on a series got a 403 error, with a 'another error occurred'. Details from log attached.

When I tried to download the new story solo I got the regular 403 error.
The series link: https://archiveofourown.org/series/5913121

I'll wait a few hours and try again. But I thought the extra error needed to be reported.

[ownedbycats]

Originally Posted by Elaine T

My turn with an Ao3 issue. Anthology update on a series got a 403 error, with a 'another error occurred'. Details from log attached.

When I tried to download the new story solo I got the regular 403 error.
The series link: https://archiveofourown.org/series/5913121

I'll wait a few hours and try again. But I thought the extra error needed to be reported.

https://www.otwstatus.org/incidents/82vzwqssqg73

CloudFlare is up right now, hence the 403s