[GUI Plugin] FanFicFare

[applegaa]

I had a story stub again on ScribbleHub, so I scrubbed and saved the current epub as book 5 and created a new book from the URL. As tends to happen with ScribbleHub's use of CloudFlare, it failed to download all of the chapters on the first try. Instead I got:

Download epub completed, 5 failed chapters, 129 total chapters.

I waited a while, then did another Update EPUB if New Chapters, figuring I could grab the last five with no problem. However, it failed again, and I got this message:

Update epub completed, added 114 chapters, 5 failed chapters, for 129 total.

That surprised me, as (I thought) that meant it had downloaded less than 15 chapters in the first attempt. I looked and verified there were more than 5 chapters that 403 errored, but I didn't actually count them. Still, (I thought) it meant there were less than 15 remaining, so I waited a while and tried again. This time, I got:

Update epub completed, added 85 chapters, 5 failed chapters, for 129 total.

Now, 114+85 is quite a bit more than 129, so I took a closer look. Strangely, the epub appeared to have 88 chapters.

So, first of all, it appears the failed chapter count is not working. I don't know if it works for counts less than 5 or if it always shows 5. I'm pretty sure this is a bug rather than a configuration error, but I'm open to being corrected.

Second, rather than just adding the missing chapters, it's apparently starting over and downloading the whole thing again rather than just the missing chapters. This kind of defeats a big reason for saving the part downloaded successfully, and it keeps the URL for the missing chapters, so it's not like it's not sure what to replace them with.

Is there something I missed that would tell it to only download the errored out chapters instead of everything? None of the other update modes seem likely to work better, but CloudFlare kind of puts a damper on extensive testing.

EDIT: I just had another one show 2 chapter errors, so it does give counts other than 5.

I'm still afraid to try and pull those, for fear it will start downloading the whole story and remove chapters currently included.

[JimmXinu]

Originally Posted by applegaa

I had a story stub again on ScribbleHub, so I scrubbed and saved the current epub as book 5 and created a new book from the URL. As tends to happen with ScribbleHub's use of CloudFlare, it failed to download all of the chapters on the first try. Instead I got:

As it is, I don't have much to work from, since you've not given even a story URL.

Run Calibre in debug and post the logs while you are seeing these issues. PM if NSFW story.

I would speculate, that if the author 'stubbed' the story by replacing each chapter, the chapters URLs all changed. In which case, discarding the old chapters is technically the correct course of action for FFF.

I am also concerned what you mean by you "scrubbed" the epub. Running other tools on FFF epubs can break update.

Originally Posted by applegaa

I'm still afraid to try and pull those, for fear it will start downloading the whole story and remove chapters currently included.

I recommend backups. Especially if this is something your authors do regularly, I would backup before every update.

[applegaa]

Originally Posted by JimmXinu

As it is, I don't have much to work from, since you've not given even a story URL.

Sorry that I didn't include a specific URL, but the specific story is not really relevant to the issue. Any story where CloudFlare throws 403 errors on some chapters will suffice. I can provide a couple of stories it's happened to me on, but I can't guarantee that CloudFlare will trigger for you.

Run Calibre in debug and post the logs while you are seeing these issues. PM if NSFW story.

I don't think it's NSFW, but I'll PM to be safe.

I would speculate, that if the author 'stubbed' the story by replacing each chapter, the chapters URLs all changed. In which case, discarding the old chapters is technically the correct course of action for FFF.

I am also concerned what you mean by you "scrubbed" the epub. Running other tools on FFF epubs can break update.

What I meant was that I removed anything from the old epub and its metadata that would cause FFF to link it to any new download attempts with that URL/Title. I included that information to make it clear that the previous epub and the removed chapters were not part of the issue. In hindsight, I probably created more confusion than I cleared up.

I recommend backups. Especially if this is something your authors do regularly, I would backup before every update.

I use Update EPUB if New Chapters, then View Log before accepting changes as a precaution. I did not realize that it would replace already downloaded chapters with error messages rather than just downloading new and missing chapters, or I would have proceeded differently.



So, to try again to more clearly explain the issues I'm concerned about, as I clearly failed in my first attempt…

1 - When some chapters fail during a download, it counts them, but for some reason the count always stops at 5. If there are 5 or less, it displays an accurate count. However, if there were 6 or more chapters with errors, it still says there were 5.

2 - Despite using Update EPUB if New Chapters, it is attempting to overwrite the EPUB instead. It is downloading and replacing existing chapters instead of just new or missing ones. FFF stores the URL of each missing chapter in the epub file for that chapter, so it should not need to download the entire story to figure out which chapter it needs to grab.

[JimmXinu]

Originally Posted by applegaa

1 - When some chapters fail during a download, it counts them, but for some reason the count always stops at 5. If there are 5 or less, it displays an accurate count. However, if there were 6 or more chapters with errors, it still says there were 5.

That sounds like continue_on_chapter_error_try_limit.

From defaults.ini:

## Because some adapters can pull chapter URLs from human posts, the
## odds of errors in the chapter URLs can be higher for some
## sites/stories.  You can set continue_on_chapter_error:true to
## continue on after failing to download a chapter and instead record
## an error message in the ebook for that chapter.
continue_on_chapter_error:false

## When continue_on_chapter_error:true, after
## continue_on_chapter_error_try_limit chapters have failed, continue
## processing, but stop trying to download chapters.  Mark all such
## chapters with chapter_title_error_mark, but chapter text will
## explain that no download attempt was made because
## continue_on_chapter_error_try_limit was exceeded.  Set to -1 for
## infinite chapter errors.
continue_on_chapter_error_try_limit:5

I suppose I could count skipped chapters as errors still. I'll need to look at the logic more to see if that would cause other problems.

If you are getting blocks like this regularly, I would definitely set slow_down_sleep_time considerably higher. Your log shows you have it set lower than the default 5sec.

Originally Posted by applegaa

2 - Despite using Update EPUB if New Chapters, it is attempting to overwrite the EPUB instead. It is downloading and replacing existing chapters instead of just new or missing ones. FFF stores the URL of each missing chapter in the epub file for that chapter, so it should not need to download the entire story to figure out which chapter it needs to grab.

The code shows that old chapters will be honored, even when continue_on_chapter_error_try_limit is exceeded. I will need to contrive a test case.

[JimmXinu]

Originally Posted by JimmXinu

That sounds like continue_on_chapter_error_try_limit.

...

The code shows that old chapters will be honored, even when continue_on_chapter_error_try_limit is exceeded. I will need to contrive a test case.

My (admittedly contrived) test does show FFF reusing existing chapters: trying new 5 chapters, then auto-failing the rest.

However, from FFF's status log output, I can see why you would assume the other 10 chapters were there:

Update epub completed, added 10 chapters, 5 failed chapters, for 25 total

In point of fact, all 10 added chapters are failures, but the later 5 are 'assumed failed'.

I will look into improving reporting when continue_on_chapter_error_try_limit trips.

[JimmXinu]

New Test Version Posted

2026-05-10
- Improve continue_on_chapter_error_try_limit exceeded reporting
- Chrome browsercache_sqldb added, but cannot read cache while browser is open. See #1341

continue_on_chapter_error_try_limit exceeded chapters counted as failed now.

I do not consider the browsercache_sqldb particularly useable.

But I did learn you can go into Chrome's Experiments flags and change the cache type to 'simple' or 'blockfile'. Enter chrome://flags/#http-cache-custom-backend into the address bar.

[applegaa]

Originally Posted by JimmXinu

That sounds like continue_on_chapter_error_try_limit.

I suppose I could count skipped chapters as errors still. I'll need to look at the logic more to see if that would cause other problems.

Yeah, that is what I was expecting. It appears to continue creating the files for all of the missing chapters, so I expected it was counting them. In hindsight, I should have figured out that it wasn't…

If you are getting blocks like this regularly, I would definitely set slow_down_sleep_time considerably higher. Your log shows you have it set lower than the default 5sec.

I normally don't, but I had a bunch of stories stub since the last update, so I was downloading more chapters at a time than usual. I thought I had my sleep time higher than the default, but maybe that was a different library or you raised it without me noticing.

The code shows that old chapters will be honored, even when continue_on_chapter_error_try_limit is exceeded. I will need to contrive a test case.

However, from FFF's status log output, I can see why you would assume the other 10 chapters were there:

In point of fact, all 10 added chapters are failures, but the later 5 are 'assumed failed'.

After more careful testing with this in mind, I think the issue was coincidental numbers adding to a misunderstanding of what the results actually meant. After testing several of them, I can confirm that Total Chapters minus Added Chapters equals Prior Good Chapters.

It's counting the placeholder error chapters as being added, even though they exist to say the chapter wasn't added. It also doesn't matter if they were already there from a previous update, so the number of "added" chapters is kind of meaningless, and it definitely doesn't mean what I thought it meant.


Anyway, sorry for bugging you on a Sunday evening over a misunderstanding, but thanks for helping me through it! I appreciate it!

[Atherton25]

@applegaa As far as ScribbleHub goes, the best way to do it is use a combination of cache and flaresolverr. I've found if I 'download' the story or view all the chapters - using webtoepub and then use FFF, it will go through flaresolverr to grab the chapter list, and maybe some metadata, and then grab everything else from cache (if you have it setup). If I remember right, there's something in Scribblehubs method of making chapters that means you have to grab it live.

And yes, before the update yesterday if you were using flaresolverr and continue on error it would stop on 5 chapters and then say only 5 chapters were 'bad' but essentially everything after that was just a blank chapter. I never thought to mention it here, because I didn't realize it wasn't working as intended.

[JimmXinu]

New Test Version Posted

2026-05-14
- Add OTW story note labels config #1347

## Customizable headers for the different OTW story notes:
## See exclude_notes.
#notelabel_authorheadnotes:Author's Note:
#notelabel_chaptersummary:Summary for the Chapter:
#notelabel_chapterheadnotes:Notes for the Chapter:
#notelabel_chapterfootnotes:Notes for the Chapter:
#notelabel_authorfootnotes:Author's Note: