[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by DevilsReject97

I wanted to see if you were still thinking about this? Or even an option to check box to keep them? Can't speak for others but I think it would be a huge help.

Already added to test version last month. (There hasn't been a new release since then.)

Originally Posted by JimmXinu

When continue_on_chapter_error was added, the use case was for old SB/SV story threads with manually crafted ToC posts that sometimes contained errors.

Today, we're seeing more and more sites that appear to start blocking after some unknown access limit is exceeded.

continue_on_chapter_error is helpful in those cases because you don't discard however many chapters did download before getting blocked. But I suspect that continuing to try each subsequent chapter after blocking starts will increase your blocked time--that's how I implemented it back in the day.

I propose to add a second setting: continue_on_chapter_error_try_limit.

FFF, (with continue_on_chapter_error:true), would record each chapter error and continue to the next chapter, but after continue_on_chapter_error_try_limit failures, would give up and mark all subsequent chapters 'ERROR' without trying to download them.

I'm thinking an appropriate default would be continue_on_chapter_error_try_limit:5

Some details:

• Why not just leave out the failed / not-tried chapters?
  Because users won't immediately notice missing chapters--I did exactly that while testing that very thing.
  
• What happens with updates?
  That's the real power of continue_on_chapter_error in this case; FFF will discard the ERROR message chapters and attempt to download them as if they were newly added. (Including be marked (new) if mark_new_chapters:true.)
  You could, for example, update a story with 1000 chapters once a day and if you get 100 or 200 chapters each time, have the whole thing in a week or so. Previously, we've instructed users to basically do that manually with chapter ranges.
  
• What about the original use case? What if there are actually that many bad URLs? How can you ever get the chapters after those?
  You'd need to increase continue_on_chapter_error_try_limit for that story, or use ignore_chapter_url_list.
  
• The ERROR chapter title would be the same, but ERROR chapter text for chapters after continue_on_chapter_error_try_limit would have different explanatory text.
  
• 'Skipping subsequent chapters without trying' does not apply to later chapters that were already in the epub. Those will still be used.

Opinions?

Originally Posted by JimmXinu

New Test Version Posted

2025-08-28
- Add continue_on_chapter_error_try_limit setting (default:5) See this post.

[MidnightRose]

I have no idea why, but this specific webnovel is producing a file that Calibre seems unable to parse.

https://www.royalroad.com/fiction/10...f-gold-digging

Trying to do any kind of conversion or using calibre's internal book editor gives this error:

calibre, version 8.11.0
ERROR: Failed to open book: Failed to open book, click "Show details" for more information.

Traceback (most recent call last):
  File "calibre\gui2\tweak_book\job.py", line 33, in run
  File "calibre\gui2\tweak_book\boss.py", line 101, in get_container
  File "calibre\ebooks\oeb\polish\container.py", line 1639, in get_container
  File "calibre\ebooks\oeb\polish\container.py", line 1232, in __init__
  File "calibre\ebooks\oeb\polish\container.py", line 279, in __init__
  File "calibre\ebooks\oeb\polish\container.py", line 282, in refresh_mime_map
  File "calibre\ebooks\oeb\polish\container.py", line 544, in opf_xpath
  File "calibre\ebooks\oeb\polish\container.py", line 649, in opf
  File "calibre\ebooks\oeb\polish\container.py", line 632, in parsed
  File "calibre\ebooks\oeb\polish\container.py", line 603, in parse
  File "calibre\ebooks\oeb\polish\container.py", line 179, in parse_xml
  File "calibre\utils\xml_parse.py", line 26, in safe_xml_fromstring
  File "src/lxml/etree.pyx", line 3428, in lxml.etree.fromstring
  File "src/lxml/parser.pxi", line 2059, in lxml.etree._parseMemoryDocument
  File "src/lxml/parser.pxi", line 1914, in lxml.etree._parseDoc
  File "src/lxml/parser.pxi", line 1930, in lxml.etree._parseDoc_unicode
  File "src/lxml/parser.pxi", line 1133, in lxml.etree._BaseParser._parseUnicodeDoc
  File "src/lxml/parser.pxi", line 647, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 765, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 689, in lxml.etree._raiseParseError
  File "<string>", line 6
lxml.etree.XMLSyntaxError: internal error, line 6, column 3040

Meanwhile when I take the internal EPUB file and stick it into kindle previewer, it's able to open the file just fine.

Could it be the emoji in the description?

[MidnightRose]

After some experimentation, it is indeed the emoji in the description.
I tried downloading and importing the same book using WebToEpub, it imported fine, and could convert to other formats as well as edit using the epub editor.
Then if I update the metadata using FFF, it breaks again, but if I then remove just the emoji, it works fine again.
Perhaps an option to strip those out?

[kovidgoyal]

Just update calibre to 8.11.1

[JimmXinu]

Originally Posted by MidnightRose

After some experimentation, it is indeed the emoji in the description.
I tried downloading and importing the same book using WebToEpub, it imported fine, and could convert to other formats as well as edit using the epub editor.
Then if I update the metadata using FFF, it breaks again, but if I then remove just the emoji, it works fine again.
Perhaps an option to strip those out?

Originally Posted by kovidgoyal

Just update calibre to 8.11.1

Confirmed--Calibre 8.11.1 works and 8.11.0 doesn't.

[ownedbycats]

Jim: Anything in this change that might affect retrieval of collections?
https://archiveofourown.org/admin_posts/33031

[chrisric]

Is there a way to have FFF automatically use "Add New Book" for a novel that gets the
"Existing epub contains xxx chapters, web site only has yyy." message? Setting this at the
site level would be good. Per novel would be better but I think that would be a lot of work.

Why not just overwrite? Because the existing version may have chapters I haven't read yet.

Yes, I end up with multiple versions of the same novel.

Am I missing a better process for this kind of issue?

Thanks.

-Chris

[JimmXinu]

Originally Posted by ownedbycats

Jim: Anything in this change that might affect retrieval of collections?
https://archiveofourown.org/admin_posts/33031

Not that I'm aware of. But then I don't pay much attention to AO3 collections.

If you have stories that you know use collections perhaps you could try them?

Originally Posted by chrisric

Is there a way to have FFF automatically use "Add New Book" for a novel that gets the
"Existing epub contains xxx chapters, web site only has yyy." message? Setting this at the
site level would be good. Per novel would be better but I think that would be a lot of work.

Why not just overwrite? Because the existing version may have chapters I haven't read yet.

Yes, I end up with multiple versions of the same novel.

Am I missing a better process for this kind of issue?

When you ask for it be settable on per story basis, that sounds like some individual author is abusing chapters. I'm not a big fan of adding features to address individual authors.

Update mode is not settable by site or story. It's something the user can change on each download 'job', and can choose a default for.

Is this something other users would find valuable? I assume, from use pattern, that +99.9% you want the same setting. I'm honestly not sure how hard it would be to do.

As a work around:

1. Use a custom column to set FFF error to, say #fanficfare_error,
2. When you bump into one, search for #fanficfare_error:"Existing epub contains", make a saved search if you need it a lot,
3. FFF > Get story URLs from selected books (I have a hotkey for that),
4. The dialog has a copy to clipboard button, click it, ESC to close,
5. FFF > Actions by Update Mode > Download from URLs > Add New Book (set another hotkey to speed it up), hit Enter.

[LittleBit]

Originally Posted by JimmXinu

open_pages_in_browser_tries_limit is a recently added feature to keep from retrying forever. If you're using Chrome, you may need the latest test version--there was a change to caching.

Thanks! The test version did fix that problem.

So, I've been trying to simplify/automate a few things recently, but I'm stuck on this one. I have a Kindle that I'm using KOreader on and I have it set up to add to folders by fandom. Which is fine until I have crossovers and it generates a bunch of folders with like, 1 story in them. I want Calibre to have both fandoms listed but not use that column to send to KOreader. Is there a way to separate the fandoms into different columns, i.e 'MHA, Harry Potter' would into go into something like #fandom1 and #fandom2 columns, and I can just send #fandom1 to Koreader? Or have a second column that has all the single fandoms with crossovers converted to just a Crossover tag that would make a single folder in KOreader?

[JimmXinu]

Originally Posted by LittleBit

So, I've been trying to simplify/automate a few things recently, but I'm stuck on this one. I have a Kindle that I'm using KOreader on and I have it set up to add to folders by fandom. Which is fine until I have crossovers and it generates a bunch of folders with like, 1 story in them.

As a fellow KOReader user (on Kobo), I do it a different way: KOReader has a 'Calibre metadata search' feature you can use if you have a Calibre metadata.calibre file available. That works well for me because I put category(fandom) and genre in Calibre's Tags.

Whether you do or not have that available depends how you're getting your books onto the device.

(I just dump all my Calibre books(6700+) into one directory on the device. I used {id} to name them, but I'd use {uuid} if I did it again from scratch.)

Originally Posted by LittleBit

I want Calibre to have both fandoms listed but not use that column to send to KOreader. Is there a way to separate the fandoms into different columns, i.e 'MHA, Harry Potter' would into go into something like #fandom1 and #fandom2 columns, and I can just send #fandom1 to Koreader?

Yes, but it's complicated and depending on the site (coughAO3cough), one story can have hundreds of fandoms.

Originally Posted by LittleBit

Or have a second column that has all the single fandoms with crossovers converted to just a Crossover tag that would make a single folder in KOreader?

That would be easier. I'd suggest doing it as a Calibre Custom column instead of doing it in FFF. That way if you manually change the value in your multi-fandom column for a story, the single fandom column will update appropriately.

[JimmXinu]

New Version Released

Version 4.49.0 - 2025-10-01
- adapter_storiesonlinenet: Change page not found error reporting
- adapter_literotica: Yet another site change, addresses #1245
- base_otw: Add collectionsUrl and collectionsHTML metadata
- adapter_literotica: Site change
- browsercache_simple: Code for 0 length stream in cache file, only seen in Mac
- Add continue_on_chapter_error_try_limit setting
- adapter_fireflyfansnet: Allow for missing authorId.

[JimmXinu]

New Test Version Posted

2025-10-01
- Spiritfanfiction login, closes #1247, thanks kilandra
- New site: althistory.com (NOT alternatehistory.com) for #1252

Note that alternatehistory.com and althistory.com, while both xenforo forum sites, are unrelated.

FYI, althistory.com is very aggressive about their captchas. I needed to have logged in recently, solving a captcha several times (or with several steps?) before trying to download with login. I wouldn't be surprised if FFF login failed a lot.

[ownedbycats]

Is the chapter range documented anywhere in docs? I sometimes forget how to use it and searching forum thread is slow.

EDIT: Doesn't seem to be. Any objections if I add to FAQ?

[ownedbycats]

Random feature request that I don't know if possible: An option in personal.ini editor to open second window with defaults.ini. Make easier to look up things.

[JimmXinu]

Go ahead.

It's in the tooltip for the URL box, but that one says [1,2] and I usually use [1-2]. Both work.