[GUI Plugin] FanFicFare - Page 561

applegaa · 05-08-2023, 02:00 AM

Quote:

Originally Posted by JimmXinu

New Test Version Posted

2023-05-06
- Add order_chapters_by_date option for literotica.com, used to be hard coded.

Code:

[literotica.com]
...
## Force chapters in a story to be sorted by date instead of the order
## given by the author.  Used to be the default for literotica.
## Note that FFF normalizes literotica.com story URLs to the first
## chapter URL.  If the first chapter is not the same by date and by
## list, you may need to set order_chapters_by_date under *both*
## [storyURL] sections.
order_chapters_by_date:false

It works great for me. Thanks!

- Andrew

Mandabar · 05-08-2023, 03:05 AM

Quote:

Originally Posted by JimmXinu

New Test Version Posted

2023-05-04
- Add force_update_epub_always option to update when EPUB has more chapters than source. See #950 #949 #942.
[*]FanFicFare is meant for fanfiction. If an author has removed their content to make money on it, I'm not interested in helping work around that.

I do, however, believe that a way to update stories even when the EPUB has more chapters is worthwhile. That way a user can continue updating such a story with publicly published chapters without having to edit the EPUB first.

See, I just always make a new book for the new chapters when that happens. So no biggie but neat a new feature that is disabled by default (and commented). Great!

So that's why I was surprised when I was updating this book and I got a log result of 'adding -27 chapters for 33 total'. Negative chapters? Whaaaa? I double checked and 1) I'm not even running the test version (I'm on version 4.23.0) 2) it's not in my ini to be enabled. So that confused me.

Checked the source and it seems the author condensed his split up chapters into single chapters so no actual story went missing.
I'm just confused why instead of getting an error message as usual (Existing epub has 61 chapters, current source has 34) I got the above.

Here's the nondebug log if it matters. (Or maybe it is debug? Says it often enough

)

Spoiler:

Code:

Download 1 FanFiction Book(s)
FFF: INFO: 2023-05-08 01:29:25,020: calibre_plugins.fanficfare_plugin.jobs(49): CPUs:6
FFF: INFO: 2023-05-08 01:29:25,020: calibre_plugins.fanficfare_plugin.jobs(52): FanFicFare v4.23.0
FFF: INFO: 2023-05-08 01:29:25,021: calibre_plugins.fanficfare_plugin.jobs(68): Launch background process for site www.royalroad.com:
https://www.royalroad.com/fiction/65226
FFF: INFO: 2023-05-08 01:29:42,639: calibre_plugins.fanficfare_plugin.jobs(130): 

================================================================================ site:(www.royalroad.com)
FFF: INFO: 2023-05-08 01:29:27,004: calibre_plugins.fanficfare_plugin.jobs(211): 

-------------------------------------------------------------------------------- https://www.royalroad.com/fiction/65226
FFF: DEBUG: 2023-05-08 01:29:27,045: configurable.py(1051): use_browser_cache:
FFF: DEBUG: 2023-05-08 01:29:27,045: configurable.py(1071): use_basic_cache:true
FFF: DEBUG: 2023-05-08 01:29:27,046: adapter_royalroadcom.py(151): URL: https://www.royalroad.com/fiction/65226
FFF: DEBUG: 2023-05-08 01:29:27,047: cache_basic.py(116): 
========== MISS (GET) BasicCache
https://www.royalroad.com/fiction/65226
FFF: DEBUG: 2023-05-08 01:29:27,047: fetcher_requests.py(114): 
---------- REQ (GET) RequestsFetcher
https://www.royalroad.com/fiction/65226
FFF: DEBUG: 2023-05-08 01:29:27,910: fetcher_requests.py(127): response code:200
FFF: DEBUG: 2023-05-08 01:29:27,910: decorators.py(112): fromcache:False
FFF: DEBUG: 2023-05-08 01:29:27,911: requestable.py(55): Encoding:utf8
FFF: DEBUG: 2023-05-08 01:29:27,911: requestable.py(89): Applying max_zalgo:2
FFF: DEBUG: 2023-05-08 01:29:28,756: adapter_royalroadcom.py(224): stars:(4.571135)
FFF: DEBUG: 2023-05-08 01:29:28,767: base_adapter.py(596): setCoverImage(https://www.royalroad.com/fiction/65226,https://www.royalroadcdn.com/public/covers-large/65226-rise-of-the-devourer-isekai-litrpg.jpg?time=1681297227)
FFF: DEBUG: 2023-05-08 01:29:28,768: story.py(1474): addImgUrl(parenturl=https://www.royalroad.com/fiction/65226,url=https://www.royalroadcdn.com/public/covers-large/65226-rise-of-the-devourer-isekai-litrpg.jpg?time=1681297227,cover=True,coverexclusion=(imgur.com/dzOACJf.png|/forum/images/smilies/)
FFF: DEBUG: 2023-05-08 01:29:28,769: cache_basic.py(116): 
========== MISS (GET) BasicCache
https://www.royalroadcdn.com/public/covers-large/65226-rise-of-the-devourer-isekai-litrpg.jpg?time=1681297227
FFF: DEBUG: 2023-05-08 01:29:28,769: fetcher_requests.py(114): 
---------- REQ (GET) RequestsFetcher
https://www.royalroadcdn.com/public/covers-large/65226-rise-of-the-devourer-isekai-litrpg.jpg?time=1681297227
FFF: DEBUG: 2023-05-08 01:29:29,041: fetcher_requests.py(127): response code:200
FFF: DEBUG: 2023-05-08 01:29:29,042: decorators.py(112): fromcache:False
FFF: INFO: 2023-05-08 01:29:31,769: calibre_plugins.fanficfare_plugin.jobs(360): Do update - epub(61) vs url(34)
FFF: INFO: 2023-05-08 01:29:31,769: calibre_plugins.fanficfare_plugin.jobs(361): write to C:\Users\Anon\AppData\Local\Temp\calibre_uz5kp7vi\fanficfare_sufnl68u\bgmeta-atync7nl.epub
FFF: INFO: 2023-05-08 01:29:31,770: base_writer.py(183): Save directly to file: C:\Users\Anon\AppData\Local\Temp\calibre_uz5kp7vi\fanficfare_sufnl68u\bgmeta-atync7nl.epub
FFF: DEBUG: 2023-05-08 01:29:37,831: adapter_royalroadcom.py(250): Getting chapter text from: https://www.royalroad.com/fiction/65226/rise-of-the-devourer/chapter/1201165/33-a-new-spear
FFF: DEBUG: 2023-05-08 01:29:37,831: cache_basic.py(116): 
========== MISS (GET) BasicCache
https://www.royalroad.com/fiction/65226/rise-of-the-devourer/chapter/1201165/33-a-new-spear
FFF: DEBUG: 2023-05-08 01:29:37,832: fetcher_requests.py(114): 
---------- REQ (GET) RequestsFetcher
https://www.royalroad.com/fiction/65226/rise-of-the-devourer/chapter/1201165/33-a-new-spear
FFF: DEBUG: 2023-05-08 01:29:38,352: fetcher_requests.py(127): response code:200
FFF: DEBUG: 2023-05-08 01:29:38,352: decorators.py(112): fromcache:False
FFF: DEBUG: 2023-05-08 01:29:38,353: requestable.py(55): Encoding:utf8
FFF: DEBUG: 2023-05-08 01:29:38,354: requestable.py(89): Applying max_zalgo:2
FFF: DEBUG: 2023-05-08 01:29:38,752: adapter_royalroadcom.py(250): Getting chapter text from: https://www.royalroad.com/fiction/65226/rise-of-the-devourer/chapter/1203429/34-astral-rift
FFF: DEBUG: 2023-05-08 01:29:38,752: cache_basic.py(116): 
========== MISS (GET) BasicCache
https://www.royalroad.com/fiction/65226/rise-of-the-devourer/chapter/1203429/34-astral-rift
FFF: DEBUG: 2023-05-08 01:29:38,753: fetcher_requests.py(114): 
---------- REQ (GET) RequestsFetcher
https://www.royalroad.com/fiction/65226/rise-of-the-devourer/chapter/1203429/34-astral-rift
FFF: DEBUG: 2023-05-08 01:29:39,288: fetcher_requests.py(127): response code:200
FFF: DEBUG: 2023-05-08 01:29:39,288: decorators.py(112): fromcache:False
FFF: DEBUG: 2023-05-08 01:29:39,291: requestable.py(55): Encoding:utf8
FFF: DEBUG: 2023-05-08 01:29:39,291: requestable.py(89): Applying max_zalgo:2
FFF: INFO: 2023-05-08 01:29:39,853: writer_epub.py(369): Saving EPUB Version 2.0
FFF: DEBUG: 2023-05-08 01:29:40,028: writer_epub.py(216): existing logfile found, appending
FFF: INFO: 2023-05-08 01:29:40,978: main.py(274): ## Polishing: EPUB
FFF: INFO: 2023-05-08 01:29:41,046: main.py(164): 
### Smartening punctuation
FFF: INFO: 2023-05-08 01:29:41,063: replace.py(163): Smartened punctuation in: OEBPS/title_page.xhtml
FFF: INFO: 2023-05-08 01:29:41,084: replace.py(163): Smartened punctuation in: OEBPS/log_page.xhtml
FFF: INFO: 2023-05-08 01:29:41,726: replace.py(163): Smartened punctuation in: OEBPS/file0015.xhtml
FFF: INFO: 2023-05-08 01:29:41,853: replace.py(163): Smartened punctuation in: OEBPS/file0019.xhtml
FFF: INFO: 2023-05-08 01:29:42,034: replace.py(163): Smartened punctuation in: OEBPS/file0025.xhtml
FFF: INFO: 2023-05-08 01:29:42,207: replace.py(163): Smartened punctuation in: OEBPS/file0033.xhtml
FFF: INFO: 2023-05-08 01:29:42,248: replace.py(163): Smartened punctuation in: OEBPS/file0034.xhtml
FFF: INFO: 2023-05-08 01:29:42,253: main.py(217): 
FFF: INFO: 2023-05-08 01:29:42,316: main.py(278): ----------------------------------------------------------------------
FFF: INFO: 2023-05-08 01:29:42,316: main.py(279): Polishing took: 1.3 seconds

FFF: INFO: 2023-05-08 01:29:42,640: calibre_plugins.fanficfare_plugin.jobs(142): 
Download Results:
Add https://www.royalroad.com/fiction/65226 Update epub completed, added -27 chapters for 34 total.

FFF: INFO: 2023-05-08 01:29:42,640: calibre_plugins.fanficfare_plugin.jobs(166): 
Add
https://www.royalroad.com/fiction/65226

The earlier epub is available if required thru dropbox previous versions archiving for a period of time if needed.

Quote:

Originally Posted by ownedbycats

Wondering: what is the reasoning for is_adult not enabled by default?

The reasoning for that seems to be the same reason you don't sell hot coffee w/o labeling and warning it's hot. So you don't get sued.

It's one thing if a kid uses this software and can't make naughty epubs of his fanfics w/o changing a setting, it's another if it does it out of the box and the parents/authorities come looking for someone to blame instead of poor horny Timmy.

That's my guess at least.

ownedbycats · 05-08-2023, 07:14 AM

@Mandabar: Is your Calibre library in dropbox? You shouldn't do that. So so many threads about libraries being messed up from that.

Mandabar · 05-08-2023, 07:26 AM

Quote:

Originally Posted by ownedbycats

@Mandabar: Is your Calibre library in dropbox? You shouldn't do that. So so many threads about libraries being messed up from that.

Never had an issue with my library in the Dropbox, actually has saved me a few times! The only 'issue' I've ever seen is when updating metadata in a batch like fanficfare does (DB tries to upload file before fff goes to do another metadata change and locks the file from Calibre use) and the solution to that is easy. Just pause syncing before hitting yes.

JimmXinu · 05-08-2023, 11:11 AM

Quote:

Originally Posted by Mandabar

So that's why I was surprised when I was updating this book and I got a log result of 'adding -27 chapters for 33 total'. Negative chapters? Whaaaa? I double checked and 1) I'm not even running the test version (I'm on version 4.23.0) 2) it's not in my ini to be enabled. So that confused me.

Checked the source and it seems the author condensed his split up chapters into single chapters so no actual story went missing.
I'm just confused why instead of getting an error message as usual (Existing epub has 61 chapters, current source has 34) I got the above.
...
The earlier epub is available if required thru dropbox previous versions archiving for a period of time if needed.

That sounds like something I should take a look at, if you can easily grab it. Thanks.

Quote:

Originally Posted by Mandabar

It's one thing if a kid uses this software and can't make naughty epubs of his fanfics w/o changing a setting, it's another if it does it out of the box and the parents/authorities come looking for someone to blame instead of poor horny Timmy.

Mandabar · 05-08-2023, 07:45 PM

Quote:

Originally Posted by JimmXinu

That sounds like something I should take a look at, if you can easily grab it. Thanks.

Yeah sure, this is the new version.

And this is the older version.

I'll keep the links up a day or two.

JimmXinu · 05-08-2023, 09:51 PM

Quote:

Originally Posted by Mandabar

I'll keep the links up a day or two.

Thanks, I'm done with them.

What you bumped into is a difference in the logic when metadata collection is run in foreground vs background. You can see in log when it's BG metadata; the temp epub file is bgmeta-xxxxxx.epub instead of title-author-xxxxxxx.epub.

I basically never use background metadata, so I tend to forget about it.

I have a fix, but I'm looking at something else I want to consider more for the next test version.

Mandabar · 05-08-2023, 10:27 PM

Quote:

Originally Posted by JimmXinu

Thanks, I'm done with them.

Quote:

What you bumped into is a difference in the logic when metadata collection is run in foreground vs background. You can see in log when it's BG metadata; the temp epub file is bgmeta-xxxxxx.epub instead of title-author-xxxxxxx.epub.

I basically never use background metadata, so I tend to forget about it.

Huh weird. Basically the only time I do foreground is only the first scrape when it's required for the first adding to the library, otherwise I always do it in the background to prevent my usage of Calibre while it gathers the metadata being blocked before doing the normal job work. Odd I've never had this sort of issue before... or at least not that I remember vividly. As long as It doesn't happen when another of my RR authors decides to go stub their fic more it should be fine. Besides this was a sorta odd situation with the author merging so many of their chapters together hah.

Quote:

I have a fix, but I'm looking at something else I want to consider more for the next test version.

And take your time. Thanks again for what you do Jimm. You are amazing!

JimmXinu · 05-08-2023, 11:59 PM

New Test Version Posted

2023-05-08
- Fix Update Always metadata in BG logic
- fix_relative_text_links: Keep #anchor links if target also in chapter. See #952

ownedbycats · 05-09-2023, 02:32 PM

Question: I have orphan_account set to replace to 'Unknown.' However, this results in the orphan_account URL getting set for Unknown in the Calibre db.

Using exclude_metadata_pre on authorUrl just creates a 'None' link that crashed ebook-viewer when clicked. (Probably no need to bother fixing it, unless there's other reasons exclude author URL during update.)

Unknown doesn't appear in book details, so the only place I see it is in the author manager, so I'm inclined to just leave it. But is there another method I didn't think of?

JimmXinu · 05-09-2023, 10:14 PM

Yeaaaah...

I'm going to file this one under "Don't remove authorUrl, weird things happen".

The same bucket also contains authorId, storyId, storyUrl, site and probably a few others I can't think of off hand.

Lesternixon · 05-10-2023, 06:29 AM

When adding to the library for the first time, I only perform the first scrape in the foreground; otherwise, I always perform it in the background to avoid blocking Calibre's use while it collects the necessary metadata. Oddly, I don't recall ever experiencing a problem of this nature before. Everything should be ok as long as it doesn't happen when one of my RR authors decides to go stub their fic more. In addition, the author's decision to combine so many of their chapters together was a bit strange.

jherazob · 05-11-2023, 11:25 AM

Hi! Was forwarded here as i'm mostly interested in the plugin version of the tool (might try the CLI version at some point but for now the plugin fulfills my needs).

Has anybody cracked how to properly use covers with anthologies? Seems like the documented instructions assume you're using the CLI tool and not the plugin. I have defined the cover on the ebook metadata and still doesn't generate it on the EPUB file, but i suspect that since anthologies are a special case they're just not accounted for. In addition, i'm taking the stories from DeviantArt (which is the whole reason i'm using anthologies in the first place, DA has no concept of index or chapters), so all the stories lack covers and therefore i cannot tell it to look for the cover in them.

What should i do to do this? Is there a way to tell it "The chapters are these, and the cover is here" or something like that?

julo · 05-12-2023, 03:04 PM

@ jherazob
Have you looked ay the 'Generate Cover' plug-in?
https://www.mobileread.com/forums/sh...d.php?t=124219
While I have it installed I never used it so cannot say any more.

jherazob · 05-12-2023, 03:32 PM

Yes, it makes no difference whether i add the cover via editing the metadata or generated using the plugin (which is nicer, honestly), same result, an EPUB file with no cover. Calibre insists that the book has a cover, but it's absent from the file, logic would say that editing the epub metadata would keep the cover but doesn't look like it does. Was pointed at this FAQ entry about it which plainly states that the cover of the first story will be used by default as the cover for the anthology, but as mentioned DeviantArt stories have no covers, so logic would suggest that it should look at the metadata entry for the book and use that, but doesn't seem to do it.

Quote:

Originally Posted by julo

@ jherazob
Have you looked ay the 'Generate Cover' plug-in?
https://www.mobileread.com/forums/sh...d.php?t=124219
While I have it installed I never used it so cannot say any more.

05-09-2023, 02:32 PM	#8410
ownedbycats Custom User Title Posts: 11,432 Karma: 81357187 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	Question: I have orphan_account set to replace to 'Unknown.' However, this results in the orphan_account URL getting set for Unknown in the Calibre db. Using exclude_metadata_pre on authorUrl just creates a 'None' link that crashed ebook-viewer when clicked. (Probably no need to bother fixing it, unless there's other reasons exclude author URL during update.) Unknown doesn't appear in book details, so the only place I see it is in the author manager, so I'm inclined to just leave it. But is there another method I didn't think of? Last edited by ownedbycats; 05-09-2023 at 02:54 PM. Reason: forgot a word

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] KindleUnpack - The Plugin	DiapDealer	Plugins	527	08-15-2025 02:36 PM
[GUI Plugin] Open With	kiwidude	Plugins	404	02-21-2025 06:42 AM
[GUI Plugin] Marvin XD	Philantrop	Plugins	126	01-29-2017 01:48 PM
[GUI Plugin] KiNotes	-axel-	Plugins	0	07-14-2013 07:39 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 01:27 PM

05-08-2023, 07:14 AM	#8403
ownedbycats Custom User Title Posts: 11,432 Karma: 81357187 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	@Mandabar: Is your Calibre library in dropbox? You shouldn't do that. So so many threads about libraries being messed up from that.

05-08-2023, 11:59 PM	#8409
JimmXinu Plugin Developer Posts: 7,196 Karma: 5006097 Join Date: Dec 2011 Location: Midwest USA Device: Kobo Clara Colour running KOReader	New Test Version Posted 2023-05-08 - Fix Update Always metadata in BG logic - fix_relative_text_links: Keep #anchor links if target also in chapter. See #952

05-09-2023, 10:14 PM	#8411
JimmXinu Plugin Developer Posts: 7,196 Karma: 5006097 Join Date: Dec 2011 Location: Midwest USA Device: Kobo Clara Colour running KOReader	Yeaaaah... I'm going to file this one under "Don't remove authorUrl, weird things happen". The same bucket also contains authorId, storyId, storyUrl, site and probably a few others I can't think of off hand.

05-10-2023, 06:29 AM	#8412
Lesternixon Junior Member Posts: 1 Karma: 10 Join Date: May 2023 Device: math	When adding to the library for the first time, I only perform the first scrape in the foreground; otherwise, I always perform it in the background to avoid blocking Calibre's use while it collects the necessary metadata. Oddly, I don't recall ever experiencing a problem of this nature before. Everything should be ok as long as it doesn't happen when one of my RR authors decides to go stub their fic more. In addition, the author's decision to combine so many of their chapters together was a bit strange.

05-11-2023, 11:25 AM	#8413
jherazob Junior Member Posts: 8 Karma: 10 Join Date: May 2023 Device: Android tablet	Hi! Was forwarded here as i'm mostly interested in the plugin version of the tool (might try the CLI version at some point but for now the plugin fulfills my needs). Has anybody cracked how to properly use covers with anthologies? Seems like the documented instructions assume you're using the CLI tool and not the plugin. I have defined the cover on the ebook metadata and still doesn't generate it on the EPUB file, but i suspect that since anthologies are a special case they're just not accounted for. In addition, i'm taking the stories from DeviantArt (which is the whole reason i'm using anthologies in the first place, DA has no concept of index or chapters), so all the stories lack covers and therefore i cannot tell it to look for the cover in them. What should i do to do this? Is there a way to tell it "The chapters are these, and the cover is here" or something like that?

05-12-2023, 03:04 PM	#8414
julo Connoisseur Posts: 72 Karma: 12 Join Date: Apr 2010 Location: Pittsburgh area Device: prs-505,900,T2	@ jherazob Have you looked ay the 'Generate Cover' plug-in? https://www.mobileread.com/forums/sh...d.php?t=124219 While I have it installed I never used it so cannot say any more.

Advert

Advert