Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 05-13-2022, 11:15 AM   #7126
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 5,382
Karma: 1434798
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
If you are updating an existing epub from FanFicFare with 74 chapters, when chapter 75 comes out, FFF should only need to fetch chapter 1 (for metadata) and chapter 75 -- assuming you don't have check_next_chapter or meta_from_last_chapter set true and that it's been long enough that chapter 1 shows 75 chapters.

I would suspect that it's the chapter 1 lookup that's failing and you're assuming it needs all chapters.

If FFF really wants to download all chapters, that suggests that the epub has been changed or something else is going on.
JimmXinu is offline   Reply With Quote
Old 05-13-2022, 05:29 PM   #7127
ownedbycats
Wizard
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 4,161
Karma: 18002570
Join Date: Oct 2018
Location: Canada
Device: Kobo Aura HD (retired), Kobo Libra H2O
Quote:
Originally Posted by JimmXinu View Post
Here are a few ideas:
  • SmartEject missing/dup searches?
  • Keep a second copy of each in another column that is New Only and compare?
  • Use Audit Log plugin?
  • Use FFF's logpage?
  • Remove changeable metadata from save path?

SmartEject catches such things well enough for me.
Actually, I think that last one might be doable. I already have #kobopath for part of my save template, so modifying that to also include the original author and title might work.
ownedbycats is online now   Reply With Quote
Old 05-13-2022, 06:18 PM   #7128
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 5,382
Karma: 1434798
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
Quote:
Originally Posted by ownedbycats View Post
Quote:
  • Remove changeable metadata from save path?
Actually, I think that last one might be doable. I already have #kobopath for part of my save template, so modifying that to also include the original author and title might work.
Actually, I meant removing title/author altogether and using {id} instead.
JimmXinu is offline   Reply With Quote
Old 05-14-2022, 07:19 AM   #7129
Ser4nb2LUY6e
Connoisseur
Ser4nb2LUY6e began at the beginning.
 
Posts: 53
Karma: 10
Join Date: Jun 2017
Device: Moon+ Reader Pro on Android
Any chance you could add a "Background Metadata" option to "Download from URLs" like it exists in "Update existing FanFiction Books"?

Also to pick up a previous discussion, "Download from URLs" is definitely not multi-threaded.

I wanted to download a bunch of stories (19) from novelfull.com, which is quite slow as I found that it doesn't work reliably without using flaresolverr.

I downloaded them all together in a single use of "Download from URLs" because I didn't want to sit around for 5 minutes each time waiting for "Downloading metadata for stories" before I can add the next one. (That's where a "Background Metadata" option would be great.)

That blocked calibre completely for almost an hour with "Downloading metadata for stories", followed by a job running for 49h which was clearly downloading one story, one chapter at a time linearly, despite calibre being configured for 18 background threads (I've got a 24core/48thread CPU).

A "Background Metadata" option to "Download from URLs" would at least have allowed me to add these manually as 19 different jobs without spending an hour waiting to add a new one every 5 minutes.

Then having "Download from URLs" either using multi-threading internally, or having a "create one job per book" option would have reduced that from 49h to maybe 3h.

(As I've said before, I would find a "create one job per book" option much more useful, as it means that I get access to the books as they are finished downloading, and don't have to wait for all of them to finish downloading.)
Ser4nb2LUY6e is offline   Reply With Quote
Old 05-14-2022, 10:47 AM   #7130
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 5,382
Karma: 1434798
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
Quote:
Originally Posted by Ser4nb2LUY6e View Post
Any chance you could add a "Background Metadata" option to "Download from URLs" like it exists in "Update existing FanFiction Books"?
No, there are too many features that depend on having the metadata, such as checking title/author dups. It's been discussed before.

Quote:
Originally Posted by Ser4nb2LUY6e View Post
Also to pick up a previous discussion, "Download from URLs" is definitely not multi-threaded.
Download is multi-threaded for different sites, but not a single site.

Some sites, if they see you downloading in parallel, will block you entirely. FFF is designed to avoid that happening. And no, I'm not interested in making that a per-site option.

novelfull.com in particular takes a long time to collect metadata because it only lists 50 chapters per page, so a story with 1500 chapters (not rare on that site) will need 30 requests just to get the chapter list for metadata.

If you like, you can experiment with making the slow_down_sleep_time setting lower than the default value of 2sec under [novelfull.com].
JimmXinu is offline   Reply With Quote
Old 05-14-2022, 10:36 PM   #7131
Ser4nb2LUY6e
Connoisseur
Ser4nb2LUY6e began at the beginning.
 
Posts: 53
Karma: 10
Join Date: Jun 2017
Device: Moon+ Reader Pro on Android
Ah well, I'll just have to start them by hand as individual jobs then.

The "slow down sleep time" probably doesn't come into effect as I'm using flaresolver for novelfull.com (had trouble without) and request take between 3.5 and 4 seconds in that.

I can download from novelfull.com without issue with 12 jobs started separately be hand though.

Quote:
Originally Posted by JimmXinu View Post
No, there are too many features that depend on having the metadata, such as checking title/author dups. It's been discussed before.



Download is multi-threaded for different sites, but not a single site.

Some sites, if they see you downloading in parallel, will block you entirely. FFF is designed to avoid that happening. And no, I'm not interested in making that a per-site option.

novelfull.com in particular takes a long time to collect metadata because it only lists 50 chapters per page, so a story with 1500 chapters (not rare on that site) will need 30 requests just to get the chapter list for metadata.

If you like, you can experiment with making the slow_down_sleep_time setting lower than the default value of 2sec under [novelfull.com].
Ser4nb2LUY6e is offline   Reply With Quote
Old 05-14-2022, 10:50 PM   #7132
Ser4nb2LUY6e
Connoisseur
Ser4nb2LUY6e began at the beginning.
 
Posts: 53
Karma: 10
Join Date: Jun 2017
Device: Moon+ Reader Pro on Android
Actually, it's all good, I just managed to implement it for myself:

dialogs.py
Code:
    
    # invoke the
    def ok_clicked(self):
        self.dialog_closing(None) # save persistent size.
        self.hide()
        for s in self.get_urlstext().splitlines(): # line added 
            self.go_signal.emit( self.get_fff_options(),
                                 s, #was self.get_urlstext()
                                 self.merge,
                                 self.extrapayload )
I acknowledge this isn't the right solution for everyone in every circumstance, but for me, that's exactly what I want.

Quote:
Originally Posted by Ser4nb2LUY6e View Post
Ah well, I'll just have to start them by hand as individual jobs then.

The "slow down sleep time" probably doesn't come into effect as I'm using flaresolver for novelfull.com (had trouble without) and request take between 3.5 and 4 seconds in that.

I can download from novelfull.com without issue with 12 jobs started separately be hand though.
Ser4nb2LUY6e is offline   Reply With Quote
Old 05-15-2022, 06:30 AM   #7133
sios00
Junior Member
sios00 began at the beginning.
 
Posts: 4
Karma: 10
Join Date: May 2022
Device: none
Anyone knows if it is possible to have FanficFare still generate a cover using Generate Cover without replacing all other images inside a fic?

For example, in this fic https://archiveofourown.org/works/14...pters/96450513 at the bottom of chapter 60, the author has placed an image. However, downloading the fic with Generate Cover in use results in that image being replaced with the cover image instead. This has also happened to other fics with images in them.

Thank you!
sios00 is offline   Reply With Quote
Old 05-15-2022, 11:09 AM   #7134
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 5,382
Karma: 1434798
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
Quote:
Originally Posted by sios00 View Post
Anyone knows if it is possible to have FanficFare still generate a cover using Generate Cover without replacing all other images inside a fic?

For example, in this fic https://archiveofourown.org/works/14...pters/96450513 at the bottom of chapter 60, the author has placed an image. However, downloading the fic with Generate Cover in use results in that image being replaced with the cover image instead. This has also happened to other fics with images in them.
I am seeing something weird going on there, but I'm not sure what yet.

Can you provide any additional examples? Especially one showing "replacing all other images"? Because the one above only appears to replace one of two places that the image appears.

UPDATE: Never mind--I think I figured it out.

Last edited by JimmXinu; 05-15-2022 at 02:42 PM.
JimmXinu is offline   Reply With Quote
Old 05-15-2022, 02:43 PM   #7135
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 5,382
Karma: 1434798
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
New Test Version Posted

2022-05-15
- Refactor img code in story.py to fix a problem when cover image also in story.

This should fix sios00's issue.
JimmXinu is offline   Reply With Quote
Old 05-15-2022, 09:35 PM   #7136
riothamus
Connoisseur
riothamus rocks like Gibraltar!riothamus rocks like Gibraltar!riothamus rocks like Gibraltar!riothamus rocks like Gibraltar!riothamus rocks like Gibraltar!riothamus rocks like Gibraltar!riothamus rocks like Gibraltar!riothamus rocks like Gibraltar!riothamus rocks like Gibraltar!riothamus rocks like Gibraltar!riothamus rocks like Gibraltar!
 
Posts: 61
Karma: 100000
Join Date: Nov 2010
Device: None
Quote:
Originally Posted by JimmXinu View Post
If you are updating an existing epub from FanFicFare with 74 chapters, when chapter 75 comes out, FFF should only need to fetch chapter 1 (for metadata) and chapter 75 -- assuming you don't have check_next_chapter or meta_from_last_chapter set true and that it's been long enough that chapter 1 shows 75 chapters.

I would suspect that it's the chapter 1 lookup that's failing and you're assuming it needs all chapters.

If FFF really wants to download all chapters, that suggests that the epub has been changed or something else is going on.
Ok, so to avoid downloading all chapters with FFDL, I would download the first chapter and new chapter(s) (with FFDL) and then FFF should update normally still?
riothamus is offline   Reply With Quote
Old 05-15-2022, 11:11 PM   #7137
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 5,382
Karma: 1434798
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
Quote:
Originally Posted by riothamus View Post
Ok, so to avoid downloading all chapters with FFDL, I would download the first chapter and new chapter(s) (with FFDL) and then FFF should update normally still?
Yes. I find it easier to use a browser for updates.
JimmXinu is offline   Reply With Quote
Old 05-16-2022, 09:06 PM   #7138
ownedbycats
Wizard
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 4,161
Karma: 18002570
Join Date: Oct 2018
Location: Canada
Device: Kobo Aura HD (retired), Kobo Libra H2O
Quote:
Originally Posted by JimmXinu View Post
I am seeing something weird going on there, but I'm not sure what yet.

Can you provide any additional examples? Especially one showing "replacing all other images"? Because the one above only appears to replace one of two places that the image appears.

UPDATE: Never mind--I think I figured it out.
This is very weird. I just came across the same glitch myself in an AO3 fic and came here to report it, only to find it's been fixed.

I found a glitch in the new test version. This happens when trying to save personal.ini, even though I don't have a fastnovels section:

Click image for larger version

Name:	2022-05-16 22_09_33-Go back to fix errors_.png
Views:	126
Size:	7.6 KB
ID:	193854

Last edited by ownedbycats; 05-16-2022 at 09:23 PM.
ownedbycats is online now   Reply With Quote
Old 05-16-2022, 09:22 PM   #7139
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 5,382
Karma: 1434798
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
That can happen if I remove a site but forget to remove the config section from defaults.ini.

But I didn't in this case--I downloaded the posted zip again to test. And [fastnovels.net] is the new name, it was [fastnovel.net] before. Which suggests you some how have old code with the new defaults.ini?

I'd download install the zip/install/restart calibre again.
JimmXinu is offline   Reply With Quote
Old 05-16-2022, 09:30 PM   #7140
ownedbycats
Wizard
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 4,161
Karma: 18002570
Join Date: Oct 2018
Location: Canada
Device: Kobo Aura HD (retired), Kobo Libra H2O
Interesting, looking at fastnovels.net in the defaults.ini was all red. Re-installing the test version fixed it though. I wonder how that happened. Maybe the first install got botched up somehow.
ownedbycats is online now   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] KindleUnpack - The Plugin DiapDealer Plugins 488 03-21-2022 03:49 AM
[GUI Plugin] Open With kiwidude Plugins 389 02-02-2022 06:03 AM
[GUI Plugin] Marvin XD Philantrop Plugins 126 01-29-2017 12:48 PM
[GUI Plugin] KiNotes -axel- Plugins 0 07-14-2013 06:39 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 11:27 PM.


MobileRead.com is a privately owned, operated and funded community.