09-04-2012, 07:17 PM | #436 |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
@vargaviktor - I have no plans to change anything. I call two different calibre utility functions which handle zips and rars, both of which are in my understanding modified versions of some standard Python libraries.
As I said on the quality check thread averaging size/dpi is not an appropriate function for that or indeed this plugin - I suggest if you want such a feature that you write your own plugin for it. Undoubtedly there would be code in plugins like this one which could assist you. |
09-07-2012, 03:12 AM | #437 | |
Member
Posts: 11
Karma: 10
Join Date: Sep 2012
Device: Alcatel T60 Tablet (Android 2.x)
|
Quote:
I would like to give a try to make a plugin, but my python is not the best. Your plugin will help me, to concentrate on the calculations. Thank you. regards. Viktor |
|
Advert | |
|
09-09-2012, 01:04 AM | #438 |
Addict
Posts: 320
Karma: 56788
Join Date: Jun 2011
Device: Kindle
|
I'm getting a wordcount that is way off the expected value... by about 90%, would be my guess. Is there any way to determine what's causing hiccup?
|
09-09-2012, 04:42 AM | #439 |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Remember that word count is for every single file in the ePub that is in the manifest. Frequently books include chapters of upcoming releases and all sorts of other cruft. Use Tweak ePub, open each HTML page in a browser, paste into word and get it to do a word count if you don't believe it. Just don't cheat and skip any pages...
|
09-10-2012, 10:32 AM | #440 | |
Addict
Posts: 320
Karma: 56788
Join Date: Jun 2011
Device: Kindle
|
Quote:
|
|
Advert | |
|
09-10-2012, 11:55 AM | #441 |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
@ElMiko - the same response applies regardless. I'm not sure what you want expect anyone to tell you. I can't do anything without seeing the book for myself, and who "says" the number is supposed to be 100K? The plugin logic is as I described above, and I've never seen any counts which are widely anomalous to the number of pages (from Goodreads) for the book. Perhaps your copy of the book is corrupted.
|
09-10-2012, 02:44 PM | #442 |
Addict
Posts: 320
Karma: 56788
Join Date: Jun 2011
Device: Kindle
|
@kiwidude - I did as you suggested and copy/pasted the text into a word document. In so doing, it seems I may discovered what the problem is, albeit not the solution. There are four html files in the MOBI file, and Count Pages isn't counting the first file.
Last edited by ElMiko; 09-10-2012 at 08:24 PM. |
09-10-2012, 05:42 PM | #443 |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
That's because it is invalid html. Look at the closing body tag immediately after the opening body tag and all the content is after the body.
|
09-10-2012, 08:25 PM | #444 |
Addict
Posts: 320
Karma: 56788
Join Date: Jun 2011
Device: Kindle
|
Thanks! Ran page count on an EPUB conversion and that did the trick. Really appreciate your looking into it!
|
09-19-2012, 02:03 AM | #445 |
null operator (he/him)
Posts: 20,568
Karma: 26954694
Join Date: Mar 2012
Location: Sydney Australia
Device: none
|
I'll make this as detailed as I can, I'm using remote control over a 64Kb/s satellite link with long latencies so I want to avoid to-ing & fro-ing with questions and answers.
I use Windows 7 64 bit with all updates installed, Calibre 0.8.69 and Count Pages 1.6.3 - My settings for the plug-in are attached. I use Process Explorer to observe what programs are doing. Some of my books are taking a long time to word count (15 minutes and more), although word counts of similar books of similar size, by the same author, in the same format, from the same source take a few seconds First what happens when I count words in a single 'slow' book. Whilst the plugin is counting a 'slow' book's words, one of the two instances of calibre-parallel uses 24.n% of my quad core I5 - i.e almost an entire core. My observation is that the main calibre process spawns a calibre-parallel process which spawns another calibre-parallel process. It's the second calibre-parallel that chews up processor resources, so I assume its the one doing the 'work'. This situation doesn't bother me too much - it gets there eventually Now what happens when I select a group of books. I can run into the situation where the group includes several of these 'slow' books. The plug-in spawns up to 5 instances of calibre-parallel. When this happens my computer can become unusable for an unacceptable period (my patience ran out after 33 minutes); because the 4 calibre-parallel secondary processes are EACH consuming 24.n% of a CPU core processing 'slow' books, totaling ~98% of the entire CPU If I fight with the sloth like mouse to stop the job, that doesn't give me back the CPU resources. Because whilst the primary instance of calibre-parallel dies, the four that were doing the work get detached from the main calibre process, and continue working independently and continue to hog the CPU, so I have to kill them individually with Process Explorer.: So rather than killing the job from calibre, its faster & easier to kill the process tree of the primary calibre-parallel process with Process Explorer. Calibre complains, but it doesn't crash and there's no apparent harm done; i.e. the books and database are OK because the plug-in is not accessing them when its doing the counting. What have I done to try to 'fix' it Based on reading this thread I progressively disabled DEP (at the command line so it was disabled for everything including Windows), disabled my AV, disabled the firewall, disconnected the router, closed all other programs (including disabling the ones that start in the tray), and restarted Windows between each, the final one into Safe Mode - all to no avail. I haven't tried Closing the Tag Browser, which I saw suggested in this thread and I've seen suggested elsewhere as a possible solution to various problems - because I can't figure out how to do that. There's no sign of any memory leaks . And the 'slow books' are in the minority, estimate < 10%. Changing the algo between 'ADE' & 'calibre E-book' makes no obvious difference to the speed. Didn't try the 'APNX ' algo because it seems to be about page counting. Is there some way to have this plug-in work in a serial manner rather than multi-tasking via spawning multiple calibre-parallel processes, i.e. limit the number of secondary instances of calibre-parallel to ONE. It would 'help' if I could determine the identity (author-title) of book(s) currently being processed in a job, or books that have completed. Then I could kill the job, put the 'slow books' aside to be done one at a time, and redo the group without the slow books. My limited tests show that a 'slow to count' book does not appear to take significantly longer to convert to & from EPUB and RTF than similar 'fast to count' books. In fact 'slow book' conversion is usually (always ?) faster 'slow book' word counting, but the opposite is true of a 'fast book' I have no doubt that its something to do with the book content, but I've no idea what - given that similar books are OK, and other plugs-ins are not slow in their processing of the same books I'll try starting calibre via the start command with the affinity switch set at 2, theoretically that should limit calibre to 2 cores only BR Last edited by BetterRed; 09-19-2012 at 03:06 AM. Reason: forgot the attachment & para re algo's |
09-19-2012, 02:59 AM | #446 | |
US Navy, Retired
Posts: 9,864
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Nexus 7
|
Quote:
You can also control cpu cores via Preferences - Miscellaneous, but once you change Job priority you should be fine. |
|
09-19-2012, 03:09 AM | #447 | |
Wizard
Posts: 1,759
Karma: 30063305
Join Date: Dec 2006
Location: Singapore
Device: Boyue
|
Quote:
|
|
09-19-2012, 03:20 AM | #448 | |
US Navy, Retired
Posts: 9,864
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Nexus 7
|
Quote:
The only thing I know for sure, changing Job priority from Normal to Low allows me the freedom to do whatever I want no matter how much work those calibre-parallel processes are doing. |
|
09-19-2012, 04:31 AM | #449 | |
null operator (he/him)
Posts: 20,568
Karma: 26954694
Join Date: Mar 2012
Location: Sydney Australia
Device: none
|
Quote:
I think I've found the root cause - I had Behaviour->Preferred Output set to EPUB and Preferred Input to RTF, because for my main workflow that's the stage I'm at i.e. I'm doing mainly doing RTF conversions to EPUB. There's a lot of earlier discussion in this thread around this issue, i.e. which format is used to do the counting - I was left with the impression that the pecking order was EPUB, MOBI, otherwise ?? Given I have an EPUB and an RTF for everything, I thought it was using the EPUBs. On a whim I pushed EPUB to the top of the Preferred Input order and the slow books become fast books and the fast books became even faster books. Kiwidude mentioned putting a preferred source format list in the plug-in Configuration similar to the one in Behaviour. I'd settle for a being able to specify a single default format as the one to use to do the count - if its not available then... whatever is easiest for kiwidude. Have to go, I'm in a bricks & mortar Library using their WiFi, its 18:30, and the real Librarian is rattling keys, and wagging her finger... BR Last edited by BetterRed; 09-19-2012 at 04:34 AM. |
|
09-19-2012, 04:55 AM | #450 | ||
US Navy, Retired
Posts: 9,864
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Nexus 7
|
Quote:
Just for fun I kicked off 1000 books counting pages. It spawned 8 callibre-parallel processes and while they're running they use their max 12.5% per core. But since I have calibre set to Low priority it never gets in the way of me responding to this post. Did you adjust the CPU items at Preferences - Miscellaneous? I don't mess with this but you can throttle things way back here. Quote:
Good Luck Last edited by DoctorOhh; 09-19-2012 at 04:59 AM. |
||
Tags |
count, count pages, page count, pages, plugin |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
[GUI Plugin] Quality Check | kiwidude | Plugins | 1184 | 04-17-2024 06:17 PM |
[GUI Plugin] Open With | kiwidude | Plugins | 403 | 04-01-2024 08:39 AM |
[GUI Plugin] Quick Preferences | kiwidude | Plugins | 62 | 03-16-2024 11:47 PM |
[GUI Plugin] Kindle Collections (old) | meme | Plugins | 2070 | 08-11-2014 12:02 AM |
[GUI Plugin] Plugin Updater **Deprecated** | kiwidude | Plugins | 159 | 06-19-2011 12:27 PM |