Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 05-28-2012, 11:02 AM   #226
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,230
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Thanks again Kovid.

(1) would mean more work for me than I want to deal with, (plus cross platform is a big problem I can't support) and (2) would likely involve a lag of months even if they did agree, leaving users stuck in the meantime. BTW if anyone else out there wants to raise the feature request to the poppler team on our behalf anyway please do so, I can't be bothered with mailing lists personally as a way of support.

Option (3) of pypdf sounds a possibility, obviously it will be slower than pdfreflow but given that isn't an option any more it should hopefully still be faster than pdftohtml on the worst pdfs. I shall have to do some testing.

Worst case it is going to be (4). Though I recall from some examples in the early days of this plugin calibre easily taking an hour or more to process the pdf which is not nice when all you want is to see if it has an ISBN - particularly if after all that time the PDF doesn't actually have one!
kiwidude is online now   Reply With Quote
Old 05-28-2012, 12:03 PM   #227
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,309
Karma: 5382313
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I just committed code that will allow you to use podofo to extract the pages, which should be pretty fast. You will need to wait till the next binary calibre release to actually use the code, since it involved making additions to the podofo C bindings in calibre.
kovidgoyal is online now   Reply With Quote
Old 05-28-2012, 01:14 PM   #228
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,230
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Thanks Kovid.

I had started doing some testing with pyPdf (and now pdofo as well, for the latter I am doing nothing but calling open at this point obviously). It is interesting that unlike pdftohtml & pdfreflow, both pyPdf and podofo choke on an epub which has security applied to it. pyPdf throws a DRMError. podofo spits an *enormous* # of console lines out all saying like this:
3 m_nPredictor=12 m_nCurPredictor=12
before throwing an exception of: ePdfError_UnsupportedFilter

Any way of preventing the console output?

Still working out which is the least evil combination as yet
kiwidude is online now   Reply With Quote
Old 05-28-2012, 01:29 PM   #229
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,309
Karma: 5382313
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by kiwidude View Post

Any way of preventing the console output?
Run it in a separate process with fork_job() in simple_worker.py. Both podofo and pypdf are pdf altering libraries and such such they choke on drmed pdfs, where poppler will not.
kovidgoyal is online now   Reply With Quote
Old 05-29-2012, 04:02 PM   #230
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,230
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@Kovid - was there any particular reason why you are steering me towards podofo rather than pyPdf? Is it purely performance? As the pyPdf API (with a DRMError) and not having to fork a console make it a bit nicer to use. Plus I can reverse order the back pages which might make a difference for a small # of books to reliability of match.

Obviously once 0.8.54 is released I will do my own testing to see how significant the performance difference is. The reality is that I just need to grab 10 front and 5 back pages so it may not be that significant?

@everyone else - just to summarise if you hadn't guessed already from all the techy speak, this plugin is broken using calibre 0.8.53. I have a fixed version here ready to go, but it needs calibre 0.8.54. I should be able to release it by this weekend I would assume, so either avoid using the plugin until then or use 0.8.52 in the meantime...
kiwidude is online now   Reply With Quote
Old 05-29-2012, 04:33 PM   #231
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,309
Karma: 5382313
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
pypdf hangs (as in goes into an infinite loop) on some PDF files. So really if you want to use it you should be running it in a worker process anyway. That's why calibre uses podofo rather than pypdf to set pdf metadata.
kovidgoyal is online now   Reply With Quote
Old 05-29-2012, 04:39 PM   #232
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,230
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Thanks for the warning, that sounds like it is best avoided then, darn it.
kiwidude is online now   Reply With Quote
Old 06-01-2012, 01:47 PM   #233
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,230
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Beta for next version - requires calibre 0.8.54

As posted above, this plugin has been broken for PDF scanning due to a change made in calibre 0.8.53. So users have either had to stick to earlier versions, or learn not to extract from PDFs .

Attached is my next intended version which requires calibre 0.8.54 - it would be appreciated if someone could give it a quick whirl before I officially release it.

I believe there is still a very slim chance in exceptional circumstances that scanning a PDF could still cause a calibre crash. I haven't had this happen, but in theory it could do. However when the next calibre 0.8.55 is available some code I have already included in this plugin version will automatically become active and safely handle that situation without crashing. I didn't want users to be stuck waiting for another week for the sake of something they probably won't have happen.

Please can one or two of you try this and let me know on the thread if any issues, then I will officially release it.

Last edited by kiwidude; 06-03-2012 at 09:10 AM. Reason: Removed attachment as officially released
kiwidude is online now   Reply With Quote
Old 06-03-2012, 09:12 AM   #234
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,230
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
v1.4.2 Released

Changes in this release:
  • Minimum version set to calibre 0.8.54 (but preferred version is 0.8.55)
  • Performance optimisation for epubs for calibre 0.8.51 to reduce unneeded computation
  • Change to calibre API for deprecated dialog which caused issues that intermittently crashed calibre
  • Minor fix to ensure HTMLPreProcessor object is initialised correctly
  • Change to using different pdf engines for pdf processing due to calibre 0.8.53 breaking the one I was using.
  • Stability improvement will activate with calibre 0.8.55 by running pdf analysis on a forked thread

Anyone who tried the beta version above, please make sure you force an update to this officially released version (has the same version number).
kiwidude is online now   Reply With Quote
Old 06-03-2012, 12:50 PM   #235
PeterT
Taking a break; Fed up
PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.
 
PeterT's Avatar
 
Posts: 7,171
Karma: 45264785
Join Date: Nov 2007
Location: Toronto
Device: Wife: Touch, Arc, Vox Me: Nexus 7, Glo
I might be pedantic but I am confused by these entries in the change log...

Quote:
Originally Posted by kiwidude View Post
[*]Minimum version set to calibre 0.8.54 (but preferred version is 0.8.55)[*]Performance optimisation for epubs for calibre 0.8.51 to reduce unneeded computation
Surely no users on calibre 0.8.51 can install this version to get the reduced computation fix..
PeterT is offline   Reply With Quote
Old 06-03-2012, 12:54 PM   #236
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,230
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Yes you are being pedantic . The performance change Kovid made was in calibre 0.8.51, and that *was* going to be the minimum version for the next release of this plugin. Then 0.8.53 came out, broke the PDF extraction so I delayed releasing and the new minimum is now 0.8.54, with 0.8.55 preferred when it gets released this week.
kiwidude is online now   Reply With Quote
Old 06-10-2012, 04:02 PM   #237
Joanna
Connoisseur
Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.
 
Posts: 65
Karma: 41772
Join Date: Feb 2012
Location: Poland
Device: none
I'm so glad to see your plugin working again! Thank you!

And one little improvement idea:
I guess plugin authors cannot change anything in the "Edit metadata" dialog but there is one thing that has been bugging me for quite a long time: I would love to have a little "Extract ISBN" button next to the IDs row in "Edit metadata" (right next to "Clear IDs" or "Paste the contents..."). It's not doable, though, is it? This would be great as I would often go to the "Edit metadata" dialog, than realize there is no ISBN for a given book, close the dialog, extract ISBN and open the "Edit metadata" dialog once again... not very practical.
Joanna is offline   Reply With Quote
Old 06-10-2012, 04:42 PM   #238
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,230
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@Joanna, no it is not currently possible. There is no provision in calibre to enable launching a plugin from the edit metadata dialog, only from the main toolbars or context menu.

All I can suggest is having a custom column to display isbn or having a look at the book details panel on the right before opening the edit metadata window.
kiwidude is online now   Reply With Quote
Old 06-10-2012, 05:11 PM   #239
Joanna
Connoisseur
Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.Joanna is an enigma wrapped up in a mystery.
 
Posts: 65
Karma: 41772
Join Date: Feb 2012
Location: Poland
Device: none
Thanks, that's what I thought . It's a pity, Extract ISBN should be a built-in Calibre feature .

Thanks for the tips; unfortunately I don't have enough space to have ISBN shown as a custom column or in book details. My workaround consists of trying to extract ISBN before I even go to the "Edit metadata" dialog .
Joanna is offline   Reply With Quote
Old 06-15-2012, 11:10 PM   #240
affa
Connoisseur
affa can extract oil from cheeseaffa can extract oil from cheeseaffa can extract oil from cheeseaffa can extract oil from cheeseaffa can extract oil from cheeseaffa can extract oil from cheeseaffa can extract oil from cheeseaffa can extract oil from cheeseaffa can extract oil from cheese
 
Posts: 56
Karma: 1234
Join Date: Apr 2011
Device: kindle
can i ask a reallllllly silly question? what is the main benefit of extracting an isbn? is it so you can then easier grab proper metadata?
affa is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[Old Thread] Extract ISBN from file name ChristianQ Calibre 56 05-20-2012 10:59 AM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 01:27 PM
[Old Thread] Auto Extract ISBN-Feature request UnraisedArc Calibre 60 03-23-2011 10:31 AM
Displaying ISBN column in the main GUI tilleydog Library Management 26 02-25-2011 05:08 AM
Extract ISBN from PDF? mdroberts Calibre 10 12-15-2009 02:35 AM


All times are GMT -4. The time now is 06:55 AM.


MobileRead.com is a privately owned, operated and funded community.