Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 05-11-2016, 07:50 AM   #1
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 28,810
Karma: 7295999
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Beta testers needed

Hi all,

I've spent the last two weeks doing some major surgery to calibre's internals to remove a dependency on ImageMagick for image processing. The result is that that the calibre installation size decreases by 15MB and (hopefully) all image processing functionality remains intact. The immediate motivation for this change was some security issues recently found in ImageMagick, though it has been on my long term TODO list for a while.

However, since these changes are fairly invasive, I may well have broken bits of functionality here and there. Particularly if you use any third party plugins that do image processing. So I would appreciate it if some of you could downloads betas and try them out.

In addition to the image processing changes, I also changed the way file open/save dialogs are used in calibre on windows (they are now used via a helper process) -- this should hopefully resolve most of the crashes caused by badly designed Shell extensions on windows.

Please report any problems you encounter here.

https://download.calibre-ebook.com/betas/

Thanks.

P.S. There is an extra windows 64-bit build in that folder that has Qt 5.5.1 and was compiled with Visual Studio 2015 (calibre-64bit-2.56.0-vs2015.msi) Dont install that unless you want to test out even more beta code (calibre 3.0 will be based on it)
kovidgoyal is online now   Reply With Quote
Old 05-11-2016, 09:12 AM   #2
MerlinMama
Groupie
MerlinMama began at the beginning.
 
MerlinMama's Avatar
 
Posts: 180
Karma: 10
Join Date: May 2014
Location: Canada
Device: Kobo Aura
Since installing the beta version, I can no longer find the Generate Cover Plug-in. I had it on my toolbar but now it's gone, and if I go to add it back, it is not available. It does, however, still show as installed in the list of plugins.
MerlinMama is offline   Reply With Quote
Old 05-11-2016, 09:25 AM   #3
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 9,148
Karma: 12508023
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo: Touch, Glo, Aura H2O, Glo HD
Quote:
Originally Posted by MerlinMama View Post
Since installing the beta version, I can no longer find the Generate Cover Plug-in. I had it on my toolbar but now it's gone, and if I go to add it back, it is not available. It does, however, still show as installed in the list of plugins.
Unfortuntely, Generate Cover uses ImageMagick, so it won't load. I don't really use it, but will have a look at it over the weekend if no one else does.

The Marvin XD plugin also uses ImageMagick. But, as it doesn't work with recent iOS versions, I don't have any reason to look at it.
davidfor is offline   Reply With Quote
Old 05-11-2016, 09:36 AM   #4
MerlinMama
Groupie
MerlinMama began at the beginning.
 
MerlinMama's Avatar
 
Posts: 180
Karma: 10
Join Date: May 2014
Location: Canada
Device: Kobo Aura
Quote:
Originally Posted by davidfor View Post
Unfortuntely, Generate Cover uses ImageMagick, so it won't load. I don't really use it, but will have a look at it over the weekend if no one else does.
I hope it can be made to work. I use it VERY often. For now I'll go back to the 'regular' Calibre version and keep my fingers crossed...
MerlinMama is offline   Reply With Quote
Old 05-11-2016, 09:42 AM   #5
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 28,810
Karma: 7295999
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I have created a portability layer for plugins, so most plugins that use ImageMagick should still work. However, the one part that could not be ported is writing text on images.

For that, the plugin would need to switch over to using QPainter (which is a better approach anyway and is what the generate covers code in calibre itself uses). Feel free to ask me if you have questions on how to port any specific functionality.

Last edited by kovidgoyal; 05-11-2016 at 09:51 AM.
kovidgoyal is online now   Reply With Quote
Old 05-11-2016, 09:56 AM   #6
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 9,148
Karma: 12508023
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo: Touch, Glo, Aura H2O, Glo HD
Quote:
Originally Posted by kovidgoyal View Post
I have created a portability layer for plugins, so most plugins should still work. However, the one part that could not be ported is writing text on images.

For that, the plugin would need to switch over to using QPainter (which is a better approach anyway and is what the generate covers code in calibre itself uses). Feel free to ask me if you have questions on how to port any specific functionality.
OK. I just quickly looked at the code for Generate Cover and couldn't find matches for a few things. The QPainter hint will probably be enough to get started.
davidfor is offline   Reply With Quote
Old 05-11-2016, 10:19 PM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 28,810
Karma: 7295999
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
So other than the Generate Cover Plugin, no issues?
kovidgoyal is online now   Reply With Quote
Old 05-11-2016, 10:31 PM   #8
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 9,148
Karma: 12508023
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo: Touch, Glo, Aura H2O, Glo HD
Nothing I saw. But, I just realised I didn't have the cover options turned on in the KoboTouch driver. They've been working fine while I have been running from source, but I'll check with the beta tonight.
davidfor is offline   Reply With Quote
Old 05-11-2016, 10:43 PM   #9
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 18,097
Karma: 72218357
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
The only thing I've really done with images since you started cutting out imagemagick is generating covers using the builtin feature after FanFicFare downloads.

But FWIW that seems to be working fine.
eschwartz is online now   Reply With Quote
Old 05-12-2016, 12:08 AM   #10
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 28,810
Karma: 7295999
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
The basic use cases should all be fine, I've tested them pretty extensively myself. The problem is calibre is so large and has so many parts that there is no way I can cover every combination -- which is why I'm hoping for a spot of testing from the user community before I release this code.
kovidgoyal is online now   Reply With Quote
Old 05-12-2016, 12:25 AM   #11
Terisa de morgan
Wizard
Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.
 
Terisa de morgan's Avatar
 
Posts: 3,185
Karma: 2981742
Join Date: Jun 2009
Location: Madrid, Spain
Device: Kobo Aura, Kobo Mini, Kobo Aura HD, elephone, iPad, Galaxy Tab3 8
I'm using it now. Quite basic use: edition and library management, but I use regularly a number of plugins (although not graphical) and no problem at all.
Terisa de morgan is online now   Reply With Quote
Old 05-12-2016, 10:29 AM   #12
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 9,148
Karma: 12508023
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo: Touch, Glo, Aura H2O, Glo HD
I've done some testing of sending the cover images tonight. There were no problems. I can't see a difference between the old and new versions. Including greyscale covers. I thought they were being skipped in Windows.
davidfor is offline   Reply With Quote
Old 05-12-2016, 11:22 AM   #13
jackie_w
Wizard
jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.
 
Posts: 3,490
Karma: 5609022
Join Date: Sep 2009
Location: UK
Device: Sony PRS-350, Kobo Glo/AuraHD/Aura6"/AuraH2O/GloHD
@ Kovid,

I found a problem when using my ScrambleEbook plugin. The problem arises when trying to scramble GIF images (JPG, JPEG, PNG, SVG seem OK).

This is the error message:
Code:
calibre, version 2.56.0
ERROR: Unhandled exception: <b>TypeError</b>:file() argument 1 must be encoded string without NULL bytes, not str

calibre 2.56 [64bit]  embedded-python: True is64bit: True
Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.9
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Successfully initialized third party plugins: DeDRM && Obok DeDRM && ScrambleEbook && Fantastic Fiction && Goodreads && JS Mod Epub && Diaps Editing Toolbag && DOC Input && KePub Metadata Writer && Barnes & Noble && Search The Internet && Count Pages && Wiki Reader && Quality Check && Open With && Copy Cover To Device && EpubSplit && JSTest Plugin && KindleUnpack - The Plugin && KePub Metadata Reader && Extract ISBN && KePub Input && Modify ePub && KoboTouchCSSEdit && Kobo Utilities && KePub Output && KoboBooks
Traceback (most recent call last):
  File "calibre_plugins.scrambleebook_plugin.scrambleebook", line 334, in create_scramble_book
  File "calibre_plugins.scrambleebook_plugin.scrambleebook", line 409, in __init__
  File "calibre_plugins.scrambleebook_plugin.scrambleebook", line 437, in scramble_main
  File "calibre_plugins.scrambleebook_plugin.scrambleebook", line 540, in scramble_img
  File "site-packages\calibre\utils\magick\legacy.py", line 135, in export
  File "site-packages\Pillow-2.4.0-py2.7-win-amd64.egg\PIL\Image.py", line 2093, in open
TypeError: file() argument 1 must be encoded string without NULL bytes, not str

This is the Python method where the error occurs (the red line). It processes all image formats except .SVG:
Spoiler:
Code:
def scramble_img(self, name, scramble_dgts=False):
    if self.eb.mime_map[name] in OEB_RASTER_IMAGES:
        data = self.eb.parsed(name)
        oldimg = Image()
        oldpath = self.eb.name_to_abspath(name)
        try:
            oldimg.load(data)
            wid, hgt = oldimg.size
        except:
            wid, hgt = (50, 50)
        try:
            fmt = oldimg.format
        except:
            x, x, fmt = get_nameparts(name)
        
        newimg = Image()
        newimg.load(self.dummyimg)
        newimg.size = (wid, hgt)
        
        self.eb.replace(name, newimg.export(fmt.upper()))

  • The relevant import: from calibre.utils.magick import Image
  • self.eb is a standard calibre container object (epub/azw)

I've attached a test epub in case you need it.
Attached Files
File Type: epub Images_test.epub (436.8 KB, 17 views)

Last edited by jackie_w; 05-12-2016 at 12:45 PM. Reason: Can't even get the name of my own plugin right!
jackie_w is offline   Reply With Quote
Old 05-12-2016, 12:41 PM   #14
jackie_w
Wizard
jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.
 
Posts: 3,490
Karma: 5609022
Join Date: Sep 2009
Location: UK
Device: Sony PRS-350, Kobo Glo/AuraHD/Aura6"/AuraH2O/GloHD
@Kovid,

I'm doing well here! I've also found a problem when using my Copy Cover plugin to send covers to Pocketbook devices which must be .BMP 4BPP grayscale format.

This is the error message:
Code:
calibre, version 2.56.0
ERROR: Unhandled exception: <b>ValueError</b>:Failed to export image as BMP3 with error: Unsupported image format

calibre 2.56 [64bit]  embedded-python: True is64bit: True
Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.9
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Successfully initialized third party plugins: DeDRM && Obok DeDRM && ScrambleEbook && Fantastic Fiction && Goodreads && JS Mod Epub && Diaps Editing Toolbag && DOC Input && KePub Metadata Writer && Barnes & Noble && Search The Internet && Count Pages && Wiki Reader && Quality Check && Open With && Copy Cover To Device && EpubSplit && JSTest Plugin && KindleUnpack - The Plugin && KePub Metadata Reader && Extract ISBN && KePub Input && Modify ePub && KoboTouchCSSEdit && Kobo Utilities && KePub Output && KoboBooks
Traceback (most recent call last):
  File "calibre_plugins.copy_cover_to_device.dialogs", line 91, in do_book_action
  File "calibre_plugins.copy_cover_to_device.dialogs", line 140, in copy_to_pocketbook_16
  File "site-packages\calibre\utils\magick\legacy.py", line 92, in save
  File "site-packages\calibre\utils\magick\legacy.py", line 139, in export
  File "site-packages\calibre\utils\img.py", line 124, in image_to_data
ValueError: Failed to export image as BMP3 with error: Unsupported image format
These are the relevant plugin methods. The error relates to the red line:
Spoiler:
Code:
def copy_to_pocketbook_16(self, screen, aspect, cdata, dest):
    img, rescale = self.cover_resize(screen, aspect, cdata)
    # Resize image then convert to greyscale
    if rescale:
        img.size = rescale
    img.quantize(number_colors=16, colorspace='GRAYColorspace')
    # save_cover_data_to(img, dest)
    img.type = "PaletteType"
    img.save(dest, 'BMP3')
    return rescale

def cover_resize(self, screen, aspect, cdata):
    img = Image()
    img.load(cdata)
    rescale = None
    if aspect:
        swidth, sheight = screen
        cwidth, cheight = img.size[:2]
        # calc new image size
        # Set rescale factor to match screen height
        rescale = (int(sheight * cwidth / cheight), sheight)
        if rescale[0] > swidth:
            # if too wide for screen, set to match screen width
            rescale = (swidth, int(swidth * cheight / cwidth))
    else:
        rescale = screen
    return img, rescale


The ImageMagick-related imports are these:
Code:
from calibre.utils.magick.draw import save_cover_data_to
from calibre.utils.magick import Image, create_canvas
I tested as well as I can given I don't own most of the supported devices (Kindle, Nook) and my Pocketbook battery is dead I discovered today. I have no reason to believe there is any problem with:
  • create_canvas - used for Kindles only
  • save_cover_data_to - used for Sony (JPG), Nook (JPG), Kindle (PNG)
    For Pocketbooks, save_cover_data_to had to be replaced by the workaround code (provided by you in 2012 if you can remember that far back ) you see above as it was not saving the in 4BPP format when using Linux.
jackie_w is offline   Reply With Quote
Old 05-12-2016, 12:44 PM   #15
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 28,810
Karma: 7295999
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by davidfor View Post
I've done some testing of sending the cover images tonight. There were no problems. I can't see a difference between the old and new versions. Including greyscale covers. I thought they were being skipped in Windows.
No, I implemented a grayscale routine that works on all Qt versions.
kovidgoyal is online now   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Windows beta testers needed kovidgoyal Calibre 19 02-08-2016 04:38 PM
Beta testers for calibre 2.0 needed kovidgoyal Calibre 142 05-08-2015 04:44 AM
Arc Updated Jelly Bean release - Beta testers needed kobo-adrian Kobo Tablets 19 05-23-2013 09:32 PM
Classic G:RSS: Optimized Google Reader (RSS) for the Nook [BETA Testers needed] Fmstrat Barnes & Noble NOOK 24 12-28-2010 12:22 PM
Web Browsing from the Ebw-1150 - BETA Testers needed Nate the great Fictionwise eBookwise 26 09-21-2008 11:52 PM


All times are GMT -4. The time now is 10:42 AM.


MobileRead.com is a privately owned, operated and funded community.