[GUI Plugin] Modify ePub

kiwidude · 10-22-2011, 10:27 AM

This plugin offers a way to perform certain modifications to your selected ePub files without performing a calibre conversion. This plugin was created a number of months ago and has a history documented in the Development forum on this thread.

Performing an ePub->ePub conversion will enforce a number of changes to your ePub, some of which can be undesirable for some users. Examples are the rewriting of CSS, margin modifications, file splitting in undesired places, changes to directory structure etc.

Instead this plugin allows a user specific subset of changes to be performed in isolation without otherwise touching the original ePub's file structure, CSS files etc. Frequently these changes have been performed manually by users either using the Tweak ePub feature (time consuming), by editing in Sigil (which introduces changes/side effects of its own), by doing ePub->ePub conversions, or by saving to disk and reimporting into calibre.

Users may also find it useful to install the Quality Check plugin, which offers the ability to identify ePubs in your library which qualify for many of the modifications this plugin can make.

Refer to the Help file accessed from the plugin dialog for full details on each of the modification options and when you might use them.

Main Features of v1.3.7:

Remove iTunes artifact files
Remove Calibre bookmark files
Remove OS artifact files such as Thumbs.db
Remove unused image files
Removing missing file entries from the .opf manifest
Add unmanifested files to manifest
Remove unmanifested files from ePub
Remove non dc: metadata from manifest
Flatten TOC hierarchy in NCX file
Remove broken link TOC entries in NCX file
Remove margins from Adobe .xpgt files
Remove Adobe .xpgt files and links
Remove Adobe resource DRM meta tags
Remove all metadata jackets
Remove legacy metadata jackets
Add/replace metadata jacket
Encode HTML in UTF-8 to fix invalid HTML encodings.
Remove embedded fonts
Modify @page and body margin styles
Smarten punctuation
Remove inline javascript and .js files
Remove html pages containing nothing but broken image links
Completely remove an existing cover
Insert a new cover or replace an existing one using your desired proportions/svg choice from Preferences->Output Options->EPUB
Update metadata
Save and restore your preferred settings with a single click
Optional script to run from command line

Special Notes:

Requires Calibre 0.8.53 or later

Installation Notes:

Download the attached zip file and install the plugin/restart Calibre/add to context menu as described in the Introduction to plugins thread.

Running from command line:

It is possible to run most of the functions of this plugin from the command line using a python script that is bundled inside the zip file. This allows users to use a feature such as smarten punctuation against an ePub without having to add it to the calibre GUI. Note that the command line script still requires that calibre and the Modify ePub plugin are installed as per usual - it just avoids requiring the books to be added to calibre and interactive gui clicks.
To make use of the command line script, open the zip file and extract the me.py file along with the readme.txt and follow the instructions within.

Paypal Donations:

If you find this or any of my other plugins useful please feel free to show your appreciation. I have spent many hundreds of unpaid hours in their development and support so any encouragement for me to continue is appreciated!

Version History:

Spoiler:

DoctorOhh · 10-22-2011, 07:45 PM

Excellent, it will now be easier to refer folks to this plugin.

bizzybody · 10-23-2011, 12:22 AM

Here's a feature request, a function to edit or remove text color settings in the stylesheet.css file. I spent a lot of time trying different converters from epub to Mobi but none of them would get rid of what had the text locked to black (actually a really really dark grey). Finally, in response to a post in the epub forum, I was pointed to Sigil and looking for references to color in stylesheet.css. I found three instances of color: #231F20; deleted them and saved. Then the book converted without the text color locked.

I figure the best setup would be to find each color setting and show an example of what changing or removing each setting will do. Might not want to remove *all* the color settings, in this book the TOC links and chapter headings are set to "blue" while the rest of the text was 231F20. I left the "blue" and removed the others.

Boggles me why anyone would set a color to look like printed "black" ink when black text is the default on readers and reader software.

I read on my LED screen phone in white on black because it improves the battery life a bunch VS black on white. Black pixels on LED screens use zero power, VS white pixels on LCD using zero power, other than the backlight shining through. No backlight on LED.

DoctorOhh · 10-23-2011, 12:54 AM

Kiwidude, let me state it a little simpler. It would be nice if you could add the option to remove text color and remove background color to the plugin.

kiwidude · 10-25-2011, 06:51 PM

@bizzybody/dwanthny - modifying the css via this plugin is one of those things I personally have avoided to date, though I guess Idolse must be doing with his additions to this plugin. It has to be done in a bulletproof way while minimising the changes to the css, which means not using third party css parsing code for instance. It should be possible using regular expressions but needs careful application and a lot of testing. I want people to have confidence that when they use this plugin it lives up to its billing of making the least changes possible to do their task, instead of the potential gotchas associated with alternatives.

You have to handle potentially mutiple css files, and also inline css as well...

If I do take the leap at some point, certainly stripping colors could be an option to support. I would add to that wish list things options like line-height, letter-spacing and font-family too, no doubt there are others I've forgotten.

ldolse · 10-25-2011, 09:45 PM

Quote:

Originally Posted by kiwidude

@bizzybody/dwanthny - modifying the css via this plugin is one of those things I personally have avoided to date, though I guess Idolse must be doing with his additions to this plugin. It has to be done in a bulletproof way while minimising the changes to the css, which means not using third party css parsing code for instance.

You have to handle potentially mutiple css files, and also inline css as well...

I haven't taken the leap and done my whole library with the plugin yet, but I've modified dozens if not a couple hundred ePub's css using this plugin. The code works in all the ways you describe at this point - multiple css files are supported, inline styles in the html, etc. It would be trivial to modify other css styles - the other thing it seems people want is to automatically add/remove justification.

I think the trick would be to apply all the css modifications at once though, so it would require a bit of a re-design of the current function - basically a top level modify css, then a bunch of sub-functions - sort of like heuristics.

Magnus · 10-26-2011, 10:15 PM

I'm quite naive here... but I noticed that when I run the plugin I get:

Looking for non dc: elements in manifest

Removing child: {http://www.idpf.org/2007/opf}meta

After I've modified the ePub, the result persists when I again run the plugin. Why is that? What have I overlooked?

kiwidude · 10-27-2011, 03:58 AM

@Magnus - what "result persists"are you referring to? Send me a PM with a link to the ePub and more detail on what it is doing the you don't expect it to.

capnm · 11-02-2011, 09:37 AM

Glad to see this made it to prime time! Thanks guys!

I looked at the issue of line spacing a while ago and discovered that a popular technique for setting line spacing is to manipulate the font size (set the paragraph class to, say, font-size 1.3em, line-spacing 1, then span all the text blocks with a font-size .75 em )

making it really hard to 'fix'.

paulfiera · 11-05-2011, 12:29 PM

It'd be really great to have the possibility of updating the jacket with all the metadata, including tags.

Sometimes I change the tags and I would like to update the whole epub jacket without doing a conversion.

kiwidude · 11-05-2011, 01:31 PM

Ummm... whats wrong with the add/replace metadata jacket option that is already in there?

DoctorOhh · 11-05-2011, 03:17 PM

Quote:

Originally Posted by kiwidude

Quote:

Originally Posted by paulfiera

It'd be really great to have the possibility of updating the jacket with all the metadata, including tags.

Sometimes I change the tags and I would like to update the whole epub jacket without doing a conversion.

Ummm... whats wrong with the add/replace metadata jacket option that is already in there?

Nothing, last time I used this feature it worked great.

paulfiera · 11-06-2011, 06:29 AM

Quote:

Originally Posted by kiwidude

Ummm... whats wrong with the add/replace metadata jacket option that is already in there?

Sorry, Kiwidude. You are absolutely right. It works perfectly. I had somehow overlooked this option.

SoulStyle · 11-06-2011, 07:47 AM

Quote:

Originally Posted by Magnus

I'm quite naive here... but I noticed that when I run the plugin I get:

Looking for non dc: elements in manifest

Removing child: {http://www.idpf.org/2007/opf}meta

After I've modified the ePub, the result persists when I again run the plugin. Why is that? What have I overlooked?

Happens here too. Seems to be related to the "Update metadata (including covers)" function of the Modify ePub Plug-In.

Steps to reproduce:

Pick a book that surely has no "Non dc: Metadata"
Do an "Update metadata (including covers)" from the Modify ePub Plug-In
Scan again with "Quality Check" --> "Non dc: Metadata"
Result: Book is shown as having non dc: metadata

It seems that the "Update metadata (including covers)" somehow adds the "{http://www.idpf.org/2007/opf}meta" tag, which then is identified by Quality Check.

Hope this helps

DoctorOhh · 11-06-2011, 08:27 AM

Quote:

Originally Posted by SoulStyle

Steps to reproduce:

Pick a book that surely has no "Non dc: Metadata"
Do an "Update metadata (including covers)" from the Modify ePub Plug-In
Scan again with "Quality Check" --> "Non dc: Metadata"
Result: Book is shown as having non dc: metadata

It seems that the "Update metadata (including covers)" somehow adds the "{http://www.idpf.org/2007/opf}meta" tag, which then is identified by Quality Check.

What is Non dc:metadata?

After I follow your steps on a book the first three lines of the content.opf file are as follows. But I get no error no matter how many times I try the above steps nor does it matter on what books I may try it on. I guess I have no books that meet the criteria needed to reproduce this error.

Code:

<?xml version="1.0"  encoding="UTF-8"?>
<package xmlns="http://www.idpf.org/2007/opf" version="2.0" unique-identifier="uuid_id">
  <metadata xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:opf="http://www.idpf.org/2007/opf" xmlns:dcterms="http://purl.org/dc/terms/"

10-23-2011, 12:54 AM	#4
DoctorOhh US Navy, Retired Posts: 7,315 Karma: 11288999 Join Date: Feb 2009 Location: North Carolina Device: Nexus 7, Sony PRS-950, Sony PRS-505, PRS-300	Kiwidude, let me state it a little simpler. It would be nice if you could add the option to remove text color and remove background color to the plugin. __________________ -- Good Reading, Walt -- 20GB of free CLOUD STORAGE: Use this link to sign up for a free 15GB Copy.com cloud storage account and we both get an extra 5GB of free space. Last edited by DoctorOhh; 10-23-2011 at 01:16 AM.

10-25-2011, 06:51 PM	#5
kiwidude calibre/Sigil Developer Posts: 4,087 Karma: 1211084 Join Date: Oct 2010 Location: London, UK Device: Kindle 3 3G, iPad 2, iPad 3	@bizzybody/dwanthny - modifying the css via this plugin is one of those things I personally have avoided to date, though I guess Idolse must be doing with his additions to this plugin. It has to be done in a bulletproof way while minimising the changes to the css, which means not using third party css parsing code for instance. It should be possible using regular expressions but needs careful application and a lot of testing. I want people to have confidence that when they use this plugin it lives up to its billing of making the least changes possible to do their task, instead of the potential gotchas associated with alternatives. You have to handle potentially mutiple css files, and also inline css as well... If I do take the leap at some point, certainly stripping colors could be an option to support. I would add to that wish list things options like line-height, letter-spacing and font-family too, no doubt there are others I've forgotten. __________________ Like my calibre plugins or Sigil work? Say thanks with PayPal

10-27-2011, 03:58 AM	#8
kiwidude calibre/Sigil Developer Posts: 4,087 Karma: 1211084 Join Date: Oct 2010 Location: London, UK Device: Kindle 3 3G, iPad 2, iPad 3	@Magnus - what "result persists"are you referring to? Send me a PM with a link to the ePub and more detail on what it is doing the you don't expect it to. __________________ Like my calibre plugins or Sigil work? Say thanks with PayPal

11-05-2011, 01:31 PM	#11
kiwidude calibre/Sigil Developer Posts: 4,087 Karma: 1211084 Join Date: Oct 2010 Location: London, UK Device: Kindle 3 3G, iPad 2, iPad 3	Ummm... whats wrong with the add/replace metadata jacket option that is already in there? __________________ Like my calibre plugins or Sigil work? Say thanks with PayPal

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search

10-22-2011, 07:45 PM	#2
DoctorOhh US Navy, Retired Posts: 7,315 Karma: 11288999 Join Date: Feb 2009 Location: North Carolina Device: Nexus 7, Sony PRS-950, Sony PRS-505, PRS-300	Excellent, it will now be easier to refer folks to this plugin. __________________ -- Good Reading, Walt -- 20GB of free CLOUD STORAGE: Use this link to sign up for a free 15GB Copy.com cloud storage account and we both get an extra 5GB of free space.

10-23-2011, 12:22 AM	#3
bizzybody Zealot Posts: 120 Karma: 185440 Join Date: Apr 2007 Location: Idaho, USA Device: Epic 4G, LifeDrive, Tungsten E2	Here's a feature request, a function to edit or remove text color settings in the stylesheet.css file. I spent a lot of time trying different converters from epub to Mobi but none of them would get rid of what had the text locked to black (actually a really really dark grey). Finally, in response to a post in the epub forum, I was pointed to Sigil and looking for references to color in stylesheet.css. I found three instances of color: #231F20; deleted them and saved. Then the book converted without the text color locked. I figure the best setup would be to find each color setting and show an example of what changing or removing each setting will do. Might not want to remove all the color settings, in this book the TOC links and chapter headings are set to "blue" while the rest of the text was 231F20. I left the "blue" and removed the others. Boggles me why anyone would set a color to look like printed "black" ink when black text is the default on readers and reader software. I read on my LED screen phone in white on black because it improves the battery life a bunch VS black on white. Black pixels on LED screens use zero power, VS white pixels on LCD using zero power, other than the backlight shining through. No backlight on LED.

10-26-2011, 10:15 PM	#7
Magnus Member Posts: 21 Karma: 1010 Join Date: Jun 2011 Device: none	I'm quite naive here... but I noticed that when I run the plugin I get: Looking for non dc: elements in manifest Removing child: {http://www.idpf.org/2007/opf}meta After I've modified the ePub, the result persists when I again run the plugin. Why is that? What have I overlooked?

11-02-2011, 09:37 AM	#9
capnm Groupie Posts: 150 Karma: 10001 Join Date: Feb 2011 Device: sony	Glad to see this made it to prime time! Thanks guys! I looked at the issue of line spacing a while ago and discovered that a popular technique for setting line spacing is to manipulate the font size (set the paragraph class to, say, font-size 1.3em, line-spacing 1, then span all the text blocks with a font-size .75 em ) making it really hard to 'fix'.

11-05-2011, 12:29 PM	#10
paulfiera Addict Posts: 312 Karma: 22 Join Date: Dec 2010 Location: EU Device: iPad, iPhone, iPod	It'd be really great to have the possibility of updating the jacket with all the metadata, including tags. Sometimes I change the tags and I would like to update the whole epub jacket without doing a conversion.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] Quality Check	kiwidude	Plugins	631	05-10-2013 04:10 PM
[GUI Plugin] Manage Series	kiwidude	Plugins	105	05-06-2013 01:43 PM
[GUI Plugin] Open With	kiwidude	Plugins	183	04-22-2013 02:25 PM
Modify ePub plugin dev thread	kiwidude	Development	343	11-22-2012 07:35 AM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM


Advertisement Enthusiast