Modify ePub plugin dev thread - Page 6

capnm · 06-05-2011, 05:12 PM

Yes, it's at the rebuild stage, so presumably the zip function doesn't like the invalid timestamp, which is probably not even the same invalid value it was originally -- after the OS re-interpreted it.

Which favors Kovid's nice, simple solution.

I suspect that what happens to invalid timestamps on the unpack end is frustratingly inconsistent across OS's.

kiwidude · 06-05-2011, 07:06 PM

Funnily enough, I just hit this problem myself again. I think perhaps I finally found my way back to the ePub that had the issue.

In my case it is the mimetype file, that has no "Date modified" value when you look at it in Windows Explorer after using Tweak ePub to explode the ePub. If you open the file using WinZip, it shows it like Kovid has changed of 1/1/1980.

Kovid - I got the latest source and I still get the same error when I try to rebuild the ePub. I can email you the ePub if you like.

kovidgoyal · 06-05-2011, 07:38 PM

That will be because the rebuild code is using the python stdlib zipfile not calibre.utils.zipfile

paulfiera · 06-06-2011, 07:27 AM

Kiwidude, first of all, kudos for all your plugins.

I am really looking forward this one and your name has been mentioned several times in this thread.

Sounds like a bit of a challenge but maybe you could come to the rescue?

kiwidude · 06-06-2011, 08:10 AM

@paulfiera - Certainly the physical removal of the font files and the removal of their entries from the manifest are trivial for this plugin. However the difficult part as mentioned on the other thread is the css file. I would need someone to define exactly what such css entries would look like (all their permutations if there are multiple ways they could be defined). And what to do with that css file - for instance can I just remove such font lines from the file, or does something else need to be put in their place etc.

If someone can define those rules then I can take a look at incorporating it.

kovidgoyal · 06-06-2011, 08:15 AM

There's no need to remove the @font-face rules from the css, they will just be ignored if the font files are missing.

paulfiera · 06-06-2011, 08:30 AM

Quote:

Originally Posted by kovidgoyal

There's no need to remove the @font-face rules from the css, they will just be ignored if the font files are missing.

Agree.

My only experience is with iPad, and it has some default fonts which it uses. How do other readers behave? Say, if I specify Charis and the font is not embedded. What font will they use?

kiwidude · 06-06-2011, 09:06 AM

@Kovid - thx for that, in that case it is indeed a trivial exercise. I'll put it in the next beta version.

paulfiera · 06-06-2011, 09:54 AM

Quote:

Originally Posted by kiwidude

@Kovid - thx for that, in that case it is indeed a trivial exercise. I'll put it in the next beta version.

Thanks, kiwidude.

Will you be uploading the next beta version so it can be downloaded from this thread?

kiwidude · 06-06-2011, 10:01 AM

I've written it just now, seems to work fine but obviously your feedback in testing would be appreciated.

I also want to add an "update metadata excluding covers" option and then will post it on this thread.

paulfiera · 06-06-2011, 11:15 AM

Quote:

Originally Posted by kiwidude

I've written it just now, seems to work fine but obviously your feedback in testing would be appreciated.

I also want to add an "update metadata excluding covers" option and then will post it on this thread.

I'd be glad to.

Just say when the new beta is available for download and I'll get on with it.

EDIT: Will this plugin show up in the Plugin Updater? Under "Not installed" perhaps?

kiwidude · 06-06-2011, 12:05 PM

Changes in this release:

Add ability to remove embedded fonts
Add ability to update the metadata (including cover)
Add an error dialog if the user clicks ok with no options selected
Ensure rebuilding the ePub uses the Calibre zip code as per change to Tweak ePub

Note the problem one of you have reported with rebuilding the zip file (for an ePub containing a file with no timestamp) will continue to exist until the next Calibre release. Kovid has fixed it in the Calibre source code if you run from that. Alternatively you could use Tweak ePub, edit the file with no timestamp so it now has one and then rebuild it. Note this problem "should" be fairly rare anyway so hopefully won't stop you testing in the meantime.

@Paul - here is the embedded fonts change for you to test (thx for the donation btw). And no, this will not appear in plugin updater until I officially release it as 1.0 in the plugin forums and add it to the plugin index thread. So you will have to install it "old school" using Preferences->Plugins->Add.

I consider the plugin "stable" in that I quite happily run it against my own ePubs. However the feature list for it is still evolving and I greatly appreciate feedback on this thread from the early adopters about its usability from trying the betas.

The new "Update metadata" feature is rather more useful than I had hoped thanks to Kovid pointing me at some Calibre code I could reuse that does the hard work. It will update both the metadata in the manifest and the cover if possible, in the same way as when you send to device/save to disk. The same cover update limitation applies as discussed earlier in this thread (in that Calibre has to be able to identify the cover from the manifest, most easily guaranteed from a past conversion). The future todo list still has an option for this plugin to "Add a cover" to cater for "other" ePubs without requiring the conversion.

jackie_w · 06-06-2011, 12:17 PM

@kiwidude,

FYI, I'm not sure what logic you've decided to use to determine whether an epub contains embedded fonts but I have come across epubs where the actual .ttf/otf files are un-manifested. Even when they are manifested I've come across a variety of values for the media-type attribute.

kiwidude · 06-06-2011, 12:26 PM

@jackie - that scenario is handled fine, I just tested it now. I am looking at the file extensions of files in the epub rather than the media type attribute in the manifest. So if it finds any then first it will delete the files, then it will go on remove from the manifest if found in there.

paulfiera · 06-06-2011, 12:28 PM

@kiwidude

I've tested it with three books and the fonts and their references in the manifest were successfully removed.

Opened the books with calibre viewer and they look perfect. The epubs have gone down from 2.5 MB to 0.8 MB.

For those not interested in embedded fonts - besides the other features the plugin has - this is a great plugin.

Thanks a lot.

06-06-2011, 12:05 PM	#87
kiwidude Calibre Plugins Developer Posts: 4,636 Karma: 2162064 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	v0.3 Beta Changes in this release: Add ability to remove embedded fonts Add ability to update the metadata (including cover) Add an error dialog if the user clicks ok with no options selected Ensure rebuilding the ePub uses the Calibre zip code as per change to Tweak ePub Note the problem one of you have reported with rebuilding the zip file (for an ePub containing a file with no timestamp) will continue to exist until the next Calibre release. Kovid has fixed it in the Calibre source code if you run from that. Alternatively you could use Tweak ePub, edit the file with no timestamp so it now has one and then rebuild it. Note this problem "should" be fairly rare anyway so hopefully won't stop you testing in the meantime. @Paul - here is the embedded fonts change for you to test (thx for the donation btw). And no, this will not appear in plugin updater until I officially release it as 1.0 in the plugin forums and add it to the plugin index thread. So you will have to install it "old school" using Preferences->Plugins->Add. I consider the plugin "stable" in that I quite happily run it against my own ePubs. However the feature list for it is still evolving and I greatly appreciate feedback on this thread from the early adopters about its usability from trying the betas. The new "Update metadata" feature is rather more useful than I had hoped thanks to Kovid pointing me at some Calibre code I could reuse that does the hard work. It will update both the metadata in the manifest and the cover if possible, in the same way as when you send to device/save to disk. The same cover update limitation applies as discussed earlier in this thread (in that Calibre has to be able to identify the cover from the manifest, most easily guaranteed from a past conversion). The future todo list still has an option for this plugin to "Add a cover" to cater for "other" ePubs without requiring the conversion. Attached Thumbnails Last edited by kiwidude; 06-12-2011 at 09:51 AM. Reason: Removed attachment as later version in thread

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Any web-to-epub plugin for internet browser?	bthoven	ePub	7	07-10-2011 05:14 AM
[Old Thread] Reading epub on viewer inexplicably changes the time stamp of epub	greenapple	Library Management	20	03-19-2011 10:18 PM
Easy way to modify thread subscription emails in bulk?	snipenekkid	Feedback	11	02-06-2011 03:47 AM
Another plugin dev question	DiapDealer	Plugins	2	12-11-2010 01:46 PM
Epub plugin dev	DiapDealer	Plugins	15	11-12-2010 09:36 AM

06-05-2011, 05:12 PM	#76
capnm Groupie Posts: 156 Karma: 10001 Join Date: Feb 2011 Device: sony	Yes, it's at the rebuild stage, so presumably the zip function doesn't like the invalid timestamp, which is probably not even the same invalid value it was originally -- after the OS re-interpreted it. Which favors Kovid's nice, simple solution. I suspect that what happens to invalid timestamps on the unpack end is frustratingly inconsistent across OS's.

06-05-2011, 07:06 PM	#77
kiwidude Calibre Plugins Developer Posts: 4,636 Karma: 2162064 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	Funnily enough, I just hit this problem myself again. I think perhaps I finally found my way back to the ePub that had the issue. In my case it is the mimetype file, that has no "Date modified" value when you look at it in Windows Explorer after using Tweak ePub to explode the ePub. If you open the file using WinZip, it shows it like Kovid has changed of 1/1/1980. Kovid - I got the latest source and I still get the same error when I try to rebuild the ePub. I can email you the ePub if you like.

06-05-2011, 07:38 PM	#78
kovidgoyal creator of calibre Posts: 43,853 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	That will be because the rebuild code is using the python stdlib zipfile not calibre.utils.zipfile

06-06-2011, 07:27 AM	#79
paulfiera Addict Posts: 378 Karma: 3102 Join Date: Dec 2010 Location: EU Device: Kobo Aura ONE, Kobo Libra H20	Kiwidude, first of all, kudos for all your plugins. I am really looking forward this one and your name has been mentioned several times in this thread. Sounds like a bit of a challenge but maybe you could come to the rescue?

06-06-2011, 08:10 AM	#80
kiwidude Calibre Plugins Developer Posts: 4,636 Karma: 2162064 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	@paulfiera - Certainly the physical removal of the font files and the removal of their entries from the manifest are trivial for this plugin. However the difficult part as mentioned on the other thread is the css file. I would need someone to define exactly what such css entries would look like (all their permutations if there are multiple ways they could be defined). And what to do with that css file - for instance can I just remove such font lines from the file, or does something else need to be put in their place etc. If someone can define those rules then I can take a look at incorporating it.

06-06-2011, 08:15 AM	#81
kovidgoyal creator of calibre Posts: 43,853 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	There's no need to remove the @font-face rules from the css, they will just be ignored if the font files are missing.

06-06-2011, 09:06 AM	#83
kiwidude Calibre Plugins Developer Posts: 4,636 Karma: 2162064 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	@Kovid - thx for that, in that case it is indeed a trivial exercise. I'll put it in the next beta version.

06-06-2011, 10:01 AM	#85
kiwidude Calibre Plugins Developer Posts: 4,636 Karma: 2162064 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	I've written it just now, seems to work fine but obviously your feedback in testing would be appreciated. I also want to add an "update metadata excluding covers" option and then will post it on this thread.

06-06-2011, 12:17 PM	#88
jackie_w Grand Sorcerer Posts: 6,212 Karma: 16534894 Join Date: Sep 2009 Location: UK Device: Kobo: KA1, ClaraHD, Forma, Libra2, Clara2E. PocketBook: TouchHD3	@kiwidude, FYI, I'm not sure what logic you've decided to use to determine whether an epub contains embedded fonts but I have come across epubs where the actual .ttf/otf files are un-manifested. Even when they are manifested I've come across a variety of values for the media-type attribute.

06-06-2011, 12:26 PM	#89
kiwidude Calibre Plugins Developer Posts: 4,636 Karma: 2162064 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	@jackie - that scenario is handled fine, I just tested it now. I am looking at the file extensions of files in the epub rather than the media type attribute in the manifest. So if it finds any then first it will delete the files, then it will go on remove from the manifest if found in there.

Advert

Advert

06-06-2011, 12:28 PM	#90
paulfiera Addict Posts: 378 Karma: 3102 Join Date: Dec 2010 Location: EU Device: Kobo Aura ONE, Kobo Libra H20	@kiwidude I've tested it with three books and the fonts and their references in the manifest were successfully removed. Opened the books with calibre viewer and they look perfect. The epubs have gone down from 2.5 MB to 0.8 MB. For those not interested in embedded fonts - besides the other features the plugin has - this is a great plugin. Thanks a lot.