Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 05-04-2011, 02:46 PM   #151
jackie_w
Wizard
jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.
 
Posts: 2,880
Karma: 4200035
Join Date: Sep 2009
Location: UK
Device: Sony PRS-350, PB360, Kobo Glo/AuraHD/Aura6"/AuraH2O
I agree totally with theducks. Nothing would persuade me to do any kind of "bulk automatic repair" on a set of epubs. I'm all for a set of magic buttons which each do a particular epub repair, but I want to look at the result afterwards to make sure I haven't wrecked anything.

Mind you I do tend to be over-cautious. I don't do bulk conversions or bulk metadata downloads either
jackie_w is offline   Reply With Quote
Old 05-04-2011, 08:00 PM   #152
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 8,888
Karma: 12755553
Join Date: Feb 2009
Location: North Carolina
Device: Nexus 7
For jacket removal and zeroing of page-template I would be willing to throw the dice once I verified it worked on a few books. These two items aren't likely to blow up any book.

Looking forward to trying these features out.
DoctorOhh is offline   Reply With Quote
 
Advertisement
Old 05-04-2011, 08:13 PM   #153
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,230
Karma: 1345754
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
I'm not far away from releasing the next Quality Check version (got one more "Check" I am going to add related to navpoints in ncx files).

You can see something else in the top right that is also in progress...

@jackie/theducks - obviously the choice is entirely yours as to whether to do stuff manually or use a tool. Quality Check will allow you to "find" such cases without touching the files. Obviously my goal with the plugin that "fixes" the ePubs is that any changes it makes are done in the most conservative and safest way possible. It has to be a "no-brainer" decision for any feature that gets added to it.
Attached Thumbnails
Click image for larger version

Name:	Screenshot_1_Toolbar.png
Views:	71
Size:	93.4 KB
ID:	70932  
kiwidude is offline   Reply With Quote
Old 05-04-2011, 10:09 PM   #154
ldolse
Wizard
ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.
 
Posts: 1,337
Karma: 123455
Join Date: Apr 2009
Location: Malaysia
Device: PRS-650, iPhone
Quote:
Originally Posted by kiwidude View Post
So i guess what I could do is write a plugin that a bit like book conversions will run in the background on your ePub books to apply various transformations. It would work on your selected books, and give you a very simple dialog of checkboxes of stuff you want it to do with select none and all buttons. Then I would make sure each of those functions is coded to work independently. Then if it is deemed appropriate in future this whole plugin could just become an item on the tweak submenu. And the existing tweak dialog could offer similar functionality reusing that part of the code.

Then it will be up to Quality Check as to how many of these things it offers "Check" functions for. It may only be a subset, the two plugins can grow independently.

How does that sound for a plan?
Sounds like a good plan to me. If you're willing to take on the initial effort of creation that's great, as I don't envision myself having much time the next few weeks to jump on this one. However if you've put the basic framework in place I can look into tying other 'safe' pieces of the conversion engine into the plugin.

@JackieW, part of the thinking behind this is to address a feature request that you opened a while back, your 'epub conversion lite' idea:
http://oldbugs.calibre-ebook.com/ticket/8252

The idea is to be able to automatically do any individual function that Calibre's conversion pipeline allows while avoiding doing a 'full' conversion. This isn't really possible with any other ebook formats, but since epub is zipped OEB and OEB is Calibre's intermediate format for all conversions it's an option in this case.
ldolse is offline   Reply With Quote
Old 05-05-2011, 01:21 PM   #155
jackie_w
Wizard
jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.
 
Posts: 2,880
Karma: 4200035
Join Date: Sep 2009
Location: UK
Device: Sony PRS-350, PB360, Kobo Glo/AuraHD/Aura6"/AuraH2O
Quote:
Originally Posted by ldolse View Post
@JackieW, part of the thinking behind this is to address a feature request that you opened a while back, your 'epub conversion lite' idea:
http://oldbugs.calibre-ebook.com/ticket/8252

The idea is to be able to automatically do any individual function that Calibre's conversion pipeline allows while avoiding doing a 'full' conversion. This isn't really possible with any other ebook formats, but since epub is zipped OEB and OEB is Calibre's intermediate format for all conversions it's an option in this case.
Excellent! I've been trying out some of my own beginners Python code to tackle some of these myself, as I wasn't sure my request would ever make it to the top of anyone's list. I don't think my code would be robust/generic enough to release to the public but it's been fun trying. I cobbled together an xgpt tweaker, a css tweaker and an embedded font stripper. I even managed an epub to html utility, then a week later user_none brought out the official convert to HTMLZ format. Still I guess no experience is wasted

I look forward to whatever you come up with. I'm sure it will be better than anything I write.
jackie_w is offline   Reply With Quote
Old 05-05-2011, 01:31 PM   #156
jackie_w
Wizard
jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.
 
Posts: 2,880
Karma: 4200035
Join Date: Sep 2009
Location: UK
Device: Sony PRS-350, PB360, Kobo Glo/AuraHD/Aura6"/AuraH2O
Quote:
Originally Posted by kiwidude View Post
@jackie/theducks - obviously the choice is entirely yours as to whether to do stuff manually or use a tool.
Just to clarify, I will be more than happy to use your automatic fix tools. I just want to apply them one at a time not in bulk. I hope no offense was taken, I love your plugins.
jackie_w is offline   Reply With Quote
Old 05-05-2011, 01:50 PM   #157
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,230
Karma: 1345754
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@jackie_w - I didn't see it as offensive in the slightest. There are certain things I don't do in bulk either. Conversions is top of my list - I cringe when I see people posting about doing thousands at a time or whatever. I suppose that is fine if you don't ever intend to actually read them. No wonder there is so many crap conversions floating around out there with lost quotes, bad encodings, embedded page numbers, etc.

The functions I had in mind for the Clean ePub plugin would never need to be applied to a book that I have converted, as I know it is already in a state I am happy with. However I do have a big backlog of stuff that hasn't yet had my TLC to make it into my "master" library - so if I can run a bunch of things against that mess in bulk then provided as I say I have confidence I am not destroying the ePub it should save me a lot of work. And others who use it of course...
kiwidude is offline   Reply With Quote
Old 05-05-2011, 04:09 PM   #158
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,230
Karma: 1345754
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
v1.5 Released

Changes in this release:
  • Move all the ePub checks onto a Check ePub submenu
  • Add epub check for legacy jackets only
  • Add epub check for missing container.xml files
  • Add epub check for files listed in manifest missing from epub
  • Add epub check for unmanifested files
  • Add epub check for iTunes plist files
  • Add epub check for Calibre bookmarks files
  • Add epub check for .xpgt margins
  • Add epub check for TOC with < 3 entries
  • Add epub check for inline Calibre cover
  • Add epub check for no inline Calibre cover
  • Add epub check for Calibre conversion
  • Add epub check for not Calibre converted
  • Add check for duplicate series index
  • Improve having jacket and multiple jacket checks to include legacy jackets
  • Restructure code internally for ease of future expansion

This started off as a minor release and become rather more.

Hopefully those of you who are concerned with the quality of their ePubs will find a few things to like out of that lot. Identifying is one thing - fixing will be another. As some will have gathered from the last few pages I have started another plugin which will allow automating the remedy for many of these items without requiring you to do a conversion.

Enjoy, I look forward to your feedback.
Attached Thumbnails
Click image for larger version

Name:	Screenshot_1_Toolbar.png
Views:	48
Size:	56.7 KB
ID:	70951  
kiwidude is offline   Reply With Quote
Old 05-05-2011, 05:04 PM   #159
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 15,096
Karma: 5939999
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Do you know how much work you caused

I just did about 10 of the checks. (Using Natty Narwhal(32) today)
I opened a number of each type of error in Sigil.

Unfortunately, the report was correct

Kiwidude does it again
theducks is online now   Reply With Quote
Old 05-05-2011, 06:18 PM   #160
Perkin
Guru
Perkin calls his or her ebook reader Vera.Perkin calls his or her ebook reader Vera.Perkin calls his or her ebook reader Vera.Perkin calls his or her ebook reader Vera.Perkin calls his or her ebook reader Vera.Perkin calls his or her ebook reader Vera.Perkin calls his or her ebook reader Vera.Perkin calls his or her ebook reader Vera.Perkin calls his or her ebook reader Vera.Perkin calls his or her ebook reader Vera.Perkin calls his or her ebook reader Vera.
 
Perkin's Avatar
 
Posts: 650
Karma: 64171
Join Date: Sep 2010
Location: Kent, England, Sol 3, ZZ9 plural Z Alpha
Device: Sony PRS-300, Kobo Aura HD
Wonderful work. Loving it. Thanks kiwidude.
Perkin is online now   Reply With Quote
Old 05-05-2011, 07:09 PM   #161
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,230
Karma: 1345754
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Thanks guys.

A post release thought I have had is that in the next release I will change the check for unmanifested files. By definition currently it is a superset of the bookmarks and plist files, plus any other "stray" files you may have hiding in your epubs.

Instead I think I should make it a check for "other" files excluding plist and bookmarks ones. That way if you decide to for instance retain Calibre's ability to open an epub at the last page read in its ebook viewer, you could still detect other stray files lurking within with this check.

@Idolse - I think that is two of the four items off your checklist you posted here. The other two items you mentioned were checking for embedded fonts in the opf file, and checking for svg files. If either of these is simple to describe in a PM let me know. Or if you want to code them yourself to send to me then take a look at check_epub.py where all the ePub checks are, and the top of config.py where you define a menu item.
kiwidude is offline   Reply With Quote
Old 05-11-2011, 12:54 PM   #162
drMerry
Addict
drMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmos
 
drMerry's Avatar
 
Posts: 293
Karma: 21022
Join Date: Mar 2011
Location: NL
Device: Sony PRS-650
I still love it.
Yet another option could be to check if date == pubdate (I do not know the value used for the date column...)

If those are the same, You have added the book but apparently not (yet) set the publication date.
drMerry is offline   Reply With Quote
Old 05-11-2011, 01:16 PM   #163
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,230
Karma: 1345754
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Nice idea drMerry, I'll add it to the list for the next version.

It would be nice if Calibre had the option to set an empty publication date by default but that has never made it very high up Kovid's list of issues to address it seems. And given the spasms the Qt datepickers have with empty dates it's likely not a trivial thing to fix.
kiwidude is offline   Reply With Quote
Old 05-11-2011, 02:01 PM   #164
drMerry
Addict
drMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmos
 
drMerry's Avatar
 
Posts: 293
Karma: 21022
Join Date: Mar 2011
Location: NL
Device: Sony PRS-650
Quote:
Originally Posted by kiwidude View Post
Nice idea drMerry, I'll add it to the list for the next version.

It would be nice if Calibre had the option to set an empty publication date by default but that has never made it very high up Kovid's list of issues to address it seems. And given the spasms the Qt datepickers have with empty dates it's likely not a trivial thing to fix.
should be no problem.
If you mass-edit, you can remove the update without any problems.
This would be the best way because you could search on pubdate:false
drMerry is offline   Reply With Quote
Old 05-15-2011, 01:14 PM   #165
nynaevelan
eBook Junkie
nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.
 
nynaevelan's Avatar
 
Posts: 1,504
Karma: 1462646
Join Date: May 2010
Location: USA
Device: Kindle Fire 6 2014, Kindle PW2, Galaxy Note 3
Kiwidude:

I just had a thought, since I was thinking about the no formats problem which interfered with my goodreads syncing, would it be possible to add an option to this script to check for books with no formats??
nynaevelan is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Open With kiwidude Plugins 250 11-20-2014 03:45 PM
[GUI Plugin] Clipboard Search kiwidude Plugins 9 08-12-2014 04:36 AM
[GUI Plugin] Search the Internet kiwidude Plugins 205 08-11-2014 03:22 PM
[GUI Plugin] Kindle Collections (old) meme Plugins 2070 08-11-2014 01:02 AM
[GUI Plugin] Book Sync **Deprecated** kiwidude Plugins 111 06-07-2011 08:47 PM


All times are GMT -4. The time now is 01:43 PM.


MobileRead.com is a privately owned, operated and funded community.