[GUI Plugin] Modify ePub

[jackie_w]

Originally Posted by Rand Brittain

Basically, I replace left-single-quote and left-double-quote with two glyphs I'm not using, then swap them back. Then I turn right-double-quote into a glyph and turn every right-single-quote that's followed by a space, a </p>, emdash, colon, comma or period into a right-double-quote. Then I change that last glyph into a right-single-quote.

This gets most of the basic stuff done, but I still have to manually find all the unusual uses of the apostrophe and correct for them, like if someone were to use a bit o’ Cockney accent, or uses a possessive like "the Jones'." It's more a matter of find/replace and persistence than regexes. I'd just like to automate the find/replace instead of having to repeat the steps for every book.

OK, thanks for that. I was hoping you may have found a 'magic formula' that didn't require manual intervention I expect you also have to be a bit careful not to wreck the straight double quotes inside the html tags.

[Rand Brittain]

Originally Posted by jackie_w

OK, thanks for that. I was hoping you may have found a 'magic formula' that didn't require manual intervention I expect you also have to be a bit careful not to wreck the straight double quotes inside the html tags.

So long as the text has been converted to smart quotes first, it's not a problem, but yeah, it means I can't do it the easy way and flip them before using smart quotes.

[JSWolf]

From what I see, the code for converting to smart quotes sees >" in the middle or a line and thinks it's at the beginning of a line.

Should be easy enough to search for after the smart quote conversion is done.

[Kirtai]

Originally Posted by kiwidude

It isn't a feature I have an interest in writing myself. My head hurts thinking of the regex expressions... on top of which I would suggest that unless your book was "perfectly" formatted with matching quotes etc (which a lot of books are not) surely relying on automated regexes could leave a bit of a mess behind...

IIRC, matching quotes are not even required in some contexts. e.g. when a person says multiple paragraphs the closing quote is only required on the last one.

Something like:

X said "First thing
"Second thing
"Last thing"

Don't quote me on that though

[kiwidude]

@Kirtai - yes, you are quite correct, I've seen that too. And you only have to look at all the issues that the smarten punctuation logic has been tweaked and now reverted on to see that it is just too "dangerous" to expect an automated regex to not create a bigger mess than you started with in a number of cases.

I still stand by my original comment in that it isn't something I am interested in adding to this plugin. I only want features that are no-brainers to choose to run without further editing by the user required. I think this sort of requirement is something that would be better located within Sigil one day.

[kiwidude]

Originally Posted by paulfiera

I'm finding some books with inline javascript...

I've seen a number of issues caused by javascript. Earlier versions of Sigil used to choke on it completely, stopping rendering of the page thereby corrupting your book. And with the new regex engine in Sigil 0.5.3 and all its awful bugs it seems I can't even remove the script blocks... my regexes just result in yet more blank lines being added to the header... sigh...

It's something I will consider at some point - of course it needs a QC to find books with such script blocks (both inline and external files) and then something to remove it all. Perhaps as part of ePub3 or whatever there are some intended usages for javascript, there is no place in my books for sure and most are just remnants of the original text being an interactive web page at some point before conversion to ePub.

[Kirtai]

Originally Posted by kiwidude

I still stand by my original comment in that it isn't something I am interested in adding to this plugin. I only want features that are no-brainers to choose to run without further editing by the user required.

Oh, I wasn't suggesting you add it. Quite the opposite, I was giving an example of why you shouldn't

[kiwidude]

@Kirtai - I didn't think otherwise, I just didn't want the discussion to continue on in this thread about it in case others got the idea it was being considered...

[Rand Brittain]

Fair enough; it would be better suited to a Sigil plugin, so I'll just have to hope that Sigil plugins one day become A Thing.

[Rand Brittain]

For another feature request, would it be possible for this plugin to remove styles that are unused in the EPUB from the stylesheet?

[kiwidude]

Originally Posted by Rand Brittain

For another feature request, would it be possible for this plugin to remove styles that are unused in the EPUB from the stylesheet?

This has been discussed previously both on this thread and in the plugin ideas thread. In short, its not something I currently have plans of supporting - there are too many ways of declaring styles, and too many issues with removing them without completely rewriting the css files (which is what a calibre conversion does). And it is a feature I think belongs in Sigil rather than this plugin, since it should only be while editing the ePub that a css style gets made redundant?

If your epub has a bunch of unused styles, then it is probably unlikely to be a retail one, in which case why not just do a calibre conversion on it? As calibre automatically removes unused styles as part of its css flattening.

[kiwidude]

Changes in this release:

• Change to require minimum calibre version 0.8.53 in order to utilise some calibre bug fixes/changes
• Change to calibre API for deprecated dialog in 0.8.49 which caused issues that intermittently crashed calibre on Mac OS
• Add a "Insert or replace cover" option to attempt to insert or replace a cover without doing a conversion
• Add a "Remove cover" option to attempt to completely remove an identified cover from the ePub.
• Rewrite "Removed unused image files" and "Remove broken cover images" features to use lxml rather than regex for better accuracy
• Add protection for numerous options against trying to apply them to a DRM encrypted book
• Better handle ebooks where the ncx file is not in same directory as opf manifest
• If user chooses redundant options (e.g. "Remove all jackets" makes "Remove legacy jackets" redundant) do not run the redundant option

All of the discussions related to the beta versions of this release have been moved into the Modify ePub development forum thread.

This is a pretty massive release in terms of effort - I don't want to even think about how many hours went into it! You wouldn't think that making sure an ePub has a cover with all the correct references to it was so difficult. However at long last it is possible to cleanly insert/replace/remove covers for ePubs without risking the side effects of a conversion, which is pretty darn cool if I say so myself.

My huge thanks to all the testers for their invaluable feedback and patience through the many iterations to get to this point... enjoy.

[JSWolf]

I hate to be the bearer of bad news, but I just found a bug in the new release. The links to the page-template file are not being removed. I'm using the almost the same options as the screen sent you before except the cover options are not selected.

Modify ePubs
Logfile for book ID 152 (Star Trek: Typhon Pact: Plagues of Night / David R. George III)
152
  Modifying:  C:\Users\Jon\AppData\Local\Temp\calibre_0.8.54_tmp_pxljof\thvdmq_modify_epub\152.epub
	Looking for files to remove: [u'iTunesMetadata.plist', u'iTunesArtwork']
	Looking for files to remove: [u'META-INF/calibre_bookmarks.txt']
	Looking for files to remove: [u'.DS_Store', u'thumbs.db']
	Looking for unused images
	  Removing unused image: ops/images/9781451649574.jpg
	  Manifest item removed: images/9781451649574.jpg (my--cover--image)
	Looking for Adobe xpgt files to remove
	  Found xpgt file to to remove: ops/styles/page-template.xpgt
	  Manifest item removed: styles/page-template.xpgt (pt)
	Looking for Adobe xpgt xhtml links to remove
	Looking for Adobe DRM meta tags to remove
ePub updated in 0.96 seconds

[kiwidude]

Originally Posted by JSWolf

I hate to be the bearer of bad news...

Since when... Like you couldn't have found this two hours ago... sigh...

Actually this is something that has been broken for a while, it isn't specific to this release (the only good news from my viewpoint - something I didn't break). It is a flaw in the regex I was using to identify the links, for some inanely stupid reason it was written with an ordering dependency in it. The QC plugin has the same flaw...

Thanks for the PM with the link btw, makes my life instantly easier to figure it out.

[kiwidude]

Changes in this release:

• Fix for remove Adobe xpgt links so it no longer is dependent on link attribute ordering to find them