Modify ePub plugin dev thread

[ldolse]

@capnm, I'm sorry, but I've got to admit I have no idea what problem you're trying to solve with your description. The way I'm planning to go forward is that quality check will flag any epubs which don't agree with your Calibre 'page setup' preferences - regardless of whether an @page declaration already exists. Basically quality check will only ignore epubs which don't agree with exactly what's set in your preferences. So if your preferences are all set to zero that means Quality Check would ignore epubs which either don't have any @page, or have an @page where all margins are set to zero. Any epubs which have any non-zero margins set anywhere are ones that would be flagged as needing fixing.

The whole point is that after you 'fix' them Quality check will no longer flag them as needing fixing...

Regarding jackie_w's and capnm's question about why Calibre specifies @page on a flow by flow level vs css, I recall Kovid mentioning in an old thread that this was in case different margins were required for different flows. That said I'm not sure if Calibre is able to follow through with that logic and set separate margins per flow, maybe someone else can comment....

[capnm]

Originally Posted by ldolse

@capnm, I'm sorry, but I've got to admit I have no idea what problem you're trying to solve with your description.

Probably because I was thinking in circles ...

If my preferences are all set to 10pt, I assume Quality Check will flag any epubs with an @page other than 10pt.
And probably also flag any epub with any margins in a body statement.
But will it also flag epubs without any @page or body statements? Such as Calibre generated epubs?

Regarding jackie_w's and capnm's question about why Calibre specifies @page on a flow by flow level vs css, I recall Kovid mentioning in an old thread that this was in case different margins were required for different flows. That said I'm not sure if Calibre is able to follow through with that logic and set separate margins per flow, maybe someone else can comment....

Supposedly that can all be done in the css like this ...

@page flow1 {....}
@page flow2 {....}

.txt_flow1 { page: flow1 }
.txt_flow2 { page: flow2 }

but it wouldn't surprise me in the least if it didn't always work, or work as desired

[ldolse]

Originally Posted by capnm

If my preferences are all set to 10pt, I assume Quality Check will flag any epubs with an @page other than 10pt.
And probably also flag any epub with any margins in a body statement.
But will it also flag epubs without any @page or body statements? Such as Calibre generated epubs?

The way it works for the moment (just got my first cut working) is it only looks at the css files that have @page or body set in the css with margins. I forgot about the fact that an ePub without any @page or body statements should also return True unless all the prefs are also zero.

Do you think that @page or body defined inside the xhtml should be a separate check from the css check?

Once I get all the css based stuff working to my satisfaction I'll move on to looking at styles inside the xhtml flows.

Originally Posted by capnm

Supposedly that can all be done in the css like this ...

@page flow1 {....}
@page flow2 {....}

.txt_flow1 { page: flow1 }
.txt_flow2 { page: flow2 }

but it wouldn't surprise me in the least if it didn't always work, or work as desired

I've got no idea on this stuff, keep hoping an expert on these nuances will chime in. Does .txt_flow1 need to go into the xhtml somewhere, and are you saying that the epub renderer knows which flow is flow1 vs. flow2 based on the manifest?

[capnm]

Option 1:
You aim to just flag commercially published epubs (where this all started).
So you only look at the css files that have @page or body set in the css with margins (where you are right now).

This will skip all Calibre generated and many bespoke epubs.
It *might* produce unexpected/unwanted results on epubs with multiple @page declarations or multiple css files. (Pretty darn rare).

Option 2:
You want to take on modifying other epubs too.
To find the 'body' margins look in the xhtml for <body class="xxxxxx" > , then go to the stylesheet (oops, do we have the right stylesheet? better check the xhtml for the link) and find .xxxxx {.....} and get the margins.
Now, you can *probably* take a shortcut and assume that the .calibre rule in the (one and only) stylesheet is what you're looking for in Calibre generated epubs. (But I don't know for sure).
And while you're scanning all the xhtml, you can check for the inline @page statements that Calibre generates.

Then you can zero the body margins, inject an @page in the stylesheet with new L/R margins, and decide whether to strip out all the inline @pages or reset them all with new T/B margins, and hope it all works.

Option 3:
Some sensible solution I've overlooked.



I'm in favor of option 1, leaving Calibre (re)generation to Calibre.

Of course, if you choose option 1, what happens if you zero the margins completely, then change your mind, and want 5pt margins?
Maybe inject @page even for zero margins?
Then the epub would still meet the search criteria of having body or @page margins set in the css ...

Clear as mud?

[capnm]

Originally Posted by ldolse

I've got no idea on this stuff, keep hoping an expert on these nuances will chime in. Does .txt_flow1 need to go into the xhtml somewhere, and are you saying that the epub renderer knows which flow is flow1 vs. flow2 based on the manifest?

Sorry for my cryptic brevity & lack of clarity

In a single-css, multiple page definition model, this:

@page flow1 {....}
@page flow2 {....}

.txt_flow1 { page: flow1; font-family: ....blah..blah.... }
.txt_flow2 { page: flow2; font-family: ....blah..blah.... }

goes in the css.

The xhtml would have something like:

<body class="txt_flow1" style="">
...
</body>

So flow1 styled blocks in the xhtml refer to the .txt_flow1 class in the css, and thus to flow1 page layout.
Similarly for flow2 styled blocks.


Or you can create different stylesheets for each flow, and each xhtml can link to the appropriate style sheet.

Or you can skin the cat by ......

Having said all that .... I ran across this trivia while trying to do something completely different quite some time ago, and I've never actually used it, or even seen it used
So I'm really not an expert, but hopefully partly correct

[ldolse]

Originally Posted by capnm

Option 1:
You aim to just flag commercially published epubs (where this all started).
So you only look at the css files that have @page or body set in the css with margins (where you are right now).

This will skip all Calibre generated and many bespoke epubs.
It *might* produce unexpected/unwanted results on epubs with multiple @page declarations or multiple css files. (Pretty darn rare).

Option 2:
You want to take on modifying other epubs too.


Option 3:
Some sensible solution I've overlooked.



I'm in favor of option 1, leaving Calibre (re)generation to Calibre.

Of course, if you choose option 1, what happens if you zero the margins completely, then change your mind, and want 5pt margins?
Maybe inject @page even for zero margins?
Then the epub would still meet the search criteria of having body or @page margins set in the css ...

Clear as mud?

I'm thinking something along the lines of option 1+3. I added the logic that ePubs without any body or @page margins flagged true in Quality Check and that wound up flagging 99% of my ePubs - I didn't particularly like that... I'm thinking to make a separate check for no @page margins in the css. That way the first check will generally only flag retail ePubs using either body or @page, and the second check will flag Calibre/bespoke/zeroed ePubs.

The search did highlight a fourth class of ePubs though - ones which have no css file whatsoever - around 1% of my ePubs. I haven't dug inside but I assume the stylesheet is inside the flow.

I don't want to get crazy with the xhtml modifications - it would be very simple to delete @page styles from each xhtml file. I think that would fix the issue kiwidude ran into with 1em specified there. It looks like that wouldn't provide ideal control to Calibre conversions though, looking at my old Calibre conversions it looks like it puts the default 5pt margins in both @page in the style tag and in the class Calibre assigns to body. I have no motivation to look up tags and class names in the html and correlate them to the css.

[capnm]

Sounding good!

I guess that for me, the only epubs I think I'd be comfortable doing this tweak to in bulk are those with body/@page in the css, so that's the search I'm most interested in


I kinda like the idea of having a tool to strip the in-line @page declarations, but I'd use it on specific epubs, not in bulk.

@kiwidude:
I'm curious, did your troublesome epub with in-line @page have them coded the same as Calibre's?

  <style type="text/css">
/*<![CDATA[*/
                @page { margin-bottom: 5.000000pt; margin-top: 5.000000pt; }
  /*]]>*/
  </style>

[jackie_w]

Originally Posted by capnm

I'm curious, did your troublesome epub with in-line @page have them coded the same as Calibre's?

  <style type="text/css">
/*<![CDATA[*/
                @page { margin-bottom: 5.000000pt; margin-top: 5.000000pt; }
  /*]]>*/
  </style>

My Calibre conversions do not look like that, they look like this:

<style type="text/css">
    @page { margin-bottom: 5.000000pt; margin-top: 15.000000pt; }
</style>

I had always assumed that CDATA statement came from Sigil not Calibre but I can't be sure because I don't use Sigil.

[capnm]

Originally Posted by jackie_w

I had always assumed that CDATA statement came from Sigil not Calibre but I can't be sure because I don't use Sigil.

You're right!
I'd been wondering why I thought I sometimes saw it wrapped in CDATA, and sometimes not .... it never occurred to me that when I opened a file for viewing in Sigil what I saw wasn't what was really there

@Idolse:
I poked around, and Calibre does use different in-line @page statements for different flows, and it can insert body statements in-line too.

[Agama]

Originally Posted by capnm

I kinda like the idea of having a tool to strip the in-line @page declarations, but I'd use it on specific epubs, not in bulk.

Agreed. I would also find such an option useful.

[ldolse]

Alright - I'll add options to Modify ePub to strip inline @page as a separate action, will need to think about whether to create a special 'check' in Quality Check. One option might be to have a single check categorize them into different groups, similar to the find dupes groupings, though that would be more difficult to intuit...

Where was Calibre setting the margins differently for a given flow - was it just for the cover page?

[capnm]

Originally Posted by ldolse

Where was Calibre setting the margins differently for a given flow - was it just for the cover page?

Yes. Cover and/or title page.

[jackie_w]

Originally Posted by ldolse

Where was Calibre setting the margins differently for a given flow - was it just for the cover page?

This is an extract from the heading of titlepage.xhtml (Calibre cover page)

<style type="text/css" title="override_css">
    @page {padding: 0pt; margin:0pt}
    body { text-align: center; padding:0pt; margin: 0pt; }
</style>

I think this must be Calibre standard because I'm not aware of manually setting these values.

The other area which might have had a 'special' header is jacket.xhtml (Calibre metadata jacket), but when I look, it looks exactly the same as the headers on my standard body text, i.e. as posted in post #158 above.

[ldolse]

Cool, that was about what I discovered too. I've passed an updated Quality Check over to kiwidude, I imagine it should go up relatively soon as long as he's happy with the changes.

The check compares all css - both css files and inline style tags against Calibre's preferences. The check is relaxed when checking inline styles, as Calibre only puts the top and bottom margins in @page and left and right go to the class assigned to body. Basically it only flags an inline margin as disagreeing with Calibre if something exceeds Calibre's prefs - if the in-line margins are less than Calibre's prefs it leaves them alone.

It also ignores titlepage.xhtml - I'll make this more robust at some later point and have it ignore anything marked explicitly as a cover in the manifest.

There's also a bug in modify ePub's margin rewrites, I'll be updating that shortly.

[ldolse]

Discussed with Kiwidude and releasing both plugin updates here for now.

Modify ePub 0.3.5 Beta - 26 Jun 2011
Fix an issue with css margin rewriting that created property names using '_' instead of '-'


Quality Check 1.5.9 Beta - 26 Jun 2011
Add the ability to compare ePub css 'book level' margins to Calibre Preferences
Add a check for ePubs with no 'book level' margins defined
Add a check for ePubs using inline @page margin definitions


Modify ePub still only operates on CSS files, but I'll be working on an update to modify all CSS including in-line in the near future. I wanted to get this release out to fix an outstanding bug - if you've rewritten any ePub files' margins using the previous plugin you'll need to do it again with this version.

Quality Check will check all types of css margins on body or @page, including Calibre generated in-line ones.

EDIT:
These have now been officially release and can be found on their respective threads in the Plugins forum.