About Word, letter-spacing and EPUB

[roger64]

Originally Posted by Hitch

(I also feel compelled to point out that you could easily build a regex that would wipe these out, in one-two passes, probably a hell of a lot faster than rewriting the macro.)
Hitch

I have of course a plain regex that does exactly that, on the EPUB.

Thank you for your findings. I hope the quest will soon be over.

Though the macro is rather old, this question has not been brought forward before because I am maybe the first to try it with an odt file and not a doc or docx.

Once the macro has been played, some characters or words have a red colour attribute. I usually take it out in LibreOffice for the whole book using Select All then (Format/caractère/Couleur de police/ Black). I did so for v4.

OK. The macro has been sent to those who showed some interest in this thread.

[BetterRed]

Originally Posted by roger64

Though the macro is rather old, this question has not been brought forward before because I am maybe the first to try it with an odt file and not a doc or docx.

Then I suggest therein lies the problem - get the person who wrote the macro to validate it on ODT files, or change your tool-chain - I'm outta here

BR

[Toxaris]

Ok, I quickly scanned through the macro's. Most of the macro's weren't that interesting at all and quite simple. One was a bit more complex and I haven't studied it thoroughly yet, that is Voir_NomsPropres_et_JeuDeCaracteres. From what I gather due to the quick scan, I assume it does some stuff in fixing names and titles.

Your actual process and intentions are important for the next step. My answer would depend on your process. I will make some assumptions and work from that. If the assumptions are not correct, please let me know.

1. You want to process one or more of the macro's and are not interested in the layout offered by the macro. With layout I mean the Styles and such.
2. You do want the marking of wrong words and perhaps the other fixes (convert hyperlinks to endnotes?)

Now, if these assumptions are correct, I understand the issues you are facing. In my opinion it has nothing to do with the handling of ODT by Word, but more that the macro's are combined with a template.

Now, for a possible solution. Most of the stuff, especially the macro Recherche_erreurs can easily be handled by my add-in. In fact, I think my add-in would be better (). It is just a long list of searches and marking (color red) of each hit found. It just marks all the words that might be wrong. That option is in my add-in and if you have known fixes, you might want to use the S/R procedure of the add-in. Of course it will take a while to put all the searches in it, but it is not hard and you can control which goes in and which not. You have control, not me. Adding a marking/search is easy enough and is not hard-coded.
My add-in will not change your document any more than that, if you want. It will leave your styles alone. Also, I saw some macro's that might influence dialogues. I want to put your attention to the Check dialogue procedure of my add-in. It should be able to handle that macro as well.

In short, I think that my add-in would be able to handle most parts of the macro without a hitch as long as you don't need the Styles in the template. Some others not, but I do not know how important are those. If they are, I could integrate them with the add-in, they are not that difficult at all (with the exception of the one mentioned earlier, I didn't check it yet).

Now, it sounds like an advertisement for my add-in, sorry about that. But I feel that it could cover most of your wishes already. The marking function is something I build in by request of BetterRed and it seems that this is also the core of your macro.

[roger64]

@Toxaris
Thank you for your reply.

This macro is quite long but technically probably not that complicated. Its usefulness lies in the fact that's it's tuned to many French OCR statistical mistakes which eases first proofreading.

If I understand you well, the root of the problem could be this question about the link between Styles and a template. Could you please elaborate a bit more about it so that I can understand better what happens here?

I will evaluate your proposal (writing this macro in your add-in) but I can tell you already it gives me no joy because I have no experience in macro, even less if possible in Word macros and it's so easy to make a misstep.

I will ask the author to test it with an odt file (where his own template is forcibly missing) to make him aware of the problem.

Maybe also there could be a way to use it with a copy of this template, like I do for now for normalvertu.dotm to preserve my titles and headings from being modified by Word original dotm template.

I'll be waiting for further comments.

[BetterRed]

Originally Posted by Toxaris

The marking function is something I build in by request of BetterRed and it seems that this is also the core of your macro.

Which I use daily - saves me heaps of time. And I wonder how I lived so long without Dialogue Checker.

And this is a plug for Toxaris' himself, whenever I've made suggestions for improvement, like marking, like simplifying some interactions in S&R, and Dialogue Checking, etc he always considers my ideas (other peoples too) and he always accommodates them in some form.

BR

[Toxaris]

No, no, you misunderstood me. You don't have to write macros of course. My add-in is not a macro at all, it is an application.

What I am saying, is that most of the stuff can already be done with my add-in. A lot in the macro are hard defined searches. Each hit will be colored red at the moment. There is a procedure in my add-in that does exactly that, but more. It uses an external file you can edit to mark everything you want in the color you want. The searches are not hard-coded as in the macro, but you can control them. You can add/remove them. In fact, you can group them and have each group marked in a different color. Editing the file is not hard at all. The add-in even has an editor if you don't want to do it by hand. If the color coding is all you need, I can even help you create the initial file. It is not that difficult.

I think the issue does not really lie in the macro's, but in the template.

[roger64]

Well, thank you for this explanaton. It looks more promising. I'll certainly have a look to check how I can break it off from this template.

[elibrarian]

I just couldn't resist - so here are my findings:

I think I've found the explanation. Whenever you apply some formatting - be it by keyboard shortcuts, setting a style or by a macro/vba program/whatever - Word (2013/2016, I'm on 2016) internally resets the style to the "DefaultParagraphFont" or in french, "Policepardfaut" style, before applying the new style. This is not visible to the naked eye, but you will see it when exporting to .odf and importing in Sigil (you can see something similar when exporting to unfiltered .htm and even in the internal .xml, if you feel brave enough to unzip a .docx-file and analyse the contents).

The DefaultParagraphFont (which is not the msoNormal-style as I asumed in a previous post) is special, and does not behave like any other (ordinary) style. It's normally hidden (in 2013/2016 anyway), and it can't be deleted. It's used for resetting text more or less to the bare necessities used in the actual paragraph.

There's an article concerning the DefaultParagraphFont here:

http://word.mvps.org/faqs/customization/DefParaFont.htm

with a link to an article about customizing the standard settings for the various Word versions up to and including 2016.

I enclose a couple of screenshots (please substitute "Policepardfaut"/"Defaultparagraphfont" for "Standardskrifttypeiafsnit". The first one is the unedited word file (I've used Roger Farney: Les Anekphantes from "Ebooks libres et gratuits"), exported to odt and imported in Sigil using the ODTImport 0.3.1 plugin by Doitsu. In shot nr. 2 I've done a little editing in the original word document before exporting (setting bold and italics using a style in the first paragraph and using keyboard shortcuts in the next). The third is an unfiltered htm-export from the same Word-document used as source for no. 2 (- just for the sake of illustration)





.

So it seems you can't avoid the "Policepardefault" in the Word-exports to ODT (I tried importing some of the odt-files from "Ebooks libres et gratuits", and the "Policepardfaut" are also found in some of them, as is also seen in screenshot no. 1) The best solution - since the source of the problem isn't your template or the VBA-macro, but Word itself - is probably making a saved regex in Sigil and run it on all html-files before doing anything else.

Regards,

Kim

[roger64]

Congratulations for a successful inquiry and lots of thanks for explaining at last how "Policepardfaut" came to infest silently the odt files in their xml innards before blooming later in the EPUB as multiple "Policepardfaut" spans.



Of course I already use a saved search to discard these pesky spans. This is easy, but finding the reason was quite tricky.

BTW, I also use ODTImport and people at ELG will be happy to see that there is at least a Danish reader.Just hope you are not alone.

Conclusion: A warning to odt users of Word 2013

So, it seems this thread finally reached a successful conclusion as all the questions which have been asked for have found, if not a perfect solution, at least an explanation and a workaround.

odt users of Word 2013 (2016?), should be ready to correct two Word induced defects appearing on their EPUB after conversion of their odt file:
1. - you'll have to correct the relative width of images and (maybe) suppress an empty paragraph
2. - you'll have to use a regex to suppress the useless and empty "Policepardfaut" spans.

As regard other points
- if a letter-spacing property appears in your body, check the kerning font setting in Word 2013
(credits to BetterRed)
- if some heading, titles, or notes change, attach a customized .dotm model to your file.

[Hitch]

Originally Posted by elibrarian

I just couldn't resist - so here are my findings:

I think I've found the explanation. Whenever you apply some formatting - be it by keyboard shortcuts, setting a style or by a macro/vba program/whatever - Word (2013/2016, I'm on 2016) internally resets the style to the "DefaultParagraphFont" or in french, "Policepardfaut" style, before applying the new style. This is not visible to the naked eye, but you will see it when exporting to .odf and importing in Sigil (you can see something similar when exporting to unfiltered .htm and even in the internal .xml, if you feel brave enough to unzip a .docx-file and analyse the contents).

The DefaultParagraphFont (which is not the msoNormal-style as I asumed in a previous post) is special, and does not behave like any other (ordinary) style. It's normally hidden (in 2013/2016 anyway), and it can't be deleted. It's used for resetting text more or less to the bare necessities used in the actual paragraph.

There's an article concerning the DefaultParagraphFont here:

http://word.mvps.org/faqs/customization/DefParaFont.htm

with a link to an article about customizing the standard settings for the various Word versions up to and including 2016.

I enclose a couple of screenshots (please substitute "Policepardfaut"/"Defaultparagraphfont" for "Standardskrifttypeiafsnit". The first one is the unedited word file (I've used Roger Farney: Les Anekphantes from "Ebooks libres et gratuits"), exported to odt and imported in Sigil using the ODTImport 0.3.1 plugin by Doitsu. In shot nr. 2 I've done a little editing in the original word document before exporting (setting bold and italics using a style in the first paragraph and using keyboard shortcuts in the next). The third is an unfiltered htm-export from the same Word-document used as source for no. 2 (- just for the sake of illustration)

Attachment 156378

Attachment 156379

Attachment 156380.

So it seems you can't avoid the "Policepardefault" in the Word-exports to ODT (I tried importing some of the odt-files from "Ebooks libres et gratuits", and the "Policepardfaut" are also found in some of them, as is also seen in screenshot no. 1) The best solution - since the source of the problem isn't your template or the VBA-macro, but Word itself - is probably making a saved regex in Sigil and run it on all html-files before doing anything else.

Regards,

Kim

Kim, I have mad respect for your skills, but we do this ALL DAY LONG, and we don't get those spans, in exporting to HTML/XML/etc.. You didn't get them, unless I've misread your post, in the third image--where you exported directly to HTML. Right?

Are you saying that the exports to .odt format can't avoid the spans?

My take on this is that these are effectively junk spans, created because the paragraphs are unstyled. Period. If the file is made correctly, using normal named styles, these should not appear.

What happens if these files are styled as they should be, before these save-as and export experiments? Rather than just using "Normal?"

Here's my stupid question, then: why is the .odt file being subsequently exported to .odt, anyway? Hasn't the process been described as .odt-->Macro-->Word-->HTML?

The only time we use the default para (class) is, as Barnhill notes, when we're swapping out direct styling for spans and vice-versa, because it's useful in Word searches, if those are needing to be done there, instead of in HTML and/or INDD. We don't see this, unless we suffer some type of brain-seizure and export an uncleaned, unfixed client's file to HTML, and even THEN, we typically don't see them.

Of course, we're not working in .odt, as I see no reason to do so.

Whatever. As long as Roger's issue or non-issue, or whatever, is solved.

Hitch

[elibrarian]

Originally Posted by Hitch

Kim, I have mad respect for your skills, but we do this ALL DAY LONG, and we don't get those spans, in exporting to HTML/XML/etc.. You didn't get them, unless I've misread your post, in the third image--where you exported directly to HTML. Right?

Wrong. I get those spans in HTML (unfiltered flavor only!) and in ODT. Study the third image a little closer The spans have a different, more detailed styling, but they're there all the same.

Originally Posted by Hitch

Are you saying that the exports to .odt format can't avoid the spans?

Apparently not, that is: not in the experiments I have made trying to solve this specific issue. (Which is of course not neither 100's or 1000's, I must admit. I usually don't make epubs from neither Word nor LibreOffice. But both programs have some useful addins for cleaning OCR, before I process the text)

Originally Posted by Hitch

My take on this is that these are effectively junk spans, created because the paragraphs are unstyled. Period. If the file is made correctly, using normal named styles, these should not appear

The paragraphs are styled in the the templates "Normal" style. And yes, it's junk, but Microsoft must have had some reason to do it so (hopefully).

Originally Posted by Hitch

What happens if these files are styled as they should be, before these save-as and export experiments? Rather than just using "Normal?"

Why should "Normal" not be correct? I mean, "Normal" may not be the best choice when we talk about conversion to epub using whatever tool one likes best, but after all, 90% or more of the text I see in all documents coming my way (my alter ego works in the danish union for "white collar workers", which is where I use the Office programs professionally) are styled as "Normal". When I "translate" Word documents from our jounalist to html for our website (using Pandoc), "Normal" translate to <p>, which is as it should be, in my opinion anyway.

Originally Posted by Hitch

Here's my stupid question, then: why is the .odt file being subsequently exported to .odt, anyway? Hasn't the process been described as .odt-->Macro-->Word-->HTML?

Dunno. It's Roger64s way of doing things. You do it your way, I do it mine and so on. We have a saying in danish (guess it exists in english too, just can't find the expression right now): "Man's will, man's heaven". Each to his own.

Originally Posted by Hitch

The only time we use the default para (class) is, as Barnhill notes, when we're swapping out direct styling for spans and vice-versa, because it's useful in Word searches, if those are needing to be done there, instead of in HTML and/or INDD. We don't see this, unless we suffer some type of brain-seizure and export an uncleaned, unfixed client's file to HTML, and even THEN, we typically don't see them.

You normally don't use the DefaultParagraphFont style as such. It's just there, lurking in the dark, sometimes springing to life as in Roger64's files. In Word 2013/2016 it's even hidden.

Originally Posted by Hitch

Whatever. As long as Roger's issue or non-issue, or whatever, is solved.

Hitch

Exactly

[roger64]

This thread is quite long and some points may have been lost on the way.

1. I always have been an odt user. (LibreOffice).

2. I decided to use Word 2013 just for this macro while keeping the odt format in the hope of creating a minimal disturbance...

3. After using Word, I come back to LibreOffice and later Sigil for converting my odt to EPUB with ODTImport.

After some uncertainties, I can now use this Word macro with minimal hassle.

A big thanks to all MR experts!!

[BobC]

Originally Posted by roger64

Hi

We just know our tools. LibreOffice is also pretty good for styles...<snip>...
Question: Is there a way to select my own template by default? I mean, when I open an odt, I would like to see that my template is attached automatically. This would be cleaner.

Roger, I may be misunderstanding the question and know that you are a fan of LibreOffice (as I am). Are you aware that once you have opened an ODT document in LO you can apply your own styles by going to "Styles and Formatting" (F11) then selecting the drop down on the icon labelled "New Style from Selection" and clicking on "Load Styles" to apply whatever template contains the styles you use for ebooks. This allows you to rapidly change the definition of the standard Paragraph Styles (I haven't used Character Styles much but assume they can also be "overwritten")

I created a template some time ago based on the FB21 format and used that when I was working with FB format books. Once you have the styles in that template it means that you don't need to keep reworking existing though you might need to re-assign a few to meet your needs.

Apologies if I'm "teaching my grandmother to suck eggs" but it took me quite some time to work out how to apply a different set of styles to an existing document.

BobC

[roger64]

@BobC

Thanks for this information. I was aware of it. In fact, you put together two unrelated items of mine, but this is because I have been unclear. The second one refers to Word templates.

Even with the odt format, Word does not use any LibreOffice .ott template. So one is facing a choice: either you attach a customized template in .dotm format or you will get by default the normal.dotm template (which may modify for example some of your titles or headings).

As all of this was new to me, and not really intuitive, I asked about the most convenient way to deal with these Word templates.