e-Book Tools - a Word add-in

[Toxaris]

Originally Posted by Jellby

No. For a prolongued quotation it would be:

«First paragraph.
»Second paragraph.
»Third and final paragraph.»

Dialogues don't use quotes (only for continuation):

—First paragraph.
»Second paragraph.
»Third and final paragraph.

That is actually not that bad I think. For quotations that could be partially automated. If this is the case for Spanish, I think I can actually do something with this. Detect the language and for Spanish honor this. If you could sent me a (short) Spanish text for testing, that would be much appreciated.

[BetterRed]

Originally Posted by Toxaris

Of course I could implement this. However, it would require attention, since if the end quote is indeed missing, there might be a mistake when the next person starts to speak and it is wrongly interpreted as continued speech.

Yes it would - intelligent, discerning, comprehending attention. And if I made a mistake and believed it to be an different speaker that would be my fault. If the alternative is what I got yesterday, then I will forego the use of this tool.

BR

[Toxaris]

Originally Posted by BetterRed

Not everything can be automated. I for one would rather it wasn't, over automation is either hard to maintain or it does not deal adequately with the exceptions - some of which we haven't even thought of yet. Manual intervention to deal with exceptions introduces real intelligence.

I just don't want to set false expectations.

Originally Posted by BetterRed

I'm confused, is the style as per the Editors Blog post I linked to yesterday, used in The Netherlands never, sometimes or always. You seem to have said all three in different places. Not that that matters.

Just from the top of my head I can think of 6 different ways quotation/dialogue is handled in Dutch in the last 100 yrs, depending on publisher, writer, target audience or time period. Currently it is a single quote and always a set that can span multiple paragraphs.

Originally Posted by BetterRed

As someone posted elsewhere pick up any quality book published in the English Speaking world and you will see the style I prefer, as per the Editors Blog, which is as per Cambridge and Chicago, maybe not the OUP, by definition they're all card carrying members of the Contrarian Society.

Define quality... For example James Joyce used dashes as marks (not honored by all publishers). Your mileage may vary. I come across almost all variations, but the one you described almost never.

No matter, I already said I will look into it.

Originally Posted by BetterRed

Mate, to be honest I don't care what the HTML looks like, its a means to an end. I only care about what the EPUB looks like in a viewer or on my Android Tablet and maybe in an Editor like Sigil or calibre's. As I already said, I am quite happy to forego the creation of EPUBs from my consideration because I already have an acceptable working solution for converting a DOCX to EPUB.

Calibre uses a different way for converting by trying to keep the layout from Word. There is an option in the tools to create a stylesheet based on the Word layout, but I don't think if it does smallcaps like you apparently want. In the end it is the same, a class specific for the smallcaps. You need to provide the stylesheet, except if you use the generate stylesheet option.

With regards to options, I can say the following. The default directory is the same directory where the DOCX is. The option is there if you want another directory. Also, the tool will only open the ePUB if the option is checked at the generate ePUB screen (off by default), so that is also not an issue. The last one is interesting, as the DOCX must be closed for the process. You can reopen it at the end if you want.

[BetterRed]

@Toxaris - regarding expectations - I expect a tool to be of assistance in getting a job done but I don't expect it to do the job in its entirety - otherwise I'd have nothing to do

I am going to stick with the Calibre DOCX conversion, it already does what I want, I've used it to convert ~50,000 'books', all of which use a common template, which I've probably unwittingly fashioned around the capabilities of the DOCXInput PI.

Regarding the placement of closing quotes in multi para speech. Many of my 'books' are transcripts, they're full of 'soliloquies', there are currently over 6000 of them each styled following Chicago guidelines which on this issue is as per the Authors Blog post. It's also how they are 'presented' by the transcribers, many of whom are former court reporters. So I won't be changing the grammatical/punctuation style from what I already use. On matters of consistency I am somewhat OCD.

But... I think I have a way to use what there is without any significant changes!

When an anomaly is found, I can put a 'mark' on the para via a 'small adjustment', and click No. Then when the dialogue checker finishes I can review the anomalies via Word's Find 'mark' - at my leisure This will suit my work patterns much, much better.

How will I know what the anomaly is about, by reading what's there, and maybe by having different 'marks' for different types of anomalies.

But I would like to have a way of suppressing all those popups, long documents can take a while to dialogue check, and I prefer to do something other than click 24 OK's whilst I'm waiting

BR

[Toxaris]

Originally Posted by BetterRed

But I would like to have a way of suppressing all those popups, long documents can take a while to dialogue check, and I prefer to do something other than click 24 OK's whilst I'm waiting

I have halved the number of pop-ups. There is still mention on what quote pair will be checked next. Otherwise you would not know what is being checked at the moment in case you need a manual adjustment.

Probably the update with that change will be posted on Sunday.

[mrmikel]

Mate, to be honest I don't care what the HTML looks like, its a means to an end. I only care about what the EPUB looks like in a viewer or on my Android Tablet and maybe in an Editor like Sigil or calibre's.

That leaves out a lot of the world having e-ink readers which are generally smaller and are far less tolerant of HTML out of spec. I have a tablet and e-ink reader and I want what I do to be ok in both, which is not an easy thing.

[BetterRed]

Originally Posted by mrmikel

That leaves out a lot of the world having e-ink readers which are generally smaller and are far less tolerant of HTML out of spec. I have a tablet and e-ink reader and I want what I do to be ok in both, which is not an easy thing.

@mrmikel - If you look at the fragment of HTML (generated by EPUB Tools), to which my comment referred you will see that it contained curious strings - ' ' - within the text, the originating document had no non breaking spaces, anywhere. And the HTML had a complete line of text (marked in red) that was not in the original document.

In saying I don't care what the HTML looks like I am referring to its appearance not to its validity. Most of all I care about the content, and its appearance because that's what the reader reads, it's to that sentiment that my 'means to an end' comment applies.

BR

[Toxaris]

Originally Posted by BetterRed

@mrmikel - If you look at the fragment of HTML (generated by EPUB Tools), to which my comment referred you will see that it contained curious strings - ' ' - within the text, the originating document had no non breaking spaces, anywhere. And the HTML had a complete line of text (marked in red) that was not in the original document.

Like I said, the   should have been  . That is fixed in my version. The fact that an   is placed, is because a lot of readers/reading programs will ignore, and thus not display, empty paragraphs. So, the only way to mimic this, is by either making the paragraphs only seem empty or by removing them and set a margin. The first option is the most simple and closest to the original situation.

[Jellby]

Originally Posted by Toxaris

If you could sent me a (short) Spanish text for testing, that would be much appreciated.

You can use this book, for example. Chapter IX, XVI-XIX, XXI have that kind of continuation quote. If you prefer, I have also uploaded the same book in ePub format.

[Toxaris]

Thanks Jellby, I will investigate.

[BetterRed]

Originally Posted by Toxaris

Like I said, the   should have been  . That is fixed in my version. The fact that an   is placed, is because a lot of readers/reading programs will ignore, and thus not display, empty paragraphs. So, the only way to mimic this, is by either making the paragraphs only seem empty or by removing them and set a margin. The first option is the most simple and closest to the original situation.

I should add that I would never normally have an empty paragraph in the source document, it was remiss of me to have them in my test document, and I should have mentioned that earlier.

I did write somewhere (maybe elsewhere than here tho') that I never have empty paragraphs and tabs in a document, indeed I have a search and destroy macro that I use in my final clean up. Patently I didn't apply my own rules in that small caps test document.

Peccāvī, peccāvī on all counts Your Grace

Re Search and Replace Settings

In the list on the left - should the up/down pointers move rows in the rules document Table 1, it didn't when I tried it. But I edited Table 1 in the rules and got the result I wanted.

Could the line items have a check boxes for whole words only and find all word forms

I'm finding the S&R really useful for dealing with transcriber abbreviations; different transcribers use different abbreviations <sigh>

Originally Posted by Toxaris

I have halved the number of pop-ups. There is still mention on what quote pair will be checked next. Otherwise you would not know what is being checked at the moment in case you need a manual adjustment.

You seem to be saying that users must remember what was in the last pop-up box they clicked through when they get the next Correction pop up. IMO that's an unrealistic expectation. They might have picked up a ringing phone, answered a call of nature, serviced their caffeine addiction, dealt with an attention seeking cat. Or perhaps they suffer short term memory loss because of their age

Would it not be better to have the 'Correction pop up' inform the user of the current Dialogue Checker phase - see attachment.


I'm getting "Paragraph mark found between opening and closing mark!" errors around some dialogue paragraphs that have no leading or trailing attribution, in passages such as this:

Mary swept into the room looking like the cat that got the cream.
"It's beautiful day."
After a glance outside Fred said glumly, "But I think it will rain later."

But it appears to be inconsistent, i.e. its not all unattributed dialogue paragraphs.

Does the presence or absence of dialogue attribution play any role in the Dialogue Checking? I think it may depend on whether the following paragraph contains dialogue. And it may only happen when the search direction is Up. If I can find a consistent pattern I'll report back.

BR

[Toxaris]

Originally Posted by BetterRed

Re Search and Replace Settings

In the list on the left - should the up/down pointers move rows in the rules document Table 1, it didn't when I tried it. But I edited Table 1 in the rules and got the result I wanted.

I think I understand what you are saying here. I will check, I haven't touched that part for quite a while. You mean editing the S/R document within the settings, right?

Originally Posted by BetterRed

Could the line items have a check boxes for whole words only and find all word forms

Not getting what you mean here. If you mean by line item a search request, than you can do that yourself by putting that into the S/R request. If you put your search between <>, it should only hit complete words (wildcard search only). That is normal S/R behavior for Word.

Originally Posted by BetterRed

You seem to be saying that users must remember what was in the last pop-up box they clicked through when they get the next Correction pop up. IMO that's an unrealistic expectation. They might have picked up a ringing phone, answered a call of nature, serviced their caffeine addiction, dealt with an attention seeking cat. Or perhaps they suffer short term memory loss because of their age

Would it not be better to have the 'Correction pop up' inform the user of the current Dialogue Checker phase - see attachment.

Perhaps. I do not thing it is an unrealistic expectation. I will think about it, but halving it is already an improvement. Not that it is hard to implement, but I do not feel the added value here.

Originally Posted by BetterRed

I'm getting "Paragraph mark found between opening and closing mark!" errors around some dialogue paragraphs that have no leading or trailing attribution, in passages such as this:

Mary swept into the room looking like the cat that got the cream.
"It's beautiful day."
After a glance outside Fred said glumly, "But I think it will rain later."

But it appears to be inconsistent, i.e. its not all unattributed dialogue paragraphs.

Does the presence or absence of dialogue attribution play any role in the Dialogue Checking? I think it may depend on whether the following paragraph contains dialogue. And it may only happen when the search direction is Up. If I can find a consistent pattern I'll report back.

This makes no sense at all. There is no difference between unattributed and attributed dialogue for the procedure. As long as curly quotes are used, all will be picked up. As a very brief explanation, part of the procedure is finding an opening mark and then grow the selection until it either finds a paragraph mark, an opening mark or a closing mark. In the latter case it is fine and the next opening mark is found. The other cases require action. On the way back it is vice versa. If you get reproducible false hits, I would be very, very interested in that since it would (almost) impossible.

[BetterRed]

Originally Posted by Toxaris

I think I understand what you are saying here. I will check, I haven't touched that part for quite a while. You mean editing the S/R document within the settings, right?

The attachment shows the pointers I refer to - not a big issue this, the document is easy to change manually

Originally Posted by Toxaris

Not getting what you mean here. If you mean by line item a search request, than you can do that yourself by putting that into the S/R request. If you put your search between <>, it should only hit complete words (wildcard search only). That is normal S/R behavior for Word.

The attachment also shows the bottom part of Words Find/Replace dialogue. I have highlighted the whole words only and find all word forms conditions. I was wondering if those two conditions could be placed in the table along with Wildcards and Case sensitive.

whole words only - Say I want to replace leaned with leant, but I wouldn't want cleaned to be changed to cleant. So in vanilla Word S&R I would check whole words only. My experience is that using a leading or trailing space wont work because of adjacent quotes and punctuation.

find all word forms - I would use this if I wanted to do replaces such as realise with realize, it will pick up words like realisation, realised etc. I import quite a lot of documents where both the US & UK spelling variants are contained within the same document The Word dictionary I use allows both variants. I could use a Word Exclusion dictionary to disallow realise and all forms thereof, but... sometimes for political reasons I prefer to have the UK spelling variant. These mixed spelling documents are most often found in translations of public domain documents emanating from Continental Europe. I think I already said I'm a bit OCD on matters of consistency.

Originally Posted by Toxaris

Perhaps. I do not thing it is an unrealistic expectation. I will think about it, but halving it is already an improvement. Not that it is hard to implement, but I do not feel the added value here.

Do you have any elderly relations or friends, if so I think its a safe bet that for some of them their short term memory is not what it once was. Or perhaps you know someone who has suffered a stroke, that's another condition that can lead to short term memory loss. And then there are interrupts, maybe you can remember the last pop up you saw after an hour on the phone, a bathroom visit and making a fresh coffee - but I can't, and I doubt that I ever could.

If I had the code (no I don't want it) the popups might be useful, but as an end user I find them irritating. If I want/need to remember them (see next section) then I will either have to run a video capture program and take snapshots or write them down, there's no chance of me doing the latter and the only time I tried a running a video capture program over Word, Word crashed.

Originally Posted by Toxaris

This makes no sense at all. There is no difference between unattributed and attributed dialogue for the procedure. As long as curly quotes are used, all will be picked up. As a very brief explanation, part of the procedure is finding an opening mark and then grow the selection until it either finds a paragraph mark, an opening mark or a closing mark. In the latter case it is fine and the next opening mark is found. The other cases require action. On the way back it is vice versa. If you get reproducible false hits, I would be very, very interested in that since it would (almost) impossible.

I hear what you say. I have some primitive macros that do in part at least what the searching part of your tool does, so it was a surprise to me when I saw this happen. Especially as it was so inconsistent. I did save a couple of pages around one occurrence, needless to say it didn't happen on that fragment. I'm not even sure its consistent between running it today and running it tomorrow. I am running it on large documents 100,000+ words.

I'm sure I would be better able to see a pattern if I didn't have to confirm the change of state popups, and if the 'error' dialogue had the current state information as I suggested yesterday.

BR

[Toxaris]

Originally Posted by BetterRed

The attachment shows the pointers I refer to - not a big issue this, the document is easy to change manually

Fixed in the current beta. Not only that, but also fixed some other bugs there that I (and everybody else...) missed, but that I found out now.

Originally Posted by BetterRed

The attachment also shows the bottom part of Words Find/Replace dialogue. I have highlighted the whole words only and find all word forms conditions. I was wondering if those two conditions could be placed in the table along with Wildcards and Case sensitive.

Again, the whole words only can also be achieved by putting your search between <> and enabling wildcards.
I must admit that I am not familiar with the find all word forms option. I have never used it, perhaps it is because it is only for English. The main issue I have with adding a column for that, is that I doubt it will be useful for most people and that it would cause issues with the S/R-documents already in use. In principle it would be incompatible and thus force the users to change their document for a feature that almost nobody uses. So, until more users tell me to incorporate this, it will be a no.

Originally Posted by BetterRed

If I had the code (no I don't want it) the popups might be useful, but as an end user I find them irritating. If I want/need to remember them (see next section) then I will either have to run a video capture program and take snapshots or write them down, there's no chance of me doing the latter and the only time I tried a running a video capture program over Word, Word crashed.

I made an alternative. As of the current beta the popups in this procedure can be switched off if you want.

I also made another option for the next version (which is in beta now). If a dialogue/quotation spans multiple paragraphs and the next paragraph starts with a opening quotation mark (English) or a closing quotation mark (Spanish), the procedure will still stop at the paragraph mark, but not at the quotation mark anymore. So, that would prevent errors and additional actions.

There are also some other changes, but not this much. So, unless you come with more things and I do not find additional bugs, this beta will be released soon.

[BetterRed]

Originally Posted by Toxaris

I must admit that I am not familiar with the find all word forms option. I have never used it, perhaps it is because it is only for English. The main issue I have with adding a column for that, is that I doubt it will be useful for most people and that it would cause issues with the S/R-documents already in use. In principle it would be incompatible and thus force the users to change their document for a feature that almost nobody uses. So, until more users tell me to incorporate this, it will be a no.

Good point - I'll edit the word lists I have into the rules document by hand.

This might be a feature you'd like to consider - importing a word list to seed a table.

Originally Posted by Toxaris

I made an alternative. As of the current beta the popups in this procedure can be switched off if you want.

That sounds perfect

Originally Posted by Toxaris

I also made another option for the next version (which is in beta now). If a dialogue/quotation spans multiple paragraphs and the next paragraph starts with a opening quotation mark (English) or a closing quotation mark (Spanish), the procedure will still stop at the paragraph mark, but not at the quotation mark anymore. So, that would prevent errors and additional actions.

There are also some other changes, but not this much. So, unless you come with more things and I do not find additional bugs, this beta will be released soon.

Are you also able to incorporate the suggestion I made to put the text that was in the last popup (even if it wasn't popped) into the error correction dialogue. IMO that would obviate the need to show the popups, except under circumstances that I can't imagine - yet

I'll leave further investigation of that unattributed dialogue anomaly until the new version is released. Whilst I'm looking forward to its release, there's no urgency, I'd rather wait a bit longer if that's necessary/beneficial.

BR