Sample Plugin for the Editor

[DiapDealer]

Originally Posted by arspr

Nevertheless one hint: I mounted my attached exception file from your example (downloaded from the initial post). And when I opened it (Windows Notepad), all the words were in the very same line with some kind of non-printable character between them. So I immediately thought about the UNIX-Windows CR/LF (or whatever) conflict and retyped those "return key" symbols in their place. So my file should be a pure genuine windows friendly one. Could it be the source of the trouble?

Shouldn't matter. The file is read in binary mode which should eliminate any EOL character nonsense. I've created those files on Win/Mac/Linux and mixed and matched their usage on the other platforms with no issues. But I'll double-check to be sure. If you get a chance, upload a small epub sample so I can be certain I'm starting with the exact same input as you are. Then I can pair that with your exceptions file to see if I can see what's going on.

[arspr]

Originally Posted by DiapDealer

Shouldn't matter. The file is read in binary mode which should eliminate any EOL character nonsense. I've created those files on Win/Mac/Linux and mixed and matched their usage on the other platforms with no issues. But I'll double-check to be sure. If you get a chance, upload a small epub sample so I can be certain I'm starting with the exact same input as you are. Then I can pair that with your exceptions file to see if I can see what's going on.

See my previous post (now edited). I can fully replicate the issue with your sample _before.epub

OTOH How do you manage to get ' correctly modified before decades in number? Because any automated decision you make can be really risky.

[DiapDealer]

Originally Posted by arspr

See my previous post (now edited). I can fully replicate the issue with your sample _before.epub

I'll be damned. I can replicate it with your exception file too! But in all fairness... that file is one weird puppy. It starts off with a UTF8 byte order mark (which is being included with the first 'bout' entry); each line is terminated with a CR/LF and then there's an additional LF character in between every entry. What did you use to create it?

I'll see if I can't come up with something to scrub the file of UTF8 BOM and additional LF characters before processing.

OTOH How do you manage to get ' correctly modified before decades in number? Because any automated decision you make can be really risky.

I don't DO anything. SmartyPants looks for a single-straight quote immediately followed by two digits that are immediately followed by the lower-case letter 's' ... and changes that straight-single quote to a curly right-single quote. There are a few other details that take care of unique situations, but that's the gist of it. I don't really see the "risk" in that. Instead of worrying about it, why not offer up a situation where my tool gets 'XXs wrong? Or calibre's smartener for that matter. They're no different in that regard. Just straight-up SmartyPants. Besides ... the decades thing is easy enough to double-check with a regex search.

I'm not trying to offer up any new, infallible quotation-smartening logic here. All the caveats for algorithmic quotation-smartening still apply. I'm just looking to add more control to WHAT you want to smarten, and to lessen the number of 'tis 'bout and 'cept'n foul-ups, and to do so without affecting any code in the document that doesn't pertain to punctuation being smartened.

[arspr]

Yes, I've also noticed those EOL rubbish and that is the indirect cause of it. Because of those EOL troubles and duplications, there are in fact BLANK lines in the text which Notepad doesn't show, (which is the software I firstly used and which leads to that buggy file) but Notepad++ does. (I was actually testing it).

And the problem is that if you leave just one single BLANK line in the exception file, (even at the end of it), the plugin fails in that way: every single word is qualified as a contraction... I really think you should filter out blank lines.

Test it, just use your .txt and type a return at the end of the file (or in the middle).

[DiapDealer]

Originally Posted by arspr

I really think you should filter out blank lines.

I plan to.
But keep in mind that I'm not interested in absolving users of any and all responsibility for providing valid input. If that file gets too hairy, it's getting ignored.

[phossler]

@DiapDealer -- Been playing with the PI a little. Very nice and going to be handy.

Q - in the delete spans, divs, etc. would you consider a little wildcard capability. Nothing like regex, but maybe something simple like ? and * for 1 char and many chars when matching class = " " ?

I know the screen shot is a little dumb since "filepos#####" is used for ID's and not classes, but the idea is basically the same

[DiapDealer]

Originally Posted by phossler

@DiapDealer -- Been playing with the PI a little. Very nice and going to be handy.

Q - in the delete spans, divs, etc. would you consider a little wildcard capability. Nothing like regex, but maybe something simple like ? and * for 1 char and many chars when matching class = " " ?

I know the screen shot is a little dumb since "filepos#####" is used for ID's and not classes, but the idea is basically the same

I'd love to try. I don't know how successful I'll be at it, but it makes perfect sense.

[DiapDealer]

Posting a somewhat beta-version of the plugin that contains many of the changes discussed.

** NOTE: because of horrible planning on my part, the atrociously named "diaps_editing_toolbag_SpanDivEdit_settings.js on" file in the plugin folder of calibre's configuration directory must be deleted before many of the changes to the Span/Div Editor Tool will appear (for those who have a previous version of the plugin installed). For that, I apologize. The file will not be recreated until I find a better way to handle customization. **

Changes to the Span/Div Editor tool:
1) inclusion of more html elements to delete/modify
2) inclusion of more choices to change those html elements to
3) more attributes to choose from in search criteria
4) the ability to use regex in the search box
(Regex is search-only at this point. There is no way currently to use regex captures to build the replacement attribute string when modifying.)

Changes to the Smarten Punctuation tool:
1) Makes sure the apostrophe exceptions file is somewhat sanitized before use (mainly makes sure blank lines and UTF-8 BOM doesn't make things barf)

[phossler]

Originally Posted by DiapDealer

... doesn't make things barf)

1. Can you put that in less technical language??

2. Thanks - even the alpha-version of the PI has been very useful to cleanup a number of inefficiently formatted epubs. More than once I've messed up the RegEx doing that manually and had to revert to a previously saved version. The PI is safer AND less work

3. Keep 'PlugIn-g' away (pun intended)

[arspr]

Noob question with plugins as I cannot find it in the Calibre User Manual:

Where is the plugin stored (Calibre in Windows) and therefore its diaps_editing_toolbag_SpanDivEdit_settings.json file? I've searched for it in the Calibre directory (and just for *.json) and I cannot find nothing relevant...

[DiapDealer]

Originally Posted by arspr

Noob question with plugins as I cannot find it in the Calibre User Manual:

Where is the plugin stored (Calibre in Windows) and therefore its diaps_editing_toolbag_SpanDivEdit_settings.json file? I've searched for it in the Calibre directory (and just for *.json) and I cannot find nothing relevant...

On Windows, calibre's configuration directory is usually in your user directory:
"<username>/AppData/Roaming/calibre"

The plugins (and the json file(s)) are in the 'plugins' sub-directory.

You can also find it from within calibre by going to Preferences->Miscellaneous and clicking the "Open calibre configuration directory" button.

I apologize for the awkwardness of this solution. I'll try to create a way to reset that file (if it exists) from within the tool itself as soon as I can.

[eschwartz]

Or even a way to edit it from the plugin...

[arspr]

Originally Posted by DiapDealer

On Windows, calibre's configuration directory is usually in your user directory:
"<username>/AppData/Roaming/calibre"

The plugins (and the json file(s)) are in the 'plugins' sub-directory.

You can also find it from within calibre by going to Preferences->Miscellaneous and clicking the "Open calibre configuration directory" button.

I apologize for the awkwardness of this solution. I'll try to create a way to reset that file (if it exists) from within the tool itself as soon as I can.

I've deleted both .json files and installed your new version.

Although I suppose you've already tested it, as I was the culprit, I'll also test the bogus exception file ASAP.



Another side comment, the more I think about it the more awkward it seems. When the tools have an explicit dialog window why don't you move the "current file/entire book" option to that dialog? Having two look for tool options in two places seems a bit un-userfriendly IMO.

[DiapDealer]

Originally Posted by arspr

Another side comment, the more I think about it the more awkward it seems. When the tools have an explicit dialog window why don't you move the "current file/entire book" option to that dialog? Having two look for tool options in two places seems a bit un-userfriendly IMO.

Because many of the tools I will be using won't have any explicit dialogs, and I'd rather the "current file/entire book" option remain in a consistent location across all tools I may publish/use. I don't really consider it "two different places" myself. All actions for each tool are available from a single icon/menu: Run (including any run options), Set Scope, and--eventually--Customization. Scope and Customization will tend to persist from session to session. Most Run options (with some exceptions like Smarten) will likely need to be chosen each time they're run.

[DiapDealer]

Originally Posted by eschwartz

Or even a way to edit it from the plugin...

Heaven help us.

Look on the same menu where we choose the "current file/entire book" scope.