[Plugin] PunctuationSmarten Sigil plugin

[DiapDealer]

OK. I'm going to post a new version of the plugin to test. It has been completely rewritten using Qt for Python rather than tk/tkinter.

Functionally, this test version should work exactly like the old version. I'm just looking to verify that everything works like it always did. New stuff will have to wait.

There's a lot of extra debug output that could come in handy if anyone runs into complications.

As always, make good backups and create lots of checkpoints with Sigil to help make sure nothing is happening that shouldn't be.

I don't expect trouble, but be smart about testing.

EDIT: in case it wasn't obvious, this new version is not going to work with anything less than Sigil 2.0 (and nothing less than Python 3.6 if you're using an external Python interpreter for plugins).

[DiapDealer]

Can anyone who has downloaded the test version at least verify that it's working as expected?

[retiredbiker]

I just grabbed a sf novel from Faded Page and made sure it had typewriter punctuation, then downloaded your new version and gave it a go. It seems to have done everything correctly, with a fairly quick look-through.

[DiapDealer]

Thanks! It wasn't perfect to start with, but I'm hoping this updated version didn't introduce any new issues.

[DNSB]

I grabbed a couple of ePubs from Gutenberg and ran them through with the old plugin. I savef the output files with _old appended to the filename. Updated the plugin and ran the same files through. I did a quick eyeball check and there seems to be no difference between the two versions. I also ran the two _old versions through the new plugin and both reported no changes made.

[carmenchu]

Originally Posted by DiapDealer

Can anyone who has downloaded the test version at least verify that it's working as expected?

Thanks a lot! Just downloaded and made a very quick try from the appimage: now files were selected and processes as expected.
My apologies for not checking the forum much earlier--I find switching from Windows to a Linux distro both possible and rewarding, but in no way hassle-free, if one wants to preserve most of thirty-years-long working habits...
again.

[DiapDealer]

No need to apologize. I should have switched to Qt for Python long ago. Glad to hear it's working for you! I was hoping you would be able to try it out.

[jd1004]

I ran a comparison test of the PunctuationSmarten plugin between versions 0.3.3 and 0.5.990.

I used seven ebooks in English with smart quotes and with lots of 'cause and 'em and 'til and the originally posted apos_exceptions file plus the additional entries Cause, course, Course, kay, nough, Nough, til, tween.

For each epub:

- export the epub to text using Epub2Text v0.1.4
- change all quotes [“”] to " and all quotes [‘’`´] to ' and save the epub
- run v0.3.3 and export the epub to text
- close the epub without saving, thereby keeping the straight quotes
- run v0.5.990 and export the epub to text

Result:

- Versions 0.3.3 and 0.5.990 produce the same output for all seven ebooks.

- (I need to add Gainst/gainst, Twere/twere, Twill/twill, Twixt/twixt, and Twould/twould to my apos_exceptions file.)

- V0.5.990 starts the log output with "Close event accepted". Is that relevant? What does it mean?

- All in all, the plugin works pretty well. Both versions fail for the HTML below, though. Don't know if anything can be done about that.

Incorrect smart quotes:

- The HTML
      <div>
         '<i>Father!</i>'
      </div>
  produces
         ‘Father!‘
- The HTML
      <div>
         But she had already begun to read. '"—And when ... and—"'
      </div>
  produces
         But she had already begun to read. ’”—And when ... and—”’
- The HTML
      <p>'Bla bla in '37.'</p>
  produces
         ‘Bla bla in ‘37.’
- The HTML
      <p>'They're speaking "twin" ', she explained.</p>
  produces
         ‘They’re speaking “twin” ‘, she explained.
  Likewise, the HTML
      <p>' "Met"? How do you ...?'</p>
  produces
         ’ “Met”? How do you ...?’

Could you update your original post with the new version and an updated history of changes?

Thanks for all your good work! It is very much appreciated.

[DiapDealer]

0.5.990 is a test version. The new version will be 6.0.0 when I get around to it. You're welcome to keep using the test version in the meantime.

There are always going to be situations where the original PunctuationSmarten Python script falls down. I'm afraid that I've given up chasing those edge cases along time ago. This latest was simply an update to using Qt for Python as the GUI instead of TKinter. Nothing more

The "Close event accepted" message in the log is just some debug output that hasn't been cleaned up (hence the testing version). It just means that the close event (user hitting the OK button or the X in the title bar) has happened. You'll note that the message is "Close event rejected" when you click the Cancel button (or use the ESC key) to end the plugin.

[carmenchu]

Hello!
Just have tripped on an anomalous behaviour with version v0.5.990 under Sigil-2.7.0-x86_64.AppImage, on Linux Mint:
* Trying to point the pluguin to apos_exceptions.txt on the default configuration folder (/home/[user]/.local/share/sigil-ebook/sigil/) simply crashes it--freezes, and on [cancel] reports crash on loading...
* Going instead to said configuration folder, and hand-editing /home/[user]/.local/share/sigil-ebook/sigil/plugins_prefs/PunctuationSmarten/PunctuationSmarten.json to

    "useFile": true,
    "useFilePath": "/home/[user]/.local/share/sigil-ebook/sigil/apos_exceptions.txt",

worked quite well...
Can there be some issue with permissions? I tried to move apos_exceptions.txt to one of my miscellaneous folders, but the trouble persisted.
Thanks for all your work!

[DiapDealer]

I'll have to take a look. I can't imagine what it might be. The appimage uses the same preferences folder as the regular version of Sigil. I'm not sure I've ever tried to store the apostrophe exceptions file in the root of Sigil's preferences folder, to be honest. I think I always tested with it being located in my Documents folder.

[DiapDealer]

I can't make the crash happen on my Arch Linux system. I can point to the apos_exceptions.txt file whether it's in my Documents folder or in /home/[user]/.local/share/sigil-ebook/sigil/ with no trouble (Using the 2.7.0 AppImage). I can't think why it would matter if the file location was read from the preferences .json, or if it was loaded via the gui QFileDialog.getOpenFilename. I'm using non-native dialogs so switching to, and browsing, normally hidden folders in /home/<usr>/.local shouldn't be a problem (and doesn't appear to be on Arch).

Do you happen to have a user name with diacritics in it? Perhaps I'm not handling something correctly when trying to print the path in the textbox after choosing a new file. Something that doesn't happen when reading the last used path from the preferences file. I'll keep looking.

[carmenchu]

Originally Posted by DiapDealer

I can't make the crash happen on my Arch Linux system. I can point to the apos_exceptions.txt file whether it's in my Documents folder or in /home/[user]/.local/share/sigil-ebook/sigil/ with no trouble (Using the 2.7.0 AppImage). I can't think why it would matter if the file location was read from the preferences .json, or if it was loaded via the gui QFileDialog.getOpenFilename. I'm using non-native dialogs so switching to, and browsing, normally hidden folders in /home/<usr>/.local shouldn't be a problem (and doesn't appear to be on Arch).

* The crash happened after inserting the full path to the apos_exceptions file in the dialog window--as it was impossible to browse to it by the hidden folders in the path.

Do you happen to have a user name with diacritics in it? Perhaps I'm not handling something correctly when trying to print the path in the textbox after choosing a new file. Something that doesn't happen when reading the last used path from the preferences file. I'll keep looking.

* No: sheer vanilla text.
At need, can replicate the process and provide screenshots--if told which may be relevant.

[DiapDealer]

Originally Posted by carmenchu

* The crash happened after inserting the full path to the apos_exceptions file in the dialog window--as it was impossible to browse to it by the hidden folders in the path.

Ah... now I understand. You're typing (or pasting) the full path in the dialog rather than browsing to it. I'll try to figure out why that might be happening. I'm still having a heard time making it crash that way myself. Just know that in the meantime, it's not "impossible" to browse hidden folders/files with that Open File Dialog.

Just right click in the file selection browser window (where the files and folders are) and select "Show Hidden Files" and browse directly to the file's location. The non-native Open File Dialog will remember that selection the next time it opens.