0.7.700 Alpha Ready For Testing

[KevinH]

Hi,

I think it is coming from the tempfile or from the plugin launcher outdir. Perhaps on Windows mkdtemp returns a unicode path? Or it could come from the unicode literal string for the name of the new epub 'new.epub'.

@contextmanager
def make_temp_directory():
    import tempfile, shutil
    temp_dir = tempfile.mkdtemp()
    yield temp_dir
    shutil.rmtree(temp_dir)

If so we should probably change this to be:

@contextmanager
def make_temp_directory():
    import tempfile, shutil
    from utf8_utils import utf8_str
    temp_dir = utf8_str(tempfile.mkdtemp())
    yield temp_dir
    shutil.rmtree(temp_dir)

utf8_utils is already imported as part of the launcher so importing from it should be no issue. Either that or it is the outdir that needs to be converted to utf8 by the launcher (but I think it already does that).

The problem is all it takes is one full unicode path piece to exist and when joining paths to it, all of the resulting path will be upconverted to full unicode. So tracking down which is the culprit will take some time but I will find it.

Perhaps simply removing the unicode literals and seeing what happens might be easiest as now I am wondering what happens with the following code:

self.bk.addotherfile('new.epub', data)

The 'new.epub" is a literal and so probably is full unicode. Then concatenating its name to the outdir path probably forces the whole thing to be full unicode.


Thanks,

KevinH

Originally Posted by DiapDealer

Hey Kevin, the full unicode strings are probably coming from me using the "from __future__ import unicode_literals" (or whatever) in my code. But it works in the calibre plugin without warning, so I'm not sure. I've not gotten that particular warning in my preliminary testing so far (mostly on Linux--a little Windows).

[gipsy]

Hi,
first, thanks for continuing Sigil!!!

I have two suggestions.

1. It's possible to add a feature for multiple dictionaries?
   For example i can only work with greek now, but in some epubs i have english, french, etc languages.
   The spellcheck mark them as error. If you seperate the Dictionary (an make them Dictionaries) from User Dictionaries would be more convenient.
2. I have some regex searches that i check the ebook... As a plugin can we have something to add them there and execute them simultaneously?

Thanks

[KevinH]

Hi DiapDealer,

Once I removed the from future unicode-literals from both mobi_stuff.py and KindleImport.py all warnings went away. When we built up the path to the epub using the literal 'mobi8' we were forcing the entire path to the new epub to be full unicode.

So unless KindleImport.py and mobi_stuff.py actually need the from future unicode_literals stuff, can we remove them?

When I build the python 3 interface, then I will move the python interfaces to be full unicode. Right now much of the launcher/wrapper code depends on the utf-8 strings much like KindleUnpack does.


Thanks,

Kevin

[JSWolf]

Originally Posted by KevinH

Hi DiapDealer,

Once I removed the from future unicode-literals from both mobi_stuff.py and KindleImport.py all warnings went away. When we built up the path to the epub using the literal 'mobi8' we were forcing the entire path to the new epub to be full unicode.

So unless KindleImport.py and mobi_stuff.py actually need the from future unicode_literals stuff, can we remove them?

When I build the python 3 interface, then I will move the python interfaces to be full unicode. Right now much of the launcher/wrapper code depends on the utf-8 strings much like KindleUnpack does.


Thanks,

Kevin

When you do the Python 3 interface, please make it run Python 2 code. Calibre doesn't do Python 3 and I don't want there to have to be two versions of the KindleUnpack code that has to be maintained due to Python version differences.

[KevinH]

Originally Posted by gipsy

Hi,[*]It's possible to add a feature for multiple dictionaries?
For example i can only work with greek now, but in some epubs i have english, french, etc languages.
The spellcheck mark them as error. If you seperate the Dictionary (an make them Dictionaries) from User Dictionaries would be more convenient.

One thing we could do is allow you to pass text through all dictionaries you have installed instead of just one. OpenOffice.org/LibreOffice has/had that option. Then if the word is spelled properly in *any* dictionary you have, it is considered properly spelled. It can create problems for some words that are similar but not exactly the same in multiple languages.

The only other way to deal with multiple dictionaries is to somehow allow the user to tag each and every word of text with the language that word is drawn from. That is very time consuming to do but is possible in both Word and OpenOffice/LibreOffice. You could possibly add span tags to somehow convey the language of the span, but this would have to be standardized to make multiple-language spell checking possible.

[*]I have some regex searches that i check the ebook... As a plugin can we have something to add them there and execute them simultaneously?

Python has very strong regular expression capability and so with minor effort you could write some code to perform any set of standard regular expressions you want on every html file. So you just need to find someone on MR to write that plugin for you or design it yourself. That is a perfect thing for a plugin to do.

Take care,

KevinH

[Doitsu]

Originally Posted by gipsy

1. I have some regex searches that i check the ebook... As a plugin can we have something to add them there and execute them simultaneously?

That feature already exists: Saved Seaches.

[KevinH]

Hi,

Given that Paul, DiapDealer, tkeo and myself are the primary authors of KindleUnpack and maintain it, I really don't think you need to worry about whether there are multiple versions needed or not.

KevinH

Originally Posted by JSWolf

When you do the Python 3 interface, please make it run Python 2 code. Calibre doesn't do Python 3 and I don't want there to have to be two versions of the KindleUnpack code that has to be maintained due to Python version differences.

[KevinH]

Hi Doitsu,

I did not know Sigil could do that.

Wow! You really do learn something new every day!
Thanks,

KevinH

Originally Posted by Doitsu

That feature already exists: Saved Seaches.

[gipsy]

Originally Posted by Doitsu

That feature already exists: Saved Seaches.

I know. I have them there.
I was asking for a plugin to run all of them at the same time, and not load every one of them.

[DiapDealer]

Originally Posted by KevinH

So unless KindleImport.py and mobi_stuff.py actually need the from future unicode_literals stuff, can we remove them?

Yep. They don't have to be there. I've just gotten in the habit of trying to keep my scripts as close to Python3-ready as I can. I think there's a couple of tweaks to some of the struct hex-string stuff in mobiHeaderLight and SectionerLight, but that's no big deal to fix. I'll take a peek later tonight/early tomorrow.

[user_none]

Ubuntu packages courtesy of DiapDealer have been uploaded.

[DiapDealer]

Looks like John added my Linux binary packages to the available alpha downloads for those who want to play. Just use the link in his first post.

Packages are hard-coded to install to the /usr/local prefix and can be removed anytime using "sudo dpkg -r sigil".

EDIT: Oops cross posted.

Thanks for making those available, John.

[theducks]

Originally Posted by gipsy

I know. I have them there.
I was asking for a plugin to run all of them at the same time, and not load every one of them.

If you open saved searches:

Group them (indent) under a Group title:

Ex I have a series of (6) searches for the CSS to rename NAMED font sizes to my EM sizing standards


(assumes standard search box is open and set for mode and scope)
I highlight the Group title in Saved Searches: click load search.
click Replace All IN SAVED SEARCHES

The whole series executes 1 after another

[KevinH]

Hi DiapDelaer,

The more I think about this the more I think you should not need to remove unicode_literals from the plugin. You are right, keeping your code as close to Python 3 as possible is a good idea. The bug has to either be in my launcher code or the kindleunpack code someplace since those are the only places pathof() are used.

I've looked, and the first thing the bk.addotherfile does is run utf8_str() on the 'new.epub' passed in so that can't be it.

The first thing the launcher does is use utf8_argv to get utf-8 encoded versions of all command line arguments such as the module name, the outdir, ebook_root etc so that can't be it either.

And I checked and the first thing kindleunpack.py does in unpackBook is run utf8_str() on the infile and outdir paths.

So I simply can not see how any unicode at all makes it past those utf8_str() calls and they should be the only interface points where full unicode can enter.

So please leave in the unicode literals import and I will try to track down where exactly the unicode path pieces are getting past the screens that should allow it to work nicely with full unicode with no warnings whatsoever.

Thanks,

Kevin

Originally Posted by KevinH

Hi DiapDealer,

Once I removed the from future unicode-literals from both mobi_stuff.py and KindleImport.py all warnings went away. When we built up the path to the epub using the literal 'mobi8' we were forcing the entire path to the new epub to be full unicode.

So unless KindleImport.py and mobi_stuff.py actually need the from future unicode_literals stuff, can we remove them?

When I build the python 3 interface, then I will move the python interfaces to be full unicode. Right now much of the launcher/wrapper code depends on the utf-8 strings much like KindleUnpack does.


Thanks,

Kevin

[DiapDealer]

Hey Kevin,

I'm finding some weird behavior with the plugin that I can't really explain (at least on Linux). Suffice to say that I think the full Tk mainloop approach may be introducing some unwanted issues. Once I select a file to process, I simply cannot programmatically stop the plugin (close/destroy the root window) based on certain criteria. It tries to unpack anything I feed it--even though I specifically tell it to jump to the quitApp function if a KF8 book isn't detected (and it does go to the quitApp function--it just also continues to try to unpack). Simply put: the only thing that seems to close the plugin's gui programmatically (once it gets to the doit function) is complete success. Puzzling.