Sigil-0.9.8 Released

[KevinH]

Good to know it works.

In case others are in a similar boat ... the instructions for using the testplugin are in the first post of its thread here on MR and I've copied them below for anyone else who is building their own on Linux (or Widnows or OS X)

1. Use the attached testplugin_v014.zip here

2. open Sigil.app to the normal nearly blank template epub it generates when opened. It is important to note that the plugin's tests rely on this nearly blank epub during its testing.

3. use Plugins->Manage Plugins menu and make sure the "Use Bundled Python" checkbox is checked on Mac OS X and Windows. On Linux please properly set the path to your system Python 3 interpreter.

4. use the "Add Plugin" button to navigate to and add testplugin.zip and then hit "Okay" to exit the Manage Plugins Dialog

5. use Plugins->Manage Plugins->Edit->testplugin to launch the plugin and hit the "Start" button to run it

6. check the plugin output window for your missing or broken plugin test results

[patrik]

I ran the testplugin after the built and it was fine, but it was an old one. Installed the latest and got success, but it says in the end "Failure - 1 Tests of Plugin Operations Failed"?

Spoiler:

Status: success


Verify sys.path settings manually
/usr/local/share/sigil/plugin_launchers/python
/usr/lib/python3.4
/usr/lib/python3.4/plat-x86_64-linux-gnu
/usr/lib/python3.4/lib-dynload
/usr/local/lib/python3.4/dist-packages
/usr/lib/python3/dist-packages
/home/patrik/.local/share/sigil-ebook/sigil/plugins/testplugin

Verifying proper Python packages are available
Checking GUI modules
Python Package: PyQt5 Missing
Python Package: tkinter (/usr/lib/python3.4/tkinter/__init__.py) Found
Python Package: PIL (/usr/lib/python3/dist-packages/PIL/__init__.py) Found
Python Package: cssselect (/usr/lib/python3/dist-packages/cssselect/__init__.py) Found
Python Package: cssutils (/usr/lib/python3/dist-packages/cssutils/__init__.py) Found
Python Package: html5lib (/usr/lib/python3/dist-packages/html5lib/__init__.py) Found
Python Package: lxml (/usr/lib/python3/dist-packages/lxml/__init__.py) Found
Python Package: regex (/usr/lib/python3/dist-packages/regex.py) Found
Python Package: chardet (/usr/lib/python3/dist-packages/chardet/__init__.py) Found
Python Package: six (/usr/lib/python3/dist-packages/six.py) Found

Verifying Sigil Python Libraries can be found/loaded
Sigil Python library: epub_utils Found
Sigil Python library: quickparser Found
Sigil Python library: compatibility_utils Found
Sigil Python library: sigil_bs4 Found

Verifying Hunspell Spell Checking
Hunspell en_US affix file and dictionary Found
Hunspell shared library Found
Hunspell spellchecking works True

Verifying Sigil Gumbo Library operation
Sigil Gumbo BS4 Adapter library Found
Sigil BS4/Gumbo Parser works True

Verifying selected basic plugin interface elements
bk.text_iter() works True
Listing manifested files...
Text/Section0001.xhtml
toc.ncx
bk.manifest_iter() works True
bk.spine_iter() works True
bk.other_iter() works True
reading a non-manifest book file works True
reading a specific OPF manifest id works True
writing to a specific OPF manifest id works True
Launcher Version 20170227
ePub Version 2.0
adding your own file to the manifest works True
basename to id mapping works True
deleting our added file works True


Failure - 1 Tests of Plugin Operations Failed

[KevinH]

The error message is here:

Python Package: PyQt5 Missing

Your python installation is missing the PyQt5 package that was recented added to Sigil with this new release.

[Doitsu]

@Patrik: You'll most likely have to install python3-pyqt5 with your package manager.

[patrik]

Originally Posted by Doitsu

@Patrik: You'll most likely have to install python3-pyqt5 with your package manager.

Yes, that was it. It seems they sneek it one more python-thing for me to install with every release to keep me on my toes...

sudo apt-get install python3-pyqt5
fixed it.

Thanks

[DiapDealer]

Actually, I think this is the first new python module we've added since we started bundling Python in the Mac/Windows builds.

The difference is that the test plugin didn't check for the presence of any of the gui modules (tkinter, pyqt5) before this latest version.

[roger64]

Originally Posted by Doitsu

An easier test might be to check for the presence of the new Main Menu Icon Size option. (Edit > Preferences > Appearance > Menus)

Thanks all for your replies and sorry for being late, I am in the wrong time-zone.

Seems I have the 0.9.8. (see screenshot) with the wrong 0.9.7. tag. He, this is a collector's edition!!


edit: also, test-plugin 0.14 just installed and running fine.

[odamizu]

Sadly, 0.9.8 will not launch on my Mac running OS 10.11.6.

I get "Sigil quit unexpectedly." I have Tcl 8.6.4.1 installed and 0.9.7 works fine.

I am not the most knowledgeable about these things, and the problem could be on my end, but if there's anything I can do to help identify the problem, please let me know.

Muchas gracias

[KevinH]

Please make sure you are using Sigil version 0.9.8 and not Sigil version 0.9.7. Once Sigil version 0.9.8 is launched, even once, it will change the sigil.ini file preventing an easy return to Sigil version 0.9.7.

If you are sure that you are trying with Sigil version 0.9.8 then rename your Sigil preferences file:

~/Library/Application Support/sigil-ebook/sigil/sigil.ini

to sigil.ini.old (or anything else really) and they try to start Sigil.

If that works, we know that one of the settings in the old sigil.ini was corrupted. It most likely is a temp path that exists but can not be written to. We can narrow it down further if renaming the sigil.inifile allows you to launch Sigil.

Please let us know what you find out.

[KevinH]

The official build will only work on OSX 12 (Sierra) by mistake.

Just got access to a OSX 10.11.6 and it SystemB library is missing a symbol getentropy that should not be needed by my official build since it was configured for OSX 10.9 and later.

So give me a day or two and I will rebuild the Mac release to actually not use that missing symbol (it is used in Python). The problem is that the Python 3.5 configure switch is NOT MACOSX_DEPLOYMENT_TARGET aware and so decides to use features that do not exist in 10.9, 10.10, and 10.11 that do exist in 10.12 (which is what I use to build the release).

So stay tuned for a new Mac Package that will work all the way back to 10.9

Sorry about that.

KevinH

[odamizu]

Cool! Glad it wasn't me Absolutely no rush.

Totally appreciate all the work you guys do to offer such marvelous software free of charge. I still think you should enable that donate button. I'd be happy to contribute

[KevinH]

Just so you know, Apple really messed up on this ... their command line tools add a /usr/include/sys/random.h that defines getentropy but it is not protected by any of the MacOSX availability macros. So any header based test for existence of getentropy will pass in "configure" even though it did not exist prior to OSX 10.12 (Sierra)

Then to top it off, Apple in their infinite stupidity has defined some weak symbols in Mac OSX 10.11 that include the following:

'$ld$weak$os10.11$_basename_r', 
'$ld$weak$os10.11$_clock_getres', 
'$ld$weak$os10.11$_clock_gettime', 
'$ld$weak$os10.11$_clock_settime', 
'$ld$weak$os10.11$_dirname_r', 
'$ld$weak$os10.11$_getentropy', 
'$ld$weak$os10.11$_mkostemp',
'$ld$weak$os10.11$_mkostemps',

Which means that any linking based test for availability of specific functions done by any "configure" tests will pass even though the damn function is not actually available until Mac OSX 10.12

Setting the MACOSX_DEPLOYMENT_TARGET=10.9 (and none of these exist in 10.9) and running Python configure will say that **all** of these will exist.

So if we use XCode 8 and build on Sierra, the resulting binary will ONLY work on Sierra if it uses any of these specific functions.

That really sucks...

So I have to modify the Python "configure"/"configure.ac" to hard code these to undefined to get something to build on 10.12 using XCode 8 and run on 10.9.

That is doable for Python but not doable for Qt given its huge configuration complexity. I just have to hope and pray that Qt does not make use of any of these new "symbols" during the build.

I should have kept an older Mac OSX 10.9 build machine around but I had faith the Apple would handle XCode properly to maintain the ability to build things backwards compatible.

Sad ...

KevinH

Originally Posted by KevinH

The official build will only work on OSX 12 (Sierra) by mistake.

Just got access to a OSX 10.11.6 and it SystemB library is missing a symbol getentropy that should not be needed by my official build since it was configured for OSX 10.9 and later.

So give me a day or two and I will rebuild the Mac release to actually not use that missing symbol (it is used in Python). The problem is that the Python 3.5 configure switch is NOT MACOSX_DEPLOYMENT_TARGET aware and so decides to use features that do not exist in 10.9, 10.10, and 10.11 that do exist in 10.12 (which is what I use to build the release).

So stay tuned for a new Mac Package that will work all the way back to 10.9

Sorry about that.

KevinH

[KevinH]

It gets worse - almost all configure scripts define a ac_fn_c_check_func macro that basically defines its own version with just the symbol name to check and then tries to link it. If it links then that function exists. If it does not link, then that function fails.

All of this seems to ignore the MACOSX_DEPLOYMENT_TARGET and even will ignore CFLAGS=-mmacosx-version-min=10.9

In addition the LDFAGS="-Wl,-no_weak_imports -Wl,-macosx_version_min,10.9: have no impact on the results of the configure tests.

So for any opensource projects that use "configure"/"configure.ac" to check for c functions being available using the AC_CHECK_FUNCS() will pass in all cases on OSX 10.12 (when that function exists there) no matter how hard you try to tell it you are building for "10.9" which does not support that symbol.

Actually compiling a real test case with the proper includes will at least detect some of this due to some Availability macros but that is not how c functions are checked for in configure.

So all of the following tests in the Python "configure" will pass when built on OSX 10.12 even if you have properly set the MACOSX_DEPLOYMENT_TARGET.

faccessat
fchmodat
fchownat
fdopendir
fstatat
getentropy
linkat
mkdirat
openat
readlinkat
renameat
symlinkat
unlinkat
clock_getres
clock_gettime
basename_r
clock_settime
dirname_r
mkostemp
mkostemps
getattrlistat
getattrlistbuilk

And none of these functions/symbols exists in OSX 10.9!!!!!!!

And I am sure there are many more. So once a person upgrades to XCode 8 and OSX Sierra, they have lost all true backwards compatibility for any open source project that uses one of these symbols/functions.

This truly sucks.

[notimp]

@devs - I have an inside baseball question - but the GUI plugin api has tickled my interest - and I'm trying to gage if it is worth an effort on my part to try something. There is an "ignorant" part of that question, that could be solved by me installing the current build and finding out - but the followup and clarification - not so much, so I'm asking right away.

I've written several regex based "methods" to automate html parsing for htmls/epubs from certain OCR sources - that rely on Sigil having implemented something like tidy (a checker that auto propagates/closes tags that were removed by certain replace all's).

At the beginning of last year, tidy (?) was cut from Sigil with an engine change I believe, and it wasn't "returned" in the immediate months after.

Two questions -

Is html tidy (or something that would serve that function) integrated in current builds? (Not talking about flightcrew - but actual tidy functionality ("fix missing tags on the fly").

If not - would you consider implementing it again, just on the basis of one guy saying that it would be great to have that functionality back.
-

Here is my rational for it - I can't solve my issue with minimal matching - because I need all the complexity in the regex to catch probable formating patterns, and they are complex ones, where minimal matching would catch too early.

I also can't compensate for it - because I don't necessarily know where a closing tag that gets replaced was a "needed" one. So I need some sort of integrity parsing after the replacements are done.

Thank you for the response, and I guess kind consideration.

[KevinH]

You can always compile and include your own tidy html5 library inside your plugin (one plugin author already does this) or you can use Sigil's internal gumbo library. Sigil's gumbo parser is a fork of Google's Gumbo parser that autocorrects using the exact same rules as browsers. Sigil gumbo library is available through the plugin api as well.

FWIW, You also might be better off parsing the file in gumbo first and then doing replacements of the text parts so that you do not break the xhtml syntax by funny search and replaces.

I think there is a simple example of using Sigil's gumbo in the testme plugin that is documented in the latest Plugin framework Developers guide.

Originally Posted by notimp

@devs - I have an inside baseball question - but the GUI plugin api has tickled my interest - and I'm trying to gage if it is worth an effort on my part to try something. There is an "ignorant" part of that question, that could be solved by me installing the current build and finding out - but the followup and clarification - not so much, so I'm asking right away.

I've written several regex based "methods" to automate html parsing for htmls/epubs from certain OCR sources - that rely on Sigil having implemented something like tidy (a checker that auto propagates/closes tags that were removed by certain replace all's).

At the beginning of last year, tidy (?) was cut from Sigil with an engine change I believe, and it wasn't "returned" in the immediate months after.

Two questions -

Is html tidy (or something that would serve that function) integrated in current builds? (Not talking about flightcrew - but actual tidy functionality ("fix missing tags on the fly").

If not - would you consider implementing it again, just on the basis of one guy saying that it would be great to have that functionality back.
-

Here is my rational for it - I can't solve my issue with minimal matching - because I need all the complexity in the regex to catch probable formating patterns, and they are complex ones, where minimal matching would catch too early.

I also can't compensate for it - because I don't necessarily know where a closing tag that gets replaced was a "needed" one. So I need some sort of integrity parsing after the replacements are done.

Thank you for the response, and I guess kind consideration.