testplugin_v019.zip
 

Hi All,

To help Linux packagers and others who build Sigil from source, we have created a simple testplugin_v019.zip that will test the available Python 3.X interpreter environment setup.

1. Use the attached testplugin_v019.zip here

2. open Sigil.app to the normal nearly blank template epub it generates when opened. It is important to note that the plugin's tests rely on this nearly blank epub during its testing.

3. use Plugins->Manage Plugins menu and make sure the "Use Bundled Python" checkbox is checked on Mac OS X and Windows. On Linux please properly set the path to your system Python 3 interpreter.

4. use the "Add Plugin" button to navigate to and add testplugin.zip and then hit "Okay" to exit the Manage Plugins Dialog

5. use Plugins->Manage Plugins->Edit->testplugin to launch the plugin and hit the "Start" button to run it

6. check the plugin output window for your missing or broken plugin test results

It tests for the full set of python 3 packages available to the Bundled Python as well as Hunspell SpellChecking, using the bs4 sigil gumbo adapter, Sigil's internal python libaries, and the many of the BookContainer basic interface functions.

I have attached it here in case anyone wants to play with it.

Windows users: please note that Sigil 0.9.0's bundled python 3 will fail the cssutils test (missing) when running this plugin. I don't think any plugins are currently using cssutils, but this problem wiil be fixed in the next Sigil release.

Mac OS X using Sigil-0.9.0 is in the same boat ....

The current Sigil-0.9.0 on Mac OS X will fail for cssutils, and for some things done in PIL. Both of these are fixed for our next release.

So creating the test plugin actually helped us track down and fix our own issues before a plugin developer ran into them!

KevinH

Cool, sounds great!

And I've verified my AUR package passes the plugin tests with flying colors.


A couple requests:

• Can you summarize the failed tests -- if any -- at the end? Just to make it easier to read. :)
• Would it be possible to report which components are part of the core functionality, and which are recommended but not crucial? I am thinking of some of the python packages, which plugins may expect but Sigil itself does not use.
  Reason I ask is, because I list those as optdepends ("python-whatever: recommended for plugins"), and debian can list them as suggests or recommends.
• You documented the plugin in docs/Bundling_Python3_With_Sigil_on_MacOSX.txt but not docs/BuildingOnLinux.md
  It isn't very helpful to linux packagers if they don't know it exists :o so I think it would be nice if you made it more likely they will see it.

Hi eschwartz,

Nicely done.

The output itself is only 1 page long in total and it tells you everything. So a quick visual inspection will tell you anything you want.

I can split out the extra Python external packages such as cssutils, cssselect, PIL, html5lib, six, into their own reporting section. That said if you or other linux distributors treat them differently from other python 3 packages, some future plugin may fail generating an unneeded bug report and more inconsistencies across platforms. I would prefer to keep all versions of Sigil with the exact same capabilties just to be safe and consistent and to make support easier.

By the way, without lxml, Sigil will not even start up, making testing the plugins impossible.

I only just added it to the Mac OSX building docs before I posted it here. The Linux docs will get a similar treatment but there are only so many hours of volunteer time this weekend. Before we release Sigil-0.9.1, we'll make that change.

Thanks,
KevinH

Quick visual inspection that requires you to fully read each line.
Maybe instead you could colorize it (red warning) or print "**** FAILED ****" if the plugin runner doesn't support color (probably not...)

It isn't that big a deal, it's just that currently it doesn't stand out very well.


As for optional components, linux distributors like splitting out optdepends, and expect users to know what they're doing.
apt-based systems can use Recommends, which I believe is installed by default -- an advanced setting lets you only install the minimal packages necessary, and advanced users are expected to know what they are doing later on down the line.
ArchLinux is a "user-centric, not user-friendly" distro, and we are expected to make note of the "optional dependencies for ____" list in pacman's output. :D

Perhaps, like the Failed checks, you can simply add a Recommended status but still group them as you do now. I'm just floating ideas here.

Again, it's your choice, but packagers may appreciate the information since it helps them figure out how to write the dependency list based on their preferred guidelines.

Thanks. Just wanted to make sure you were aware. :)

I don't think any of the python modules should truly be considered "optional." Sure, Sigil could run without them, but as Kevin mentioned, those are the modules that are being packaged with Sigil (and eventually, all three platforms will have a package). Plugin developers will not be required to "declare" any of those modules as needing to be installed to use their plugin. It will be assumed that any Sigil installation will have access to all of those. At the very least, they should be considered "Most Strenuously Recommended If You Want Help Figuring Out Why A Particular Downloaded Plugin Isn't Working for You."

I can guarantee my first trouble-shooting step will be to require that the Test Plugin passes every single test before proceeding. ;)

Couldn't agree more. Why on earth would maintainers want to have the same release have different versions with different capabilities flying around. If you want to call it "Sigil", it should be Sigil and have the exact same capabilities as Sigil on all other platforms. Any other way makes for a support nightmare.

So Linux distributors liking to use "optional" or "recommended" packages will simply result in crippled versions of applications existing in the wild causing problems.

KevinH

If plugins are optional, then the modules that are only needed for plugins, are optional as well.

The opinion of your average distro packaging standards, is that plugins which aren't necessary for core functionality, are optional extras.


That's why its nice to list them separately, and if a distro prioritizes user-friendliness, they can default to installing them, and if a distro prioritizes giving users the tools to make unusually informed decisions, they can do that too.


And for a standalone binary download, it makes sense to include them as a matter of course.

...

More information makes it more likely that packagers include those modules in any shape or form, rather than having some maintainer somewhere say "huh, this isn't doing anything, let's ax it". :whistle:

How's this then: we don't consider any of the python modules optional for core functionality. So our list IS prioritized. Distros are, of course, free to prioritize what they consider optional (which they're going to do regardless of what we say anyway). Which is ultimately going to result in us getting a metric crap-ton of support requests/bug reports because people are installing versions of Sigil with crippled/incomplete plugin frameworks. Call me crazy, but I don't really feel compelled to supply the bullets that are going to be used to shoot us with later.

Updated the first post to testplugin_v012.zip to work with the forthcoming Sigil 0.9.4 release that includes a Nav by default under epub3.

KevinH

Would it be useful to have the testplugin check for python3-tk?

I just ran a couple of plugins for the first time, and they all failed due to not finding the module _tkinter.
Which was easily solved by installing python3-tk.

The plugins are optional, and the testplugin checks the basic installation, but maybe it could save some time for some users?

The linux build instructions specifically mention python3-tk as one of the minimum requirements in the "Getting Python 3.4" section. But we'll certainly take it under advisement.

Test plugin failed (error log included)
 

I tried the plugin test and it failed. Can anyone help me interpret the errors? This was on an Ubuntu computer:

Status: failed

Verify sys.path settings manually
/usr/local/share/sigil/plugin_launchers/python
/usr/lib/python3.4
/usr/lib/python3.4/plat-x86_64-linux-gnu
/usr/lib/python3.4/lib-dynload
/usr/local/lib/python3.4/dist-packages
/usr/lib/python3/dist-packages
/home/kingmidas/.local/share/sigil-ebook/sigil/plugins/testplugin

Verifying proper Python packages are available
Python Package: PIL Found
Python Package: cssselect Found
Python Package: cssutils Found
Python Package: html5lib Found
Python Package: lxml Found
Python Package: regex Found
Python Package: chardet Found
Python Package: six Found

Verifying Sigil Python Libraries can be found/loaded
Sigil Python library: epub_utils Found
Sigil Python library: quickparser Found
Sigil Python library: compatibility_utils Found
Sigil Python library: sigil_bs4 Found

Verifying Hunspell Spell Checking
Hunspell en_US affix file and dictionary Missing
Hunspell shared library Missing
Hunspell spellchecking works False

Verifying Sigil Gumbo Library operation
Sigil Gumbo BS4 Adapter library Traceback (most recent call last):
File "/usr/local/share/sigil/plugin_launchers/python/launcher.py", line 135, in launch
self.exitcode = target_script.run(container)
File "/home/kingmidas/.local/share/sigil-ebook/sigil/plugins/testplugin/plugin.py", line 182, in run
import sigil_gumbo_bs4_adapter as gumbo_bs4
File "/usr/local/share/sigil/plugin_launchers/python/sigil_gumbo_bs4_adapter.py", line 35, in <module>
import sigil_gumboc as gumboc
File "/usr/local/share/sigil/plugin_launchers/python/sigil_gumboc.py", line 161, in <module>
SourcePosition.EMPTY = SourcePosition.in_dll(_dll, 'kGumboEmptySourcePosition')
ValueError: /usr/bin/python3: undefined symbol: kGumboEmptySourcePosition
Error: /usr/bin/python3: undefined symbol: kGumboEmptySourcePosition

Not sure what the deal is with the gumboc error, but the hunspell error implies you a) don't have a recognized en_US hunspell dictionary, and b) do not have libhunspell.so which is supposed to be a stable symlink to libhunspell-1.3.so.${sover}

@darkbreath: would probably have to see what cmake options you used to build Sigil (I'm assuming it's at least been built from source since the 0.9.5 tag). Are you overriding any settings with environment variables when launching Sigil? Unless you specifed the -DUSE_SYSTEM_LIBS=1 option with cmake, Sigil should have built hunspell from its own sources (and would automatically be available unless you modified the files after installation).

The gumboc error is quite baffling. Does Sigil function normally otherwise? Is the spellchecker working to highlight misspelled words in Code View?

Hi there. Maybe some background information will help. I just want to say I am not a programmer and therefore had significant difficulty compiling Sigil from source, which I unfortunately had to do with Linux. To be quite honest, I was surprised Sigil compiled at all. I would appreciate any help correcting compilation mistakes I may have made.

I used this page as my general guide: https://github.com/Sigil-Ebook/Sigil...dingOnLinux.md

I managed to get through every section except "Advanced Stuff."

However, I deviated on several occasions when I had to:

1. When installing Pillow, I got some error. I googled the error, and found a page that suggested these commands:

sudo apt-get build-dep python-imaging
sudo apt-get install libjpeg8 libjpeg62-dev libfreetype6 libfreetype6-dev

I installed those dependencies and it worked again.

2. I'm on Ubuntu 14.04. The version of cmake in official repositories is only about 2.8. I compiled and installed the latest version of cmake myself from cmake.org. I believe I have version 3.52. I followed cmake's readme instructions.

3. I was confused as to which python packages to install (3 or 3.4), so I installed BOTH when both were options. I am uncertain if this led to any problems. The packages this could have applied to are as follows:

python3
python3-dev
libpython3
libpython3-dev
python3-pip
python3-tk
python3-lxml
python3-six

I surmise this may have led to the wrong version of python being used during the plugin test?

4. I see -DUSE_SYSTEM_LIBS under "Advanced Stuff" in the Github guide I followed. If this is what you meant, I didn't touch it; I was only at the "Testing Sigil's Python plugin framework" section when I discovered those errors and posted for help.

Unfortunately I haven't tried the spell checker; I didn't think Sigil would work properly unless the errors were resolved.

As for the cmake command, the one I used was similar to the one in the guide but corrected for location of Qt:

cmake -G "Unix Makefiles" -DCMAKE_PREFIX_PATH=/opt/Qt5.4.2/5.4/gcc_64/lib/cmake -DCMAKE_BUILD_TYPE=Release ../sigil-src

I'm not sure if this made a difference, but I used Qt 5.5.1 (64-bit version).

Please help! I'm in over my head. If someone could just make a ppa for Linux it'd be great. Having to compile from source was like climbing a mountain with both feet tied together. Painful. But since I've gotten this far and Sigil is important for my work, I'll do what I have to do to fix it.

@darkbreath: Since the error was most likely caused by the hunspell library, have you tried to install the hunspell package and a hunspell dictionary, e.g. hunspell-en-us?
Also install the FlightCrew plugin (or any other plugin written by KevinH/DiapDealer) and test it.
If it works fine, you can most likely ignore the testplugin error messages unless you plan to use a spellcheck plugin.

Well, building software on Ubuntu is fun. :rolleyes:

I don't really know what packages etc. you need for that. I do know that Sigil is available in the Debian unstable repositories (but not yet in the current Debian release).
Give it a few years and Ubuntu might ship it too. :rofl:

Arch Linux has had it since forever. And I maintain the sigil-git package for the Arch User Repository.

...

I've looked once or twice at how to package a Debian/Ubuntu package (for calibre), but couldn't really figure it out. :o
Maybe I could copy the version in Debian and turn it into a PPA. I hear they've got git dailybuild recipes now, and everything...

If you're not trying to use system libs, then no hunspell packages should need to be manually installed. They're included with Sigil (including the necessary dictionaries).

What's the gcc version on 14.04? Anything short of gcc 4.8 is probably asking for trouble with Sigil's C++11 requirements.

In addition to that--lets start simple. If the plugin framework can't find/use the hunspell library/dictionaries, lets make sure they're where they're supposed to be. It looks like you went with the default install prefix of /usr/local, so the libhunspell.so shared library should be located in /usr/local/lib/sigil directory (along with the libsigilgumbo.so file and the sigil binary).

The hunspell dictionaries should be in the /usr/local/share/sigil/hunspell_dictionaries directory (a bunch of .dic and .aff files).

Sorry for the late reply. I'm happy to report that the problem was caused by a mistake in the way I ran Sigil. I made a novice mistake and tried to run Sigil right from the directory in which it was built. Running "sigil" (the sigil in the path variable) from Terminal or running Sigil using the shortcut (I didn't know one was created) fixed the problem completely. The problem was not related to language packages in any way.

One thing I am uncertain of is whether the Sigil shortcut is visible until the computer is started. The shortcuts of some newly installed programs can't be found via the dash until a restart, and I think Sigil might be one of these. It's not a big deal but if this is indeed the case, and if it's easy to fix, it might be helpful to new users to have the shortcut show up right away before a restart is done.

Thanks.

I've no idea if there's any way to force menus on various desktops environments to refresh after a new program is installed or not. But I'm pretty sure there's no way for me to make it happen through the "make, make install" process. All I can do is put the Sigil desktop file where the desktop environment can make use of it. The DE has to do the rest.

I don't typically have to restart before the shortcuts appear in the DE's menus. But I might have to log out and back in. There's a db of some kind that needs to update/refresh. I have no idea if it would show up after certain period of time has elapsed or not.

It is also dependent on the DE. ;)

But yes, a logout/login should definitely cover any DE out there.

xdg-desktop-menu forceupdate

Thanks. I can at least tell users they can run this command after building/installing Sigil if Sigil has not yet appeared in their DE's menu. Provided they have xdg-utils installed, of course.