Problems using sigil_bs4 and gumbo_bs4.parse

[slowsmile]

For DiapDealers benefit, here is the plugin.py code for my plugin. This file is by no means complete(I have yet to add the update plugin section and osx and linux capability and it needs a tidy up):

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import unicode_literals, division, absolute_import, print_function

import globals
import convert
from cutils2 import validateHTMLFile

import os, os.path, sys, codecs, platform
from time import sleep

import tkinter as tk
import  tkinter.filedialog as fdbox
from dialogs import MetadataDialog
    
# main routine
def run(bk):
    print('Python Version:', platform.python_version(), '\n')  

    # initialize variables
    google_html_file = None
    prefs = bk.getPrefs()
    
    # get update check prefs
    if 'update_check' in prefs:
        update_check = prefs['update_check']
    else:
        update_check = True
        prefs['update_check'] = True
        
    # initialize file open default dirs    
    if 'initialdir' in prefs:
        globals.HTML_OPTIONS['initialdir'] = prefs['initialdir']
        globals.COVER_OPTIONS['initialdir'] = prefs['initialdir']    
    else:
        prefs['initialdir'] = os.path.expanduser('~')
        bk.savePrefs(prefs)        
             
    # set default output to epub 2
    output_format = 'epub 2.0'
    prefs['output_format'] = output_format
    
    # get the file locations, ebook
    # metadata and validate user input
    root = tk.Tk()
    root.title('Edit Metadata')
    root.resizable(width=False, height=False)
    MetadataDialog(root).pack(fill=tk.BOTH, expand=True)
    root.mainloop()
    
    # if dialog 'Cancel' or exit is pressed then exit run()
    if globals.SYS_EXIT== True:
       return(0)
    
    # get the input html file name
    html_file_path = globals.META_OPTIONS['filename'] 
    
    # save the dir path for next time
    prefs['initialdir'] = os.path.split(html_file_path)[0]
    bk.savePrefs(prefs)
    
    # check and validate the html file
    status = validateHTMLFile(html_file_path)
    if status != 0:   
        print(' >>> The HTML file FAILED validation checks.')
        return(0)
    
    # process the html file
    if html_file_path:
        
        # run the html to epub conversion
        epub_path = convert.convert2Epub(html_file_path)

        # if conversion fails, exit run()
        if globals.SYS_EXIT == True:
            return(0)                
        
        # import the epub into Sigil
        if os.path.isfile(epub_path):
            with codecs.open(epub_path,'rb')as fp:
                data = fp.read()
            bk.addotherfile('dummy.epub', data)
            os.remove(epub_path)
            return(0)
    else:
        print(' >>> Unable to process html file due to critical conversion errors.')
        return(0)
        

def main():
    print('I reached main when I should not have\n')
    return -1

if __name__ == "__main__":
    sys.exit(main())

[KevinH]

Also, one other thought ... could your larger code have a library or some code someplace that imports from bs4 directly itself. If so that would cause a name clash and explain why only an external bs4 works.

KevinH

[slowsmile]

@KevinH...No, this plugin is my first public python program of any note. I've also been developing a larger windows app in python that converts HTML docs derived from Word, AbiWord, Google Docs etc to epub. These app libraries are in entirely different locations to the plugin app so there should be no problems. I'm an ancient C/C++ developer(now retired) so I know how to be careful with version control. I also have only one python version installed on my computer -- python 3.4. There are no other virtual python installs on my laptop.

[slowsmile]

@KevinH....I've just sent the complete plugin as a zip file to your private email.

[slowsmile]

@KevinH...I've also just noticed another instance of peculiar behaviour from the Sigil Plugin Manager. You can perhaps try this yourself.

Put this into your import code:

try:
    from sigil_bs4 import BeautifulSoup
    print('Running sigil_bs4...')
except:
    from bs4 import BeautifulSoup
    print('Running regular bs4...')

Then set the PM to bundled python and click OK.
When you run the plugin it will correctly show this message in the plugin window:

Running sigil_bs4...

But, most odd, if you then untick bundled python and set system python(to bs4) and press ok then, when you run the plugin, you get this message in the plugin window:

Running sigil_bs4...

The only way that I've found to run your own system python version is to untick bundled python and put just this in the import code:

from bs4 import BeautifulSoup
print('Running regular bs4...')

...With no reference at all to the sigil_bs4 version.
Then you will get the correct message:

Running regular bs4...

It seems that if you have the import try/except code shown at the top then no matter whether you set bundled python or not -- the PM will always select sigil_bs4.

Is this proper behaviour for the Plugin Manager?

[KevinH]

Yes, as all of the python launcher code including the sigil_bs4 module are added to the python path of whichever version you are using so that the plugin code (launcher.py, etc) will all run correctly.

So that "try" should always succeed with internal or external python when invoked from within Sigil.

I will take a look at what you e-mailed me to see if I can figure out what on earth is going on and get back to you.

[slowsmile]

@Kevin...Thanks for that.

[KevinH]

slowsmile,

While I wait for your updated sample .html file to use to test your actual plugin ...

As a secondary test, I took Doitsu's code and tweaked it to operate on every xhtml file in the epub and actually write it back out to Sigil.

So this is an edit plugin. I have attached it to this issue.

Please download and install it.
- Open an empty Sigil document
- Add an id="removeme" to any p tag in Section0001.xhtml
- Then invoke the "junk" edit plugin.

It will both print to the screen the modified xhtml without the id="removeme" on the p tag and will then update your copy in Sigil once you hit "Ok"

Please try this on your system with the bundled Python 3 interpreter enabled.

If this works, then there is no problem with sigil_bs4 when it comes to how you want to invoke it.

Please let us know the results of this test.

As for why it only works in an external Python, my guess is that your actual plugin has external requirements that are not being met in the internal version of python 3. For example, your actual plugin uses tidylib but does not include the actual shared library in the plugin. If it is needed, then it must be added so that the internal python interpreter can access it.

KevinH