ScrambleBook and Calibre 4

[jackie_w]

Originally Posted by eschwartz

Perfect! So I'd advise that, then. e.g. let the plugin be able to run with calibre-debug --run-plugin entry points, accepting the path of the ebook to scramble. Make the GUI element simply figure out the path to the book to be scrambled, and use run_calibre_debug to spawn a new instance of the plugin which is set up to do webengine stuff.

You could be onto something here. The standalone version is just a single scrambleebook.py file (exact same file as in the plugin) plus a .bat containing

set ProgDir=%~dp0
calibre-debug "%ProgDir%scrambleebook.py" %1

I feel I may already have all the right pieces of code ... but not necessarily in the right order, or place, as yet

[kovidgoyal]

@jackie_w: Yes I think @escwartz has a good idea. Have the GUi part of your plugin simply copy the ebook file to a temp dir. Then launch the standalone part in a separate process using calibre-debug which will allow you to use QWebEngine with no issues on that copied file.

[jackie_w]

Originally Posted by kovidgoyal

@jackie_w: Yes I think @escwartz has a good idea. Have the GUi part of your plugin simply copy the ebook file to a temp dir. Then launch the standalone part in a separate process using calibre-debug which will allow you to use QWebEngine with no issues on that copied file.

More help, pretty please ...

I had to abandon the above plan as I forgot to mention that the plugin needs a few small dummy images to scramble the book's original images. In the standalone these are in an images subdir with the .py and .bat files. In the plugin they're stored like any other plugin's images. I had no idea whether it would be possible to pass the images.

What I'd hoped to try instead was to pass the running of the whole Preview 3-column child QDialog to run_calibre_debug activated by the Preview button in the main plugin QDialog. The child dialog will (eventually) need the QWebEngineViews but doesn't need any of the plugin's images until they've already been written to disk in the scrambled container.

Rather than get complicated too soon I've tried to strip everything back to bare bones to see whether you can tell me where I'm going wrong.

For now, all I want to do is use run_calibre_debug to pass a single string to viewer.py which opens a single QLabel to display that text. If I ever achieve proof of concept hopefully I can start to make some progress.

This is the code in scrambleebook.py when the Preview button is clicked. I've left the original 2-line contents as comments in case you needed to know what used to be there:

    def previewbutton_clicked(self):    
        #dlg = EbookScramblePreviewDlg(self.ebook, self.eborig, self.is_scrambled, self.rename_file_map, parent=self.gui)
        #dlg.exec_()
        from calibre.debug import run_calibre_debug
        
        msg = 'preview run via calibre-debug'
        print('%s - Start' % msg)
        run_calibre_debug(['-c', 'from calibre_plugins.scrambleebook_plugin.viewer import main; import sys; main(sys.argv[-1]);', 'running OK'])
        print('%s - End' % msg)

viewer.py is this:

from __future__ import (unicode_literals, division, absolute_import, print_function)

def main(some_text):
    from PyQt5.Qt import QLabel
    from calibre.gui2 import Application    
    app = Application([])
    
    w = QLabel(some_text)
    w.setMinimumHeight(100)
    w.setMinimumWidth(100)
    
    w.show()
    w.raise_()
    app.exec_()

When I
- put the above in the plugin
- open calibre via calibre-debug
- run the plugin on a library epub
- press the Preview button

I see the 2 Start/End print statements in the debug log and no error messages but no sign of any QLabel widget. What stupid error have I made?

[kovidgoyal]

@jackie_w: some_text is not defined in viewer.py And if you need to pass in dummy images I would suggest you simply create a temporary directory, copy in the dummy images into that directory and pass the path to th edirectory

[jackie_w]

Originally Posted by kovidgoyal

@jackie_w: some_text is not defined in viewer.py

I don't understand? Am I having a brain fade?

def main(some_text):
    ...
   w = QLabel(some_text)
    ...

What are you suggesting I change?

Originally Posted by kovidgoyal

And if you need to pass in dummy images I would suggest you simply create a temporary directory, copy in the dummy images into that directory and pass the path to th edirectory

I don't want to even think about this until I see evidence of run_calibre_debug doing something with the trivial example I posted.

[kovidgoyal]

Oh sorry my mistake, I dont see anything wrong in the snippet you post. Post the full plugin and I will have a look.

[jackie_w]

Originally Posted by kovidgoyal

Oh sorry my mistake, I dont see anything wrong in the snippet you post. Post the full plugin and I will have a look.

Here it is.

[eschwartz]

@jackie_w, @kovidgoyal,

Am I missing something? Surely the run_calibre_debug process, which is executing code imported from the calibre plugin itself, likewise has access to all plugin resources, and especially the ability to access plugin utilities that extract an image file from the plugin.

The only thing the subprocess *won't* have is the ability to share internal state and memory, and thus, the ability to have a writable handle on the metadata.db.

It should be literally the same as running the plugin normally, but launched from a custom QApplication rather than calibre's.
In general, I don't think there is a reason that calibre-debug --run-plugin should be unable to import all Calibre code, consult the constant variable which says "here is where the user config dir is", or execute the same functions that are available to the GUI.

[jackie_w]

@eschwartz,

Much as I'd love to have a sensible discussion with you about this, I'm afraid it's above my head.

If you're able to tell me exactly what to change and where I'll be happy to test but I'm not going to be able to figure it out for myself based on hints.

[eschwartz]

I'm mobile right now, cannot check the source code and offer specific examples, but what I'm basically thinking is that you should be able to get the images by instantiating your plugin in the calibre-debug subprocess and using https://manual.calibre-ebook.com/plu...load_resources

Also see cli_main() there. This seems like a much simpler approach: You'd want to allow the cli_main entry point to spin up your QApplication with a path to an ebook file in order to run the plugin, just as if you had used the batch file instead.

In fact, this could replace the batch file. Users would simply install the plugin, and then the batch file would run 'calibre-debug --run-plugin ScrambleEbook path-to-epub'.

Then the final step would be that on newer versions of Calibre with webengine instead of webkit, the plugin will simply call run_calibre_debug and start up the command line interface for the plugin.

If this isn't enough of a hint for you, I will try to explain in greater detail and hopefully with code samples, tonight or tomorrow or something. Or Kovid can fill in the blanks.

[jackie_w]

Originally Posted by eschwartz

I'm mobile right now, cannot check the source code and offer specific examples, but what I'm basically thinking is that you should be able to get the images by instantiating your plugin in the calibre-debug subprocess and using https://manual.calibre-ebook.com/plu...load_resources

Also see cli_main() there. This seems like a much simpler approach: You'd want to allow the cli_main entry point to spin up your QApplication with a path to an ebook file in order to run the plugin, just as if you had used the batch file instead.

In fact, this could replace the batch file. Users would simply install the plugin, and then the batch file would run 'calibre-debug --run-plugin ScrambleEbook path-to-epub'.

Then the final step would be that on newer versions of Calibre with webengine instead of webkit, the plugin will simply call run_calibre_debug and start up the command line interface for the plugin.

If this isn't enough of a hint for you, I will try to explain in greater detail and hopefully with code samples, tonight or tomorrow or something. Or Kovid can fill in the blanks.

I can wait. I'm not so much in a hurry to complete the fix as I am to know whether it is fixable in a way I have a chance of understanding and supporting long-term.

I don't use cli_main() and have never planned to become an expert. It's not something I'll ever use anywhere else. I'm not against a one-line solution which uses it, though, if handed to me on a plate. I'm afraid your hints (while well-intentioned and gratefully received ) aren't enough.

Re: the standalone version. I don't really want a different solution to that as I don't think it's broken as-is.

[eschwartz]

So I've imported ScrambleEbook into a git repository here: https://github.com/eli-schwartz/ScrambleEbook

And this patch implements the CLI interface: https://github.com/eli-schwartz/Scra...775c440f515b85

Notice it's pretty simple to do. The heart of it is in __init__.py, where ScrambleEbookActionBase needs to be extended with a new method: cli_main(), which is responsible for initializing a new application and doing all that good stuff.

The thing that cli_main does is the same thing that running scrambleebook.py on its own needs to do: call a QApplication, set up the EbookScramble(QDialog), and exec the QApplication. Luckily, all that is already in a

if __name__ == '__main__'

, so I could simply shift the lines around, turn it into a main() function that is then called by if __name__ ..., and rewire it to pass sys.argv around sliiiiiightly different, so that I can pass my own array into main() if I really want to.

And then cli_main() runs calibre_plugins.scrambleebook_plugin.scrambleebook .main -- that's all.

[eschwartz]

Now that you can run:

calibre-debug --run-plugin ScrambleEbook ./path/to/mytroublesomebookthatneedshelp.epub

A couple things become possible:

• The standalone tool from https://www.mobileread.com/forums/sh...19#post3188419 could be simplified... it no longer needs to contain scrambleebook.py or that image folder, instead it can simply say "you must first install the plugin in calibre", then it can run the above command line.
  
• You can use:
  

  from calibre.debug import  run_calibre_debug
  
  run_calibre_debug('--run-plugin', 'ScrambleEbook', ebook_file)

  to call the plugin in another process.

Now that this works, once you can figure out how to run it using WebEngine as a standalone script, you can change the plugin to call itself... as a standalone script.

[kovidgoyal]

@jackie here is a demo plugin that shows you how to run webengine in a separate process. Note that it only works with calibre version >= 3.99.4 which will be released in an hour or so. No need to mess around with calibre-debug anymore.

When its button is clicked it asks for a URL and then opens that in a QwebEngineView

[jackie_w]

Thanks for this. I've tried your suggestions and have been able to run the plugin as standalone and from the calibre GUI - with a working QWebEngineView widget.

After thinking about it some more I can see that there is an advantage to the standalone not needing to have a local copy of scrambleebook.py and the images directory. A simple .bat should be OK

echo off
calibre-debug --run-plugin ScrambleEbook %1

A few questions though ...

In the plugin's uiaction.py to start the dialog it used to have

dlg = EbookScramble(path_to_ebook, book_id=book_id, from_calibre=True, parent=self.gui)
dlg.exec_()

and now I think you're suggesting it runs itself from itself?

from calibre.debug import run_calibre_debug
run_calibre_debug('--run-plugin', 'ScrambleEbook', path_to_ebook)

I'm not sure yet whether the book_id arg is still needed because the path_to_book has already been set at this point, but I do still want the from_calibre parameter set to True when run from calibre GUI. Where do I put this?

I thought this might work but it doesn't.

run_calibre_debug('--run-plugin', 'ScrambleEbook', path_to_ebook, 'True')