Executing calibre searches from a browser extension

[kiwidude]

A user (Comfy.n) asked about performing calibre searches direct from a browser in the Plugin ideas thread. Being the curious sad person I am I started dabbling with writing a firefox extension to do this using that nice calibre URL scheme stuff:
https://manual.calibre-ebook.com/url_scheme.html

I have something working locally (sticking plasters and bandaids - don't ask me for a copy of it yet). However I don't know if there is a "less clunky" approach possible as I have no idea how I could nicely even package this all up.

The issue is that it seems the only way (or at least the only way I have come across) to actually send that calibre:// url to the "outside" from a browser on your machine is to use browser.runtime.sendNativeMessage(). And that seems to require...
- some running application which will accept/respond using stdin/out the message sending from the browser. e.g. a python script or node.js script. This can then do the actual process launch with the calibre:// url
- a json file manifest defining stuff including the path the above python script/batch to launch it
- registry entries pointing to the json file

https://developer.mozilla.org/en-US/...dNativeMessage

Just curious if any of the fellow geeks in here have dabbled with writing extensions and have any thoughts/alternatives I may be missing?

The fact that I have something running I can use is a partial win for me, but trying to decide whether I bother to tell anyone else it exists and implement the endless list of stuff that would invariably lead to (multi-browser support, multi-os, keyboard shortcuts, toolbar buttons, configuration options, translations etc).

[kovidgoyal]

Well one way to do it would be to add support for it to the calibre content server. Add a URL endpoint that passes calibre:// URLs to calibre when the content server is running from within calibre itself. Your browser extension then just needs to connect to this endpoint, the only tricky bit being how it finds out what ip address/port the server is listening on, you can always punt to having the user tell you that.

Also, exposing the calibre url scheme to the net may not be the safest so best to only pass it on for connections from localhost.

[kiwidude]

Hi Kovid,

Thanks for the calibre content server suggestion.

Pros:
- avoids the registry keys/json stuff
- avoids additional download/install on top of the extension itself

Cons:
- would have to add the endpoint to calibre codebase. I personally have no idea where to start beyond there being an srv folder
- I don't think there is a way to do any kind of background web request with the extensions JavaScript API. So would have to launch another tab/window temporarily? Would need to test to see how annoying this could be.
EDIT: Actually after some investigation I think I could in fact use the fetch() function in javascript to invoke the url endpoint in calibre, so not an issue.
- user must have calibre content server running (I know this can be as easy as a checkbox to run at startup but I don't normally run it myself).

At the very least I can see it being a nice option to offer in the extension as an alternative to a user downloading/installing another app bundle.

[kiwidude]

Good news - the dev work and documentation for the extensions seems to be (mostly) done.

Before the wall of text begins and your eyes glaze over, I would like two favours from the community:

1. Can some of you kind folks try installing below (Windows users) and offer feedback. Ideally on both functionality and documentation?

2. Are there any icon gurus wanting to contribute something for this? I picked something random off a free site, but something that looks more "calibre-ish" would be nice. I need 16,32,48,128 bit pngs and an svg for just one icon. Is there somewhere else I should ask if they won't see it here?

These should work on Chrome, Chromium, Edge, Firefox, Waterfox G5, Waterfox Classic (tested all of those) and allegedly Thunderbird (untested).

When I have some feedback I will submit to the various marketplaces and that way most of the pain of installation steps below will go away for you (except installing the host application).


First the documentation link which is here:
https://github.com/kiwidude68/calibr...bre-Search-FAQ

That does not however include instructions on how to install the extension itself when not in the marketplace - here are a few pointers of what you need to do:

1. Download the relevent repository as a zip or clone it:
https://github.com/kiwidude68/calibre-search-chrome
https://github.com/kiwidude68/calibre-search-firefox

2. Extract to a folder

3. For Chrome users:
In your address bar type: chrome://extensions/
Turn on Developer mode in the top right
Click on "Load unpacked" in the top left
Navigate to the calibre-search-chrome/addon folder
Click on "Select Folder"
Take a copy of the ID: value, you will need this later.

For Firefox users:
In your address bar type: about:debugging#/runtime/this-firefox
Click on "Load Temporary Add-on"
Navigate to the calibre-search-firefox/addon folder
Double click on the manifest.json file

4. Chrome users need to pin the Calibre Search button to the extensions bar if they want it visible. For Firefox it should just appear.

5. At this point you should just go to the Options page for the extension and follow the documentation in either the Getting Started link within that options page or the FAQ link above. So get the host application installed and your calibre library name set.



6. Chrome users Because I haven't registered this in the marketplace yet your machine will give the extension a temporary id. You need to copy that id into the following file after you have installed the calibre-search-app above using install-host.bat:

%localappdata%\com.kiwidude.calibre_search\manifest-chrome.json

The last line in the file has a "chrome-extension://XXX/" value, replace XXX with your ID from the //chrome:extensions page above.

Now you should be able to click on Test connection in the options page and see it go green...

Finally you can now try searching - as shown here I could either select a mix of text and hyperlink, just text or just the hyperlink (the last case will do a search for the text displayed on the hyperlink, e.g. "Wilbur Smith").

So by default this will do a search in calibre of:

PHP Code:

Titans of War by Wilbur Smith 


But with a simple replace expression of either "by" or "/\bby\b/gi" configured the calibre search becomes:

Titans of War Wilbur Smith

Feedback, thoughts, suggestions welcomed... it was a crazy amount of work and re-work to get this working on all those browsers with all the mess out of there of Manifest V2/V3 and cross-browser stuff... ugh.

[Comfy.n]

Originally Posted by kiwidude

Good news - the dev work and documentation for the extensions seems to be (mostly) done.
[...]

Feedback, thoughts, suggestions welcomed... it was a crazy amount of work and re-work to get this working on all those browsers with all the mess out of there of Manifest V2/V3 and cross-browser stuff... ugh.

It works!

[FlyGoneCrazy]

Hey! Is this extension still working? After a lot of struggle I've managed to connect to host on macos. However when I'm trying to search for books, nothing is happening...

I thought that it might be related to CALIBRE_OVERRIDE_DATABASE_PATH=. Therefore created new local library and attached extension to it. However, nothing has changed.

Let me know if it's still working. Thank you.

[kiwidude]

Originally Posted by FlyGoneCrazy

Hey! Is this extension still working? After a lot of struggle I've managed to connect to host on macos. However when I'm trying to search for books, nothing is happening...

I thought that it might be related to CALIBRE_OVERRIDE_DATABASE_PATH=. Therefore created new local library and attached extension to it. However, nothing has changed.

Let me know if it's still working. Thank you.

Sorry just saw this post. If you are having a problem with it the best way to ensure I see it is to raise an issue on the repo where you download it from.

I know for sure that on Windows my Firefox extension works, I use it every week. The Chrome extension I haven't used in a while so it is possible that might have got broken along the way. As for other operating systems I have no ability to test them/support them...