Feature Proposal: Gemini/LLM API Integration in E-book Viewer

[amirthfultehrani]

Originally Posted by DNSB

One other item to remember is that many of the people posting on MobileRead are well into the old curmudgeon stage of their existence. Even minor changes tend to meet with resistance. Recent example was Sigil going to a non-native file dialog when running under Windows which was already used for MacOS and Linux. Baby duckling syndrome is very real.

@DNSB, what an insightful response - thank you. Today I not only learned what "curmudgeon" means (bad-tempered) but also "baby duckling syndrome" (first experience with a UI being the preferred one) - I feel like you have hit why the poll results are the way they are right on the head.

My goal from the start has been to implement this feature in the most unobtrusive, "Calibre-like" way possible, precisely because I respect the ethos of the Calibre community's healthy skepticism toward change. We shall see in the ensuing posts/threads/days what shall be the end-case!

[amirthfultehrani]

Originally Posted by Quoth

Even if off by default, it's the thin end of a wedge that could import nonsense and eventually cost the user and environment.

People that don't know what LLM really is and costs will turn it on.

People can separately use LLM / generative AI if they want. No need to spoil the best ebook management system and set of tools.

Quoth, you have raised some very important points that were central to how I approached this feature, thank you for the reply. Let me address each point of yours directly:

• "Thin end of a wedge / Spoil the best ebook system": 100% agreed, and I hope you may see how my intention is not to be this "thin end" or to "spoil" anything. My proposed feature is to be completely inert/invisible unless a user explicitly enables it - in other words, 100% opt-in. If you don't go into the settings and provide your own API key, it adds 0 overhead, makes 0 network calls, and uses virtually 0 resources. It is fundamentally just another tool in the toolbox, like the existing "Lookup" feature.
• "Import nonsense": as it is, the suggested/proposed feature is strictly read-only. It never, under any circumstances, modifies the book's content. What it does at most is simply display the LLM's output in a separate, dockable side panel for the user to read, similar to how the current Lookup panel works - the book remains untouched.
• "Cost the user and environment": This is for sure a hot-button issue. I think a wider discussion on this would definitely hold value, but, for sake of this thread, and to be neutral + clear, I shall say the following two things about the feature's design (in regards to LLM call transparency): 1. The user must provide their own API key -> they are in full control of their own billing/usage - Calibre itself wouldn't be connected to any paid service 2. The UI includes a session-based API call counter so the user is always aware of how much they are using the feature (and this can be expanded as desired)
• "People can separately use LML": Absolutely, and this feature is meant to be for them! My goal is not to force AI on anyone. My goal is instead to streamline the workflow for users who already copy-paste text between the viewer and an external AI service. It is meant to be integration for convenience and not a fundamental change of purpose.

Thank you again for the valuable perspective, Quoth.

[kovidgoyal]

@amirthfultehrani I suggest you make a PR on github we can discuss the implementation further there. I am somewhat inclined to make this a tab in the lookup panel, because it is essentially doing the same function as lookup, just using a different type of lookup. That way no need for a new shortcut or a new button. calibre can simply remember which tab the user was on last when the panel is re-opened.

[amirthfultehrani]

Originally Posted by kovidgoyal

@amirthfultehrani I suggest you make a PR on github we can discuss the implementation further there. I am somewhat inclined to make this a tab in the lookup panel, because it is essentially doing the same function as lookup, just using a different type of lookup. That way no need for a new shortcut or a new button. calibre can simply remember which tab the user was on last when the panel is re-opened.

Kovid, thank you for your quick + helpful response! It has been done! Both your vision of what you'd like it to look like (i.e., it being a tab in the lookup panel + remembering tab) and the pull request on GitHub.

Thank you for all the time and support, Kovid. If there is any more interaction on this thread, I shall respond accordingly, but until then, I shall carry everything forward on GitHub!

[un_pogaz]

Okay, so I'm going to put aside all my extreme disapproval of such a feature because all the criticism toward the LLM have already been presented (and because, if I start to add my voice, it will absolutly not became polite),
and so I will only talk about your implementation as presented in your PR:

Actualy, you implementaion use a third party service to uniforme the API call wich raise a great concern of my part. If this can easier the task, I realy not confident to depend to a third party service wich on existence and continuity is not guaranteed for use others third party services.
Think about it: What happen if OpenRouter shutdown? Whether it's for an hour, a day, or permanently? We cannot use any LLM service.
No, the better way is to implement it in the same way that the devices do: Create a Base class that will be herited by other class that individualy implement the code for a single LLM. Then, in the GUI, the user select the third party service wich they want to use it, and fill the specific settings and parameters.

calibre/gui2/viewer/llm/__init__.py -> contain the Base class inherited in others API call.
calibre/gui2/viewer/llm/openrouter.py -> contain the code for running OpenRouter
calibre/gui2/viewer/llm/chatgpt.py -> contain the code for running Chat-GPT
...

But that implie to implements the code for many LLM services. I don't very know how is the technical side of the LLM landscape, but from what I can see, it is constantly changing, which means that their API is too... which means that the code implemented in Calibre as a high risk to become obsolete and non-functional.
So, it would be possible to implement the feature in the same way as the Bookstore, which can be shared across Calibre versions, but that would not change the fact that its implementation would require a active maintenance (and a potentialy high level of) to satisfy a niche use case.

More I think about, more I think that it would not be a good idea to implement any LLM feature into Calibre, primary because of the implementation behind is the biggest concern.

I accept that I'm not a great programmer, and so my concern and critique is certainly very imperfect, but I'm very very concern that is a bad idea for the the long term health of Calibre's development and maintenance.

[un_pogaz]

Sorry for the double post, but I have further thoughts on the matter.

I'm strongly oppose to any LLM feature buildin to Calibre, no matter wich implementation, because the sustainability of them is not garanted.

Calibre is, in my opinion, an example of mature software that has been developed over a long period of time and has always implemented only stable features.
LLMs are nothing like that. They are just the latest technological hype from Silicon Valley, and although they are impressive, their future and longevity are by absolutly no means guaranteed.

Remember the Metaverse, NFTs, the cryptocurrencies, the Big Data, etc. The list of disruptive and revolutionary technologies that everyone forgot five years later is long, and right now, LLMs are all to be the next addition to that list.
The future of web development is AI. Get on or get left behind. by Alex.Party

There are chance, too significant to be ignored, that the feature discussed here will have to be completely removed from Calibre in the next years because there will no longer be any LLM service to support it.

So no, no LLMs whatsoever. If, in 10 years, LLM has proven to be a stable, sustainable technology with sufficient usage/demand, then yes, it will be feasible to implement a technology that has proven to be mature.

[thiago.eec]

I think it's a good idea to integrate AI with the viewer. The topic of AI generates a lot of controversy and heated opinions, but as Kovid said: only those who want to use it will. Even a simple Google search today produces an AI-generated summary, even within calibre. For me, it would be useful in certain situations, like when I'm reading a new release in a series and want to recap the previous book.

[kovidgoyal]

@un_pogaz: Dont worry about robustness, this is just like any of the various other bits of calibre that query web services, such as metadata plugins, recipes, get books, etc. etc. Nowadays calibre has a robust mechanism for live loading code (see live.py) which can be used to manage multiple backends easily.

And in any case if this feature breaks its not actually critical to usage of the viewer.

[rantanplan]

Man, it’s so disconcerting that so many posts were clearly created by an LLM. Can’t you at least use DeepL instead, if you don’t speak the language?