Do builtin AI prompts need explict 'say I don't know' instruction?

[kovidgoyal]

Users can create custom actions. Whether order matters or not is largely model dependent and unknown.

[WillAdams]

Yes, if the model knew what it doesn't know --- since it doesn't know that, such an instruction will not preclude hallucinations based on compression errors or flaws in the training methodology.

[mitsie]

Originally Posted by kovidgoyal

In my experience they still make stuff up. I have had conversations with AI that go along the line of "I ask a question, it gives a made up answer, I tell it the answer is made up, it apologises and comes back with a new made up answer". If you happen to ask it a question about something not in its training data it's pretty hopeless. It's very much a case of you need to understand the limitations of the tool, if you do and use it keeping those in mind, its very useful.

That said, I have no objection to adding such a line to the prompt. It wont do any harm. You can try and see what effect it has, if any, by asking a custom question instead of using one of the quick actions.

That's why in my AI plugin using Retrieval Augmented Generation (RAG) and a Local LLM, you get a successful Answer 99% of the time.

With a Cloud AI, the temperature is midway between Creativity and Truthful Accuracy. The higher you turn the temperature, the more creative an AI's response..

With a Cloud Service such as Gemini, you don't get to dictate the Temperature.. with a local LLM, you have full control.

Like others have pointed out, an AI will simply answer do not know that book, and return nothing.. that's where RAG comes in.. you load the ebook into the AI model from your filesystem.. So no matter what eBook it is, doesn't matter if it wasn't published or your Mother wrote it.. the AI still has access to the material and can summarise it..

Not only is it beneficial to use Local LLMs because you have full control, it also stops the transfer of Authors data to Cloud AI services to Harvest eBooks..

Turn the temperature down on LLM to 0 so it doesn't "Make stuff up", use RAG for local eBooks for reference gives successful and accurate results 99% of the time.

That's why I written the AI plugins the way I did in the first place..

[kassandr]

The hallucination problem discussed here is real, and "say I don't know" helps, but it's a patch on a structural issue. When calibre's Discuss sends a single book to an LLM, the model can only work with what it receives in that session. It has no memory of your library, no access to your annotations, no way to cross-reference between titles. For the question "what does this book say about X?" that's often enough. But for "what do my books say about X?" - across hundreds or thousands of volumes - it's a fundamentally different problem. That's where Retrieval-Augmented Generation (RAG) comes in: instead of asking the model to recall from training data, you index your actual library and feed the relevant passages to the model before it answers. The model responds based on your sources, with citations.

I've been building an open-source tool called ARCHILLES that does exactly this. It connects to Calibre (among others), indexes full text, metadata, and annotations via multilingual embeddings, and exposes the library to any AI model via MCP (Model Context Protocol), so it works with Claude, ChatGPT, local models, whatever you prefer. Everything runs locally, no data leaves your machine. It's MIT-licensed and on GitHub: https://github.com/kasssandr/archilles. Still early, but the core search and citation pipeline is functional.