[GUI Plugin] AI Vision Metadata

[RelUnrelated]

[Plugin] AI Vision Metadata

Overview
AI Vision Metadata is a custom calibre plugin designed to automate the extraction of metadata from publication covers. By leveraging state-of-the-art AI vision models, it analyzes cover art to identify specific issue numbers, publication dates, publishers, and creators, making it an invaluable tool for cataloging comics, magazines, and vintage periodicals.

Originally built around Google's Gemini API, the plugin has evolved into a robust, multi-agent engine tailored for power users.

Key Features

• Multi-Provider Routing: Seamlessly switch between cloud-based AI models (Google Gemini, OpenAI, Anthropic) or route requests to your own local, offline models using Ollama or LM Studio.
• Sequential Batch Processing: Select multiple publications at once. The plugin intelligently queues the requests in the background, preventing rate-limit bans and UI lockups.
• Side-by-Side Review GUI: Never fly blind. The plugin presents a crisp, scaled thumbnail of the cover image right next to the extracted metadata, allowing you to easily verify the AI's accuracy.
• Isolated Memory Banks: The configuration menu securely remembers your distinct API keys, model selections, and custom system prompts for every individual provider.
• Advanced Prompt Tuning: Directly edit the AI's core instructions to fine-tune extraction behavior for the unique quirks of your specific collection.
• Thread-Safe Architecture: Background processing ensures your main calibre window never freezes, while gracefully catching and reporting network or API errors.

Installation

1. Download the attached AI_Vision_Metadata_v1.0.0.zip file at the bottom of this post. (Do not unzip this plugin file).
2. Open calibre and click on Preferences (the gear icon) in the top toolbar.
3. Under the "Advanced" section, click on Plugins.
4. Click the Load plugin from file button in the bottom right corner.
5. Navigate to and select the zip file.
6. Click Yes to accept the security warning and install the plugin.
7. Restart calibre for the changes to take effect.

Configuration
Before using the tool, you must configure it with an API key or a local server address.

1. Go to Preferences > Plugins and locate AI Vision Metadata under the User interface action category. Double-click to open the configuration window.
2. AI Provider: Select your preferred AI engine from the dropdown (Google Gemini, OpenAI, Anthropic, or Local). The UI will dynamically update to show the settings for that specific provider.
3. API Key / Local URL: Paste your API key for the selected cloud provider. If using a local model, ensure your Local Base URL is correct (e.g., http://localhost:11434 for Ollama).
4. Model Name: Click Fetch Available Models to populate the dropdown menu directly from your chosen provider, then select the specific model you wish to use.
5. System Prompt (Advanced): You can safely tweak the AI's core instructions here. Every provider remembers its own prompt.
6. Click Apply or OK to save.

Usage
Once configured, the plugin integrates seamlessly into your standard calibre workflow.

1. Select Publications: Highlight one or more entries in your calibre library that have cover images. (Batch processing is fully supported).
2. Trigger the Plugin: Click the AI Vision Metadata button in your main toolbar, or right-click the highlighted books and select it from the context menu.
3. Wait for Processing: The plugin runs in a safe background thread. It will analyze the first image and compile the data.
4. Review the Data: A "Review AI Metadata" window will appear, featuring the cover image on the left and the extracted data on the right.
   • Action Indicators: Every field includes a muted sub-label (e.g., Replaces, Merges, Appends) so you know exactly whether checking the box will overwrite your existing calibre data or safely add to it.
   • Checkboxes: Use the checkboxes to select exactly which fields you want to import. Unchecked fields will be ignored, preserving your existing calibre database entries.
   • Editable Dropdowns: Fields like Series Index offer auto-generated formats, but you can manually type directly into the box for edge cases.
5. Apply & Auto-Advance: Click OK to save the checked metadata directly to calibre. If you selected multiple books, the plugin will seamlessly load the next cover in your queue and begin processing it immediately.

Provider Setup Guide
To use the cloud features of this plugin, you will need to generate an API key from your preferred provider. Treat these keys like passwords.

Google Gemini (Recommended for Free Tier)

• Navigate to Google AI Studio to generate a free API key.
• Note on Limits: gemini-2.0-flash offers generous free daily quotas. Using gemini-2.5-pro for complex covers and deep web searching is highly recommended, but it requires adding a billing profile to your Google Cloud account to lift strict rate limits.

OpenAI (ChatGPT)

• Navigate to the OpenAI Platform to generate a key.
• Requirements: OpenAI no longer offers free API grants. You must add prepaid credits (minimum $5) to your developer dashboard for the API to process requests.

Anthropic (Claude)

• Navigate to the Anthropic Console to generate a key.
• Requirements: Like OpenAI, Anthropic requires you to load prepaid credits to your account before API requests will be authorized (otherwise you will receive an immediate HTTP 400 error).

Local Models (Ollama / LM Studio)

• You can run vision-capable models (like llava) completely offline on your own hardware.
• Download Ollama or LM Studio. Make sure your local server is running, verify the Base URL in the plugin settings, and fetch the models you have downloaded.

License
This plugin is licensed under the GNU GPL v3.

Version History

Spoiler:

• v1.0.0 - Initial public release. Features multi-provider AI routing, background batch processing, and the side-by-side review GUI.
• v0.9.0 - Pre-release testing, architecture validation, and UI stabilization.

[RelUnrelated]

Reserved for future test versions

[akasdh]

Hello!
Thank you so much for this amazing plugin, it is as game changer!

Sadly i experience 503 errors every few books consistently ever since i started using it 3 days ago. I am using the Google Cloud API.

I have added billing to my Google Cloud Account, and the limits there are miles away from being reached. I tried every possible model to see if that is the problem.
There are no 503 errors anywhere in the Google Cloud logs.
I used the Google Cloud Gemini assistant to check the errors, and its unable to find anything.

Geminis hypothesis:

Hypothesis 1: Ghost Project / ID Issue. Despite the ID matching your UI context, the lack of logs and the disabled Service Usage API suggest this project might not be the one actively serving traffic.
--> Not possible, i only started using Google Cloud API for the plugin, it is the only project and the name and id is the same as i gave to Gemini to analyze.

Hypothesis 2: External Errors. If you are seeing 503s at a specific URL, it's possible that the URL is pointing to a resource in a different project, or the error is coming from an external provider (like a DNS provider or third-party proxy) before it reaches Google Cloud.
--> I use no VPN, and i set cloudflares DNS servers in my router and have 0 issues with that.

Hypothesis 3: Extreme Propagation Delay. If resources were very recently created or deleted, there may be a delay in them appearing in discovery tools.
--> Its been 3 days, i doubt Googles infrastructure is that slow to propagate.

This plugin is really a game changer, so its extremely sad if i can not fix this.
Any ideas? Thanks in advance!

[RelUnrelated]

Originally Posted by akasdh

Hello!
Thank you so much for this amazing plugin, it is as game changer!

Sadly i experience 503 errors every few books consistently ever since i started using it 3 days ago. I am using the Google Cloud API.

Thank you so much for the kind words! I'm thrilled to hear it's a game changer for your workflow.

Regarding that 503 error: you haven't done anything wrong. That is Google's server telling the plugin, "We are temporarily at full capacity." Google AI Studio is incredibly popular, so during peak hours, they occasionally shed load by returning a 503 Service Unavailable error. I'm getting them more frequently here, too.

The best workaround right now is simply to wait a few minutes and try the batch again. If you are doing very large batches, breaking them down into smaller chunks during busy times of the day can also help.

I'll look into adding an automatic "retry" feature using 'exponential backoff' for these specific Google traffic jams in the next version. I'm also working on some more user-friendly error reporting.

Thanks again for letting me know that I'm not the only one who wanted something like this to make things a bit easier for me. Cheers!