Limit the number of batch download metadata in one go

Rellwood · 08-18-2026, 09:52 PM

I did submit a bug report, but I figured it could be metioned here.

I think it's time that Calibre impletmented a limit on the number of books that can be queued up to bactch download metadata and covers in one go.

Calibre use has been growing and people onboarding libraries and those who want to get new tags or whatever new metadata they can get with the new source plugins are loading up hundreds if not thousands of books to just churn through overnight with the expectation that it's going to have them waking up to all pristine metadata without any effort.

It's a problem when websites have been cutting access due to bots and scraping.

Im hoping that a limit on the number of books that can be queued up for a one click download is put in a new update. Im not trying to be some kind of police here,but Amazon is gone, Fantastic Fiction is gone, all these sites are using Cloudflare and WAF/Authentication to prevent automated access, yet Calibre still lets users select every book and download the metadata automatically - and doesn't stop the process even after the calls have been blocked. Users don't know that after so many books all they are doing is getting their ip's flagged and banned for continuing after each block.

Am I alone here or am I just being overly worried about future access to metadata sources?

(I am now seeing that this was posted in plugins - am moving it to Calibre unless a mod does it for me)

Rellwood · 08-18-2026, 09:57 PM

(Initally posted in Plugins by mistake - I didn't realize I was on that forum)

I did submit a bug report, but I figured it could be metioned here.

I think it's time that Calibre impletmented a limit on the number of books that can be queued up to bactch download metadata and covers in one go.

Calibre use has been growing and people onboarding libraries and those who want to get new tags or whatever new metadata they can get with the new source plugins are loading up hundreds if not thousands of books to just churn through overnight with the expectation that it's going to have them waking up to all pristine metadata without any effort.

It's a problem when websites have been cutting access due to bots and scraping.

Im hoping that a limit on the number of books that can be queued up for a one click download is put in a new update. Im not trying to be some kind of police here,but Amazon is gone, Fantastic Fiction is gone, all these sites are using Cloudflare and WAF/Authentication to prevent automated access, yet Calibre still lets users select every book and download the metadata automatically - and doesn't stop the process even after the calls have been blocked. Users don't know that after so many books all they are doing is getting their ip's flagged and banned for continuing after each block.

Am I alone here or am I just being overly worried about future access to metadata sources?

Sirtel · 08-18-2026, 10:53 PM

Personally I don't care, as I never download metadata, but always edit it manually. Yes, I have thousands of books. Editing metadata manually to start with is still faster than downloading and then editing it, because I'm never satisfied with downloaded metadata as is. I have certain preferences how titles, author and series names, comments etc should be formatted, and I use my custom tags, not those that come with books.

Mr_H_Poirot · 08-19-2026, 02:30 AM

Ultimately, if these sites don't want to be scraped they will put blockers in. And, tbh, fair enough. They do have bandwidth constraints.

I'll let users of this service opine on whether it is a problem. I personally would never want to use someone else's metadata which might have all sorts of categories I don't like or would conflict with my own. I like to catalogue all my books myself and don't really get why someone would want to bulk download like that

But then I have less than a 1000 books in my library (mainly those off Standard ebooks and Gutenberg)

Sirtel · 08-19-2026, 02:56 AM

Quote:

Originally Posted by Mr_H_Poirot

I personally would never want to use someone else's metadata which might have all sorts of categories I don't like or would conflict with my own. I like to catalogue all my books myself and don't really get why someone would want to bulk download like that.

Same, and I have a lot more books than 1000.

sgmoore · 08-19-2026, 03:12 AM

Quote:

Originally Posted by Rellwood

all these sites are using Cloudflare and WAF/Authentication to prevent automated access,

I could be totally wrong, but I suspect these measures are not taken to prevent the likes of us as calibre users, but other 'bots'. They say over half the internet traffic is now bots, and most of that appears to be malicious.

There are 'only' about 3 million Calibre users and would only spend a few minutes download meta at a time and that is probably a 'rounding error' in Amazon's total download traffic.

Quote:

Am I alone here or am I just being overly worried about future access to metadata sources?

No, I am worried as well, but I don't think we are responsible and don't think restricting calibre would help.

DNSB · 08-19-2026, 03:15 AM

For myself, the only parts of the metadata I am interested in is covers, book summaries, titles, authors & series/index. I don't have any tags in my library. Given my rather limited reading interests (SF, Fantasy, Biography, Electronics/Computers, Photography & Food), I don't bother with genres either. It's hard to mistake an AMD CPU guide for a southern cooking book.

When I first started using calibre and added my Baen, Gutenberg, etc. ebooks, I downloaded metadata individually. I remember being limited to Google and Amazon (at least going by my memory, those were the two sources) though I'm not sure if they had bulk download capability. Later I expanded to using other sources when calibre got it's plugin interface and metadata pugins plugins started to be created.

I really don't agree with trying to limit people's ability to try bulk metadata downloads. It will be nothing but a headache when people complain about not being able to download metadata for a thousand books at a time. In my testing, most of the sites I tested recently blocked me after downloading metadata for between 4 to 10 books.

DNSB · 08-19-2026, 03:21 AM

Quote:

Originally Posted by sgmoore

No, I am worried as well, but I don't think we are responsible and don't think restricting calibre would help.

I agree!

rantanplan · 08-19-2026, 04:20 AM

Quote:

Originally Posted by Rellwood

Calibre use has been growing and people onboarding libraries and those who want to get new tags or whatever new metadata they can get with the new source plugins are loading up hundreds if not thousands of books to just churn through overnight with the expectation that it's going to have them waking up to all pristine metadata without any effort.

It's a problem when websites have been cutting access due to bots and scraping.

Websites are implementing access restrictions because more than half of the internet traffic is now (AI) bots. You and I and everyone using Calibre isn't making any dent at all. We aren't the problem, bots are.

EDIT
Of course if Calibre implements a change, why not, but it won't really make a difference.

Sirtel · 08-19-2026, 04:45 AM

Quote:

Originally Posted by Rellwood

(I am now seeing that this was posted in plugins - am moving it to Calibre unless a mod does it for me)

Just a note: if you post in a wrong forum, don't post it twice. Send a pm to a mod and ask them to move it. Otherwise people start to reply in both threads, depending on which of them they notice first, and it will be harder to merge the threads than it would have been just to move the post. And posting the same message multiple times is against the forum rules too.

theducks · 08-19-2026, 10:24 AM

I think it would not hurt. We already have a 'request per period limit' to be kind to metadata servers.
Inferred from a few posts I have seen, some folk appear to think that they will get better (metadata content) if the regularly repeat downloading for their library

(I expect Calibre will get the same responses in most cases.)
The only time I DO expect a better result is for a SINGLE just released book.
So setting a limit, just helps doing our part to be a good network citizen, even if it would be a small blip in the overall traffic.

JSWolf · 08-19-2026, 01:31 PM

Even if a limit is set, it could be that too many people are downloading metadata at similar times.

Personally, I don't download metadata. I find too many errors or things that need editing. So for me, it's easier to do it manually.

I don't want tags or genres. I don't want to have to edit every description because to came with junk in it. I don't recall many descriptions would come clean. I just want the metadata to be correct. Mostly all I need is a description and maybe a cover.

issybird · 08-19-2026, 06:37 PM

Quote:

Originally Posted by Rellwood

(Initally posted in Plugins by mistake - I didn't realize I was on that forum)

In that situation, per the guidelines, please report the post and ask for it to be moved rather than double-posting.

BetterRed · 08-19-2026, 07:52 PM

Moderator Notice

Threads merged and moved to where they belong.

08-18-2026, 09:52 PM	#1
Rellwood Library Breeder (She/Her) Posts: 1,352 Karma: 1977877 Join Date: Apr 2015 Location: California Device: Paperwhite 2015 (2), PW 2024 (12 GEN), PW 2023 (11 GEN), Scribe (1st)	Limit the number of batch download metadata in one go I did submit a bug report, but I figured it could be metioned here. I think it's time that Calibre impletmented a limit on the number of books that can be queued up to bactch download metadata and covers in one go. Calibre use has been growing and people onboarding libraries and those who want to get new tags or whatever new metadata they can get with the new source plugins are loading up hundreds if not thousands of books to just churn through overnight with the expectation that it's going to have them waking up to all pristine metadata without any effort. It's a problem when websites have been cutting access due to bots and scraping. Im hoping that a limit on the number of books that can be queued up for a one click download is put in a new update. Im not trying to be some kind of police here,but Amazon is gone, Fantastic Fiction is gone, all these sites are using Cloudflare and WAF/Authentication to prevent automated access, yet Calibre still lets users select every book and download the metadata automatically - and doesn't stop the process even after the calls have been blocked. Users don't know that after so many books all they are doing is getting their ip's flagged and banned for continuing after each block. Am I alone here or am I just being overly worried about future access to metadata sources? (I am now seeing that this was posted in plugins - am moving it to Calibre unless a mod does it for me) Last edited by Rellwood; 08-18-2026 at 09:55 PM. Reason: Noticing that it was in the wrong forum

08-18-2026, 09:57 PM	#2
Rellwood Library Breeder (She/Her) Posts: 1,352 Karma: 1977877 Join Date: Apr 2015 Location: California Device: Paperwhite 2015 (2), PW 2024 (12 GEN), PW 2023 (11 GEN), Scribe (1st)	Limiting one-click batch download metadata (Initally posted in Plugins by mistake - I didn't realize I was on that forum) I did submit a bug report, but I figured it could be metioned here. I think it's time that Calibre impletmented a limit on the number of books that can be queued up to bactch download metadata and covers in one go. Calibre use has been growing and people onboarding libraries and those who want to get new tags or whatever new metadata they can get with the new source plugins are loading up hundreds if not thousands of books to just churn through overnight with the expectation that it's going to have them waking up to all pristine metadata without any effort. It's a problem when websites have been cutting access due to bots and scraping. Im hoping that a limit on the number of books that can be queued up for a one click download is put in a new update. Im not trying to be some kind of police here,but Amazon is gone, Fantastic Fiction is gone, all these sites are using Cloudflare and WAF/Authentication to prevent automated access, yet Calibre still lets users select every book and download the metadata automatically - and doesn't stop the process even after the calls have been blocked. Users don't know that after so many books all they are doing is getting their ip's flagged and banned for continuing after each block. Am I alone here or am I just being overly worried about future access to metadata sources?

08-19-2026, 03:15 AM	#7
DNSB Bibliophagist Posts: 53,899 Karma: 182150167 Join Date: Jul 2010 Location: Vancouver Device: Kobo Sage, Libra Colour, Lenovo M8 FHD, Paperwhite 4, Tolino epos	For myself, the only parts of the metadata I am interested in is covers, book summaries, titles, authors & series/index. I don't have any tags in my library. Given my rather limited reading interests (SF, Fantasy, Biography, Electronics/Computers, Photography & Food), I don't bother with genres either. It's hard to mistake an AMD CPU guide for a southern cooking book. When I first started using calibre and added my Baen, Gutenberg, etc. ebooks, I downloaded metadata individually. I remember being limited to Google and Amazon (at least going by my memory, those were the two sources) though I'm not sure if they had bulk download capability. Later I expanded to using other sources when calibre got it's plugin interface and metadata pugins plugins started to be created. I really don't agree with trying to limit people's ability to try bulk metadata downloads. It will be nothing but a headache when people complain about not being able to download metadata for a thousand books at a time. In my testing, most of the sites I tested recently blocked me after downloading metadata for between 4 to 10 books. Last edited by DNSB; 08-19-2026 at 03:20 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Batch Metadata Download Broken 9.2(?)	johnelle	Library Management	7	02-12-2026 05:41 AM
Metadata download create limit	yamusing	Library Management	14	12-28-2022 10:02 PM
Limit metadata download to single field	zakman	Library Management	1	04-24-2022 06:58 PM
Limit to Number of Books?	TheArtfulDodger	Devices	3	03-13-2013 02:25 AM
Limit on Number of books?	kblechman	Library Management	1	04-02-2011 10:05 AM

08-18-2026, 10:53 PM	#3
Sirtel Grand Sorcerer Posts: 15,590 Karma: 249250073 Join Date: Jan 2014 Location: Estonia Device: Kobo Sage & Libra 2	Personally I don't care, as I never download metadata, but always edit it manually. Yes, I have thousands of books. Editing metadata manually to start with is still faster than downloading and then editing it, because I'm never satisfied with downloaded metadata as is. I have certain preferences how titles, author and series names, comments etc should be formatted, and I use my custom tags, not those that come with books.

08-19-2026, 02:30 AM	#4
Mr_H_Poirot Connoisseur Posts: 72 Karma: 16722 Join Date: Mar 2025 Device: Kobo Clara BW	Ultimately, if these sites don't want to be scraped they will put blockers in. And, tbh, fair enough. They do have bandwidth constraints. I'll let users of this service opine on whether it is a problem. I personally would never want to use someone else's metadata which might have all sorts of categories I don't like or would conflict with my own. I like to catalogue all my books myself and don't really get why someone would want to bulk download like that But then I have less than a 1000 books in my library (mainly those off Standard ebooks and Gutenberg)

08-19-2026, 10:24 AM	#11
theducks Well trained by Cats Posts: 31,931 Karma: 64187770 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	I think it would not hurt. We already have a 'request per period limit' to be kind to metadata servers. Inferred from a few posts I have seen, some folk appear to think that they will get better (metadata content) if the regularly repeat downloading for their library (I expect Calibre will get the same responses in most cases.) The only time I DO expect a better result is for a SINGLE just released book. So setting a limit, just helps doing our part to be a good network citizen, even if it would be a small blip in the overall traffic.

08-19-2026, 01:31 PM	#12
JSWolf Resident Curmudgeon Posts: 84,568 Karma: 153744815 Join Date: Nov 2006 Location: Roslindale, Massachusetts Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3	Even if a limit is set, it could be that too many people are downloading metadata at similar times. Personally, I don't download metadata. I find too many errors or things that need editing. So for me, it's easier to do it manually. I don't want tags or genres. I don't want to have to edit every description because to came with junk in it. I don't recall many descriptions would come clean. I just want the metadata to be correct. Mostly all I need is a description and maybe a cover.

08-19-2026, 07:52 PM	#14
BetterRed null operator (he/him) Posts: 22,790 Karma: 34252730 Join Date: Mar 2012 Location: Sydney Australia Device: none	Moderator Notice Threads merged and moved to where they belong.

Advert

Advert