Big Project-Best Way to Start?

Kbobnook · 07-09-2025, 12:08 PM

I want to get a better handle on exactly what books (and audiobooks) we have (my wife and I). We have roughly 9000 ebooks from Kindle, 3000 from Barnes and Noble, and approaching 2500 from Google. There are also 600 audio Books from Audible, and roughly 300 from Chirp.

Our primary goal is to create a tagged catalog that shows what books we have, in what formats (ie which store), and more importantly, what tags are associated with each title.

One reason for the organization is simple: none of the stores make it easy to find say. Action adventure novels located in the Florida Keys, or Cozy Mysteries set in North Carrolina- that you already own!

Another is to avoid buying second copies of books we already own. It's easy to find good deals on books. Not such a good deal when you have three copies of the same book bought on differ sales from different stores... Wawawa.

Ideally, I'd like to just throw copies of all our ebooks into one folder, let calibre chew for a while, and poof, We have our catalog!. But I know it's not going to be that easy.

Is there a way to export just catalog info from our various bookstores, without doing the ebook import thing?

Once we have our perfected, up to date catalog, is there a source database from which we can import decent tag information

14,250 ebooks to go, 14,250 to import
Take one down, import it in, 14,249 ebooka to go...

theducks · 07-09-2025, 01:13 PM

Never trust Otto (anything, not just metadata downloads).
Doing that is pretty much resulting in GIGO.

Your end goals are similar to what I did.
I have a #usertags column where I log which source (store) and I have a #MstrFmt for the starting exact format (A store is not always a single format)

My original start was my 900 paperbacks (empty books. Metadata only).
I had a barcode scanner (only 60% had those) and I would do batches of 20 using Add from ISBN (the scanner was Keyboard wedge, so those scans landed in the entry box as if I had typed them), I then resolved errors and not founds while the copies were on the table. If I did not like or get a cover, I flopped the book on my flatbed scanner). Fix as you go, not let it pile up.
It took me almost a year to get that task done in my free time.

I fixed Author names (consistency, not always exact as on the cover. Not all publishers did it the same for a single author. Fn [M] Ln) I also used the Cover Author name (might be a pen name) and logged the real name in #psudo4

I also learned that Tags downloaded is a total mess, and did weeding.
Do we need 23 ways to say Science Fiction (or other genre)?
Be ready to Move from the {title} Series, other odd junk to where it fits in Calibre (or the bit bucket). Same goes for tags with a Series, Author... .
Sometimes Calibre just splits tags in the wrong place (Adams, Joey becomes 2 tags because they put a character in as Ln, Fn)

[Plug]Quality check Plugin is your friend to help keep things under control.

Have a PLAN when you need to break an unrelated (Series) Tie (I chose series (Author initials) <

Calibre mostly ignores the (contents) when sorting , so I use this feature other places as well.

Things to consider: Calibre only allows for 1 series and 1 series Index. No [5-9]. My choice is biased towards the Cover. I also use hierarchical series and Tags. eg Pern.Dragonriders That helps keep universes together Or tags Primary.refined (Fantasy.Urban)
Then there is what I call OBVIOUS. Do I need a Fiction Tag for Fantasy or Science FICTION

Rememer: Your books. Your Calibre... Your Way

Sirtel · 07-09-2025, 01:21 PM

There is no way to do this automatically. I suspect it's either creating empty book entries in calibre and populating them with the metadata you want, or download the books, deDRM them and import them to calibre. The latter is what most of us here do, although many of us started doing this way back when we had not yet accumulated thousands of books.

theducks · 07-09-2025, 01:38 PM

If you do download. do it in small chunks (~20) to avoid getting blocked or getting access banned.
Be a good Net Neighbor. Those sites let us use for free.

Kbobnook · 07-09-2025, 03:21 PM

Yeah. I get this is a long process. What frustrates me is I can search for keywords when shopping for new books, but click on library column, and it's title or author only.

Follow up question! Has anyone successfully merged three larger databases like this? Probably 30 percent of thee Barnes and Noble books are duplicates of Kindles, and well over 50 percent of the Google Play Books are duplicates as well., just to make things more fun.

Thanks for the tips, everyone, keep'em coming!

Sirtel · 07-09-2025, 04:02 PM

Your main problem here is not merging databases (that requires manual work anyway, as there's no automated way to do it), but removing the DRM from all your books, if you decide to import them into calibre. While it's still pretty easy for Google Play books, it's no longer so easy for B&N and Kindle books, and requires jumping through some hoops. Plus Amazon is actively trying to stop DRM removing, so if you wait too long, you may no longer be able to do it at all.

07-09-2025, 12:08 PM	#1
Kbobnook Junior Member Posts: 3 Karma: 10 Join Date: Jul 2025 Device: Android tablet	Big Project-Best Way to Start? I want to get a better handle on exactly what books (and audiobooks) we have (my wife and I). We have roughly 9000 ebooks from Kindle, 3000 from Barnes and Noble, and approaching 2500 from Google. There are also 600 audio Books from Audible, and roughly 300 from Chirp. Our primary goal is to create a tagged catalog that shows what books we have, in what formats (ie which store), and more importantly, what tags are associated with each title. One reason for the organization is simple: none of the stores make it easy to find say. Action adventure novels located in the Florida Keys, or Cozy Mysteries set in North Carrolina- that you already own! Another is to avoid buying second copies of books we already own. It's easy to find good deals on books. Not such a good deal when you have three copies of the same book bought on differ sales from different stores... Wawawa. Ideally, I'd like to just throw copies of all our ebooks into one folder, let calibre chew for a while, and poof, We have our catalog!. But I know it's not going to be that easy. Is there a way to export just catalog info from our various bookstores, without doing the ebook import thing? Once we have our perfected, up to date catalog, is there a source database from which we can import decent tag information 14,250 ebooks to go, 14,250 to import Take one down, import it in, 14,249 ebooka to go...

07-09-2025, 01:21 PM	#3
Sirtel Grand Sorcerer Posts: 13,451 Karma: 239219453 Join Date: Jan 2014 Location: Estonia Device: Kobo Sage & Libra 2	There is no way to do this automatically. I suspect it's either creating empty book entries in calibre and populating them with the metadata you want, or download the books, deDRM them and import them to calibre. The latter is what most of us here do, although many of us started doing this way back when we had not yet accumulated thousands of books. Last edited by Sirtel; 07-09-2025 at 01:26 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Hacks Kindle Kintsugi Project - A Mutual Aid E-Reader Distribution Project	erikhoudini	Amazon Kindle	6	07-27-2023 10:51 PM
Big Project Strategies	trefoil	General Discussions	0	03-01-2021 10:28 AM
Project Management Advice and Tips: How Good Project Managers Manage Project	amazon author	Self-Promotions by Authors and Publishers	0	04-07-2015 04:04 AM
Setting-up a big library - where to start?	greenrun	Library Management	11	11-20-2014 10:27 AM
big trouble. iLiad doesn't start	Mike Kostousov	iRex	6	01-20-2007 09:27 AM

07-09-2025, 01:13 PM	#2
theducks Well trained by Cats Posts: 31,038 Karma: 60358908 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Never trust Otto (anything, not just metadata downloads). Doing that is pretty much resulting in GIGO. Your end goals are similar to what I did. I have a #usertags column where I log which source (store) and I have a #MstrFmt for the starting exact format (A store is not always a single format) My original start was my 900 paperbacks (empty books. Metadata only). I had a barcode scanner (only 60% had those) and I would do batches of 20 using Add from ISBN (the scanner was Keyboard wedge, so those scans landed in the entry box as if I had typed them), I then resolved errors and not founds while the copies were on the table. If I did not like or get a cover, I flopped the book on my flatbed scanner). Fix as you go, not let it pile up. It took me almost a year to get that task done in my free time. I fixed Author names (consistency, not always exact as on the cover. Not all publishers did it the same for a single author. Fn [M] Ln) I also used the Cover Author name (might be a pen name) and logged the real name in #psudo4 I also learned that Tags downloaded is a total mess, and did weeding. Do we need 23 ways to say Science Fiction (or other genre)? Be ready to Move from the {title} Series, other odd junk to where it fits in Calibre (or the bit bucket). Same goes for tags with a Series, Author... . Sometimes Calibre just splits tags in the wrong place (Adams, Joey becomes 2 tags because they put a character in as Ln, Fn) [Plug]Quality check Plugin is your friend to help keep things under control. Have a PLAN when you need to break an unrelated (Series) Tie (I chose series (Author initials) < Calibre mostly ignores the (contents) when sorting , so I use this feature other places as well. Things to consider: Calibre only allows for 1 series and 1 series Index. No [5-9]. My choice is biased towards the Cover. I also use hierarchical series and Tags. eg Pern.Dragonriders That helps keep universes together Or tags Primary.refined (Fantasy.Urban) Then there is what I call OBVIOUS. Do I need a Fiction Tag for Fantasy or Science FICTION Rememer: Your books. Your Calibre... Your Way

07-09-2025, 01:38 PM	#4
theducks Well trained by Cats Posts: 31,038 Karma: 60358908 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	If you do download. do it in small chunks (~20) to avoid getting blocked or getting access banned. Be a good Net Neighbor. Those sites let us use for free.

07-09-2025, 03:21 PM	#5
Kbobnook Junior Member Posts: 3 Karma: 10 Join Date: Jul 2025 Device: Android tablet	Yeah. I get this is a long process. What frustrates me is I can search for keywords when shopping for new books, but click on library column, and it's title or author only. Follow up question! Has anyone successfully merged three larger databases like this? Probably 30 percent of thee Barnes and Noble books are duplicates of Kindles, and well over 50 percent of the Google Play Books are duplicates as well., just to make things more fun. Thanks for the tips, everyone, keep'em coming!

07-09-2025, 04:02 PM	#6
Sirtel Grand Sorcerer Posts: 13,451 Karma: 239219453 Join Date: Jan 2014 Location: Estonia Device: Kobo Sage & Libra 2	Your main problem here is not merging databases (that requires manual work anyway, as there's no automated way to do it), but removing the DRM from all your books, if you decide to import them into calibre. While it's still pretty easy for Google Play books, it's no longer so easy for B&N and Kindle books, and requires jumping through some hoops. Plus Amazon is actively trying to stop DRM removing, so if you wait too long, you may no longer be able to do it at all.