Why does Calibre Library take up so much space? 7000 books equals 50GB when exported

droopy · 07-21-2020, 06:55 PM

Why does Calibre Library take up so much space?

7000 books equals 50GB when exported. Most books in my library are epubs, and most of these epubs are 2MB on average I'm guessing. Yes, I do have some epubs that have many high-res images. Yes, I do have about 200 PDFs, some of which are image-based PDFs.

But I wouldn't think it would take 50 gigabytes? Are these one-gigabyte library000X.data slivers not compressed?

And what about the calibre-library itself (I mean, not just the "exported data" form)?

ownedbycats · 07-21-2020, 07:23 PM

If you're on Windows, run WinDirStat on your library folder. It can be a little slow but it'll show you where the space is going.

I'd put money on the PDF files you mentioned.

For ePubs, you can run Polish > Losslessly compress images and see if that helps.

droopy · 07-21-2020, 07:25 PM

I'm on Linux.

ownedbycats · 07-21-2020, 07:28 PM

K4DirStat or QDirStat may be of more use then, though I can't vouch for them myself. I recommended WinDirStat specifically because it also shows which extensions are taking up the most space.

Quoth · 07-22-2020, 09:56 AM

200 off 200 M byte Image PDFs is 40G.

You can easily find out how much space your PDFs take.
Then there are 7000 cover images, stored separately.

ownedbycats · 07-22-2020, 04:39 PM

Quote:

Originally Posted by ownedbycats

K4DirStat or QDirStat may be of more use then, though I can't vouch for them myself. I recommended WinDirStat specifically because it also shows which extensions are taking up the most space.

JDiskReport may also be of use, if you don't mind Java. Like WinDirStat, it'll also breakdown the filetypes for you.

droopy · 07-22-2020, 05:15 PM

i avoid java if i can.

anyway, I found that "Disk Usage Analyzer" comes pre-installed with Mint 20.

Bookstooge · 07-22-2020, 06:48 PM

Let us know the breakdown. You've got me curious, at least

retiredbiker · 07-22-2020, 11:21 PM

This makes me curious too. I hadn't checked for a long time, but I find I've got 38,273 "books", 22.9 gb, for an average of 600 kb per "book". Virtually all epubs, lots of short stories and so on. I see books getting fatter all the time, it's getting ridiculous.

ownedbycats · 07-22-2020, 11:29 PM

My library is around 20 GB, but the vast bulk of that is taken up by image-heavy PDF files.

DNSB · 07-23-2020, 12:51 AM

My library clocks in at 22.4GB for about 11,700 books. Mostly epub with the occasional PDF. If I dropped my average cover size from 1000x1500 to 500x750, I'd drop 10GB of that space.

droopy · 07-23-2020, 01:20 AM

Quote:

Originally Posted by DNSB

My library clocks in at 22.4GB for about 11,700 books. Mostly epub with the occasional PDF. If I dropped my average cover size from 1000x1500 to 500x750, I'd drop 10GB of that space.

I have a forma. when i do "Download Metadata" in Calibre, the cover sizes vary:
Some examples are:
1XX by 1XX
3XX by 5XX
1200 by 2000

What image size is ideal for a Kobo Forma?

DNSB · 07-23-2020, 02:34 AM

Quote:

Originally Posted by droopy

I have a forma. when i do "Download Metadata" in Calibre, the cover sizes vary:
Some examples are:
1XX by 1XX
3XX by 5XX
1200 by 2000

What image size is ideal for a Kobo Forma?

Whatever size you like. I still use the 1000x1500 or 1125x1500 from the days when I had the cover displayed as the sleep screen. The smallest I found usable for that was the 500x750 but on some covers, it made the sleep image look pixelated. It may cost me a bit of storage but that's rather cheap these days.

Ask yourself how often do I actually look at the cover? For more than 1 second?

Not to mention that you have to be careful with some cover sources. Google seems to love 1800x2700 images that when you look at the full size image are horribly pixelated.

droopy · 07-23-2020, 03:26 AM

After I decide on my preferred image size, is there a way to tell the caliber program to choose a certain size as my first option and then another size as my second option and then the third size as my third option?

pdurrant · 07-23-2020, 04:27 AM

My calibre ebooks folder is 45.88GB for 4787 books.

3,452 MOBI: 18.44GB
2,220 ePub: 11.54GB
203 CBZ: 9.69GB
71 PDF: 3.79GB
791 AZW3: 1.04GB
4757 JPEG Covers: 789MB
31 PDB & PMLZ: ca. 25MB
original copies: ca 20MB

Total of ebook files: 45.29 GB

It doesn't look like there's a lot of overhead to me.

07-21-2020, 06:55 PM	#1
droopy Guru Posts: 834 Karma: 2912460 Join Date: Apr 2009 Device: Kobo Forma	Why does Calibre Library take up so much space? 7000 books equals 50GB when exported Why does Calibre Library take up so much space? 7000 books equals 50GB when exported. Most books in my library are epubs, and most of these epubs are 2MB on average I'm guessing. Yes, I do have some epubs that have many high-res images. Yes, I do have about 200 PDFs, some of which are image-based PDFs. But I wouldn't think it would take 50 gigabytes? Are these one-gigabyte library000X.data slivers not compressed? And what about the calibre-library itself (I mean, not just the "exported data" form)?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Library File Structure for Books Exported to eBook Device	otta56	Library Management	7	04-15-2020 11:29 AM
How to sync library after exported library	chantw0968	Library Management	3	10-30-2017 09:40 PM
Books blank when exported	Joosca77	Library Management	2	06-10-2011 11:33 PM
7000 books ! A new milestone for the MR Library !	zelda_pinwheel	Announcements	26	12-12-2008 12:46 PM

07-21-2020, 07:23 PM	#2
ownedbycats Custom User Title Posts: 11,359 Karma: 79528341 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	If you're on Windows, run WinDirStat on your library folder. It can be a little slow but it'll show you where the space is going. I'd put money on the PDF files you mentioned. For ePubs, you can run Polish > Losslessly compress images and see if that helps.

07-21-2020, 07:25 PM	#3
droopy Guru Posts: 834 Karma: 2912460 Join Date: Apr 2009 Device: Kobo Forma	I'm on Linux.

07-21-2020, 07:28 PM	#4
ownedbycats Custom User Title Posts: 11,359 Karma: 79528341 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	K4DirStat or QDirStat may be of more use then, though I can't vouch for them myself. I recommended WinDirStat specifically because it also shows which extensions are taking up the most space.

07-22-2020, 09:56 AM	#5
Quoth Still reading Posts: 15,004 Karma: 111111255 Join Date: Jun 2017 Location: Ireland Device: All 4 Kinds: epub eink, Kindle, android eink, NxtPaper	200 off 200 M byte Image PDFs is 40G. You can easily find out how much space your PDFs take. Then there are 7000 cover images, stored separately.

07-22-2020, 05:15 PM	#7
droopy Guru Posts: 834 Karma: 2912460 Join Date: Apr 2009 Device: Kobo Forma	i avoid java if i can. anyway, I found that "Disk Usage Analyzer" comes pre-installed with Mint 20.

07-22-2020, 06:48 PM	#8
Bookstooge Member Retired Posts: 805 Karma: 2091358 Join Date: May 2019 Device: Kindle Oasis 1st Gen, PB Era	Let us know the breakdown. You've got me curious, at least

07-22-2020, 11:21 PM	#9
retiredbiker Evangelist Posts: 454 Karma: 3886916 Join Date: May 2013 Location: Ontario, Canada Device: Kindle KB, Oasis, Pop_Os!, Kobo Forma	This makes me curious too. I hadn't checked for a long time, but I find I've got 38,273 "books", 22.9 gb, for an average of 600 kb per "book". Virtually all epubs, lots of short stories and so on. I see books getting fatter all the time, it's getting ridiculous.

07-22-2020, 11:29 PM	#10
ownedbycats Custom User Title Posts: 11,359 Karma: 79528341 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	My library is around 20 GB, but the vast bulk of that is taken up by image-heavy PDF files.

07-23-2020, 12:51 AM	#11
DNSB Bibliophagist Posts: 48,175 Karma: 174315444 Join Date: Jul 2010 Location: Vancouver Device: Kobo Sage, Libra Colour, Lenovo M8 FHD, Paperwhite 4, Tolino epos	My library clocks in at 22.4GB for about 11,700 books. Mostly epub with the occasional PDF. If I dropped my average cover size from 1000x1500 to 500x750, I'd drop 10GB of that space.

07-23-2020, 03:26 AM	#14
droopy Guru Posts: 834 Karma: 2912460 Join Date: Apr 2009 Device: Kobo Forma	After I decide on my preferred image size, is there a way to tell the caliber program to choose a certain size as my first option and then another size as my second option and then the third size as my third option?

07-23-2020, 04:27 AM	#15
pdurrant The Grand Mouse 高貴的老鼠 Posts: 74,449 Karma: 318076944 Join Date: Jul 2007 Location: Norfolk, England Device: Kindle Oasis	My calibre ebooks folder is 45.88GB for 4787 books. 3,452 MOBI: 18.44GB 2,220 ePub: 11.54GB 203 CBZ: 9.69GB 71 PDF: 3.79GB 791 AZW3: 1.04GB 4757 JPEG Covers: 789MB 31 PDB & PMLZ: ca. 25MB original copies: ca 20MB Total of ebook files: 45.29 GB It doesn't look like there's a lot of overhead to me.