Zipped metadata.calibre for reduce size

[un_pogaz]

I know that the metadata.calibre file is used to index the books on our device, and thus to reduce the analysis time of the presents epubs files.
But this has a default, it takes up space, and mine is 31 MB (was huge on a device, compared to 500 KB average of an epub).
Always in search of a gain of disk space, and out of curiosity, I archived my metadata.calibre in a ZIP.

And I went from 31 Mo to 3.5 Mo
...
What ?
I was expecting a result, but not so effective
8 I divided by 8, and it's a so simple idea, that I wonder (almost) how nobody thought about it and that it's not the default working of Calibre?!
(or make it an option)

Okay, I can guess that it will slow down the reading of metadata.calibre slightly, because it will first have to unzipped it to a temporary folder, and when updating data, it will have to remplace the metadata.calibre.zip on the device, but it should be "negligible".
(Or else, read and edit the metadata.calibre directly in the ZIP file, I do not know that is possible.)


So here's my idea: Archived metadata.calibre file in a ZIP.

[ilovejedd]

With HDDs at $0.03/GB and SSDs at $0.25/GB, I expect most would find such a change pretty pointless. Besides, there is likely a huge performance penalty for people with very, very large libraries.

Personally, I find deleting per book conversion options and compacting the database saves quite a bit of space already.

[PeterT]

Originally Posted by ilovejedd

With HDDs at $0.03/GB and SSDs at $0.25/GB, I expect most would find such a change pretty pointless. Besides, there is likely a huge performance penalty for people with very, very large libraries.

Personally, I find deleting per book conversion options and compacting the database saves quite a bit of space already.

We're not talking about a file on the PC but one on the eReader itself.

Sent from my Nexus 7 using Tapatalk

[kovidgoyal]

Not worth the performance code complexity overhead to me. If you have a 30MB metadata.calibre file on your device that means you are storing thousands of books on the device, comared to that the size of metadata.calibre is negligible.

[un_pogaz]

Originally Posted by kovidgoyal

Not worth the performance code complexity overhead to me. If you have a 30MB metadata.calibre file on your device that means you are storing thousands of books on the device, comared to that the size of metadata.calibre is negligible.

Yes
Not fault
Point taken
(but a need more )

Thanks for the answer, the proposal will have been made (all I wanted).
Good continuation o/

[93terp]

Originally Posted by kovidgoyal

Not worth the performance code complexity overhead to me. If you have a 30MB metadata.calibre file on your device that means you are storing thousands of books on the device, comared to that the size of metadata.calibre is negligible.

I've run into a scenario where the metadata.calibre file on my Voyage is 84MB, with only 1,812 books on (all AZW3 files, no manga, no PDFs, etc...). Yet my Oasis 2 metadata.calibre file is only 40MB and that device has almost 5,900 books on it (all AZW3 files, no manga, no PDFs, etc...). Why the disparity? And is there any sort of repair that can be done to the metadata.calibre file?

[DNSB]

Originally Posted by 93terp

I've run into a scenario where the metadata.calibre file on my Voyage is 84MB, with only 1,812 books on (all AZW3 files, no manga, no PDFs, etc...). Yet my Oasis 2 metadata.calibre file is only 40MB and that device has almost 5,900 books on it (all AZW3 files, no manga, no PDFs, etc...). Why the disparity? And is there any sort of repair that can be done to the metadata.calibre file?

You can try deleting the file and calibre will recreate it on the next connect.

[ilovejedd]

Originally Posted by 93terp

I've run into a scenario where the metadata.calibre file on my Voyage is 84MB, with only 1,812 books on (all AZW3 files, no manga, no PDFs, etc...). Yet my Oasis 2 metadata.calibre file is only 40MB and that device has almost 5,900 books on it (all AZW3 files, no manga, no PDFs, etc...). Why the disparity? And is there any sort of repair that can be done to the metadata.calibre file?

Are custom columns same on both? Because that could really make the file balloon. Every single book JSON entry contains all the metadata fields, even the empty ones.

[theducks]

The size of Comments varies per book, therefore the only comparison would be if the exact same books (and upload) were done.

I also don't think existing (device) metadata auto updates (like can happen with CC WDC mode). You need to send the book again to change what was stored on the device.

[Sarmat89]

Putting a database inside an unseekable, noneditable container like ZIP pretty much defeats its purpose.

[93terp]

Originally Posted by DNSB

You can try deleting the file and calibre will recreate it on the next connect.

Thanks, that's exactly what I did and it worked like a charm.

[93terp]

Originally Posted by ilovejedd

Are custom columns same on both? Because that could really make the file balloon. Every single book JSON entry contains all the metadata fields, even the empty ones.

yes, everything's the same.