Forma Large Database Question

[davidfor]

Originally Posted by 93terp

After reloading my Aura One LE, there's virtually no change in the database size, both before Kobo Utilities compression (827MB) and post-compression (822MB). Any idea what could be leading to such a large database size?

Actually, when I quoted my database size, I was probably looking at the wrong device. That one has a lot of short stories and books with only two or three ToC entries. Looking at another device with about 1000 books, mostly novels, the database is about 90MB

In the database, you are looking at:

For each book
- One entry in the content table with the book details. This includes the metadata read from the book plus several timestamps and ids
- One entry in the content table for each entry in the ToC (for all book formats)
- For kepubs, one entry in content table for each entry in the manifest.
- One entry in the volume_shortcovers table for each entry in the ToC (for all book formats)
- The Event table will have a least one entry for each book. More get added as you open, read and finish books. And some have extra data added as you read.

Overall, these can get large. The design is terrible and there is a lot of wasted space. And the more ToC entries or internal files, the more wasted space (a lot of the columns in the content table are not used for these entries). Exporting the data for my current book to CSV, it is about 24KB. It is a sideloaded kepub with 20 internal files and ToC entries. Another with 73 chapters is about 115KB. Multiplying those out over 6000 books, it wouldn't be hard to get a large database size. And that doesn't take into account the "overhead" such as indexes and anything else in the data structures.

The size database size you have immediately after the import is probably as good as it gets. The compress will have reorganized the database and it is probably has the least wasted space it will ever have.

[93terp]

Originally Posted by davidfor

Actually, when I quoted my database size, I was probably looking at the wrong device. That one has a lot of short stories and books with only two or three ToC entries. Looking at another device with about 1000 books, mostly novels, the database is about 90MB

In the database, you are looking at:

For each book
- One entry in the content table with the book details. This includes the metadata read from the book plus several timestamps and ids
- One entry in the content table for each entry in the ToC (for all book formats)
- For kepubs, one entry in content table for each entry in the manifest.
- One entry in the volume_shortcovers table for each entry in the ToC (for all book formats)
- The Event table will have a least one entry for each book. More get added as you open, read and finish books. And some have extra data added as you read.

Overall, these can get large. The design is terrible and there is a lot of wasted space. And the more ToC entries or internal files, the more wasted space (a lot of the columns in the content table are not used for these entries). Exporting the data for my current book to CSV, it is about 24KB. It is a sideloaded kepub with 20 internal files and ToC entries. Another with 73 chapters is about 115KB. Multiplying those out over 6000 books, it wouldn't be hard to get a large database size. And that doesn't take into account the "overhead" such as indexes and anything else in the data structures.

The size database size you have immediately after the import is probably as good as it gets. The compress will have reorganized the database and it is probably has the least wasted space it will ever have.

Copy all, thanks.

[anacreon]

I've just compressed my database, and it went from 486 to 462 Mb.
I have some 5000 books on the Libra, mostly kepubs. There are only 21 PDFs, those who can be read comfortably on 7", the rest of the PDFs are on the Forma.

From what davidfor writes, the nature of the books is responsible: I have as much of my reference library as I can on the readers, complete works with one entry per chapter or per poem, which makes for huge TOCs.
All the books have more or less detailed metadata, and a number are heavily annotated.