View Single Post
Old 05-27-2014, 01:54 PM   #56
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,645
Karma: 5433388
Join Date: Nov 2009
Device: many
Hi,

Both headers are not at the beginning of the file. Typically the mobi6 header comes right after the palm section table, then there are lots of additional sections that hold all of the text of the file, all of the resources (fonts, images, resc section) and then a ncx index, flis, fcis, srcs sections, datp, etc and then a boundary section and then finally comes the mobi8 header, followed by its own text sections, and its indexes, and then a new boundary section containing a CONT section which is an HD Container with lots of HD images.

So to edit both headers you need to split the file at the headers and then recreate the entire file twice.

There really is no other way to deal with this unless you want to use file io to build the new version from smaller chunks and pieces which will be much slower than doing it in memory.

I will take a look at it when I get a free moment.

Kevin


Quote:
Originally Posted by AcidWeb View Post
Input file = 409MB

Memory usage before:

420.35546875

After that line:
819.8828125

And it crash line later on:

On 64bit Python memory usage after that line is around 1225.

Using append don't impact memory usage.

EDIT:
Well after spending another 6h on tests now I'm quite sure that is not an error. It just use too much memory.
All these anomalies were caused by fact that standalone program was running very close to memory limit and success depended on the number of imports (lol!).

Both of headers are on beginning on the file? Why we loading entire file?
KevinH is online now   Reply With Quote