epub load time, possible file optimizations?

takamatsu · 07-06-2010, 03:41 AM

I'm reading a longish epub, Man, Economy, and State (with Power and Market) by Murray N. Rothbard on my Kobo.

From clicking on the book in the menu it takes around 2 minutes to get to the page I am on. Once loaded page turns are usually not too bad. The epub is stored on an SD card. I have found shorter books open quite quickly so I don't think the SD card is the problem.

The epub is 4.2MB before uncompressing. I unzipped the epub to check out its structure. After decompressing the total size is 8.6MB. The book is split into html files with uncompressed size of 100-300KB. There are 4.7MB of black and white jpg images, but the largest is only 80KB. Very few pages actually show images (they are small graphs).

I was wondering if there is something I can do in Calibre to help this and similar books load faster, for example is it better to split the html into more or less files? or compress it more or less? If there is no other alternative is there a good way to split it into smaller epubs?

I appreciate your help; please feel free to get technical.

hyperb · 07-06-2010, 06:05 AM

From playing with the included free books I can tell you that the books with large chapter sizes have very long load and font resize times. How many pages are chapters. The load times seem to long for anything over 100 pages per chapter on medium or smaller font.

goinveg · 07-06-2010, 06:12 AM

Fancy that! I have the same book loaded as well as Human Action and a few others. I hadn't yet got to reading them so I didn't see that issue but, yes, it takes a while. I will be looking a that later today when I've more time.

Stinger · 07-06-2010, 05:12 PM

indeed!

I tried reading 'On The Origin Of Species' by Darwin and noticed this problem too. The entire bulk of that book is pretty much 1 chapter, so I figured it had something to do with that.

@takamatsu:
Have you tried further reducing the size of the (split) HTML files? In Calibre, in the conversion option->Epub Output section, you can reduce the split size.

takamatsu · 07-07-2010, 02:14 AM

@goinveg very amusing that you are looking at similar books. Having a chance to read them was a big motivator for buying an ebook reader.

@hyperb Based on Kobo's page count there are about 100 pages per chapter.

@Stinger regarding size of split HTML files. This was my first thought too. I had a big fight with Calibre (7.7) trying to make it split the files at 50KB. It kept splitting at 260KB anyway. I erased settings manually, reinstalled etc. Eventually I figured out that it silently ignores requested split levels below 100KB. Huge waste of time.

After at last managing to split the HTML at 100KB chunks (confirmed again by unzipping) the book now loads in a blistering.... 3.5 minutes. This is not an artifact of running the book through Calibre; I tested again at 260KB splits and the time went back to about 2 minutes.

I am really curious as to whether Kobo is unzipping the whole epub into memory, or unzipping individual files on demand. I also wonder how much flow calculation is being done on initial load.

I'm leaning toward splitting this book into 2 halves, but can't see how to do that with Calibre.

Any other ideas would be great.

kiwikobo · 07-07-2010, 07:01 PM

Try using Sigil to tweak the book. This allows you to see everything that's going on. You can also insert chapter breaks wherever you like (make some <h2>s)

takamatsu · 07-07-2010, 09:01 PM

Thanks kiwikobo, I will try Sigil.

I also did another experiment on chapter splitting by setting the chapter split threshold very high. This way the book is still split into chapters, but the chapters are not split into parts.

The result was significantly faster load time, 1.5 minutes, down from 2 minutes. I think this indicates that a significant amount of time is spent gluing back together chapters that have been split into parts. If the Darwin book mentioned above is all one chapter, maybe the huge chapter is being glued back together on load.

Truly separate smaller chapters look like a good approach.

goinveg · 07-08-2010, 12:19 PM

I just downloaded a PDF version from http://mises.org/books/mespm.pdf. Calibre converted it as one long chapter - bookmarked near the end. But it is only 1.2MB in size and loads quickly.

takamatsu · 07-09-2010, 03:54 AM

Sigil turned out to be useful. I used it to delete the chapters belonging to "Man, Economy, and State" since I had finished it, leaving me with just "Power and Market". This slimmed down version loads quickly.

goinveg, I can see why this book converted well from pdf - it is basically linear. I have tried converting several books before that have margin notes and it ended up a confusing mish mash.

07-06-2010, 03:41 AM	#1
takamatsu Person Posts: 5 Karma: 10 Join Date: Jul 2010 Device: Kobo	epub load time, possible file optimizations? I'm reading a longish epub, Man, Economy, and State (with Power and Market) by Murray N. Rothbard on my Kobo. From clicking on the book in the menu it takes around 2 minutes to get to the page I am on. Once loaded page turns are usually not too bad. The epub is stored on an SD card. I have found shorter books open quite quickly so I don't think the SD card is the problem. The epub is 4.2MB before uncompressing. I unzipped the epub to check out its structure. After decompressing the total size is 8.6MB. The book is split into html files with uncompressed size of 100-300KB. There are 4.7MB of black and white jpg images, but the largest is only 80KB. Very few pages actually show images (they are small graphs). I was wondering if there is something I can do in Calibre to help this and similar books load faster, for example is it better to split the html into more or less files? or compress it more or less? If there is no other alternative is there a good way to split it into smaller epubs? I appreciate your help; please feel free to get technical.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Long load time	Mostly Math	Kobo Reader	8	09-06-2010 09:21 PM
Help to load file in calibre customized plugin	William A	Amazon Kindle	0	03-21-2010 12:57 PM
Ebook load time	richough3	Which one should I buy?	11	08-21-2009 09:22 AM
Speeding Up The Load Time of Scanned Books	owl123	iRex	0	06-04-2009 10:11 AM
Could Book Designer Load an LRF File?	Fitzwaryn	Sony Reader	6	01-30-2008 04:43 AM

07-06-2010, 06:05 AM	#2
hyperb Junior Member Posts: 6 Karma: 12 Join Date: Jul 2010 Device: Kobo	From playing with the included free books I can tell you that the books with large chapter sizes have very long load and font resize times. How many pages are chapters. The load times seem to long for anything over 100 pages per chapter on medium or smaller font.

07-06-2010, 06:12 AM	#3
goinveg Enthusiast Posts: 32 Karma: 12 Join Date: Jun 2010 Location: Meriden, CT, USA Device: Kobo, Entourage eDGe DualBook	Fancy that! I have the same book loaded as well as Human Action and a few others. I hadn't yet got to reading them so I didn't see that issue but, yes, it takes a while. I will be looking a that later today when I've more time.

07-06-2010, 05:12 PM	#4
Stinger Asha'man Posts: 335 Karma: 844 Join Date: May 2010 Location: Canada Device: Kobo	indeed! I tried reading 'On The Origin Of Species' by Darwin and noticed this problem too. The entire bulk of that book is pretty much 1 chapter, so I figured it had something to do with that. @takamatsu: Have you tried further reducing the size of the (split) HTML files? In Calibre, in the conversion option->Epub Output section, you can reduce the split size.

07-07-2010, 02:14 AM	#5
takamatsu Person Posts: 5 Karma: 10 Join Date: Jul 2010 Device: Kobo	@goinveg very amusing that you are looking at similar books. Having a chance to read them was a big motivator for buying an ebook reader. @hyperb Based on Kobo's page count there are about 100 pages per chapter. @Stinger regarding size of split HTML files. This was my first thought too. I had a big fight with Calibre (7.7) trying to make it split the files at 50KB. It kept splitting at 260KB anyway. I erased settings manually, reinstalled etc. Eventually I figured out that it silently ignores requested split levels below 100KB. Huge waste of time. After at last managing to split the HTML at 100KB chunks (confirmed again by unzipping) the book now loads in a blistering.... 3.5 minutes. This is not an artifact of running the book through Calibre; I tested again at 260KB splits and the time went back to about 2 minutes. I am really curious as to whether Kobo is unzipping the whole epub into memory, or unzipping individual files on demand. I also wonder how much flow calculation is being done on initial load. I'm leaning toward splitting this book into 2 halves, but can't see how to do that with Calibre. Any other ideas would be great.

07-07-2010, 07:01 PM	#6
kiwikobo Enthusiast Posts: 47 Karma: 120 Join Date: Jun 2010 Device: Kobo	Try using Sigil to tweak the book. This allows you to see everything that's going on. You can also insert chapter breaks wherever you like (make some <h2>s)

07-07-2010, 09:01 PM	#7
takamatsu Person Posts: 5 Karma: 10 Join Date: Jul 2010 Device: Kobo	Thanks kiwikobo, I will try Sigil. I also did another experiment on chapter splitting by setting the chapter split threshold very high. This way the book is still split into chapters, but the chapters are not split into parts. The result was significantly faster load time, 1.5 minutes, down from 2 minutes. I think this indicates that a significant amount of time is spent gluing back together chapters that have been split into parts. If the Darwin book mentioned above is all one chapter, maybe the huge chapter is being glued back together on load. Truly separate smaller chapters look like a good approach.

07-08-2010, 12:19 PM	#8
goinveg Enthusiast Posts: 32 Karma: 12 Join Date: Jun 2010 Location: Meriden, CT, USA Device: Kobo, Entourage eDGe DualBook	I just downloaded a PDF version from http://mises.org/books/mespm.pdf. Calibre converted it as one long chapter - bookmarked near the end. But it is only 1.2MB in size and loads quickly.

07-09-2010, 03:54 AM	#9
takamatsu Person Posts: 5 Karma: 10 Join Date: Jul 2010 Device: Kobo	Sigil turned out to be useful. I used it to delete the chapters belonging to "Man, Economy, and State" since I had finished it, leaving me with just "Power and Market". This slimmed down version loads quickly. goinveg, I can see why this book converted well from pdf - it is basically linear. I have tried converting several books before that have margin notes and it ended up a confusing mish mash.

Advert

Advert