Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Development

Notices

Reply
 
Thread Tools Search this Thread
Old 02-16-2013, 04:09 AM   #16
kc7zzv
Member
kc7zzv began at the beginning.
 
Posts: 12
Karma: 10
Join Date: Jan 2009
Device: Handspring Palm
Quote:
Originally Posted by davidfor View Post
My apologies for pointing out an error in your post and explaining it. But, as it was the first time I referred to this in this thread, I think "Belabouring this point", is a bit much.

Now if you had accused me of being pedantic in my explanation, then I will plead guilty. I was trying to make sure you understood what was going on. The only issue with the "close enough" is that Kobo might decide this is a bug (reading a epub 3 attribute in an epub 2 file) and fix it.
At least twice I've said that "I think Kobo might do X non-standards conforming thing, but it's close enough to being a normal epub that I won't worry about it." This was about the cover piece, and mis-remembering that the Kobo uses a custom attribute in the span.

My point has been that using the epub input code will work fine. I will do more investigation before I start programming in earnest. I understand that you want to help keep me technically accurate, but correcting me on things which are not going to be technical problems, and that aren't likely to actually be implemented in the broken way I stated does nothing except derail this discussion.

I'm sorry I snapped at you, but I was only trying to state the worst case that I remembered might be true, as a way of "padding my estimates".

Basically, if I say, "Even if X is true, we should still be fine," please don't correct me to tell me X is false, unless you think that's something I could somehow miss later.


Quote:
Originally Posted by davidfor View Post
But, to belabour a point before I continue. I think that having calibre capable of putting properly formatted kepubs on the Kobo devices is a good idea. But, I am not convinced that having kepub as a format inside calibre is very useful. And I think don't think it is more useful than the driver method.
Actually, I had found the plugin, and I learned it existed, but I think I got it confused with an older branch. I've been testing it, and been happy to find that it does much of what I wanted, and more than I expected.


Quote:
Originally Posted by davidfor View Post
I am not trying to convince you that splitting a epub or kepub into smaller chunks won't be faster when turning the page. I am fairly sure that it will be. But there is a minimum size that makes it slower as going from one file in the epub/kepub to another is slower.

From what I see for performance and usability, the best split is at chapter marks [pedantic me: where it says "Chapter x" or "Part x" or just "x"]. It is rare to find a book that has chapters large enough to want to split them into smaller files. But, it is common to find epubs that have not been split at chapters.
I have an epub (not kepub) generated by Calibre, where Calibre didn't find the chapter marks. (This isn't really Calibre's fault, since even a human would have had a hard time) It contains 653k characters, and 7.9k lines. Aside from the header and footer, half the lines are a single "br" tag, and the other half are lines of text beginning and ending with a "p" tag.

The file contains literally no formatting, tags or css, and no links to external css. Near the end of the book, turning the page takes 2-5 seconds near the beginning of the book (I didn't time this) but over 15 seconds near the end of the book. (I timed this)

You might argue that this is a bug in the kobo, and/or this would be fixed if the book had proper chapter marks. I agree with both of those statements, but I consider them tangent to my point.

I think this means that books of this size should have been automatically split by Calibre into smaller pieces before being sent to my Kobo. Since 700KB in size was way too long, I'm planning to try splitting at 25KB, 75KB, 100KB, 150KB, and 250KB. I'm hoping that a default of 100KB will be a "small enough" default piece to fix this problem, and large enough that users won't get annoyed at "too many" page breaks. Then I want to test what effect using Kepubs has on the speed with the diffierent sizes. (I'll be testing this as soon as I manage to get my Kobo working, since this file seems to have caused it to hang.)

Quote:
Originally Posted by davidfor View Post
Why one earth would you do that? It is not needed on any of the Kobo devices to split the text into "tiny pieces". Or do we disagree on the how big "tiny" is? And if you read my posts, I am a big advocate of splitting at chapter breaks. That will not put page-breaks in ugly places.
I think this was a miscommunication. To me, "tiny pieces" is somewhere between 100KB and 20KB depending on testing.

Quote:
Originally Posted by davidfor View Post
Huh? Of course all that is the case. Why would you do any of that if the size is a big concern to you. The spans and ids have nothing to do with other readers (that was the point I was asking about) as they are only added to kepubs. Same with the kepub specific javascript. But I don't understand how this affects other readers as kepubs don't go onto them.
I think there was some confusion here. I only said that I don't want to add the Kobo Javascript to, or split the epubbs in my library. I have no objections doing that to kepubs, or epubs that will only be on my Kobo.

Firstly, I won't correct you because I think "kepub reader" is a perfectly good description. I do use "kepub renderer" in some places, but that is because it fits the discussion.


Quote:
Originally Posted by davidfor View Post
Sorry, but calibre does work very well with the Kobo device out of the box with no configuration changes. Any changes people make are because they want to fine tune it. The issues you are talking about are related to the structure of the books being read or maybe some firmware bugs.
If you believe this, then either you are pretty lucky, or I'm very unlucky, because I've used several ebooks that all took over 5 seconds per page turn.
kc7zzv is offline   Reply With Quote
Old 02-16-2013, 06:06 AM   #17
jgoguen
Generally Awesome Person
jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.
 
Posts: 1,061
Karma: 2178845
Join Date: Jan 2013
Location: /dev/kmem
Device: Kobo Clara HD, Kindle Oasis
Quote:
Originally Posted by kc7zzv View Post
I think there was some confusion here. I only said that I don't want to add the Kobo Javascript to, or split the epubbs in my library. I have no objections doing that to kepubs, or epubs that will only be on my Kobo.
If this means you're considering adding the Kobo JavaScript code to any kepubs you generate, be sure that your local copyright laws will permit you to distribute it. In general, the lack of copyright notice and license (which is the case for the official kepubs I've taken apart) means "copyright is applied by the owner, all rights reserved, no redistribution allowed". Same goes for the Kobo-specific CSS, but rhe only CSS change that seems to be important for a kepub is forcing 'margin:0'.
jgoguen is offline   Reply With Quote
Old 02-17-2013, 06:11 AM   #18
kc7zzv
Member
kc7zzv began at the beginning.
 
Posts: 12
Karma: 10
Join Date: Jan 2009
Device: Handspring Palm
Quote:
Originally Posted by jgoguen View Post
If this means you're considering adding the Kobo JavaScript code to any kepubs you generate, be sure that your local copyright laws will permit you to distribute it. In general, the lack of copyright notice and license (which is the case for the official kepubs I've taken apart) means "copyright is applied by the owner, all rights reserved, no redistribution allowed". Same goes for the Kobo-specific CSS, but rhe only CSS change that seems to be important for a kepub is forcing 'margin:0'.
Ya. I'd hoped to, but I haven't had any luck so far finding a redistributable version. I know about this, but only because of: https://github.com/jgoguen/calibre-kobo-driver/issues/7

At this point, I'm trying to find any books Kobo might have put under free distribution or something similar.
kc7zzv is offline   Reply With Quote
Old 02-17-2013, 08:35 AM   #19
jgoguen
Generally Awesome Person
jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.jgoguen ought to be getting tired of karma fortunes by now.
 
Posts: 1,061
Karma: 2178845
Join Date: Jan 2013
Location: /dev/kmem
Device: Kobo Clara HD, Kindle Oasis
The problem isn't finding free distribution books (although that's not exactly easy either) it's finding that specific file freely redistributable. Even if you find a book that you can redistribute that has the Kobo files, that probably only allows you to redistribute the whole package and not the component files.

If you do find a freely re-distributable version though, do let us all know. I'd like to be able to include it.
jgoguen is offline   Reply With Quote
Old 02-17-2013, 09:46 PM   #20
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by kc7zzv View Post
At least twice I've said that "I think Kobo might do X non-standards conforming thing, but it's close enough to being a normal epub that I won't worry about it."
Good, I am glad you realise that. I never said in any way that that wasn't the case. In fact, if you have a look at my posting history on this forum, you will find me explaining exactly that far too many times.
Quote:
This was about the cover piece, and mis-remembering that the Kobo uses a custom attribute in the span.
Well, to be precise, you made a reference to "non-conforming part that I know of is extra attributes in the span tags". I pointed out that it was valid code in an epub. You then stated that you remembered wrong and that it was a "custom attribute" in the metadata. At that point I knew what you were talking about, so I explained my understanding of that.

My apologies for explaining something that I happened to have done some research on in the past. And also my apologies for correcting some terminology in the attempt to make sure we were all talking about the same thing.
Quote:
My point has been that using the epub input code will work fine.
When have I said that wasn't going to be the case? The conversion code from epub to kepub is going to be a doddle. Especially with examples of how to add the spans that are already around. And the conversion the other way is basically just renaming the file.
Quote:
I will do more investigation before I start programming in earnest. I understand that you want to help keep me technically accurate, but correcting me on things which are not going to be technical problems, and that aren't likely to actually be implemented in the broken way I stated does nothing except derail this discussion.

I'm sorry I snapped at you, but I was only trying to state the worst case that I remembered might be true, as a way of "padding my estimates".

Basically, if I say, "Even if X is true, we should still be fine," please don't correct me to tell me X is false, unless you think that's something I could somehow miss later.
What do you think I was doing? I am not answering your posts or picking on you for sport. I was reading what you were saying and trying to understand it. I found what I considered to be errors, so I commented on them. I found things that seemed to be lack of understanding, so I commented on them. I saw some statements from you that seemed to be contradicting each other, so I commented on them to try and sort it out. I also wasn't sure I understood somethings, so I asked about those.
Quote:

I have an epub (not kepub) generated by Calibre, where Calibre didn't find the chapter marks. (This isn't really Calibre's fault, since even a human would have had a hard time) It contains 653k characters, and 7.9k lines. Aside from the header and footer, half the lines are a single "br" tag, and the other half are lines of text beginning and ending with a "p" tag.
I have done plenty of conversions where calibre didn't pick up the chapters. Generally I use Sigil to fix this afterwards.

To get a 653KB file in an epub after a conversion, you had to make a change to the conversion settings. The default split size on the 260KB. Try redoing the conversion but make sure the split size is the default or smaller.

What do you mean by "header" and "footers"? They don't makes sense in an epub. Was this a conversion from a PDF that had headers and footers that weren't removed?
Quote:
The file contains literally no formatting, tags or css, and no links to external css. Near the end of the book, turning the page takes 2-5 seconds near the beginning of the book (I didn't time this) but over 15 seconds near the end of the book. (I timed this)

You might argue that this is a bug in the kobo, and/or this would be fixed if the book had proper chapter marks. I agree with both of those statements, but I consider them tangent to my point.

I think this means that books of this size should have been automatically split by Calibre into smaller pieces before being sent to my Kobo. Since 700KB in size was way too long, I'm planning to try splitting at 25KB, 75KB, 100KB, 150KB, and 250KB. I'm hoping that a default of 100KB will be a "small enough" default piece to fix this problem, and large enough that users won't get annoyed at "too many" page breaks. Then I want to test what effect using Kepubs has on the speed with the diffierent sizes. (I'll be testing this as soon as I manage to get my Kobo working, since this file seems to have caused it to hang.)
OK, with times like that, I can understand you wanting to look at it. But, as Kovid said, size isn't the real factor. Complexity of code is.

To explain this, I mentioned a epub I have earlier. This is one I constructed specifically to test some size related questions someone else had. The book has about 30 chapters. The original version had each chapter in a separate file between 10KB and 50KB in size. Plus there were three small files for cover and other non-story things. I combine the chapters into three files that were 342KB, 296KB and 739KB in size. They are in that order in the file. When I uses this before, I was testing battery life, and didn't pay attention to page turning speed. But, I put it back on and did some quick tests.

At the beginning of the book the page turning is sub-second. At the end of the first file, it is slower, but still sub-second. At the beginning of both the other files, it is fast. The only real noticeable speed drop is at the end of the third file, the 739KB file. And then it is maybe three seconds. No were near as bad you are seeing with smaller file sizes.

The difference is that the code in this book is clean. There are no extra tags for weird things. CSS is used to handle everything nicely. The epub you are describing is badly formatted. This will cause performance problems. Cleaning this up will help a lot more that splitting the files up. But, splitting on chapters (when calibre can detect them) and using the default split size (mainly for when the chapter detection fails) will usually result in reasonable performance.

For curiosity, I also put the book on as a kepub. It didn't have the extra spans, but that shouldn't affect the performance much. Page turning on it was sub-second throughout the book.

I would be interested to see this book you are having trouble with. It would be interesting to see exactly what sort of code in the book caused this as I have never seen performance as bad as you describe. If you are willing to send it to me, send a PM with a place I can get it, or I can send email address.

As to determining an optimum split size, which device are you going to use? Touch, Glo or Mini? Or maybe the original Kobo or Kobo WiFi? And which firmware version? The performance is different on all these devices and the different firmware has had a lot of effect on performance as well.
Quote:
I think this was a miscommunication. To me, "tiny pieces" is somewhere between 100KB and 20KB depending on testing.
Yes, that is a miscommunication. I was thinking two or three pages. 100KB is a novella and 20KB is 20 or 30 pages. I usually see split files shorter than that as a 20 page chapter is on the long side. Or did you mean 200KB?
Quote:

If you believe this, then either you are pretty lucky, or I'm very unlucky, because I've used several ebooks that all took over 5 seconds per page turn.
My comment about calibre working well out of the box was related to how much configuration you stated was needed. All of those configuration items are personal desires. Calibre handles the Kobo devices very well without doing any of these.

The performance of books on the Kobo device is a different matter. These books would behave like that if you put them on without using calibre.
davidfor is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Touch .kepub.epub davidfor Kobo Reader 233 01-12-2014 08:15 AM
Kepub - how does it work? Lynx-lynx Kobo Reader 5 01-31-2013 06:09 AM
Plugin calibre epub to kepub Tumeconnais Kobo Developer's Corner 0 01-16-2013 04:26 PM
Plugin for making a summary list about your books equilan1986 Plugins 2 01-08-2013 07:21 AM
A little help and direction... stevejay Deals and Resources (No Self-Promotion or Affiliate Links) 12 02-19-2009 05:24 PM


All times are GMT -4. The time now is 01:10 AM.


MobileRead.com is a privately owned, operated and funded community.