[Device Plugin] Kobo Touch Extended Driver - Page 100

patrik · 04-07-2016, 07:25 AM

Quote:

Originally Posted by davidfor

[*]Tap "Highlight" on the menu. The menu closes and the underlines change to grey background on the text.

At which point does it stop? I assumed the third step, and that the last wasn't done, but I am not completely sure.

Exactly.

As can be seen in this video I made last time I asked about it here.
https://dl.dropboxusercontent.com/u/...ghlighting.mp4

Quote:

And some more questions:

As you are seeing a change when you change font settings, what font and size are you using?
If you close the book and reopen it, does the highlighting start working again?
After changing the font size and the highlighting starts to work again, how many highlights can you get? About 20 again? Or less?

I'm using Georgia.
Is there a way to see exactly which font size I'm using?

Usually like this, or one step smaller:

I will test more, but I'm guessing around 8-12 highlights before the problem arises again.

Thanks David

davidfor · 04-07-2016, 10:00 AM

Quote:

Originally Posted by patrik

Exactly.

As can be seen in this video I made last time I asked about it here.
https://dl.dropboxusercontent.com/u/...ghlighting.mp4

That video tells me what is going on. Or at least one reason the highlights are disappearing.

If you look at the underline while the menu is displayed, the space before the sentence is selected. That's the problem. The kepub format has spans wrapping each sentence. But, the space between sentences isn't included. When selecting text, if start or end of the selection is at a different level of the spans, the highlight won't get saved. In this case, the space is outside the spans and that causes a problem.

I remember this being discussed some time ago, and I thought it had been fixed in the driver. It might not be all that is going on, but it is probably a contributing factor.

Quote:

I'm using Georgia.
Is there a way to see exactly which font size I'm using?

Usually like this, or one step smaller:

The simplest way is to look in the config file on the device. Look for the line with "readingFontSize". But, that is roughly what I am using.

Quote:

I will test more, but I'm guessing around 8-12 highlights before the problem arises again.

OK.

Another thought is to look for a stack dump on the device. This will be in the .kobo directory and have the name "stack_nn.log" where "nn" is two digits. These can be produced by the firmware when there is a problem. If there is one, it can give a hint of what is going on, but we can also give it to Kobo to look at.

patrik · 04-07-2016, 10:37 AM

Quote:

Originally Posted by davidfor

That video tells me what is going on. Or at least one reason the highlights are disappearing.

If you look at the underline while the menu is displayed, the space before the sentence is selected. That's the problem. The kepub format has spans wrapping each sentence. But, the space between sentences isn't included. When selecting text, if start or end of the selection is at a different level of the spans, the highlight won't get saved. In this case, the space is outside the spans and that causes a problem.

Hmm, ok, I will check for this when doing the highlights, thanks.

Kind of surprised that there is not more mentioning of this. I can't be the only one reading a lot of non-fiction on sideloaded kepubs.

Quote:

The simplest way is to look in the config file on the device. Look for the line with "readingFontSize". But, that is roughly what I am using.

32

Quote:

Another thought is to look for a stack dump on the device. This will be in the .kobo directory and have the name "stack_nn.log" where "nn" is two digits. These can be produced by the firmware when there is a problem. If there is one, it can give a hint of what is going on, but we can also give it to Kobo to look at.

No such logs.

Thanks,

patrik · 04-07-2016, 02:06 PM

Ok, I just played around doing highlights, actually trying to start and/or end on a space (which is not easy...).

All these highlights worked. The space doesn't seem to matter.

(The video was taken on an earlier firmware, maybe they fixed that part?)

But I will be sure to note what the highlights look like that doesn't work.

davidfor · 04-07-2016, 10:42 PM

I tested this before my post on my Glo HD with firmware 3.19.5761. Did you make sure the spaces were between the sentences?

The code the extended driver generates looks like:

Code:

<span class="koboSpan" id="kobo.3.1">Sentence one.</span> <span class="koboSpan" id="kobo.3.2">Sentence two.</span> <span class="koboSpan" id="kobo.3.3">Sentence three.</span>

The kepub bookmarking uses those spans and the ids. If the start or end of a selection is outside the spans, then the highlight cannot be save. Selecting the space between the spans as the start of the highlight will trigger this. A side effect of this is that when you select across the space, say from the word "one" to the word "two", when the highlight is drawn after it has been save, the space between the sentences is not highlighted.

To see this, put the attached kepub your device. This has been converted from an epub. Then I edited the "Forward" to move the spaces between the spans into the spans. If you select a sentence and make sure you include the space before it, the highlight will be saved.

If you go to the chapter "Font Display" and do the same thing, the highlight won't be saved. And if you do a highlight that goes from the middle of one sentence to the middle of the next, the highlight will be saved, but the space in between the sentences won't be marked.

To make this easier to do and see, I also edited the first paragraph in the chapter "Special Characters". In this case, I removed the span wrapping the second sentence. If you try to make a highlight from somewhere in the first sentence, to somewhere in the second, it won't save. Or from somewhere in the second sentence to anywhere else in the chapter, it won't save. And if you select from somewhere in the first sentence to somewhere in the third sentence, the highlight will save. But, when it is redrawn, the second sentence will not be highlighted.

From what you are describing, there might be something else going on. But, it is a problem and eliminating it might expose the real problem. And it is possible that these errors are causing other problems in the highlighting engine (memory leaks etc) and eliminating these errors will mean the other problems don't occur.

patrik · 04-08-2016, 01:39 PM

Ok, I continued to read some more, and everything worked fine in the beginning (it seems it always works fine first, and then it starts to not work, and get more and more errournous...).

After a while this happened:

As you can see the underline does not go all the way (is this because it goes over separate spans?).
Hightlight didn't stick.

Choosing a part of the text, the hightlight worked.
Trying to add the second part, made this:

No matter what I did, I could not get all the text to work.

I tried to do one word at a time.

That seemed to work.

On the next page, it worked to mark everything.

On the page after that, which looks the same, I couldn't get the whole page.

After that I gave up and continued reading in the epub-version of the same book (which I simple file copied on the device). Worked fine with highlights for the time I read.

Any more thoughts?

davidfor · 04-08-2016, 10:52 PM

patrik: Yes, some of those screen shots show the problem with the space between sentences not included in the spans. The third (after "Egghead.") and fourth (after "attitudes." ) shot show them the best.

The rest of it is weird. I don't remember seeing the missing lines while doing the selections. I think this a point were we need to see the book and see what is in it. Is there anywhere we can get it? A sample might be enough. You can also use Scramble Book to post a version were all the text has been scrambled but the formatting should still be the same. Or you can PM me to discuss other methods of exchange.

patrik · 04-09-2016, 04:46 AM

Since I have had the problem with highlights, as far as I can remember, ever since I bought the Kobo, I tried with a book coming from our "local" library, thus accessible to anyone. I just started to flip pages and make a highlight or two on each page.

This is highlight number 11:

(All 10 first highlights worked fine, even those starting with a space.)

I assume the book should be ok, regarding formatting/code.

J.S. Fletcher - The Middle Temple Murder
https://www.mobileread.com/forums/showthread.php?t=29405

If you still think it would be helpful to look at the book I was reading, I'm happy to send it to you via pm.

jgoguen · 04-09-2016, 09:30 AM

Can you post the converted KePub of that book here? Just to make it easy for me, gives me a specific example to look at.

davidfor · 04-09-2016, 10:20 AM

I've grabbed that and converted it. It might have been better to start with an epub rather than a PRC. But, looking at the generated code, there are no problems with it. It does use "div"s instead of "p"'s for the paragraphs, but I don't think that is the problem.

I went straight to that line and selected it. I didn't get the same missing underline. It was just the last couple of characters. And when I tried to save it, the highlight didn't save. This was the first selection I tried.

So, I did some experimenting. Are all the problems you are having when you select the end of a paragraph? If I make sure the selecting doesn't end at the end of a paragraph, then there is no problem. If it does end with the end of a paragraph, sometimes there is. And it seems to happen more often if the paragraph ends with two punctuation characters.

Unfortunately, it doesn't seem to be as simple as "select to the end of the paragraph". It seems that if I stop the selection at the right place, the highlight will be saved. I don't have a feel for where this spot is.

I also tested with a purchased kepub. I was able to reproduce it there as well. This happens to be DRM free, so I was able to look at the code. Again clean and this one has "p" for the paragraphs.

So, I'm down to there being a bug in the selection engine of the kepub renderer. There is a problem with the spaces between sentences and hence the spans when the extended driver creates the kepub. That can be fixed. But, the rest is up to Kobo to sort out.

patrik · 04-09-2016, 10:32 AM

Quote:

Originally Posted by jgoguen

Can you post the converted KePub of that book here? Just to make it easy for me, gives me a specific example to look at.

Sure, here is the kepub of The Middle Temple Murder: https://dl.dropboxusercontent.com/u/...her.kepub.epub

Thanks for taking a look at this.

patrik · 04-09-2016, 10:39 AM

Quote:

Originally Posted by davidfor

Are all the problems you are having when you select the end of a paragraph?

No, at least I don't think so.

Quote:

I also tested with a purchased kepub. I was able to reproduce it there as well. This happens to be DRM free, so I was able to look at the code. Again clean and this one has "p" for the paragraphs.

So, I'm down to there being a bug in the selection engine of the kepub renderer. There is a problem with the spaces between sentences and hence the spans when the extended driver creates the kepub. That can be fixed. But, the rest is up to Kobo to sort out.

Ok, thanks for testing. I guess the good news is that it's not me that is going nuts...

, it would probably have been easier if there was a bug in the calibre-plugin as I'm sure jgoguen and/or you could have fixed that.

Have you seen any correlation to the amount of, either time, or "things" you do in the book before the problem arises? Or can you make it happen directly? For me it seems that the problem occurs after a (fairly short) while. Is there some utility to check memory usage during reading/highlighting? (In case there is some memory leak that somehow triggers this.)

Is this something you can bring to Kobo? (Last time when I reported this all I got was them telling me to do a factory reset...)

Thanks David, you give amazing support, Kobo really should hire you.

jgoguen · 04-09-2016, 10:50 AM

Taking a look at this and a couple other converted books, it looks like the most obvious difference is that in a purchased KePub the space between sentences is included in the next kobo span:

Code:

<p class="para"><span id="kobo.5.1">The Guild gardens stretched out before her.</span><span id="kobo.5.2"> The University building loomed to the right.</span></p>

While in a converted KePub it's between spans:

Code:

<p class="calibre8"><span class="koboSpan" id="kobo.14.1">Lorkin met the man’s eyes and shrugged.</span> <span class="koboSpan" id="kobo.14.2">“Hardly.”</span></p>

I'll take a look at fixing that as soon as I can, but I can't say how soon that'll be.

jgoguen · 04-09-2016, 07:07 PM

I've uploaded a new version that puts the whitespace between sentences in the same place as Kobo KePubs do. I haven't had a chance to try out highlighting yet, once calibre offers the update (next couple hours) give it a try and let me know how it goes.

patrik · 04-10-2016, 03:22 AM

Thanks jgoguen!

I just updated to 2.6.0 and haven't done any reading/highlighting yet, but I did a quick comparison between the old kepub and the new. Maybe I'm doing something wrong but as far as I can see, they are the same regarding spaces between spans.

Code:

<span class="koboSpan" id="kobo.9.1">&nbsp;&nbsp;&nbsp;&nbsp;The barrister and the journalist, left thus unceremoniously on a crowded pavement, looked at each other.</span> <span class="koboSpan" id="kobo.9.2">Breton laughed.</span>

I will read and make highlights later and see if there is a difference.

Thanks for the amazingly quick update!

04-09-2016, 04:46 AM	#1493
patrik Guru Posts: 682 Karma: 4568205 Join Date: Jan 2010 Location: Sweden Device: Kobo Forma	Since I have had the problem with highlights, as far as I can remember, ever since I bought the Kobo, I tried with a book coming from our "local" library, thus accessible to anyone. I just started to flip pages and make a highlight or two on each page. This is highlight number 11: (All 10 first highlights worked fine, even those starting with a space.) I assume the book should be ok, regarding formatting/code. J.S. Fletcher - The Middle Temple Murder https://www.mobileread.com/forums/showthread.php?t=29405 If you still think it would be helpful to look at the book I was reading, I'm happy to send it to you via pm. Last edited by patrik; 04-09-2016 at 08:02 AM.

04-09-2016, 10:50 AM	#1498
jgoguen Generally Awesome Person Posts: 1,100 Karma: 2191133 Join Date: Jan 2013 Location: /dev/kmem Device: Kobo Clara HD, Kindle Oasis	Taking a look at this and a couple other converted books, it looks like the most obvious difference is that in a purchased KePub the space between sentences is included in the next kobo span: Code: <p class="para"><span id="kobo.5.1">The Guild gardens stretched out before her.</span><span id="kobo.5.2"> The University building loomed to the right.</span></p> While in a converted KePub it's between spans: Code: <p class="calibre8"><span class="koboSpan" id="kobo.14.1">Lorkin met the man’s eyes and shrugged.</span> <span class="koboSpan" id="kobo.14.2">“Hardly.”</span></p> I'll take a look at fixing that as soon as I can, but I can't say how soon that'll be.

04-10-2016, 03:22 AM	#1500
patrik Guru Posts: 682 Karma: 4568205 Join Date: Jan 2010 Location: Sweden Device: Kobo Forma	Thanks jgoguen! I just updated to 2.6.0 and haven't done any reading/highlighting yet, but I did a quick comparison between the old kepub and the new. Maybe I'm doing something wrong but as far as I can see, they are the same regarding spaces between spans. Code: <span class="koboSpan" id="kobo.9.1">    The barrister and the journalist, left thus unceremoniously on a crowded pavement, looked at each other.</span> <span class="koboSpan" id="kobo.9.2">Breton laughed.</span> I will read and make highlights later and see if there is a difference. Thanks for the amazingly quick update!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Kobo Device Driver Update	davidfor	Devices	284	05-24-2019 05:24 PM
[Device Plugin] Kindle 2, 3, 4, Touch Device Interface MBP Update Mod	cryzed	Plugins	7	10-28-2012 04:58 PM
[Device Interface Plugin] Update for Nook Color Driver	jmricker	Plugins	0	10-22-2011 10:11 AM
Touch Kobo Touch Extended 2 Year Warranty - Is it worth the $34.99?	EverC	Kobo Reader	11	08-31-2011 11:47 PM
Touch Kobo Touch Extended Warranty Program	SensualPoet	Kobo Reader	1	07-17-2011 04:08 AM

04-07-2016, 02:06 PM	#1489
patrik Guru Posts: 682 Karma: 4568205 Join Date: Jan 2010 Location: Sweden Device: Kobo Forma	Ok, I just played around doing highlights, actually trying to start and/or end on a space (which is not easy...). All these highlights worked. The space doesn't seem to matter. (The video was taken on an earlier firmware, maybe they fixed that part?) But I will be sure to note what the highlights look like that doesn't work.

04-08-2016, 01:39 PM	#1491
patrik Guru Posts: 682 Karma: 4568205 Join Date: Jan 2010 Location: Sweden Device: Kobo Forma	Ok, I continued to read some more, and everything worked fine in the beginning (it seems it always works fine first, and then it starts to not work, and get more and more errournous...). After a while this happened: As you can see the underline does not go all the way (is this because it goes over separate spans?). Hightlight didn't stick. Choosing a part of the text, the hightlight worked. Trying to add the second part, made this: No matter what I did, I could not get all the text to work. I tried to do one word at a time. That seemed to work. On the next page, it worked to mark everything. On the page after that, which looks the same, I couldn't get the whole page. After that I gave up and continued reading in the epub-version of the same book (which I simple file copied on the device). Worked fine with highlights for the time I read. Any more thoughts?

04-08-2016, 10:52 PM	#1492
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	patrik: Yes, some of those screen shots show the problem with the space between sentences not included in the spans. The third (after "Egghead.") and fourth (after "attitudes." ) shot show them the best. The rest of it is weird. I don't remember seeing the missing lines while doing the selections. I think this a point were we need to see the book and see what is in it. Is there anywhere we can get it? A sample might be enough. You can also use Scramble Book to post a version were all the text has been scrambled but the formatting should still be the same. Or you can PM me to discuss other methods of exchange.

04-09-2016, 09:30 AM	#1494
jgoguen Generally Awesome Person Posts: 1,100 Karma: 2191133 Join Date: Jan 2013 Location: /dev/kmem Device: Kobo Clara HD, Kindle Oasis	Can you post the converted KePub of that book here? Just to make it easy for me, gives me a specific example to look at.

04-09-2016, 10:20 AM	#1495
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	I've grabbed that and converted it. It might have been better to start with an epub rather than a PRC. But, looking at the generated code, there are no problems with it. It does use "div"s instead of "p"'s for the paragraphs, but I don't think that is the problem. I went straight to that line and selected it. I didn't get the same missing underline. It was just the last couple of characters. And when I tried to save it, the highlight didn't save. This was the first selection I tried. So, I did some experimenting. Are all the problems you are having when you select the end of a paragraph? If I make sure the selecting doesn't end at the end of a paragraph, then there is no problem. If it does end with the end of a paragraph, sometimes there is. And it seems to happen more often if the paragraph ends with two punctuation characters. Unfortunately, it doesn't seem to be as simple as "select to the end of the paragraph". It seems that if I stop the selection at the right place, the highlight will be saved. I don't have a feel for where this spot is. I also tested with a purchased kepub. I was able to reproduce it there as well. This happens to be DRM free, so I was able to look at the code. Again clean and this one has "p" for the paragraphs. So, I'm down to there being a bug in the selection engine of the kepub renderer. There is a problem with the spaces between sentences and hence the spans when the extended driver creates the kepub. That can be fixed. But, the rest is up to Kobo to sort out.

04-09-2016, 07:07 PM	#1499
jgoguen Generally Awesome Person Posts: 1,100 Karma: 2191133 Join Date: Jan 2013 Location: /dev/kmem Device: Kobo Clara HD, Kindle Oasis	I've uploaded a new version that puts the whitespace between sentences in the same place as Kobo KePubs do. I haven't had a chance to try out highlighting yet, once calibre offers the update (next couple hours) give it a try and let me know how it goes.