Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > Kobo Reader

Notices

Reply
 
Thread Tools Search this Thread
Old 07-16-2020, 05:44 AM   #16
ndriver
Member
ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.
 
Posts: 18
Karma: 7940
Join Date: Apr 2013
Device: Kobo Glo; Kobo AuraHD; Kobo Aura One
After posting my last message, I also gave it a second look and found the content table. I also saw the information was there, ready to use...
Though for some reason my content table is really small, it has only the data for the last 30 days or so. Maybe I did something in the past that wiped it out.

With the hard part out-of-the-way and after Quoth's minimalist post questioning why not simply use Calibre, I decided to do just that.

In order to make all this work wirelessly, I use bash scripts that run in the Kobo device, which communicates through SSH to my little linux server that hosts Calibre.
The scripts are called by using the excellent NickelMenu, so everything is well-integrated in the Kobo main menu.

Next weekend I will try to implement some simple graphs, which will show how much was read in a specific week/month etc. These will have to be hosted outside Calibre of course.

The docker suggestion was so that everyone could deploy locally, in a safe and easy way, a SSH server and some web server.

But on second thought, I guess only the maniac people who are interested in stats will bother with this. A container is probably over complicating things for such small gains....
Attached Thumbnails
Click image for larger version

Name:	stats.jpg
Views:	97
Size:	176.9 KB
ID:	180677  

Last edited by ndriver; 07-16-2020 at 09:11 AM.
ndriver is offline   Reply With Quote
Old 07-16-2020, 09:54 AM   #17
rigogsilva
Enthusiast
rigogsilva doesn't litterrigogsilva doesn't litter
 
Posts: 29
Karma: 136
Join Date: May 2020
Location: US
Device: kobo libra h20, kobo aura h20, kindle, boox nova 2
Quote:
Originally Posted by ndriver View Post
After posting my last message, I also gave it a second look and found the content table. I also saw the information was there, ready to use...
Though for some reason my content table is really small, it has only the data for the last 30 days or so. Maybe I did something in the past that wiped it out.

With the hard part out-of-the-way and after Quoth's minimalist post questioning why not simply use Calibre, I decided to do just that.

In order to make all this work wirelessly, I use bash scripts that run in the Kobo device, which communicates through SSH to my little linux server that hosts Calibre.
The scripts are called by using the excellent NickelMenu, so everything is well-integrated in the Kobo main menu.

Next weekend I will try to implement some simple graphs, which will show how much was read in a specific week/month etc. These will have to be hosted outside Calibre of course.

The docker suggestion was so that everyone could deploy locally, in a safe and easy way, a SSH server and some web server.

But on second thought, I guess only the maniac people who are interested in stats will bother with this. A container is probably over complicating things for such small gains....
About the content table being small, I wonder if kobo does some housekeeping in the sqlite db after a while.

I am curious about the shell script. When using the script, do you trigger it from the Kobo through the menu? (that is awesome!). So, technically I could do the same but point to my cloud server instead (cool). I am looking at the NickelMenu tool. Thanks!

I mean, the data doesn't have to be pushed to my server. I could definitely put it out there as a local web app and people can use locally (saves me some cloud money).

I will take a closer look at the calibre plugin.

Last edited by rigogsilva; 07-16-2020 at 10:14 AM.
rigogsilva is offline   Reply With Quote
Old 07-16-2020, 10:12 AM   #18
ndriver
Member
ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.
 
Posts: 18
Karma: 7940
Join Date: Apr 2013
Device: Kobo Glo; Kobo AuraHD; Kobo Aura One
Yes, you can easily call scripts by using the menu.

There are some wizards in this forum who provide us these awesome tools.

Until recently I was using kfmon to run scripts, now I am using NickelMenu, which is even better and easier to use.

Check this post:

https://www.mobileread.com/forums/sh...d.php?t=329525
ndriver is offline   Reply With Quote
Old 07-16-2020, 10:51 AM   #19
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 19,572
Karma: 33152421
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by rigogsilva View Post
About the content table being small, I wonder if kobo does some housekeeping in the sqlite db after a while.
The content table contains at a record for each book known to the device, plus a record for each ToC entry for books actually on the device, and an extra record per internal file for kepubs. "Known to the device" is sideloaded books, purchased books (either downloaded or not), preview and recommendations. The latter are cleaned up frequently when the device is synced. If the table is small, it means there aren't many books on the device.
davidfor is offline   Reply With Quote
Old 07-16-2020, 11:21 AM   #20
ndriver
Member
ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.
 
Posts: 18
Karma: 7940
Join Date: Apr 2013
Device: Kobo Glo; Kobo AuraHD; Kobo Aura One
Quote:
Originally Posted by davidfor View Post
The content table contains at a record for each book known to the device, plus a record for each ToC entry for books actually on the device, and an extra record per internal file for kepubs. "Known to the device" is sideloaded books, purchased books (either downloaded or not), preview and recommendations. The latter are cleaned up frequently when the device is synced. If the table is small, it means there aren't many books on the device.

I described my situation in a misleading way.
I meant the query of the content table after filtering the results of the ReadStatus column to either 1 or 2 (reading and finished).
There are books I have finished a year ago, which report the correct time. Others finished very recently also seem to be correct. But some books between these periods either disappeared or are reporting a very low value.

I had to repair my Kobo account at some point in the past, because there were a lot of missing covers. And I remember that it had to download a lot of books that were already in my device. Maybe it is related to that.

Last edited by ndriver; 07-16-2020 at 11:49 AM.
ndriver is offline   Reply With Quote
Old 07-16-2020, 08:54 PM   #21
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 19,572
Karma: 33152421
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by ndriver View Post
I described my situation in a misleading way.
I meant the query of the content table after filtering the results of the ReadStatus column to either 1 or 2 (reading and finished).
There are books I have finished a year ago, which report the correct time. Others finished very recently also seem to be correct. But some books between these periods either disappeared or are reporting a very low value.

I had to repair my Kobo account at some point in the past, because there were a lot of missing covers. And I remember that it had to download a lot of books that were already in my device. Maybe it is related to that.
The reading time is only there for books actually read on the device. For purchased books, it isn't synced in any way. If you read on multiple devices or the Kobo apps, each place only has the time spent reading there. And if you reset the database on the device (sign out or factory reset), you will lose the time spent.

I'm not sure what the repair will do. It probably depends on whether it has to recreate the database entries. I think it checks everything and then replaces anything that is questionable. The latter would probably mean the reading time and related stats would be lost.
davidfor is offline   Reply With Quote
Old 07-20-2020, 01:08 PM   #22
ndriver
Member
ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.
 
Posts: 18
Karma: 7940
Join Date: Apr 2013
Device: Kobo Glo; Kobo AuraHD; Kobo Aura One
Just for fun, I proceeded with the test of trying to implement a solution to keep an eye on and plot my reading patterns.

This is self hosted. The e-reader sends the result of a query to my PC, where it is parsed.
(A shared site would not be something I would use because of the privacy issues that are inherent to that type of setup.)

I can then lookup and search the results in a web page (though at the moment it is in need of a bit of styling...). After selecting the date range, a stacked bar chart is displayed, showing what and how much was read on a daily, weekly or monthly scale.

Main drawbacks are that ideally the script which sends the query result to the PC should be run everyday the Kobo is used and secondly the results are only valid in scenarios where you use exclusively one device, since this data is not synced to the Kobo servers as davidfor warned on the previous post.
Attached Thumbnails
Click image for larger version

Name:	plot.png
Views:	79
Size:	78.3 KB
ID:	180816  

Last edited by ndriver; 07-20-2020 at 07:57 PM.
ndriver is offline   Reply With Quote
Old 07-21-2020, 12:37 PM   #23
NiLuJe
BLAM!
NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.
 
NiLuJe's Avatar
 
Posts: 10,599
Karma: 18345810
Join Date: Jun 2010
Location: Paris, France
Device: Kindle 2i, 3g, 4, 5w, PW & PW2; Kobo H2O & Forma
Oh, hai, KOReader feature ^^.
NiLuJe is offline   Reply With Quote
Old 07-22-2020, 07:18 AM   #24
ndriver
Member
ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.ndriver knows the square root of minus one.
 
Posts: 18
Karma: 7940
Join Date: Apr 2013
Device: Kobo Glo; Kobo AuraHD; Kobo Aura One
Quote:
Originally Posted by NiLuJe View Post
Oh, hai, KOReader feature ^^.
Indeed, I ought to try KoReader again, I haven't done so in ages...

What made me return to Nickel, if I remember correctly, was Pocket, yeah I Know...It is closed source and there is also Wallabag, but I'm so used to Pocket that I just wanted it back.

Would you point me in the right direction regarding the execution of my little script over here?
I would like to trigger the script just before Nickel is suspended. Is there any hook available to make that possible?
Thanks for your great work.

Last edited by ndriver; 07-22-2020 at 11:05 AM.
ndriver is offline   Reply With Quote
Old 07-22-2020, 01:25 PM   #25
NiLuJe
BLAM!
NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.
 
NiLuJe's Avatar
 
Posts: 10,599
Karma: 18345810
Join Date: Jun 2010
Location: Paris, France
Device: Kindle 2i, 3g, 4, 5w, PW & PW2; Kobo H2O & Forma
Not yet. Sherman is working on something that may help, but keep in mind that when a device goes to sleep, it goes to *suspend*, so that's generally a terrible trigger to do stuff, because you can't actually do stuff ;p.
NiLuJe is offline   Reply With Quote
Old 07-22-2020, 06:09 PM   #26
sherman
Guru
sherman ought to be getting tired of karma fortunes by now.sherman ought to be getting tired of karma fortunes by now.sherman ought to be getting tired of karma fortunes by now.sherman ought to be getting tired of karma fortunes by now.sherman ought to be getting tired of karma fortunes by now.sherman ought to be getting tired of karma fortunes by now.sherman ought to be getting tired of karma fortunes by now.sherman ought to be getting tired of karma fortunes by now.sherman ought to be getting tired of karma fortunes by now.sherman ought to be getting tired of karma fortunes by now.sherman ought to be getting tired of karma fortunes by now.
 
Posts: 617
Karma: 1223950
Join Date: Aug 2008
Location: Auckland - NZ
Device: Kobo Aura H2O, Kobo Forma
Quote:
Originally Posted by NiLuJe View Post
Not yet. Sherman is working on something that may help, but keep in mind that when a device goes to sleep, it goes to *suspend*, so that's generally a terrible trigger to do stuff, because you can't actually do stuff ;p.
I haven't tested any of these yet, but Nickel does emit the following signals:
Code:
SIGNAL :: void N3PowerWorkflowManager::aboutToSleep()
SIGNAL :: void N3PowerWorkflowManager::resumingFromSleep()
SIGNAL :: void N3PowerWorkflowManager::resumedFromSleep()
SIGNAL :: void N3PowerWorkflowManager::aboutToPowerOff()
SIGNAL :: void N3PowerWorkflowManager::aboutToReboot()
So I think it will be possible to listen for them using my current project. Just got to plumb them in...
sherman is offline   Reply With Quote
Old 07-29-2020, 10:42 AM   #27
rigogsilva
Enthusiast
rigogsilva doesn't litterrigogsilva doesn't litter
 
Posts: 29
Karma: 136
Join Date: May 2020
Location: US
Device: kobo libra h20, kobo aura h20, kindle, boox nova 2
Quote:
Originally Posted by ndriver View Post
Just for fun, I proceeded with the test of trying to implement a solution to keep an eye on and plot my reading patterns.

This is self hosted. The e-reader sends the result of a query to my PC, where it is parsed.
(A shared site would not be something I would use because of the privacy issues that are inherent to that type of setup.)

I can then lookup and search the results in a web page (though at the moment it is in need of a bit of styling...). After selecting the date range, a stacked bar chart is displayed, showing what and how much was read on a daily, weekly or monthly scale.

Main drawbacks are that ideally the script which sends the query result to the PC should be run everyday the Kobo is used and secondly the results are only valid in scenarios where you use exclusively one device, since this data is not synced to the Kobo servers as davidfor warned on the previous post.
Ndriver, that is pretty cool. That exactly what I am kind of looking for.

After all the responses about private concerns for sharing data, I have decided to create something different than a website where I would need to keep users' data.

I am thinking about creating a:
- python pip library that will easily export the data out of kobo. I it will also get the page turns out of it since that is encoded. The pip library will decode.
- support exporting data from multiple KoboReader.db files. This will be great if you have backups.
- this pip library will also allow us to run notebooks in a docker container. Notebooks will provide very detail information about your reading. There are so many plots I want to do, the one you have in that picture is one of them
- all data will be kept locally so no sharing of data going on. Keeping your data and having it exported from the device so you can save it.

Maybe one day, I can use ndriver script to automate exporting the data automatically from kobo without a computer.
rigogsilva is offline   Reply With Quote
Old 07-30-2020, 09:09 AM   #28
rigogsilva
Enthusiast
rigogsilva doesn't litterrigogsilva doesn't litter
 
Posts: 29
Karma: 136
Join Date: May 2020
Location: US
Device: kobo libra h20, kobo aura h20, kindle, boox nova 2
Quote:
Originally Posted by NiLuJe View Post
Oh, hai, KOReader feature ^^.
Tried Koreader. That is pretty good.
rigogsilva is offline   Reply With Quote
Reply

Tags
annotations, ebook, highlights, kobo, site

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Desktop App KoboNotes.com, a new website to organize your kobo highlights and annotations fferreira Kobo Developer's Corner 26 07-27-2020 08:13 AM
theKoboNotes - a new website for your Kobo's highlights and annotations fferreira Kobo Reader 10 09-30-2019 12:16 PM
kindle forgets progress and highlights each time I open a book Braid Amazon Kindle 2 11-27-2017 03:16 PM
Management of annotations with Kobo and Calibre Ziggi Plugins 2 11-13-2015 01:10 PM
Annotations/highlights on Kobo MHaensel Kobo Reader 3 04-05-2015 12:44 AM


All times are GMT -4. The time now is 03:08 AM.


MobileRead.com is a privately owned, operated and funded community.