Kobo book management site (highlights/progress/annotations)

[rigogsilva]

Hi Kobo users,

I have been building this website for myself and I would like to see if there is any interest for a site like this.

Basically I love my Kobo but I also think that it has some key features missing. Like:

Highlights management; I highlight things in my kobo and I can't export them anywhere. I would like to have a place to search my highlights per book. Same applies for annotations; all the notes I write in my kobo should be available somewhere.

The other main issue is the ability to better visualize my reading statistics with graphics. Page turns, amount of time read by day, month, year. Books read by month, year, and things like that.

Would you like to use a site like this? Here is a preview of some of what I have done already (added pics as attachments). Cheers.

[FrozenTrout]

This would be great! I still miss Readmill, even though it’s been dead for years; it was a combination ebook app and social network. Social part was kind of like Goodreads if it didn’t suck, with a really nice UI that showed time read, highlights and annotations, etc etc.

[MGlitch]

Hasn’t this been tried before to uh less than great results because it requires sharing your database?

https://www.mobileread.com/forums/sh...d.php?t=323167 (Among other posts by the op of the linked thread which delve further into this)

[rigogsilva]

Originally Posted by MGlitch

Hasn’t this been tried before to uh less than great results because it requires sharing your database?

https://www.mobileread.com/forums/sh...d.php?t=323167 (Among other posts by the op of the linked thread which delve further into this)

MGlitch, thanks for sharing this. I was not aware of that before trying to create this.

Yes, I believe security would definitely be an issue. To mitigate some of that, I was thinking of creating a tool that would run in your local machine that would pull just the data needed out of the database. So, no need to share the whole database which has emails and other personal info.

After pulling the data, this "tool" would show to the user with checkboxes where the user can accept and decline sharing that specific data (e.g.: percentage read, and others).

In regards to storing the data. The data can be anonymized, so there is no email or other personal info attached to the user; just the user id.

Thoughts about that?

[rigogsilva]

Originally Posted by FrozenTrout

This would be great! I still miss Readmill, even though it’s been dead for years; it was a combination ebook app and social network. Social part was kind of like Goodreads if it didn’t suck, with a really nice UI that showed time read, highlights and annotations, etc etc.

FrozenTrout, wow. I had never heard of http://readmill.com/. Seems like they terminated their services based on the note on that site . I wish I could see what it looked like or use it.

Yeah, the social network part is a bit tough. I do wish there was something better than goodreads. For now I was just thinking of tackling the issues of tracking books read, highlights and notes, and book statistics.

[joaomcoelho]

Originally Posted by rigogsilva

MGlitch, thanks for sharing this. I was not aware of that before trying to create this.

Yes, I believe security would definitely be an issue. To mitigate some of that, I was thinking of creating a tool that would run in your local machine that would pull just the data needed out of the database. So, no need to share the whole database which has emails and other personal info.

After pulling the data, this "tool" would show to the user with checkboxes where the user can accept and decline sharing that specific data (e.g.: percentage read, and others).

In regards to storing the data. The data can be anonymized, so there is no email or other personal info attached to the user; just the user id.

Thoughts about that?

Hello,
First of all congrats on your work! It is very similar to what we have on thekobonotes.

Since I am the developer of thekobonotes.com I have to just clarify "the security issue" that was addressed. When you upload your notes on our app, the .sqlite file is processed on the browser! We then only upload the data related to the annotations and highlights to the server so you can access them everywhere (including in our new mobile app). This way any sensitive data that is on the kobo sqlite file stays on your browser (on the client) and we do not have access and we do not store it in any way on our server. This was an important aspect that we addressed in our app for security reasons.

Again congrats on your work

All the best

[Quoth]

Security and privacy are huge issues.
I wouldn't go near any such service.
The Browser is not a secure and private program either. Especially when you add more than basic HTML to the page.
Years of working in computer security and managing programmers as well as programming has shown me that even if the originator has the best motives it will end badly for the users.

I decided too that I'd stop posting what books I was reading and what I thought of them anywhere because that information isn't just going to or informing fellow readers. So now I only communicate such stuff to friends and family and by private internet means.
There is a good reason that Amazon owns GoodReads and IMDB. Also that Google hijacked newsgroups and runs YouTube.

[rigogsilva]

Originally Posted by joaomcoelho

Hello,
First of all congrats on your work! It is very similar to what we have on thekobonotes.

Since I am the developer of thekobonotes.com I have to just clarify "the security issue" that was addressed. When you upload your notes on our app, the .sqlite file is processed on the browser! We then only upload the data related to the annotations and highlights to the server so you can access them everywhere (including in our new mobile app). This way any sensitive data that is on the kobo sqlite file stays on your browser (on the client) and we do not have access and we do not store it in any way on our server. This was an important aspect that we addressed in our app for security reasons.

Again congrats on your work

All the best

Hey João.

I spent a bit more time on your app. Looks great! I tried contacting you own facebook messenger to see if you guys were interested in working on this and ask if you are from Brasil (portugal). Or even add some of the features to your app, but that may not apply to kobonotes since what I am trying to create is not just for notes.

Anyway, thanks for the feedback. Yeah, the browser idea and not saving the whole db is great.

Checkout that facebook messenger message if you want to chat about this. The tech stack for what I used is: vue, vuex, js, aws dynamo db, aws lambda functions.

[MGlitch]

Originally Posted by rigogsilva

MGlitch, thanks for sharing this. I was not aware of that before trying to create this.

Yes, I believe security would definitely be an issue. To mitigate some of that, I was thinking of creating a tool that would run in your local machine that would pull just the data needed out of the database. So, no need to share the whole database which has emails and other personal info.

In regards to storing the data. The data can be anonymized, so there is no email or other personal info attached to the user; just the user id.

Thoughts about that?

Honestly, personally i don't care. I don't make notes in the books I read, nor do I highlight them. Which ticks off two major features of these sorts of tools.

I just wanted to remind folks that this has been done before, with shall we say mixed results.

[rigogsilva]

Originally Posted by MGlitch

Honestly, personally i don't care. I don't make notes in the books I read, nor do I highlight them. Which ticks off two major features of these sorts of tools.

I just wanted to remind folks that this has been done before, with shall we say mixed results.

Yeah, I get it. Definitely not for everyone; mostly for people that do highlights and want to track their book (without having to use goodreads). The big reason I started this is to get the page turns out of it to get some cool graphs.

The current tools out there doesn't really have that. No book tracking, no graphs (e.g.: books read in a year or how many hours read in a day).

To be honest, I just wish kobo had their own. Not sure they will do anything like that anytime soon.

Thanks for sharing that though!

[ndriver]

I want this!...
personally, I don't care so much for annotations but I would love to be able to track my book reading habits, stats and progress.

I assume you are using the Event table in the SQLite database, which after a quick glance I decided it was a mess...

Would it be complicated to to try to make it a self-hosted solution, deployed in a docker container for example? That would quiet down the privacy and security issues, as well as any project's longevity concerns.

[Quoth]

Calibre?

[ndriver]

Originally Posted by Quoth

Calibre?

Yes, you are right.
Calibre equipped with the excellent Kobo Utilities plugin by Davidfor, gets you very close to this. That's what I'm using in fact.

The only missing piece of information (at least for my preferences and as far as I know it is not achievable), is how long it took to read a specific book.

But in my setup, I don't connect my Kobo very often by USB. So the information in Calibre is usually outdated by weeks or even months.
What I am really looking is for a solution to extract and process the data in the e-reader and then push it by Wifi to my home server. Even if it means an additional tool on top of Calibre.

[rigogsilva]

Originally Posted by ndriver

Yes, you are right.
Calibre equipped with the excellent Kobo Utilities plugin by Davidfor, gets you very close to this. That's what I'm using in fact.

The only missing piece of information (at least for my preferences and as far as I know it is not achievable), is how long it took to read a specific book.

But in my setup, I don't connect my Kobo very often by USB. So the information in Calibre is usually outdated by weeks or even months.
What I am really looking is for a solution to extract and process the data in the e-reader and then push it by Wifi to my home server. Even if it means an additional tool on top of Calibre.

Yeah, that would be awesome if we could add a program to export that data from the device.

Yes, I did dig into the events and got page turn events.

How long it took to read a book? Like date started and end date, or the amount of minutes it took? The amount of minutes it took is inside the database contents.TimeSpentReading. So, it is possible to get that.

Unfortunately, I don't see a way to get the data out without plugging the device to your computer. Maybe there is. I will keep that in mind.

[rigogsilva]

Originally Posted by ndriver

I want this!...
personally, I don't care so much for annotations but I would love to be able to track my book reading habits, stats and progress.

I assume you are using the Event table in the SQLite database, which after a quick glance I decided it was a mess...

Would it be complicated to to try to make it a self-hosted solution, deployed in a docker container for example? That would quiet down the privacy and security issues, as well as any project's longevity concerns.

ndriver,

Can you elaborate a little bit on the docker container? What part of the system would need to be containerized? Maybe the "sync" part where the data is sent to the cloud?

As for hosting the static site, I was thinking in using a cloud based approach. AWS is right up my alley.

I do like the kobonotes approach where the sync is done through the site itself, but not sure.