Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Development

Notices

Reply
 
Thread Tools Search this Thread
Old 06-30-2016, 08:05 AM   #1
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
SQLite version

I am doing some experimenting with the KoboTouch driver had an error with SQLite access to a database. After a bit of investigation, I discovered the version of SQLite that the sqlite3 library is built with is different in Windows and Linux.

The Windows build is using SQLite version 3.6.21. The Linux version of calibre I had to check with was 1.48. This has a 3.7 version of SQLite.

Is there a reason for the different versions? Any chance of getting the Windows version updated?
davidfor is offline   Reply With Quote
Old 06-30-2016, 08:20 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
The version of sqlite is whatever is bundled with python on windows and the system sqlite on linux.

If you want to have a modern sqlite version use the apsw module rather than the sqlite3 module (that is what calibre uses for its own database as well)

Note that migrating may not be trivial as there a few differences in the interfaces of the two modules.
kovidgoyal is offline   Reply With Quote
Old 06-30-2016, 09:17 AM   #3
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
I was afraid you were going to say that. I saw the use of apsw, but, there is enough mention of sqlite3 in the code, that I wasn't sure what was going on.

I had a very quick pass over the changes, and I think I might be lucky in the driver. The biggest problem might be the use of "fetchone". The plugins will probably be the painful part.
davidfor is offline   Reply With Quote
Old 06-30-2016, 10:20 AM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Historically, calibre used to use sqlite3, but I switched to apsw for calibre 1.0 (part of the big database layer rewrite) because it has better performance, and more control over sqlite semantics (apsw is sqlite specific whereas sqlite3 has to conform to the more general DB-API).

Not to mention that one is no longer tied to whatever sqlite is shipped with python.

However, as for your original issue -- I am not aware of any incompatibilities between 3.6 ans 3.7 as long as the db is not using any features that were introduced in 3.7
kovidgoyal is offline   Reply With Quote
Old 06-30-2016, 09:49 PM   #5
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Yes, it's about a database feature added in SQLite 3.7 and a possible change coming from Kobo. With the possibility of better performance from apsw, I think it's worth my time to convert over at this point. There are some things about how the driver handles the database connection that I don't like, so I'll fix that at the same time. If I do that properly, that should flow down to the plugins nicely as well.
davidfor is offline   Reply With Quote
Old 07-03-2016, 09:37 AM   #6
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
For anyone that is interested, I have converted the KOBO and KOBOTOUCH driver over to apsw. It turned out to be a lot quicker that I expected. Especially when I refactored all the database connects out to a simple method. I did some other refactoring for some code that was almost the same between the two drivers.

The code is in my branch at https://github.com/davidfor/calibre.git. If there are any other Kobo users out there who want to run from source and check everything works, I would be very happy to hear the results. This is especially if you have a WiFi or the original Kobo. I don't have one of these to test with. I can do some testing of this using my Touch. I think they will be OK, but would like confirmation.

This will have no effect on extended driver, my Kobo Utilities or the Annotations plugin. I will be making changes to the latter two after these changes have been released to use apsw as well.

My plan is to use this for the next week and fix any problems I see. If everything goes OK, I will create a pull request next weekend.
davidfor is offline   Reply With Quote
Reply

Tags
sqlite3

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Sqlite on Kobo Lucas Malor Kobo Developer's Corner 6 03-26-2014 08:06 AM
SQLite Meeky Kindle Developer's Corner 2 01-01-2013 11:02 AM
Koboreader.sqlite gundged maxwelledison Kobo Reader 21 01-12-2011 08:24 AM
Kindle SQLite DB akira.spb Kindle Developer's Corner 2 10-18-2010 10:20 PM


All times are GMT -4. The time now is 03:36 PM.


MobileRead.com is a privately owned, operated and funded community.