Issue between Calibre and Kobo Forma

[Fox1977]

Hello all,

I'm facing an issue that i cannot explain.
I'm adding books from Calibre to my Kobo Forma. They appear with the green tick showing that they are on my reader, and if i check the metadata.calibre file on the reader, the books appear on it.
If i eject the reader, i can see the books on it.
When I replug the reader, Calibre rebuilds the metadata.calibre file and deletes the books from it, and therefore Calibre considers that the books i added are not on the reader.

Anyone has a clue of what's going on and how i can do to fix the problem ?
Thanks

[theducks]

What Version of Calibre? What OS?

Do you have the Reading List Plugin installed? <<<

[Fox1977]

I'm using the latest version of Calibre (7.3) as well as the latest Kobo firmware.
I tried adding a book from my wife's Calibre on her computer and it does the same (but if she adds a book on her own Kobo reader, everything is fine).
I suppose something got wrong on the database on my Kobo, but i don't understand what.
If i delete the koboreader.sqlite to rebuild the database, then it's fine, but it's not the first time it behaves this way and i cannot rebuild the whole database every time i want to add a book.
There must be an expanation, but i don't see which one...

[kovidgoyal]

Its not calibre deleting your book. Your reader is failing to index it. On re-connect calibre reads the updated index from the device. Since the book is not present in that index, calibre shows it as not on the device.

Why your device is not indexing the book, only Kobo can tell you. You can try resetting the device and also try doing an epub to epub conversion to fix any issues in the file itself.

[DNSB]

You may want to try rebooting your Forma to force the on-disk and in memory database copies to sync though is is more of a bandaid over an underlying issue.

If you have Windows and calibre on your computer, do you get the Windows "There's a problem with this drive. Scan the drive now and fix it." popup when you connect your Forma? This suggests either you are not safely ejecting your Forma or you are running into the database corruption issue with 4.38.21908 firmware. That error seems to worse on the Sage and Elipsa models but has been seen on most models and is the reason I now use a NickelMenu command to reboot my Sage before connecting to my computer.

[Fox1977]

I don’t get any error message, neither from Windows, nor from Calibre. I always eject the reader with the button in Calibre before disconnecting it from the computer.
I will reset once more the reader, and try to see if there is a pattern when the problem comes back.
I will also look into this NickelMenu that is mentioned, I don’t know what it is but if it can help, why not…

@kividgoyal when you say that my device is not indexing the book, it’s then possible that the book is not indexed but still appears on it ? Because I’m currently reading the book on my Kobo, but if I plug it on my computer, Calibre doesn’t show that it’s on it (no green tick)

[chsf35]

Originally Posted by Fox1977

I
If i delete the koboreader.sqlite to rebuild the database,

I've got the same issue. How do you delete the koboreader.sqlite ? Where ?
Thanks for your help

[JSWolf]

Originally Posted by Fox1977

I don’t get any error message, neither from Windows, nor from Calibre. I always eject the reader with the button in Calibre before disconnecting it from the computer.
I will reset once more the reader, and try to see if there is a pattern when the problem comes back.
I will also look into this NickelMenu that is mentioned, I don’t know what it is but if it can help, why not…

@kividgoyal when you say that my device is not indexing the book, it’s then possible that the book is not indexed but still appears on it ? Because I’m currently reading the book on my Kobo, but if I plug it on my computer, Calibre doesn’t show that it’s on it (no green tick)

There is 1 of 2 things you need to do before you connect your Kobo to the computer.

1st thing to try is to go to the home screen before you plug in your reader.
If that fails to work then the 2nd thing to do is to go to the home screen and then reboot your Kobo by holding the power button until it reboots.

Please let us know which option works for you.

[DNSB]

For what it may be worth, here is the current content of my nm_config.txt file:

The relevant lines for reboot and shutdown are ( <location> is which screen the menu item shows on (i.e. main for the Home, library for My Books, etc.):

menu_item : <location>     : Reboot            : power          : reboot
menu_item : <location>     : Shutdown          : power          : shutdown

#
# ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #
menu_item : main    : Dump Syslog        : cmd_spawn      : logread > /mnt/onboard/.adds/syslog.log
menu_item : main    : Free Memory        : cmd_output     : 500  : free -m
menu_item : main    : Kernel Version     : cmd_output     : 500  : uname -a
menu_item : main    : IP Address         : cmd_output     : 500  : /sbin/ifconfig | /usr/bin/awk '/inet addr/{print substr($2,6)}'
menu_item : main    : Telnet             : cmd_spawn      : quiet: /bin/mount -t devpts | /bin/grep -q /dev/pts || { /bin/mkdir -p /dev/pts && /bin/mount -t devpts devpts /dev/pts; }
     chain_success  : cmd_spawn          : quiet: /usr/bin/pkill -f "^/usr/bin/tcpsvd -E 0.0.0.0 1023" || true && exec /usr/bin/tcpsvd -E 0.0.0.0 1023 /usr/sbin/telnetd -i -l /bin/login
     chain_success  : dbg_toast          : Started Telnet server on port 1023.
menu_item :main     :Typography          :cmd_output      :500   :/mnt/onboard/.adds/fontcheck.sh
#
menu_item : main    : Dark Mode          : nickel_setting : toggle : dark_mode
menu_item :main     : Invert & Reboot    : nickel_setting :toggle: invert
    chain_success   : power              : reboot
#
menu_item : main    : Screenshots        : nickel_setting : toggle : screenshots
menu_item : main    : Import books       : nickel_misc    : rescan_books_full
menu_item : main    : Toggle Auto USB    : nickel_setting : toggle : auto_usb_gadget
menu_item : main    : USB Connect        : nickel_misc    : force_usb_connection
menu_item : main    : Screensaver Status : cmd_output     : 500: quiet : test -e /mnt/onboard/.kobo/screensaver_old
      chain_success    : dbg_toast       : Screensaver is off
      chain_failure    : dbg_toast       : Screensaver is on
#
menu_item : main : Toggle screensaver    : cmd_output     : 500: quiet: test -e /mnt/onboard/.kobo/screensaver_old
      chain_failure  : skip : 3
      chain_success  : cmd_spawn         : quiet: mv /mnt/onboard/.kobo/screensaver_old /mnt/onboard/.kobo/screensaver
      chain_success  : dbg_toast         : Screensaver on
      chain_always   : skip              : -1
      chain_failure  : cmd_spawn         : quiet          : mv /mnt/onboard/.kobo/screensaver /mnt/onboard/.kobo/screensaver_old
      chain_success  : dbg_toast         : Screensaver off
menu_item : main     : Reboot            : power          : reboot
menu_item : main     : Shutdown          : power          : shutdown
#
menu_item : main    : Force WiFi on      : nickel_setting  : toggle: force_wifi
     chain_success  : nickel_wifi        : autoconnect
	 
menu_item : main    : Pocket             : nickel_open     : library: pocket

# ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #

menu_item : reader  : Dark Mode          : nickel_setting : toggle : dark_mode
menu_item : reader  : Invert & Reboot    : nickel_setting : toggle : invert
    chain_success   : power              : reboot
#
menu_item : reader  : Pocket             : nickel_open     : library: pocket
menu_item : reader  : Screenshots        : nickel_setting : toggle : screenshots
menu_item : reader  : Free Memory        : cmd_output     : 500: free -m
menu_item : reader : Reboot             : power          : reboot
# ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #

menu_item : browser : Invert & Reboot    : nickel_setting : toggle: invert
    chain_success   : power              : reboot
# ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #

menu_item : library : Dark Mode          : nickel_setting : toggle : dark_mode
menu_item : library : Free Memory        : cmd_output     : 500: free -m
menu_item : library  : Invert & Reboot   : nickel_setting : toggle: invert
    chain_success   : power              : reboot
#
menu_item : library : Pocket             : nickel_open     : library: pocket
menu_item : library : Screenshots        : nickel_setting : toggle : screenshots
menu_item : library : Import books       : nickel_misc    : rescan_books_full
menu_item : library : Toggle Auto USB    : nickel_setting : toggle : auto_usb_gadget
menu_item : library : USB Connect        : nickel_misc    : force_usb_connection
menu_item : library : Reboot             : power          : reboot
menu_item : library : Shutdown           : power          : shutdown
# ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #
#

[Fox1977]

Thanks for your messages.
I am not familiar with NickelMenu. I can try it on my Kobo. What does extra convenient features does it bring ?

[Fox1977]

Originally Posted by chsf35

I've got the same issue. How do you delete the koboreader.sqlite ? Where ?
Thanks for your help

The KoboReader.sqlite file is in the .kobo folder

[Fox1977]

I hadn't plug the reader back to my computer since last time as i had two books to read. I finished the second today and wanted to follow your suggestions, but now i get :
CorruptError: database disk image is malformed

Traceback (most recent call last):
File "calibre\gui2\device.py", line 104, in run
File "calibre\gui2\device.py", line 558, in _books
File "calibre\devices\kobo\driver.py", line 2028, in books
File "C:\t\t\apsw-42s16m2x\src\cursor.c", line 240, in resetcursor
apsw.CorruptError: CorruptError: database disk image is malformed

I couldn't do anything, kept getting this error, so i deleted the Kobo.sqlite file to have the database rebuilt.
Now, it's fine but till when... I have no idea what causes this issue to happen. It's the 3rd of 4th time it happens over the last few months. Last time, i think it happened at the beginning of November.
I took a new USB cable, in case...

[DNSB]

There is a known issue with Kobo's 4.38.21908 firmware which allows the ereader to connect to the computer without first closing all the SQLite databases. My solution has been to do a reboot before connecting, hence, the NickelMenu stuff in a previous message.

[Fox1977]

I installed NickelMenu and used your config file, thanks. So, you recommend that i reboot first the reader everytime i need to connect it to the computer ?

[DNSB]

Originally Posted by Fox1977

I installed NickelMenu and used your config file, thanks. So, you recommend that i reboot first the reader everytime i need to connect it to the computer ?

For me, I reboot anytime I've been reading. It keeps me from getting database errors while waiting for a new firmware to drop. I haven't seen database corruption after book imports but some others have reported this.