Kobo Utilities Plugin

The Kobo Utilities Plugin adds extra function to calibre for working with Kobo e-ink ereaders. This will allow changes on the ereades that either cannot be done or will make them easier to do. The current functions of the plugin are:

• Set/Remove font settings for specific books.
• Update metadata directly to the database on the device. This is: Title, Author, Series, Comments/Synopsis, Publisher and Date, ISBN, Language and Rating.
• Change the reading status.
• Manage series info for books on the device. This is mainly for books not in the calibre library including kepubs.
• Store/record current reading position for epubs.
• Upload/Remove covers for all books in the library. This includes kepubs.
• Dismiss selected tiles from the new home screen and block them from returning.
• A couple of annotations options that I'm not happy with.
• List the books missing from the device database. This only lists the books that calibre thinks should be in the database.
• Backup the device database.
• Run integrity check on the database. It doesn't fix anything, but it tells you if the database is OK or not.
• Different function depending on whether you are looking at the library or the device list.

The plugin supports the WiFi, Touch, Glo, Mini, Aura HD and Aura. When a WiFi is being used, the options are limited options it supports. The plugin will probably work with the original Kobo eReader, but this has not been tested.

Known problems:

• There are currently no known problems.

Warning

Almost all function of this plugin directly update the database on the Kobo device. If anything goes wrong, the database could get corrupted. It shouldn't happen, but it could. Before using any function of the plugin, it is recommended that you backup the database.

The plugin has been created by reverse engineering the function of the Kobo ereaders. This has been done by examining the database on the Touch and Glo and experimenting with it and the interface. There is a good chance that my conclusions from this are wrong. This is unlikely to damage the device, but it could result in the need to reset it. Again, if you cannot accept this, please do not try this plugin.

Acknowledgements

A large chunk of the code is borrowed from the plugins written by kiwidude.

A few people have tried the beta version of the plugin. Their reports and suggestions have been important in getting it working.

Current Version

The current version of the plugin is 1.4.1.

Changes in this plugin include:

• Update: Option added when ordering series shelf to update shelf sorting in config file

Installation and Configuration

The plugin is installed through the calibre prefences dialog. During the installation, you will be prompted on which toolbar or menu to put the plugin button. As there is no available function when no devices are connected, put it on the "Main toolbar when a device is connected" or "The menubar when a device is connected". After the plugin is installed, restart calibre before attempting to use it.

To use the plugin, it needs to be configured. This can be done throught the calibre preferences, or from the "Customise plugin" option on the plugins menu. There are two areas that need to be configured: the columns to be used by the plugin and the action to be taken when the plugin button is pressed.

Columns for the Plugin

The plugin can use several columns to store the current reading position of epubs on the ereader. If you do not want to use this part of the plugin, they do not need to be configured.

The columns and types needed are:

• Current Reading Location: This is used to store the link to the current location in the book. It must be a "Text, column shown in the tag browser". It is recommended that the column is not shown in the column list and hidden in the tag browser.
• Percent Read: This is the current percentage read for the book. It must be an "Integers" column.
• Rating: This is the rating of the book. The standard "Rating" column in calibre can be used, or a custom used. If a custom column is used, it must be either a "Ratings, shown with stars" or an "Integers" type column.
• Last Read Timestamp: This is the date the book was last read. This must be a "Date" type column.

To add a custom column to calibre:

1. Open the preferences and select "Add your own columns".
2. Press the "Add custom column" button at the bottom, or the plus at the side of the window.
3. Enter a lookup name for the column. This is used internally by calibre and the plugin.
4. Enter the display name for the column. This is displayed in calibre with the column. This can be anything but should be meaningful to you and the purpose of the column.
5. Select the "Column type" from the drop-down list. Select the appropriate type for the column from those mentioned above.
6. Press OK to create the column.
7. When you have finished adding columns, press the "Apply" button to save the changes. You will be promped to restart calibre. You must do this before the columns can be used.

Configuration

Once you have created the columns and restarted calibre, you can set the columns to be used by the plugin. To do this:

1. Open the plugin configuration dialog by either:
   • If the device is connected, open the plugin menu and select "Customize plugin"
   • If the device is not connected, open the calibre preferences. The select "Plugins" and find the plugin in the list. Select the plugin and press the press the "Customize plugin" button.
2. Choose the columns you want to use.
   • For each column, the columns that can be used are listed in the drop-down list. This includes columns of the same type that may be used for something else. Make sure you select the correct column.
3. Choose the function to use when the toolbar button is pressed.
   • The toolbar button can have a different function depending on whether you are viewing the library list or the device list.
   • For each setting, only functions that are valid for that list are listed.
   • From the drop down list, select the desired function.
4. When finished customizing the plugin, press the "OK" button to save the changes.

Using the Plugin

The following section details each function available and how to use them. Some functions are only available for the library list, some only on the device list and some work for both. Most functions act on the books that are selected in the current list.

For most of the functions available, the way to use them are:

1. Select the books you want to set change something for.
2. Select the desired menu option.
3. When the dialog is displayed, adjust the options.
4. Press the OK button. If the Cancel button is pressed, no changes are made.

Set Reader Font for Selected Books

When the reader settings are changed in the device, they are stored for that book and for any future book opened. This allows the user to change these setting for one or more books. And, for some of the settings, it allows finer grade options than the device settings dialg.

This option works the same for both the library list and the device list.

The options on the dialog are:

• Font Face: This is the list of standard fonts supplied by Kobo. It does not include any sideloaded fonts.
• Font Size: The device can use font sizes between 12 and 58. The settings on the device provide about 20 options in that size range. The plugin allows any font sizes in the range.
• Line Spacing:
  • The device settings allow nine different line spacings. These can be selected using the spin button. "0" is the left-most line spacing on the device and "8" is the right-most. The line spacing value used is displayed in the read-only field.
  • If the "Custom setting" option is checked, a custom line spacing can be entered. The value can be ay number, but, from experimentation, the minimum value that has an effect is 1.225. Any number lower than that gives the same spacing.
• Left and Right Margin: The device setting use the same value for both the left and right margin, But they can be set separately. The value can be from "0", not margin, to "16". These are the same as selecting the left-most and right-most settings on the device. On the device, each setting jumps by two, The dialog allows single step increments.
• Lock margins: Lock the left and right margins to the same value. Changing the left margin will also set the right margin.
• Justification: From the drop-down, select "Off", "Left" or "Justify".
• Update config file: Update the 'Kobo eReader.conf' file with the new settings. These will be used when opening new books or books that do not have stored settings.
• Get configuration from device: Pressing this button read the "Kobo Reader.conf" file to retrieve the current settings used on the device.
• Get settings from device: Pressing this button read the databae on the device to get the settings for the current book. This will be disabled if more than one books is selected.

Once the settings are how you want them, press the OK button to save the settings and update the database on the device.

Dismiss Tiles from Home Screen

The new home screen on the Aura HD and Glo uses as set of tiles to display recent activities. These can be pressed to do the same activity or dismissed. The tiles displayed change over time as different as the device is used.

This function is available for both the library and device lists.

When "Dismiss Tiles from Home Screen" is selected, a dialog is displayed. This has:

• The top section lists the type of tiles that can be dismissed. Select each type of tile that is to be dismissed.
• The middle section is to dismiss the tiles for new books and finished books.
• The bottom section creates or removes the database trigger. The trigger will work for the options selected from the top sections. Check the option, and then select whether to create or delete the trigger.

When you have selected the options, press the OK button. This updates the device and displays a message when finished.

Notes:

• This will dismiss all instances of a tile type. Some, such as "Sync" can only appear once. Others such as "Shelf" and "Award" can be on the home screen multiple times.
• Unless the database trigger is created, this does not permamently dismiss the tile. If the activity for the tile is used again, the tile will return to the home screen.
• Sometimes, after the device is disconnected, the home screen needs to be refreshed before the dismissed tiles are removed. This should be done by leaving the home screen and returning to it.
• If this function is used with devices that do not use the new home screen, there will be no affect or damage.

Remove Reader Font for Selected Books

The reader settings are stored in a database for each book. This option removes the settings for all books selected. It will prompt you to continue by pressing "Yes" or stop by pressing "No".

This function is available for both the library and device lists. It will work for all book types that store the settings.

Update metadata in device library

The metadata displayed on the device is read from sideloaded books the first time the device sees them. If the book and metadata is changed and sideloaded again, the metadata is not reread. There is also some standard metadata that is not read from epubs.

For kepubs synced from the Kobo server, the metadata is populated from the server. It might get changed at later syncs.

This option only works from the library list.

The options on the dialog are:

• Metadata options:
  • Title
  • Author - the calibre {author} is used. This will correctly populate the author on the device for multiple authors
  • Series and Index
  • Comments/Synopsis
  • Publisher
  • Published Date
  • ISBN
  • Language
  • Rating - this is only available if a rating column has been configured for the plugin.
• Reading Status - These options can be used to change the reading status of the books on the device. The function is identical to "Change Reading Status in device library" and it is described there.

Change Reading Status in device library

This allows the reading status of books in the device that are not in the calibre library to be changed. An intended use is to reset an accidently opened book to unread. It can also be used to mark books as finished after a factory reset or other reason for relaoding books.

The function on dialog is shared with the "Reading Status" section of the "Update metadata in device library" function.

• Change reading status - Select this if you want to change the reading status.
• Unread, Reading and Finished - Select the status to change the books to.
• Reset reading position - If this is selected, when the status is changed,the current reading position is removed. If this is not done, the next time the book opened, it will be opened to the stored position.

Note: When the OK button is pressed, the changes are made on the device. But, the reading status options are not save. The next time the dialogs is opened, all options will be reset. This is deliberate. If this function is used, the decision of what changes to make must be made each time.

Manage Series Information in device library

This allows you to change the series information for books already on the device. It allows you to set the series name and number for books that you do not have in your calibre. This can change the series information for kepubs.

The interface and most of the function has been taken from the "Manage Series" plugin written by kiwidude. If you are familiar with that plugin, this works in a very similar way, but is restricted to books on the device. If you want to manage series for books in your calibre library, please use the "Manage Series" plugin. The Kobo Touch driver can update the device with this information.

If you find a problem with the "Manage Series" function of the Kobo Utilities plugin, report it to the developer of the Kobo Utilities plugin. DO NOT report it as a bug in the original "Manage Series" plugin.

To manage series on the device, from the device list:

1. Select the books that are in the same series.
2. Select "Manage Series Information in device library"
3. A dialog displaying the list of books with several options is displayed. Here you can:
   • In the list at the top, select the series name or enter a new one if the names not in the list.
   • Set the starting number for the series number.
   • Some Kobo books include the series name in the title. This can look something like Book title (The Whatever Series: Book #1). Selecting "Clean titles of Kobo books" will remove this.
   • The title and the publishing date can be edited by clicking in the field and pressing the F2 key. Pressing the escape key will cancel the changes and any other action will save them.
   • Change the order of the books by selecting one and pressing the up and down arrows at the right of the list.
   • Remove a book from the list by press the trash button.
   • Lock or unlock the series number for a book by selecting the book and pressing the lock or unlock buttons on the right. Locking the series number will prevent it an any books after it in the list from changing if the order is changed.
   • To undo all the changes, press the "Restore Original Series" button.
4. When finished making changes, press the OK button to save them, or the cancel button to quit. No changes are made to the device database unless the OK button is pressed.

Store/Restore current bookmark

This allows the current reading position in an epub to be stored in the calibre library. This can then be sent back to the device or another Kobo device to restore the reading position. As well, the rating set on the device can also be stored and restored.

The device store the reading position in the database on the device. This is stored in several parts:

• The current chapter and the position in the chapter.
• The percentage read of the book
• The reading status: Unread, Read and Finished
• When the book was last read.

These can all be retrieved from the device and stored in custom columns in the calibre library.

This is only available in the library list.

To use this function, you need to:

1. Create the necessary custom columns and configure the plugin. This is described above.
2. Select the books you want to store or restore the reading position for. If books are selected that are not on the device or do not have an ePub format, they will be ignored.
3. Select the "Store/Restore current bookmark".
4. Choose either "Store" to copy the current bookmark to the library, or "Restore" to set the current bookmark on the device.
5. Set the options needed.
6. Press the OK button to do the action.
7. The function will run and a message will be shown with the changes made.

The options on the dialog are:

• Clear if unread - If using "Store" and the book has not been read on the device, the bookmark values in the lubrary will be cleared.
• Set reading status - This will change the reading status on the device based on the bookmark settings. If the book is 100% read, it will be marked as "Finished". If it is 0% read, then it will be marked as "Unread". Otherwise, it will be marked as "Reading".
• Set date to now - If this is set, then the last read timestamp will be set to now. This will push the book to the top of the "Recently Read" list and the Home screen. If this is not set, then the stored last read timestamp will be used.
• Update rating - Sets the rating on the device from the calibre library.

Upload covers for Selected Books

With this, you can change the cover on the device to the current cover in the calibre library.

To use it, select the books that you want to change the cover for, and choose the desired options. Then press the OK button to upload the covers.

The options on the dialog are:

• Black and White Covers - Covers are converted to black and white before uploading.
• Keep cover aspect ratio - When the covers images are sized for device, the current aspect ratio is kept. This can improve the look of the covers on the home screen and library lists.
• Upload covers for Kobo epubs - Replace covers for Kobo kepubs with the cover from the calibre library.

Remove covers for Selected Books

This will remove saved covers for the selected books from the device. This will allow the device to regenerate covers for these books when they are next needed.

To use it, select the books that you want to change the cover for, and choose the desired options. Then press the OK button to remove the covers.

The options on the dialog are:

• Remove covers for Kobo epubs - Replace covers for Kobo kepubs with the cover from the calibre library.

This function is available for both the library and device lists. It will work for all book types that store the settings.

Clean Images Directory

This looks at the images directory on the device and looks for any cover images that are not for books currently in the device database. The cover images can be removed or simply listed.

To use it, select the menu option and choose the desired options. Then press the OK button to check and clean the images directory. The clean is run as a background job.

The options on the dialog are:

• Delete extra cover image files - Delete an extra cover images found in the directory.

When the job is finished, how many extra cover image files were found and deleted is displayed. Pressing the "Show Details" button will list the extra cover image files that were found.

Order Series Shelves

Using the KoboTouch driver, shelves can be created for each book series on the device. The books are then added to the shelf. But, there is no way to sort these shelves into the series order. This option allows uses the "Date Added" sort to show the books in the series order.

To use it, select the menu option. A dialog with the list of series shelves will be displayed. The shelves listed are those that have the same name as for series that are on the device. The number of books on each shelf will be displayed.

Select the options and press the OK button to order the shelves. The options on the dialog are:

• The "Series Order" order is either "Ascending" or "Descending". Choosing "Ascending" will put the books in the series order. Choosing "Descending" will put them in reverse order.
• If "Update config file" is selected, the selected sort order for the series shelves will be set to "Date Added" in the "Kobo eReader.conf" file. The is the sort order needed to display the books in the series order. If this option is not selected, the sort order will need to be changed when viewing the shelves.
• To skip ordering a shelf, select it in the list and press the "Remove" button.

When the shelves are ordered, a message box will be displayed with the number of shelves whose order was set. If the shelf only had one book on it, the shelf order will not be changed.

To see the books in the series order, select "Date Added" when viewing the shelf on the device.

A few notes about the ordering:

• Only books that have a series name that match the shelf are ordered.
• The books also have to have a series number. Some books from Kobo do not have the number.
• If the series number is not just a number, an attempt is made to extract the number. Again, books from Kobo frequently have a hash symbol in front of the number. If a number cannot be found, the book is not ordered.

This function is available for both the library and device lists.

Copy annotations for Selected Book

This will retrieve any annotations for the selected books and display them in a dialog. The annotations can be copied and save elsewhere. The annotations are retrieve using the same code as the "Fetch annotations".

Backup Annotation File

Annotations for ePubs are stored on the device in two places: the internal database and files. The files have type "ANNOT" and are stored within the directory Digital Editions\Annotations. These files are contain XML that define the annotations. They can be copied to another Kobo device, or with a little fiddling, used with ADE.

Selecting this option will prompt for a directory to copy the files to. Either enter the directory name, or press the "..." button to select a directory. When the directory has been entered, press the OK button to copy the files.

Show books not in the device database

There are several cirumstances where calibre will think a book is on the device, but it is not in the device database. This usually means the book has been sent to the device by calibre, but the book has not been processed yet.

When selected, a list of books is displayed in a separate window. This is simply a list of the books and has no function other than to display the books.

Note: This does not show any books that are on the device and calibre does not know about and are not in the device database. If books have been sideloaded in some other way and are not in the device database, they will not be displayed.

This function is only available when viewing the device list.

Refresh the list of books on the device

When this is used, the books on the device will be reread. This is done in the same way as when the device is first connected. The main use for this is it will cause any changes for shelves an series information to be sent to the device.

Fix Duplicate Shelves

Due to some errors with syncing to the Kobo server, the shelves occasionally appear to be duplicated. There are several things that can happen, but in one case old shelves that have been deleted are recreated.

To use it, select the menu option. A list of all shelves with the following information is displayed:

• Shelf Name
• Oldest - Date when oldest version of the shelf was created
• Newest - Date when newest version of the shelf was created
• Number - Number of versions of the shelf

To fix the duplicates, choose the options and press the OK button. The options are:

• Shelf to Keep: This can be either "Newest" or "Oldest". This determines which of the duplicate copies of the shelves should be kept.
• Purge duplicate shelves: Selecting this will delete the rows for the duplicate shelves from the database.

When the OK button is pressed, the duplicate shelves will be marked for deletion. The version of each shelve that is kept will be the one with either the newest or oldest date, depending on the options. After this is done, you should disconnect the device and sync to the Kobo server. This should remove all the duplicates from the server and should solve the problem.

If the Purge duplicate shelves option is selected, the duplicates will be deleted from the device database. If this is done and a sync to the Kobo server is then done, it is possible that the server will send all the shelves to the device again. But, if you don't sync the shelves in the future, this probably isn't important.

This function is available for both the library and device lists.

Block Analytics Events

The Kobo devices collect information about actions taken on the device. This is stored in a database table called "AnalyticsEvents". There is evidence that this information is sent to Kobo during a sync. It is unknown how Kobo store or use this.

To use it, select the menu option and choose the desired options. Then press the OK button. The entries already in the AnalyticsEvents table will be deleted and a database trigger installed to prevent further entries from being created.

The options on the dialog are:

• Create or change trigger - When this is selected, the database trigger will be created or updated if it already exists.
• Delete trigger - This will remove the existing trigger and let the device work as Kobo intended it.

This function is available for both the library and device lists.

Check the device database

Unfortunately, there are circumstances where the database on the device can become corrupted. When this happens, the database usually has to be replaced. This means loss of annotations and reading status for any sideloaded books. With this option, the status of the database can be checked.

One of the signs of a corrupt database is an error from calibre when the device is connected. This error is "DatabaseError: database disk image is malformed". When using the device, the symptoms can include things like the loss of reading position or the same books being processed multiple times.

Selecting this option will run PRAGMA integrity_check against the database and display the output in a window. If there are no problems with the database, "ok" will be displayed in the window. If the database is corrupt, the errors found in the database will be displayed.

Compress the device database

As books are added or removed from the device, their details are added or removed from the device database. This causes the size of the database file to increase. This option will compress the database file to clear unused space.

Selecting this option will run VACUUM against the database. When finished the uncompressed and compressed sizes of the database are displayed. If there is an error, the error message will be displayed.

It is recommended that you backup the device database before compressing it.

Backup device database

This will copy the device database to somewhere on your computer. This will allow it to be

Customize plugin

About Plugin

Problems

If you problems, you can find me on the MobileRead forums. My profile is davidfor. Visit there and send me a private message explaining what the problem is. When describing the problem, please give as much information as possible. In general, there is no such thing as to much information when trying to debug remotely.

If you send me a message, I will respond as quickly as I can. But, remember, it will take time to read the report, think about what could be going on and then write a response. And I am probably in a different timezone to you. Plus, this is done in my spare time. I work, have a family and like to use my Kobo devices for their intended purpose, reading.

After hearing of a problem, I might ask you to run calibre in debugging mode and send me the output produced. If I do, these are the steps:

1. Restart calibre in debug mode by clicking on the arrow next to the prefences button and selecting "Restart in Debug mode".
2. Calibre will close and reopen. When calibre reopens a message about being in debug mode will be displayed.
3. Perform the actions in calibre that are not working as expected. This will normally mean connecting the the device and waiting for it to sync and then using whichever of the plugins functions didn't work for you.
4. Close calibre.
5. The calibre debug log will then be displayed.
6. Send a copy of the log file to me when requested.