[GUI Plugin] Manage Sony x50 Reader Book List

[kpw]

I've decided to take a shot at developing a Calibre plugin, so I've put one together to help manage the Book list on the Sony x50 reader Home screen. The default Sony behaviour is to display the 3 most recently loaded books, but I'd like to have more control in what books show up there.

Incidentally, there is a standalone application that enables this as well that has been posted in the Sony Reader forum (https://www.mobileread.com/forums/sho...d.php?t=107826), and the application is excellent. However, I wanted to have something that is integrated with Calibre, and I also wanted to extend the functionality specific to my use cases.

If anyone is interested in this functionality, I've put together a simple User Guide with installation, usage, and configuration information, as well as a description of the main use cases that I was trying to address. I've attached that below in both EPUB and PDF format, but I will repeat some of the information here for convenience.

Main Features

• Home page Books list
  • You can interactively select books from one of the Calibre device book lists (Device, Card A, Card B) to set books for the book list.
  • [NEW v1.2.0] You can automate selection of books by configuring Calibre library queries from which books will be chosen.
  • You can automate selection of books for the book list by configuring collections from which books will be chosen.
  • You can control the strategy controlling how books will be chosen from the sources, for cases where a source contains more books than will fit on the book list (i.e. more than 3).
  • You can control the sort order for books in the book list.
• [NEW v1.2.0] Unread Books collection
  • You can interactively select books from one of the Calibre device book lists (Device, Card A, Card B) to add books to or remove books from the reader unread books collection.
  • You can automate selection of books for the reader unread books collection by configuring Calibre library queries which will be synchronized with the unread collection.
  • You can automate selection of books for the reader unread books collection by configuring collections which will be synchronized with the unread collection.
• You have access to a full UI for managing configuration options.
• You can configure separate distinct sets of options for easy management of multiple readers for different people, each having their own favourite way of using the book list.

Version History

Spoiler:

1.2.11 September 30, 2014 Fix regression: 'QLabel' object has no attribute 'activated'.

1.2.9 September 27, 2014 Migrate to Calibre 2.x.

1.2.8 October 21, 2011 Fix Unread Books collection issue that may occur when using this feature in conjunction with the PRS+ support for the same.

1.2.7 September 11, 2011

• Handle scenario in a better way where books are added through the OS before reader connects to Calibre and before reader firmware processes the books.
• Change library query support so that queries are not subject to Library View restrictions (i.e. queries work against the full Calibre library).
• Add support for plugboards so that library query result matching to device books takes plugboard metatdata manipulation into account.

1.2.6 September 6, 2011 Fix issues that cause plugin failure when books are deleted from the reader using Calibre or directly from the file system while reader is connected to Calibre.

1.2.5 September 2, 2011 Ensure Books List entries remain in place semi-permanently (they will stay there for a year or until you choose other books to replace them). In this way, the Books List will not be overwritten when you add new books to your reader.

1.2.4 August 27, 2011

• Code cleanup.
• Fix issue in Ubuntu where "author sort" for book author wasn't being preserved when using the "Use author sort for author" setting.
• Add warning dialog when invoking "Set Unread List" action to make user aware the action is resetting your Unread Books collection. Hopefully, this will avoid someone inadvertently wiping out the list when "exploring" functionality. Warning can be turned off in the dialog, and enabled/disabled in the plugin customization (in the "Unread List" tab).

1.2.3 August 25, 2011 Test version of plugin that got released with a new version number.

1.2.2 August 20, 2011 Fix plugin so that it handles user input that includes accented character (or any non-English text).

1.2.1 August 4, 2011 Minor update to use better default values for library query (i.e. default query will execute without error and is specific to the context)

1.2.0 July 31, 2011

• Enable configuration of Calibre Library queries, in addition to the current collection sets, for book selection. Query expressions can be anything that you can enter in the main Calibre UI search field. Examples: 'tag:read', '#reading:yes', 'search:"My Short List"'
• Add new features to allow management of the Sony x50 reader Unread Books collection. Options include:
  • Manually add books to or remove books from the unread collection by selecting books in the device book list views (Device/Card A/Card B).
  • Synchronize the reader unread collection with Calibre Library queries or collection sets.
• User guide has been updated to cover the new features.

1.1.1 April 12, 2011 When updating book date/timestamp and timezone values, use the most recent updated book as the reference point rather than the computer date/time and timezone. This addresses issues that occur when there are discrepancies between the device and computer with regard to these values. One significant situation where this can occur is when a non-North American user has a PRS-950, which only allows North Americna timezones to be set.

1.1.0 April 2, 2011

• Restructure plugin to make use of newly introduced Calibre plugin API. NOTE: The new API is only available in Calibre version 0.7.53 or greater, so to use this version of the plugin, you need to move to this level of Calibre as well.
• Rework the preference store mechanism to use the recommended approach rather than the legacy approach I was using previously. The plugin should automatically migrate any existing configuration, so this should be transparent.
• Added new feature to allow explicit management of profile order. The configuration dialog now includes "Up"/"Down" buttons for the profile list to control how profiles are presented to the user when using the "Book List" action.
• Updated documentation to include new feature in preceding bullet.

1.0.6 March 18, 2011 Ensure cache entry date value is formatted properly on non-English installations

1.0.5 March 16, 2011 Account for DST when setting tz timesone offset value

1.0.4 February 28, 2011 Use locale neutral version of strptime to fix date parse error on non-English installations

1.0.3 February 27, 2011

• Add check to make sure cache file has not been updated separately before saving book timestamp updates. That way, the file shouldn't get updated with stale data.
• Change order of processing to ensure read/write of cache does not get interrupted by user interaction. This should minimize the possibility of the cache file being updated separately.
• Fix date sort bug.
• Restructure code for style consistency and to separate code out into separate .py files for better organization.

1.0.2 February 23, 2011 Fix regression introduced in 1.0.1 which can result in JSON error loading configuration string

1.0.1 February 11, 2011 Restructure code to ensure deferral of module loading

1.0.0 Febrauary 10, 2011 Initial release

Installation

1. Install plugin
   
   Spoiler:

   • Download attached plugin zip file (SetSonyBookList.zip).
   • Go to Preferences->Plugins->Add a new plugin.
   • Select the downloaded plugin zip file.
   • Apply the changes.
2. Add action button to toolbar
   
   Spoiler:

   • Go to Preferences->Customize the toolbar.
   • Select option The main toolbar when a device is connected.
   • Move the Book List action to the Current action list from the list of Available actions. Position the new action to your liking.
   • Apply the changes.

Configuration and Usage

• I'll let you refer to the User Guide for detailed information on usage and configuration (or more detailed, anyway).
• Out of the box, without any configuration, you can use the plugin to interactively select books from the Calibre device book list (i.e. first bullet in the Main Features section). For other features, you will need to configure accordingly.

Known (or Potential) Issues

Spoiler:

• If you choose a periodical to include it in the book list, it won't appear. I need to figure out how to ignore these.
• I haven't figured out a way of differentiating the PRS-600 from a PRS-x50, so that means you can run the plugin against a PRS-600. It won't really do anything useful, but it shouldn't break anything either.
• I think this should work with the PRS-900, as I understand it has the same UI as the x50 series, but I don't have one to test with.
• My date/timezone approach is working for the scenarios I've tested on my reader, but I'm not convinced it's bullet-proof... I've seen much discussion in the forums about how much grief this has caused folks. The main side effect (as far as I can tell) if the timestamp in the cache file for a book gets out of sync with the timestamp on the actual book file is that the Sony Reader will reprocess the record when it boots up and you may lose the "author sort" value if you've enabled that.

A Note on Testing

Spoiler:

• I've tested with the following:
  • Calibre 0.7.44
  • Windows 7 Home Premium, 64 bit
  • Windows Vista Home Premium, 64 bit
  • Windows XP
  • Sony PRS-650 with ~400 books
  • Memory stick and SD card
  • Sony PRS-600 - not very useful as the UI doesn't have the Books list, but it doesn't seem to break anything
  • Kobo Original (non-wifi) - i.e. detects that Kobo is not a Sony and pops up an error dialog

Under the Covers

Spoiler:

• The plugin does not update anything in the Calibre Library. The only file updates that occur are on the device.
• The files that are updated are the following:
  • For each book that is chosen to appear on the Books list, the book file (e.g. EPUB file) modification date is updated (i.e. the Python equivalent of a "touch"). This is what the reader uses to populate the Books list.
  • For each chosen book, the corresponding record in the media.XML or cache.XML file (for main memory and card respectively) is updated to synchronize the date with the new book file timestamp.
  • [NEW v1.2.0] For books that are added to or removed from the reader Unread Books collection, the corresponding record in the cacheExt.XML file is updated so that the book record "opened" attribute is set appropriately.
  • You may want to take a backup of the media.XML and/or cache.XML and cacheExt.XML files if you are worried.

Thanks all.

Karl

[kiwidude]

Thanks for that - I don't own a Sony reader myself but I am sure others will appreciate the effort particularly with so much work on the documentation etc. I will add this to the plugins index sticky thread shortly.

I haven't looked at the code in detail, but I did notice (as per your comment) your usage of lots of classes at the module level with imports of Qt code etc. That is one of those "best not to do that" situations for various reasons according to Kovid when I got him to do a code review a while ago when I first started doing GUI configuration screens. Reasons included how Calibre can be invoked from the command line (plugin base classes still get loaded) as well as potential namespace collisions.

One approach to get around this is to use a proxy approach that chaley kindly came up with for me and I have since put in place in a number of plugins in this forum. I can send you a "plugin template" I put together for it if you PM me an email (I now use it as the basis for all my plugins). Or else you can copy code from one of the recently updated plugin versions of mine. A "simpler" one to have less other stuff to remove from would be the Clipboard Search one. Of course I am sure there are other approaches to this, but as it came from one of the Calibre developers who knows far more about Python/Calibre than I ever will that was good enough for me .

I couldn't even being to repeat with any confidence all the technical reasons why that approach is better beyond what I mentioned above. I just trust that it does have his and Kovid's seal of approval over dumping everything into the top level namespace in one code file. It would still be nice to have even more code files (I believe this is possible by extending the approach) but I haven't had a need to as yet. Not having to nest all my classes was the big win for me.

[kpw]

Thanks for the feedback kiwidude. This is my first foray into Calibre and Python development so I'm figuring things out as I go along. I'd love to get that "plugin template" so I can clean things up. I'll PM that email address.

[kpw]

Updated plugin as per kiwidude's suggestions (Thanks!) to ensure deferral of module loading until required. New version is v1.0.1.

[meme]

kpw - great to see this plugin. I don't have a Sony, but I'm having a look over your plugin code to give me ideas on how to start re-organizing my code into classes.

Like you I've considered using multiple .py files to separate out the code sections - not strictly necessary but it might make things more readable.

[goaspy]

Hi there,

I see you're trying to steal my customers ... nah, I was just kidding

Good luck with your implementation, maybe I will try Python myself at some point.

I just wanted to let you know how I calculate the time stamp. I just released a new beta version that updates the metadata, I hope that it works on all time zones. I'm waiting feedback on this.

First I set the modified date to local date and time, then I extract the UTC (Coordinated Universal Time) date from this local date and write it in the xml file.
Then I calculate the offset between UTC and local time, in minutes, and multiply this with -1. I write the result in the tz attribute.

[woodfrog]

hi kpw,
thanks for this cool plugin.

just tried it with calibre 0.7.18 in Ubuntu Linux 10.10 64 bit, but it doesn't work for me.

I get this error:

Spoiler:

ERROR: <b>SyntaxError</b>:encoding declaration in Unicode string (<string>, line 0)

Traceback (most recent call last):
File "/usr/lib/calibre/calibre/gui2/preferences/plugins.py", line 150, in add_plugin
add_plugin(path)
File "/usr/lib/calibre/calibre/customize/ui.py", line 370, in add_plugin
plugin = load_plugin(path_to_zip_file)
File "/usr/lib/calibre/calibre/customize/ui.py", line 76, in load_plugin
exec raw in locals
File "<string>", line 0
SyntaxError: encoding declaration in Unicode string (<string>, line 0)

Not sure, if it is supposed to work in linux, but I thought plugins would be platform-independent.

I'll try it in my virtual machine...

[DoctorOhh]

Originally Posted by woodfrog

just tried it with calibre 0.7.18 in Ubuntu Linux 10.10 64 bit, but it doesn't work for me.

GUI Plugins won't work on a version as old as the one you're using. Since distribution versions of calibre aren't supported by Kovid my advice is to uninstall that old version, go here and do the recommended binary install of the newest version according to the instructions.

sudo python -c "import urllib2; exec urllib2.urlopen('http://status.calibre-ebook.com/linux_installer').read(); main()"

[woodfrog]

hi dwanthny,
thanks for your reply, i've upgraded to the latest calibre release and successfully installed the plugin now.

unfortunately, once i start the plugin by clicking on "book list" I get the following error message:

Spoiler:

calibre, version 0.7.45
ERROR: Unhandled exception: <b>ValueError</b>:No JSON object could be decoded

Traceback (most recent call last):
File "/home/christoph/.config/calibre/plugins/Manage Sony x50 Reader Book List.zip/ManageSonyX50BookList_plugin_code.py", line 71, in set_sony_book_list
config = Config(json.loads(self.site_customization))
File "json/__init__.py", line 326, in loads
File "json/decoder.py", line 360, in decode
File "json/decoder.py", line 378, in raw_decode
ValueError: No JSON object could be decoded

btw, I'm using an PRS-650. I hope it's ok to post my error messages, it might help to find bugs.
the rest seems to work, I can cofigure the plugin and everything, it just doesn't execute the script correctly.

[DoctorOhh]

Originally Posted by woodfrog

btw, I'm using an PRS-650. I hope it's ok to post my error messages, it might help to find bugs.

Including error messages is encouraged in these forums.

Originally Posted by woodfrog

the rest seems to work, I can configure the plugin and everything, it just doesn't execute the script correctly.

This is a little vague. You might want to elaborate on exactly what "the rest seems to work, I can configure the plugin and everything" means.

kpw will have to drop in to further help troubleshoot this problem. The only thing I would suggest (out of my own ignorance) is to restart your Linux box and restart (shutdown and turn on, not sleep mode) your 650.

Good Luck.

[woodfrog]

yay, it's working now. I just had to deinstall the plugin, redownload and reinstall it. There must have been something wrong with my first download, since the error occured in windows as well.

thanks again kpw, this is a very useful plugin.

And I can say:

Tested and works on:

• Ubuntu 10.10 64bit,
• Ubuntu 10.04 LTS, 64bit
• Windows 7 64 bit

All tested with Calibre 0.7.45, plugin version 1.0.1

[Calibrefan]

Thanks kpw, that's just what I was looking for, an integrated plugin which would accomplish this. The problem is that upon install and trying to use it with my Sony PRS-350 attached, I get the following error message:

calibre, version 0.7.45
ERROR: Unhandled exception: <b>ValueError</b>:No JSON object could be decoded

Traceback (most recent call last):
File "C:\Users\Colauto\AppData\Roaming\calibre\plugins\ Manage Sony x50 Reader Book List.zip\ManageSonyX50BookList_plugin_code.py", line 71, in set_sony_book_list
File "json\__init__.py", line 310, in loads
File "json\decoder.py", line 346, in decode
File "json\decoder.py", line 364, in raw_decode
ValueError: No JSON object could be decoded

Could you please give me a hand to get it up and running? I'm using Win7-64bit with Calibre 0.7.45. Just let me know if you need any additional information.

[woodfrog]

Hi Calibrefan,
this is exactly the same error that I got.
I had to deinstall the plugin, redownload it from here, and reinstall it. Then it worked. Still don't know why...
(maybe restarting the pc helped, maybe I still had the old version out of my browser cache, who knows?)

If you want I can drop you my file with which it definitely worked under Win 7.

[Calibrefan]

Hi woodfrog,

I didn't notice that you had the same problem at first because I thought you were having the trouble in Ubuntu rather than Win7x64. In the mean time I've reinstalled the plug-in, like you did, but I had no luck there. The problem persists. If you don't mind I'd like to try your working version to see if that works for me also if you care to drop me the file.

Thanks in advance!

[ThR]

With W7 32bit, calibre 7.45 - no problems. Works as suggested.

Thank you kpw!