[GUI Plugin] Reading List

[kiwidude]

@Wretch - can you see it on your toolbar? Go to Preferences -> Toolbar and make sure it is on all the toolbars relevant to you (take note of different ones for when a device is connected).

[Wretch]

Thanks kiwidude, it was because my device was plugged in, I didnt realise it was a different set of icons when a device was connected!

Great stuff!

[Kraftwerk]

Not too sure where to ask this question, so I'm trying here:

I save my 'Calibre Library' folder on a USB stick, just in case something goes wrong with my computer's hard drive.

Is my list of books in the 'Reading list' plugin saved in the 'Calibre Library'?

I ask because if something does go wrong with 'Calibre' or my computer's hard drive, I would want to add all the books to 'Calibre' aswell as adding all books I had in my 'Reading list'.

Thanks

[kiwidude]

@Kraftwerk - the data for the reading list plugin (or indeed any of my plugins) is stored in a .json file which is located within the calibre plugins folder.

You can find this folder using:
Preferences->Advanced->Miscellaneous and click Open calibre configuration directory

There is a "plugins" folder that contains all user installed plugins and their json files containing configuration and plugin data. I would suggest you backup that folder, so that if you restore your library off usb you are able to restore your plugin configuration data as well.

[davidfor]

Originally Posted by kiwidude

I think kiwidude is wondering what on earth you guys are up to...

I can't decide whether to be insulted by that or not

You have it roughly right. And I could do it the way you suggest. What I do has developed over time and I should revisit. I do like using different lists as they are easy to display. The "Read" list isn't as needed, but because moving a book between the lists will manage my status column for me, it works well. I have a couple of other lists who could easily just be tags, but again, the way the Reading List plugin maintains a column makes this a sensible way to handle it.

My "Current" list is actually fairly long. But, this contains a lot of stories downloaded from the web and are still being written.

You are right about my suggestion to formalise the workflow, but it doesn't really fit into this plugin.

As to the "Move to list" function, I keep trying to use this when I am not actually looking at a list. I can see a few ways of handling it in this case:

- If the book isn't in a list, do the same thing as "Add to list".
- If the book is in one list, do a move from that list.
- If it is in multiple lists, either disable the option, or prompt for the source list.

But the consistency of handling this and the other functions the plugin supplies means I think leaving it the way it is probably best.

[kiwidude]

@davidfor - I was totally tongue in cheek of course in response to your own comment in the previous post.

I'm not against allowing the Move option on the library. It is a far simpler solution to what I thought was originally being asked for when people start talking about automating workflows.

It is just a case of finding/making the time to do the development. Whenever I next revisit this plugin for an update it will be top of the list.

[kiwidude]

I've been wondering for a while about an idea that I know also came up on the Plugin Ideas sticky thread ages ago. It is the idea of being able to import lists of books.

These lists might be things like Hugo nominees from Wikipedia, or someone's ad hoc list of favourite books/top 100/whatever. It could be a "new/upcoming releases" list or ... you get the idea. The intent will be to try to make this as streamlined as possible (once a source has been configured), minimising the clicks and pain involved. My ideal goal is where you are just browsing away, see someone's list of recommended books in a forum post, you paste it into calibre and as quickly as possible see which books you have etc.

Right now with calibre there is no way to do anything like this with a group/list of books.

So what would you actually "do" with the list? Well you might simply want them available as a retrievable list to view. You might want to tag/apply a custom column value like "Hugo" or whatever. You might want to put the books on that list onto a device. These are features the Reading List plugin already has, which is why I think it would make sense to extend this plugin rather than just creating a new one.

There are obviously a lot of technical difficulties/challenges. I want the ability to paste a text based list of books from the clipboard. I also want to be able to point at a web page and import the list from there. It *will* involve the user creating regular expressions for the former, and xpath expressions for the latter, so some technical expertise is required. However I want those sets of expressions to be named/saved to be retrievable from a dropdown, and potentially shareable. So for arguments sake there might be a "Wikipedia Hugo" configuration which works with any Wikipedia pages that have data displayed like this one. Or maybe it only works with that one page, depends on each usage.

From a UI perspective there would be a series of steps.

Step 1 - the user would specify their source - be it pasting in text from the clipboard, pointing to a URL, or importing from a file.

Step 2 - this is all about converting the source into a list of titles/authors. The user creates or selects an import profile to match that source. So here is where you will be doing your regex or xpath thing, with a view on the left of the raw text and a grid on the right showing the results. A checkbox will let you flip the author LN, FN to match whatever convention you use in your library.

Step 3 - the matching against books in your library. This will have lots of fun to resolve. How titles/authors are named in the list might be different to your library - author initials, suffixes, missing authors, title mismatches etc. Obviously there might be no matches because you simply don't have that book. Or there might be multiple matches, perhaps you have multiple editions or the title is too ambiguous. The user must be able to easily refine each book in the list to find their matching calibre book.

Step 4 - saving your list (if you decide to) by creating/updating a Reading List, and configuring the list as you do now.

For text based lists (like pasting from the clipboard) you will just specify a single regex, a bit like what you do when adding books, just to identify the title/author from it.

For web pages it will be more complex - I will assume that data is presented in a table and the user must specify the xpath expressions to identify the table of interest, the rows, and then the title and author data within. Anyone who has ever had to write a metadata source or Store plugin will be well familiar with the scraping techniques and challenges. It won't work with every website list out there, but hopefully it will cover enough.

There's also loads of scope for what else this feature might ultimately do. Things I haven't mentioned but may need thinking about early on. Exporting/sharing lists with other users. Sharing these configuration profiles - is it done like the Search the Internet plugins approach of some bundled but also exportable individually. What about updatable lists - if I have a "top 50 books of 2012" list from goodreads, this will change every month and I will want to minimise the effort of list updating.

Any thoughts appreciated... in particular what examples of websites/lists would you like to be able to import?

[Zetmolm]

That's a great idea, kiwidude!
If you build a feature like that I'm certainly going to use it. I'd like to be able to import lists of winners (and also shortlists) of literary awards, e.g. Booker, Pulitzer. And some time ago I found these lists of bestsellers for each year of the 20th century (starting from 1900); I'd like to import those lists as well.

[Stampercam]

I think its a fantastic idea, and I would use it regularly...

[PatNY]

I am having a problem with the Reading List Plugin. Now when I add a book to the reading list, it is not automatically given the matching tag in Calibre. I just noticed this today, soon after I updated to the latest version of Calibre.

However, it happened soon after I had to restore the calibre database when it had some errors in it, so that might be the cause?

Prior to this issue, I had been using the Reading List plugin for months, and had no problems with books automatically getting matching Calibre tags when I add them to the reading list.

Separate Questions: Can I uninstall and reinstall the plugin without losing my reading lists? Is there a way to backup the reading lists?

[kiwidude]

@PatNY - yes, I think you will have lost your configuration for your list. The plugin supports per library settings, storing the settings in the config file within the unique id calibre generates for that library. By "restoring" your library calibre will have given it a new library id, which now means that the plugin will not see any settings for it. Note this is not the only plugin that might be affected for the same reasons - restoring your library should only be used as a "last resort" if you hit irrecoverable problems with your main calibre database.

The settings still exist in the .json configuration file, but are just not visible to the plugin. Either you will need to setup your list(s) manually again, or you could close calibre and edit the json file manually. You would replace the new library id over top of the old one where all your configuration data is and remove the section for the new library id so it wasn't duplicated. Make a copy of the file before you edit, in case you screw it up, you do need to be careful to make sure you get all the commas and brackets etc correct if you go chopping stuff out.

The configuration and reading lists themselves are stored in a JSON file called "Reading List.json". You can find it from calibre by going to Preferences -> Miscellaneous -> Open calibre configuration directory, and looking in the "plugins" subfolder. There you will see both the Reading List.zip file for the plugin itself, and the Reading List.json file for its configuration.

You can uninstall/reinstall the plugin as much as you like, because it only changes the zip file and does not touch the configuration json file. If you want to completely start from scratch then you would delete/rename that json file.

Hope that helps...

[PatNY]

Hello kiwidude. Yes, unfortunately, I did lose my old reading lists. But since each book in the old lists had calibre tags, I was able to recreate the list by sorting on the relevant tag and then adding them to a new reading list. I gave the new list the same name as the old one. The problem is, now when I add a book to this new reading list, the tags aren't being added to Calibre anymore automatically.

I made sure the new lists are set up so that "tags" appear in the "Column to update" fields. So something is not working as before.

[kiwidude]

Did you remember to set the "When changing this list" dropdown to "Update column for add or remove" or similar rather than "Do not update calibre column"? And did you definitely configure it for the same list name that you are adding to if you have multiple lists?

It will be something obvious like that, the plugin hasn't changed and works fine with the latest calibre...

[PatNY]

Yes, the lists are set to "Update column for add/remove."

And I definitely gave the new lists the same name as before. Not only that, but I created 2 brand new "test" reading lists with names not previously used, and when I add books to them, the corresponding tags are NOT being added to Calibre anymore. Apparently, reconstructing the library somehow "broke" things, but I don't know how to fix it.

In addition to adding brand new "test" lists, other things I've tried are rebooting my PC; reinstalling (repairing) Calibre; uninstalling the plugin, then restarting calibre, then reinstalling the plugin. I also did a Calibre database integrity check and it found no major problems. Upon letting Calibre check the files against the database, it did find extra jpegs in some of the folders, but I had Calibre delete them and it made no difference. It's a mystery to me.

[kiwidude]

Interesting. You should try running calibre in debug mode (Ctrl+Shift+R) and see if there are any error messages being reported by the plugin at the time you add an item to a list. You could also try PM'ing me a link to your JSON config (or email it to me - address is at top of each .py file in the plugin zip).