Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 09-13-2012, 09:11 PM   #16
Sidetrack
Enthusiast
Sidetrack began at the beginning.
 
Posts: 34
Karma: 10
Join Date: Jan 2009
Location: South Pacific
Device: Kindle DX
@Kiwidude
Excellent, thanks! That will work for the moment. Just a little search and replace after the fact to set the index. Just to be clear, there is no option that will set any metadata on existing/matched books, right?

I can see I still have a bit of learning to do in the XPath department. Love your plugins

Cheers mate.

edit- FYI, as it turns out that info seems to make a better tag than series info so I'm running a little search and replace regex on it after the fact anyway. Having it all in one place in the series field is workable. I think I'd be more apt to want to regex actual series info out of the title and set a tag with the other info.

Oryx and Crake (MaddAddam Trilogy, #1)
by Margaret Atwood
Man Booker Prize Nominee for Shortlist (2003)

Title: Oryx and Crake
Author: Margaret Atwood
Series: MaddAddam Trilogy [1]
tag: accolade.Man Booker Prize.Nominee.Shortlist.2003

Last edited by Sidetrack; 09-13-2012 at 10:15 PM.
Sidetrack is offline   Reply With Quote
Old 09-14-2012, 08:35 AM   #17
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,228
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@Sidetrack - correct, this plugin has no ability to update metadata for an existing book.

That is the second time this week it has been mentioned though, so I'm sure it might be a feature one day. I just haven't had time to give it any thought as to how that might actually work from a UI perspective, as it is potentially complicated to manage. The next release I am working on currently allows a variable number of fields to be imported. Somehow the user would need the option somewhere to choose to overwrite data for an existing book...
kiwidude is offline   Reply With Quote
 
Enthusiast
Old 09-14-2012, 01:06 PM   #18
cirocco
Member
cirocco began at the beginning.
 
Posts: 13
Karma: 10
Join Date: Aug 2007
Device: Kindle 4
I too wanted to use the plugin today to correct some database errors

Being a programmer (although I only know very little python) I modified your plugin a little to allow database modification. For now I added two buttons "Update author" and "Update title", I cannot be sure if they always work correctly but I used it few times with success. Unfortunately my source site has author as LN FN and I have not resolved how to handle that.

I could upload modified file, but it for sure needs a lot of improvements.
cirocco is offline   Reply With Quote
Old 09-14-2012, 02:36 PM   #19
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,228
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@cirocco - kudos to you for being brave enough to go fiddling with the code .

It has been massively restructured for the new version and large swathes of things had to be changed, so the code is very different now. The problem is that if *I* add such a feature I will be doing it in such a way that *any* field could be imported. Which will involve a lot of navel contemplation and effort - sadly sneaking a few extra buttons on screen won't be an option. Particularly when you have to start supporting custom columns, date fields etc it all starts getting very complicated.

However as you have already used this plugin for such a purpose, you or of course others could provide valuable input into how the "official" version might look/work. Especially since I don't have a lot of time to think about it right now.

Completely off the top of my head, I'm thinking that there would be a button somewhere on the "Resolve" screen, which would give the user a popup allowing them to select which column(s) of data to import (by default it will be none). A bit like metadata download. So the user will immediately see the effect in the matches grid. It wouldn't be a per book setting, it would be an all or nothing overwrite.

Would that work well enough? Is it fair that it always defaults to importing no columns, and every time you use the wizard you decide which column(s) to import?

Any comments welcomed, no promises on it being implemented of course.
kiwidude is offline   Reply With Quote
Old 09-14-2012, 04:31 PM   #20
cirocco
Member
cirocco began at the beginning.
 
Posts: 13
Karma: 10
Join Date: Aug 2007
Device: Kindle 4
regexp problem

Choosing columns to update is better solution than only few buttons, just too hard for me , default to update nothing is good too. But I think "all or nothing overwrite" is not good, my buttons work for selected books (with single or multiple matches) which is is more flexible.

I have had a lot of troubles with XPath for series idx. Here is the sample html:

Spoiler:
Code:
<li>
   <a href="book.aspx?id=106">Johnny i zmarli</a>
   <a class="icon" href="/book.aspx?id=106#reviews">
      <img src="images/report.png" style="display:inline; vertical-align:-4px; margin-left:5px;" title="do książki dodano opisy lub recenzje" /></a><a class="icon" href="/book.aspx?id=106&mode=bookshare#bookshare"><img class="bookExchangeIconAjax" id="106" src="images/group_go.png" style="display:inline; vertical-align:-4px; margin-left:5px;" />
   </a> 
   (tom: 2) 
</li>


with
Code:
../text()
I got (with newlines)
Code:
     (tom: 2)
but could not get the number out of it. I eventually gave up and used python to get it.
Is it possible to do it with XPath only? If not maybe some support in python? seeking number anywhere in given string?
cirocco is offline   Reply With Quote
Old 09-14-2012, 04:50 PM   #21
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,228
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@cirocco - there are some things you can do in xpath with replacements, but really it is all an awful lot easier using a regex. The next version allows you to specify a strip at the per field level - its the same issue as the recent posts about extracting series information from Goodreads....
kiwidude is offline   Reply With Quote
Old 09-14-2012, 08:30 PM   #22
Sidetrack
Enthusiast
Sidetrack began at the beginning.
 
Posts: 34
Karma: 10
Join Date: Jan 2009
Location: South Pacific
Device: Kindle DX
Quote:
Originally Posted by kiwidude View Post
@Sidetrack - Somehow the user would need the option somewhere to choose to overwrite data for an existing book...
Can't speak for everyone, but the ability to pick out information and create a tag to add, which wouldn't have to overwrite anything, would fit my needs.
Sidetrack is offline   Reply With Quote
Old 09-15-2012, 04:20 AM   #23
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,228
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@Sidetrack - indeed, tag type fields would be treated specially to append data to them, not overwrite.

I think the bigger complication now around a ui for this is if trying to support only applying updates to some books.

Either I stick to it applying to update all matched books (which I see as the majority use case). And if you don't want one books data to be overwritten, then you just clear any match for it/ leave it unmatched.

Or I make it a selection based action (like all the others are) with a right click menu option and button on the right side. So this "Update metadata" option will give you the dialog to choose which columns to update, remembering your choice from last time. I can see that working quite well, though you will have the complications of if the user changes their mind etc. or the user could just clear the match and start again.

There are pros and cons with both. The second option is more flexible, but more prone to user error. That flexibility means if the user forgets to select a row it won't get updated.

Any thoughts welcomed.
kiwidude is offline   Reply With Quote
Old 09-15-2012, 04:55 AM   #24
cirocco
Member
cirocco began at the beginning.
 
Posts: 13
Karma: 10
Join Date: Aug 2007
Device: Kindle 4
Just one more problem remark, I had to use python for the author too , to change LN FN into FN LN, and to deal with extra data like in this one "Zajdel Janusz A. (Zajdel Janusz Andrzej)". And this time I have no good idea how to do it generally.

So now I have customized plugin which can update title, author and series information, but works only on one site

And to add my opinion
Quote:
That flexibility means if the user forgets to select a row it won't get updated.
I think that not updating by mistake is much better than updating by mistake (when there is no undo function)
cirocco is offline   Reply With Quote
Old 09-15-2012, 05:03 AM   #25
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,228
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@cirocco - Thanks for the comments, all input is appreciated around this.

The extra data is easily stripped off with a regex on the author field. And flipping author names is one of those topics that gives me continuous grief - however I would just apply the same approach that I added to the calibre metadata download options dialog of a "Swap author names from FN LN to LN, FN" checkbox. So you can decide at the time you tick the fields whether to flip it or not. The authors column in the Resolve grid is currently editable only for empty books, I could loosen that restriction. So if the automatic name swapping doesn't guess it right (it is impossible to correctly handle all cases) you can manually type the correct name. Or you can wait until you click Finish on the wizard and the books are displayed in the library, and fix them there.
kiwidude is offline   Reply With Quote
Old 09-25-2012, 06:40 PM   #26
Sidetrack
Enthusiast
Sidetrack began at the beginning.
 
Posts: 34
Karma: 10
Join Date: Jan 2009
Location: South Pacific
Device: Kindle DX
Hey kiwidude,

How difficult would it be to iterate through pages of a particular website all at once. I'm thinking that the user could feed the plugin the largest page url or suffix (eg. '?Page=13') easily enough so that it could build a complete list from the individual pages.
Sidetrack is offline   Reply With Quote
Old 09-25-2012, 06:52 PM   #27
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,228
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@Sidetrack - it isn't trivial but I imagine not impossible. Like so many of these options most of the work comes in trying to figure out a UI for it without making a complex plugin even more incomprehensible! How well it might work depends on whether websites really do have a simple page=xxx type of querystring to manipulate, not all do.

I'll take a look at it "one day", but there are a whole bunch of other bigger changes that I'm part way through that I want to finish off and release first. And since we are fine tuning the Sigil release at the moment I've had no time to come back to this plugin, perhaps next week...
kiwidude is offline   Reply With Quote
Old 10-16-2012, 10:51 AM   #28
SFReader2012
Junior Member
SFReader2012 began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Oct 2012
Device: Kindle for Android
Loved the idea of this. I have just tried to import my list from LibraryThing where I have 695 books recorded.
Whichever export I use - xls or csv - I get the message "UnicodeDecodeError:'utf8' codec can't decode byte )xff in position 0: invalid start byte.
The files open in excel, and I have saved them again as csv but it does not help.

Sad that the add-on does not support this great cataloguing site. Goodreads which is supported seems more of a social network site than a catalogue.
SFReader2012 is offline   Reply With Quote
Old 10-16-2012, 11:00 AM   #29
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,228
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@SFReader2012 - welcome to MobileRead

As it happens I am literally working on the next version of this plugin at the moment, so if there is an issue with the csv import now would be a good time to fix it. Any chance you could either attach a sample file which has the error to a post, or PM me a link to it, (or email it to me, my email address is in any of the files inside the plugin zip). I don't use LibraryThing myself to generate my own sample and it could well be specific to some title in your catalogue.
kiwidude is offline   Reply With Quote
Old 10-19-2012, 05:43 PM   #30
SFReader2012
Junior Member
SFReader2012 began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Oct 2012
Device: Kindle for Android
Information sent. Librarything exports as either csv or xls (tab delimited text). Also as MARC data, which I have never heard of before.

Sorry for delay. Thanks for looking at it.
SFReader2012 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Reading List kiwidude Plugins 660 10-10-2014 04:36 PM
[GUI Plugin] Manage Sony x50 Reader Book List kpw Plugins 170 10-02-2014 08:23 PM
Import List plugin idea thread kiwidude Development 45 08-13-2012 06:05 AM
[GUI Plugin] WebOS Kindle-Import CranstD Plugins 0 01-24-2012 03:36 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 12:05 PM.


MobileRead.com is a privately owned, operated and funded community.