Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 10-22-2012, 12:48 PM   #31
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,588
Karma: 2089838
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
@SFReader2012 - I just took a look at the CSV file, thanks. It comes down to some nasty quality data coming out of LibraryThing. I have put in place a workaround for the next plugin version which handles this situation appropriately.
kiwidude is offline   Reply With Quote
Old 10-22-2012, 04:52 PM   #32
SFReader2012
Junior Member
SFReader2012 began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Oct 2012
Device: Kindle for Android
Thanks. Very prompt and helpful response. Look forward to the revised plugin in due course.
SFReader2012 is offline   Reply With Quote
Old 10-22-2012, 05:49 PM   #33
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,588
Karma: 2089838
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
Beta for version 1.1

Ok, this is a major advancement of this plugin, in three key areas:
  1. You can now import into a variable number of columns, including custom columns and identifier fields
  2. You can now use this plugin as a way to update metadata for existing books in your library.
  3. More granular control at field level when scraping from a web page of regexes to include/exclude against the data retrieved. This makes it easier to more precisely extract a series name/index for instance.
And there are a bunch of other fixes and tweaks as well. So the range of possible uses for this plugin have increased for those who need some more flexible behaviour.

It would appreciated if a few of you could give this build a whirl before I officially release it to see if you can break it etc.

Last edited by kiwidude; 10-25-2012 at 04:49 PM. Reason: Removed attachment as officially released
kiwidude is offline   Reply With Quote
Old 10-25-2012, 07:22 AM   #34
Gunnerp245
Gadget Freak
Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.
 
Gunnerp245's Avatar
 
Posts: 1,169
Karma: 1043832
Join Date: Nov 2007
Location: US
Device: EE, Note 8
Thumbs up Beta for version 1.1

Quote:
Originally Posted by kiwidude View Post
Beta for version 1.1
Short test drive last night, did not see any obvious issues. Still have to figure out the series.idx regex, though.
Gunnerp245 is offline   Reply With Quote
Old 10-25-2012, 07:42 AM   #35
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,588
Karma: 2089838
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
@Gunnerp245 - thanks for feeding back. I think I am going to release it as is tonight since no-one has reported an issue, just need to update all the screenshots.

As for the regex, you can take a look at any of the Goodreads examples included with the plugin for what they do which should hopefully help. Point both series and series index at the same text, then use something like these:

Series field: "Include matching text" of: \(([^,\.]+)
Series index field: "Include matching text" of: #([\d\.]+)
kiwidude is offline   Reply With Quote
Old 10-25-2012, 04:51 PM   #36
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,588
Karma: 2089838
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
v1.1.0 Released

Changes in this release:
  • Allow importing into custom columns
  • Allow updating metadata of existing books in your library from data off the list (standard or custom columns)
  • Allow clipboard, csv and web page import to retrieve into a dynamic set of calibre standard columns including identifier fields
  • Dynamically change the Preview, Search and Match columns presented to match what is configured for the import source
  • Remove the ability to specify columns to display on the configuration screen
  • Fix matched count on the Resolve step so multiple matches are not counted as matched
  • Fix for importing non utf-8 csv files such as for titles from LibraryThing
  • Allow direct editing of plugin configuration data from config dialog (use at your own risk!)

To those of you who tried the beta in the post above a few days ago, please force another download of the plugin (it has the same version number as the beta but I did find a couple of things to fix).
kiwidude is offline   Reply With Quote
Old 10-25-2012, 08:26 PM   #37
Gunnerp245
Gadget Freak
Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.Gunnerp245 ought to be getting tired of karma fortunes by now.
 
Gunnerp245's Avatar
 
Posts: 1,169
Karma: 1043832
Join Date: Nov 2007
Location: US
Device: EE, Note 8
Thumbs up Fantastic!

Quote:
Originally Posted by kiwidude View Post
As for the regex, ... Point both series and series index at the same text, then use something like these:
Series field: "Include matching text" of: \(([^,\.]+)
Series index field: "Include matching text" of: #([\d\.]+)
@kiwidude - That worked perfectly!

An update to my original post https://www.mobileread.com/forums/sho...0&postcount=11. I desired to know the date I added books to my Goodreads 'to read shelf' (I use recommendations to find interesting books). Adding a custom column in calibre;
  1. Lookup name - gadd
  2. Column Heading - GA
  3. Column Type - Text,column shown in the tag browser
  4. Restart calibre!
Added GA to the 'select fields to import'. After some trial and reading, reading XML Path Language (XPath), determined the following expression would find and display the correct information:
  • td[@class="field date_added"]/div[@class="value"]/span[@title=@*]/text()

Spoiler:


Excellent kiwidude!
Gunnerp245 is offline   Reply With Quote
Old 10-29-2012, 01:32 PM   #38
tcrabb
Reader
tcrabb began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Jul 2010
Device: PRS-300
Quote:
Originally Posted by kiwidude View Post
Changes in this release:
Allow importing into custom columns
I've exported two Goodreads accounts as csv files, and attempted to import them such that 'My Rating' from one file goes in one custom column, and into a different custom column when I import the second file. [MyRating1; MyRating2]

(I'm actually doing this for many fields, but let's keep the example simple!)

These two accounts have several common/matching books, so the same book will be in both files.

It appears that the second import erases the information in the columns related to the first file as it populates the columns related to the second file.

Is there a way to prevent this?
tcrabb is offline   Reply With Quote
Old 10-29-2012, 01:45 PM   #39
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,588
Karma: 2089838
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
@tcrabb - I'm not quite sure exactly what you are doing?

You should have two separate CSV configurations setup. One will populate a MyRating1 column, and have no field defined to import for MyRating2. And vice versa.

You import from file 1, that creates all the books and populates MyRating1.

You then load the configuration for file 2. When you hit the Step 2 match screen, it will show some books already exist (from your first file). At this point you use the Update Metadata feature, and tick only the MyRating2 column to be updated for those selected books. All the other books (which are new that you have created empty books for) should have values automatically set for MyRating2.

Even if you did put both MyRating1 and MyRating2 in your same configuration, then it should still work, provided on the Update metadata screen for the selected rows you make sure to only tick the MyRating2 column.
kiwidude is offline   Reply With Quote
Old 10-29-2012, 03:03 PM   #40
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 29,659
Karma: 54369090
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Just got around to trying this to build a list of missing series (using the Clip board).
The creation of (missing) Empty books works Great but some data was not available.

Some questions:
How to merge non-included/incomplete data?
ex. Nomads of Gor (1969) ISBN 0-7592-5445-1
I can trap all fields shown:

the date 1969 is not acceptable to Calibre (failed to show up in Published), it needs to be 1/1/1969
I need to pad a fake MM/DD/

The Author name, series name, is not available,
I would like to force it: John Norman and Gor

The series index is not available, but is implied by the position in the list (first is 1)

BTW I am snagging series info off of the Wikipedia page for the Author
theducks is offline   Reply With Quote
Old 10-29-2012, 03:40 PM   #41
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,588
Karma: 2089838
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
@theducks - I just used this expression for your example:
Code:
(?P<title>.*?) \((?P<pubdate>.*)\) ISBN (?P<identifier:isbn>.*)
If you want to populate the author, just paste it directly into the preview grid on the right side of that first Step 1 of the wizard. All columns are editable there, so you can do any "input" cleanup you require before moving to Step 2.

I had no issues with populating the published date with just that 1969 - calibre created the empty book with a date of 1 Jan 1969 with no extra help.
kiwidude is offline   Reply With Quote
Old 10-29-2012, 04:50 PM   #42
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 29,659
Karma: 54369090
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Quote:
Originally Posted by kiwidude View Post
@theducks - I just used this expression for your example:
Code:
(?P<title>.*?) \((?P<pubdate>.*)\) ISBN (?P<identifier:isbn>.*)
If you want to populate the author, just paste it directly into the preview grid on the right side of that first Step 1 of the wizard. All columns are editable there, so you can do any "input" cleanup you require before moving to Step 2.

I had no issues with populating the published date with just that 1969 - calibre created the empty book with a date of 1 Jan 1969 with no extra help.
Weird The year showed in the column during the test, but not after import.
Ah! I did not realize I could edit fields.

theducks is offline   Reply With Quote
Old 10-29-2012, 05:49 PM   #43
tcrabb
Reader
tcrabb began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Jul 2010
Device: PRS-300
Quote:
Originally Posted by kiwidude View Post
When you hit the Step 2 match screen, it will show some books already exist (from your first file). At this point you use the Update Metadata feature, and tick only the MyRating2 column to be updated for those selected books. All the other books (which are new that you have created empty books for) should have values automatically set for MyRating2.

Even if you did put both MyRating1 and MyRating2 in your same configuration, then it should still work, provided on the Update metadata screen for the selected rows you make sure to only tick the MyRating2 column.
My problem boiled down to being unable to find the option to select the matched books - then I decided to try a right-click and found it!

Thanks!
tcrabb is offline   Reply With Quote
Old 12-09-2012, 07:48 AM   #44
Pepin33
Zealot
Pepin33 has a complete set of Star Wars action figures.Pepin33 has a complete set of Star Wars action figures.Pepin33 has a complete set of Star Wars action figures.Pepin33 has a complete set of Star Wars action figures.Pepin33 has a complete set of Star Wars action figures.
 
Posts: 109
Karma: 419
Join Date: Aug 2012
Location: Spain
Device: Kindle Touch
A question about quotes in CSV files: When using quotes to delimit fields, if inside the data are already quotes, the field is badly chopped.

For example, if the CSV contains
Code:
"Mi book","This is a "description" of the book","My author"
the fields after importation are not correct.

With other programs (like Notepad++), it's possible to have quotes inside the fields, because it only keep in mind the first quotes after the field separator and the last before new separator.

Is it possible for the plugin working at the same way?

Last edited by Pepin33; 12-09-2012 at 07:50 AM.
Pepin33 is offline   Reply With Quote
Old 12-09-2012, 08:31 AM   #45
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,588
Karma: 2089838
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
@Pepin33 - that's a pretty ugly CSV file . This plugin actually uses a standard python library (csv) to read the file. Looks like it doesn't handle this situation...
kiwidude is offline   Reply With Quote
Reply

Tags
column types

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Reading List kiwidude Plugins 1309 03-16-2024 11:52 PM
[GUI Plugin] Manage Sony x50 Reader Book List kpw Plugins 170 10-02-2014 08:23 PM
Import List plugin idea thread kiwidude Development 45 08-13-2012 06:05 AM
[GUI Plugin] WebOS Kindle-Import CranstD Plugins 0 01-24-2012 03:36 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 03:16 AM.


MobileRead.com is a privately owned, operated and funded community.