Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 12-19-2009, 11:52 AM   #1
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Firstname as author - metadata.db and SQLite3

I've got about 10,000 books imported into Calibre, and most were originally in the lastname, firstname - optional series name - title.ext format.

I customized the regex so the import (5 hours) went well, but now realize that I've inadvertently imported them as multiple author books where the firstname is a second author.

I'd like to fix this, but could use some suggestions.

I'm comfortable in SQL, and would like to simply run a query to change the author name in metadata.db from lastname| firstname to firstname lastname in the authors table.

However, I realize there are several problems doing that. First, there is the author_sort field in the books table that would also need to be fixed. That's pretty easy too.

The next issue is that the directory structure is now lastname_ firstname and it should be firstname lastname. That needs to be fixed in both the table and in the actual folder structure.

I've noticed that all this is handled automatically inside Calibre when the author is changed, so I'm wondering if I can change just the author field inside metadata.db, then run Calibre or python to fix the directory.

Alternatively, Does anyone have any suggestions for fixing this? I can manually change the database and folders, if necessary. Another alternative is to go back to the import (another 5 hours), but I wasn't sure if the regex would allow me to swap firstname and lastname using groups and \1 \2 etc.

Another possibility is to modify the names of the files, then reimport. Finally, I can possibly create an xml file from the names, use macros in ultraedit to fix the xml file, then import from the xml.

I'd appreciate some comments before I go forward.

Thanks!
Starson17 is offline   Reply With Quote
Old 12-19-2009, 12:05 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,809
Karma: 5006091
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
If you're comfortable with scripting you could use the calibredb command to do this
kovidgoyal is offline   Reply With Quote
 
Enthusiast
Old 12-19-2009, 12:27 PM   #3
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by kovidgoyal View Post
If you're comfortable with scripting you could use the calibredb command to do this
Thanks. I really appreciate it. Can you give me a bit more info to steer me in the right direction? I'm comfortable with scripting.
Starson17 is offline   Reply With Quote
Old 12-19-2009, 12:31 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,809
Karma: 5006091
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
The user manual contains full documentation for all command line calibre utilities. Using calibredb you can read/write metadata and the write operation will cause calibre to move files into the correct folders for the new names.
kovidgoyal is offline   Reply With Quote
Old 12-19-2009, 12:45 PM   #5
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by kovidgoyal View Post
The user manual contains full documentation for all command line calibre utilities. Using calibredb you can read/write metadata and the write operation will cause calibre to move files into the correct folders for the new names.
OK, so does this basic strategy seem OK:

1)calibredb list into a text file (author names wrong, but get the id)

2)process the text file to change it into a batch to do multiple calibredb writes with the fixed author name and matching id?

I've done step 1 already
Starson17 is offline   Reply With Quote
Old 12-19-2009, 12:53 PM   #6
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by Starson17 View Post
process the text file to change it into a batch to do multiple calibredb writes with the fixed author name and matching id?
It looks like set_metadata is what I want for step 2 and I need to figure out the opf format to tell it to write a new author?
Starson17 is offline   Reply With Quote
Old 12-19-2009, 01:27 PM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,809
Karma: 5006091
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
use show_metadata to see the format of an OPF (it's just a simple XML file)
kovidgoyal is offline   Reply With Quote
Old 12-19-2009, 02:04 PM   #8
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by kovidgoyal View Post
use show_metadata to see the format of an OPF (it's just a simple XML file)
I've done that, and I've done a set_metadata (which worked great), and it looks like I can set just the author in the opf file if I want.

Do you have some procedure in mind that I might try? I'm thinking of something like this metascript:

cycle through all possible id's, doing a show_metadata
pipe to sed to fix firstname and lastname
pipe to set_metadata.

Last edited by Starson17; 12-20-2009 at 09:56 AM.
Starson17 is offline   Reply With Quote
Old 12-20-2009, 09:54 AM   #9
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by Starson17 View Post
I'm thinking of something like this metascript:

cycle through all possible id's, doing a show_metadata
pipe to sed to fix firstname and lastname
pipe to set_metadata.
Just an update and a thanks. I ended up doing this in Windows and used batch files with FOR /L loops to go through all the IDs. The first step wrote all the opf files, the second step used sed to process them and fix the author names, and the last loop did the set_metadata to put them back into the database.

I don't know if it's a bug, but for some reason, calibredb always seemed to stop execution of the batch file when it completed. This caused it to ignore any other lines in the same batch file. I had to do this job in two batch files, one to make the opf files, and one to sed process then set the metadata.

The process time was about 10 seconds per book, so this is taking a while, (still running 8 hours later). Anyway, thanks for the help.
Starson17 is offline   Reply With Quote
Old 12-20-2009, 10:58 AM   #10
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,809
Karma: 5006091
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Ah yes on windows for some reason the calibre command line utilities dont play well with the cmd.exe interpreter. Better to use powershell.
kovidgoyal is offline   Reply With Quote
Old 12-20-2009, 01:11 PM   #11
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by kovidgoyal View Post
Ah yes on windows for some reason the calibre command line utilities dont play well with the cmd.exe interpreter. Better to use powershell.
It confused me for a while. First I thought my batch file logic was faulty, then I thought the program after the calbredb command was causing the trouble, then I tried a couple of tricks to work around it. I even thought about doing this on one of my *nix boxes, or under Cygwin, but I wasn't sure if the db would need to be converted, so I gave up and just ran the calibredb stuff separately. Interestingly, multiple invocations of the calibredb command in a single FOR loop worked fine. I just couldn't get any subsequent lines or calls to batch subroutines or external batch files to work.

Anyway, thanks again. If I do this again, I'll try powershell.
Starson17 is offline   Reply With Quote
Old 12-20-2009, 02:01 PM   #12
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,518
Karma: 5567081
Join Date: Aug 2009
Location: The (original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by Starson17 View Post
It confused me for a while. First I thought my batch file logic was faulty, then I thought the program after the calbredb command was causing the trouble, then I tried a couple of tricks to work around it. I even thought about doing this on one of my *nix boxes, or under Cygwin, but I wasn't sure if the db would need to be converted, so I gave up and just ran the calibredb stuff separately. Interestingly, multiple invocations of the calibredb command in a single FOR loop worked fine. I just couldn't get any subsequent lines or calls to batch subroutines or external batch files to work.

Anyway, thanks again. If I do this again, I'll try powershell.
Calibre does works Dual Boot (XP <->Ubuntu Karmic) with the same XP folder used for the library.

The Gotcha
is folder names are CASE sensitive in Linux and correcting a title case in Windows Calibre does NOT change the original (named) folder's case (since it matters naught to M$),but does matter to Calibre.
Running the DB Integrity tool showed many errors. Looking in the file manager had me until I thought of Case
I fixed the folder names externaly and Calibre was happy.
I believe the "Use Lower case for folder names" might keep this from happening ON NEW ENTRIES.
theducks is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
How To: Getting summary metadata and images, not tags, author, title Starson17 Calibre 21 01-10-2011 09:51 AM
List index out of range, 0.6.46 downloading metadata, no author. highwaykind Calibre 4 04-04-2010 07:31 AM
jetBook reads metadata with FB2 for title & author tselling Ectaco jetBook 0 01-07-2010 09:54 PM
Lastname, Firstname filenames - Any way to import correct author name? Starson17 Calibre 5 12-23-2009 11:03 PM
MetaData/Cover picture via librps? Author sort? OUTATIME Sony Reader Dev Corner 5 02-23-2008 06:44 PM


All times are GMT -4. The time now is 07:20 AM.


MobileRead.com is a privately owned, operated and funded community.