Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 03-27-2012, 08:12 AM   #1
PatNY
Zennist
PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.
 
PatNY's Avatar
 
Posts: 1,022
Karma: 47809468
Join Date: Jul 2010
Device: iPod Touch, Sony PRS-350, Nook HD+ & HD
Custom metadata in archived/backup books?

I have 5 custom metadata fields. When I archive books for backup purposes, I would like to make sure any data in those fields is kept in the book so that if I had to use the backup copies in the future all the custom metadata would still be there and show up.

Copying a book from within one library to another seems to be no problem. All the custom metadata shows up in the Calibre interface. But if I save the book to disk, and then import it into a library using "add books ..." not all the custom metadata shows up. One field does, but others do not.

I have a custom tweak to hide custom metadata from the content server, but that is so books imported into Stanza don't show that metadata. I would think that when you archive/save books, this shouldn't matter and all custom metadata should be saved.

So the question is, what settings/tweaks should I have to ensure that all books backed up have all custom metadata included? And what is the best way to process and copy these books for archiving purposes (ie, save to disk, or manually copy from a library directory or another way?).
PatNY is offline   Reply With Quote
Old 03-27-2012, 08:28 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,771
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
save to disk writes all custom metadata when creating the opf file it saves alongwith the book files. But if you want to backup your library, just copy the library folder.
kovidgoyal is online now   Reply With Quote
Advert
Old 03-27-2012, 08:52 AM   #3
PatNY
Zennist
PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.
 
PatNY's Avatar
 
Posts: 1,022
Karma: 47809468
Join Date: Jul 2010
Device: iPod Touch, Sony PRS-350, Nook HD+ & HD
Hi Kovid. I have 5 custom metadata fields, 4 of which are "text, column shown in tag browser" and 1 is "long text." Only the "long text" custom metadata shows up when I import a book into a library that was previously "saved to disk" from another library. The other 4 fields are empty.
PatNY is offline   Reply With Quote
Old 03-27-2012, 08:58 AM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,771
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
When you import a book custom metadata is read only if your target library has the exact same custom column definitions as the source library.
kovidgoyal is online now   Reply With Quote
Old 03-27-2012, 09:12 AM   #5
PatNY
Zennist
PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.
 
PatNY's Avatar
 
Posts: 1,022
Karma: 47809468
Join Date: Jul 2010
Device: iPod Touch, Sony PRS-350, Nook HD+ & HD
I don't understand. If it's read-only, shouldn't the data at least show up? Also why is the data in the "long text" field showing up but not in the other fields?

All custom column definitions are replicated in the target library, including the long text definition.
PatNY is offline   Reply With Quote
Advert
Old 03-27-2012, 09:26 AM   #6
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,771
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
The two libraries have to have thesame custom column names, datatypes and so on. Every custom column that matches will be imported, those that do not, will not.
kovidgoyal is online now   Reply With Quote
Old 03-27-2012, 10:21 AM   #7
PatNY
Zennist
PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.
 
PatNY's Avatar
 
Posts: 1,022
Karma: 47809468
Join Date: Jul 2010
Device: iPod Touch, Sony PRS-350, Nook HD+ & HD
Kovid, all my secondary libraries have the same exact structure/custom columns as the main library. A few months ago, I made sure of that by recreating all secondary libraries using the "copy structure from the current library" option checked. (Previous to that I didn't know that option existed and had been manually re-creating the custom columns in each separate library).

To double check, from within the main library, I just now created a brand new empty test library with that option checked. Then I imported a book into the test library that had previously been "saved to disk" from within the main library. Again, the long text field imported fine; but all the short text fields did not.

Also, if I simply copy that same exact book from within the main library to the test library (or all secondary libraries), all the custom metadata shows up fine -- both the long text and short text. If the library structures weren't exact, then that wouldn't happen.

Last edited by PatNY; 03-27-2012 at 10:28 AM.
PatNY is offline   Reply With Quote
Old 03-27-2012, 12:05 PM   #8
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,771
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Note that save to disk and Add Books will work only if you leave the default save to disk options and then use add books from subfolders one book per folder when adding books. That will allow the OPF file generated bya default save to disk to be used. Which is why I said, in the first post, that if you want to backup a library, simply copy its folder, its a lot easier and more bulletproff than save to disk.
kovidgoyal is online now   Reply With Quote
Old 03-27-2012, 02:02 PM   #9
PatNY
Zennist
PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.
 
PatNY's Avatar
 
Posts: 1,022
Karma: 47809468
Join Date: Jul 2010
Device: iPod Touch, Sony PRS-350, Nook HD+ & HD
Quote:
Originally Posted by kovidgoyal View Post
Note that save to disk and Add Books will work only if you leave the default save to disk options and then use add books from subfolders one book per folder when adding books. That will allow the OPF file generated bya default save to disk to be used.
Just to be sure, I went into Preferences>Saving books to disk and clicked on "Restore defaults" and there was no difference. The custom long text field carried over, but none of the "text, column shown in tag browser" data shows up. And I was adding from a subfolder with just one book in it.

Quote:
Which is why I said, in the first post, that if you want to backup a library, simply copy its folder, its a lot easier and more bulletproff than save to disk.
Well, I actully do have a mirror copy of my entire library on sugar sync. But I also like to keep another backup copy after I finish processing a book. Maybe that is unnecessary, but I don't like to rely solely on the cloud for backups. I've had really bad luck losing data with dropbox. So instead of having to backup the entire library every time I process a book, I just like to add it incrementally to a manual backup on an external hard drive.

What I don't get is why there is no issue if I copy a book from within one library to another, and why the long text custom metadata field will show up but not the shorter text field??
PatNY is offline   Reply With Quote
Old 03-28-2012, 03:58 AM   #10
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by PatNY View Post
Just to be sure, I went into Preferences>Saving books to disk and clicked on "Restore defaults" and there was no difference. The custom long text field carried over, but none of the "text, column shown in tag browser" data shows up. And I was adding from a subfolder with just one book in it.
Did you use "Add books from directories, including sub-directories (One book per directory ...)"? That option uses the metadata in the .opf that save-to-disk created in the same directory as the book format.
chaley is offline   Reply With Quote
Old 03-28-2012, 04:05 PM   #11
PatNY
Zennist
PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.
 
PatNY's Avatar
 
Posts: 1,022
Karma: 47809468
Join Date: Jul 2010
Device: iPod Touch, Sony PRS-350, Nook HD+ & HD
Hello Charles,

No, I wasn't using that. I didn't realize that was a separate option. I thought Kovid was just talking in general terms. But I just tried importing a book previously saved to disk using that menu option and it made no difference. The long text custom data field shows up, but the short text fields will not.

Am I the only one experiencing this issue? I can't imagine why.

I have the latest version of Calibre installed and am otherwise not seeing any issues with how it runs.

Also, keep in mind if I simply copy a book in the main library to a secondary library all fields show up!

Any other ideas?
PatNY is offline   Reply With Quote
Old 03-29-2012, 07:49 AM   #12
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
It mostly works for me. There are problems with text/non-multiple columns.

My steps to test:

1) create an empty library with copy structure.
2) switch back to the original library
3) Find a book with only .mobi format. I chose mobi so that there wouldn't be any significant metadata stored in the book.
4) ensure that all the custom columns of every type for that book have data in them. Includes text, enumeration, text/multiple, int, float, yes/no, date, and composite.
5) Save the mobi format to disk. After the save, I see in the saved-to folder:
-- book.mobi
-- metadata.opf
-- book.jpg
6) switch to the empty library
7) do "add books from directories, including sub-directories (One book per directory ...)"

At this point I see the book with all the custom columns filled in *except* the text/non-multiple ones. There is indeed a problem here, and I have submitted a fix to Kovid.
chaley is offline   Reply With Quote
Old 03-29-2012, 04:56 PM   #13
PatNY
Zennist
PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.
 
PatNY's Avatar
 
Posts: 1,022
Karma: 47809468
Join Date: Jul 2010
Device: iPod Touch, Sony PRS-350, Nook HD+ & HD
TYVM, Charles. I appreciate your help.

Is the issue in the "saved to disc" books? Or in how the program imports those books? If it's in the books themselves, then I might look into redoing my manual archives to ensure they include all the custom metadata.
PatNY is offline   Reply With Quote
Old 03-29-2012, 05:10 PM   #14
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by PatNY View Post
Is the issue in the "saved to disc" books? Or in how the program imports those books? If it's in the books themselves, then I might look into redoing my manual archives to ensure they include all the custom metadata.
It was in the import. The info written to the OPF was/is correct and has not changed. The error was in interpreting the info in the file as it was read back.
chaley is offline   Reply With Quote
Old 03-29-2012, 06:50 PM   #15
PatNY
Zennist
PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.PatNY ought to be getting tired of karma fortunes by now.
 
PatNY's Avatar
 
Posts: 1,022
Karma: 47809468
Join Date: Jul 2010
Device: iPod Touch, Sony PRS-350, Nook HD+ & HD
OK. great. That means I don't have to re-save any of my previously saved books. But as a result of going through this issue, I am rethinking my backup strategy anyway, which may make the issue moot for me. I now realize that restoring a library via the "add books" method may not be very feasible or time efficient. So I am considering doing a second mirror backup of the entire library locally instead of just keeping a mirror in the cloud plus manual backups. Or I may just abandon the cloud backup and stick with a local mirror which I feel is safer. At any rate, thanks again.
PatNY is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Metadata Backup? Mountainjack Calibre 2 07-17-2011 04:42 PM
New Metadata class (empty books) and custom columns kiwidude Development 14 03-12-2011 06:47 AM
Kindle, Calibre and Archived Books nynaevelan Calibre 2 11-23-2010 06:10 PM
Kindle, Calibre and Archived Books nynaevelan Amazon Kindle 5 11-23-2010 05:21 PM
Troubleshooting Kindle for PC: Won't Download any Archived Books matt314159 Amazon Kindle 11 08-05-2010 09:20 PM


All times are GMT -4. The time now is 04:05 AM.


MobileRead.com is a privately owned, operated and funded community.