Duplicate library mess

[travger]

Hello,
Some months ago I apparently decided to change the location of the library (at the time must have seemed like a good idea...). Of course I promptly forgot it and managed to add books into whichever one was open.
Now I have 2 libraries with lot of duplicates, but probably both have some entries with just metadata.opf.
As I understand, I have to select like 10-100 books from lib2 and right-click 'copy to lib1'.

How can I be sure duplicates are really the same file?
What happens when some books are just metadata files?

I did try 'check database integrity' on both libraries, but folders with no books remained.

Then I have folder named 'David Drake - RCN 05' in lib1, but there's no author with such name in the Calibre! How to force Calibre to see it?

And, lib2 shows in book details panel always format, but lib1 is very erratic. I can understand it in case of just metadata file, but some books have pdf and jpg files in a folder, still no format info in the panel.

Thank you!

[travger]

Ok, I try again.

1. Can I use Calibre to get rid of the author that I see only in folder view? (I'm afraid creating even more mess by just deleting it)

2. Why some books 'do not have available formats' ? There is a prc file in the corresponding folder!

3. Now I get error trying to check database integrity and trying to switch libraries. (I did copy 1 book from other library, now it shows formats).

ERROR: ERROR: Unhandled exception: <b>DatabaseException</b>:Connection closed
Remote

Traceback (most recent call last):
File "site-packages\calibre\gui2\actions\choose_library.py", line 246, in qs_requested
File "site-packages\calibre\gui2\actions\choose_library.py", line 243, in switch_requested
File "site-packages\calibre\gui2\ui.py", line 371, in library_moved
File "site-packages\calibre\gui2\library\views.py", line 363, in set_database
File "site-packages\calibre\gui2\library\views.py", line 242, in save_state
File "site-packages\calibre\gui2\library\views.py", line 236, in write_state
File "site-packages\calibre\library\prefs.py", line 51, in set
File "site-packages\calibre\library\prefs.py", line 44, in __setitem__
File "site-packages\calibre\library\sqlite.py", line 202, in run
DatabaseException: Connection closed
Remote

Another click gave:
Traceback (most recent call last):
File "site-packages\calibre\gui2\dialogs\book_info.py", line 48, in slave
File "site-packages\calibre\gui2\dialogs\book_info.py", line 96, in refresh
File "site-packages\calibre\gui2\library\models.py", line 376, in get_book_info
File "site-packages\calibre\gui2\library\models.py", line 367, in current_changed
File "site-packages\calibre\gui2\library\models.py", line 340, in get_book_display_info
File "site-packages\calibre\library\database2.py", line 693, in get_metadata
File "site-packages\calibre\library\database2.py", line 1333, in authors_with_sort_strings
File "site-packages\calibre\library\sqlite.py", line 202, in run
DatabaseException: Connection closed
Remote


Please help!

[itimpi]

Calibre will only show formats that have been inserted via the Calibre GUI. Files in a Calibre library folder that do not have entries in the Calibre metadata database are simply ignored.

[cybmole]

why not export both libraries to separate folders ( via send to folder) then merge those 2 folders in windows,
then use that merged folder to create a new calibre library ( after backing up & deleting both of the old ones)

[travger]

Yeah, but I hoped that there's a way to somehow point it to the right direction. I copied books from the second library and it helped, though C did not copy covers. Have to do copy-paste.

BUT, if I copy a pdf to another library that already contains different pdf with the same name - I will lose original one or must create double entries for every book and choose/delete manually.

I really, really do not want to start third library (for the books I've manually checked) while I've not sorted out the ones I have.

[travger]

Originally Posted by cybmole

why not export both libraries to separate folders ( via send to folder) then merge those 2 folders in windows,
then use that merged folder to create a new calibre library ( after backing up & deleting both of the old ones)

I feel quite stupid. I selected all books, >save to disk via single directory. Is this process called exporting a library? I can't find 'send to folder' anywhere.
In the resulting folder are only e-book files - no opf, and no jpg. If I import it, I lose all my metadata!

[cybmole]

sorry - yes it's called save to disc, not send to folder.

doing that should add the metadata to each book, as it's saved, so your metadata is "saved" within the books even though there is no metadata.db file.

so when you create new library using that output , all your metadata should still be there.

NB i'm being theoretical , I've not actually done this, so test with a few books & backup everythign beforehand.

[travger]

Nnoo-oo, that's not good. I have very few books with good metadata, most are in the process of aquisition. It involves lot of decisions, careful reading and editing - nothing so easy as bulk download. In this stage I definitely do not want metadata in the book.

Must I really start from scratch?

And
1. Can I use Calibre to get rid of the author that I see only in folder view? (I'm afraid creating even more mess by just deleting it).
But if C won't see it anyway, maybe it's ok to navigate to library and delete offending folders in win explorer...?

[speakingtohe]

Nnoo-oo, that's not good. I have very few books with good metadata, most are in the process of aquisition. It involves lot of decisions, careful reading and editing - nothing so easy as bulk download. In this stage I definitely do not want metadata in the book.

Must I really start from scratch?

Sounds like you have still a lot of work to do in the metadata editing part anyway.
It might be a very good time to merge in one way or another.
If you know which books you have manually checked, you can (if you haven't already) create custom columns and mark those as checked.
I had 8000 ebooks to sort and format etc. and it took me a couple of months but with the custom columns I was not constantly rechecking.

I have one y/n column for whether it was checked. Y=good format N=format problem indeterminate =not checked.

Another short text column indicates what the formatting is. When you have made an entry then it will give you a dropdown list so you can make your comments consistent.

That way you can sort your books based on formatting problems and fix a bunch at once if you so desire.

1. Can I use Calibre to get rid of the author that I see only in folder view? (I'm afraid creating even more mess by just deleting it).
But if C won't see it anyway, maybe it's ok to navigate to library and delete offending folders in win explorer...?

Pretty sure if you move your library it will leave those folders behind.
Also there is a faster database check which I cannot seem to locate in 7.023 which points out a lot of inconsistencies

[chaley]

Originally Posted by speakingtohe

Also there is a faster database check which I cannot seem to locate in 7.023 which points out a lot of inconsistencies

It is on the library pull-down menu, under 'Library Maintenance/Check Library'

[travger]

Originally Posted by speakingtohe

Also there is a faster database check which I cannot seem to locate in 7.023 which points out a lot of inconsistencies

what chaley said.
I get a window, but don't know what to do with it.

'Invalid authors,metadata.opf,metadata.opf,
Extra book formats,Cross the Stars (138),David Drake\Cross the Stars (138),Thumbs.db
Extra book formats,Fallen Angels (88),David Drake\Fallen Angels (88),Thumbs.db' -and lot more Thumbs.

Main reason why I want to save as much as possible - I need to learn for the future, so I know what I'm doing if something happens when my library has thousands of books.

[speakingtohe]

I think the thumbd.db are probably made by windows
You can safely delete them but they might come back
You can turn them off
http://www.annoyances.org/exec/show/article03-204

Main reason why I want to save as much as possible - I need to learn for the future, so I know what I'm doing if something happens when my library has thousands of books.

Best practice is AFAIK to copy entire library regularly (and to a separate device/computer every so often as well)
Hard drives do fail and I at least do something idiotic periodically
Helen

[DoctorOhh]

I'm a little confused on what you want. The following are areas I'm not confused on, maybe this info will be of some help.

Originally Posted by travger

In the resulting folder are only e-book files - no opf, and no jpg. If I import it, I lose all my metadata!

When you use the save to disk feature, if you check the appropriate boxes then the opf and jpg files are also saved (see attached).

Originally Posted by travger

1. Can I use Calibre to get rid of the author that I see only in folder view? (I'm afraid creating even more mess by just deleting it).
But if C won't see it anyway, maybe it's ok to navigate to library and delete offending folders in win explorer...?

If you go to the library icon on the top row you can choose to move your library to a new location. This will only move items calibre is aware of. After the move anything left in the original folder is yours to do with as you see fit.

As far as backing up the library is concerned, the best way is a complete copy of the library to another drive.

[travger]

Originally Posted by dwanthny

I'm a little confused on what you want. The following are areas I'm not confused on, maybe this info will be of some help.

When I moved the library, I expected to be left with empty place where it was. Instead, on the first glance it was indistinguishable from the new library. So after some weeks and lot of clicking around ('what happens if I click this' type) I probably switched them around and added new books to both. Somewhen I forgot that I have 2 libraries and noticed it only recently. (Well, I expected 1 full and 1 empty)
What I'm trying to determine is, what did I do wrong and is there an easy way to merge them.

I will try save to disk with appropriate checks and see what happens.

Now I'm somewhat leery about just moving the library, I am fairly sure that some of the books I moved lacked some data or formats. Hence this post - if in the future I move x000 books, how can I be sure that not a single one has left anything behind.
Like, should 'check Library' window be empty when library is ok? Can I correct things from there or is it just for info where the discrepancies are? Then why all those clickable buttons in the bottom?

[Perkin]

One thing you might be able to do is :

Enter library 1
Select all files
Save to disk, point to one store directory (in prefs ticking save opf+cover)
Enter library 2
Select all files
Save to disk, point to same directory or another
Create New library
Import from directory (the one saved to)
(if using different second director, Import again)