Refreshing the library

[Patrick Derwael]

Hi there,

I have installed Calibre a few weeks ago, and unleashed it on my library

The lib is organised in subdirectories, based on the author's name initial:
\A
\B
\C
etc...

I've been fiddling a bit with Calibre, doing tags updates etc... So far so good

I have also done some activity at file name level: adding/deleting a few books to/from the directories, correcting a few typos on some titles and author's names...

Now, I would like to refresh the whole stuff, while keeping the updates done in Calibre (tags etc) and the ones done at file level
I'm a bit worried about using the "Add books" functionality as I don't want duplicates by adding everything again

Question: what is the proper way to do an effective "refresh", which will take every add/update/delete into consideration, at metadata level and at file name level?

Thanks a bunch :=)

[stumped]

Next time read the instuctions before meddling.
You are probably screwed but someone may help you salvage the mess

[Sirtel]

Calibre doesn't recognize any changes done with the files manually. Your library is messed up now, as the Calibre database and the actual files are different.

All and any changes in a Calibre library must be made using the Calibre interface, not the OS.

[ilovejedd]

Any changes made at file level are effectively lost but you can still salvage the calibre library. First and foremost, create a back-up of metadata.db and keep it somewhere safe. I'd do multiple backups just to be extra safe. I know I've overwritten my backup with a bad/corrupt copy on occasion.

Then, assuming your books are in the following:

\Calibre Library\A\Author A\Title (id)\Title - Author A.ext
\Calibre Library\Z\Author Z\Title (id)\Title - Author Z.ext

Move the authors back to the top level so you have:

\Calibre Library\Author A\Title (id)\Title - Author A.ext
\Calibre Library\Author Z\Title (id)\Title - Author Z.ext

Then run Calibre and do:
Library maintenance -> Check library -> Do you want calibre to check if the files in your library match the information in the database? Yes

Don't run any of the fixes. Instead expand the results so you can rename the files and folders you changed back to their original names in Windows Explorer or Finder (or whatever you use as file manager).

If you've made more massive changes to the directory and file structure than the above such as removing (id) you're better off copying the library to a different location and re-importing with merge duplicates.

[Patrick Derwael]

Ok guys, thank you for the slap in the face

Actually, at this very minute, I don't really care about loosing the DB as I did not put the full collection under Calibre management

Having said this, whenever I notice a typo in the file name, I must corrected. Fullstop
I've been again and again searched the various menus, but did not find how to rename a book (read file name) from within Calibre
Any pointer would be MUCH appreciated

Second question: with time, I will be adding new FILES to the library directory tree and I will be wanting to do a full refresh
Again, how do I do this without loosing the metadata updates already done and without duplicating everything, "as per the rules"

Thanks!

[davidfor]

Originally Posted by Patrick Derwael

Ok guys, thank you for the slap in the face

Actually, at this very minute, I don't really care about loosing the DB as I did not put the full collection under Calibre management

Having said this, whenever I notice a typo in the file name, I must corrected. Fullstop
I've been again and again searched the various menus, but did not find how to rename a book (read file name) from within Calibre
Any pointer would be MUCH appreciated

It is automatic. Change the author or title in calibre, and the file is renamed.

Second question: with time, I will be adding new FILES to the library directory tree and I will be wanting to do a full refresh
Again, how do I do this without loosing the metadata updates already done and without duplicating everything, "as per the rules"

Add the books to calibre. If they are new books, use the Add button, or drag-and-drop them on the window. There is also a command line tool (calibredb I think) to do it. If the book is already in the library, I usually select the book in list and drop the new version or extra file in the details window.

[ilovejedd]

Note, Calibre requires a very specific directory structure which you are not supposed to change manually via file manager.

https://manual.calibre-ebook.com/faq.html#id35

Why doesn’t calibre let me store books in my own directory structure?

The whole point of calibre’s library management features is that they provide a search and sort based interface for locating books that is much more efficient than any possible directory scheme you could come up with for your collection. Indeed, once you become comfortable using calibre’s interface to find, sort and browse your collection, you wont ever feel the need to hunt through the files on your disk to find a book again. By managing books in its own directory structure of Author -> Title -> Book files, calibre is able to achieve a high level of reliability and standardization. To illustrate why a search/tagging based interface is superior to folders, consider the following. Suppose your book collection is nicely sorted into folders with the following scheme:

Genre -> Author -> Series -> ReadStatus

Now this makes it very easy to find for example all science fiction books by Isaac Asimov in the Foundation series. But suppose you want to find all unread science fiction books. There’s no easy way to do this with this folder scheme, you would instead need a folder scheme that looks like:

ReadStatus -> Genre -> Author -> Series

In calibre, you would instead use tags to mark genre and read status and then just use a simple search query like tag:scifi and not tag:read. calibre even has a nice graphical interface, so you don’t need to learn its search language instead you can just click on tags to include or exclude them from the search.

To those of you that claim that you need access to the filesystem, so that you can have access to your books over the network, calibre has an excellent Content server that gives you access to your calibre library over the net.

If you are worried that someday calibre will cease to be developed, leaving all your books marooned in its folder structure, explore the powerful Save to Disk feature in calibre that lets you export all your files into a folder structure of arbitrary complexity based on their metadata.

Finally, the reason there are numbers at the end of every title folder, is for robustness. That number is the id number of the book record in the calibre database. The presence of the number allows you to have multiple records with the same title and author names. It is also part of what allows calibre to magically regenerate the database with all metadata if the database file gets corrupted. Given that calibre’s mission is to get you to stop storing metadata in filenames and stop using the filesystem to find things, the increased robustness afforded by the id numbers is well worth the uglier folder names.

If you are still not convinced, then I’m afraid calibre is not for you. Look elsewhere for your book cataloguing needs. Just so we’re clear, this is not going to change. Kindly do not contact us in an attempt to get us to change this.

[Sirtel]

Originally Posted by Patrick Derwael

Second question: with time, I will be adding new FILES to the library directory tree and I will be wanting to do a full refresh
Again, how do I do this without loosing the metadata updates already done and without duplicating everything, "as per the rules"

Thanks!

That's the one thing you absolutely MUST NOT do. Add the books via the Calibre interface, as explained in a previous post. If you don't want duplicates cluttering your PC, then you either have to delete the original files or store them somewhere else (for example, an external drive) for back-up.

[theducks]

Originally Posted by Sirtel

That's the one thing you absolutely MUST NOT do. Add the books via the Calibre interface, as explained in a previous post. If you don't want duplicates cluttering your PC, then you either have to delete the original files or store them somewhere else (for example, an external drive) for back-up.

Set up an Auto Add folder (preferences:adding books), Drop the book there: Once the ADD has been accepted, the book is deleted fom the Auto Add folder

Doing it this way *assumes* nothing goes wrong
IMHO Archiving the original ON A DIFFERENT STORAGE DEVICE is a

[BetterRed]

FTR : on Windows, if you you put a shortcut to calibre.exe in the SendTo folder you can then use the Send To option in the File (Windows) Explorer context menu to add books to the current library - calibre will start if its not already running.

To open the SendTo folder press Winkey+R then type shell:sendto

BR

[Patrick Derwael]

Ok, I now understand my confusion about directory structures

Other library managers I use for media (music, movies & pictures) are only storing metadata and pointers to effective data in the library database
Calibre is storing the data along with the metadata; and this changes the whole picture!!!

When I said I was modifying file nd directories names with the file manager, I meant the source directories where my ebooks were stored before being imported in Calibre
Obviously, I would never mess around in the "Calibre Library" sudirectories tree!

Thanks a bunch to TheDucks whose last message made me get it!

Originally Posted by theducks

Once the ADD has been accepted, the book is deleted fom the Auto Add folder

As far as I'm concerned: issue closed!