Book Database update issue

[pnhearer]

So, I have alot of books of various formats and i only want them in epub format. no problem Calibre converts them ez as pie. A problem however occurs when I try and delete the files manually from the files aren't updated and removed in the calibre database.
I'll give an example

1.)I have lets say 100 file(s).pdf
2.)I convert file.pdf to Epub
3.)I use a dos prompt and a little dos magic and I delete all of the .pdf files that reside in this directory.
4.) Now my calibre says I have all these pdf's I don't actually have, but I go ahead and add the new epub versions of the books anyway and now the library states that I have epub and pdf formats of these files, which is inaccutate and annoying when you are trying to convert everything to epub.

Is there anyway to mitigate this problem? My temp work around is to go into the Calibre Library folder and delete the metadata db file and readd all of my books which takes hours... on quad core system with 8gb of ram.

I want a simple way of Converting all these files, deleting the original, have calibre reflect the changes I have made without having to trash the entire metadata file. The only thing I can figure to maybe try is to go into the metadata file and manuall delete all .pdf entries but looking at that file it would be a nightmare......

[itimpi]

Why do this at the DOS level? Why not use the Calibre facility for removing additional formats of a book?

[Synamon]

Originally Posted by itimpi

Why do this at the DOS level? Why not use the Calibre facility for removing additional formats of a book?

Specifically, the "remove all formats from selected books, except..." option in the Remove books dropdown menu.

[theducks]

You went under the skirts and made changes
Calibre does not know about your changes.

You have damaged your Database (files are part of the DB, even tough they are in the file system.) Stay OUT (unless you absolutely know your activity is safe. Rule 1:Change only the Insides of the file. There is no Rule 2. )

Calibre can delete Single formats (at least 8.37 will do it correctly, again)

[transmitthis]

What they said.
Also, dos magic? -
Win7 search would do that kind of thing in 2 shakes of a lambs tail
Select folder, type *.pdf in search, select pdfs and delete.

But as said, don't do it via windows.

[pnhearer]

v.v...I completely misunderstood/misread/overlooked that delete option...would have made my life so much easier..sigh.

Anyway, Windows search ...pfffffffft.

cmd prompt

DEL /S C:\Users\Username\Calibre Library\*.pdf

....I think mine works better.




By the way.... anyone having problems with the program hanging and taking its sweet ass time during converts/deletes? I'm sitting on a quad code with 8gb of ram, I cant really fathom why it is hanging so much and taking so long....

[theducks]

Originally Posted by pnhearer

v.v...I completely misunderstood/misread/overlooked that delete option...would have made my life so much easier..sigh.

Anyway, Windows search ...pfffffffft.

cmd prompt

DEL /S C:\Users\Username\Calibre Library\*.pdf

....I think mine works better.




By the way.... anyone having problems with the program hanging and taking its sweet ass time during converts/deletes? I'm sitting on a quad code with 8gb of ram, I cant really fathom why it is hanging so much and taking so long....

I suspect that it is DB housekeeping. Every single title, das entries for eash format. These need to be kept up-to-date (removed in this case).
I/O, not cores in this case

[pnhearer]

Originally Posted by theducks

I suspect that it is DB housekeeping. Every single title, das entries for eash format. These need to be kept up-to-date (removed in this case).
I/O, not cores in this case

Actually that would make sense, the Read/Write overhead would be the issue. I wonder if it would make more sense to load the entire db into memory, make changes there and then write it all at one time to the disk....aka overwrite the old file

[theducks]

Originally Posted by pnhearer

Actually that would make sense, the Read/Write overhead would be the issue. I wonder if it would make more sense to load the entire db into memory, make changes there and then write it all at one time to the disk....aka overwrite the old file

One word: Fragile.

[pnhearer]

Fragile? Hardly. ram and flash memory are volitile memory yes but they would store just fine. I think I'll propose this to the author of the program maybe...

[transmitthis]

I did a test on something simmiler
used calibre to copy a library, ie select all books and copy to another library:
Windows copy via explorer
Calibre - over 3hours - went to sleep, could have been 11h
Windows 14min

[itimpi]

I would expect major differences.

When you copy via Explorer then you are simply copying the files. The actual contents of files are not bheing examined in any way. if you only want to make a backup then this is the way to go.

When you copy via Calibre as well as copying the files related to each book, it is also doing the required inserts into the target metadata database for each book and carrying out checks for duplicates etc and this is a lot of additional work. You can use this feature to guarantee that the target library is valid. The same code is also used to copy books to an existing library where merging needs to rake place.

[transmitthis]

The copy was to a blank empty library, it should have not included any duplicate checking, the target database would also be empty, all metadata is already included in the folders.
Not saying its good or bad, just something I noticed.
Result being if you want to copy a library - don't do it from within calibre

[pnhearer]

Honestly, no matter how you slice it read/write to memory is exponentially faster than writing to a SATA drive, and light years faster than to a pata. I'm running RAID 0 for cryin out load and the converting 226 of files took well over 24 hrs

[ilovejedd]

Originally Posted by pnhearer

Honestly, no matter how you slice it read/write to memory is exponentially faster than writing to a SATA drive, and light years faster than to a pata. I'm running RAID 0 for cryin out load and the converting 226 of files took well over 24 hrs

I reckon yours is more CPU bound than disk bound. While it's true that using an SSD can significantly cut down conversion time, 226 files in 24 hours suggest it's the CPU that's getting stressed.