Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 01-24-2012, 03:28 AM   #1
pnhearer
Junior Member
pnhearer began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Jan 2012
Device: IPad
Book Database update issue

So, I have alot of books of various formats and i only want them in epub format. no problem Calibre converts them ez as pie. A problem however occurs when I try and delete the files manually from the files aren't updated and removed in the calibre database.
I'll give an example

1.)I have lets say 100 file(s).pdf
2.)I convert file.pdf to Epub
3.)I use a dos prompt and a little dos magic and I delete all of the .pdf files that reside in this directory.
4.) Now my calibre says I have all these pdf's I don't actually have, but I go ahead and add the new epub versions of the books anyway and now the library states that I have epub and pdf formats of these files, which is inaccutate and annoying when you are trying to convert everything to epub.

Is there anyway to mitigate this problem? My temp work around is to go into the Calibre Library folder and delete the metadata db file and readd all of my books which takes hours... on quad core system with 8gb of ram.

I want a simple way of Converting all these files, deleting the original, have calibre reflect the changes I have made without having to trash the entire metadata file. The only thing I can figure to maybe try is to go into the metadata file and manuall delete all .pdf entries but looking at that file it would be a nightmare......
pnhearer is offline   Reply With Quote
Old 01-24-2012, 03:48 AM   #2
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,030
Karma: 777817
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Why do this at the DOS level? Why not use the Calibre facility for removing additional formats of a book?
itimpi is offline   Reply With Quote
 
Enthusiast
Old 01-24-2012, 09:10 AM   #3
Synamon
Lunatic
Synamon ought to be getting tired of karma fortunes by now.Synamon ought to be getting tired of karma fortunes by now.Synamon ought to be getting tired of karma fortunes by now.Synamon ought to be getting tired of karma fortunes by now.Synamon ought to be getting tired of karma fortunes by now.Synamon ought to be getting tired of karma fortunes by now.Synamon ought to be getting tired of karma fortunes by now.Synamon ought to be getting tired of karma fortunes by now.Synamon ought to be getting tired of karma fortunes by now.Synamon ought to be getting tired of karma fortunes by now.Synamon ought to be getting tired of karma fortunes by now.
 
Synamon's Avatar
 
Posts: 1,559
Karma: 3861560
Join Date: Dec 2011
Location: Land of the Loonie
Device: Kindle Paperwhite and Keyboard, Kobo Aura, iPad mini, iPod Touch
Quote:
Originally Posted by itimpi View Post
Why do this at the DOS level? Why not use the Calibre facility for removing additional formats of a book?
Specifically, the "remove all formats from selected books, except..." option in the Remove books dropdown menu.
Synamon is offline   Reply With Quote
Old 01-24-2012, 10:09 AM   #4
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,274
Karma: 5495472
Join Date: Aug 2009
Location: The (original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
You went under the skirts and made changes
Calibre does not know about your changes.

You have damaged your Database (files are part of the DB, even tough they are in the file system.) Stay OUT (unless you absolutely know your activity is safe. Rule 1:Change only the Insides of the file. There is no Rule 2. )

Calibre can delete Single formats (at least 8.37 will do it correctly, again)
theducks is offline   Reply With Quote
Old 01-24-2012, 11:54 AM   #5
transmitthis
Addict
transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.
 
transmitthis's Avatar
 
Posts: 285
Karma: 1003542
Join Date: May 2011
Device: Google Nexus 7 16GB
What they said.
Also, dos magic? -
Win7 search would do that kind of thing in 2 shakes of a lambs tail
Select folder, type *.pdf in search, select pdfs and delete.

But as said, don't do it via windows.
transmitthis is offline   Reply With Quote
Old 01-24-2012, 12:09 PM   #6
pnhearer
Junior Member
pnhearer began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Jan 2012
Device: IPad
v.v...I completely misunderstood/misread/overlooked that delete option...would have made my life so much easier..sigh.

Anyway, Windows search ...pfffffffft.

cmd prompt

DEL /S C:\Users\Username\Calibre Library\*.pdf

....I think mine works better.




By the way.... anyone having problems with the program hanging and taking its sweet ass time during converts/deletes? I'm sitting on a quad code with 8gb of ram, I cant really fathom why it is hanging so much and taking so long....

Last edited by pnhearer; 01-24-2012 at 12:13 PM.
pnhearer is offline   Reply With Quote
Old 01-24-2012, 12:43 PM   #7
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,274
Karma: 5495472
Join Date: Aug 2009
Location: The (original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by pnhearer View Post
v.v...I completely misunderstood/misread/overlooked that delete option...would have made my life so much easier..sigh.

Anyway, Windows search ...pfffffffft.

cmd prompt

DEL /S C:\Users\Username\Calibre Library\*.pdf

....I think mine works better.




By the way.... anyone having problems with the program hanging and taking its sweet ass time during converts/deletes? I'm sitting on a quad code with 8gb of ram, I cant really fathom why it is hanging so much and taking so long....
I suspect that it is DB housekeeping. Every single title, das entries for eash format. These need to be kept up-to-date (removed in this case).
I/O, not cores in this case
theducks is offline   Reply With Quote
Old 01-24-2012, 01:44 PM   #8
pnhearer
Junior Member
pnhearer began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Jan 2012
Device: IPad
Quote:
Originally Posted by theducks View Post
I suspect that it is DB housekeeping. Every single title, das entries for eash format. These need to be kept up-to-date (removed in this case).
I/O, not cores in this case
Actually that would make sense, the Read/Write overhead would be the issue. I wonder if it would make more sense to load the entire db into memory, make changes there and then write it all at one time to the disk....aka overwrite the old file

Last edited by pnhearer; 01-24-2012 at 01:47 PM.
pnhearer is offline   Reply With Quote
Old 01-24-2012, 03:22 PM   #9
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,274
Karma: 5495472
Join Date: Aug 2009
Location: The (original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by pnhearer View Post
Actually that would make sense, the Read/Write overhead would be the issue. I wonder if it would make more sense to load the entire db into memory, make changes there and then write it all at one time to the disk....aka overwrite the old file
One word: Fragile.
theducks is offline   Reply With Quote
Old 01-24-2012, 05:32 PM   #10
pnhearer
Junior Member
pnhearer began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Jan 2012
Device: IPad
Fragile? Hardly. ram and flash memory are volitile memory yes but they would store just fine. I think I'll propose this to the author of the program maybe...
pnhearer is offline   Reply With Quote
Old 01-25-2012, 10:55 AM   #11
transmitthis
Addict
transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.
 
transmitthis's Avatar
 
Posts: 285
Karma: 1003542
Join Date: May 2011
Device: Google Nexus 7 16GB
I did a test on something simmiler
used calibre to copy a library, ie select all books and copy to another library:
Windows copy via explorer
Calibre - over 3hours - went to sleep, could have been 11h
Windows 14min
transmitthis is offline   Reply With Quote
Old 01-25-2012, 11:54 AM   #12
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,030
Karma: 777817
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
I would expect major differences.

When you copy via Explorer then you are simply copying the files. The actual contents of files are not bheing examined in any way. if you only want to make a backup then this is the way to go.

When you copy via Calibre as well as copying the files related to each book, it is also doing the required inserts into the target metadata database for each book and carrying out checks for duplicates etc and this is a lot of additional work. You can use this feature to guarantee that the target library is valid. The same code is also used to copy books to an existing library where merging needs to rake place.
itimpi is offline   Reply With Quote
Old 01-25-2012, 03:02 PM   #13
transmitthis
Addict
transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.
 
transmitthis's Avatar
 
Posts: 285
Karma: 1003542
Join Date: May 2011
Device: Google Nexus 7 16GB
The copy was to a blank empty library, it should have not included any duplicate checking, the target database would also be empty, all metadata is already included in the folders.
Not saying its good or bad, just something I noticed.
Result being if you want to copy a library - don't do it from within calibre
transmitthis is offline   Reply With Quote
Old 01-25-2012, 06:56 PM   #14
pnhearer
Junior Member
pnhearer began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Jan 2012
Device: IPad
Honestly, no matter how you slice it read/write to memory is exponentially faster than writing to a SATA drive, and light years faster than to a pata. I'm running RAID 0 for cryin out load and the converting 226 of files took well over 24 hrs
pnhearer is offline   Reply With Quote
Old 01-25-2012, 10:07 PM   #15
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 2,309
Karma: 5761596
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PRS-350, Nexus S, Galaxy S, Nook Color, iPhone4, iPT4, iPad 2012
Quote:
Originally Posted by pnhearer View Post
Honestly, no matter how you slice it read/write to memory is exponentially faster than writing to a SATA drive, and light years faster than to a pata. I'm running RAID 0 for cryin out load and the converting 226 of files took well over 24 hrs
I reckon yours is more CPU bound than disk bound. While it's true that using an SSD can significantly cut down conversion time, 226 files in 24 hours suggest it's the CPU that's getting stressed.
ilovejedd is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Database update errors nynaevelan Calibre 5 06-09-2011 12:23 PM
desktop cannot update database after 1.8.1 upgrade michael_h Kobo Reader 4 12-21-2010 10:01 AM
unable to update your devices database.syncing cannot be completed? baronrus Kobo Reader 5 12-20-2010 10:25 AM
We are unable to update your device's database. Syncing cannot be completed. talios Kobo Reader 1 12-10-2010 03:16 AM
Update Metadata in database from Plugin DokaMax Plugins 0 05-22-2010 05:58 AM


All times are GMT -4. The time now is 07:40 AM.


MobileRead.com is a privately owned, operated and funded community.