Losing books/formats is not normal.-Chaley

theducks · 09-30-2010, 08:50 AM

How about losing track of Formats?

I know better than to add/remove formats outside of the GUI.

I had a Surprise when I went to fix

an ePub with Sigil.
I clicked open in the book browser, that showed epub as my only format.
Surprise: there was the same book in PRC in the folder (many of my Baen books were converted from PRC, so PRC would not be an unusual format to see

).
The DB Integrity check reports no errors.

Am I correct, that the check only tests for "Inner (file name.type) Joins", when checking for files present?

I have zero Python-Fu, so writing a tool to test for "un-expected files" report, is not within my skill set. Using Eyeball Mk I method, is (mostly) out of the question with 1100 books

chaley · 09-30-2010, 09:19 AM

Quote:

Originally Posted by theducks

How about losing track of Formats?

You know the answer to that.

Quote:

I had a Surprise when I went to fix

an ePub with Sigil.
I clicked open in the book browser, that showed epub as my only format.
Surprise: there was the same book in PRC in the folder (many of my Baen books were converted from PRC, so PRC would not be an unusual format to see

).
The DB Integrity check reports no errors.

It is normal that the integrity check does not report on extra files. It doesn't know why they are there, but perhaps they should be. For example, calibre2opds puts (or at least used to put) thumbnails in there.

Do these extra formats have the correct name? Their name should be absolutely identical to the epub, except for the extension.

Quote:

Am I correct, that the check only tests for "Inner (file name.type) Joins", when checking for files present?

To use database-speak (even though the filesystem isn't a database), calibre would use a left-join and look for nulls where formats are supposed to be. It does not do a right-/full-outer-join, looking for nulls in its database.

Quote:

I have zero Python-Fu, so writing a tool to test for "un-expected files" report, is not within my skill set. Using Eyeball Mk I method, is (mostly) out of the question with 1100 books

Am I hearing a feature request?

theducks · 09-30-2010, 09:47 AM

I will have to wait and find another one. I removed the book

and added it back via the GUI.

I would expect that the tool would have an "Exclude or advisory" list.
So other standard filesystem (usually hidden) would not cause alarm.
Calibre optional files seems a candidate for the user maintained list.

Yes, a command line tool would be cool (enough. So as to not clutter the GUI)

chaley · 10-02-2010, 06:43 AM

Taaa Daaa (trumpets blow

)

Take a look under preferences -> advanced -> miscellaneous and push the button 'Check the library folders for potential problems'. The function reports. It doesn't change/fix anything.

The function is also available in the the command line under calibredb. Try

Code:

calibredb check_library --help

This function is (obviously) new. It could find problems that aren't problems, and it could miss problems. Experience will tell.

ldolse · 10-02-2010, 08:26 AM

I just tried the feature out. While it seems like it could be helpful, nothing I did in the 'ignore filename' or 'ignore file extension' boxes could get it to filter .DS_Store from the results. And seeing as OSX is such a big fan of that hidden file it winds up overwhelming the results...

chaley · 10-02-2010, 08:41 AM

Quote:

Originally Posted by ldolse

I just tried the feature out. While it seems like it could be helpful, nothing I did in the 'ignore filename' or 'ignore file extension' boxes could get it to filter .DS_Store from the results. And seeing as OSX is such a big fan of that hidden file it winds up overwhelming the results...

Fixed in the next release. For some reason I forced the ignore items to lower case. I was probably going to take the next step and make the filename compare case-insensitive, but I didn't, and in any event it would be wrong.

You could fix it by renaming all those pesky .DS_Store files to .ds_store. The mac file system is case-insensitive, isn't it?

ldolse · 10-02-2010, 09:01 AM

Cool, glad to hear it. I certainly don't have a problem renaming them, even if OSX got upset.. problem is it will just put them back

theducks · 10-02-2010, 11:11 AM

That was quick

I tried on my test DB (on Ubuntu Lucid) and it found a log file I had placed there a while back.

Now to try on my main DB (XP)

09-30-2010, 08:50 AM	#1
theducks Well trained by Cats Posts: 29,784 Karma: 54830978 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Losing books/formats is not normal.-Chaley How about losing track of Formats? I know better than to add/remove formats outside of the GUI. I had a Surprise when I went to fix an ePub with Sigil. I clicked open in the book browser, that showed epub as my only format. Surprise: there was the same book in PRC in the folder (many of my Baen books were converted from PRC, so PRC would not be an unusual format to see ). The DB Integrity check reports no errors. Am I correct, that the check only tests for "Inner (file name.type) Joins", when checking for files present? I have zero Python-Fu, so writing a tool to test for "un-expected files" report, is not within my skill set. Using Eyeball Mk I method, is (mostly) out of the question with 1100 books

10-02-2010, 06:43 AM	#4
chaley Grand Sorcerer Posts: 11,734 Karma: 6690881 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	Taaa Daaa (trumpets blow ) Take a look under preferences -> advanced -> miscellaneous and push the button 'Check the library folders for potential problems'. The function reports. It doesn't change/fix anything. The function is also available in the the command line under calibredb. Try Code: calibredb check_library --help This function is (obviously) new. It could find problems that aren't problems, and it could miss problems. Experience will tell.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Normal books and technical papers (PDFs with annotation support)	NautilusIII	Which one should I buy?	7	08-05-2010 04:42 AM
For Charles/Chaley Merging user defined metadata	Starson17	Calibre	20	07-22-2010 10:27 AM
Books losing their formatting...	Shiren	Sony Reader	38	05-24-2010 12:42 PM
Books with no Formats	Talung	Calibre	10	02-03-2009 08:01 PM
BOOKS in all formats	hesire	Deals and Resources (No Self-Promotion or Affiliate Links)	0	04-27-2003 12:10 PM

09-30-2010, 09:47 AM	#3
theducks Well trained by Cats Posts: 29,784 Karma: 54830978 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	I will have to wait and find another one. I removed the book and added it back via the GUI. I would expect that the tool would have an "Exclude or advisory" list. So other standard filesystem (usually hidden) would not cause alarm. Calibre optional files seems a candidate for the user maintained list. Yes, a command line tool would be cool (enough. So as to not clutter the GUI)

10-02-2010, 08:26 AM	#5
ldolse Wizard Posts: 1,337 Karma: 123455 Join Date: Apr 2009 Location: Malaysia Device: PRS-650, iPhone	I just tried the feature out. While it seems like it could be helpful, nothing I did in the 'ignore filename' or 'ignore file extension' boxes could get it to filter .DS_Store from the results. And seeing as OSX is such a big fan of that hidden file it winds up overwhelming the results...

10-02-2010, 09:01 AM	#7
ldolse Wizard Posts: 1,337 Karma: 123455 Join Date: Apr 2009 Location: Malaysia Device: PRS-650, iPhone	Cool, glad to hear it. I certainly don't have a problem renaming them, even if OSX got upset.. problem is it will just put them back

10-02-2010, 11:11 AM	#8
theducks Well trained by Cats Posts: 29,784 Karma: 54830978 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	That was quick I tried on my test DB (on Ubuntu Lucid) and it found a log file I had placed there a while back. Now to try on my main DB (XP)

Advert

Advert