Searching for 'no' formats

ldolse · 09-17-2010, 11:26 AM

I guess the bug where Calibre occasionally decides to send your books to /dev/null is still around. Just found a book I was recently working with gone, probably because of some metadata change, but not sure, haven't touched it for a week or so. The book entry is there in the Calibre database, but the books are gone.

I have a database integrity check running, which I think is the only workaround for this, but I wanted to see what other books were affected. Isn't there some search you can do to look for book records with no books? I thought it was format"=NONE" or something along those lines, but no luck. (DB integrity check says two books are affected, but would like to be able to do the search)

Is there any way to make Calibre more bulletproof in this regard, maybe create new directories and copy files over, and then kill the old directory only after the transfer is verified?

kovidgoyal · 09-17-2010, 11:28 AM

format:false

and calibre does crete a anew directory and copy files over and only then delete the old directory

ldolse · 09-17-2010, 11:36 AM

Thanks - well, I'll try to keep an eye out for it happening. Interestingly enough I have two books missing. The only thing they have in common is that there is a : in the title. Colon isn't supported in hfsplus filenames (not sure about other filesystems).

Calibre is using an underscore instead of a colon when I manually rename books using a colon, but I wonder if there is some specific code path where it forgets to escape it? I'll see if I can retrace my steps.

kovidgoyal · 09-17-2010, 11:38 AM

While it's possible, I doubt it. All file renaming/moving happens ina single function in database2.py called set_path

theducks · 09-17-2010, 11:43 AM

Quote:

Originally Posted by ldolse

Thanks - well, I'll try to keep an eye out for it happening. Interestingly enough I have two books missing. The only thing they have in common is that there is a : in the title. Colon isn't supported in hfsplus filenames (not sure about other filesystems).

Calibre is using an underscore instead of a colon when I manually rename books using a colon, but I wonder if there is some specific code path where it forgets to escape it? I'll see if I can retrace my steps.

Manually? Like, where??

BTW Colons are not allowed in Windows file names, That is what makes it a "Device Name" (LPT1: COM1: NUL: )

ldolse · 09-17-2010, 11:48 AM

I took a book that didn't have a colon in the name and manually added it - using Calibre. Don't worry, I'm not trying to trip Calibre up by managing files myself

theducks · 09-17-2010, 03:44 PM

Quote:

Originally Posted by ldolse

I took a book that didn't have a colon in the name and manually added it - using Calibre. Don't worry, I'm not trying to trip Calibre up by managing files myself

Calibre should have made sure the filename was valid by substituting a char for the invalid one. AFAIK the title always remains as typed in the booklist.

"Genellan: Planetfall" is my example. The colon was replaced with an underscore in the filename.

ldolse · 09-17-2010, 09:27 PM

It did replace the character in my testing, I'm not saying it didn't. The character just jumped out as something in common for both affected files. But it did do 'something', as the books have vanished and I don't see any way it could have been user error.

Anyway I need to dig the missing book off of my iPhone and see if I can reproduce the steps. The original title didn't have a colon in it from the mobi metadata, I think I pulled that down when I downloaded more metadata from the web. Maybe it was something else in the data downloaded from the web that caused Calibre to choke.

09-17-2010, 11:26 AM	#1
ldolse Wizard Posts: 1,337 Karma: 123455 Join Date: Apr 2009 Location: Malaysia Device: PRS-650, iPhone	Searching for 'no' formats I guess the bug where Calibre occasionally decides to send your books to /dev/null is still around. Just found a book I was recently working with gone, probably because of some metadata change, but not sure, haven't touched it for a week or so. The book entry is there in the Calibre database, but the books are gone. I have a database integrity check running, which I think is the only workaround for this, but I wanted to see what other books were affected. Isn't there some search you can do to look for book records with no books? I thought it was format"=NONE" or something along those lines, but no luck. (DB integrity check says two books are affected, but would like to be able to do the search) Is there any way to make Calibre more bulletproof in this regard, maybe create new directories and copy files over, and then kill the old directory only after the transfer is verified?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Convert Epub and Msreader formats to Kindle formats	bruc79	Calibre	17	06-22-2010 04:50 AM
Searching for an old movie	LucasCorso	Lounge	2	04-09-2010 05:58 AM
Unutterably Silly Collect more 'no' than 'yes' votes	ProDigit	Lounge	52	05-02-2009 02:13 PM
How Robust is Searching?	ewayde	iRex	17	11-02-2007 03:33 AM

09-17-2010, 11:28 AM	#2
kovidgoyal creator of calibre Posts: 45,216 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	format:false and calibre does crete a anew directory and copy files over and only then delete the old directory

09-17-2010, 11:36 AM	#3
ldolse Wizard Posts: 1,337 Karma: 123455 Join Date: Apr 2009 Location: Malaysia Device: PRS-650, iPhone	Thanks - well, I'll try to keep an eye out for it happening. Interestingly enough I have two books missing. The only thing they have in common is that there is a : in the title. Colon isn't supported in hfsplus filenames (not sure about other filesystems). Calibre is using an underscore instead of a colon when I manually rename books using a colon, but I wonder if there is some specific code path where it forgets to escape it? I'll see if I can retrace my steps.

09-17-2010, 11:38 AM	#4
kovidgoyal creator of calibre Posts: 45,216 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	While it's possible, I doubt it. All file renaming/moving happens ina single function in database2.py called set_path

09-17-2010, 11:48 AM	#6
ldolse Wizard Posts: 1,337 Karma: 123455 Join Date: Apr 2009 Location: Malaysia Device: PRS-650, iPhone	I took a book that didn't have a colon in the name and manually added it - using Calibre. Don't worry, I'm not trying to trip Calibre up by managing files myself

09-17-2010, 09:27 PM	#8
ldolse Wizard Posts: 1,337 Karma: 123455 Join Date: Apr 2009 Location: Malaysia Device: PRS-650, iPhone	It did replace the character in my testing, I'm not saying it didn't. The character just jumped out as something in common for both affected files. But it did do 'something', as the books have vanished and I don't see any way it could have been user error. Anyway I need to dig the missing book off of my iPhone and see if I can reproduce the steps. The original title didn't have a colon in it from the mobi metadata, I think I pulled that down when I downloaded more metadata from the web. Maybe it was something else in the data downloaded from the web that caused Calibre to choke.

Advert

Advert