Calibre emptying pdf files

[kidpixo]

First of all: I didn't find a Calibre bug/problem dedicated category, so I´m posting here and yes, I searched!

Hi everybody,
I´m adding a huge PDF library of scientific books to calibre in order to attach metadata and to organise the Jungle on my machine.

I´m running MacOsx 10.6.6 and Calibre.

Since a lot of the file are poorly optimize-scan, after addition to calibre I started to optimize some of that.
I'm running AdobePro9 scanned file optimization, OCR and files "size decrease" (Adobe magic stuff).

By the way, I found out that the iPad iOs 4.2.1 could visualize properly PDF until version 1.4 (Adobe v.5). After that the images are not displayed, irrespective to wich program I using to visualize it (I tested: iBook, Safari integrated pdf engine, Goodreader, iAnnotate).

After that process the file size is always different, but calibre does´n not refresh the metadata, so I try to cancel the format from the metadata (pointing to the actual file that is the new, small one) and to add it again.

After that it results in a Zero bytes file! I don't know where is the problem.

It happens exactly after clicking OK in the 'Edit Metadata Information' window.

I also try with other format (djvu), but the result is the same.

I guess that Calibre isn't happy to find a record in his db with the same name of an old, differing from the last saved file.

NOTE: If the file reside somewhere else on my machine, it's fine. Calibre creates a copy of that file in the right folder and Amen.

[itimpi]

If you are trying to replace a PDF file that is already under Calibre control then you can do one of two things:

• Replace directly the file that is already in the Calibre library taking care that you do not change the filename. This will work although it is not the recommended method.
• In the Edit Metadata dialog add a file that is NOT under Calibre control using either drag-and-drop to the Formats window or via one of the Add buttons. This is the recommended way to do things.

What you must NOT do (and I think you are trying to do) is to try and combine these methods and use the second of the above options to try and add again the file that is already under Calibre control. This will go wrong because Calibre always takes a copy when adding a file and this results in calibre trying to copy the file to itself.

[kidpixo]

thanks for the answer! As you said, the method one or two are working fine, I feel just stupid

Originally Posted by itimpi

If you are trying to replace a PDF file that is already under Calibre control then you can do one of two things:

• Replace directly the file that is already in the Calibre library taking care that you do not change the filename. This will work although it is not the recommended method.

that´s what I did, and it worked fine until I tried to fix the file size.

Originally Posted by itimpi

• In the Edit Metadata dialog add a file that is NOT under Calibre control using either drag-and-drop to the Formats window or via one of the Add buttons. This is the recommended way to do things.

What you must NOT do (and I think you are trying to do) is to try and combine these methods and use the second of the above options to try and add again the file that is already under Calibre control. This will go wrong because Calibre always takes a copy when adding a file and this results in calibre trying to copy the file to itself.

I agree, but why at the end it results in a zero-length file?
It should keep the old version (size X,let´s say) or the new one (X-1 size).

I assume that calibre keeps the new file (wherever it is), then erases the old one and makes a copy of the new new one in the position once occupied by the old file.

Add to this picture that I overwrite the new on the old file and you see the problem.

That sounds like a program bug to me, even if my action was not standard: shouldn't be here a sort of idiot-protection?

bye!

[itimpi]

Originally Posted by kidpixo

Add to this picture that I overwrite the new on the old file and you see the problem.

That sounds like a program bug to me, even if my action was not standard: shouldn't be here a sort of idiot-protection?

whether it should be considered a bug or not is an interesting point as I suspect that the developers probably never though of that as something anyone would try and do. It is probably an enhancement request to add such a check - but it IS a good idea to have it as anything that can lead to data loss is highly undesireable.

[kidpixo]

Originally Posted by itimpi

whether it should be considered a bug or not is an interesting point as I suspect that the developers probably never though of that as something anyone would try and do. It is probably an enhancement request to add such a check - but it IS a good idea to have it as anything that can lead to data loss is highly undesireable.

Indeed!
I'm wondering open a ticket about it for Calibre..

[DoctorOhh]

So you did a drag-n-drop from the calibre library folder to the available formats window in the edit metadata area. This is essentially a "move" and after a file is successfully copied/moved to the new location (same location in your example) the original would be deleted. This causes a 0.0 byte file.

You can't do this via two different file managers in Windows because it recognizes they are the same and throws up a error box and an annoying sound to let you know this isn't right.

It seems calibre (python) isn't so forgiving. I tried this myself and did end up with a 0.0 byte file that was garbage.

Read here to find out how to create a ticket. But like the old joke, Patient: Doctor, Doctor it hurts when I do this.... Doctor: Well don't do that.

Congrats on being unique enough to find this "Opening of destruction".

[theducks]

Originally Posted by dwanthny

Congrats on being unique enough to fine this "Opening of destruction".

(We really need a 'foot-shot' emoticon )

[kovidgoyal]

Should be fixed in next release. calibre will now simply ignore attempts to overwrite a file with itself.

[kidpixo]

Originally Posted by dwanthny

So you did a drag-n-drop from the calibre library folder to the available formats window in the edit metadata area. This is essentially a "move" and after a file is successfully copied/moved to the new location (same location in your example) the original would be deleted. This causes a 0.0 byte file.

Yes, but the whole story is:

• I erased the specific file format from Calibre (namely PDF here)
• I did a drag-n-drop in the same position of the old file
• I added a new file format for this entry, that by chance is a file in the same position and w. the same name

Originally Posted by dwanthny

You can't do this via two different file managers in Windows because it recognizes they are the same and throws up a error box and an annoying sound to let you know this isn't right.
It seems calibre (python) isn't so forgiving. I tried this myself and did end up with a 0.0 byte file that was garbage.

It's right, but I was assuming that once the file format was erased from Calibre DB it should release the manage of this specific file. It turned out to be wrong...

Originally Posted by dwanthny

Read here to find out how to create a ticket. But like the old joke, Patient: Doctor, Doctor it hurts when I do this.... Doctor: Well don't do that.

But I like to do that!

Originally Posted by dwanthny

Congrats on being unique enough to find this "Opening of destruction".

Thanks

Originally Posted by kovidgoyal

Should be fixed in next release. calibre will now simply ignore attempts to overwrite a file with itself.

Great!And that means that I don't need to annoy you with a new bug ticket!

Bye!

--
"The Answer to the Great Question …"
"Of Life, the Universe and Everything ..." said Deep Thought.
"Is ... " said Deep Thought, and paused.
"Forty-two," said Deep Thought, with infinite majesty and calm.

[Manichean]

Originally Posted by kidpixo

Yes, but the whole story is:

• I erased the specific file format from Calibre (namely PDF here)
• I did a drag-n-drop in the same position of the old file
• I added a new file format for this entry, that by chance is a file in the same position and w. the same name

That's what dwanthny was saying. It doesn't matter whether or not you delete the format from the Calibre database before. As soon as you start mucking about inside Calibre's library folder tree, Bad Things are going to happen to you sooner or later, as you have seen. It's best just to leave the library folder alone and use external folders as import sources.

[kidpixo]

Originally Posted by Manichean

That's what dwanthny was saying. It doesn't matter whether or not you delete the format from the Calibre database before. As soon as you start mucking about inside Calibre's library folder tree, Bad Things are going to happen to you sooner or later, as you have seen. It's best just to leave the library folder alone and use external folders as import sources.

I'm not sure about this point and I have to disagree.
Calibre is just pointing to some file stored in some folder. If I add new thing there, it should not destabilize Calibre file management structure, because it is not scanning the folder tree to find new file (in this aspect is really close to iTunes).
That's why I mentioned that I erased that specific file from inside Calibre: then it should be possible to add a new file without problem.

Cheers.

--
"The Answer to the Great Question …"
"Of Life, the Universe and Everything ..." said Deep Thought.
"Is ... " said Deep Thought, and paused.
"Forty-two," said Deep Thought, with infinite majesty and calm.

[DoctorOhh]

Originally Posted by kidpixo

Yes, but the whole story is:

• I erased the specific file format from Calibre (namely PDF here)
• I did a drag-n-drop in the same position of the old file
• I added a new file format for this entry, that by chance is a file in the same position and w. the same name

If you just did the first two steps you would be fine. The file would be updated and everything would work as expected. I essentially do this every time I edit a ePub with Sigil and save over an old version.

The last step, which was not needed and caused your problem, is what I described in post 6.

Originally Posted by kidpixo

That's why I mentioned that I erased that specific file from inside Calibre: then it should be possible to add a new file without problem.

You can always (even now) add in the manner you were using except in your third step you were adding from the calibre directory to the calibre directory. There was no need for this step, so no one else had ever stumbled across this error. After the fix feel free to do step three, it still won't be a needed step, but it won't hurt anything because it will essentially be ignored.

[kidpixo]

Originally Posted by dwanthny

If you just did the first two steps you would be fine. The file would be updated and everything would work as expected. I essentially do this every time I edit a ePub with Sigil and save over an old version.

The last step, which was not needed and caused your problem, is what I described in post 6.

If I just did the first two steps I had just an empty entry in Calibre, that's non fine.
Note that in step 2 I meant "I moved the pdf file from /somewhere/ to the folder where Calibre stored the old file in the file system".

I agree that there is a safer solution, but my problem was that I updated the pdf file and Calibre didn't recognize the change (namely, the file size was untouched).

That's why I simply though to erase and re-add it to the library, assuming that the metadata are extracted once when the file were added to the database.

I never suspected that add a file format would overwrite the old one: I'm maybe ultra-safe, but I'd create a secondary copy of it.

That said, I get it, thanks!

bye!


--
"The Answer to the Great Question …"
"Of Life, the Universe and Everything ..." said Deep Thought.
"Is ... " said Deep Thought, and paused.
"Forty-two," said Deep Thought, with infinite majesty and calm.

[DoctorOhh]

Originally Posted by kidpixo

That said, I get it, thanks!

I'm glad you get it, because your explanations confuse me no end.

I just know that I was able to recreate your results in a repeatable way that will be corrected in the next version.

Good Luck.

[kidpixo]

Originally Posted by dwanthny

I'm glad you get it, because your explanations confuse me no end.

I'm glad to help

Originally Posted by dwanthny

I just know that I was able to recreate your results in a repeatable way that will be corrected in the next version.

Good.

Originally Posted by dwanthny

Good Luck.

You too!