![]() |
#1 |
Addict
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 265
Karma: 724240
Join Date: Aug 2013
Device: KyBook
|
Problem with Calibre Library
Since it seems to be going around I assume it might be easier to just tag it to this topic. I sadly have to get in line with the others as I just lost a library too...
That is, the booklist is empty after switching back from another lib. All physical folders and files are still present. Calibre also lists each and every author/title when I do a check library as "extra titles" Steps I did just prior, I was moving some books to another library(copy+delete) and received an error about a file not being accessible (windows error 5) so did an "open folder" on the book and found there was nothing the matter with the file that I could see, it was readable, had no specific security setting or encryption. Could move it and copy it myself without objections. So I assume it encountered a temporary deadlock situation where two or more processes/threads were trying to access the same file? So closed the folder window, switched to the library I was moving to so I could check what Calibre did move. Found the file there too (no duplicate, target library was freshly made before move) and switched back to source lib...empty list...AI! Now I am trying to get back from a Restore Library, it does find every book but it gives me this... Code:
Traceback (most recent call last): File "site-packages\calibre\db\restore.py", line 131, in load_preferences File "site-packages\calibre\db\restore.py", line 28, in __init__ File "site-packages\calibre\db\backend.py", line 321, in __init__ ValueError: Path to library too long. Must be less than 89 characters. and everything below that I did not touch. All Calibre's creation. So anything too long is not my doing. Meanwhile I cannot recover as I do not get a helpful hint which folder name or book title might then be too long. Neither does it offer to correct this and shorten the name itself. Trying to go in there manually and hoping I can find what it considers too long I don't dare as the credo goes "DONT TOUCH" so I won't. But also ain't too looking forward to go import each and every book by hand...adding them via Add Books is likely going to yield the same error and I got more formats per single book folder...and that format ain't addable in bulk. Last edited by At_Libitum; 10-19-2013 at 02:21 PM. |
![]() |
![]() |
![]() |
#2 | |
Well trained by Cats
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 30,950
Karma: 60358908
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
|
Quote:
are there: (assume you are viewing the Empty Library in Calibre: No device connected) Right click the Library Icon: Library Maintenance: Restore database Let it run |
|
![]() |
![]() |
Advert | |
|
![]() |
#3 | |
Addict
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 265
Karma: 724240
Join Date: Aug 2013
Device: KyBook
|
Thanks for the suggestion, but
Quote:
And The only thing that I could think of doing was to make a 1character long folder in the root of the drive. Move the content of the library folder in there. Mount it in Calibre and repeat the "Restore Library"... same result... Code:
calibre, version 1.7.0 ERROR: Failed: Restoring database failed, click Show details to see details Traceback (most recent call last): File "site-packages\calibre\db\restore.py", line 112, in run File "site-packages\calibre\db\restore.py", line 228, in create_cc_metadata File "site-packages\calibre\db\restore.py", line 28, in __init__ File "site-packages\calibre\db\backend.py", line 321, in __init__ ValueError: Path to library too long. Must be less than 89 characters. |
|
![]() |
![]() |
![]() |
#4 | |
Well trained by Cats
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 30,950
Karma: 60358908
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
|
Quote:
This is a Local DRIVE (and NOT shadowed to a backup service) This Drive has passed chkdisk? What was the last book added? verify that there is not a huge name there . Also verify there are NOT any Links inside the Library folders. |
|
![]() |
![]() |
![]() |
#5 |
Addict
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 265
Karma: 724240
Join Date: Aug 2013
Device: KyBook
|
Every file is still in place, it's a local drive, passes chkdsk and several other. No links or shortcuts in the library, no humongously long names (did an expand all on the top level folder.
The Y:\0 folder I created in the root I created by letting Calibre create an empty library, then I copied the contents of the Y:\Books\Transfer folder, so basically all of the subfolders but not the metadata.db and metadata_db_prefs_backup as these are likely corrupted anyways if the booklist is empty. All of the books still have their opf files present in their associated folders. Just for kicks I created again a 1 character long folder in the root. Copied in just one author folder with 1 book folder (and everything there, .opf, cover, epub) Andre Kyte - The Hidden Realm same result. same error about the path having to be < 89 All of the test library folders created in the process where created by creating a blank library copying the structure of an existing one (which passes all of the Calibre checks) Sorry, I ain't going to waste an evening hunting this down. There's clearly an obscure situation where during a batch move files do not get moved due to an external system error (windows error 5) but the booklist still got emptied as if they were moved. And Restore not working...I'll just lift out all of the books in a search and will dump them on a new empty lib, less aggravation from having to recreate missing metadata as from having to figure why a 1char long libfolder name can possibly be too long, and author/title folders? Well Calibre made these itself so if there's no bug in that part of the code, these could not be too long as this would have violated it's own length limitations. EDIT: And I also just did a check if there weren't any remnants from exploded epubs left behind from Tweak/Modify. None found so that ain't causing a length violation either Last edited by At_Libitum; 10-19-2013 at 03:07 PM. |
![]() |
![]() |
Advert | |
|
![]() |
#6 |
Wizard
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 4,812
Karma: 26912940
Join Date: Apr 2010
Device: sony PRS-T1 and T3, Kobo Mini and Aura HD, Tablet
|
It might be just as fast to create a new library and import the contents. Use the copy structure from current library and the second option in add books.
Helen |
![]() |
![]() |
![]() |
#7 | |
US Navy, Retired
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 9,895
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Kindle PaperWhite SE 11th Gen
|
Moderator Notice Moved this post to its own thread. It is not easier to jump on an existing thread when the two problems are entirely different. One is on OSX one is on Window. One person says his entire file structure is gone another acknowledges their file structure is intact. Quote:
When a library appears empty it is because the metadata.db file has been moved or is corrupt. |
|
![]() |
![]() |
![]() |
#8 | |
null operator (he/him)
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 21,662
Karma: 29711016
Join Date: Mar 2012
Location: Sydney Australia
Device: none
|
Quote:
![]() Its rarely best practice to jump onto an existing thread - unless its to post a ![]() As often as not, whilst one believes one understands what the thread originator wrote, its often the case that what one reads is not what the thread originator meant ![]() Besides which it draws attention away from the OPs problem, which IMO is bad form. BR |
|
![]() |
![]() |
![]() |
#9 | |
Addict
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 265
Karma: 724240
Join Date: Aug 2013
Device: KyBook
|
Sorry about that, like said, sometimes what the OP meant and what was read are two different things.
@Helen, Quote:
As for the main issue. I didn't bother to dig deeper what happened. I've recovered all books by simply doing a search all files and dropping the result on a nnew blank library window. I'll worry bout the metadata.opf's later. I've got a fairly good idea what happened but since it's an obscure situation and prolly not occurring frequent enough it's likely going to take a while before the cause is found. That its a bug I am pretty confident about because files do not get corrupted just by themselves and present day file systems are reliable enough to rule out system corruption. As I said I got a "windows error 5" during the copy+delete. Which means there was a deadlock situation (the error means 'access denied') during the copy+delete. Since I let nothing else but Calibre touch my libraries (having read the various warnings) I am pretty sure that this is something in Calibre or possibly in combination with a plugin. Now, all that is part 1 of the problem. Part 2 of the problem is why on earth a 1 character library path can be considered too long when trying to performing a "Recover Library". And that I cannot figure nor even understand. Plus that the error stack trace isn't helping at all in this case as it would have been much more useful if it had reported the title/author it tripped over. Even restarting in debug mode did not gave more information Last edited by At_Libitum; 10-19-2013 at 09:44 PM. |
|
![]() |
![]() |
![]() |
#10 |
US Navy, Retired
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 9,895
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Kindle PaperWhite SE 11th Gen
|
Hopefully between you and Kovid some sense can be made of your situation.
|
![]() |
![]() |
![]() |
#11 |
creator of calibre
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 45,255
Karma: 27110894
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
You were trying to restore a library located in a location whose path is longer than specified. That error happens if and only if the path of the library being restored is too long, and it refers to the path to the library not the path to anything inside the library. As can be seen here:
https://github.com/kovidgoyal/calibr...ackend.py#L318 I have changed that error message to include the problem path, so that it is clearer. And as for your original problem, of a duplicate hardlink causing a spurious file locked message, that was fixed in 1.7, as is documented in the whats new page. |
![]() |
![]() |
![]() |
#12 |
creator of calibre
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 45,255
Karma: 27110894
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
And I think I figured out why you were getting the restore failure even though your library path was shorter than 89 chars. The restore process creates a temporary library, I'm guessing that the path to the temp folder was longer than 89 chars on your system. IIRC this can happen if your username is very long as the temp folder is typically, some thing like
C:\Users\username\AppData\Local\Temp\calibre_XXXXX X\XXXXXX_library_restore which is already 75 characters. I have shortened that path to C:\Users\username\AppData\Local\Temp\calibre_XXXXX X\XXXXXX_rlib which saves 11 characters EDIT: I've further changed the code to first try to create the temp library in the parent folder of the library you are tring to restore, and only if that fails, to use the temporary directory. That should make it much more robust. Last edited by kovidgoyal; 10-20-2013 at 12:12 AM. |
![]() |
![]() |
![]() |
#13 |
Addict
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 265
Karma: 724240
Join Date: Aug 2013
Device: KyBook
|
Ahh...the temp folder, that option I missed completely and the worst thing is I could have realized this...been seeing the temp folder often enough as of late...
indeed it can be quite long...but not from my username... reconstructing what then must have happened, the path on my system would have been C:\Users\Ruud\AppData\Local\Temp\calibrePortableTe mp\calibre_XXXXXX\XXXXXX_library_restore\ which is already over the 89 char limit. Good to hear the access denied situation also is fixed, just my luck that I had still to apply the 1.7 update and was still running on 1.6 Last edited by At_Libitum; 10-20-2013 at 08:19 AM. |
![]() |
![]() |
![]() |
#14 |
null operator (he/him)
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 21,662
Karma: 29711016
Join Date: Mar 2012
Location: Sydney Australia
Device: none
|
@At_Libitum : you can give calibre a dedicated temp folder via the CALIBRE_TEMP_DIR environment variable - see http://manual.calibre-ebook.com/cust...ment-variables
I point it at c:\caltemp\ ![]() BR |
![]() |
![]() |
![]() |
#15 | |
Addict
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 265
Karma: 724240
Join Date: Aug 2013
Device: KyBook
|
Quote:
![]() |
|
![]() |
![]() |
![]() |
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
Problem splitting a calibre library... | hiitsmescott | Calibre | 3 | 11-23-2012 12:18 AM |
Calibre Library problem | Beaverbank | Library Management | 3 | 06-15-2011 05:56 AM |
Kindle and Calibre user with problem importing large library into Calibre | pleabargain | Calibre | 1 | 12-07-2010 10:19 AM |
problem with calibre library | slainte | Calibre | 20 | 08-11-2010 05:12 PM |