Copying Library to Windows from Linux (or other Windows installation)

[grumbles]

I am attempting to copy my Calibre library from Linux to Windows (XP in this case but the problem remains for any other version of Windows.) In theory all I have to do is copy the Calibre library but this won't work. Every time I try this I run into the path length error, even if the destination library name is a single character. This has been a bug in Windows forever, Microsoft calls it a limitation but if you can't copy a directory from on installation to another it is a bug. The bug is in explorer, the underlying NTFS file system accepts path lengths up to 32k.

Alternatively I can simply add the books to the library on Windows, but I need to also copy the dates the books were added. If I simply add the books I will lose this information as well as any metadata, I don't want to have to have to download it all again.

Any thoughts on how I can get the library over to Windows. And yes, I need to physically more the file, setting up a Calibre server would not be useful in my case.

[readx]

Would't it work by zipping the bibliothek on linux, moving it over and unzipping it on windows?

[DaltonST]

@grumbles,


An option (a variation of which I have personally used for a different purpose and is described in detail at https://www.mobileread.com/forums/sho...84760#poststop ) would be to make a copy of your metadata.db, put it safely aside, then add your books into Calibre in Windows. Your valuable metadata that will be changed in the process can be manually copied from your sequestered original metadata.db to the new Windows metadata.db using a SQLite manager. Your SQL would have to match on title & author_sort in the books table (more if at all possible, such as isbn from the identifiers table), since the integer book ids and the integer author ids would be different in your new library.

The Calibre table to look at would be "books". Attach the "old" metadata.db, and do something like [example for published date here]: "UPDATE main.books SET pubdate = (SELECT pubdate FROM old.books WHERE old.books.title = main.books.title AND old.books.author_sort = main.books.author_sort AND (whatever else you can match on) . The above example is for pubdate, but it would work for last_modified and timestamp, too. Be careful not to mess with anything that Calibre will create uniquely for the books in the new Windows metadata.db (e.g. path, uuid, etc.). Back up the "new" metadata.db after your books have been added but before executing any of your own SQL queries.


Best of luck.


DaltonST

[JSWolf]

Originally Posted by grumbles

I am attempting to copy my Calibre library from Linux to Windows (XP in this case but the problem remains for any other version of Windows.) In theory all I have to do is copy the Calibre library but this won't work. Every time I try this I run into the path length error, even if the destination library name is a single character. This has been a bug in Windows forever, Microsoft calls it a limitation but if you can't copy a directory from on installation to another it is a bug. The bug is in explorer, the underlying NTFS file system accepts path lengths up to 32k.

Alternatively I can simply add the books to the library on Windows, but I need to also copy the dates the books were added. If I simply add the books I will lose this information as well as any metadata, I don't want to have to have to download it all again.

Any thoughts on how I can get the library over to Windows. And yes, I need to physically more the file, setting up a Calibre server would not be useful in my case.

If you still have access to the library via Calibre for Linux and Linux can talk to where you want the library moved, use Calibre/Linux to copy the library.

If not, use WinRAR (under Windows) to compress the library and then decompress it to where you want it.

[PeterT]

Jon; I don't believe that would help at all. The issue is that the Windows build of calibre is aware of the need for path / file name shortening, while the Linux version is not. So even if you use the Linux version to copy to a new location, the resulting path / file names will still cause issues.

[JSWolf]

Originally Posted by PeterT

Jon; I don't believe that would help at all. The issue is that the Windows build of calibre is aware of the need for path / file name shortening, while the Linux version is not. So even if you use the Linux version to copy to a new location, the resulting path / file names will still cause issues.

Then how about copying all the directories that are not too long, then use check library to check the library and then manually add in the books that were in directories that are too long.

[BetterRed]

The Windows legacy file i/o API's which is where the path length restrictions lie, and which every software vendor I know of uses, predate NTFS by several years.

IIRC they were inherited from the MS/IBM OS/2 partnership and HPFS. MS could have done what Apple did in the late 90,s with Next, and revamped Windows from the bottom up. But MS had and still has a huge user base, especially large corporate users. Apple managed to survive cutting off their already disaffected and dwindling user base - there's no way MS could have done that or even now do that.

The other issue is name casing - e.g. on most file systems in LinuxLand - mydata, myData, MyData, mYdATA, mYdAtA... MYDATA can coexist, in Windows they can't (nor in OSX if IIRC).

If you massage the library on Linux to ensure that no path length exceeds 250 characters (I think that'll be short enough) and you eliminate authors and titles that differ only in their character casing and you're happy to have the library at the root of a drive then you should be able to copy the library - I think

BR

[grumbles]

I found a partial solution. I copied the book directories to a memory stick, took it over the windows computer and the used the add books function from the stick. Calibre kept the metadata and cover along with the book. This kept the date added which was my main concern. I would have happier if the actual order in the new library was kept but this is good enough, at least for now.

The path length limit goes all the way back to DOS and the original version of Windows. Zipping the directories. What is needed is a utility or addon that will create an image of the library and then restore the library in a new location updating the database metadata for the changed paths/filenames. A project that is beyond my programming skills. I'm just an old, retired Pascal programmer.

[eschwartz]

That is not a partial solution, that is the official one.

Alternatively, use Library Maintenance ==> Restore database. To get around the file paths, use a bulk renamer to shorten the filenamepaths.
This has the advantage of retaining settings and custom metadata columns.

[grumbles]

Originally Posted by eschwartz

That is not a partial solution, that is the official one.

It may be official but it is not complete. A complete solution would maintain the order in the database as well. I'm not concerned about keeping the settings and I found that Calibre did import my custom data (i.e. if I've read the book or not.) It will do as long as I have to have a copy of my library on Windows.
My primary library remains on Linux (or maybe BSD in the future.) The copy on Windows is so my wife can move books on and off her reader by herself. I loathe the Windows user interface (but not as much as I loathe Ubuntu's unity!) I really wish programmers would stop trying to pretend that they are designers. They are so very very bad at it!

[eschwartz]

Originally Posted by grumbles

It may be official but it is not complete. A complete solution would maintain the order in the database as well. I'm not concerned about keeping the settings and I found that Calibre did import my custom data (i.e. if I've read the book or not.) It will do as long as I have to have a copy of my library on Windows.

You can blame Microsoft for that...

What I would do is use a filename shortener and make sure the id # in parentheses is preserved (in the per-book directory). After shortening the filenames to the point where Windows will accept them, your library will be horribly broken (that is rather the point) but it should copy over perfectly.

Then use the Restore database feature from Library Maintenance to rebuild the database.

Adding the books to calibre via save-to-disk and reimport is just as though it was the same library, except the actual id references change. Honestly, I could care less about that since the add date and all is preserved.

Restore database will use the ids from the filename paths, by assuming there is a valid library with a broken database.

Restore will preserve all custom columns and other per-library setings et cetera using the metadata_db_prefs_backup.json file.

My primary library remains on Linux (or maybe BSD in the future.) The copy on Windows is so my wife can move books on and off her reader by herself. I loathe the Windows user interface (but not as much as I loathe Ubuntu's unity!) I really wish programmers would stop trying to pretend that they are designers. They are so very very bad at it!

I too hated Unity. My solution was to stop using it.

I switched from Ubuntu to ArchLinux and haven't looked back. My current desktop is Cinnamon, the face of Linux Mint (a Ubuntu derivative) which is a distro I would recommend as the current best it-just-works distro for linux newbies and for veterans who don't want to fight their system.

I would say that Cinnamon is worlds better even than GNOME2 which I was a big fan of.