Move Library : function and performance

[BetterRed]

I haven't moved a library for a long time

If a Library is moved via the calibre Switch/Create Library option should it 'disappear' from the old location - I just did a move where the old and new locations were on the same internal SATA3 NTFS drive - the library was copied from old to new and old wasn't deleted.

I 'expected' the operation to be done 'ctrl/x ctrl/v quick' - or am I misremembering.

I use Windows 10 64bit version 1511

BR

[theducks]

Was there any stragglers (non-calibre files) left over?

[eschwartz]

The expected behavior is that each individual file that calibre knows about will be moved, not copied.

So if all the files are copied something is wrong (file locking?) but if there are extra files and a folder structure left behind then that is expected.

[BetterRed]

Originally Posted by theducks

Was there any stragglers (non-calibre files) left over?

No, it was the entire library left behind, and it had a clean bill of health from Check Library. I retested with a freshly created empty library and the library folder with its metadata.db was left behind.

BR

[BetterRed]

Originally Posted by eschwartz

The expected behavior is that each individual file that calibre knows about will be moved, not copied.

So if all the files are copied something is wrong (file locking?)

The time taken to do the so-called 'move' (27 secs) is the approximately the same as File Explorer takes to 'copy' the same folder (24 secs) - a File Explorer 'move' is of course instantaneous

If files are locked calibre will normally thrown an error.

The contents of the old and now library folders are not symlinked, hardlinked or otherwise exotically associated.

I'll have to look up out how to get the file Object-Ids (fsutil I think) - if they're being copied they will be different, if they're being moved they'll be the same.

BR

[PeterT]

fsutil file [quaeryfileid] <FileName>

[BetterRed]

Originally Posted by BetterRed

I'll have to look up out how to get the file Object-Ids (fsutil I think) - if they're being copied they will be different, if they're being moved they'll be the same.BR

Well, I'm pretty sure the files are being copied and not moved

I made sure a couple of files in my test library had object ids before moving the library - the same files in the new location had no object id (object ids are only created when they're needed)

If I move files with File Explorer, xplorer2 or File Menu Utils the Object Ids are retained, if I copy the same files the Object Ids are not retained.

Also - the centre-screen blue progress bar flashes "Copied xxxx..." as it progresses

=================

If for some reason a library folder is deleted in the file manager (intentionally or inadvertently), then next time calibre starts it recreates the folder as an empty library without any warning.

I am almost certain it previously threw an message of some sort - I reset the confirmation dialogues to make sure it I hadn't hidden it.

===================

Peter I used "fsutil objectid query"

C:>fsutil objectid query "D:\CalibreLibraries\_Test\Kovid Goyal\DOCX Demo (185)\DOCX Demo - Kovid Goyal.docx"
Object ID :        f64aaebbac9ee5119cad0002721fc94a
BirthVolume ID :   1cfeae92c7d42d4cbe6eab8da2df5b55
BirthObjectId ID : f64aaebbac9ee5119cad0002721fc94a
Domain ID :        00000000000000000000000000000000

C:>

Thanks

BR

[kovidgoyal]

move library copies and then deletes after the copy operation succeeds. That is an operation mode that is far more tolerant to failures and works even if the destination is on another drive.

[BetterRed]

Originally Posted by kovidgoyal

move library copies and then deletes after the copy operation succeeds. That is an operation mode that is far more tolerant to failures and works even if the destination is on another drive.

Point taken, that was what I suspected I could be misremembering - but my main issue is that it's not doing the deletion - i.e. the old library remains as was (although its no longer known to calibre) much like Remove Library operates. Is that intentional ?

BR

[kovidgoyal]

Works for me. All I get left behind is the original folder with metadata_db_prefs_backup.json in it. All the actual books get moved. I should probably fix metadata_db_prefs_backup.json

[BetterRed]

Originally Posted by kovidgoyal

Works for me. All I get left behind is the original folder with metadata_db_prefs_backup.json in it. All the actual books get moved. I should probably fix metadata_db_prefs_backup.json

Yes I think it would be better to remove those files and the folder at the end of the move.

Should the deletes happen progressively as books are coped? I can set a task to monitor the filesystem operations on the library as its being moved.

The library folder that failed to delete anything would have had a disposition of Read Only (only the library folder not the contents) -- I wonder?

Aside : if some bright spark created shortcuts to their favourite books and then they moved their library, the shortcuts would be rendered irreparable; the repair service (Distributed Link Tracking) relies on the fact that files and folders will retain their Object Ids when they are renamed or moved within the same domain.

BR

[kovidgoyal]

No, deleting happens after all copying has completed successfully, that way you are never left with a partially mangled library in case of unexpected failures.

The deleting code simply ignores failures, since they are not critical. So yes, something about the original library prevented it from being deleted.

[BetterRed]

Originally Posted by kovidgoyal

No, deleting happens after all copying has completed successfully, that way you are never left with a partially mangled library in case of unexpected failures.

The deleting code simply ignores failures, since they are not critical. So yes, something about the original library prevented it from being deleted.

Speaking for myself - I would prefer to know via an end of 'task' message that calibre detected failures during the deletion phase. Otherwise I will be left wondering why the old library folder is still there. This is what I happened this morning - my initial 'problem' wasn't all the books as I assumed it was, just a goodly proportion of them -- I didn't run a diff across the two folders

I can't recreate the problem now - Indexing, Malware Detection, Compression, Defragging - they've all changed in Win 10, to some degree at least.

BR