Books all over the place Oh my

[ScottRKahler]

I am looking for suggestions to get my files in order on my hard-drive. Right now I have the files in two locations. Performed scott@home/dir1 $ ls -R > dir1.txt and cott@home/dir2 $ ls -R > dir2.txt. I am using diff to sort out what is happening. Also, Nautilus search feature.
So, right now. I am :
1. reviewing diff output to find something out of place.
2. searching for the files in Nautilus
3. reviewing results
4. removing extra's/ renaming files/looking at things

Things that complicate the solution. one of the directories is a SMB share on a windows box. I think I have both a mounted folder and a Nautilus mount. I actually didn't realize there was a difference.


I have seen command line solutions that also add an MD5 hash into the mix. I don't know if I need that level of detail.

[Adoby]

If it is the same books, with the same names and the same folder structure, but from different points in time, you could use Midnight Commander to copy both trees to the same new overlapping location.

When asked if you want to write over existing files specify update so you only write over if newer.

The "cp" command can do the same using -R for recursive and -u for update.

One complication could be that you may end up with duplicates with very similar names. This is because SMB/CIFS has a different case sensitivity compared to most Linux file systems.

Yet another method could be to import both folder into a new library in calibre and use calibre to find, examine and handle duplicates.

[latepaul]

What I would probably do if it was me is write a simple script to do a find for ebooks and add them to a Calibre library (probably one set up purely for the purpose). I'd then use Calibre to find duplicates and tidy up the metadata. Once happy, if I wanted a non-Calibre archive, I could then do a Save to Disk.

[darryl]

What is it that you actually want to achieve? If you just want to make sure you have all books in one directory I would have thought rsync was the logical answer.

For example, the command:

rsync -avzP /home/scott/dir2/ /home/scott/dir1/

should result in dir1 containing its original files and the files in dir2 that were not already in dir 1.

Adding the n option will do a dry run, listing on the console what actions would be performed.

rsync -anvzP /home/scott/dir2/ /home/scott/dir1/

This is a very simple explanation of using rsync in these types of circumstances:

https://www.digitalocean.com/communi...ories-on-a-vps

My recommendation would be to use Calibre to manage your ebooks. Once imported, you could safely delete the original source. You should, of course, keep at least one backup of your Calibre directories.

[ScottRKahler]

What is it that you actually want to achieve?

I want to make sure that the books that I have:
good directory tree for Calibre.
is not a duplicate of another book. ( This does do not include other formats. For example, if I have a book called The Best Book in the World, in pdf and epub those two are in the proper folder for Calibre. )
No empty listings in Calibre
Overall, I just want to organize them. Make sure the books have a nice tidy place to live. Then remove the garbage.

I would have thought rsync was the logical answer.

does rsync work over samba share? for some reason, I thought that only worked with ssh.

[ScottRKahler]

Originally Posted by latepaul

What I would probably do if it was me is write a simple script to do a find for ebooks and add them to a Calibre library (probably one set up purely for the purpose). I'd then use Calibre to find duplicates and tidy up the metadata. Once happy, if I wanted a non-Calibre archive, I could then do a Save to Disk.

You mean like a bash script?
What would this script look like? When adding books what does Calibre like?

[ScottRKahler]

I think this is more of a library management question. can I have this thread moved there?
I can start my library over. I think I did something wrong in the setup of it. What I would like to do is gather all of my books in one place. Point Calibre at it and tell it to add all of the metadata. I think I have to many errors to do it any other way. I want to make sure I am doing the setup correctly. I want to get rid of anything that isn't a book for Calibre. It will create the metadata needed.
Does that sound like a good plan? What is the best way of doing that?

[Adoby]

One possible method:

Create a new calibre library. Call it "in". This is a repository for all books, unchanged.

Add all books from the two trees to "in".

Handle duplicates and bad books. Delete obvious junk.

Create a new library. Call it "finished". (Or "fiction", "non-fiction" and so on if needed.)

Copy a few books to "finished" from "in". 5-10?

Keep the books in "in" but tag them as "added to finished". Makes it easier to avoid duplicates in "finished" in the future. And you have a backup is something goes wrong.

Fix ALL the books you just added to "finished". Perfect metadata and cover and so on.

If something remains fixing, move the book back to "in" and tag it "todo". Or possibly use a small "working" library.

Setup a system to backup your calibre libraries.

When you download new books, add them to the "in" repository and handle them when you have time. Perhaps one author at a time or a series at a time or when you need something to read.

[sealbeater]

Originally Posted by latepaul

I'd then use Calibre to find duplicates and tidy up the metadata.

If I were you, I would just make a directory, put both directories in it and run some sort of de-duplication software such as fdupes or dupseek before you import them into calibre.


It might also be a good idea to pre-process your books before adding them. Look at https://github.com/na--/ebook-tools