Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 07-11-2018, 06:24 AM   #1
ScottRKahler
Member
ScottRKahler began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Jul 2018
Device: Moon Reader Pro
Books all over the place Oh my

I am looking for suggestions to get my files in order on my hard-drive. Right now I have the files in two locations. Performed scott@home/dir1 $ ls -R > dir1.txt and cott@home/dir2 $ ls -R > dir2.txt. I am using diff to sort out what is happening. Also, Nautilus search feature.
So, right now. I am :
1. reviewing diff output to find something out of place.
2. searching for the files in Nautilus
3. reviewing results
4. removing extra's/ renaming files/looking at things

Things that complicate the solution. one of the directories is a SMB share on a windows box. I think I have both a mounted folder and a Nautilus mount. I actually didn't realize there was a difference.


I have seen command line solutions that also add an MD5 hash into the mix. I don't know if I need that level of detail.
ScottRKahler is offline   Reply With Quote
Old 07-11-2018, 06:39 AM   #2
Adoby
Handy Elephant
Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.
 
Adoby's Avatar
 
Posts: 1,286
Karma: 8142344
Join Date: Dec 2009
Location: Southern Sweden, far out in the quiet woods
Device: Ubuntu LTS, Android, Bouye Likebook Plus
If it is the same books, with the same names and the same folder structure, but from different points in time, you could use Midnight Commander to copy both trees to the same new overlapping location.

When asked if you want to write over existing files specify update so you only write over if newer.

The "cp" command can do the same using -R for recursive and -u for update.

One complication could be that you may end up with duplicates with very similar names. This is because SMB/CIFS has a different case sensitivity compared to most Linux file systems.

Yet another method could be to import both folder into a new library in calibre and use calibre to find, examine and handle duplicates.

Last edited by Adoby; 07-11-2018 at 06:43 AM.
Adoby is offline   Reply With Quote
Old 07-11-2018, 06:46 AM   #3
latepaul
Wizard
latepaul ought to be getting tired of karma fortunes by now.latepaul ought to be getting tired of karma fortunes by now.latepaul ought to be getting tired of karma fortunes by now.latepaul ought to be getting tired of karma fortunes by now.latepaul ought to be getting tired of karma fortunes by now.latepaul ought to be getting tired of karma fortunes by now.latepaul ought to be getting tired of karma fortunes by now.latepaul ought to be getting tired of karma fortunes by now.latepaul ought to be getting tired of karma fortunes by now.latepaul ought to be getting tired of karma fortunes by now.latepaul ought to be getting tired of karma fortunes by now.
 
latepaul's Avatar
 
Posts: 1,092
Karma: 8378220
Join Date: Dec 2011
Device: a variety (mostly kindles and kobos)
What I would probably do if it was me is write a simple script to do a find for ebooks and add them to a Calibre library (probably one set up purely for the purpose). I'd then use Calibre to find duplicates and tidy up the metadata. Once happy, if I wanted a non-Calibre archive, I could then do a Save to Disk.
latepaul is offline   Reply With Quote
Old 07-11-2018, 06:53 AM   #4
darryl
Wizard
darryl ought to be getting tired of karma fortunes by now.darryl ought to be getting tired of karma fortunes by now.darryl ought to be getting tired of karma fortunes by now.darryl ought to be getting tired of karma fortunes by now.darryl ought to be getting tired of karma fortunes by now.darryl ought to be getting tired of karma fortunes by now.darryl ought to be getting tired of karma fortunes by now.darryl ought to be getting tired of karma fortunes by now.darryl ought to be getting tired of karma fortunes by now.darryl ought to be getting tired of karma fortunes by now.darryl ought to be getting tired of karma fortunes by now.
 
darryl's Avatar
 
Posts: 2,313
Karma: 35576750
Join Date: Nov 2011
Location: Australia
Device: Kobo Aura H2O, Kindle Oasis, Huwei Ascend Mate 7
What is it that you actually want to achieve? If you just want to make sure you have all books in one directory I would have thought rsync was the logical answer.

For example, the command:

Code:
rsync -avzP /home/scott/dir2/ /home/scott/dir1/
should result in dir1 containing its original files and the files in dir2 that were not already in dir 1.

Adding the n option will do a dry run, listing on the console what actions would be performed.

Code:
rsync -anvzP /home/scott/dir2/ /home/scott/dir1/
This is a very simple explanation of using rsync in these types of circumstances:

https://www.digitalocean.com/communi...ories-on-a-vps

My recommendation would be to use Calibre to manage your ebooks. Once imported, you could safely delete the original source. You should, of course, keep at least one backup of your Calibre directories.
darryl is offline   Reply With Quote
Old 07-12-2018, 03:06 AM   #5
ScottRKahler
Member
ScottRKahler began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Jul 2018
Device: Moon Reader Pro
Quote:
What is it that you actually want to achieve?
I want to make sure that the books that I have:
good directory tree for Calibre.
is not a duplicate of another book. ( This does do not include other formats. For example, if I have a book called The Best Book in the World, in pdf and epub those two are in the proper folder for Calibre. )
No empty listings in Calibre
Overall, I just want to organize them. Make sure the books have a nice tidy place to live. Then remove the garbage.
Quote:
I would have thought rsync was the logical answer.
does rsync work over samba share? for some reason, I thought that only worked with ssh.
ScottRKahler is offline   Reply With Quote
Old 07-12-2018, 03:25 AM   #6
ScottRKahler
Member
ScottRKahler began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Jul 2018
Device: Moon Reader Pro
Quote:
Originally Posted by latepaul View Post
What I would probably do if it was me is write a simple script to do a find for ebooks and add them to a Calibre library (probably one set up purely for the purpose). I'd then use Calibre to find duplicates and tidy up the metadata. Once happy, if I wanted a non-Calibre archive, I could then do a Save to Disk.
You mean like a bash script?
What would this script look like? When adding books what does Calibre like?
ScottRKahler is offline   Reply With Quote
Old 07-12-2018, 06:35 AM   #7
ScottRKahler
Member
ScottRKahler began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Jul 2018
Device: Moon Reader Pro
I think this is more of a library management question. can I have this thread moved there?
I can start my library over. I think I did something wrong in the setup of it. What I would like to do is gather all of my books in one place. Point Calibre at it and tell it to add all of the metadata. I think I have to many errors to do it any other way. I want to make sure I am doing the setup correctly. I want to get rid of anything that isn't a book for Calibre. It will create the metadata needed.
Does that sound like a good plan? What is the best way of doing that?
ScottRKahler is offline   Reply With Quote
Old 07-12-2018, 06:58 AM   #8
Adoby
Handy Elephant
Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.
 
Adoby's Avatar
 
Posts: 1,286
Karma: 8142344
Join Date: Dec 2009
Location: Southern Sweden, far out in the quiet woods
Device: Ubuntu LTS, Android, Bouye Likebook Plus
One possible method:

Create a new calibre library. Call it "in". This is a repository for all books, unchanged.

Add all books from the two trees to "in".

Handle duplicates and bad books. Delete obvious junk.

Create a new library. Call it "finished". (Or "fiction", "non-fiction" and so on if needed.)

Copy a few books to "finished" from "in". 5-10?

Keep the books in "in" but tag them as "added to finished". Makes it easier to avoid duplicates in "finished" in the future. And you have a backup is something goes wrong.

Fix ALL the books you just added to "finished". Perfect metadata and cover and so on.

If something remains fixing, move the book back to "in" and tag it "todo". Or possibly use a small "working" library.

Setup a system to backup your calibre libraries.

When you download new books, add them to the "in" repository and handle them when you have time. Perhaps one author at a time or a series at a time or when you need something to read.
Adoby is offline   Reply With Quote
Old 07-17-2018, 04:34 AM   #9
sealbeater
Addict
sealbeater ought to be getting tired of karma fortunes by now.sealbeater ought to be getting tired of karma fortunes by now.sealbeater ought to be getting tired of karma fortunes by now.sealbeater ought to be getting tired of karma fortunes by now.sealbeater ought to be getting tired of karma fortunes by now.sealbeater ought to be getting tired of karma fortunes by now.sealbeater ought to be getting tired of karma fortunes by now.sealbeater ought to be getting tired of karma fortunes by now.sealbeater ought to be getting tired of karma fortunes by now.sealbeater ought to be getting tired of karma fortunes by now.sealbeater ought to be getting tired of karma fortunes by now.
 
Posts: 379
Karma: 1700000
Join Date: Jan 2008
Device: Sony Reader PRS-505 : Onyx Boox Max : Sony PRS-900 : Onyx Kepler Pro
Quote:
Originally Posted by latepaul View Post
I'd then use Calibre to find duplicates and tidy up the metadata.
If I were you, I would just make a directory, put both directories in it and run some sort of de-duplication software such as fdupes or dupseek before you import them into calibre.


It might also be a good idea to pre-process your books before adding them. Look at https://github.com/na--/ebook-tools
sealbeater is online now   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Short Fiction Cobb, Irvin S.: From Place to Place (collected shorts). v1. 02 Aug 2017 GrannyGrump Kindle Books 0 08-01-2017 08:02 AM
Short Fiction Cobb, Irvin S.: From Place to Place (collected shorts). v1. 02 Aug 2017 GrannyGrump ePub Books 0 08-01-2017 07:27 AM
Importing books in place Tim L Library Management 2 03-28-2016 11:04 PM
Books that love the place you live Mercury Reading Recommendations 22 09-25-2010 10:56 PM


All times are GMT -4. The time now is 05:25 PM.


MobileRead.com is a privately owned, operated and funded community.