Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 02-24-2011, 07:03 AM   #1
sam_uk
Member
sam_uk began at the beginning.
 
Posts: 15
Karma: 10
Join Date: Feb 2011
Device: Android
Uber Calibre script - ubuntu

Hi all

I am new to ebooks. I have got a 'mess' of ebook files on my hardrive, different formats, some compressed, some in a folder directory structure, some by author, some by title.

I was hoping someone might have a script that

1) Recursively extracts all tarred files [ find ./ -name '*.rar' -exec unrar e -o+ {} \; ]

2) Organises those files into a single flat directory [ find . -type f -exec mv '{}' . \; ]

3) Removes duplicates [ yes 1 | fdupes -rd . ]

4) Deletes .rar files (space saving) [ find -name "*.rar" -delete ]

5) Copies all epub files to a new folder called 'calibre' or similar

6) Coverts all the remaining .pdf .txt etc files to .epub and moves them to 'calibre' folder

7) Sorts resulting folder alphabetically

8) Goes and grabs the cover art from the internet

9) Makes me a cup of tea whilst it completes!

Not much to ask!

Any pointers or part-solutions welcomed..

Thanks

Sam

Last edited by sam_uk; 02-24-2011 at 02:46 PM.
sam_uk is offline   Reply With Quote
Old 02-24-2011, 08:53 AM   #2
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by sam_uk View Post
I have got a 'mess' of ebook files on my hardrive, different formats, some compressed, some in a folder directory structure, some by author, some by title.
I had that mess when I started with Calibre.
Quote:
I was hoping someone might have a script that
Not a script, but I did write some functions into Calibre that will help.
Quote:
1) Recursively extracts all tarred files [ find ./ -name '*.rar' -exec unrar e -o+ {} \; ]
Calibre reads *.rar and *.zip files and extracts embedded ebooks.
Quote:
2) Organises those files into a single flat directory [ find . -type f -exec mv '{}' . \; ]
Why? Calibre doesn't care where they are.
Quote:
3) Removes duplicates [ yes 1 | fdupes -rd . ]
Turn on the Automerge function. Duplicate authors with near duplicate (fuzzy matched) titles are ignored if it's the same format.
Quote:
4) Copies all epub files to a new folder called 'calibre' or similar
Why?
Quote:
5) Coverts all the remaining .pdf .txt etc files to .epub and moves them to 'calibre' folder
Global; conversions usually suck, but you can do it easily enough inside Calibre after import.

Quote:
6) Sorts resulting folder alphabetically
Sort however you like in Calibre, after import.
Quote:
7) Goes and grabs the cover art from the internet
Calibre will do it globally.

Quote:
8) Makes me a cup of tea whilst it completes!
Nope - sorry

Last edited by Starson17; 02-24-2011 at 02:04 PM.
Starson17 is offline   Reply With Quote
Old 02-24-2011, 11:58 AM   #3
sam_uk
Member
sam_uk began at the beginning.
 
Posts: 15
Karma: 10
Join Date: Feb 2011
Device: Android
Great thanks for the responses! Seems like I just need to play around some more.

Could you clarify a couple of things?

I am indexing *lots of Gigs of ebooks with calibre. Does it basically copy every file?

I have separate partitions:

/
/home

As Calibre indexes this content I can see space disappearing in my / (system/ root) partition. What is it doing here?

Thanks

Sam
sam_uk is offline   Reply With Quote
Old 02-24-2011, 12:12 PM   #4
Manichean
Wizard
Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!
 
Manichean's Avatar
 
Posts: 3,130
Karma: 80446
Join Date: Feb 2008
Location: Germany
Device: Cybook Gen3
Calibre copies books added to the library into the library folder you specified (which, by the way, should be totally empty when you start your libary, it's a bad idea to select a folder with books already in it). If your library is somewhere on your root partition, you'll see the free space being taken up by Calibre's copies of your books.
Manichean is offline   Reply With Quote
Old 02-24-2011, 01:50 PM   #5
sam_uk
Member
sam_uk began at the beginning.
 
Posts: 15
Karma: 10
Join Date: Feb 2011
Device: Android
Quote:
Originally Posted by Manichean View Post
Calibre copies books added to the library into the library folder you specified (which, by the way, should be totally empty when you start your libary, it's a bad idea to select a folder with books already in it). If your library is somewhere on your root partition, you'll see the free space being taken up by Calibre's copies of your books.

My library folder was empty when I started. It is on my /home directory.

Is there a command line option I can run to delete the original files as it runs? I do not have enough hard drive space to duplicate the entire collection.

Is there also a way to limit the RAM that Calibre uses? It is using 100% of 2Gigs on Ubuntu which means I cannot run the import as a background process..

Thanks

Sam
sam_uk is offline   Reply With Quote
Old 02-24-2011, 01:58 PM   #6
Manichean
Wizard
Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!
 
Manichean's Avatar
 
Posts: 3,130
Karma: 80446
Join Date: Feb 2008
Location: Germany
Device: Cybook Gen3
Quote:
Originally Posted by sam_uk View Post
My library folder was empty when I started. It is on my /home directory.
Good, I wrote that just to be on the safe side
Edit to add: Wait, you said your /home is on another partition, and free space on / is shrinking? Then this is not caused by Calibre adding the files to the library. Might be temporary files?

Quote:
Is there a command line option I can run to delete the original files as it runs? I do not have enough hard drive space to duplicate the entire collection.
No. You could create a script that first adds the book to the Calibre library via the CLI tools and then deletes the file, though. Or you could just add smaller batches to the GUI (you should be doing that anyway) and manually delete the books afterwards.

Quote:
Is there also a way to limit the RAM that Calibre uses? It is using 100% of 2Gigs on Ubuntu which means I cannot run the import as a background process..
I don't think there is a way.
Manichean is offline   Reply With Quote
Old 02-24-2011, 02:56 PM   #7
sam_uk
Member
sam_uk began at the beginning.
 
Posts: 15
Karma: 10
Join Date: Feb 2011
Device: Android
calibredb add -r .

Seemed to be a better way of adding stuff to the library. It did not use all the ram which was nice.
sam_uk is offline   Reply With Quote
Old 02-24-2011, 03:35 PM   #8
user_none
Sigil & calibre developer
user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.
 
user_none's Avatar
 
Posts: 2,433
Karma: 950001
Join Date: Jan 2009
Location: Florida, USA
Device: Nook STR
The space you're seeing used in / is temporary files. They are typically only deleted when calibre quits.

There is no way to limit RAM usage. calibe will use as much as it needs / can to complete the task as quick as possible.
user_none is offline   Reply With Quote
Old 02-24-2011, 05:30 PM   #9
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 8,770
Karma: 12516053
Join Date: Feb 2009
Location: North Carolina
Device: Nexus 7
Quote:
Originally Posted by Starson17 View Post
Calibre reads *.rar and *.zip files and extracts embedded ebooks.
I may be wrong but I believe calibre only expects one book per archive. If there is more then one book in the file only one gets added.
DoctorOhh is offline   Reply With Quote
Old 02-25-2011, 09:02 AM   #10
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by dwanthny View Post
I may be wrong but I believe calibre only expects one book per archive. If there is more then one book in the file only one gets added.
Correct. The "ebooks" are extracted from their corresponding "files".
Starson17 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Does anyone have a launchd script for calibre-server? offby1 Calibre 8 06-06-2014 08:04 AM
Converting a film script (PDF) to EPUB with Calibre alanjay Calibre 7 10-19-2010 10:41 AM
Calibre under Ubuntu 9.04 ebeckhusen Calibre 4 04-10-2009 04:22 PM
Cannot install calibre on ubuntu 8.04 kkkidder Calibre 1 03-19-2009 10:44 PM
Error in script on 0.4.113 - 64bit Ubuntu CaptOzone Calibre 10 02-12-2009 05:27 PM


All times are GMT -4. The time now is 02:14 PM.


MobileRead.com is a privately owned, operated and funded community.