Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Related Tools

Notices

Reply
 
Thread Tools Search this Thread
Old 04-25-2012, 10:05 PM   #1
vibbix
Junior Member
vibbix began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Apr 2012
Device: Kindle Touch
Extra-large Database sizes?

My library is entirely in .MOBI, and consists of over 8000 books, with Metadata reaching 20MB... It takes about 5 to 10 minutes for Calibre to boot, as it starts with approx. 100MB of ram on my comp, and requires 180MB+ to start working. The large database size also makes it hard to use Calibre2OPDS, which I want to host on my NAS, giving me a "Out of memory" error in the command prompt(it still allows me to go into C2O, but trying to generate the library and it refuses). The library is already stored on the NAS, and even without the ethernet/USB2.0 bottleneck, it takes several minutes to boot up Calibre. Is there anyway to manually create (Via terminal/command prompt?) a Calibre2OPDS HTML library that can actually store my database?
vibbix is offline   Reply With Quote
Old 04-25-2012, 11:07 PM   #2
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 15,061
Karma: 5939999
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Moderator Notice
Moved to (OPDS) Catalog area


While Calibre has not published a definitive: Minimum Equipment required to run Spec, it sure sounds like you are trying to run on a sub-standard system.
A lowly P4 with 2G RAM opens a 4000+ book Library in about 20 seconds.
theducks is online now   Reply With Quote
Old 04-26-2012, 12:25 AM   #3
PeterT
Taking a break; Fed up
PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.
 
PeterT's Avatar
 
Posts: 7,183
Karma: 45264785
Join Date: Nov 2007
Location: Toronto
Device: Wife: Touch, Arc, Vox Me: Nexus 7, Glo
When you say "The library is already stored on the NAS" what type of NAS is it?

How are you connecterd to the NAS?

How are the drives attached to the NAS?

What happens if you put everything on a local hard drive?
PeterT is offline   Reply With Quote
Old 04-26-2012, 02:30 AM   #4
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,094
Karma: 780247
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
What is the spec of your machine, particularly in terms of RAM? I use calibre2opds with my library on a NAS and about 16k books with an average of about 4 formats per book. My metadata.db file is over 70MB. Calibre2Opds tends to peak at around 350MB RAM usage when handling this.

In practise I would have thought that 512MB RAM was a minimum for both Calibre and Calibre2Opds, and 1GB was more realistic. This is very low for most modern machines which tend to have considerably more RAM.
itimpi is offline   Reply With Quote
Old 04-26-2012, 04:09 PM   #5
vibbix
Junior Member
vibbix began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Apr 2012
Device: Kindle Touch
6GB of ram, 2x 640Gb HDD's at 7200RPM, Core i5 2410m, GT555m graphics(Dell XPS 17)... the NAS is probobally the what is slowing it down. Its a WD 1TB hard drive(usb 2.0), attached to my router(Netgear n600)... I can get 20mb while downloading through uSB, closer to 4000k while in NAS... I want to store the data on the NAS so I can access it outside the LAN...
vibbix is offline   Reply With Quote
Old 04-26-2012, 04:15 PM   #6
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,094
Karma: 780247
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
It does not sound to me as if your machine spec is a problem. I can understand why the NAS might slow down Calibre2Opds, but not why it would make it fail to run. I use Calibre2Opds against my NAS with a much bigger library. In my case Calibre2Opds can take a few hours to complete the run, but I do not have the problem you report of it failing to run.

In terms of speeding Calibre up, you might want to look for the Calibre-Portable.bat file that is held in the resources sub-folder of the Calibre install location. It shows you how you can control where all the Calibre files are located. I use it to keep the calibre metadata.db file on my local hard disk with all the Book files on the NAS. I then periodically copy the metadata.db file into the NAS Calibre library location.
itimpi is offline   Reply With Quote
Old 04-26-2012, 05:58 PM   #7
vibbix
Junior Member
vibbix began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Apr 2012
Device: Kindle Touch
It allows opens the client, it gives me a SQL out of ram exception.... It wont let me install Calibre2OPDS, so i have to manually run the batch script to let it run... I will try the database move....
vibbix is offline   Reply With Quote
Old 04-26-2012, 08:41 PM   #8
PeterT
Taking a break; Fed up
PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.
 
PeterT's Avatar
 
Posts: 7,183
Karma: 45264785
Join Date: Nov 2007
Location: Toronto
Device: Wife: Touch, Arc, Vox Me: Nexus 7, Glo
I hate to say it but expecting reasonable performance through a USB 2.0 attached drive to a consumer router is a bit much. Routers are limited in both CPU and memory performance.

From a review of this particular router they show the following speeds for reads / writes

Code:
FAT32 Write (MBytes/s)	12.5	
FAT32 Read (MBytes/s)	13.3	
NTFS Write (MBytes/s)	4.3	
NTFS Read (MBytes/s)	12.0
shows that NTFS writes are relatively slow.
PeterT is offline   Reply With Quote
Old 04-27-2012, 10:44 AM   #9
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,094
Karma: 780247
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Quote:
Originally Posted by vibbix View Post
It wont let me install Calibre2OPDS, so i have to manually run the batch script to let it run... I will try the database move....
If you are using 64-bit Java on Windows 64-bit then there is a known issue with the installer not running properly. 32-bit Java on 64-bit Windows is OK for the installer. However if you follow the instructions for doing the manual install from the ZIP file then the main Calibre2Opds program is happy with 64-bit Java on 64-bit Windows.

I have never seen the SQLite error that you mentioned so am not sure what could be causing that.

When you talk about running the batch script, exactly what do you mean? It was not clear if you meant running the rungui.cmd file to start Calibre2Opds in graphical mode, or the run.cmd file to start in batch (non-graphical) mode. If the latter then note that you have to have used the graphical mode successully at least once before the batch mode will run as it is the graphical mode that sets up the required configuration parameters.
itimpi is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
PRS-950 Extra large font on a Kobo book Alleyoop Sony Reader 49 03-12-2012 07:33 AM
Extra large left margin? peartree Amazon Kindle 5 02-23-2012 01:01 PM
Splitting large file sizes ... Hangman Calibre 1 04-05-2010 11:01 PM
Fictionwise extra 10% large purchase micropay at $50 instead of $100?? texasnightowl Deals, Freebies, and Resources (No Self-Promotion) 4 12-21-2009 05:27 PM
Extra Large Print Kindle ebooks wallcraft Amazon Kindle 1 12-20-2008 11:53 AM


All times are GMT -4. The time now is 11:35 AM.


MobileRead.com is a privately owned, operated and funded community.