Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 10-13-2012, 11:33 AM   #16
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,428
Karma: 5560777
Join Date: Aug 2009
Location: The (original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
The Tag browser (usually) remembers those pesky custom column names
I just open the tag browser, select my choices, the close the browse to get my screen width back
theducks is offline   Reply With Quote
Old 10-13-2012, 11:46 AM   #17
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,052
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad1, Kindle3KB
Quote:
Originally Posted by theducks View Post
The Tag browser (usually) remembers those pesky custom column names
I just open the tag browser, select my choices, the close the browse to get my screen width back
Yeah. But it's faster to just type the lookup names in search box, if no mistakes are made.
unboggling is offline   Reply With Quote
Old 10-14-2012, 12:33 PM   #18
lbik
Reader
lbik doesn't litterlbik doesn't litter
 
Posts: 45
Karma: 162
Join Date: Nov 2010
Location: Hannover
Device: Kindle KB and Kindle Fire HD 8.9
The best way to speedup Calibre, ist an SSD for the Library.
lbik is offline   Reply With Quote
Old 10-14-2012, 03:18 PM   #19
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 2,309
Karma: 5761596
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PRS-350, Nexus S, Galaxy S, Nook Color, iPhone4, iPT4, iPad 2012
Quote:
Originally Posted by lbik View Post
The best way to speedup Calibre, ist an SSD for the Library.
I completely agree. Well, it does help for the library folder itself but if space is a premium, then you can use it just for metadata.db. Of course, RAM disk is so much faster.
ilovejedd is offline   Reply With Quote
Old 10-22-2012, 05:08 PM   #20
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,052
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad1, Kindle3KB
Quote:
Originally Posted by lbik View Post
The best way to speedup Calibre, ist an SSD for the Library.
Quote:
Originally Posted by ilovejedd View Post
I completely agree. Well, it does help for the library folder itself but if space is a premium, then you can use it just for metadata.db. Of course, RAM disk is so much faster.
FYI.

I upgraded startup drive to SSD from hard disk, and RAM to 16 GB from 8 GB, in an early 2011 Macbook Pro 17 inch notebook that has a 2.2 GHz quad core i7 processor. Did it myself.

Apple says this model only can be upgraded to 8 GB maximum RAM, but some research showed the real max is 16 GB. I haven't had any problems since installing the 16 GB RAM last week.

The SSD has 6 gigabit link speed, while the old hard disk has 3 gigabit. To preserve the life of the SSD, I turned on trim support and noatime, turned off local Time Machine snapshots, turned off sleep mode memory backup to persistent storage, and disabled sleep disk image.

Speedup results, without doing anything with temp folders and ramdisk:
  • OS X startup time: more than twice as fast on SSD, a significant difference.
  • calibre startup time: 13 seconds on SSD, 18 seconds on hard drive, 25 to 30% faster with calibre app, libraries, and config dir all on SSD instead of hard drive. That is with startup restriction off. (Starting up in restriction requiress extra time for the restriction search).
  • bulk conversion time: shaved 1 or 2 seconds off each conversion during bulk conversion operation, approx 5% faster on SSD. Not as much speedup as I expected.

Last edited by unboggling; 10-22-2012 at 08:14 PM. Reason: variable leveling, test result corrections
unboggling is offline   Reply With Quote
Old 10-22-2012, 05:42 PM   #21
speakingtohe
Wizard
speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.
 
Posts: 4,596
Karma: 25170848
Join Date: Apr 2010
Device: sony PRS-T1 and T3, Kobo Mini and Aura HD, Tablet
Quote:
Originally Posted by unboggling View Post
FYI.

I upgraded startup drive to SSD from hard disk, and RAM from 8 GB to 16 GB, in an early 2011 Macbook Pro 17 inch notebook that has a 2.2 GHz quad core i7 processor. Did it myself.

Apple says this model only can be upgraded to 8 GB maximum RAM, but some research showed the real max is 16 GB. I haven't had any problems since installing the 16 GB RAM last week.

The SSD has 6 gigabit link speed, while the old hard disk has 3 gigabit. To preserve the life of the SSD, I turned on trim support and noatime, turned off local Time Machine snapshots, turned off sleep mode memory backup to persistent storage, and disabled sleep disk image.

Speedup results, without doing anything with temp folders and ramdisk:
  • OS X startup time: over 500% faster with SSD, a significant difference.
  • calibre startup time: faster by approx 1 second, approx 5%, with calibre app, libraries, and config dir all on SSD. Not as much speedup as I expected.
  • bulk conversion time: shaved 1 or 2 seconds off each conversion during bulk conversion operation, approx 5% faster on SSD. Not as much speedup as I expected.
Interesting.

I am not really surprised that calibre does not convert faster as it is pretty cpu intensive and you have lots of RAM.

My calibre startup time seems to be about 10 seconds (I have way too many books in there) with SATA HD and 8G RAM, so even 5% does not excite me.

Nice to see some actual numbers though.

Thanks

Helen
speakingtohe is offline   Reply With Quote
Old 10-22-2012, 05:42 PM   #22
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,038
Karma: 777825
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
On Windows I found that using a RAM disk for temporary files helped speed thins up. I presume the same an be done on MacOS ? It also has the additional benefit of reducing wear on the SSD.
itimpi is offline   Reply With Quote
Old 10-22-2012, 06:53 PM   #23
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,052
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad1, Kindle3KB
Quote:
Originally Posted by speakingtohe View Post
Interesting.

I am not really surprised that calibre does not convert faster as it is pretty cpu intensive and you have lots of RAM.

My calibre startup time seems to be about 10 seconds (I have way too many books in there) with SATA HD and 8G RAM, so even 5% does not excite me.

Nice to see some actual numbers though.

Thanks

Helen
Oops, I made some mistakes earlier so retested with variables more leveled, and revised my prior post with the corrections. OS X startup after shutdown is twice as fast, not five times as fast. And I'd previously tested with calibre startup restriction off, but on SSD with startup restriction on. I turned off the startup restriction, removed some recently added books from library that were added since testing with the hard disk drive, and got speedier results, over 25% faster without restriction than with restriction.

Last edited by unboggling; 10-22-2012 at 07:25 PM.
unboggling is offline   Reply With Quote
Old 10-22-2012, 07:19 PM   #24
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,052
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad1, Kindle3KB
Quote:
Originally Posted by itimpi View Post
On Windows I found that using a RAM disk for temporary files helped speed thins up. I presume the same an be done on MacOS ? It also has the additional benefit of reducing wear on the SSD.
Regarding temp files and RAMdisk, I'm still researching and testing exactly which temp folders would be appropriate. Apparently on OS X, not all temp folders should be put on RAMdisk, such as /var/db, while /private/tmp and /var/run are okay on RAMdisk. So I'm still trying to determine which specific temp folders appropriate for RAMdisk would speed up various specific calibre operations. I noticed that on OS X, conversion uses /var/folders. Does anyone know all the other specific temp folders that various calibre operations use on OS X, or where I could look to determine that?

What size RAMdisk do you use?

Last edited by unboggling; 10-22-2012 at 07:28 PM.
unboggling is offline   Reply With Quote
Old 10-22-2012, 10:39 PM   #25
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,617
Karma: 4998447
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
calibre uses only /tmp

On OS X, /tmp is usually a symlink pointing elsewhere.
kovidgoyal is offline   Reply With Quote
Old 10-23-2012, 02:13 AM   #26
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,052
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad1, Kindle3KB
Quote:
Originally Posted by kovidgoyal View Post
calibre uses only /tmp

On OS X, /tmp is usually a symlink pointing elsewhere.
Thanks. So if I understand correctly how symlinks work, that means I still don't know which temp folders to mount as ramdisks. I've done some testing, trying to adapt the script in the spoiler to mount relevant temp folders each to its own ramdisk at startup. Script is from a blog article (which took it from GitHub):

http://blog.alutam.com/2012/04/01/op.../#time-machine

https://raw.github.com/gist/931579/1...disk_MacOSX.sh

It works but doesn't speed up calibre startup or bulk conversions. Tried various ramdisk sizes. I added a line for another ramdisk for /var/folders which is where calibre conversions seem to happen on OS X. /var/folders actually mounts on ramdisk as /private/var/folders. But that didn't speed up bulk conversions either. So I commented out my extra added line.

Obviously I'm missing or misunderstanding something. I'm new to command line and symlink. Is there a way with command line to figure out exactly which folders the symlink /tmp points to?

Spoiler:

Code:
#!/bin/bash

# +----------------------------------------------------------------------+
# |                                                                      |
# |  Set up Mac OS X to store temporary files in RAM rather than on disk.|
# |                                                                      |
# |  By Philipp Klaus <http://blog.philippklaus.de>                      |
# |                                                                      |
# |  Originally by Ricardo Gameiro <http://blogs.nullvision.com/?p=357>  |
# |  Changes by Daniel Jenkins                                           |
# |     <http://blogs.nullvision.com/?p=357#comment-1140>                |
# |                                                                      |
# +----------------------------------------------------------------------+

cd /System/Library/StartupItems
sudo mkdir RamFS
sudo chown -R root:wheel RamFS
sudo chmod -R u+rwX,g+rX,o+rX RamFS
cat << "EOF" | sudo tee RamFS/RamFS > /dev/null
#!/bin/sh
# Create a RAM disk with same perms as mountpoint

RAMDisk() {
    mntpt=$1
    rdsize=$(($2*1024*1024/512))
    echo "Creating RamFS for $mntpt"
    # Create the RAM disk.
    dev=`hdik -drivekey system-image=yes -nomount ram://$rdsize`
    # Successfull creation...
    if [ $? -eq 0 ] ; then
        # Create HFS on the RAM volume.
        newfs_hfs $dev
        # Store permissions from old mount point.
        eval `/usr/bin/stat -s $mntpt`
        # Mount the RAM disk to the target mount point.
        mount -t hfs -o union -o nobrowse $dev $mntpt
        # Restore permissions like they were on old volume.
        chown $st_uid:$st_gid $mntpt
        chmod $st_mode $mntpt
    fi
}

# Test for arguments.
if [ -z $1 ]; then
    echo "Usage: $0 [start|stop|restart] "
    exit 1
fi

# Source the common setup functions for startup scripts
test -r /etc/rc.common || exit 1 
. /etc/rc.common

StartService () {
    ConsoleMessage "Starting RamFS disks..."
    RAMDisk /private/tmp 1024
    RAMDisk /var/run 128
    # RAMDisk /var/folders added by unboggling. 
    # RAMDisk /var/folders 1024
    # RAMDisk /var/db 1024
    # mkdir -m 1777 /var/db/mds
}
StopService () {
    ConsoleMessage "Stopping RamFS disks, nothing will be done here..."
    # diskutil unmount /private/tmp /private/var/run
    # diskutil unmount /private/var/run
}

RestartService () {
    ConsoleMessage "Restarting RamFS disks, nothing will be done here..."
}

RunService "$1"
EOF
sudo chmod u+x,g+x,o+x RamFS/RamFS


cat << EOF | sudo tee RamFS/StartupParameters.plist > /dev/null
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist SYSTEM "file://localhost/System/Library/DTDs/PropertyList.dtd">
<plist version="0.9">
    <dict>
        <key>Description</key>
        <string>RamFS Disks Manager</string>
        <key>OrderPreference</key>
        <string>Early</string>
        <key>Provides</key>
        <array>
                <string>RamFS</string>
        </array>
        <key>Uses</key>
        <array>
                <string>Disks</string>
        </array>
    </dict>
</plist>
EOF

Last edited by unboggling; 10-23-2012 at 02:19 AM.
unboggling is offline   Reply With Quote
Old 10-23-2012, 02:20 AM   #27
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,617
Karma: 4998447
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I'm not an OS X user so I can't help. Basically you need to find out where /tmp points to and ensure that location is mounted in RAM.
kovidgoyal is offline   Reply With Quote
Old 10-23-2012, 02:21 AM   #28
cybmole
Wizard
cybmole ought to be getting tired of karma fortunes by now.cybmole ought to be getting tired of karma fortunes by now.cybmole ought to be getting tired of karma fortunes by now.cybmole ought to be getting tired of karma fortunes by now.cybmole ought to be getting tired of karma fortunes by now.cybmole ought to be getting tired of karma fortunes by now.cybmole ought to be getting tired of karma fortunes by now.cybmole ought to be getting tired of karma fortunes by now.cybmole ought to be getting tired of karma fortunes by now.cybmole ought to be getting tired of karma fortunes by now.cybmole ought to be getting tired of karma fortunes by now.
 
Posts: 2,750
Karma: 1089170
Join Date: Sep 2010
Device: Kobo aura HD, Kobo Arc, Kindle Fire HDX 8.9 , Kindle for PC
sheesh - how often do you guys NEED to re-convert your books ?

what do you plan to do with the saved seconds , apart from contribute even more to this thread
cybmole is offline   Reply With Quote
Old 10-23-2012, 02:41 AM   #29
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,052
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad1, Kindle3KB
Quote:
Originally Posted by kovidgoyal View Post
I'm not an OS X user so I can't help. Basically you need to find out where /tmp points to and ensure that location is mounted in RAM.
Thanks.

I guess I need to buy a book with discussion of temp files, symlinks, and ramdisks on OS X Mountain Lion.

For now I'm not using ramdisk until I know what I'm doing.

Last edited by unboggling; 10-23-2012 at 02:49 AM.
unboggling is offline   Reply With Quote
Old 10-23-2012, 02:43 AM   #30
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,052
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad1, Kindle3KB
Quote:
Originally Posted by cybmole View Post
sheesh - how often do you guys NEED to re-convert your books ?

what do you plan to do with the saved seconds , apart from contribute even more to this thread
I convert all books that I add to calibre. But I'm trying to speed up conversions with temp files on ramdisks mainly out of curiosity, not necessity.
unboggling is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
How to speedup page turn on sideloaded epub on Kobo Touch with Firmware 2.0 MisterMax Kobo Reader 40 12-06-2012 05:31 AM
Patch: Calibre adds tags to identify ebook formats created by calibre. siebert Calibre 1 07-18-2011 02:07 PM
Speedup Metadata download libertytrooper Library Management 4 07-04-2011 09:40 AM
iLiad Some speedup ideas.. henris42 iRex Developer's Corner 2 05-28-2008 07:59 AM
Processor Speedup? sirmaru Sony Reader Dev Corner 4 01-05-2008 06:23 PM


All times are GMT -4. The time now is 07:07 AM.


MobileRead.com is a privately owned, operated and funded community.