Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 06-28-2010, 08:16 PM   #16
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,747
Karma: 10215946
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I use a heavily modified environment to build the installer (the build is fully automated and runs in a VM controlled from a linux box) which means it's a little hard to replicate.

The command is

python setup.py win32_freeze

but you will have to jump through a gazillion hoops to get the environment setup right (calibre depends on a ton of third party libraries)
kovidgoyal is offline   Reply With Quote
Old 06-28-2010, 11:34 PM   #17
toddos
Guru
toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.
 
toddos's Avatar
 
Posts: 695
Karma: 822675
Join Date: May 2010
Device: Kobo Aura, Nokia Lumia 920 (Freda)
Ah, it seems CostFinalize is still very costly, spending about 30 seconds or so processing all of the components. Turning off file costing does appear to be useful, but that just stops costing for flies while CostFinalize will still do costing for components. I think the issue now is that you have a one-to-one mapping of components to files. That's extreme overkill, and I think what's causing CostFinalize to run for so long. If you could instead put all files in a single directory into a single component you should see a reduction in CostFinalize time.

I'm not very familiar with python, but in psuedocode I'd want to change the process_dir() function to do something like:

Code:
For each item in a directory listing:
    if item is a directory:
        Add <Directory /> node
        Recurse on process_dir()
    else:
        If no component node for this current directory exists:
            Create <Component /> node with a unique name (probably need to keep a self._comp_id similar to the self._file_id that's already there)

        Add <File /> node to this directory's <Component />

If a <Component> node was created, add its appropriate </Component> closing node.
Append component node to the output of the subdirectory recursion to get an XML chunk with <Directory /> nodes before the singular <Component /> node.
This should minimize the number of components to a maximum of the number of directories (~400 items, probably less if there are a lot of dirs without files) vs. the number of files (~3750). An order of magnitude in the number of components should significantly reduce costing.

Edit: If you do decide to take out file costing (reducing the components may reduce costing enough that removing file costing doesn't gain anything), you can suppress the warnings by adding -sw1008 to the light.exe command line in wix.py.

Last edited by toddos; 06-28-2010 at 11:37 PM.
toddos is offline   Reply With Quote
Advert
Old 06-29-2010, 01:49 AM   #18
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,747
Karma: 10215946
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Will changing the component structure cause problems with upgrades?
kovidgoyal is offline   Reply With Quote
Old 06-29-2010, 05:09 AM   #19
toddos
Guru
toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.
 
toddos's Avatar
 
Posts: 695
Karma: 822675
Join Date: May 2010
Device: Kobo Aura, Nokia Lumia 920 (Freda)
Quote:
Originally Posted by kovidgoyal View Post
Will changing the component structure cause problems with upgrades?
Yeah, it probably will. Never mind about splitting up the files into components. You'd have to generate a new guid for each (the "*" guid only works when a component has a single file), you'd have to make sure those guids stayed the same across builds, etc. So a non-starter unless you wanted to manually maintain the 3700 or so files in the installer

The more I look into this, the more it appears that you were right all along and there's not a whole that can be done . The way the Wix UI bits are set up, costing has to complete before you can do anything really useful. I can understand why they went with this approach (costing has to be done before you can show disk usage or feature lists to include in the install), but since Calibre doesn't expose features for users to enable/disable it gets in the way.

One thought is to remove costing from the UI sequence entirely (leaving it in the InstallExecuteSequence). I don't know what impact that would have on the ability to change directories. I'm going to build me a sample MSI and play around with what happens if I turn off UI costing while still trying to use the WIX UI extensions.
toddos is offline   Reply With Quote
Old 06-29-2010, 10:53 AM   #20
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,747
Karma: 10215946
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
IIRC the installer does let you choose some components like whether to put the icon on the desktop or install the start menu icons if you click Advanced.

The only thing that occurs to me is whether costing can be completely disabled and instead values for the components can be supplied manually at installer build time.

The other option is of course to put all the files into a zip file, but I dont really want to do that as it makes it difficult for people to monkey patch the install, something that is often used by calibre developers to get people to test small changes easily.
kovidgoyal is offline   Reply With Quote
Advert
Old 06-29-2010, 05:38 PM   #21
toddos
Guru
toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.
 
toddos's Avatar
 
Posts: 695
Karma: 822675
Join Date: May 2010
Device: Kobo Aura, Nokia Lumia 920 (Freda)
Quote:
Originally Posted by kovidgoyal View Post
IIRC the installer does let you choose some components like whether to put the icon on the desktop or install the start menu icons if you click Advanced.
I don't see that when I hit the Advanced button. Just the option to choose a directory.

Quote:
Originally Posted by kovidgoyal View Post
The only thing that occurs to me is whether costing can be completely disabled and instead values for the components can be supplied manually at installer build time.
Unfortunately there's no way for you to feed costing data to the installer at build time, since it has to know information about the installation state of the machine (what files/components already exist, what versions, etc).

Quote:
Originally Posted by kovidgoyal View Post
The other option is of course to put all the files into a zip file, but I dont really want to do that as it makes it difficult for people to monkey patch the install, something that is often used by calibre developers to get people to test small changes easily.
Yeah, that's not a very good option.

I played around with creating my own simple installer last night to see if I could suppress all of costing during the UI sequence. I ran into a problem where even if I suppress CostFinalize in the wix file a CostFinalize step is still added to InstallUISequence. This causes the MSI to immediately fail because CostInitialize is not in the sequence. If I manually remove CostFinalize from the MSI, it does work and seems to install correctly as long as you don't present any feature selection or disk space usage to the user. Unfortunately manually editing the MSI after generating it with Wix is a pain in the rear and not something I'd expect you to do (nor should you).

At this point, short of using something other than WIX I think we'll just have to live with the installer being annoying.
toddos is offline   Reply With Quote
Old 06-29-2010, 05:46 PM   #22
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,747
Karma: 10215946
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Should I leave the suppression of FileCost in?
kovidgoyal is offline   Reply With Quote
Old 06-29-2010, 05:50 PM   #23
chaley
CC Android & calibre dev
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 8,800
Karma: 2692874
Join Date: Jan 2010
Location: France
Device: Many android devices
Out of curiosity, is it faster to uninstall then reinstall than to upgrade? The commentary indicates that it is. If so, could the install script go that route (force uninstall)?
chaley is offline   Reply With Quote
Old 06-29-2010, 05:53 PM   #24
toddos
Guru
toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.
 
toddos's Avatar
 
Posts: 695
Karma: 822675
Join Date: May 2010
Device: Kobo Aura, Nokia Lumia 920 (Freda)
Quote:
Originally Posted by kovidgoyal View Post
Should I leave the suppression of FileCost in?
If it gives any improvement and doesn't break anything, it shouldn't hurt to keep suppressing it. I'd only suppress it in InstallUISequence, though. Once the installer is at the progress bar dialog, users are fine with that taking more time. It's just the startup behavior where it looks like it's not doing anything that's annoying.
toddos is offline   Reply With Quote
Old 06-29-2010, 06:07 PM   #25
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,747
Karma: 10215946
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
OK I'll leave it in InstallUISequence. Hopefully it wont break anything.

@chaley: Uninstalling is pretty slow too.

Another thought I had is that is there a way to tell MSI to always copy the new file even if there has been no change, that should effectively reduce the timing to that of a new install.
kovidgoyal is offline   Reply With Quote
Old 07-02-2010, 01:53 AM   #26
kgn
Groupie
kgn has learned how to buy an e-book online
 
Posts: 169
Karma: 94
Join Date: May 2010
Location: Brisbane, Australia
Device: Pocketbook 302
Quote:
Originally Posted by toddos View Post
Why not? MSI-based installers are as customizable as you want them to be, with an open-source framework for building them (WIX). They get you nice standard behaviors (install/repair/upgrade/uninstall) easily integrated into the Add/Remove Programs control panel, and are fully supported by Microsoft. Sure, they can be arcane but that's where 3rd parties often step in and try to make things nicer.
Your comments and this thread are very clear examples of why I would never use an MSI based installer. I could use your term 'arcane' - but I would rather use such words as 'bloated', 'over complicated', 'poorly designed', 'slow', etc.

Another big problem with MSI installers (or perhaps it is just Windows) is that an Uninstall does not do a full uninstall.

I would rather installers such as Wise (a bit dated now) or Setup Factory. These are far more flexible, simpler to use, and generally just work.
kgn is offline   Reply With Quote
Old 01-21-2011, 11:05 PM   #27
Arachne
Junior Member
Arachne began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Dec 2010
Location: New Zealand
Device: Nook Color
[QUOTE=toddos;985070]I don't see that [option to not create a desktop icon] when I hit the Advanced button. Just the option to choose a directory.

I love that Kovid is pushing out small incremental updates frequently. However, for goodness' sake, can you PLEASE remove the desktop icon creation step from the installer? I don't want it, I can't turn it off in the installer, and it's a minor inconvenience every few days when a new version gets released. Thanks for considering this.
Arachne is offline   Reply With Quote
Old 01-22-2011, 01:07 AM   #28
Lady Fitzgerald
Wizard
Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.
 
Lady Fitzgerald's Avatar
 
Posts: 2,013
Karma: 251649
Join Date: Apr 2010
Location: Tempe, AZ, USA, Earth
Device: JetBook Lite (away from home) + 1 spare, 32" TV (at home)
[QUOTE=Arachne;1351433]
Quote:
Originally Posted by toddos View Post
I don't see that [option to not create a desktop icon] when I hit the Advanced button. Just the option to choose a directory.

I love that Kovid is pushing out small incremental updates frequently. However, for goodness' sake, can you PLEASE remove the desktop icon creation step from the installer? I don't want it, I can't turn it off in the installer, and it's a minor inconvenience every few days when a new version gets released. Thanks for considering this.
I agree it is an annoyance but it takes me all of 1 second to drag and drop it into the recycle bin.
Lady Fitzgerald is offline   Reply With Quote
Old 01-22-2011, 04:43 AM   #29
toddos
Guru
toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.
 
toddos's Avatar
 
Posts: 695
Karma: 822675
Join Date: May 2010
Device: Kobo Aura, Nokia Lumia 920 (Freda)
Or just turn it off during deployment, as was pointing out a long time ago in this necro thread.
toddos is offline   Reply With Quote
Old 01-23-2011, 11:40 AM   #30
solomon
Connoisseur
solomon began at the beginning.
 
Posts: 91
Karma: 10
Join Date: May 2010
Device: Win 10
Quote:
Originally Posted by toddos View Post
Or just turn it off during deployment, as was pointing out a long time ago in this necro thread.
Perhaps a compromise would be to remember the install options from last time and default to that. I typically go through the "Advanced" path and de-select it, but it would be nice if the default were the same as last time, so I'd only have to do that once!

As others have said, while I agree with the comment, this is certainly a nit!

Richard
solomon is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Installer for Reader shmish Sony Reader 6 09-01-2010 02:47 PM
New Inept Installer.... Voltare ePub 13 05-04-2010 09:28 PM
Installer partition Mike_73 Sony Reader 0 03-12-2010 04:56 PM
Windows Installer? radamo Calibre 4 12-27-2009 09:09 AM
6.17 installer not executable?? petercreasey Calibre 3 10-13-2009 06:28 PM


All times are GMT -4. The time now is 01:03 PM.


MobileRead.com is a privately owned, operated and funded community.