Changes to Installer???

[kovidgoyal]

I use a heavily modified environment to build the installer (the build is fully automated and runs in a VM controlled from a linux box) which means it's a little hard to replicate.

The command is

python setup.py win32_freeze

but you will have to jump through a gazillion hoops to get the environment setup right (calibre depends on a ton of third party libraries)

[toddos]

Ah, it seems CostFinalize is still very costly, spending about 30 seconds or so processing all of the components. Turning off file costing does appear to be useful, but that just stops costing for flies while CostFinalize will still do costing for components. I think the issue now is that you have a one-to-one mapping of components to files. That's extreme overkill, and I think what's causing CostFinalize to run for so long. If you could instead put all files in a single directory into a single component you should see a reduction in CostFinalize time.

I'm not very familiar with python, but in psuedocode I'd want to change the process_dir() function to do something like:

For each item in a directory listing:
    if item is a directory:
        Add <Directory /> node
        Recurse on process_dir()
    else:
        If no component node for this current directory exists:
            Create <Component /> node with a unique name (probably need to keep a self._comp_id similar to the self._file_id that's already there)

        Add <File /> node to this directory's <Component />

If a <Component> node was created, add its appropriate </Component> closing node.
Append component node to the output of the subdirectory recursion to get an XML chunk with <Directory /> nodes before the singular <Component /> node.

This should minimize the number of components to a maximum of the number of directories (~400 items, probably less if there are a lot of dirs without files) vs. the number of files (~3750). An order of magnitude in the number of components should significantly reduce costing.

Edit: If you do decide to take out file costing (reducing the components may reduce costing enough that removing file costing doesn't gain anything), you can suppress the warnings by adding -sw1008 to the light.exe command line in wix.py.

[kovidgoyal]

Will changing the component structure cause problems with upgrades?

[toddos]

Originally Posted by kovidgoyal

Will changing the component structure cause problems with upgrades?

Yeah, it probably will. Never mind about splitting up the files into components. You'd have to generate a new guid for each (the "*" guid only works when a component has a single file), you'd have to make sure those guids stayed the same across builds, etc. So a non-starter unless you wanted to manually maintain the 3700 or so files in the installer

The more I look into this, the more it appears that you were right all along and there's not a whole that can be done . The way the Wix UI bits are set up, costing has to complete before you can do anything really useful. I can understand why they went with this approach (costing has to be done before you can show disk usage or feature lists to include in the install), but since Calibre doesn't expose features for users to enable/disable it gets in the way.

One thought is to remove costing from the UI sequence entirely (leaving it in the InstallExecuteSequence). I don't know what impact that would have on the ability to change directories. I'm going to build me a sample MSI and play around with what happens if I turn off UI costing while still trying to use the WIX UI extensions.

[kovidgoyal]

IIRC the installer does let you choose some components like whether to put the icon on the desktop or install the start menu icons if you click Advanced.

The only thing that occurs to me is whether costing can be completely disabled and instead values for the components can be supplied manually at installer build time.

The other option is of course to put all the files into a zip file, but I dont really want to do that as it makes it difficult for people to monkey patch the install, something that is often used by calibre developers to get people to test small changes easily.

[toddos]

Originally Posted by kovidgoyal

IIRC the installer does let you choose some components like whether to put the icon on the desktop or install the start menu icons if you click Advanced.

I don't see that when I hit the Advanced button. Just the option to choose a directory.

Originally Posted by kovidgoyal

The only thing that occurs to me is whether costing can be completely disabled and instead values for the components can be supplied manually at installer build time.

Unfortunately there's no way for you to feed costing data to the installer at build time, since it has to know information about the installation state of the machine (what files/components already exist, what versions, etc).

Originally Posted by kovidgoyal

The other option is of course to put all the files into a zip file, but I dont really want to do that as it makes it difficult for people to monkey patch the install, something that is often used by calibre developers to get people to test small changes easily.

Yeah, that's not a very good option.

I played around with creating my own simple installer last night to see if I could suppress all of costing during the UI sequence. I ran into a problem where even if I suppress CostFinalize in the wix file a CostFinalize step is still added to InstallUISequence. This causes the MSI to immediately fail because CostInitialize is not in the sequence. If I manually remove CostFinalize from the MSI, it does work and seems to install correctly as long as you don't present any feature selection or disk space usage to the user. Unfortunately manually editing the MSI after generating it with Wix is a pain in the rear and not something I'd expect you to do (nor should you).

At this point, short of using something other than WIX I think we'll just have to live with the installer being annoying.

[kovidgoyal]

Should I leave the suppression of FileCost in?

[chaley]

Out of curiosity, is it faster to uninstall then reinstall than to upgrade? The commentary indicates that it is. If so, could the install script go that route (force uninstall)?

[toddos]

Originally Posted by kovidgoyal

Should I leave the suppression of FileCost in?

If it gives any improvement and doesn't break anything, it shouldn't hurt to keep suppressing it. I'd only suppress it in InstallUISequence, though. Once the installer is at the progress bar dialog, users are fine with that taking more time. It's just the startup behavior where it looks like it's not doing anything that's annoying.

[kovidgoyal]

OK I'll leave it in InstallUISequence. Hopefully it wont break anything.

@chaley: Uninstalling is pretty slow too.

Another thought I had is that is there a way to tell MSI to always copy the new file even if there has been no change, that should effectively reduce the timing to that of a new install.

[kgn]

Originally Posted by toddos

Why not? MSI-based installers are as customizable as you want them to be, with an open-source framework for building them (WIX). They get you nice standard behaviors (install/repair/upgrade/uninstall) easily integrated into the Add/Remove Programs control panel, and are fully supported by Microsoft. Sure, they can be arcane but that's where 3rd parties often step in and try to make things nicer.

Your comments and this thread are very clear examples of why I would never use an MSI based installer. I could use your term 'arcane' - but I would rather use such words as 'bloated', 'over complicated', 'poorly designed', 'slow', etc.

Another big problem with MSI installers (or perhaps it is just Windows) is that an Uninstall does not do a full uninstall.

I would rather installers such as Wise (a bit dated now) or Setup Factory. These are far more flexible, simpler to use, and generally just work.

[Arachne]

[QUOTE=toddos;985070]I don't see that [option to not create a desktop icon] when I hit the Advanced button. Just the option to choose a directory.

I love that Kovid is pushing out small incremental updates frequently. However, for goodness' sake, can you PLEASE remove the desktop icon creation step from the installer? I don't want it, I can't turn it off in the installer, and it's a minor inconvenience every few days when a new version gets released. Thanks for considering this.

[Lady Fitzgerald]

[QUOTE=Arachne;1351433]

Originally Posted by toddos

I don't see that [option to not create a desktop icon] when I hit the Advanced button. Just the option to choose a directory.

I love that Kovid is pushing out small incremental updates frequently. However, for goodness' sake, can you PLEASE remove the desktop icon creation step from the installer? I don't want it, I can't turn it off in the installer, and it's a minor inconvenience every few days when a new version gets released. Thanks for considering this.

I agree it is an annoyance but it takes me all of 1 second to drag and drop it into the recycle bin.

[toddos]

Or just turn it off during deployment, as was pointing out a long time ago in this necro thread.

[solomon]

Originally Posted by toddos

Or just turn it off during deployment, as was pointing out a long time ago in this necro thread.

Perhaps a compromise would be to remember the install options from last time and default to that. I typically go through the "Advanced" path and de-select it, but it would be nice if the default were the same as last time, so I'd only have to do that once!

As others have said, while I agree with the comment, this is certainly a nit!

Richard