Automatically Keep Calibre Updated?

[BetterRed]

Originally Posted by GeekDrop

So, set it's default to "Off".

And as soon as someone set it to ON it would stay ON, otherwise there'd be no point in setting ON, hence Kovid's out of pocket costs would increase over time.

Have you considered running from source then you will only need to download what's changed and I think you can get the changes as soon as Kovid posts them to Github. The instructions to do that are at the back of the calibre user manual.

BR

[GeekDrop]

Originally Posted by BetterRed

And as soon as someone set it to ON it would stay ON, otherwise there'd be no point in setting ON, hence Kovid's out of pocket costs would increase over time.

He already hosts his downloads on Fosshub (free) and Github (free). I myself develop software and host files for download via my website, on Github, for free ...

Regarding the "On" setting; as you said, 'not everyone has a need to keep it always updated', therefore not everyone will turn it ON, leaving only a certain percentage, which automatically (and according to you, probably dramatically) knocks down those original numbers you calculated.

Have you considered running from source then you will only need to download what's changed and I think you can get the changes as soon as Kovid posts them to Github. The instructions to do that are at the back of the calibre user manual.

Well now we'd be right back to talking about conjuring up our own schemes to keep his software updated on our machines. It would take probably a couple hours at least to bake up a system to check Git for changes, if found download and compile, and update the system. All remotely and automatically.

[kovidgoyal]

Sorry I am not willing to rely on third party services for the ~100TB of bandwidth a month calibre needs even without automatic updates.

And if you think an automated update system that works *robustly* across three OSes can be coded up in a couple of hours, you obviously have never actually deployed any consumer grade software at scale.

[GeekDrop]

Originally Posted by kovidgoyal

And if you think an automated update system that works *robustly* across three OSes can be coded up in a couple of hours, you obviously have never actually deployed any consumer grade software at scale.

An automated update system could be as easy as automatically downloading the normal installer that one downloads from the webpage, and when the download is complete, closing Calibre and launching the installer with a c-line flag to install silently using the current installation path (which the current installer already is aware of). Thousands (or more) of programs do this.That's a hard thing to do across 3 OS's? Download a file and launch it?

You're paying out of pocket currently, for your fosshub and github file hosting?

btw, when I said "It would take probably a couple hours at least to bake up a system to check Git for changes, if found download and compile, and update the system. All remotely and automatically." I was talking about me specifically having to write up my own personal system, to keep Calibre updated on my own machine ...

[kovidgoyal]

that's not an automated update at least for any self respecting software developer. Download the normal installer and run, really? That's the best you can come up with? Forgot about delta updates, dealing with conflicts on OSes such as windows where resources in use can cause installations to require reboots. Permissions issues, updating software that is installed with system permissions from software running as a non-admin user is not as simple as download a file and run it. Debugging failed downloads/updates and providing support for them. Non-standard installations, for example binary installations, vs distro packages vs run from source setups. Hell even detecting that calibre is running in order to shut it down for an update is far from trivial, since calibre has about 20 different binaries, many of which can run concurrently with the main program.

And what part of "I dont want to rely on free services for calibre hosting" did you fail to comprehend? Did it ever occur to you that I might have relied on free services in the past as primary hosting and been burned by them? So no I am not paying out of pocket for fosshub and github, though I fail to see how that is relevant to anything.

[GeekDrop]

Originally Posted by kovidgoyal

that's not an automated update at least for any self respecting software developer. Download the normal installer and run, really? That's the best you can come up with? Forgot about delta updates, dealing with conflicts on OSes such as windows where resources in use can cause installations to require reboots. Permissions issues, updating software that is installed with system permissions from software running as a non-admin user is not as simple as download a file and run it. Debugging failed downloads/updates and providing support for them. Non-standard installations, for example binary installations, vs distro packages vs run from source setups. Hell even detecting that calibre is running in order to shut it down for an update is far from trivial, since calibre has about 20 different binaries, many of which can run concurrently with the main program.

lol, who knew the dev of Calibre was so hostile? Especially toward suggested software enhancements .. it sounds like you're way too stressed, maybe you need a break from developing for awhile until you lose the burnout.

I suppose 'self respecting developers' would just rather not even tackle such a common feature, and force people to a web page each time, to manually download and run the installer instead ... talk about high tech!

At least on Windows, simply dl'ing the installer in the background, and running it with a /silent flag would work just fine. I've seen it done a billion times. You know you have as well. If you prefer to make your life harder to 'be self respecting ' then that's your own issue.

Actually I'm well aware of all it takes to update software via code; I've been a software developer, including owning and developing for shareware company, since 1996. If you think the above list is anything out of the ordinary for a dev, I have to start questioning your actual coding skills that I had just assumed were pretty decent up until now.

And what part of "I dont want to rely on free services for calibre hosting" did you fail to comprehend? Did it ever occur to you that I might have relied on free services in the past as primary hosting and been burned by them? So no I am not paying out of pocket for fosshub and github, though I fail to see how that is relevant to anything.

Perhaps you're right on this one; that I'm 'failing to comprehend' ... looking at the current download page for Calibre, you have them linked to both FossHub, and GitHub ... which are both free services, no? Sooooo, you're currently relying on said free services for calibre hosting, no?

Hell, it even says it right on their front page 'in big lights' ..



You still fail to see how it's relevant?

Anyway, hey, if updating your own software programatically is beyond your skills, and prefer to blame it on a list of 'difficult' roadblocks, then I suppose we'll just be SOL and deal with it.

Can't wait to hear from all of the White Knights who will come to your defense! lol ...

[pazos]

Hi GeekDrop. Just a reminder, Calibre is free software, if you want to contribute you can. If your vision doesn't match with the main author you can fork it, provide the improvements you want, and giving it free to the world.

I'm not a developer myself, but tried to contribute on a few free software projects. It really hurts to see how easy is to throw negative critics on someone who spent years of his life providing good software for free. It hurts even more comming from a windows shareware developer, no pun intended.

[eschwartz]

Originally Posted by GeekDrop

lol, who knew the dev of Calibre was so hostile? Especially toward suggested software enhancements .. it sounds like you're way too stressed, maybe you need a break from developing for awhile until you lose the burnout.

I suppose 'self respecting developers' would just rather not even tackle such a common feature, and force people to a web page each time, to manually download and run the installer instead ... talk about high tech!

At least on Windows, simply dl'ing the installer in the background, and running it with a /silent flag would work just fine. I've seen it done a billion times. You know you have as well. If you prefer to make your life harder to 'be self respecting ' then that's your own issue.

Actually I'm well aware of all it takes to update software via code; I've been a software developer, including owning and developing for shareware company, since 1996. If you think the above list is anything out of the ordinary for a dev, I have to start questioning your actual coding skills that I had just assumed were pretty decent up until now.

What an inelegantly clumsy way to provide a builtin updater. If that was all that people needed, then why not use the existing methods that have been repeatedly pointed out which accomplish exactly that?

And of course the one place this doesn't work with any reliability is, yup, Windows.

(Linux has lots of excellent methods for auto-updating all software ever, starting with the use of distribution package managers.)

Perhaps you're right on this one; that I'm 'failing to comprehend' ... looking at the current download page for Calibre, you have them linked to both FossHub, and GitHub ... which are both free services, no? Sooooo, you're currently relying on said free services for calibre hosting, no?

Hell, it even says it right on their front page 'in big lights' ..

[removed big fat huge annoying bloated image]

You still fail to see how it's relevant?

It's pretty darn irrelevant. Both of those websites are secondary mirrors, with the primary downloads being served directly from his site. As he's repeatedly pointed out, it doesn't matter how free either one is, if it isn't reliable enough to guarantee continued availability no matter what then it's just a terrible idea to rely on something that may disappear.

And the primary download site was once sourceforge, wasn't it -- and look how well that turned out.

And IIRC Github has, in the past, stated they would reserve the discretion to throttle your release download bandwidth if they felt it was generating an unreasonably large amount of it -- something hooked into an automatic updater is more likely than anything else to trigger such measures. Although currently the only language I can find on their website is a 2GB per-file limitation, so maybe that changed (very predictable, yay!).

Meanwhile Fossahub is an ad-supported website and they don't actually permit you to download anything in a non-interactive fashion -- it is categorically impossible to build an auto-updater using Fossahub bandwidth. Their download links are dynamically generated and if you tried to build some scraper that fought against the site in order to trick it into yielding the correct download link, they would a) change the algorithm, and b) ban Kovid for violating their ToS.

Bottom line: no, there actually isn't any such thing as fairy tales like "hey, free bandwidth!" There are always conditions and caveats, and the most honest ones always boil down to "here's how many dollars it will cost you -- knock yourself out, and it's been a pleasure doing business with you".

Anyway, hey, if updating your own software programatically is beyond your skills, and prefer to blame it on a list of 'difficult' roadblocks, then I suppose we'll just be SOL and deal with it.

There is no need to impugn his skills. Be content with impugning his interest -- anyway I'm sure he will not be at all hurt by accusations that he totally doesn't care about implementing such a feature enough that he won't bother even trying.

Observations on the difficulty of doing so, are simply observations on "look at how much work it would take for me to do something silly I totally don't care about at all, why should I waste my time doing it for no benefit".

Why do you even need automatic updates -- calibre doesn't stop working just because it comes out with minor patchlevel releases every single week containing the latest rollup of minor new feature requests that only enhance functionality a minority of people use, bug fixes that only affect a small minority of users, translation string updates, and an updated cache of builtin news recipes.
If you are one of the unusual people who crave the new version, then a) you're quite dedicated, b) you can run one of the existing scripts as a weekly timer on your OS of choice, c) you can run from a git clone and update every five minutes by running git pull and only update the installer on that unusual occasion when the C extensions get updated. Most changes are just changes to the python code.

Can't wait to hear from all of the White Knights who will come to your defense! lol ...

Cool. Happy to help. If it helps your ego even more, you managed to get defended against by one of the very rare and impressive eschwartz sightings on my thrice-yearly visit to the site.

You now have the questionable honor of being rebutted by a *genuine* Linux neckbeard and distribution developer, even.

[stumped]

Originally Posted by BetterRed

Very few people need to download every new version, and there is a new version every couple of weeks. Calibre has ~3,000,000 active users, with ~25 releases per annum, at ~70MB each that is 5.25 billion Megabytes.

And who would pay for the network bandwidth at the server end? Kovid Goyal.

I'll leave you to calculate the cost he would have to bear.

BR

As a direct comparison, Plex media server updates every few weeks and they have that automated.
The Plex media player client also auto updates.
Dunno exactly how many active users but a big number.
And it's freeware, for most users, yet they are able to find auto updates somehow.
A small minority pay for plex pass but most do not. I do only because a cheap lifetime offer came around. All the features I like are in the free version

[BetterRed]

Originally Posted by stumped

As a direct comparison, Plex media server updates every few weeks and they have that automated.
The Plex media player client also auto updates.
Dunno exactly how many active users but a big number.
And it's freeware, for most users, yet they are able to find auto updates somehow.
A small minority pay for plex pass but most do not. I do only because a cheap lifetime offer came around. All the features I like are in the free version

If you want calibre to provide an auto-update option out-of-the-box, download the source, make the necessary changes, and submit them for inclusion in the next calibre release. Kovid has spelt out his reservations, so you should take them in account.

BR

[DoctorOhh]

Originally Posted by stumped

As a direct comparison, Plex media server updates every few weeks and they have that automated.

Let me veer off topic with you for a moment. I've running a Plex server over 6 years and calibre for over 10 years. In that time I have never had either software auto-update. Plex informs me when an update is available and I choose whether to update the server. When I do choose to update the server more times than not the update fails and I have to go to the site, download the install and run the process manually. The same as I do for calibre.

Back on topic, if anyone wants calibre to auto-update there are apparent solutions for Windows, Linux and Mac users in the sticky posts of the Related Tools forum.

I say apparent because I haven't tried them. I suspect none of them would work without me first pausing sync on Dropbox, since for my setup pausing sync in Dropbox is required prior to updating calibre. If I don't pause it the install will fail.