Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 01-26-2015, 10:02 PM   #1
uriuri
Junior Member
uriuri began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Jan 2015
Device: Kobo Glo
Enhancement Guidance: Auto-update

Hey I've been using Calibre on and off for a while, and it always seems like there's an update every time I launch it (Kudos to Kovid and the active dev community).

I consider myself to be a capable developer and I would like to explore the possibility of adding auto-updates to Calibre. I haven't started digging through the code yet but I was wondering if this has been attempted in the past, and if anyone has any tips.

Thanks!
uriuri is offline   Reply With Quote
Old 01-26-2015, 10:31 PM   #2
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 11,624
Karma: 25094357
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
This has simply not been a priority. See: http://manual.calibre-ebook.com/faq....tomatic-update

The issue has come up several times I can think of, but only from people who want loudly expect it.

If someone else develops the feature, I believe Kovid would accept it. It should be incremental however, as noted in the FAQ.

Last edited by eschwartz; 01-26-2015 at 10:37 PM.
eschwartz is offline   Reply With Quote
 
Advertisement
Old 01-26-2015, 10:33 PM   #3
jgaiser
Omnivorous
jgaiser ought to be getting tired of karma fortunes by now.jgaiser ought to be getting tired of karma fortunes by now.jgaiser ought to be getting tired of karma fortunes by now.jgaiser ought to be getting tired of karma fortunes by now.jgaiser ought to be getting tired of karma fortunes by now.jgaiser ought to be getting tired of karma fortunes by now.jgaiser ought to be getting tired of karma fortunes by now.jgaiser ought to be getting tired of karma fortunes by now.jgaiser ought to be getting tired of karma fortunes by now.jgaiser ought to be getting tired of karma fortunes by now.jgaiser ought to be getting tired of karma fortunes by now.
 
jgaiser's Avatar
 
Posts: 3,114
Karma: 21944965
Join Date: Feb 2008
Location: Rural NW Oregon
Device: Kindle Fire HD, Kindle 3, KPW1
No. No. No. No automatic updates. I want to decide when to update.

And yes, this gets brought up from time to time and the answer is always no. There are scripts available if you feel a need.
jgaiser is offline   Reply With Quote
Old 01-26-2015, 11:28 PM   #4
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 4,844
Karma: 3752329
Join Date: Mar 2012
Location: NSW Australia
Device: none
Quote:
Originally Posted by uriuri View Post
Hey I've been using Calibre on and off for a while, and it always seems like there's an update every time I launch it (Kudos to Kovid and the active dev community).

I consider myself to be a capable developer and I would like to explore the possibility of adding auto-updates to Calibre. I haven't started digging through the code yet but I was wondering if this has been attempted in the past, and if anyone has any tips.

Thanks!
If you peruse the sticky threads here ==>> Related Tools you might find something to suit you.

but my needs are jgaiser's

BR

Last edited by BetterRed; 01-26-2015 at 11:34 PM.
BetterRed is offline   Reply With Quote
Old 01-26-2015, 11:53 PM   #5
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 11,624
Karma: 25094357
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
Quote:
Originally Posted by jgaiser View Post
No. No. No. No automatic updates. I want to decide when to update.

And yes, this gets brought up from time to time and the answer is always no. There are scripts available if you feel a need.
Seriously?
  1. It would be optional.
  2. The answer is "I have better things to do with my life" -- that does not preclude accepting a merge request.
  3. The previous times you allude to were whiny demands. This is the first time I am aware of that someone approaches from the other end and offers to contribute it.
  4. I helped author, and currently maintain, one of those scripts. I think I have the right to say, this way is better (if coded in a way that meets Kovid's approval).

Both of you -- I managed to answer all your concerns without dismissing the OP out of hand.
eschwartz is offline   Reply With Quote
Old 01-26-2015, 11:58 PM   #6
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 11,624
Karma: 25094357
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
I think the model of Firefox and Google Chrome is an excellent one. If every application out there could do the same, that would be awesome. Granted, most don't have the popularity that makes it worth the developer's time to implement.

If someone wishes to scratch that itch anyway, they should not be rejected out of hand.
eschwartz is offline   Reply With Quote
Old 01-27-2015, 12:29 AM   #7
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 4,844
Karma: 3752329
Join Date: Mar 2012
Location: NSW Australia
Device: none
When I am notified of new releases (mainly via RSS) I download whatever (including Firefox & Chrome) at a time that suits me, over a link that suits me. It goes into an install queue, which I process after taking a system checkpoint at a time that suits me.

Some applications, including Firefox, are first installed into a VM where I check them out for surprises. Been bitten too many times with things not working satisfactorily. I didn't install Firefox 34.0 because it had a bug that affected me, a few days later a fix release was issued, 34.05 I think.

I've been following that basic process for 30+ years for my personal systems and longer for work. It isn't going to change until up pushing up daisies.

BR
BetterRed is offline   Reply With Quote
Old 01-27-2015, 12:35 AM   #8
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 11,624
Karma: 25094357
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
And that makes you unusual, so surely you should realize that your perspective is not the most accurate guide to the acceptability of such a feature in a more general sense.

Witness: That feature exists in FF/Chrome, even though you don't use it.

Assuming the trend continues, you wouldn't use this either -- but tons of other people would.
eschwartz is offline   Reply With Quote
Old 01-27-2015, 12:54 AM   #9
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 27,113
Karma: 5606957
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Auto-updates are not something that can be implemented in a pull request. They require a lot of backup infrastructure, at a minimum, the release process has to be modified to maintain binary deltas of the various installers, and the distribution servers have to be modified to host those deltas.

That said if someone is willing to do the coding for the calibre side of the equation, I am willing to help with the rest.

What would be needed, is

1) A way to *reliably* download binary diffs, and use them to update installer images. This means hash checks of the patched installer images at a minimum and the ability to rollback if a patch fails.

2) A way to *atomically* update the calibre installation on launch, when an update is waiting. This is particularly important, a background update must *never* break a working calibre install.

3) A way to notify the user if auto-updating is failing repeatedly, so that the user can perform manual updates.

4) Auto-updates have to be optional, defaulting to off. That is necessary for calibre, since unlike Chrome and Firefox it does not have a large development and QA team to ensure releases are relatively bug free. I do a fairly good job with that on my lonesome ownsome, but I absolutely refuse to responsible for an on-by-default auto update process breaking working software on people's systems.

5) You need to figure out a way to have auto updates work for installations in system folders for which the normal user account does not have permissions, without prompting the user. Most applications get around this by running a root background updater service, which is a particularly ugly (security nightmare) and resource intensive solution, but is probably the only robust way.

Auto-updates are particularly problematic on windows, where the installer uses MSI and files are mandatorily locked by the OS. I have no idea how one would incrementally update an MSI based installation in the background.

So while, to the uninitiated it might seem like enabling auto-updates is relatively easy, doing so *reliably* and *robustly* is a whole other ball game.
I'd estimate it would take over a man month of work -- for only the initial implementation, assuming there were no nasty surprises along the way, and there are almost always nasty surprises. And then will come all the work to make it robust.

Last edited by kovidgoyal; 01-27-2015 at 12:57 AM.
kovidgoyal is offline   Reply With Quote
Old 01-27-2015, 01:16 AM   #10
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 4,844
Karma: 3752329
Join Date: Mar 2012
Location: NSW Australia
Device: none
Quote:
Originally Posted by eschwartz View Post
And that makes you unusual, so surely you should realize that your perspective is not the most accurate guide to the acceptability of such a feature in a more general sense.

Witness: That feature exists in FF/Chrome, even though you don't use it.

Assuming the trend continues, you wouldn't use this either -- but tons of other people would.
Multi user, metadata maintenance via the web browser, author link like fields on all categories, searching across physical libraries, full text search of book contents, add files of any type to a book via dropping it into book details... are all higher up my wish list - and a good many others I suspect.

There is also the issue that auto updating will probably push up the download traffic by a significant volume. I doubt Kovid's ongoing revenue would increase in proportion to the ongoing expense of the download traffic charges he would have to pay.

BR

@Kovid - please replace "I do a fairly good job with that..." with "I do an awesome job with that...", it will scan more better

Last edited by BetterRed; 01-27-2015 at 01:29 AM.
BetterRed is offline   Reply With Quote
Old 01-27-2015, 03:13 AM   #11
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,168
Karma: 781311
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Quote:
Originally Posted by BetterRed View Post
There is also the issue that auto updating will probably push up the download traffic by a significant volume. I doubt Kovid's ongoing revenue would increase in proportion to the ongoing expense of the download traffic charges he would have to pay.
If an auto-update process can be implemented as deltas then it is likely to significantly reduce traffic I would have thought.

If the default setting is off, then I would not be surprised if those who switch it to on are also likely to b those who download the full (new) release every week.
itimpi is offline   Reply With Quote
Old 01-27-2015, 10:47 AM   #12
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 11,624
Karma: 25094357
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
Quote:
Originally Posted by BetterRed View Post
Multi user, metadata maintenance via the web browser, author link like fields on all categories, searching across physical libraries, full text search of book contents, add files of any type to a book via dropping it into book details... are all higher up my wish list - and a good many others I suspect.
That is an entirely different point.

And I am sure Kovid will not delay release of those *tentative* enhancements... any more than they already are.
Perhaps @uriuri would like to work on those instead, but then again, perhaps not.

Where an item ranks on the average wishlist is an important qualification... to someone who isn't specifically interested in a specific feature. I wouldn't automatically assume that is not the case here.


Quote:
There is also the issue that auto updating will probably push up the download traffic by a significant volume. I doubt Kovid's ongoing revenue would increase in proportion to the ongoing expense of the download traffic charges he would have to pay.
That is why I recommended looking into incremental patching. As itimpi said, that should result in a net decrease in bandwidth costs... especially if it defaults to off.

Quote:
@Kovid - please replace "I do a fairly good job with that..." with "I do an awesome job with that...", it will scan more better
Agreed!
eschwartz is offline   Reply With Quote
Old 01-27-2015, 11:03 AM   #13
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 16,210
Karma: 6613876
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: Astak Pocket Pro, K4NT,Galaxy Tab 2
Lightbulb Tossing in an idea

Would it be possible to shift the built in News recipes to a system more like the get user PI notice uses?:

saving 5 M of base traffic (the Zip file) on every calibre installation

Even the full available 'list' would only be pulled (refreshes) only if the user has active subscriptions/uses the Get News icon

Individual news-source updates would be near instantly available, instead of needing to wait for the next release cycle

If those of us who do not us any news source, the News related PI traffic would approach zero
theducks is offline   Reply With Quote
Old 01-27-2015, 11:07 AM   #14
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 11,624
Karma: 25094357
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
I believe Kovid has said recipes are automatically updated when run -- I think it first looks at the up-to-date version hosted on the calibre website, before running the job.

I suspect a list of the current ones is needed, so calibre knows what is available without pinging the server first.
eschwartz is offline   Reply With Quote
Old 01-27-2015, 11:10 AM   #15
uriuri
Junior Member
uriuri began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Jan 2015
Device: Kobo Glo
Quote:
And I am sure Kovid will not delay release of those *tentative* enhancements... any more than they already are.

Perhaps @uriuri would like to work on those instead, but then again, perhaps not.
Is there a list of enhancements or "todos"? I wouldn't mind starting to work on something simpler.
uriuri is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Auto-update Calibre? Geremia Calibre 17 03-14-2014 02:24 AM
Auto update covers? mdamsma Library Management 4 01-29-2013 06:18 PM
Auto-update kapsi Calibre 6 07-24-2011 12:24 PM
Auto update Toxaris Sigil 1 11-17-2010 05:36 AM
Auto Update to Connect Xsavior Sony Reader 0 10-18-2007 03:07 PM


All times are GMT -4. The time now is 03:42 AM.


MobileRead.com is a privately owned, operated and funded community.