Howto recompile opensource firmware for KOBO reader

[fastrobot]

Originally Posted by PeterT

Just wondering if the OP has bothered getting in touch with George Talusan @ Kobo regarding the GPL'ed source issues? (He has an account here; george.talusan)

Just so you know, I wrote him since you posted this, and he still has not replied.
I let you know if he ever does, but since it has already been several days -- I doubt it.

[PeterT]

I've poked him....

[PeterT]

George would like you to contact him via GitHub.

[fastrobot]

OK.
I'm only vaguely familiar with github's system.
I had to create a different account name, as someone else took fastrobot; Although, I did find a kobo-labs web page, https://github.com/chillerpr34/https...bs-Kobo-Reader

But, there is no evidence of any activity on the page since Jun30'th 2013.
So the repository is roughly 2 years out of date, as there have clearly been many KOBO reader upgrades since that date.

I'm looking for a way to write George Talusan, and I don't see any obvious way of doing it.
The kobolabs page is registered to a "Chillerpr34", but all the links go to dead ends.

So -- How, exactly, does he want me to contact him through github ?

[PeterT]

I'd say go to https://github.com/kobolabs/Kobo-Reader/issues and open an issue there.

[fastrobot]

Well, Peter....

Look here:
https://github.com/kobolabs/Kobo-Reader/issues/53

George closed the thread, for he apparently doesn't want to answer the question I actually asked; but wants to be 'seen' to answer it when he was being superficial ( or careless ? )

Note, he referenced me to: https://github.com/kobolabs

And if you look on that page, today, you will not find a repository for AuraHD -- but only ARC.
I already downloaded the generic kobo-reader archive, and proved it's out of date.
So -- AFAICT individual download, must mean gnu 'packages' and not 'device' repositories.


So to be fair to George, as I mentioned busybox to him explicitly -- check the third page of the repository, which has busybox: https://github.com/kobolabs/busybox but doesn't tell you which version it is.

Now, notice -- the repository is two years old, and never updated since then exactly the same as the repository I already downloaded. So, that's no better than the repository I downloaded before --eg: it's also an old one; and there is no way to tell if it is or isn't the version that is on the KoboAura or Glo, and worse -- there are no Aura or Glo configure scripts -- so I don't know which parts of busybox are used in the kobo AuraHD and which ones are not. The Kobo is not using all the busybox code, as I already know from hard won experience that a telnet host is not compiled in KoboAURAHD firmware update 3.12.1 or 3.15.1 ; and there are some patches in the archive, so it's not pure busybox -- either.

So if I naively try to compile that repository for the KOBO -- it's going to introduce bugs or break dependencies ; and there is no guarantee it's even the version that's on the Aura in the first place, as it's not labeled as being part of the Aura. Nor is there a readme file saying which version of what is for which reader.

So -- George just kicked me back down to doing everything the hard way. He didn't give me a chance to ask him how to tell which version is used on which reader, and how it needs to be compiled to be compatible with the reader. ( The ToDO's, readmes, etc. are 4 to 6 years old, so they obviously aren't going to accurately reflect a firmware update/release from a month ago. )

It's an attack on the purpose of open source code on the Aura to hide things like this, because it makes it very difficult for users to replicate what is already there -- and then fix only what is broken (the bugs).

Notice: I mentioned the scripts issue, regarding QT and George didn't bother to comment on that, either.

I can't replicate what is there, unless I know what it is and I think George likely knows that and chose to close the thread prematurely; I expect he likely knows (or ought to know better) that it's superficial to show a link to a giant source tree when there is no labeling of version numbers, or text files, or databases, telling you what is and is not used on a particular version of a KOBO reader.

He might as well just point me at the gnu software site, and say "it's on there somewhere..."
What he's doing is extremely evasive, whether done on purpose or not -- it looks bad.

Besides: It's not like I'm asking for the compiler flags for nickel, or anything proprietary. Just the open source stuff.

git is meant to do versions of software, and revision control. It's not like Talusan doesn't have the correct tool for the job -- he's just not using it, or else there should be individual firmware releases indicators of some kind. I'll play with git and see if I can get it to list any hidden branches or anything, but it's strange that the website says there were no commits in two years if he's actually using git while doing firmware maintenance of busybox.

[fastrobot]

Originally Posted by eschwartz

Kobo is in violation of the GPL. Wow. So is everyone else, for example Amazon.

Sigh: That doesn't make it right,or legal.
If "everyone" runs red lights, would you too?
The truth is, not everyone is in violation of GPL.
And no one will change, unless people speak up about it -- and even then it might require that the FSF sues.

If Amazon is in violation, I hope someone talks to them -- and if they remain that way, in spite of conversation -- I hope they get what they deserve.

I'm the consumer, and I support FSF with $. So -- If I got screwed by companies like Amazon over software that I helped develop, it would tick me off.
If they are only doing little things that can be worked around, it's not so bad; but when it's blatant there is a point where legal remedies are in order.
It's just a matter of time, and courts have long memories when lots of money is involved.


Hopefully, a consensus can be achieved rather than a battle. But, that's seldom the case when people obey the letter of law only when enforced by threat, and violate it's spirit continuously.

[PeterT]

Well when I look at the makefile at KoboLabs in the busybox directory I see version 1 patch level 22 extraversion kobo which would appear to match the eye catcher from the run you did of busybox.

I do wonder what issues you hope to fix seeing I expect most issues are in the non open source portions of the device ie the main application nickel?

[eschwartz]

Originally Posted by fastrobot

Sigh: That doesn't make it right,or legal.
If "everyone" runs red lights, would you too?
The truth is, not everyone is in violation of GPL.
And no one will change, unless people speak up about it -- and even then it might require that the FSF sues.

If Amazon is in violation, I hope someone talks to them -- and if they remain that way, in spite of conversation -- I hope they get what they deserve.

I'm the consumer, and I support FSF with $. So -- If I got screwed by companies like Amazon over software that I helped develop, it would tick me off.
If they are only doing little things that can be worked around, it's not so bad; but when it's blatant there is a point where legal remedies are in order.
It's just a matter of time, and courts have long memories when lots of money is involved.


Hopefully, a consensus can be achieved rather than a battle. But, that's seldom the case when people obey the letter of law only when enforced by threat, and violate it's spirit continuously.

Granted. And I do agree that we should have access to those sources.

I am merely pointing out that:

• It shouldn't really come as a surprise. There is a pre-existing trend of such companies only offering token obedience to the GPL (if that) and offering sources that are outdated and/or missing vital components -- if anything.
• Asking nicely is not likely to be very effective. Companies get asked all the time, and they have a lousy track record for answering.
• Lawsuits might help. It seems to be the only thing giant multinationals listen to.

Consensus will never be achieved -- because it contradicts the business model of said companies.


The fight between FLOSS/FSF/GPL and giant multinationals is hardly new, or news

[fastrobot]

Originally Posted by PeterT

Well when I look at the makefile at KoboLabs in the busybox directory I see version 1 patch level 22 extraversion kobo which would appear to match the eye catcher from the run you did of busybox.

I do wonder what issues you hope to fix seeing I expect most issues are in the non open source portions of the device ie the main application nickel?

Cool! So the Makefile does have a reference to the version! and this busybox is more likely the one used by the KoboGlo, and AuraHD, after all, then. Thanks!

The makefile in Gnu software is usually created by a configure / confiugre.in script, and the M4 macro system ; so I didn't think about checking the makefile contents... it often doesn't have a readable version number in it. I guess in busybox, it does!


As to bug fixes, all kinds of them.... some of which affect the native kobo apps, some of which don't. ( Most of the things which affect native kobo apps are bugs in the linux kernel and drivers, or just failing to use the hardware that is available in the lowest power mode possible... )

For a few examples of things that irritate me because I'm trying to do maintenance and develop some Kobo apps, eg: the first thing I would like to work RIGHT in busybox -- is the basic unix shell, eg: ash.

Ash should be able to receive 'control C' from the USBpty, and interrupt a program that is being run over telnet; but for some reason the default setting when I log in is that control C is disabled. So when I work on the kobo, I can't break a running process without logging into another telnet session and running 'kill' remotely. I figured out that I can run "stty sane" to enable control C -- if I forget to do that -- then it's a pain... and startup scripts don't appear to be read if I don't login as superuser.

eg: There should be a telnet or ssh server that allows me to login to ASH as non superuser, and still works.

Another example, I have written a very small vt52u (unicode) terminal emulator that runs on an e-paper display. It knows about update ioctls, and can efficiently queue terminal screen updates so as to not have the user forced to wait for redrawing, and supports type-ahead and power conservation. (I'm working on figuring out the KOBO's dynamic clocking system, because I want to reduce power usage even more -- but that's tricky.. ) The terminal is fully supported by ncurses, plays nice with vim, readline, and even has a basic stroke recognition input software that lets me emulate a keyboard with the touch-screen via. gestures, so that I don't need a graphics keyboard that has to be redrawn for each button press, or mode change, and which isn't missing a lot of key combinations that I can normally enter (like control C) on a real keyboard.

The terminal emulator was originally for my sony reader, prs-600, and allows me to do text editing for well over 12 hours continuously; and with a very little effort could run for days on a Kobo once I figure out the power conservation modes of the Kobo.

It would be nice to make an app out of the vt52u terminal so that it can run as a 'beta' feature for the kobo under nickel; eg: a system terminal not requiring X11; but even if nickel is too broken to let me develop my terminal as an app, I would still like to be able to use it stand alone to be able to take notes and edit files, and run python from the command line. ( Python, and frobtads, are apps I really like to use, and share with my kids, along with either pygame which has to be modified to run on a framebuffer ; or pyQT -- which could be interfaced to QTanywhere library already on the KOBO. ).

But, several of the tools in busybox are not compiled to play nicely with terminals in general. The "ls" command, for example, is hardwired to use ANSI escape sequences which are not supported by several kinds of terminals; and does not accept termcap definitions. There should be a way to configure ASH by a startup script to never use color; but that's also broken in various ways. eg: busybox assumes a color linux vt-102 terminal by default even though an e-paper display is presently, by definition, only black and white, so that the default doesn't make sense.

Oh, and there was one other thing I already know about -- though I forget exactly which expressions were broken -- but the built in grep does not parse all regular expressions correctly. Certain pattern combinations cause incorrect results in the output.

These kinds of bugs aren't serious because they can be worked around, but they do exist and I'm sure there are many more I simply haven't found -- but I think as a user of open software, that I should verify that I actually have the tools necessary to fix bugs and that they are ready to be used.

[george.talusan]

Lot of noise here about nothing.

The git repositories that are hosted on github are kept up-to-date. Just because you see that something hasn't been updated since 2013 or whatever doesn't mean it isn't the latest. It is in fact the latest.

The problem that I see here is that there's a request for source code, the request is fulfilled, then there is a trust issue regarding whether or not this source code is the latest.

If the build instructions are followed from the Kobo-Reader repository using the individual git repos, you should be able to build up your own chroot. The main reason for the big split is because github doesn't allow binaries greater than 100MB and it's easier to just track upstream repositories if we use git as intended.

In other words, just clone and compile.

UPDATE: Also if there's anything missing, then file an issue on github instead of complaining here.

[fastrobot]

Originally Posted by davidfor

No WiFi? All the setup can be done over WiFi most of the devices. The Touch was the last that couldn't out of the box, but if it has newer firmware on it, it can.

Hi David.
I figured out what's going on.

Wait until one of your KOBO's gets out of sync firmware wise. eg: an old version of firmware.

Then take it to a coffee shop, or someplace commercial, which has a virus firewall on the wireless router like most commercial/company installations do in the USA to protect themselves from being sued by customers. ( Eg: Dan's guardian is fairly common around here where I live. ).

What happens is that the Kobo checks the store, realizes that the firmware is old -- and tries to do an upgrade; the upgrade fails -- and the system, rather than allowing you to log in using the old firmware, wants to restart and install firmware that it failed to download. This is especially true of a KOBO that has no books on it at all, and is factory reset ; as once you have books, and have read the tutorial book -- the Kobo acts differently.

Of course, after restarting -- it checks for firmware upgrades, and finds that an upgrade is required... attempts to download it and the cycle begins all over again.

I'm not sure exactly how the KOBO wireless upgrades work, but the cycle I just described is what I think is going on; there appears to be two reboots/restarts sometimes when I upgrade without a firewall involved, so there may be other failure modes that are more complicated than I just described because it might be possible that *part* of the upgrade happens when a firewall is present, but not the whole upgrade or it might be possible that the virus firewall's are inconsistent in what they block.

A wireless firewall is not supposed to stop text, and internet pages from downloading -- so that explains why text such as ebooks, and epubs, download just fine but a program which could have a virus, such as an unprotected firmware download -- does not work.

Note: When I upgraded the firmware manually, that allowed me to use the KOBO store without incident from then on; so -- I'm thinking the issue is that the firmware upgrade comes in a format that is incompatible with standard firewall software. eg: All the problems go away as soon as I install a wireless access point that has no virus protection (a bad idea, but necessary). Therefore, I suspect that KOBO's aren't smart enough (yet) to figure out when a standard firewall is interfering with upgrades.

It would be nice if KOBO had a message warning the user when a firewall was causing problems.
Lots of users don't have control over their internet access, especially those who go on vacation and rely on coffee shops as places to download books.

Also, as I mentioned before -- Kobo's don't appear to support ad-hoc mode as of May 2015, so merely buying a USB dongle to work in ad-hoc mode isn't good enough to test the problem out with; I had to buy an actual wireless access point and learn how to configure it, in order to even run a test.

Most non-technical people I know of would have just thrown the KOBO out.

If you can't replicate the problem, I'd be willing to film an upgrade and post it on YouTube to show exactly what it was doing to me; Let me know.

[eschwartz]

Uh, most people don't have router firewalls... and how is Kobo supposed to know when you do?

That being said, Amazon has wisely avoided the issue by not demanding frivolous checks for Kindle firmware updates. When an update is available it will try to auto-update, but that is entirely independent of the other firmware components. Logging in on old firmware works just fine -- as it should.

[fastrobot]

Originally Posted by george.talusan

Lot of noise here about nothing.

George, let me point out something obvious.

If anyone, here, at this site had known for sure that the source code to the latest release of the KoboGlo or Aura, was in that upper directory -- they would have pointed out where it was, or made fun of me for not looking.

The very fact that a thread has to go this far before we can even guess where the right source code is -- shows there is a problem.

The problem that I see here is that there's a request for source code, the request is fulfilled, then there is a trust issue regarding whether or not this source code is the latest.

Yes, there is a trust issue for an obvious reason; I already downloaded a two year old archive that wasn't labeled as archive only; and didn't realize it was the wrong one until I compiled the code and checked the version numbers and noticed it was wrong. There's nothing which says which archive is up to date for the Glo or Aura; and your comment was merely "you could look" ... it wasn't "the most recent code for busybox is here"

There's a saying -- once burnt, twice shy. Well ... I've been burnt more than twice with Kobo support.

In the new Busybox directory that I did find, there is a ALSO a note saying "of course you can always download the latest source from ..."; which is a generic pointer to the busybox respository for the developers of Busybox. So it was not possible for me to tell if the latest version of busybox used on the Aura or Glo, was even in the git-hub repository, or if the git hub version was only there because you had patched that particular version historically. There's no documentation of the reason for a particular directory existing or not existing or whether I should use a newer version from busybox, with an old configuration file from your archive -- or what.

Most companies working with source code document at least a tiny bit better, for example:
If you go to packages.slackware.com, they use a system that has older releases, slackware 12, 13, 13.1, etc. and a separate directory labeled "current"; which at least gives the user a clue between historical archives, and present day working directory.

It's pretty counter-intuitive to look at an archive which is two years old and go -- oh yeah, that must be the one he used for the release that came out last month; unless there is a note of some kind.

If the build instructions are followed from the Kobo-Reader repository using the individual git repos, you should be able to build up your own chroot. The main reason for the big split is because github doesn't allow binaries greater than 100MB and it's easier to just track upstream repositories if we use git as intended.

In other words, just clone and compile.

UPDATE: Also if there's anything missing, then file an issue on github instead of complaining here.

I have no intention of building a change root.
I intend on fixing the code that the Aura uses natively.

Of course, I will post to git hub as I see problems in the future; and hopefully you won't close the thread so quickly that I can't fully ask the question? I mean, if I have a problem -- and no-one else, here could answer it -- then it's clear that many people have a problem knowing how to compile your archives; and you'll just end up answering the same question multiple times if you cut the answer short -- with unhappy customers.

I can't read minds, and I know that you know how you did it, and why -- and I'm sure it's all very reasonable in your own mind; but the rest of us are guessing.

I want to continue posting the results, here -- or, at least links to the Github comments, for since you don't have individual archives for all the different readers, it seems that posting the successful recompilation steps for the Arua and Glo will aid the public in general.

It would be also be bad to leave this thread hanging as if the problems were never resolved.
People doing searches often find threads like these, hoping for solutions.

As a note, I figured out a way to disable the kobo accessing the store while still allowing it to remain logged in, eg: with an app. I wrote, so at least my son (who's birthday was yesterday), got to use the Kobo Reader for the first time -- for he can now read books that I have bought for him, without my having to worry about my credit card being charged for purchases I didn't authorize should the logged in reader get lost or stolen --so; there's no risk of chargebacks to Kobo over unauthorized purchases.

That should make us both happy, as open source benefits us both in reducing paperwork.

Cheers.

[fastrobot]

Originally Posted by eschwartz

Uh, most people don't have router firewalls... and how is Kobo supposed to know when you do?

That being said, Amazon has wisely avoided the issue by not demanding frivolous checks for Kindle firmware updates. When an update is available it will try to auto-update, but that is entirely independent of the other firmware components. Logging in on old firmware works just fine -- as it should.

yes, I am surprised that KOBO doesn't allow mildly older firmware to 'sync'.
It's just a small oversight.

But, not everyone owns their own internet connection, either; so I don't know if "most" people is a reality.
You may be right, or at least I'd bet that the kobo developers aren't behind firewalls...

But, I know lots of Kids are at their parents mercy with "parental" controls in place as many DSL's have those now if the parents care to set them up ... and sometimes people live in a large apartment high-rise / commercial building where the connection is supplied by the owner of the building; (Hotels very commonly do this, too); and if the building has its own PBX, as is common at college campuses, dorms, etc. the only other option may require a cell phone which is both expensive, and not always compatible as an AP.

So those of us being forced to deal with firewalls, at least for most of the day, is a significant number of people.

As for Detecting a firewall ; that's simple:

Most firewalls actually send a web page saying "forbidden" when you try to download a file that's not allowed. That's generally true of windows servers, and linux servers alike.
What is certain is that it WONT allow the firmware to download, but will allow webpages to load. So, that alone -- is a sufficient test to warn the user that there is potential firewall interference.

If I had known what was wrong, I would have tried other places wifi connections until I found one that wasn't blocked; quite possibly the local library which is anti-censorship, would have been a good bet ; but not the local grocery store or coffee shop, both of which it turns out are firewalled against most internet services, except basic http.

I can think of a really simple solution, too-- if I were running a firmware upgrade site; eg: firewalls don't block epub books, or other text files (html) from downloading, so if KOBO just used unix's uuencode program to turn their firmware into text, and then published their firmware as an epub book; the Kobo could download it without incident.