Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > Amazon Kindle > Kindle Developer's Corner

Notices

Reply
 
Thread Tools Search this Thread
Old 03-01-2012, 02:39 PM   #16
geekmaster
Carpe diem, c'est la vie.
geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.
 
geekmaster's Avatar
 
Posts: 6,433
Karma: 10764860
Join Date: Nov 2011
Location: Multiverse 6627A
Device: K1 to PW3
Quote:
Originally Posted by knc1 View Post
Some Kindle models are not so lucky but according to the FreeScale git repo, they are maintaining a 2.6.38 version (which would be a nice, general purpose, upgrade to all Kindle models).
...
Nice! That kernel version would let us run compache with immediate release notification so we can free compressed cache immediately. The kernel on the K4NT and Touch will run compcache fine, but freeing memory may be delayed until a process terminates. Freeing memory immediately on release is MUCH better.

But then, as the kindles are now with NO cache, even limited compcache support is a huge plus.

I think that putting cache files on the mmc instead of compressed RAM is a bad idea, because I did not see write wear levelling code in the mmc read and write routines. Compcache was designed for tablet devices like the kindles, and I think that we should be using it. The touch runs out of memory much too quickly, especially when using optware while the framework is still running.

Last edited by geekmaster; 03-01-2012 at 02:44 PM.
geekmaster is offline   Reply With Quote
Old 03-01-2012, 03:17 PM   #17
knc1
On Holiday
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 16,212
Karma: 17256900
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
Quote:
Originally Posted by geekmaster View Post
Nice! That kernel version would let us run compache with immediate release notification so we can free compressed cache immediately. The kernel on the K4NT and Touch will run compcache fine, but freeing memory may be delayed until a process terminates. Freeing memory immediately on release is MUCH better.

But then, as the kindles are now with NO cache, even limited compcache support is a huge plus.

I think that putting cache files on the mmc instead of compressed RAM is a bad idea, because I did not see write wear levelling code in the mmc read and write routines. Compcache was designed for tablet devices like the kindles, and I think that we should be using it. The touch runs out of memory much too quickly, especially when using optware while the framework is still running.
Another thing to keep in mind while mucking about with things...
Once you have a writable overlay system, then you can have a ro base.

A quick comparison, the k-3.2.1 base filesystem image as ext3 and as squashfs:
Code:
mszick@core2quad /bun1/Builds/Kindle/k-3.2.1/bin.tgz/img $ file rootfs.img
rootfs.img: Linux rev 1.0 ext3 filesystem data, UUID=96b81823-9626-43b1-a98b-a8085e28e614
- - -
mszick@core2quad /bun1/Builds/Kindle $ ls -l /bun1/Builds/Kindle/k-3.2.1/bin.tgz/img/rootfs.img
-rw-r--r-- 1 mszick mszick 419454976 2012-03-01 13:56 /bun1/Builds/Kindle/k-3.2.1/bin.tgz/img/rootfs.img
Now make a quick squashfs image of that, without any selecting or trimming or anything other than the default options:

Code:
mszick@core2quad /bun1/Builds/Kindle $ sudo mksquashfs /mnt/k321 k321.sfs
- - - snip - - -
mszick@core2quad /bun1/Builds/Kindle $ ls -l *.sfs
-rw-r--r-- 1 root root 248823808 2012-03-01 13:56 k321.sfs
Loop mount both images, check for amount of space used in the images (the ext3 image wasn't 100% full):

Code:
mszick@core2quad /bun1/Builds/Kindle/k-3.2.1/bin.tgz $ du -s /mnt/k321
359379	/mnt/k321
- - -
mszick@core2quad /bun1/Builds/Kindle $ du -s /mnt/k-sfs
355850	/mnt/k-sfs
Not much difference, but "free space" doesn't count in squashfs - you can't write to it.

Here is where the difference lies (if the reader didn't see it above):
The ext3 image requires: 419,454,976 bytes to store on flash.
The squashfs image requires: 248,823,808 bytes to store on flash.

Just because lab126 is still learning to build an embedded Linux system does not mean we have to follow their examples. We should know better than to do that here (follow lab126 example that is).

Last edited by knc1; 03-01-2012 at 03:20 PM.
knc1 is offline   Reply With Quote
Advert
Old 03-02-2012, 08:11 AM   #18
knc1
On Holiday
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 16,212
Karma: 17256900
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
Quote:
Originally Posted by knc1 View Post
Another thing to keep in mind while mucking about with things...
Once you have a writable overlay system, then you can have a ro base.
I posted a comparison of the two file system sizes for the K4NT here:
https://www.mobileread.com/forums/sho...06&postcount=7
Just to give an idea of what might be gained.
knc1 is offline   Reply With Quote
Old 03-04-2012, 10:56 PM   #19
Nyoxi
Connoisseur
Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.
 
Posts: 55
Karma: 124493
Join Date: Jan 2012
Device: Kindle Touch
Do you also have an idea how could we use that? Just by switching mmcblk0p1 from ext3 to squashfs would not help. To make proper use of the free space we would have to create new partition for our overlay between *p1 and *p2 which whould cause renumbering of all the other partitions. And would mean changing lot's of scripts and files on both main and diag partitions, but most importantly you probably wouldn't be able to boot into diags at all (or is there some idme or other variable that sais which partition to use?). The other option is to move all the partitions to move the free space to the end but this has also a few problems. First, are we sure diags will still work? Isn't there anything that relies on the original address of mmcblk0p2 on MMC? And you would have to do that before user puts his data onto userspace or you risk not having enough free space to move the partition.

But maybe I missed something.

By the way, I wouldn't shun people at lab126 because of this -- they didn't decide to use overlay (we did). And simply exchanging the free space on root filesystem for free space on overlay doesn't make you gain anything.

PS: Could you also post the real filesystem usage to be completely fair? I don't have those devices and I am too lazy to check that for KT.
Nyoxi is offline   Reply With Quote
Old 03-04-2012, 11:40 PM   #20
knc1
On Holiday
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 16,212
Karma: 17256900
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
Quote:
Originally Posted by Nyoxi View Post
Do you also have an idea how could we use that? Just by switching mmcblk0p1 from ext3 to squashfs would not help. To make proper use of the free space we would have to create new partition for our overlay between *p1 and *p2 which whould cause renumbering of all the other partitions. And would mean changing lot's of scripts and files on both main and diag partitions, but most importantly you probably wouldn't be able to boot into diags at all (or is there some idme or other variable that sais which partition to use?). The other option is to move all the partitions to move the free space to the end but this has also a few problems. First, are we sure diags will still work? Isn't there anything that relies on the original address of mmcblk0p2 on MMC? And you would have to do that before user puts his data onto userspace or you risk not having enough free space to move the partition.

But maybe I missed something.

By the way, I wouldn't shun people at lab126 because of this -- they didn't decide to use overlay (we did). And simply exchanging the free space on root filesystem for free space on overlay doesn't make you gain anything.

PS: Could you also post the real filesystem usage to be completely fair? I don't have those devices and I am too lazy to check that for KT.
I agree, it would be a major re-design effort.

I can't even guess at what might be gained or lost by it.

But still, a window of oppertunity to be kept in mind.

Right now, if people want to keep adding features to the system....
Well, there is a lot of storage available, and so what if the end result is the machine holds 1,200 books instead of 1,500?

Another oppertunity to keep in mind -
U-Boot has the ability to load a compressed kernel image and (on the k3) it is currently loading an uncompressed image.
That would result in a kernel image in about 1/2 the current storage area.
Or more features (modules) in the kernel without resizing its partition.

Just thoughts to keep in the back of one's mind, not any sort of plan or justification at the moment.
knc1 is offline   Reply With Quote
Advert
Old 03-05-2012, 07:35 AM   #21
Nyoxi
Connoisseur
Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.
 
Posts: 55
Karma: 124493
Join Date: Jan 2012
Device: Kindle Touch
Quote:
Originally Posted by knc1 View Post
But still, a window of oppertunity to be kept in mind.
Yup, for now I treat it that way.
Nyoxi is offline   Reply With Quote
Old 03-05-2012, 09:44 AM   #22
knc1
On Holiday
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 16,212
Karma: 17256900
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
A possible alternative

http://archlinuxarm.org/

If not familar with Arch Linux, it is a "binary distribution" with packages built from the Gentoo distribution sources.

You can get an Arch Linux LiveCD to try out on your x86{,_64} machine if curious.
http://www.archlinux.org/

Minimum kernel version is now 2.6.32 but from the looks of Freescale's git repo, they are actively maintaining 2.6.38 (although 2.6.35 is the most recent tagged as a release).

unionFS sucks (it always has to some degree), overlayFS isn't of much use before the kernel 3.x series - so auFS is the current best choice.

Will put building auFS modules for the Kindle-Kernels (KK?) on my to-do list - warning, don't hold your breath, my to-do list is very long.
knc1 is offline   Reply With Quote
Old 03-20-2012, 05:23 PM   #23
lukeroge
Enthusiast
lukeroge began at the beginning.
 
Posts: 35
Karma: 10
Join Date: Feb 2012
Device: Kindle Touch
This seems like a pretty useful idea
lukeroge is offline   Reply With Quote
Old 03-20-2012, 08:23 PM   #24
geekmaster
Carpe diem, c'est la vie.
geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.
 
geekmaster's Avatar
 
Posts: 6,433
Karma: 10764860
Join Date: Nov 2011
Location: Multiverse 6627A
Device: K1 to PW3
Quote:
Originally Posted by Nyoxi View Post
Do you also have an idea how could we use that? Just by switching mmcblk0p1 from ext3 to squashfs would not help. To make proper use of the free space we would have to create new partition for our overlay between *p1 and *p2 which whould cause renumbering of all the other partitions. And would mean changing lot's of scripts and files on both main and diag partitions, but most importantly you probably wouldn't be able to boot into diags at all (or is there some idme or other variable that sais which partition to use?). The other option is to move all the partitions to move the free space to the end but this has also a few problems. First, are we sure diags will still work? Isn't there anything that relies on the original address of mmcblk0p2 on MMC? And you would have to do that before user puts his data onto userspace or you risk not having enough free space to move the partition.

But maybe I missed something.

By the way, I wouldn't shun people at lab126 because of this -- they didn't decide to use overlay (we did). And simply exchanging the free space on root filesystem for free space on overlay doesn't make you gain anything.

PS: Could you also post the real filesystem usage to be completely fair? I don't have those devices and I am too lazy to check that for KT.
The startup logic for both main and diags respect whatever they find for the 4 main primary partitions. The values in mntus.params are only used if the USB drive cannot be mounted successfully.

Because p4 has its own partition table, the USB drive resides in "mmcblk0p4p1", and you have 3 more empty "inner" partitions you can use. You need to either loop mount mmcblk0p4 then mount the second partition inside that (mmcblk0p4p2), or you just loop mount it directly with an offset (like how /mnt/us is sometimes mounted). Of course you need to shrink mmcblk0p4p1 to make room to expand one or more of the empty inner partitions.

Last edited by geekmaster; 03-20-2012 at 08:27 PM. Reason: severe typophrenia
geekmaster is offline   Reply With Quote
Old 03-20-2012, 08:31 PM   #25
geekmaster
Carpe diem, c'est la vie.
geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.
 
geekmaster's Avatar
 
Posts: 6,433
Karma: 10764860
Join Date: Nov 2011
Location: Multiverse 6627A
Device: K1 to PW3
Quote:
Originally Posted by knc1 View Post
...
unionFS sucks (it always has to some degree), overlayFS isn't of much use before the kernel 3.x series - so auFS is the current best choice.
auFS is a fork of unionFS, so if unionFS "always sucked" that impies that auFS also sucked in its early days.

Apparently they forked it so they could give it an "unsuck" job.

Last edited by geekmaster; 03-20-2012 at 08:34 PM. Reason: severe typophrenia
geekmaster is offline   Reply With Quote
Old 03-21-2012, 08:45 AM   #26
knc1
On Holiday
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 16,212
Karma: 17256900
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
Quote:
Originally Posted by geekmaster View Post
auFS is a fork of unionFS, so if unionFS "always sucked" that impies that auFS also sucked in its early days.

Apparently they forked it so they could give it an "unsuck" job.
Not so.

When JRO and the unionFS project parted ways, he left behind the parts of the original design that he saw as stumbling blocks to an effective design.

Which left him with a minimal feature set and then he worked forward from that point. The auFS-version-1 rarely gets any mention in recent years.

I don't recall at the moment what was the major change between auFS-version-1 and auFS-version-2.

With the change from auFS-version-2 to auFS-version-3 JRO did another 'feature strip' of his own project to eleminate rarely needed and difficult to maintain features.

The 'they' in this project is a single person and over the years it has been a very impressive, single person, project to follow.
knc1 is offline   Reply With Quote
Old 04-12-2012, 04:23 PM   #27
idoit
hub
idoit ought to be getting tired of karma fortunes by now.idoit ought to be getting tired of karma fortunes by now.idoit ought to be getting tired of karma fortunes by now.idoit ought to be getting tired of karma fortunes by now.idoit ought to be getting tired of karma fortunes by now.idoit ought to be getting tired of karma fortunes by now.idoit ought to be getting tired of karma fortunes by now.idoit ought to be getting tired of karma fortunes by now.idoit ought to be getting tired of karma fortunes by now.idoit ought to be getting tired of karma fortunes by now.idoit ought to be getting tired of karma fortunes by now.
 
idoit's Avatar
 
Posts: 714
Karma: 2151032
Join Date: Jan 2012
Location: Iranian in Canada
Device: K3G, DXG, Kobo mini
Quote:
Originally Posted by Nyoxi View Post
Right now we seem to have one only option: disable the overlay, apply update, create new clean overlay.

At some later point we can make use of the reconstruction of /var/local ... but I can't remember what was the use. I realy had an idea here that I can't remember right now (I should have written all this sooner).
Thanks for this neat thread.
I wonder if there are any thing new regarding this. The motivation for me to try this would be to be able to update Touch firmware more easily. Right now, if I want to let it update the firmware, I have to go through the list of mods, hack and file changes I have made which is cumbersome. Now with this overlay file system, would it be possible to just disable overlay and update and then re-enable and have all the modifications in place?
idoit is offline   Reply With Quote
Old 05-09-2012, 06:10 PM   #28
Nyoxi
Connoisseur
Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.Nyoxi can talk all four legs off a donkey... then persuade it to go for a walk.
 
Posts: 55
Karma: 124493
Join Date: Jan 2012
Device: Kindle Touch
No, nothing new ... at least not from me.

As for the updates the answer is still no. For simple reason already mentioned here: https://www.mobileread.com/forums/sho...5&postcount=12 in point 2)

The point 1) however could be solved if somebody figures out how to safely deal with the kernel/initrd changes in updates (kexec?).
Nyoxi is offline   Reply With Quote
Reply

Tags
mini_fo, overlay, root filesystem

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Creating the Apple Read Aloud Media Overlay sgtgrom ePub 1 08-08-2011 01:53 AM
Accessories Kindle 3 number-key overlay? tovare Amazon Kindle 25 02-21-2011 05:30 PM
To Root, or not to Root... that is the question t3l01v Barnes & Noble NOOK 8 01-24-2011 06:54 PM
Development Alternate root method / "1-click root" Oneiros enTourage Archive 0 09-06-2010 02:04 PM
DR800 root filesystem contents Mr. X iRex 2 03-05-2010 07:31 AM


All times are GMT -4. The time now is 04:37 PM.


MobileRead.com is a privately owned, operated and funded community.