DIY - KeK - Page 18

knc1 · 08-12-2012, 09:56 AM

The Rob Landley emulator image has been quite solid.
It was only posted for the person who wanted to do: "one or two small programs".

Our "abusing" it by trying to build complete toolchains and small Linux distributions is showing some of its limitations but have not disqualified it for its original purpose.

Two things for the newbe to keep in mind:

The emulator runs the same command shell (of a different version) as the Kindles.
A good place to answer the question: "Will this script of mine run under ash?"

It does contain a complete "C" and "C++" gcc toolchain (in addition to the mis-functional tcc build).

twobob · 08-12-2012, 11:22 AM

Fully concur. I set out to see what would happen if it was splattered with the big stuff. now we know. happy with that.

Hopefully my documented processes may stir others into seeing if the issues were specific to my machine / implementation or indeed general effects of applying the processes I undertook.

Still good for the little stuff.

eureka · 08-13-2012, 04:34 PM

@knc1, as you've collected some cross-compiling toolchains in scope of KeK project, you might be interested in The Embedded LLVM Compiler Collection: ellcc.

It's a cross-compilation environment based on Clang (which is modern C/C++ compiler using LLVM infrastructure). ELLCC has support of compiling for Linux and includes ARM as cross-compilation target (amongst many other targets). Building and installation from sources is as easy as svn co ... && ./configure && make && make install and there are also pre-built binaries for Linux x86-64.

It doesn't include (e)glibc/libstdc++, it incorporates musl and libc++. But it could be seen as advantage for static builds

(Nice small example of use of ELLCC.)

Disclaimer: I didn't use it, however it looks like actively and seriously developed project.

NiLuJe · 08-13-2012, 05:57 PM

@eureka: Oooh, good to know, I wasn't aware of musl, and that there was something like that using an x-clang tc

. (clang's veeeery nice. If you've never checked it out, even on a native system, do so. The static code analysis tools and address sanitizer introduced in 3.1 are pretty nice features, especially compared to the mess that is setting up/using valgrind on a recent distro...)

eureka · 08-13-2012, 06:10 PM

Quote:

Originally Posted by NiLuJe

@eureka: Oooh, good to know, I wasn't aware of musl, and that there was something like that using an x-clang tc

. (clang's veeeery nice. If you've never checked it out, even on a native system, do so. The static code analysis tools and address sanitizer introduced in 3.1 are petty nice features, especially compared to the mess that is setting up/using valgrind on a recent distro...)

Well, I did use Clang for some native builds and have an interest in using it for cross-compiling (I didn't use ELLCC just because I already have Clang installed and have an urge to adapt this installation for cross-compiling needs). Seconding your promotion.

knc1 · 08-13-2012, 10:03 PM

This article takes a slightly different approach and is fairly recent (2010):
http://balau82.wordpress.com/2010/04...t-on-qemu-arm/

It might be worth checking if the mentioned patches are already in our qemu sources before starting to follow that article. Mainstream may have picked them up but implemented the features a bit differently.

Edit:
One point you will not find in the general articles . . .
The i.MX Freescale SoC has a hardware watchdog which can not be disabled and if not serviced, will reset the core.

So any program running in control of the processor (iBootloader, U-boot, kernel in that order) has to provide code to service the watchdog often enough to keep it from tripping.

Things may not be the same under emulation - I don't know if qemu emulates the non-maskable hardware watchdog.

knc1 · 08-14-2012, 11:26 AM

Make the location of the AL files the current directory.
Check that you have at least 4Gbytes of free disk space:

Code:

core2quad armv6l $ df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/md126             37G   21G   15G  60% /

Create a new, sparse, file for the emu /home file system:

Code:

core2quad armv6l $ dd if=/dev/zero of=hdb2.img bs=1024 seek=$[1024*1024*4] count=1
core2quad armv6l $ mke2fs -q -b 1024 -i 4096 -F hdb2.img
  warning: Unable to get device geometry for hdb2.img
core2quad armv6l $ tune2fs -j -c 0 -i 0 hdb2.img
  tune2fs 1.41.14 (22-Dec-2010)
  Setting maximal mount count to -1
  Setting interval between checks to 0 seconds
  Creating journal inode: done
  This filesystem will be automatically checked every -1 mounts or
  0 days, whichever comes first.  Use tune2fs -c or -i to override.

When run as the /home directory of the emulator, we want a (e)glibc
development environment, rooted at /home/emd, that can be run as a
chroot.

That development environment should be tailored for the purpose it will
be used for, such as one modeled on a Kindle's filesystem.
For this document, the Ubuntu-core, rootfs image is used.

Code:

core2quad armv6l $ wget http://drpbox.knetconnect.com/KeK/ubuntu-core-12.04-core-armel.tar.gz

Make a mount point for our empty system image file and mount it:

Code:

core2quad armv6l $ sudo mkdir /mnt/hdb2
core2quad armv6l $ sudo mount hdb2.img /mnt/hdb2

Running within the emulator, our /mnt/hdb2 location is known as /home.
Make the emd sub-directory and populate it with the Ubuntu-core files:
These two steps MUST be done as 'root' to get the image correct!

Code:

core2quad armv6l $ sudo mkdir /mnt/hdb2/emd
core2quad armv6l $ sudo tar -C /mnt/hdb2/emd -xf ubuntu-core-12.04-core-armel.tar.gz

Since it is expected that relatively large jobs will be done within this
chroot, build a swap file for the emulator to use:
NOTE: This must NOT be a sparse file.

Code:

core2quad armv6l $ cd /mnt/hdb2
core2quad hdb2 $ sudo dd if=/dev/zero of=swapf bs=$[1024*1024] count=512
  (wait for it)
core2quad hdb2 $ sudo mkswap swapf
  Setting up swapspace version 1, size = 524284 KiB
  no label, UUID=a352dd5b-2dce-424e-a158-7e4901e5744f
core2quad hdb2 $ cd -

Unmount the new file system, turn it into a device and run e2fsck on it:

Code:

core2quad armv6l $ sudo umount /mnt/hdb2
core2quad armv6l $ sudo losetup -f
  /dev/loop0
core2quad armv6l $ sudo losetup /dev/loop0 hdb2.img
core2quad armv6l $ sudo e2fsck -f /dev/loop0
  e2fsck 1.41.14 (22-Dec-2010)
  Pass 1: Checking inodes, blocks, and sizes
  Pass 2: Checking directory structure
  Pass 3: Checking directory connectivity
  Pass 4: Checking reference counts
  Pass 5: Checking group summary information
  /dev/loop0: 8347/1048576 files (0.5% non-contiguous), 908743/4194304 blocks
core2quad armv6l $ sudo losetup -d /dev/loop0

Next, modify the Aboriginal Linux scripts for the new name(s).
Only two are shown here, make a similar change to native-build.sh if you
are using that script.

Code:

core2quad armv6l $ cp -a dev-environment.sh dev-ucore.sh
core2quad armv6l $ cp -a run-emulator.sh run-ucore.sh

Edit the dev-ucore.sh script to change the two lines shown:

Code:

core2quad armv6l $ diff -u dev-environment.sh dev-ucore.sh
--- dev-environment.sh    2012-07-14 08:57:55.000000000 -0500
+++ dev-ucore.sh    2012-08-14 09:44:41.000000000 -0500
@@ -46,7 +46,7 @@
   [ $? -ne 0 ] && exit 1
 }
 
-source ./run-emulator.sh --norun || exit 1
+source ./run-ucore.sh --norun || exit 1
 
 [ -z "$QEMU_MEMORY" ] && QEMU_MEMORY=256
 QEMU_EXTRA="-m $QEMU_MEMORY $QEMU_EXTRA"
@@ -55,7 +55,7 @@
 
 if [ "$HDBMEGS" != "0" ]
 then
-  [ -z "$HDB" ] && HDB=hdb.img
+  [ -z "$HDB" ] && HDB=hdb2.img
   if [ ! -e "$HDB" ]
   then

Fire the little turkey up and see what three hours of typing this how-to
has done to our emulator environment:

Code:

core2quad system-image-armv6l $ . pset.sh
core2quad system-image-armv6l $ ./dev-ucore.sh
- - - -
Not using distcc.
Type exit when done.
(armv6l:1) /home # ls
  emd         lost+found  swapf
(armv6l:1) /home # ls emd
  bin      etc      media    proc     sbin     sys      var
  boot     home     mnt      root     selinux  tmp
  dev      lib      opt      run      srv      usr

All bow down and say "Thank you Rob":

Code:

(armv6l:1) /home # swapon swapf
  /bin/ash: swapon: not found

To make up for that lack, move the swap file inside of the chroot tree.
It would be better to leave it where it is and start it before doing the
following tmpfs assignments but that would take building swapon/swapoff.

Code:

(armv6l:1) /home # mv swapf emd

Setup the chroot dynamic filesystems:

Code:

(armv6l:1) /home # mount -t tmpfs tmpfs /home/emd/tmp
(armv6l:1) /home # mount -t tmpfs tmpfs /home/emd/var/tmp
(armv6l:1) /home # mount -t sysfs sysfs /home/emd/sys
(armv6l:1) /home # mount -t proc proc /home/emd/proc
(armv6l:1) /home # mount -o rbind /dev /home/emd/dev

Try to enter the new chroot and then say: "Thank you Rob":

Code:

(armv6l:1) /home # chroot /home/emd /bin/sh
  Segmentation fault

Well, this bright idea isn't going to work as-is.
Need to (at least) build swapon/swapoff/chroot for the emulation environment.
Once that is done, then this method should work for (e)glibc development.

For now, just shutdown the chroot dynamic filesystems and exit:

Code:

(armv6l:1) /home # umount /home/emd/dev/pts
(armv6l:1) /home # umount /home/emd/dev
(armv6l:1) /home # umount /home/emd/proc
(armv6l:1) /home # umount /home/emd/sys
(armv6l:1) /home # umount /home/emd/var/tmp
(armv6l:1) /home # umount /home/emd/tmp
(armv6l:1) /home # cd /root
(armv6l:1) /root # umount /home
(armv6l:1) /root # exit
Restarting system.
core2quad armv6l $ 

NOTE:  Whatever became of /dev/shm Rob?  Could this be part of the problem?

Once the chroot is entered for the first time, this Ubuntu-core image will need the few site-local files setup.
Things like creating a regular user name, tweaking the sudoers file, . . . .

knc1 · 08-14-2012, 03:03 PM

Lets see if we can fix the Busybox chroot problem in the above post with this bundle of goodies.

Build notes:

Spoiler:

This is another file blob that you just un-pack with /home of the emulator as the current directory. Puts chroot + 102 other small utilities into /home/bin.

Now the $64 million dollar question - Did it fix the above problem?

Spoiler:

Answer: No.

Next step: Install our KeK strace in this copy of the emulator image.

But folks, I've had all the fun I can stand today. Maybe more tomorrow.

knc1 · 08-15-2012, 05:44 PM

See what is to be learned from strace about this problem:

Code:

(armv6l:1) /home # strace -fF chroot /home/emd /bin/sh
  * * snip the usual memory setup stuff * *
  chroot("/home/emd")                     = 0
  chdir("/")                              = 0
  execve("/bin/sh", ["/bin/sh"], [/* 9 vars */]) = 0
  --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x1be} ---
  +++ killed by SIGSEGV +++
  Segmentation fault
(armv6l:1) /home #

Hmm...
The chroot, chdir and execve system calls all returned 0 (success).
It looks like the Ubuntu-core, /bin/sh stumbled into the mapping error segfault.

In Ubuntu-core /bin/sh is a sym-link to /bin/dash.
About the simplest compiled program in any distribution is:

Code:

int main(void) { return 0; }

Also known in public as: /bin/true.

Code:

(armv6l:1) /home # strace -fF chroot /home/emd /bin/true
  * * snip the usual memory setup stuff * *
chroot("/home/emd")                     = 0
chdir("/")                              = 0
execve("/bin/true", ["/bin/true"], [/* 9 vars */]) = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x1be} ---
+++ killed by SIGSEGV +++
Segmentation fault
(armv6l:1) /home #

Hmm...
That should tell me something, not sure what.
How in the world can: "return 0" cause a seg-fault?

An ABI mis-match?
Will have to investigate that possibility.

geekmaster · 08-15-2012, 06:48 PM

Quote:

Originally Posted by knc1

About the simplest compiled program in any distribution is:

Code:

int main(void) { return 0; }

Also known in public as: /bin/true.

Isn't "return 0" returning FALSE instead of TRUE? Oh, nevermind. In C, zero is FALSE, but in the shell it is TRUE. Oh well, never mind then...

"Three blank lines copyrighted by AT&T" : http://linuxlife.blogspot.com/2012/0...copyright.html

knc1 · 08-15-2012, 07:03 PM

No matter if it returns 0 or 1.
No matter if it is called true or false.
The question remains of how that program can generate a segment mapping violation and seg-fault.

There is a one or two register difference between eabi and gnueabi - which may be the problem here

I.E: I either built A.L. to the wrong ABI or at least not to the same ABI that Ubuntu built their rootfs.

And I just knew you would want to comment on the "excessive whitespace" in my representation of the program.

twobob · 08-16-2012, 07:50 AM

Meh Copyright is ten a penny. The day they get the patent on returning 1 or 0 I begin to worry.

@knc1 Hmm yes - do you think this may go some way to explain my experiences with the internal 'ar' etc. Well. I shall resume cheerleading - bloody good catch mate.

Intrigued to know the result mate.

ra ra emu!

knc1 · 08-16-2012, 08:28 AM

Quote:

Originally Posted by twobob

@knc1 Hmm yes - do you think this may go some way to explain my experiences with the internal 'ar' etc. Well. I shall resume cheerleading - bloody good catch mate.

Intrigued to know the result mate.

ra ra emu!

I recall your reporting programs that did that and yes, it may be related.

The 'main' executable code of /bin/true (or /bin/false) should be two machine instructions. One to set the register holding the returned value and the "return" instruction itself. Neither directly accesses memory (not even in RISC).

But every compiler generated executable contains a preamble and a post-amble (setting stack pointer, saving the return address, +a bunch of other things). Most of those involve setting registers to point at the proper parts of memory.

Those, if using a register that wasn't set to the expected value, or not set at all, can cause a memory segmentation violation.

There have been various "ABI"s for ARM over the years, Rob's compiler may be set to generate one (the compiler generated pre/post ambles) and the Ubuntu-core man be using another (with different register usage defines).

It will take a bit of research and maybe some disassembly to identify where the mismatch is originating.

Although in the end, it may be time to replace Rob's rootfs that we are using.
For new work, with one of the Ubuntu-core rootfs images.
For Kindle re-work, by adding a native tool-chain to the one you just built against the Amazon kernel.

Hmm...
Maybe the fasted way forward is to just get the emulator to boot these other rootfs images.

knc1 · 08-16-2012, 02:06 PM

Quote:

Originally Posted by twobob

@knc1 Hmm yes - do you think this may go some way to explain my experiences with the internal 'ar' etc. Well. I shall resume cheerleading - bloody good catch mate.

Intrigued to know the result mate.

ra ra emu!

This one looks like the most likely:
-mpic-register=reg
Specify the register to be used for PIC addressing. The default is R10 unless stack-checking is enabled, when R9 is used.

Not yet confirmed in the dis-assembly, but I seem to recall that Rob's gcc-4.2 does not have stack checking enabled and the CS/MG gcc-4.6 cross-compiler does have stack checking enabled.

Will see if I can build a static linked chroot, then it will not matter which register is used for PIC addressing base.

twobob · 08-16-2012, 03:09 PM

hmm.. sounds like you are close to the solution there mate.

I tend to agree about mounting something a bit more - for the job - now.

Although this has proved excellent and will still remain useful for certain things without question.

Assuming the BR image works fine as you say it will seemingly be a pretty decent platform for devving - and as you say - for the purpose.

Interesting to see how the static stuff fairs.

08-13-2012, 05:57 PM	#259
NiLuJe BLAM! Posts: 13,477 Karma: 26012494 Join Date: Jun 2010 Location: Paris, France Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E	@eureka: Oooh, good to know, I wasn't aware of musl, and that there was something like that using an x-clang tc . (clang's veeeery nice. If you've never checked it out, even on a native system, do so. The static code analysis tools and address sanitizer introduced in 3.1 are pretty nice features, especially compared to the mess that is setting up/using valgrind on a recent distro...) Last edited by NiLuJe; 08-13-2012 at 06:16 PM. Reason: petty != pretty... Oops.

08-13-2012, 10:03 PM	#261
knc1 Going Viral Posts: 17,212 Karma: 18210809 Join Date: Feb 2012 Location: Central Texas Device: No K1, PW2, KV, KOA	This article takes a slightly different approach and is fairly recent (2010): http://balau82.wordpress.com/2010/04...t-on-qemu-arm/ It might be worth checking if the mentioned patches are already in our qemu sources before starting to follow that article. Mainstream may have picked them up but implemented the features a bit differently. Edit: One point you will not find in the general articles . . . The i.MX Freescale SoC has a hardware watchdog which can not be disabled and if not serviced, will reset the core. So any program running in control of the processor (iBootloader, U-boot, kernel in that order) has to provide code to service the watchdog often enough to keep it from tripping. Things may not be the same under emulation - I don't know if qemu emulates the non-maskable hardware watchdog. Last edited by knc1; 08-13-2012 at 10:34 PM.

08-15-2012, 05:44 PM	#264
knc1 Going Viral Posts: 17,212 Karma: 18210809 Join Date: Feb 2012 Location: Central Texas Device: No K1, PW2, KV, KOA	See what is to be learned from strace about this problem: Code: (armv6l:1) /home # strace -fF chroot /home/emd /bin/sh * * snip the usual memory setup stuff * * chroot("/home/emd") = 0 chdir("/") = 0 execve("/bin/sh", ["/bin/sh"], [/* 9 vars /]) = 0 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x1be} --- +++ killed by SIGSEGV +++ Segmentation fault (armv6l:1) /home # Hmm... The chroot, chdir and execve system calls all returned 0 (success). It looks like the Ubuntu-core, /bin/sh stumbled into the mapping error segfault. In Ubuntu-core /bin/sh is a sym-link to /bin/dash. About the simplest compiled program in any distribution is: Code: int main(void) { return 0; } Also known in public as: /bin/true. Code: (armv6l:1) /home # strace -fF chroot /home/emd /bin/true * snip the usual memory setup stuff * * chroot("/home/emd") = 0 chdir("/") = 0 execve("/bin/true", ["/bin/true"], [/* 9 vars */]) = 0 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x1be} --- +++ killed by SIGSEGV +++ Segmentation fault (armv6l:1) /home # Hmm... That should tell me something, not sure what. How in the world can: "return 0" cause a seg-fault? An ABI mis-match? Will have to investigate that possibility.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Accessories DIY KT Case	echo013	Amazon Kindle	2	01-27-2012 06:53 PM
DIY Stylus	scottjl	Apple Devices	3	05-10-2010 01:52 AM
DIY Scanner	Eratosthenes	News	14	04-16-2010 04:21 PM
DIY Idea	sørensundsans	Sony Reader	6	01-26-2010 01:25 AM
DIY stylus	-Thomas-	iRex	5	02-12-2009 12:05 PM

08-12-2012, 09:56 AM	#256
knc1 Going Viral Posts: 17,212 Karma: 18210809 Join Date: Feb 2012 Location: Central Texas Device: No K1, PW2, KV, KOA	The Rob Landley emulator image has been quite solid. It was only posted for the person who wanted to do: "one or two small programs". Our "abusing" it by trying to build complete toolchains and small Linux distributions is showing some of its limitations but have not disqualified it for its original purpose. Two things for the newbe to keep in mind: The emulator runs the same command shell (of a different version) as the Kindles. A good place to answer the question: "Will this script of mine run under ash?" It does contain a complete "C" and "C++" gcc toolchain (in addition to the mis-functional tcc build).

08-12-2012, 11:22 AM	#257
twobob ( ͡° ͜ʖ ͡°){ʇlnɐɟ ƃǝs}Týr Posts: 6,586 Karma: 6299991 Join Date: Jun 2012 Location: uti gratia usura (Yao ying da ying; Mo ying da yieng) Device: PW-WIFI\|K5-3G+WIFI\| K4\|K3-3G\|DXG\|K2\| Rooted Nook Touch	Fully concur. I set out to see what would happen if it was splattered with the big stuff. now we know. happy with that. Hopefully my documented processes may stir others into seeing if the issues were specific to my machine / implementation or indeed general effects of applying the processes I undertook. Still good for the little stuff.

08-13-2012, 04:34 PM	#258
eureka but forgot what it's like Posts: 741 Karma: 2345678 Join Date: Dec 2011 Location: north (by northwest) Device: Kindle Touch	@knc1, as you've collected some cross-compiling toolchains in scope of KeK project, you might be interested in The Embedded LLVM Compiler Collection: ellcc. It's a cross-compilation environment based on Clang (which is modern C/C++ compiler using LLVM infrastructure). ELLCC has support of compiling for Linux and includes ARM as cross-compilation target (amongst many other targets). Building and installation from sources is as easy as svn co ... && ./configure && make && make install and there are also pre-built binaries for Linux x86-64. It doesn't include (e)glibc/libstdc++, it incorporates musl and libc++. But it could be seen as advantage for static builds (Nice small example of use of ELLCC.) Disclaimer: I didn't use it, however it looks like actively and seriously developed project.

08-15-2012, 07:03 PM	#266
knc1 Going Viral Posts: 17,212 Karma: 18210809 Join Date: Feb 2012 Location: Central Texas Device: No K1, PW2, KV, KOA	No matter if it returns 0 or 1. No matter if it is called true or false. The question remains of how that program can generate a segment mapping violation and seg-fault. There is a one or two register difference between eabi and gnueabi - which may be the problem here I.E: I either built A.L. to the wrong ABI or at least not to the same ABI that Ubuntu built their rootfs. And I just knew you would want to comment on the "excessive whitespace" in my representation of the program.

08-16-2012, 07:50 AM	#267
twobob ( ͡° ͜ʖ ͡°){ʇlnɐɟ ƃǝs}Týr Posts: 6,586 Karma: 6299991 Join Date: Jun 2012 Location: uti gratia usura (Yao ying da ying; Mo ying da yieng) Device: PW-WIFI\|K5-3G+WIFI\| K4\|K3-3G\|DXG\|K2\| Rooted Nook Touch	Meh Copyright is ten a penny. The day they get the patent on returning 1 or 0 I begin to worry. @knc1 Hmm yes - do you think this may go some way to explain my experiences with the internal 'ar' etc. Well. I shall resume cheerleading - bloody good catch mate. Intrigued to know the result mate. ra ra emu!

08-16-2012, 03:09 PM	#270
twobob ( ͡° ͜ʖ ͡°){ʇlnɐɟ ƃǝs}Týr Posts: 6,586 Karma: 6299991 Join Date: Jun 2012 Location: uti gratia usura (Yao ying da ying; Mo ying da yieng) Device: PW-WIFI\|K5-3G+WIFI\| K4\|K3-3G\|DXG\|K2\| Rooted Nook Touch	hmm.. sounds like you are close to the solution there mate. I tend to agree about mounting something a bit more - for the job - now. Although this has proved excellent and will still remain useful for certain things without question. Assuming the BR image works fine as you say it will seemingly be a pretty decent platform for devving - and as you say - for the purpose. Interesting to see how the static stuff fairs.

Advert

Advert