Help with cross-compiling tools?

[brianinmaine]

Originally Posted by twobob

the remote-debian thing was a "bad idea" in terms of compilation.

it runs things just fine but extremely heavy traffic (caused by compilation) can lock up the device sadly

Seemed like a great idea - too bad. Maybe I'll try to set up a remote swap file instead for this kind of stuff. I'd have to do some testing to see if it's faster or not.

Also, sometimes when some of you guys give answers it seems like you are talking in another language I have so much to learn just to get on the level when I can understand what you all are talking about!

I've seen buildroot and tried installed it, but then I did 'make xconfig' and didn't know what to select? I am running my own custom kernel on my laptop right now, there are just SO MANY options to choose from? Can you explain/share your config files? They are on a git site?

[knc1]

twobob at this moment is running some large compile jobs using a local swap file on the kindle.

too soon to speculate what the results may be.
(Other than the Kindle isn't smoking (yet). )

lots of information in the output of: cat /proc/vmstat
I think the main ones would be pswpin, pswpout ("Page Swap {In,Out}")

And the pswpout should be equivalent to the number of erase blocks your re-writing in the flash.
The flash's on-board controller will cycle through all available ones, and there are a lot - it may be a long time before each has been written too even once.

Like I wrote above, we will have some metrics on using an internal, flash based, swap file - soon.

PS: Geekmaster was a big fan of using the NBD driver to access a remote swap file. Details in some of his posts.

[twobob]

Originally Posted by brianinmaine

Seemed like a great idea - too bad. Maybe I'll try to set up a remote swap file instead for this kind of stuff. I'd have to do some testing to see if it's faster or not.

Also, sometimes when some of you guys give answers it seems like you are talking in another language I have so much to learn just to get on the level when I can understand what you all are talking about!

I've seen buildroot and tried installed it, but then I did 'make xconfig' and didn't know what to select? I am running my own custom kernel on my laptop right now, there are just SO MANY options to choose from? Can you explain/share your config files? They are on a git site?

yup. What knc1 said. I just bumped a post for you about swappage (with the amend of how I run it) - NOTE: I dont care if I destroy a kindle or two in the name of progress... You might.

Also: buildrooting a kindle 3 kernel has the vast majority of the options you need on it.


Just swap arm1136jf_s for CortexA8

THumb 2

hmmmm.. gimme sec EDIT: okay that should get you going

Niluje knows this stuff inside out but that is the thrust of it.
(and knc1)

[NiLuJe]

Yep, for the K4/K5/PW: ARCH: arm 32bit, le; CPU: cortex-a8; FPU: neon; FPU ABI: softfp; Instruction Set: Thumb2

Unless you are about accurate backtraces, you can safely bump the GCC optimization level to -O2 .

[knc1]

For the K5 (both) with the exception of programs using SIMD code, you should get better results using VFP2 for the FPU specification (the 508 has dual VFP2 co-processors) than using neon.

Some of the newer GCC versions have an "auto-vectorize" feature, but I haven't kept track of which versions have good ARM SIMD (vector) support.

[NiLuJe]

@knc1: That's vfpv3 in terms of GCC flags, IIRC.

Thanks for clarifying that, though. I had it as a possible choice in a comment, but I never bothered to bench it ;p.

[knc1]

Originally Posted by NiLuJe

@knc1: That's vfpv3 in terms of GCC flags, IIRC.

Thanks for clarifying that, though. I had it as a possible choice in a comment, but I never bothered to bench it ;p.

Might be - I was typing from memory.
Anyone would be best to read the gcc on-line manual arm options descriptions.

(Then they can make their own typos. )

[twobob]

3 hour compilation of the most ridiculously complex app.

root@kindle:/usr/local/src/PokerTH-1.0.1-src# cat /proc/vmstat
nr_free_pages 1877
nr_inactive_anon 811
nr_active_anon 34
nr_inactive_file 47421
nr_active_file 4899
nr_unevictable 0
nr_mlock 0
nr_anon_pages 785
nr_mapped 685
nr_file_pages 52544
nr_dirty 3706
nr_writeback 0
nr_slab_reclaimable 2467
nr_slab_unreclaimable 1156
nr_page_table_pages 261
nr_unstable 0
nr_bounce 0
nr_vmscan_write 48910
nr_writeback_temp 0
pgpgin 2447857
pgpgout 2067216
pswpin 6376
pswpout 33043
pgalloc_dma 1315218
pgalloc_normal 4687178
pgalloc_movable 0
pgfree 6004292
pgactivate 413375
pgdeactivate 1065840
pgfault 6823807
pgmajfault 17259
pgrefill_dma 384646
pgrefill_normal 736266
pgrefill_movable 0
pgsteal_dma 222540
pgsteal_normal 571762
pgsteal_movable 0
pgscan_kswapd_dma 205506
pgscan_kswapd_normal 544616
pgscan_kswapd_movable 0
pgscan_direct_dma 107361
pgscan_direct_normal 248259
pgscan_direct_movable 0
pginodesteal 5052
slabs_scanned 357888
kswapd_steal 585212
kswapd_inodesteal 72054
pageoutrun 7581
allocstall 3024
pgrotated 26801
unevictable_pgs_culled 2058
unevictable_pgs_scanned 0
unevictable_pgs_rescued 2163
unevictable_pgs_mlocked 2313
unevictable_pgs_munlocked 2163
unevictable_pgs_cleared 0
unevictable_pgs_stranded 0
unevictable_pgs_mlockfreed 0

[knc1]

pswpin 6376
pswpout 33043

With about 1 million erase blocks, I don't think erasing 33,000 of them, once each, is going to shorten your Kindle's life span a noticeable amount.

[twobob]

excellent - thanks!

should mention I had the framework - and X utterly stopped for this escapade.

[brianinmaine]

Originally Posted by twobob

excellent - thanks!

should mention I had the framework - and X utterly stopped for this escapade.

How big was the swap you made? Do you think it helped or will you try the same test without swap?


Also:
I have a chroot on my debian laptop of the arm version of debian with qemu on it so I can chroot into it. If I specify:
System types:
--build=BUILD configure for building on BUILD [guessed]
--host=HOST cross-compile to build programs to run on HOST [BUILD]
during builds, isn't this the same thing as compiling on the Kindle? Can I 'force the chroot to pretend to be a Kindle? This would allow me to have as much space as I want and a MUCH faster processor..

[knc1]

Originally Posted by brianinmaine

How big was the swap you made? Do you think it helped or will you try the same test without swap?


Also:
I have a chroot on my debian laptop of the arm version of debian with qemu on it so I can chroot into it. If I specify:
System types:
--build=BUILD configure for building on BUILD [guessed]
--host=HOST cross-compile to build programs to run on HOST [BUILD]
during builds, isn't this the same thing as compiling on the Kindle? Can I 'force the chroot to pretend to be a Kindle? This would allow me to have as much space as I want and a MUCH faster processor..

He used a 256Mbyte swap file.

You should not have to specify --build --host or --target inside of the chroot being run under QEMU.
Just be sure you start QEMU with the matching processor emulation.

Yes, you can have more (and faster - the flash is slow to write) space than you can on a Kindle.

Of course, depending on the horsepower of your host, you might even reach (or beat) the same emulated speed as that which is natural to a K5.

Under QEMU, you will probably need at least a 2.5Ghz, quad-core machine.

If really serious, you probably want to use OVP rather than QEMU:
http://www.ovpworld.org/

[brianinmaine]

This is what I did:

To be able to chroot into a target file system, the qemu emulator for the target CPU needs to be accessible from inside the chroot jail. For this to work, you need first to install the qemu-user-static package:

# apt-get install qemu-user-static
You cannot use the dynamically linked qemu because the host libraries will not be accessible from inside the chroot.

Next, copy the emulator for the target architecture to the path registered by binfmt-support. For example, for an ARM target file system, you need to do the following:

# cp /usr/bin/qemu-arm-static /target_fs/usr/bin
You should now be able to chroot into the file system:

# chroot /target_fs/

seems pretty quick, not like loading a regular Debian in a virtual machine?

[twobob]

Originally Posted by brianinmaine

This is what I did:

To be able to chroot into a target file system, the qemu emulator for the target CPU needs to be accessible from inside the chroot jail. For this to work, you need first to install the qemu-user-static package:

# apt-get install qemu-user-static
You cannot use the dynamically linked qemu because the host libraries will not be accessible from inside the chroot.

Next, copy the emulator for the target architecture to the path registered by binfmt-support. For example, for an ARM target file system, you need to do the following:

# cp /usr/bin/qemu-arm-static /target_fs/usr/bin
You should now be able to chroot into the file system:

# chroot /target_fs/

seems pretty quick, not like loading a regular Debian in a virtual machine?

I'm gonna try this now.

[knc1]

You still have to get the start-up command line passed to QEMU correct for the Kindle model processor your emulating.
(Which is doable, just missing from Brian's post.)

See the "qemu sub-tree" of the Buildroot distribution for examples.

And if your starting QEMU in 'system' mode (not running its own kernel, but using your host's kernel) -
Keep in mind that QEMU is just re-directing system calls to your host's linux kernel.
Which, sometimes, means your host needs to be running the same kernel version (to get the system calls to match the emulated system).

- - - - -

Yes, I know, the water is getting deeper and the current stronger - - but its all doable - just swim harder.