Benchmark. Benchmarking Ciphers. (And a handy test of the scripty wangle WIP)

[twobob]

Extract somewhere on FAT drive

[root@kindle us]# ./benchmark.sh

Spoiler:

MD5 : 9639 Kb/s, 0 cycles/byte
SHA-1 : 5841 Kb/s, 0 cycles/byte
SHA-256 : 2808 Kb/s, 0 cycles/byte
SHA-512 : 1488 Kb/s, 0 cycles/byte
ARC4 : 4279 Kb/s, 0 cycles/byte
3DES : 466 Kb/s, 2 cycles/byte
DES : 1085 Kb/s, 1 cycles/byte
AES-128 : 1832 Kb/s, 0 cycles/byte
AES-192 : 1650 Kb/s, 0 cycles/byte
AES-256 : 1504 Kb/s, 0 cycles/byte
CAMELLIA-128 : 1281 Kb/s, 0 cycles/byte
CAMELLIA-192 : 1000 Kb/s, 0 cycles/byte
CAMELLIA-256 : 1004 Kb/s, 0 cycles/byte
HAVEGE : 62 Kb/s, 16 cycles/byte
CTR_DRBG (NOPR) : 1607 Kb/s, 0 cycles/byte
CTR_DRBG (PR) : 1149 Kb/s, 0 cycles/byte
RSA-1024 : 604 public/s
RSA-1024 : 30 private/s
RSA-2048 : 188 public/s
RSA-2048 : 6 private/s
RSA-4096 : 53 public/s
RSA-4096 : 1 private/s

The point here is not what does benchmark do? it does ^this, at about this speed on all threes.

This was about, getting libs reffed right and packaging stuff neatly, which is now done to a large degree.

Here's a readelf -d pruned.

Dynamic section at offset 0x25d8 contains 25 entries:
Tag Type Name/Value
0x00000001 (NEEDED) Shared library: [libpolarssl.so.1]
0x00000001 (NEEDED) Shared library: [libc.so.6]

This should now work OOTB on all 3's. possibly others FB welcomed on that.

As I said this is about establishing working practices.
The application is now made available with the better script.

Thanks.

[twobob]

The updated loader script will read:

#!/bin/sh
# http://www.gnu.org/licenses/gpl-2.0.html

cd $( dirname "$0" )

#Get libs
/lib/ld-linux.so.3 --library-path $PWD/lib:/usr/lib:/lib --inhibit-rpath libpolarssl.so.1 ./benchmark

[twobob]

EDIT: EVERYTHING AFTER THE TOP THREAD IS NOW NOT reffing the TOP THREAD SCRIPT. IT HAS BEEN UPDATED TO THE LATEST VERSION. JUST TO BE CLEAR. THANKS

: )Thanks

[knc1]

Those "php code" tags make it look pretty enough to work.
None of the ugly black and white scripting.

I **think** you have the pairs in the search order paths interchanged.
As in, for the trusted system library paths:
/usr/lib:/lib rather than /lib:/usr/lib
Putting all the */usr/<whatever> before the */<whatever> allows the local install to "override" the system installed objects.

Same reasoning for */local/* those go before the "fixed" directories in the path to allow the local install to "override" the system installed objects.

There is probably a "spec" on this, somewhere, I don't know the reference off-hand.

An example:
/mnt/us/usr/bin:/mnt/us/bin:/usr/local/bin:/usr/bin:/bin
Same for sbin

And invert the order of ld_library_path - same reasoning, the most specific first.
Hmm... Once you do that, you might not have to pre-load the "C" library.

[twobob]

EDIT: EVERYTHING AFTER THE TOP THREAD IS NOW NOT reffing the TOP THREAD SCRIPT. IT HAS BEEN UPDATED TO THE LATEST VERSION. JUST TO BE CLEAR. THANKS


The environment variable LD_LIBRARY_PATH is referenced *AFTER* the system library paths. According to what I have read on Stupid ld Tricks. That was of course for a RHEL system IIRC but the info seemed good.

This was the only way - so far - I could get it to swallow the c libs in this manner - with no ldconfig update - but I'll go check your alternate method now

Much appreciate the feedback. I'm not even 100% this way works as my test device refuses to NOT work at the moment. lol. Hence the "Does this really work?" header : D

[knc1]

RedHat has the most heavly patched, non-standard system of any distribution.

Also, the author of the reference did not know he could have avoided all of those handsprings by using some of the standard binutil tools.

And he is wrong about the LD_LIBRARY_PATH being searched **after** the standard libraries. Think about it, it wouldn't serve its purpose if it did.

- - - -

I would suggest that you take an "example" program that will not run, clean all of the environment variables (i.e: "unset ..." them), and then "export LD_DEBUG= <whatever helps>" to see what the problem is.

Ref:
http://tldp.org/HOWTO/Program-Librar...libraries.html
Page down to the usb-section 3.3.2 on LD_DEBUG

[twobob]

okay - will do that then.

see how I do : ) Thanks.

Checked your posts on the BR threads about the Goose

also burned several hours seeing if *I* could determine whether FB support really does exist in Webkit or not anymore (sadly I fear the answer is no.. but I may be wrong)

Hopefully the likely shift to Qt will happen sooner rather than later.

Right then. back to breaking stuff.

[twobob]

Originally Posted by knc1

There is probably a "spec" on this, somewhere, I don't know the reference off-hand.

An example:
/mnt/us/usr/bin:/mnt/us/bin:/usr/local/bin:/usr/bin:/bin
Same for sbin

And invert the order of ld_library_path - same reasoning, the most specific first.
Hmm... Once you do that, you might not have to pre-load the "C" library.

Hmm I thought this looked familiar. That was the first thing I ever did IIRC.
Dies on the threading libraries not being able to create virtual directories on the FAT system I believe.

Which is the reasoning behind attempting to force the multi-threaded apps to use specifically the version that exists on the extfs partition.

I'll go back and check this but essentially the straces on my early reports of why stuff dies on FAT indicate this.

Let me just check.

[twobob]

EDIT: EVERYTHING AFTER THE TOP THREAD IS NOW NOT reffing the TOP THREAD SCRIPT. IT HAS BEEN UPDATED TO THE LATEST VERSION. JUST TO BE CLEAR. THANKS

---------------------------------------------------------------

Okay well it DIDNT die in the way I expected.

using knc.sh like: source /mnt/us/knc.sh

where knc.sh =

PHP Code:

#!/bin/sh
export LD_LIBRARY_PATH=/mnt/us/usr/lib:/mnt/us/lib:/usr/lib:/lib
export PATH=/mnt/us/usr/bin:/mnt/us/bin:/mnt/us/usr/sbin:/mnt/us/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
unset LD_PRELOAD 


I get the ld fail again. (pictured, 3rd thing).
But it doesn't die (yet) in the way I expected.
so guess I need another way to skin this cat.

I'll have a look at the

3.3.2. LD_DEBUG

Another useful environment variable in the GNU C loader is LD_DEBUG. This triggers the dl* functions so that they give quite verbose information on what they are doing. For example:
export LD_DEBUG=files
command_to_run
displays the processing of files and libraries when handling libraries, telling you what dependencies are detected and which SOs are loaded in what order. Setting LD_DEBUG to ``bindings'' displays information about symbol binding, setting it to ``libs'' displays the library search paths, and setting ti to ``versions'' displays the version depdendencies.

Setting LD_DEBUG to ``help'' and then trying to run a program will list the possible options. Again, LD_DEBUG isn't intended for normal use, but it can be handy when debugging and testing.

stuff then.

In the picture you can see

1) it running correctly with my wangle.
2) it starting to fail as I remove the specific call to the right zlib (in this instance)
3) it failing completely when just using the lib and path reordering.

I'll have a think. : )

[knc1]

I think the last line shown in that attached image tells the whole story.

Note the reference to release 3 of the program loader.

Doesn't the K3 use the ld-linux.so.2 (earlier version) of the loader rather than ld-linux.so.3?

Hmm...

Your building on Linux-Mint-13, correct?
That is based on Ubuntu-12.04 (Precise Pangolin) - which is the release that switched to the newer loader (*.3) and for ARM, started with the multi-lib/multi-arch support.

My system, Linux-Mint-12 is based on Ubuntu-11.10 (Oneiric Ocelot) - which still used the older *.2 loader.

Would be nice to know how the reference to the *.3 loader slipped into the build. (Leaked in from the build host? Reference from inside of the CS/MG toolchain? Buildroot built glibc that way?)
A mystery that would be nice to solve, but might be hard to do.

You have a copy of the *.3 loader in the build?
Use the explicit loader call rather than the implicit call.
Like (section 3.3.1):
${LOADER} --library-path PATH EXECUTABLE

Guessing at where the loaders are at in your build -

LOADER=/lib/ld-linux.so.2 And you should have the same failures;

LOADER=/mnt/us/lib/ld-linux.so.3 And it will work or display some other symptom.

Use whatever object utility you have handy to dump the sysmbols of the experimental victum application, filtered by | grep "NEEDED" **Not** ldd!

That will give you the exact libraries names required, use the full pathname to them in the PATH part of the explicit loader call.

Once the complete fix is determined, if it involves using the new loader, then it may be possible to replace the Kindle's loader with the new one.
I.E: the *.3 loader should be able to load any of the Amazon objects build for being loaded by the *.2 loader.

Be real certain to also test loading some of the Amazon originals with the new loader and the specific path to the old libraries before replacing the old one with the new one.
I.E: Try to avoid adding to the "successful de-brick" reports.

[twobob]

okay Ill give that a go.

Many thanks mate. superstar.

I'm perusing the BR mailing list archives to get up to speed on the politics. Cheers.

Just going to use Nilujes patch to get the headers out now I think I have finally worked out how to generate them without trashing my main system. hopefully.

: )

[NiLuJe]

@twobo/knc1: The K3 does use ld-linux.so.3 (while my x86 host uses ld-linux.so.2. ARM ABI specific thingy?).

@twobob: That doesn't change the fact that the binutils ld you built seems to want to rely on the ld helper from the glibc you built, not the one shipped with the K3. (EDIT: because your 'new' glibc is seen before the K3's with your custom LD_LIBRARY_PATH, which I guess is what the LD_PRELOAD hack was used to workaround, although I have *no* idea if it's safe, or even a good idea at all. Having to use LD_PRELOAD isn't usually a great thing to begin with, and preloading the glibc does seem a bit extreme to me ^^).

Use readelf -l to check what ld helper your binaries are looking for, which is probably /lib/ld-linux.so.3 right now.

Keep in mind that ld-linux is a part of glibc, and I've never tried what you're doing (ie. loading a newer glibc with an older loader), but it's probably not all that surprising that it's failing . The 'best' way to handle this is probably to use the newer version of everything (glibc AND loader), or the older version of everything (which is what I'm usually doing).

As for how to achieve that (using the new loader), besides replacing the Kindle's loader in the rootfs (which is not necessarily safe, like knc1 said, check, check, and triple check, before you try it ^^), I have no idea. Messing with the INTERP header on your custom binaries?

EDIT²: A quick google search leads to this utility: http://nixos.org/patchelf.html
So I guess switching the interp header in an elf binary isn't so crazy after all .
The info about the dynamic-loader LDFLAGS is good to know, if you want to go that way without having to patch every single binaries after building them .

[knc1]

The tool-chain I found, the one that Twobob is using, is only a "close match" to what lab126 used.

The gcc-id in the lab126 binaries I looked at where only a generic ID. So there is no telling what **specific** 4.1 series tool-chain they used. Perhaps the lack of anything more specific means they built it themselves. If so - who knows with what options.

- - - -

What version of the K3 firmware?
It could be that your machine has had the loader OTA updated (replaced).
Since I don't think the ld-linx.so.3 existed when the K3's where first marketed.

- - - -

If 2Bob does have ld-linux.so.3 on his K3 - then we need a complete header and symbol dump to determine what the error message is trying to say.

With the same for the ld-linux.so.X that the build system churned out.

nm should give code letters for the symbol types.

[NiLuJe]

@knc1: I checked my K3 rootfs dump, which was done back in the 3.0.something days, IIRC. (Yup, checked the version file, 3.0.2).

See my previous message, after multiple edits, I think I got a handle on what the issue is. (short answer: loading the custom built glibc 2.x libc with the vanilla glibc-2.5 loader from the Kindle).

I haven't checked the glibc version twobob built, but I'm guessing it's far, far more recent than the Kindle's glibc 2.5, which might be the issue here, if you want it to work without resorting to using a custom loader.

[knc1]

Yeah, I just checked the one image I have (3.2.1) and ld-linux.so.3 is a symlink to ld-2.5.so
The file dates are 4/6/2011

- - - -

Whatever, trying both loaders with an explicit call and a fixed list of libraries to load as I suggested in my above post will tell us more.