View Single Post
Old 09-28-2012, 04:04 PM   #84
knc1
Helpdesk Junkie
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 6,998
Karma: 6314536
Join Date: Feb 2012
Device: Too many.
DX(G) source checks

@twobob: Naw, I did that, see attached.

I recognize that the amount of documentation on LFA is a bit light, and what there is, is about auditing mips binaries.

So with the sources of DX(G) firmware as a test example ;
Using LFA under Lua-5.2 for the first time (it was written for Lua-5.1) ;
Without installing lfa.lua ;

A quick and dirty example :

Un-archive the parts of the three Amazon release bundles into parallel directories:
Code:
core2quad dxg $ ls -ld 2.*
drwxrwxr-x 4 mszick mszick 4096 2012-09-28 10:44 2.3_399380047
drwxrwxr-x 4 mszick mszick 4096 2012-09-28 10:44 2.5.5_495460008
drwxrwxr-x 4 mszick mszick 4096 2012-09-28 10:45 2.5.7_550650009
drwxrwxr-x 4 mszick mszick 4096 2012-09-28 10:45 2.5.8_555370010
We already know that the kernel and u-boot sources in 2.5.7 and 2.5.8 are the same - so we ignore 2.5.7 here.

Use sha1deep to build a list, by sha1sum of the three directories of interest, then sort that into a text file:
Code:
core2quad dxg $ sha1deep -r 2.3_399380047 2.5.5_495460008 2.5.8_555370010 | sort -n >ku_sources.sha1
Looking at the --help output of lfa, we can see we will need two magic numbers:
Code:
core2quad dxg $ /usr/local/bin/lua ../lfa/file_audit-1.0/lfa.lua --help
    --help        -    This text.
    -c Number    -    of records in each output file.
    -f Number    -    of the first path element of the target tree.
    -h        -    This text.
    -i Input_File    -    Without this option, reads stdin.
    -o OutBase    -    Base filename used for output filename construction.
    -p Number    -    of the path element containing the package name.
    -r CN        -    Common file names contained in package tree.
    -r CS        -    Common hash sums contained in package tree.
    -r List        -    of reports to produce from: CN,CS,NO,NP,SO,SP
    -r NO        -    File names ordered by number of occurances.
    -r NP        -    File names ordered by package name in package tree.
    -r SO        -    Hash sums ordered by number of occurances.
    -r SP        -    Hash sums ordered by package name in package tree.
There is a more detailed description of those as comments in the Lua script file.

Displaying the text file and grabbing a copy of the first record:
Code:
core2quad dxg $ less ku_sources.sha1
000ad28937f69ea948b567ae0ec973dfd022b592  /home/mszick/Downloads/dxg/2.3_399380047/linux-2.6.22/arch/powerpc/platforms/cell/Kconfig
We find the package name in field 5 of the record and the fileset name in field 6 of the record.
(Which gives us the: -p 5 -f 6 options.)
We know that we are using a more modern than version 2.4 of OpenOffice (and forked clones thereof), and we know there are less than 90,000 records in the input file (wc -l) so set the record count per output file to 90,000 (the -c 90000 option).

Toss in the input filename, the output filename prefix and the list of reports to generate gives us (in all its glory):
Code:
core2quad dxg $ /usr/local/bin/lua ../lfa/file_audit-1.0/lfa.lua -i ku_sources.sha1 -p 5 -f 6 -c 90000 -r CN,CS,NO,NP,SO,SP -o dx_ku_sources-
And in a handful of seconds, the script hands us back:
Code:
core2quad dxg $ ls -l dx_ku*
-rw-rw-r-- 1 mszick mszick  9976015 2012-09-28 12:36 dx_ku_sources-CN-p0001.csv
-rw-rw-r-- 1 mszick mszick 11894411 2012-09-28 12:36 dx_ku_sources-CS-p0001.csv
-rw-rw-r-- 1 mszick mszick 10077500 2012-09-28 12:36 dx_ku_sources-NO-p0001.csv
-rw-rw-r-- 1 mszick mszick 12380206 2012-09-28 12:36 dx_ku_sources-NP-p0001.csv
-rw-rw-r-- 1 mszick mszick 12005340 2012-09-28 12:36 dx_ku_sources-SO-p0001.csv
-rw-rw-r-- 1 mszick mszick 18754600 2012-09-28 12:36 dx_ku_sources-SP-p0001.csv
Standard comma separated values with a single record label header.
Suitable for import into your favorite data file viewer.

Here, I used OpenOffice to read the files, then output them with a touch of formatting as *.ods (Open Document Spreadsheet) files.

Code:
core2quad dxg $ ls -l *.ods
-rw-rw-r-- 1 mszick mszick 1835625 2012-09-28 13:09 dx_ku_sources-CN-p0001.ods
-rw-rw-r-- 1 mszick mszick 2089383 2012-09-28 13:00 dx_ku_sources-CS-p0001.ods
-rw-rw-r-- 1 mszick mszick 1763603 2012-09-28 13:10 dx_ku_sources-NO-p0001.ods
-rw-rw-r-- 1 mszick mszick 4217524 2012-09-28 13:13 dx_ku_sources-NP-p0001.ods
-rw-rw-r-- 1 mszick mszick 2015423 2012-09-28 13:18 dx_ku_sources-SO-p0001.ods
-rw-rw-r-- 1 mszick mszick 4674535 2012-09-28 13:24 dx_ku_sources-SP-p0001.ods
All ready to be browsed for things that are the same and things that are different (at the checksum level) of the three source bundles.

And/Or
For import into your favorite Database System (MySQL can import either the *.cvs or the *.ods format files for example).

Sorry folks, even though an *.ods file is another name for *.zip - this web-site does not know that, so the compressed files got compressed (again) for up-loading.
Attached Files
File Type: gz dx_ku_sources-CN-p0001.ods.gz (1.70 MB, 36 views)
File Type: gz dx_ku_sources-CS-p0001.ods.gz (1.86 MB, 32 views)
File Type: gz dx_ku_sources-NO-p0001.ods.gz (1.63 MB, 35 views)
File Type: gz dx_ku_sources-NP-p0001.ods.gz (3.95 MB, 37 views)
File Type: gz dx_ku_sources-SO-p0001.ods.gz (1.82 MB, 37 views)
File Type: gz dx_ku_sources-SP-p0001.ods.gz (4.12 MB, 35 views)
knc1 is offline   Reply With Quote