Google PageRank Checksum Algorithm

[alirezan]

Hi guys,

I still can't get it to work. Has anybody got a working version? Can you post the code here so that we all can use it?

Thanks

Ali

[ontananza]

Hi there:

This is a working 64bit version of the google checksum algorithm.

Enjoy it

TO use it :

$checksum= new google_checksum();

$ch="6".$checksum->getGoogleChecksum('domain.com');

And the class:

<?PHP
/*
Written and contributed by
Alex Stapleton,
Andy Doctorow,
Tarakan,
Bill Zeller,
Vijay "Cyberax" Bhatter
traB
This code is released into the public domain
*/

define('GOOGLE_MAGIC', 0xE6359A60);

class google_checksum{

var $checksum;

function google_checksum(){
$this->checksum='';
}

//unsigned shift right
function zeroFill($a, $b){
$z = hexdec(80000000);
if ($z & $a)
{
$a = ($a>>1);
$a &= (~$z);
$a |= 0x40000000;
$a = ($a>>($b-1));
}
else
{
$a = ($a>>$b);
}
return $a;
}

function mixOld($a,$b,$c) {
$a -= $b; $a -= $c; $a ^= ($this->zeroFill($c,13));
$b -= $c; $b -= $a; $b ^= ($a<<8);
$c -= $a; $c -= $b; $c ^= ($this->zeroFill($b,13));
$a -= $b; $a -= $c; $a ^= ($this->zeroFill($c,12));
$b -= $c; $b -= $a; $b ^= ($a<<16);
$c -= $a; $c -= $b; $c ^= ($this->zeroFill($b,5));
$a -= $b; $a -= $c; $a ^= ($this->zeroFill($c,3));
$b -= $c; $b -= $a; $b ^= ($a<<10);
$c -= $a; $c -= $b; $c ^= ($this->zeroFill($b,15));

return array($a,$b,$c);
}

function mix($a,$b,$c) {
$a -= $b; $a -= $c; $this->toInt32($a); $a = (int)($a ^ ($this->zeroFill($c,13)));
$b -= $c; $b -= $a; $this->toInt32($b); $b = (int)($b ^ ($a<<8));
$c -= $a; $c -= $b; $this->toInt32($c); $c = (int)($c ^ ($this->zeroFill($b,13)));
$a -= $b; $a -= $c; $this->toInt32($a); $a = (int)($a ^ ($this->zeroFill($c,12)));
$b -= $c; $b -= $a; $this->toInt32($b); $b = (int)($b ^ ($a<<16));
$c -= $a; $c -= $b; $this->toInt32($c); $c = (int)($c ^ ($this->zeroFill($b,5)));
$a -= $b; $a -= $c; $this->toInt32($a); $a = (int)($a ^ ($this->zeroFill($c,3)));
$b -= $c; $b -= $a; $this->toInt32($b); $b = (int)($b ^ ($a<<10));
$c -= $a; $c -= $b; $this->toInt32($c); $c = (int)($c ^ ($this->zeroFill($b,15)));
return array($a,$b,$c);
}

//converts a string into an array of integers containing the numeric value of the char
function strord($string){
for($i=0;$i<strlen($string);$i++) {
$result[$i] = ord($string{$i});
}
return $result;
}

// calculates the google checksum
function GoogleCH($url, $length=null, $init=GOOGLE_MAGIC){

if(is_null($length)) {
$length = sizeof($url);
}
$a = $b = 0x9E3779B9;
$c = $init;
$k = 0;
$len = $length;
while($len >= 12) {
$a += ($url[$k+0] +($url[$k+1]<<8) +($url[$k+2]<<16) +($url[$k+3]<<24));
$b += ($url[$k+4] +($url[$k+5]<<8) +($url[$k+6]<<16) +($url[$k+7]<<24));
$c += ($url[$k+8] +($url[$k+9]<<8) +($url[$k+10]<<16)+($url[$k+11]<<24));
$mix = $this->mix($a,$b,$c);
$a = $mix[0]; $b = $mix[1]; $c = $mix[2];
$k += 12;
$len -= 12;
}

$c += $length;
switch($len) /* all the case statements fall through */
{
case 11: $c+=($url[$k+10]<<24);
case 10: $c+=($url[$k+9]<<16);
case 9 : $c+=($url[$k+8]<<8);
/* the first byte of c is reserved for the length */
case 8 : $b+=($url[$k+7]<<24);
case 7 : $b+=($url[$k+6]<<16);
case 6 : $b+=($url[$k+5]<<8);
case 5 : $b+=($url[$k+4]);
case 4 : $a+=($url[$k+3]<<24);
case 3 : $a+=($url[$k+2]<<16);
case 2 : $a+=($url[$k+1]<<8);
case 1 : $a+=($url[$k+0]);
/* case 0: nothing left to add */
}
$mix = $this->mix($a,$b,$c);
/*-------------------------------------------- report the result */

return $mix[2];
}

// converts an array of 32 bit integers into an array with 8 bit values. Equivalent to (BYTE *)arr32
function c32to8bit($arr32){
for($i=0;$i<count($arr32);$i++) {
for ($bitOrder=$i*4;$bitOrder<=$i*4+3;$bitOrder++) {
$arr8[$bitOrder]=$arr32[$i]&255;
$arr32[$i]=zeroFill($arr32[$i], 8);
}
}
return $arr8;
}

// gets the google checksum!
function getGoogleChecksum($url){

$url="info:"."http://".str_replace('http://','',$url);

$tmp_ch=$this->strord($url);

$this->checksum=sprintf("%u", $this->GoogleCH($tmp_ch));

return $this->checksum;
}

// converts 64 bit 2 32 bit
function toInt32(& $x){
$z = hexdec(80000000);
$y = (int)$x;
// on 64bit OSs if $x is double, negative ,will return -$z in $y
// which means 32th bit set (the sign bit)
if($y==-$z&&$x<-$z){
$y = (int)((-1)*$x);// this is the hack, make it positive before
$y = (-1)*$y; // switch back the sign
//echo "int hack <br>";
}
$x = $y;
}

}

?>

[dbase]

I have read every post here, and my php skills are not what I thought they were. I have worked at getting a 32bit version working on my PHP4 box for several hours. Could someone please post the newest working version.

[AboutSledge]

Originally Posted by alexanderg

Url http://www.gamesaga.net/pagerank.php is not valid and site http://www.gamesaga.net it seems does not contain information about PageRank anymore.

Do you know another resources with description of the new GooglePR algorithm version 3+ ?

http://home.zhiwei.li/pagerank/

Google Toolbar 5.0.x for IE (difficult)
Google Toolbar for Firefox (easy)

[saf]

I tried the code below, but I also could not get it to work on my 64 bit server.
I added the lines

PHP Code:

$url = "www.google.co.uk";
echo "PageRank of ".$url." is: ".get_page_rank($url); 


to the end of the function, so it would get googles PR (just as a test)

but whenever it was run it gives a value of -1 which is that thing that happens when I use the code for a 32 bit machine.

Can anyone help me (all of us) on this? Maybe someone has got it working, if so can you please post the code here.

Thanks

Originally Posted by Gagget

Sorry, I have completly forgotten to post my code here. It´s tested by some people now and it works very fine.
To make it easier to update existing scripts, i commented the changes.

PHP Code:

/* 
        Written and contributed by 
        Alex Stapleton, 
        Andy Doctorow, 
        Tarakan, 
        Bill Zeller, 
        Vijay "Cyberax" Bhatter 
        traB 
    This code is released into the public domain 
*/ 
define('GOOGLE_MAGIC', 0x00000000E6359A60); // CHANGED (64Bit)

//unsigned shift right 
function zeroFill($a, $b) 
{   
        $z = 0x0000000080000000;  // CHANGED (64Bit)
        $a = $a & 0x00000000FFFFFFFF;  // ADDED (64Bit)
        
        if ($z & $a) 
        {     
            $a = ($a>>1); 
            $a &= (~$z); 
            $a |= 0x0000000040000000;  // CHANGED (64Bit)
            $a = ($a>>($b-1)); 
        } 
        else 
        {     $a = ($a>>$b);    
        } 
        return $a; 
} 
  
  
function mix($a,$b,$c) { 
    
  $a = $a & 0x00000000FFFFFFFF;  // ADDED (64Bit)
  $b = $b & 0x00000000FFFFFFFF;  // ADDED (64Bit)
  $c = $c & 0x00000000FFFFFFFF;  // ADDED (64Bit)
  
  $a -= $b; $a -= $c; $a ^= (zeroFill($c,13)); 
  $b -= $c; $b -= $a; $b ^= ($a<<8); 
  $c -= $a; $c -= $b; $c ^= (zeroFill($b,13)); 
  $a -= $b; $a -= $c; $a ^= (zeroFill($c,12)); 
  $b -= $c; $b -= $a; $b ^= ($a<<16); 
  $c -= $a; $c -= $b; $c ^= (zeroFill($b,5)); 
  $a -= $b; $a -= $c; $a ^= (zeroFill($c,3));   
  $b -= $c; $b -= $a; $b ^= ($a<<10); 
  $c -= $a; $c -= $b; $c ^= (zeroFill($b,15)); 

  return array($a,$b,$c); 
} 
 
 
function GoogleCH($url, $length=null, $init=GOOGLE_MAGIC) { 
    if(is_null($length)) { 
        $length = sizeof($url); 
    } 
    $a = $b = 0x000000009E3779B9;  // CHANGED (64Bit)
    
    $c = $init ; 
    $k = 0; 
    $len = $length; 
    while($len >= 12) { 
        $a += ($url[$k+0] +($url[$k+1]<<8) +($url[$k+2]<<16) +($url[$k+3]<<24)); 
        $b += ($url[$k+4] +($url[$k+5]<<8) +($url[$k+6]<<16) +($url[$k+7]<<24)); 
        $c += ($url[$k+8] +($url[$k+9]<<8) +($url[$k+10]<<16)+($url[$k+11]<<24)); 
        $mix = mix($a,$b,$c); 
        $a = $mix[0]; $b = $mix[1]; $c = $mix[2]; 
        $k += 12; 
        $len -= 12; 
    } 

    $c += $length; 
    switch($len)              /* all the case statements fall through */ 
    { 
        case 11: $c+=($url[$k+10]<<24); 
        case 10: $c+=($url[$k+9]<<16); 
        case 9 : $c+=($url[$k+8]<<8); 
          /* the first byte of c is reserved for the length */ 
        case 8 : $b+=($url[$k+7]<<24); 
        case 7 : $b+=($url[$k+6]<<16); 
        case 6 : $b+=($url[$k+5]<<8); 
        case 5 : $b+=($url[$k+4]); 
        case 4 : $a+=($url[$k+3]<<24); 
        case 3 : $a+=($url[$k+2]<<16); 
        case 2 : $a+=($url[$k+1]<<8); 
        case 1 : $a+=($url[$k+0]); 
         /* case 0: nothing left to add */ 
    } 
    $mix = mix($a,$b,$c); 
    /*-------------------------------------------- report the result */ 
    return $mix[2]; 
} 

//converts a string into an array of integers containing the numeric value of the char 
function strord($string) { 
    for($i=0;$i<strlen($string);$i++) { 
        $result[$i] = ord($string{$i}); 
    } 
    return $result; 
} 

// converts an array of 32 bit integers into an array with 8 bit values. Equivalent to (BYTE *)arr32 
function c32to8bit($arr32) { 
    for($i=0;$i<count($arr32);$i++) { 
        for ($bitOrder=$i*4;$bitOrder<=$i*4+3;$bitOrder++) { 
            $arr8[$bitOrder]=$arr32[$i]&255; 
            $arr32[$i]=zeroFill($arr32[$i], 8); 
        }     
    } 
    return $arr8; 
} 


function GoogleCHNew($ch){
    $ch=sprintf("%u", $ch); 
    $ch = ((($ch/7) << 2) | (((int)fmod($ch,13))&7)); 
    $prbuf = array(); 
    $prbuf[0] = $ch; 
    for($i = 1; $i < 20; $i++) { 
      $prbuf[$i] = $prbuf[$i-1]-9; 
    } 
    $ch = GoogleCH(c32to8bit($prbuf), 80); 
    return sprintf("%u", $ch);
}


function get_page_rank($url){
    $url = preg_replace('/\?.*$/','?',$url);
    $reqgr = "info:".$url;
    $reqgre = "info:".urlencode($url);
    $gch = GoogleCH(strord($reqgr));
    
    $gch = "6".GoogleCHNew($gch);
    

  
    $patern = '/^http:/';
    $patern2 = '/^http:\/\/.*google\..*\/(search|images|groups|news).*/';
    $patern3 = '/^http:\/\/localhost.*/';
    $patern4 = '/^http:\/\/(127\.|10\.|172\.16|192\.168).*/'; //local ip
    if(!preg_match($patern, $url) || preg_match($patern2, $url) ||
       preg_match($patern3, $url) || preg_match($patern4, $url)){
           return -1;
    }else{
        // BEGIN CHANGES (fsockopen to request PR) 
        
        $fsock = fsockopen('toolbarqueries.google.com', 80, $errno, $errstr);
        if ( !$fsock ){  
            return -1;
        }
        $base_get = "/search?client=navclient-auto&ch=".$gch."&ie=UTF-8&oe=UTF-8&features=Rank:FVN&q=".$reqgre;
        fputs($fsock, "GET $base_get HTTP/1.1\r\n");
        fputs($fsock, "HOST: toolbarqueries.google.com\r\n");
        fputs($fsock, "User-Agent: Mozilla/4.0 (compatible; GoogleToolbar 2.0.114-big; Windows XP 5.1)\r\n");
        fputs($fsock, "Connection: close\r\n\r\n");
        while(!feof($fsock)){ 
            $res['content'] .= fread($fsock, 1024);
        }
        fclose($fsock);

        // END CHANGES (fsockopen to request PR)
    
        if(preg_match('/Rank_.*?:.*?:(\d+)/i', $res['content'], $m)){    
            return $m[1];
        }else{    
            return -1;
        }      
    }
} 
?> 


[saf]

OK i got it working now

[saf]

Ok, I thought I had it working but some websites give an error. Can anyone help me with this?

Thanks

[bonusrobber]

I get
int(-6288256054)
int(-6288256054)
int(-6288256054)
int(-6288256054)

Host provider is bluehost and PHP 4.4.4 (cgi)
Linux 2.6.17-11_1.BHsmp #1 SMP Thu Aug 24 09:39:29 MDT 2006 x86_64 x86_64 x86_64 GNU/Linux

Not a single example here to fetch pr work there =(

Me
http://www.findmatch.org
100% Free dating & penpals service

Originally Posted by AboutSledge

PHP Data Type Converting Bug.


<?php

//header("Content-Type: text/plain; charset=utf-8");

$n = -6288256054;
var_dump($n);

$a = intval($n);
var_dump($a);

$b = (int) $n;
var_dump($b);

settype($n,"int");
var_dump($n);

?>

============good==========
float(-6288256054)
int(-1993288758)
int(-1993288758)
int(-1993288758)


============bad=================

float(-6288256054)
int(-2147483648)
int(-2147483648)
int(-2147483648)

[hm2k]

I've tried all the scripts on here, and I appear to have a problem generating the ch correctly, which is apparently down to an issue with the way certain versions of PHP handle the bitwise operations. Any ideas on a fix?

I have documented my findings here: http://www.googlecommunity.com/about14098.html

My Server information:
Operating system CentOS Linux
Kernel version 2.6.8-022stab078.14-smp
Machine Type i686
Apache version 1.3.37 (Unix)
PERL version 5.8.7
PHP version 4.4.4
MySQL version 4.1.21-standard
cPanel Build 10.8.2-RELEASE 119

If you think you can resolve this problem, I will give you an account on my server to attempt a solution.

[AboutSledge]

Originally Posted by hm2k

I've tried all the scripts on here, and I appear to have a problem generating the ch correctly, which is apparently down to an issue with the way certain versions of PHP handle the bitwise operations. Any ideas on a fix?

I have documented my findings here: http://www.googlecommunity.com/about14098.html

My Server information:
Operating system CentOS Linux
Kernel version 2.6.8-022stab078.14-smp
Machine Type i686
Apache version 1.3.37 (Unix)
PERL version 5.8.7
PHP version 4.4.4
MySQL version 4.1.21-standard
cPanel Build 10.8.2-RELEASE 119

If you think you can resolve this problem, I will give you an account on my server to attempt a solution.

Maybe I can resolve it.

GTalk/MSN/E-Mail: anykai [at] gmail <dot> com

[hm2k]

There is still no working solution for people with newer version of PHP...

[AboutSledge]

google toolbar 4.0.x checksum algo

Server-side scripting demo
http://pagerank.gamesaga.net/

PHP Server-side scripting source code
http://pagerank.gamesaga.net/pagerank.zip

source code for Python
http://pagerank.gamesaga.net/pagerank.txt

source code for C
http://pagerank.gamesaga.net/pagerank.c


2005-09-13 v0.1 first release
2005-10-21 v1.1 fix a bug for the final character
2006-09-21 v1.2 compatible with PHP 4.4/PHP 5.x
2006-09-29 v1.3 X86_64 CPU supported


Any suggestions and feedback is welcome.

ps:
this script works on hm2k's server which is i686 arch

[hm2k]

In addition to this, i've released some background about this here:

http://www.hm2k.com/projects/pagerank/

[rak]

nice work

[Raja Amer Khan]

Hello guys!


This script is awesome!

thank you..


Now i want some other help..

I want to save the pagerank in mysql, because i have lots of websites saved in mysql and i displayed sort option for other columns, but i cannot sort by page rank.

So is there any option so i could save pagerank function in my mysql or any other solution to make it sortable?

thank you for quick reply..