I wonder if this would work, I have no idea how feasible it would be, so don't be mad:
1. Choose three random words which appear at least a few times. Excluding "a", "the", "in" and such.
2. Remove all punctuation and spacing.
3. Calculate letters between those words.
4. Generate a hash which would contain information about those three words, number of times the words appear and number of letters in-between each appearance.
5. When identifying a book, to prevent headers and formatting from interfering just set a certain threshold at which identification would be set as positive. If a certain word is almost always nearly the same distance from other word, then set it as positive.
Example (random quote from one book):
Quote:
Lawrence was holding the next to last sheet up to Prime Intellect's TV eye when the phone rang. "They didn't believe me. I'm shitcanned," Stebbins said.
"Didn't believe you about what?"
"The papers man, the goddamn Correlation Effect papers. I'm gonna kill you for this, I really am."
"The papers are right here. I just got through showing them to Prime Intellect. You need them back?"
"It don't matter now, I don't work here any more." There was a pause. "I bet they're gonna put you in jail for this."
Prime Intellect's face disappeared from the TV, and words began to scroll across the screen:
* JOHN TAYLOR IS IN THE ROOM WITH HIM. HE IS DIRECTING STEBBINS.
Lawrence read this as he talked. "Jail for what? I just borrowed the papers to see if Prime Intellect could expand on them."
Another pause. "What? It didn't come up with anything, did it?"
"Well, it's..." (Why do you care if you've just been fired? Lawrence wondered.)
* STEBBINS IS LYING. HE WENT TO TAYLOR AS SOON YOU LEFT AND TOLD HIM THAT YOU BROUGHT THEM TO ME.
"...too early..."
* TELL HIM YES.
"Actually, I think it's just noticed something. Hang on."
* TELL HIM IT POINTS TO A NEW FORM OF COSMOLOGY WHICH THEY DID NOT CONSIDER. INFINITE RANGE IS PROBABLY POSSIBLE WITH EXISTING HARDWARE. TELEPORTATION OF MATTER IS PROBABLY POSSIBLE.
Prime Intellect paused a moment, and the words PROBABLY were replaced with DEFINITELY.
Lawrence blinked, then typed into the little-used keyboard of his console,
> Is this true?
* YES.
"It says it will give you the stars," Lawrence said flatly.
"What? You been eating mushrooms, Lawrence? Lawrence?"
> What will it take to implement this?
* LET ME TRY SOMETHING.
"It says it will give you the stars. It says your faster than light chips can be made to work at infinite range. It says you can teleport matter."
Now there was a long, long pause. "That's bullshit," Stebbins finally said. "We tried everything."
Lawrence heard a small uproar through the phone, an uproar that would have been very loud on Stebbins' end. Men were arguing. A loud voice (Military Mitchell's, Lawrence thought) bellowed, "WHAT THE FUCK DO YOU MEAN?" Then there was the faint pop of a door slamming in the background.
* I'VE GOT IT. HANG ON.
None of them knew it at the time, but that was really the moment the world changed.
|
Same quote with everything that's not needed stripped off:
Quote:
|
LawrencewasholdingthenexttolastsheetuptoPrimeIntel lectsTVeyewhenthephonerangTheydidntbelievemeImshit cannedStebbinssaidDidntbelieveyouaboutwhatThepaper smanthegoddamnCorrelationEffectpapersImgonnakillyo uforthisIreallyamThepapersarerighthereIjustgotthro ughshowingthemtoPrimeIntellectYouneedthembackItdon tmatternowIdontworkhereanymoreTherewasapauseIbetth eyregonnaputyouinjailforthisPrimeIntellectsfacedis appearedfromtheTVandwordsbegantoscrollacrossthescr eenJOHNTAYLORISINTHEROOMWITHHIMHEISDIRECTINGSTEBBI NSLawrencereadthisashetalkedJailforwhatIjustborrow edthepaperstoseeifPrimeIntellectcouldexpandonthemA notherpauseWhatItdidntcomeupwithanythingdiditWelli tsWhydoyoucareifyouvejustbeenfiredLawrencewondered STEBBINSISLYINGHEWENTTOTAYLORASSOONYOULEFTANDTOLDH IMTHATYOUBROUGHTTHEMTOMEtooearlyTELLHIMYESActually IthinkitsjustnoticedsomethingHangonTELLHIMITPOINTS TOANEWFORMOFCOSMOLOGYWHICHTHEYDIDNOTCONSIDERINFINI TERANGEISPROBABLYPOSSIBLEWITHEXISTINGHARDWARETELEP ORTATIONOFMATTERISPROBABLYPOSSIBLEPrimeIntellectpa usedamomentandthewordsPROBABLYwerereplacedwithDEFI NITELYLawrenceblinkedthentypedintothelittleusedkey boardofhisconsoleIsthistrueYESItsaysitwillgiveyout hestarsLawrencesaidflatlyWhatYoubeeneatingmushroom sLawrenceLawrenceWhatwillittaketoimplementthisLETM ETRYSOMETHINGItsaysitwillgiveyouthestarsItsaysyour fasterthanlightchipscanbemadetoworkatinfiniterange ItsaysyoucanteleportmatterNowtherewasalonglongpaus eThatsbullshitStebbinsfinallysaidWetriedeverything Lawrenceheardasmalluproarthroughthephoneanuproarth atwouldhavebeenveryloudonStebbinsendMenwerearguing AloudvoiceMilitaryMitchellsLawrencethoughtbellowed WHATTHEFUCKDOYOUMEANThentherewasthefaintpopofadoor slamminginthebackgroundIVEGOTITHANGONNoneofthemkne witatthetimebutthatwasreallythemomenttheworldchang ed
|
Say we make the 3 words: tell (7 occurances), said (3 occurances), just (4 occurances)
Tell: 1-222-2-108-3-186-4-203-5-49-6-152-7
Said: 1-1047-2-260-3
Just: 1-297-2-127-3-134-4
It appears that 1st "tell" from the second is 222 letters apart, 2nd from 3rd - 108 letters apart and so on.
So once the hash is generated, it's up to statistics to decide whether the book in question is the same.