Software Accurately Predicts Books' Popularity By Analyzing Their Sentences

[drjenkins]

Popular Science reports that a new software program can predict the popularity of a book by analyzing the writing style.

http://www.popsci.com/article/techno...heir-sentences

I wonder if you could revere engineer the software to generate a best seller written by a computer?

[eschwartz]

Maybe that's where the news comes from.

[morriss003]

Originally Posted by drjenkins

Popular Science reports that a new software program can predict the popularity of a book by analyzing the writing style.

http://www.popsci.com/article/techno...heir-sentences

I wonder if you could revere engineer the software to generate a best seller written by a computer?

Perhaps this has already occurred.

[MichelleLouring]

I'm pretty sure I read about software designed to write actual fiction somewhere... I wonder if it has better grammar than some of the latest best sellers or if it's part of the package!

[elemenoP]

Very interesting but note that the software is analyzing public domain books. So I read this more as "software accurately predicts old books that still hold up to contemporary audiences."

eP

[Catlady]

Deleted

[pdurrant]

It seemed to me that they haven't actually tested it's predictions so far, just analysed a set of books to try to find common factors in the popular ones.

Surely the next test must be running their algorithm against a completely new set of test books, and seeing if the can identify the popular ones in that set.

[Doitsu]

Originally Posted by elemenoP

Very interesting but note that the software is analyzing public domain books. So I read this more as "software accurately predicts old books that still hold up to contemporary audiences."

IMHO, high-download counts of PG books don't necessarily correlate with high popularity per se, since many of the most frequently downloaded PG books are required reading at schools and universities and many downloaders most likely chose to download free PG books to save money.

BTW, I've also learned a new word from the study: unigram = [single] word.

[j.p.s]

Originally Posted by drjenkins

I wonder if you could revere engineer the software to generate a best seller written by a computer?

That would probably result in the literary equivalent of the uncanny valley of computer graphics animation. http://en.wikipedia.org/wiki/Uncanny_valley

Synthesis is quite different from analysis.

[Namekuseijin]

Software sentences books to death by consumption

[FatDog]

Forget about popularity - what if it can analyze and create a "DNA" profile of each book. Then you group the similar genomes together.

Like Lois McMaster Bujold - the groupings can predict what other books or authors you would also like.

In truth - Amazon already suggests similar books based on previous shoppers. But this is based on searching and shopping statistics. Not that people really "like" the second book they buy.

[latepaul]

Originally Posted by FatDog

Forget about popularity - what if it can analyze and create a "DNA" profile of each book. Then you group the similar genomes together.

You mean like Booklamp?

[QuantumIguana]

Originally Posted by pdurrant

It seemed to me that they haven't actually tested it's predictions so far, just analysed a set of books to try to find common factors in the popular ones.

Surely the next test must be running their algorithm against a completely new set of test books, and seeing if the can identify the popular ones in that set.

Agreed, if they are really onto something, they should be able to use this to predict which new books should be popular. There are a number of alternative explanations for why they got these results with PG books. If publishers were to adopt this algorithm when they select books, then authors would face pressure to tailor their books to match the algorithm. This could lead to books becoming more similar, and we ought to at least find out if it really holds up first.

[drjenkins]

Originally Posted by QuantumIguana

If publishers were to adopt this algorithm when they select books, then authors would face pressure to tailor their books to match the algorithm.

That would open "opportunities" for consultants to game the publishing system the way they try to skew Google search rankings today.

[Rbneader]

Originally Posted by drjenkins

That would open "opportunities" for consultants to game the publishing system the way they try to skew Google search rankings today.

That wouldn't be ideal, but I'm not going to be upset if more authors start writing more effectively. The tool so far seems to confirm one principle of good writing, and a way to get concrete feedback on that might not be a bad thing. Since this is a university program, I'd expect it to go open-source at some point and then it's free for everyone to use.