Ebook tagging service (idea phase)

[jorm]

the idea is not bad. The only catch is how to reliably identify the same 3 words in all formats of the book. Is it fair to have a small dictionary of words that we use? I think this would be an interesting project and would be fun to see how large and accurate a database we could build.

It would also save people potentially thousands of man hours of organization if we could get this widely into use.

The key is even if we don't get a 1-1 match for all editions it should not matter eventually you will have fingerprints for the other editions and those would be tagged as well.

I am still looking for volunteers.

[Fanas]

Small dictionary of words is the way to go, there are words in any language that are used often enough to appear many times in any book, obviously 1-3 letter words would appear to often so those would be off limits. But any 4-5 letter long words would be both abundant and yet not so much as to make hashing way too resource consuming. Though someone else with more know-how should decide on algorithms to be used for identifying. I say, take the hash and scan through book until you find 2 consecutive occurrences of the word having same amount of letters in-between. When found you see if the next occurrence is correct as well, then you repeat until there's mismatch. After you finish scanning the book you've got more or less the right idea of how identical the two texts are.

[transmitthis]

Took me 2sec to identify
The Metamorphosis of Prime Intellect
A Novel by Roger Williams

Shame you can't use google - wonder what the state of AI is these days?

[Benji99]

Just want to my 2 cents to this discussion...

I spend a lot of time manually editing my book collection like I'm sure most of you guys do and no one else benefits but me.

I've also been thinking of how great a tagging service for books would be.
Instead of thinking of FreeDB, I was thinking of something closer to Musicbrainz of which I love for music. Ideally, the best would be get the musicbrainz developpers interested in this project as a book database has a lot in common with a music database.

[Fanas]

Originally Posted by transmitthis

Took me 2sec to identify
The Metamorphosis of Prime Intellect
A Novel by Roger Williams

Shame you can't use google - wonder what the state of AI is these days?

Well you are human, you are good at this sort of thing and it's a very good novel.

[Backi]

I took a look at the plug-ins for calibre and found this one:

[GUI Plugin] Find Similar Stories

I think, Ian_Stott did some preparatory work.

[transmitthis]

Originally Posted by Fanas

Well you are human, you are good at this sort of thing and it's a very good novel.

Will have to put it in my "toread" list.

The point I was making though, is it took a simple google search on some random text, and then another search on a link to positivity identify the book - so if the state of AI has improved since last I looked, maybe there are some options to explore in that area?
Probably too expensive - processor,monitory, and time but things are changing pretty quick in tech.

[Fanas]

Originally Posted by transmitthis

Will have to put it in my "toread" list.

The point I was making though, is it took a simple google search on some random text, and then another search on a link to positivity identify the book - so if the state of AI has improved since last I looked, maybe there are some options to explore in that area?
Probably too expensive - processor,monitory, and time but things are changing pretty quick in tech.

Yes, because the full text of novel is available free on the internet. With other books it might be different, you can't exactly scan torrents and rars of pirated books.