Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book General > Writers' Corner

Notices

Reply
 
Thread Tools Search this Thread
Old 02-05-2018, 12:01 PM   #1
Nabeel
Zealot
Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.
 
Nabeel's Avatar
 
Posts: 122
Karma: 611158
Join Date: Sep 2010
Location: Britain
Device: Kobo Aura One
Word use frequency - Is there a programme?

Dear All,

A question came up in my Creative Writing group: there must be a computer programme or app that, when given a text, will analyse how frequently different words are used.

Obviously, the really useful thing would be a programme that points out that you have unwittingly used a word like 'vast' five times in the same paragraph, but we're not looking for miracles.

Any advice or information would be very useful,

Nabeel
Nabeel is offline   Reply With Quote
Advert
Old 02-05-2018, 01:20 PM   #2
arjaybe
Guru
arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.
 
arjaybe's Avatar
 
Posts: 761
Karma: 5237178
Join Date: Aug 2013
Location: Canada
Device: Sony PRS-650
Calibre can do that. Select the book and edit it. Under Tools, select Reports.

Maybe there's an easier way, but ...
arjaybe is offline   Reply With Quote
Old 02-05-2018, 01:50 PM   #3
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 21,672
Karma: 20975546
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: K4NT, Galaxy Tab 2(RIP)
Spell check in Sigil or Calibre editor: Show all words, also gives a count for words of more than 1 letter.
theducks is offline   Reply With Quote
Old 02-05-2018, 06:03 PM   #4
gmw
cacoethes scribendi
gmw ought to be getting tired of karma fortunes by now.gmw ought to be getting tired of karma fortunes by now.gmw ought to be getting tired of karma fortunes by now.gmw ought to be getting tired of karma fortunes by now.gmw ought to be getting tired of karma fortunes by now.gmw ought to be getting tired of karma fortunes by now.gmw ought to be getting tired of karma fortunes by now.gmw ought to be getting tired of karma fortunes by now.gmw ought to be getting tired of karma fortunes by now.gmw ought to be getting tired of karma fortunes by now.gmw ought to be getting tired of karma fortunes by now.
 
gmw's Avatar
 
Posts: 3,579
Karma: 86147493
Join Date: Nov 2010
Location: Australia
Device: Kobo Aura One, Sony PRS-650
I expect most editing software will do this, I know that Editor from Serenity Software showed both word and phrase frequency counts (the latter can be rather interesting). Please note that I only ever trialled v4 of that software; I don't know what has happened in v5, so this is not a recommendation, just a suggestion of where to look.

A few posts on page 2 of this thread talk about editing software (with links).
gmw is offline   Reply With Quote
Old 02-06-2018, 12:52 AM   #5
skb
Addict
skb ought to be getting tired of karma fortunes by now.skb ought to be getting tired of karma fortunes by now.skb ought to be getting tired of karma fortunes by now.skb ought to be getting tired of karma fortunes by now.skb ought to be getting tired of karma fortunes by now.skb ought to be getting tired of karma fortunes by now.skb ought to be getting tired of karma fortunes by now.skb ought to be getting tired of karma fortunes by now.skb ought to be getting tired of karma fortunes by now.skb ought to be getting tired of karma fortunes by now.skb ought to be getting tired of karma fortunes by now.
 
skb's Avatar
 
Posts: 226
Karma: 1571730
Join Date: Mar 2010
Device: Ipod G4, MacOS 10.12, Calibre, Kindle Paperwhite
I think Scrivener has a word frequency feature.
skb is offline   Reply With Quote
Old 02-06-2018, 05:51 AM   #6
Nabeel
Zealot
Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.
 
Nabeel's Avatar
 
Posts: 122
Karma: 611158
Join Date: Sep 2010
Location: Britain
Device: Kobo Aura One
Much thanks for this useful advice. Calibre is amazing!

But...

When I run a spell check, it's obvious that Calibre is only looking at a part of the text: in this case, 8,000 words of a 73,000 word text. How do I get the edit function to look at the whole text?

Nabeel
Nabeel is offline   Reply With Quote
Old 02-06-2018, 05:57 AM   #7
Nabeel
Zealot
Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.
 
Nabeel's Avatar
 
Posts: 122
Karma: 611158
Join Date: Sep 2010
Location: Britain
Device: Kobo Aura One
Spoke too soon. The answer, obviously, is to untick 'show only mispelled words'.

Nabeel
Nabeel is offline   Reply With Quote
Old 02-17-2018, 04:39 AM   #8
Araucaria
Bibliophile
Araucaria ought to be getting tired of karma fortunes by now.Araucaria ought to be getting tired of karma fortunes by now.Araucaria ought to be getting tired of karma fortunes by now.Araucaria ought to be getting tired of karma fortunes by now.Araucaria ought to be getting tired of karma fortunes by now.Araucaria ought to be getting tired of karma fortunes by now.Araucaria ought to be getting tired of karma fortunes by now.Araucaria ought to be getting tired of karma fortunes by now.Araucaria ought to be getting tired of karma fortunes by now.Araucaria ought to be getting tired of karma fortunes by now.Araucaria ought to be getting tired of karma fortunes by now.
 
Araucaria's Avatar
 
Posts: 143
Karma: 934516
Join Date: Jul 2011
Location: Cantal in the French Auvergne
Device: Kindle Voyage, Moon Pro on Lenovo tablet
8,000 is a lot to be wrong. Perhaps a case of Jasper Fforde's mispeling vyrus?

Last edited by Araucaria; 02-17-2018 at 04:40 AM. Reason: inserting link
Araucaria is offline   Reply With Quote
Old 02-17-2018, 11:48 AM   #9
arjaybe
Guru
arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.arjaybe ought to be getting tired of karma fortunes by now.
 
arjaybe's Avatar
 
Posts: 761
Karma: 5237178
Join Date: Aug 2013
Location: Canada
Device: Sony PRS-650
Quote:
Originally Posted by Araucaria View Post
8,000 is a lot to be wrong. Perhaps a case of Jasper Fforde's mispeling vyrus?
It might just be the wrong dictionary.
arjaybe is offline   Reply With Quote
Old 02-17-2018, 02:01 PM   #10
Gregg Bell
Gregg Bell
Gregg Bell ought to be getting tired of karma fortunes by now.Gregg Bell ought to be getting tired of karma fortunes by now.Gregg Bell ought to be getting tired of karma fortunes by now.Gregg Bell ought to be getting tired of karma fortunes by now.Gregg Bell ought to be getting tired of karma fortunes by now.Gregg Bell ought to be getting tired of karma fortunes by now.Gregg Bell ought to be getting tired of karma fortunes by now.Gregg Bell ought to be getting tired of karma fortunes by now.Gregg Bell ought to be getting tired of karma fortunes by now.Gregg Bell ought to be getting tired of karma fortunes by now.Gregg Bell ought to be getting tired of karma fortunes by now.
 
Gregg Bell's Avatar
 
Posts: 2,075
Karma: 3741360
Join Date: Jan 2013
Location: Itasca, Illinois
Device: Kindle Touch Sony PRS300
This is what you want. https://www.online-utility.org/text/analyzer.jsp
Gregg Bell is offline   Reply With Quote
Old 02-18-2018, 11:57 AM   #11
Nabeel
Zealot
Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.Nabeel ought to be getting tired of karma fortunes by now.
 
Nabeel's Avatar
 
Posts: 122
Karma: 611158
Join Date: Sep 2010
Location: Britain
Device: Kobo Aura One
Thank you Greg! Calibre doesn't do the job badly, but the site you recommend is more sophisticated.

Nabeel
Nabeel is offline   Reply With Quote
Old 02-20-2018, 03:31 AM   #12
evanhson357
EvnHrsn
evanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exercise
 
Posts: 49
Karma: 38446
Join Date: Sep 2016
Location: Australia
Device: none
Wow! Thanks! I didn't know Calibre could do that.
evanhson357 is offline   Reply With Quote
Old 02-20-2018, 04:48 AM   #13
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 10,149
Karma: 9778562
Join Date: Mar 2012
Location: Sydney Australia
Device: none
And, there's a English Noun Frequency plugin for calibre.

BR
BetterRed is offline   Reply With Quote
Old 02-23-2018, 02:19 AM   #14
evanhson357
EvnHrsn
evanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exerciseevanhson357 juggles running chainsaws for a bit of light exercise
 
Posts: 49
Karma: 38446
Join Date: Sep 2016
Location: Australia
Device: none
Thanks again.
There's a bunch of good info here. Anything to make the job easier.
evanhson357 is offline   Reply With Quote
Old 03-02-2018, 12:25 AM   #15
Tex2002ans
Guru
Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.Tex2002ans ought to be getting tired of karma fortunes by now.
 
Posts: 996
Karma: 5860817
Join Date: Jul 2012
Device: Nook
Quote:
Originally Posted by Nabeel View Post
A question came up in my Creative Writing group: there must be a computer programme or app that, when given a text, will analyse how frequently different words are used.
As others have stated, a user-friendly way for single words is to use Sigil or Calibre's Spellcheck lists. If you use Word, Toxaris's EPUB Tools also has this built in.

If you want to get "repeated phrases", that's called n-grams. Gregg Bell linked to one such tool, but there are plenty others.

Side Note: I personally use a commandline tool for n-grams. This gives me full control over the variables. Then I import it into a spreadsheet so I can sort by frequency.

Code:
This is an example of an n-gram example with an n-gram example.
2-grams would be all 2 words in a row:

Code:
1 This is
2 an n-gram
1 is an
1 an example
1 example of
1 of an
2 n-gram example
1 example with
1 with an
3-grams:

Code:
1 This is an
1 is an example
[...]
2 an n-gram example
[...]
Typically smaller n-grams are so full of cruft, they aren't really helpful ("he said" + "she said"). But I find the helpful patterns start to pop out at 4-grams and higher.

When you run this on a book-length text, you tend to see the author's own writing patterns.

I recently ran this on a ~70k word novel, and there were 26 "XYZ took a deep breath and" and 34 "XYZ shook her head". That's 292 words of characters taking a deep breath and shaking their heads.

Or a different author had the tendency to write "she said with an evil smirk on her face", "she said with a smile". So that author would probably want to go through and focus on chopping down "she said with".

A different book had 15 "What the f*** do you think you are doing?" That's 9 * 15 = 135 words.

These are typically a sign that you have to go through your book again and spice it up with variations.

Nobody wants to read hundreds of the same exact words again and again and again. Or slight variations of the words again and again... and again.

Quote:
Originally Posted by Nabeel View Post
Obviously, the really useful thing would be a programme that points out that you have unwittingly used a word like 'vast' five times in the same paragraph, but we're not looking for miracles.
The only tool I've come across that does this is TeXStudio:

https://www.texstudio.org/

It is a LaTeX editor, but you could use it for plain text if you wanted to.

It has a function called "Word Repetition":

https://tex.stackexchange.com/questi...from-texstudio

What it does is gives you a little green squiggly for the same word repeated within X number of words (you can set the min/max variables).

It tends to gives a lot of false positives though. One of the ways it could be made better would be if you could have some sort of whitelist, so you could ignore very common words ("the" + "if" + "and" + "but" + [...]).

Last edited by Tex2002ans; 03-02-2018 at 12:52 AM.
Tex2002ans is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Hyphenation frequency frapperla Kobo Reader 18 01-30-2016 07:40 AM
Repeatedly Crashing - Frequency Increasing Nyssa Calibre 16 01-24-2015 09:17 PM
Update frequency FinancialWar Calibre 98 08-29-2014 01:10 PM
Eink Refresh frequency Smartie Amazon Kindle 25 07-07-2014 09:45 AM
Programme TV en ePub Komenor Sony Reader 17 01-12-2009 11:34 AM


All times are GMT -4. The time now is 07:37 AM.


MobileRead.com is a privately owned, operated and funded community.