"Physicists decode theBard," by Philip Ball. Nature Science Update, 21 September 2001.
The relative frequencies of words has significance when analyzing a document,but these frequencies by themselves don't say much about what part of speech aword belongs to. This article summarizes work done by two physicists inArgentina linking a statistical analysis to the meanings of words. The toolthey used, Shannon entropy, measures how context-specific a word is. All of theanalysis was done on works from English literature---Shakespeare, Dickens andRobert Louis Stevenson---with the researchers finding that using frequency andShannon entropy together can determine a word's grammatical nature.
--- Mike Breen