Gauging Lexical Diversity
Posted On
Lexical Diversity refers to the various words used in a text, with a greater range indicating a higher diversity.
Lexical diversity encompasses a wide variety of measures, all of which seek to quantify the range of vocabulary deployed in a given text.
Researchers use measures of lexical diversity in many fields, including language acquisition, stylistics, neuropathology, and linguistic forensics.
Instruments to measure lexical diversity are as follows:
- Type-token ratio (TTR)
TTR is the magnitude relation obtained by dividing the kinds (the total range of various words) occurring in every text or utterance by its tokens (the total range of words). A high TTR indicates a high degree of lexical variation whereas a low TTR indicates the opposite. It is simple to use as a measure of lexical diversity.It is highly affected by text length, i.e., the longer the text, the less reliable it gets.
- D (Malvern & Richards 1997)
D allows for comparisons across samples of varying lengths. D, has been developed (by Malvern & Richards, 1997) to address the problem faced by a number of different words (NDW) i.e., the requirement of input samples of equivalent length. It randomly samples 35-50 tokens to compensate for text length. It prefers short texts and is moderately affected by text length.
- Measure of Textual Lexical Diversity (MTLD)
The Measure of Textual Lexical Diversity (MTLD) employs a sequential analysis of a sample to estimate an LD (Lexical diversity) score. Conceptually, MTLD reflects the average number of words in a row for which a certain TTR is maintained. To generate a score, MTLD calculates the TTR for increasingly longer parts of the sample. Every time the TTR drops below a predetermined value, a count (called the factor count) increases by 1, and the TTR evaluations are reset.
It randomly samples 50 tokens forwards and backwards to compensate for text length and calculates lexical diversity at a precise point of decline. It is also reliable for essays of 100+ tokens. It is new and needs further validation with long texts to see if it is reliable under 100 tokens and beyond 2000 tokens.
There are various games and exercises at Languify that will help you develop rich diversity in your vocal as well as written speech. Reading blogs, practicing your speech in unison will automatically improve your lexical diversity.
