Published: May 14, 2018 By

Choosing an appropriate reading text for a class is complex. The content, vocabulary, length and a myriad of other factors might affect text selection. To that end, this document outlines some of the research that has guided text selection/adaption for the reading final exams at the IEC. Research on lexical coverage, Lexile levels and other considerations for text selection are described. The guidelines for the reading exam exemplify how this research could be applied to the IEC. Instructors can utilize these guidelines in order to create practice tests and in-class reading activities.

Lexical Coverage

The first concept is lexical coverage, which is how much vocabulary a learner knows in a given text. In order to estimate lexical coverage, we need to know students’ vocabulary size. Several studies have tried to estimate the size of students’ vocabulary at the different CEFR levels (Capel, 2010; Milton, 2010; Milton & Alexiou, 2009). While the research is not definitive, for the IEC’s purpose, Table 1 below should prove helpful:

Table 1

Vocabulary Size at Different Common European Framework Levels (CEFR)

Common European Framework Level

Vocabulary Size Estimate

A1

1,000 words

A2

2,000

B1

2,500

B2

3,000

C1

5,000+

Once we understand students’ vocabulary size, we can begin to analyze the vocabulary used in a text and estimate the lexical coverage. However, the threshold for lexical coverage is quite high. Research estimates that only if learners know at least 95% of the words in a text will they be able to adequately understand it (Laufer, 1989; Laufer & Ravenhorst-Kalovski, 2010). The ideal percentage might actually be higher (Nation, 2001; Laufer & Ravenhorst-Kalovski, 2010). This is relevant when selecting reading passages. For example, CEFR B2 level students might know about 3,000 words, so an appropriate reading passage would have 95% of the words from the list of the 3,000 most common words in English. Tools, such as Lextutor, can help determine that percentage. In that way, both the vocabulary in a text and students’ level can influence the selection of reading passages.

Holistic Text Measures

While lexical coverage is important, sentence difficulty may also affect reading comprehension. Both grade level scores and Lexile levels provide some indication of sentence and vocabulary difficulty. When making former versions of the IEC exam, reading passages were assessed using Flesch Reading Ease and an aggregate of other grade level scores. These scores measure word and sentence length to determine grade level and/or text difficulty. They are also relatively easy to interpret, so a 12th grader should be able to read a text at the 12th grade level. As students exit the IEC at Advanced 2, they should be reading at that level.  

While Lexile measures correlate with other readability scores, they may be more precise (Wright & Stone, 2004). In part, this is because the Lexile tool analyzes each word in a passage, comparing it to a 600-million word corpus to determine vocabulary difficulty (Lexile, 2017). Using this vocabulary difficulty rating along with sentence length, the system then assigns a Lexile level (L). Scores range from 200L to the most difficult 1700L. In a study of university textbook demands, Williamson (2008) estimates that university materials are at a 1200L-1400L level. Therefore, students exiting the IEC should be prepared to read at that level. However, students at the B2 CEFR level (Intermediate 3 and Advanced 1) might only able to read at the 1000L level (Smith & Turner, 2016). Table 2 below is based on Smith and Turner’s (2016) research and estimates students’ Lexile levels. These Lexile levels should be taken into account when selecting reading passages for in-class use and final exams. (See Appendix 1 for information on the estimated Lexile level of common IEC reading textbooks, using the Lexile Analyzer tool.)

Table 2

Lexile Levels for Intensive Reading at Different Levels

IEC

Levels

CEFR

Levels

Intensive Reading

Lexile Levels

(approximate) 

Intro

A1

0L to 600L

Basic 1

A2-

180L to 600L

Basic 2

A2+

180L to 800L

Intermediate 1

B1-

700L to 1000L

Intermediate 2

B1+

700L to 1200L

Intermediate 3

B2-

1000L to 1250L

Advanced 1

B2+

1000L to 1350L

Advanced 2

C1

1250L to 1400L

Other Considerations

While the metrics listed above are helpful in selecting passages, determining text difficulty still requires further analysis. In some cases the scores might be misleading, because they cannot detect idiomatic language or in-text definitions. The scores, especially the grade level scores, can be easily skewed by repeated words that are long, so an article on “biodiversity” might get a higher score just because it has that six-syllable word repeated often. Additionally, the vocabulary coverage score could be low, but depending on the students’ language background the uncommon words could be easily interpretable cognates. Therefore, these scores are not enough to justify text selection.

Other factors must also be considered. Often these are influenced by the reading task. To take the final reading exam as an exam, text length is important because students have limited time to complete the exam. At the upper levels, the texts for the final exams are all 500-600 words to account for the time constraint. The final exam also requires students to write a summary and a response; these writing tasks influence the text selection. For ease of summary writing, the passages should be clearly organized. Most importantly for response writing, students should be able to respond to the text, which means the topic must be familiar, relatable, and possibly controversial. All these elements are as important as the quantitative metrics outlined above.

Passage Selection Guidelines

The above research could influence passage selection and text analysis across the IEC. Specifically, this body of research informed the IEC’s final reading exam passage selection guidelines. These guidelines have relatively large ranges for the different metrics, but the goal should be to have most of the metrics within the given ranges. Since the reading tests are multi-level integrated skills tests, they should not be designed only for the top level. Most students taking the exam should be able to understand at least part of the reading. Therefore, for the lower level, we targeted the reading passage at the Basic 2 level and the A2+ CEFR level. The lower-level guidelines are in Table 3.

Table 3

Lower-Level Reading Text Guidelines for Final Exam

Word Count

300-400 words

Respond-able Topic

Yes

Average Grade Level

6-8

Reading Ease

65-80 (larger easier)

Lexile

600-800L

Text Coverage
Percentage of Words from the 2,000 most common words or below

~95%

For the upper level exam guidelines, see Table 4. The difficulty of the upper-level reading passage targeted the Intermediate 3 and Advanced 1 (B2 CEFR level).

Table 4

Upper-Level Reading Text Guidelines for Final Exam

Word Count

500-600 words

Respond-able Topic

Yes

Average Grade Level

11.5-12.5

Reading Ease

45-55 (larger easier)

Lexile

1050L-1350L

Text Coverage: Percentage of Words from the 2,000 most common words or below

~90%

Text Coverage: Percentage of Words from the 3,000 most common words or below

~95%

With this information, passage difficulty for the final exam is expected to be more consistent across sessions. Moreover, these methods for selecting and adapting texts for the final exams can be applied to other classroom materials. Faculty can evaluate, adapt, and incorporate outside materials more effectively using these guidelines. For more information on the exact websites and tools to use, faculty should consult the list of text analysis tools in Appendix 2. With the tools to analyze texts more rigorously and the above guidelines, faculty can more readily adapt authentic materials without compromising level appropriateness. The final exam just serves as one example of how to apply these tools and guidelines; all faculty might find them useful when adapting or selecting additional reading materials for their classes.

References

  • Capel, A. (2010). Insights and issues arising from the English Profile Wordlists project, Research Notes, 41: 2-7. Cambridge: Cambridge ESOL.
  • Laufer, B. (1989). What percentage of text lexis is essential for comprehension? In C. Lauren & M. Nordman (Eds.), Special language: From humans thinking to thinking machines (pp. 316–323). Clevedon: Multilingual Matters.
  • Laufer, B, & Ravenhorst-Kalovski, G. C. (2010). Lexical threshold revisited: Lexical text coverage, learner’s vocabulary size and reading comprehension. Reading in a Foreign Language, 22, 1, 15-30.
  • Lexile. (2017, May). How is the Lexile measure of a text determined? Retrieved from https://lexile.desk.com/customer/portal/articles/508829-how-is-the-lexil...
  • Milton, J. (2010). The development of vocabulary breadth across the CEFR levels. In I. Bartning, M. Martin, & I. Vedder (Eds.), Communicative proficiency and linguistic development: intersections between SLA and language testing research (pp. 211-232), Online: EUROSLA Monographs.
  • Milton, J., & Alexiou, T. (2009). Vocabulary size and the Common European Framework of Reference for Languages. In B. Richards et al. (Eds.), Vocabulary studies in first and second language acquisition (pp. 194–211). Basingstoke: Palgrave.
  • Nation, I. S. P. (2001). Learning vocabulary in another language. Cambridge: Cambridge University Press.
  • Smith, M. & Turner, J. (2016). The Common European Framework for Reference for Languages (CEFR) and The Lexile Framework for Reading. Retrieved from https://lexile.co.uk/lexile-international-prod-media/pdfs/CEFR_1.pdf
  • Williamson, G. L. (2008). A text readability continuum for postsecondary readiness. Journal of Advanced Academics, 19(4), 602-632.
  • Wright, B. D., & Stone, M. H. (2004). Making measures. Chicago: Phaneron Press.

Appendix 1

Estimated Lexile Level of IEC Textbooks

Below are some common IEC textbooks. One passage from each was selected and run through the free Lexile Analyzer Pro tool in order to estimate the Lexile level. This table shows how Smith and Turner’s (2016) estimates for each CEFR level are born out in the IEC’s textbooks.

IEC Level (Recommended Lexile Range)

Textbook Name(s)

Estimated

Lexile Level*

Intro

(0L to 600L)

All New Very Easy True Stories

310L

B1

(180L to 600L)

Password 1

580L

B2

(180L to 800L)

Reading Explorer: Foundations

780L

I1

(700L to 1000L)

Making Connections 1

830L

I2

(700L to 1200L)

Making Connections 2

920L

I3

(1000L to 1250L)

Reading Explorer 4

1140L

 

I3

(1000L to 1250L)

Academic Encounters 4 1140L

A1

(1000L to 1350L)

Making Connections 3

1150L

A2

(1250L to 1400L)

Making Connections 4

1260L

*These levels were not prepared by MetaMetrics staff, nor did the researcher receive training from them.

Appendix 2

Tools Used

Lextutor

This tool analyzes the vocabulary in a text and determines the percentage of words from the one thousand, two thousand, three thousand, etc. most common words in English. These bands are labeled K1 (1,000), K2 (2,000) etc. in the tool. The bands are based on the Corpus of Contemporary American English and the British National Corpus.

Lexile Analyzer (Pro)

This tool requires registration. For the PRO version, which gives a more exact Lexile level, email the site administrators. When you copy-and-paste in a text, this tool will give you a Lexile score.

Grade Level

This tool gives users an array of grade level and readability scores. For the final exams, the IEC has focused on the Flesch Reading Ease and this site’s “average grade level.” Note that there is a daily limit of texts that can be analyzed.

When using these tools, remove titles and in-text citations because these distort the sentence length and word frequency algorithms.