READING INSTRUCTION STRATEGIES META-ANALYSIS

An Explanation and a Disclaimer: 

As teachers, we get exposed to so many different teaching interventions each year that it would be impossible to effectively implement all of them in the classroom. Perhaps more to the point, it should be noted that these different methodologies naturally vary in their efficacy as teaching tools and cannot, therefore, produce equal results in regard to student learning.


As, of course, there are only so many teaching hours in a day and given the fact that this time is subdivided to accommodate multiple subjects, it can be an overwhelming task to wade through the array of recommended teaching interventions and determine which of these will best serve to promote student success. To that end, I wanted this article to help demystify the process of choosing appropriate teaching methods and highlight those which have the highest probability of improving student learning outcomes. I was inspired by John Hattie’s work, which ranks teaching factors according to their effect sizes (ES) in meta-analysis. Hattie’s work, however, provides a more generalized view of teaching interventions, which, although broadly useful in determining the effectiveness of strategies that apply to all subjects, can overlook those strategies that are subject-specific and which target individual learning goals. I, therefore, decided to try and replicate Hattie’s ideas and apply them more specifically to individual subjects, in this article, I explore meta-research for reading.

I compiled the results of nine separate meta-analyses, which examined 681 different quantitative studies, on the efficacy of different reading interventions. While I do not think specific effect size numbers are overly important, I do believe they provide us with a less subjective measure for determining which teaching interventions are the most likely to be successful. And, while small differences in effect size, like the 0.02 point difference between Read-Aloud and Answering Questions, may not be materially instructive, it would, I believe, be fair to say that large differences, like the 1.37 point difference between RTI and Ear Reading, are substantive enough to arbitrate which strategies are likely to have the greatest success in motivating classroom learning. By viewing the extant literature and available statistics in this light, I believe it is possible to empower teachers to make more informed decisions about how best to optimize the use of their available time and resources.  

Generally speaking, if an intervention has an effect size (ES) of greater than 0.70 the intervention could be called a high yield instructional strategy. If the intervention has an ES of 0.40 - 0.69, the intervention could be called a moderate yield instructional strategy, and if the intervention has an ES lower than 0.40, it should be referred to as a low yield strategy. While the actual statistical data can, at times, be less clear-cut, particularly when variables are inconsistent or when studies employ different effect size calculations, the use of meta-studies to synthesize this data does help to mediate possible discrepancies and provides a means of comparing and cross-examining results as a way of holding individual studies to a higher degree of accountability. And, although this article does analyze studies with different effect size calculators, which has been a criticism of John’s Hattie’s work, the existing data on this subject is limited and there are too few studies to allow for more restrictive inclusion criteria. Acknowledging, then, that the specific numbers which appear in this article may be off by slight degrees relative to one another, I posit that the individual numbers, themselves, are less relevant than the proportional deviation between different interventions, as the resulting hierarchy is not so dependent on numerical exactness as to be upended by the difference of a few hundredths of a decimal point.  


All of the studies included in this analysis used Cohen's D or Hedge’s effect sizes calculation except for one study, which used an experimental design calculation. While these calculations are different, they all provide comparable results and are meant to be interpreted within the same context. The average effect size for all factors looked at in this analysis was 0.58. Ultimately, I think the precise effect sizes are almost meaningless, as these numbers represent a reflection of an average created by a very diverse set of numbers. What these numbers really allow us to do is approximate the comparative values of different reading interventions in a more educated way.

Personally, I am a big fan of this type of research, because it allows teachers to make quick and evidence-based inferences about what interventions would be the most time-efficient for them to implement within their classrooms. While it may be argued that data is changeable and that these effect sizes could shift as more research is conducted or that the numbers, as they are now represented, could be influenced by errors in execution, this type of comparative analysis, in the very least, provides us with the foundations of a scientific meter for evaluating mathematical interventions and a  jumping-off point for further research into their efficacy.


Ultimately, while I believe in this research as a tool for exposing teachers to new or more evidence-based teaching methods, I also acknowledge that this list is a starting point only and that teachers will need to experiment with these interventions (and any strategies they find in their own research ) to determine what best fits with and supports the unique learning environment of their classrooms. Simply because an intervention on this list has a low ES, does not mean that it cannot be a powerful instructional tool with the right variables or that there are no students who will benefit from its use. So much of successful classroom teaching comes down to the different and often highly individual needs of students. What works for one class, or what worked for the classes in these studies, may not work for your students. There is an element of trial and error that is inescapable, therefore, in classroom teaching, but this does not mean that teachers should have to rely on guesswork or single-handedly navigate the labyrinth of purported best practices. It is my hope, then, that this article will be viewed not as a ranking system in the strictest sense, but rather as a springboard for directing the implementation of mathematical strategies in an evidence-based way.

Intervention List:


  1. RTI: Response to intervention, is a complex teaching framework. Teachers implement this strategy by first collectively setting learning goals for their students at the start of each learning block (each learning block is usually 4-6 weeks). Once the learning goals are established, teachers use formative assessment to monitor their students learning, using objective data collection each week.Teachers then collectively reflect on this data, to make decisions on which students need extra help and as to which instructional strategies are working. Ideally a teacher would use this strategy to alter how they teach in real time, to benefit the individual needs of their students. 

  2. Main Idea: Refers to the concept of getting students to focus on interpreting what the main idea of a text is. This popular intervention has been primarily shown to increase students reading comprehension skills, not their decoding or fluency skills. Teachers should start to focus on implementing this strategy more, after their students have mastered their basic reading skills.

  3. Continuous Reading: In this intervention small groups of students go to a separate classroom, to independently read under the supervision of a special education teacher for an extended period of time. The teacher then monitors the students and helps them to decode, identify, and explain unfamiliar words. While this strategy is a Whole-Language strategy, it also provides students with extended learning time, and individual instruction. This strategy can be a strong addition to a special education reading program that also includes explicit phonetic instruction. 

  4. Single Skills Phonics: refers to phonetic instruction that looks at the individual sound of each letter. 

  5. Expository Interventions: This intervention refers to giving students explicit instruction on aspects such as text structure, summarizing, and main idea. This has been shown to help with reading comprehension and is not a strategy meant to increase vocabulary, fluency, or decoding skills. 

  6. Repeated Reading: This is a fluency intervention, in which students read the same text over and over again, while being timed, with the goal of increasing reading speed and accuracy. 

  7. Phonics: This intervention refers to any strategy that seeks to increase students' phonetic knowledge. 

  8. Spelling: This intervention refers to any strategy that seeks to increase a students spelling vocabulary, such as direct instruction and wrote-memorization. 

  9. Writing Reactions: This intervention refers to having students read a text and then write a personal response to that text. 

  10. Spelling Strategies: This intervention refers to directly instructing students on phonetic strategies for increasing their spelling vocabulary. 

  11. Analytic Phonics: Refers to interventions that focus on directly teaching students their blend sounds. For example, teaching students the “oot” sound, would be an example of analytic phonics. 

  12. Reciprocal Reading: This teaching strategy involves the direct instruction of a meta-cognition strategy for reading comprehension. In this intervention a teacher directly explains to students how to interpret texts, then demonstrates this skill, then has students practice this skill in groups, and then has students practice this skill alone. 

  13. Increased Writing: This strategy refers to just encouraging students to write more. 

  14. ICT: This intervention simply refers to the average impact size of providing students technology during language instruction. However, the individual quality of ICT instruction would obviously affect the impact of any ICT intervention. 

  15. Small Group Instruction: This strategy refers to providing students with any small group instruction. The quality of this instruction would obviously also depend on the other interventions being used. 

  16. Whole-Word Instruction: This strategy refers to teaching students how to read, via reading practice and modelling, without phonetic instruction. Obviously this strategy cannot be paired in natural conjunction with phonetic instruction and should only be compared to phonetic instruction as the two strategies are diametrically opposed. It is important to note, that despite vigorous debate on the topic, almost every single meta-analysis shows Whole-Word Instruction as underperforming phonetic instruction. 

  17. Guided Reciprocal Reading: This strategy combines Reciprocal Reading with Guided Readed, where Guided Reading can be defined as having a large group of students silently read, with one instructor present to help struggling students. 

  18. Writing Notes: Refers to any intervention were students write notes.

  19. Synthetic Phonics: This intervention refers to teaching students the sounds of individual letters and showing how this can be used to sound out words. 

  20. Guided Reading: This intervention refers to having students read in large groups silently, while a teacher supervises. 

  21. Sing Spell Write: In this intervention students first sing a word, then orally spell the word, and lastly write the word. 

  22. Whole Language: Please see Whole-Word. Both interventions are the same; however, these data points were based on separate meta-analysis. 

  23. Read Aloud: This intervention refers to having students either individually read aloud in a group, or read aloud in unison, while in a group. 

  24. Answering Questions: This intervention refers to having students answer individual questions after reading a text. 

  25. Data Informed Teaching: This method refers to teaching based off of data collected during formative assessment. While this meta-analysis ES demonstrates a low yield, DIT is a core aspect of RTI which is a high yield reading strategy. This difference should reflect the importance of execution. DIT is only so effective as the intervention changes based on that data. 

  26. Reading Meta-Cognition Strategies: These are strategies taught to students to help them better understand a text, without necessarily focusing on the skill of reading itself. For example, telling students to look at the pictures, or use the recognized words in a sentence to help them guess the un-recognized words in a sentence. This type of teaching strategy has been especially popular in the Balanced Literacy Approach. However, the weakness of this effect size, clearly demonstrates that this is not a cost-effective strategy. 

  27. Direct Instruction of Writing Methods: This intervention refers to providing students with explicit explanations of how to improve their writing. While this method clearly was a low yield strategy for reading outcomes. It is important for upper grade students, in order to improve their procedural writing practices. For example, students headed towards university require the knowledge of how to write an essay. This skill takes years of scaffolding, practice and direct instruction. 

  28. Listening While Reading: This strategy has students attempt to silently read a text while a teacher or computer software reads aloud to the student. This strategy has been often looked at as a possible solution to Dyslexia. However, the weakness of this ES should demonstrate the lack of efficacy for this idea. 




References:

NRP. (2010). National Reading Panel. Teaching Children to Read. An Evidence-Based Assessment of the Scientific Research On Reading and Its Implications for Reading. Retrieved from < https://www.nichd.nih.gov/sites/default/files/publications/pubs/nrp/Documents/report.pdf>. 


Graham, Steve, and Michael Hebert. Writing to Read: Evidence for How Writing Can Improve. Carnegie Corporation Time to Act Report. Washington, DC: Alliance for Excellent Education, 2010. Print.


A Synthesis of Read-Aloud Interventions on Early Reading Outcomes Among Preschool Through Third Graders at Risk for Reading Difficulties. (2011). Journal of Learning Disabilities., 44(3), 258–275.


OKKINGA, M. et al. Effectiveness of Reading-Strategy Interventions in Whole Classrooms: a Meta-Analysis. Educational Psychology Review, [s. l.], v. 30, n. 4, p. 1215–1239, 2018. DOI 10.1007/s10648-018-9445-7. Disponível em: http://search.ebscohost.com.ezproxy.lakeheadu.ca/login.aspx?direct=true&db=eue&AN=133311304&site=ehost-live. Acesso em: 1 ago. 2020.


Filderman MJ, Toste JR, Didion LA, Peng P, Clemens NH. Data-Based Decision Making in Reading Interventions: A Synthesis and Meta-Analysis of the Effects for Struggling Readers. Journal of Special Education. 2018;52(3):174-187. doi:10.1177/0022466918790001.

Hall MS, Burns MK. Meta-analysis of targeted small-group reading interventions. Journal of School Psychology. 2018;66:54-66. doi:10.1016/j.jsp.2017.11.002.


Stevens EA, Park S, Vaughn S. A Review of Summarizing and Main Idea Interventions for Struggling Readers in Grades 3 Through 12: 1978–2016. Remedial & Special Education. 2019;40(3):131-149. doi:10.1177/0741932517749940.


Archer K, Savage R, Sanghera-Sidhu S, Wood E, Gottardo A, Chen V. Examining the effectiveness of technology use in classrooms: A tertiary meta-analysis. Computers & Education. 2014;78:140-149. doi:10.1016/j.compedu.2014.06.001.


Hudson A, Koh PW, Moore KA, Binks-Cantrell E. Fluency Interventions for Elementary Students with Reading Difficulties: A Synthesis of Research from 2000–2019. Education Sciences. 2020;10(3):52. doi:10.3390/educsci10030052.