A Meta-Analysis of Orton Gillingham Interventions for Students in Grades K-2
Over the next several weeks, I hope to review all of the most popular language instruction programs, by either reviewing a meta-analysis, or conducting one. Once finished I hope to conduct a secondary meta-analysis, in order to determine what is the best language instruction program according to the meta-analysis literature. I wanted to start this series off by examining the Orton Gillingham programs.
The Orton Gillingham (OG) approach is a linguistic phonics approach, meaning it teaches sounds before it teaches letters. Invented in the late 19th century, the conceptual model for teaching language is named after its founders Samuel T. Orton and Anna Gillingham. According to the Orton Academy, an OG approach “is a direct, explicit, multisensory, structured, sequential, diagnostic, and prescriptive way to teach literacy”. At first glance the program looks like most other structured literacy programs. However, one unique aspect might be its multisensory approach, meaning it incorporates kinesthetic aspects to the teaching method.
The meta-analysis research done on OG interventions, is not the most positive and there have been several previously done. The NRP found a mean ES of .23, which is significantly lower than the average education intervention of .40 and even lower than the average phonics intervention of .60. More recently, in 2021, a meta-analysis by Stevens, Et al, found an effect size of .22 for phonological awareness, phonics, fluency, and spelling. They also found an ES of .14 for comprehension. As Solari, Et al points out in her review of the topic, for the Reading League, the effect size found in the Stevens meta-analysis is consistent with all other meta-analyses on the topic (which there have been several of).
Normally, I would stop my research right here and conclude that OG interventions are not statistically significant and therefore not evidence-based. However, I wanted to dig deeper for two reasons. Firstly, the OG programs are research based, meaning they are based on sound principles according to the literature. OG programs include explicit instruction, which has an ES of .57 according to Hattie, Phonics which has an ES of .60 according to Hattie, and Individualized Curriculum which has an ES of 2.35 according to Steenbergen-Hu. Moreover, OG programs have an extremely loyal following within the science of reading community, despite its poor performance in the literature. I thought maybe, I could at the very least unpack the program's particular strengths.
I started by taking a look at the individual studies included in the Steven’s, Et, al meta-analysis and right away, I noticed a factor that might not be the most fair in analyzing a phonics intervention. 66% of the studies included were for students older than grade 3. Indeed many of the studies were on secondary students. This, in my opinion, is problematic, because phonics research consistently shows that phonics interventions work best on students between kindergarten and grade 2. Indeed the NRP meta-analysis only showed an ES of .27 for students in grades 2-6; moreover, that ES was lower for older students and higher for younger students. Another factor that came to mind, was the fact that most studies on the topic were on reading disabled students. Because reading disabled students show impaired learning they often show less progress in experimental studies. The NRP ES for the impact of phonics on reading disabled students was only .32.
When I read the individual studies included in Steven's meta-analysis, many of them were specifically measuring very low fluency outcomes, which was dragging down the overall effect size. However, this actually makes sense, as OG programs are clearly primarily aimed to increase decoding, and phonemic awareness, not fluency or comprehension. Indeed any increase in fluency or comprehension would only be a by-product of the students' increased decoding ability. For these reasons, I decided to conduct my own meta-analysis that excluded all studies for students after grade 2 and to parse out the impact of OG programs on individual reading outcomes.
Bisplinghoff, Et al. Written in 2015. The study included 121 students, who received 36 hours of instruction.
Christodoulou, Et, al. Written in 2017. The study included 47 students described as primary. Students received 100 hours of instruction.
Fritts, Et al. Written in 2016. The study included 86 students in grades 1-4. Students received 33 hours of instruction.
Stewart, Et al. Was written in 2011. The study included 51 grade 1 students. Students received 45 hours of instruction. This study was included in Steven's meta-analysis; however, it did not fit the inclusion criteria as it did not have a control group. I included it only, because it was included in Steven's meta-analysis. It’s effect size was within the average range for the intervention.
Torgesen et al. Was written in 1999. The study included 180 kindergarten students, who received 90 hours of instruction.
Wade, Et al. Written in 1993. The study included 80 students, in grades 1-5. I eventually excluded the study for multiple reasons. Firstly, the majority of students were older than the age cut off. Secondly, the only ES calculated was for fluency outcomes. Thirdly, it looked like there were higher outcomes in the study found for decoding and phonological awareness; however, the authors did not include the data that would make coding those results possible.
Daniels, Et al. Written in 2016, this study included 700 students and provided 41 hours of instruction. All students were placed in groups no larger than 6 students.
Joshi, Et al. Written in 2002. This study included 56 grade 1 students, who received 42 hours of instruction.
Scheffel, Et al. This study was written in 2008, included 451 grade 1 students, who received 62 hours of instruction.
In total this meta-analysis included 8 studies, with a combined sample size of 1721 students, with an average of 56 hours of instruction. 7 of the 8 studies included a control group. None of the studies were randomnized control trials.
A few things stuck out to me immediately after completing this analysis. Firstly, the ES when excluding older students was 44% higher. Moreover, if we exclude fluency based outcomes, which I actually think might be fair, we get an ES of .40. That being said, while the average phonics intervention shows a much greater impact than this, I don’t know if the comparison is entirely fair. As most of the studies included in this meta-analysis were for Reading Disabled students, whereas many of the studies on phonics overall are on regular class instruction. If we compare the OG ES (excluding fluency outcomes) to the NRP impact of phonics on Reading disabled students, we actually get a 29% higher outcome. So it may be fair to say that OG interventions show a greater impact on Reading Disabled students, than “regular phonics” instruction. Indeed to this point, I do not know if we can make any generalizations about the impact on OG interventions in classroom settings, as we have very little data directly studying this.
Finally, it is interesting to note that the outcomes of the OG interventions appeared directly correlated to the number of hours of instruction included within the study. The average hours of instruction for the top 3 studies was .56, whereas the number of hours of instruction for the bottom 3 was .15. For these reasons, I think some of what we are seeing in terms of the range of results, is actually just a difference, in the number of hours of instruction.
B-: Research based: IE there are no direct studies and some, but not most of the principles are evidence based.
A meta-analysis found an ES of .30-.39
Written by Nathaniel Hansford
Last Edited 2022-04-10
Orton Academy. (2022). What Is The Orton Gillingham Approach? Retrieved from <https://www.ortonacademy.org/resources/what-is-the-orton-gillingham-approach/>.
S, Bisplinghoff. (2015). THE EFFECTIVENESS OF AN ORTON-GILLINGHAM-STILLMAN-INFLUENCED APPROACH TO READING INTERVENTION FOR LOW ACHIEVING FIRST-GRADE STUDENTS. Florida Gulf State University. Retrieved from <https://fgcu.digital.flvc.org/islandora/object/fgcu%3A27407/datastream/OBJ/view/THE_EFFECTIVENESS_OF_AN_ORTON-GILLINGHAM-STILLMAN-INFLUENCED_APPROACH_TO_READING_INTERVENTION_FOR_LOW_ACHIEVING_FIRST-GRADE_STUDENTS.pdf>.
Christodoulou JA, Cyr A, Murtagh J, et al. Impact of Intensive Summer Reading Intervention for Children With Reading Disabilities and Difficulties in Early Elementary School. Journal of Learning Disabilities. 2017;50(2):115-127. doi:10.1177/0022219415617163
Christodoulou, J. A., Cyr, A., Murtagh, J., Chang, P., Lin, J., Guarino, A. J., Hook, P., & Gabrieli, J. D. (2017). Impact of Intensive Summer Reading Intervention for Children With Reading Disabilities and Difficulties in Early Elementary School. Journal of learning disabilities, 50(2), 115–127. https://doi.org/10.1177/0022219415617163
Kuveke, S. H. (1996). Effecting instructional change: A collaborative approach. https:// files.eric.ed.gov/fulltext/ED392029.pdf
Litcher JH, Roberge LP. First grade intervention for reading achievement of high risk children. Orton Society: Bulletin of Orton Society. 1979;29:238-244. Accessed January 26, 2022. https://search-ebscohost-com.ezproxy.lakeheadu.ca/login.aspx?direct=true&db=eue&AN=519782451&site=ehost-live
Torgesen, J. K., Wagner, R. K., Rashotte, C. A., Rose, E., Lindamood, P., Conway, T., & Garvan, C. (1999). Preventing reading failure in young children with phonological processing disabilities: Group and individual responses to instruction. Journal of Educational Psychology, 91(4), 579-593. doi:http://dx.doi.org/10.1037/0022-06220.127.116.119
Wade, J. J. (1993). Project Read versus basal reading program with respect to reading achievement, attitude toward school and self-concept (Publication No. 9417849) [Master’s thesis, The University of Southern Mississippi]. ProQuest Dissertations and Theses Global.
Wille, G. A. (1993). Project Read as an early intervention program (Publication No. 1353782) [Master’s thesis, California State University–Fullerton]. ProQuest Dissertations and Theses Global.
E, Solari, Et al. (2021). The Reading League. What Does Science Say About Orton-Gillingham Interventions? An Explanation and Commentary on the Stevens et al. (2021) Meta-Analysis Retrieved from <https://www.thereadingleague.org/wp-content/uploads/2021/05/Article-for-May-2021-TRLJ.pdf>.
Patterson, D. (2016). An Investigation of the Effectiveness of an Orton-Gillingham Based Reading Intervention in Kindergarten and First Grade Using a Fuzzy Regression Discontinuity Design. UC Riverside. ProQuest ID: Patterson_ucr_0032D_12527. Merritt ID: ark:/13030/m5bc8m8b. Retrieved from https://escholarship.org/uc/item/3jz3x80s
Joshi, R. M., Dahlgren, M., & Boulware-Gooden, R. (2002). Teaching Reading in an Inner City School through a Multisensory Teaching Approach. Annals of Dyslexia, 52, 229–242. https://doi-org.ezproxy.lakeheadu.ca/10.1007/s11881-002-0014-9
Scheffel, D. L., Shaw, J. C., & Shaw, R. (2008). The Efficacy of a Supplemental Multisensory Reading Program for First-Grade Students. Reading Improvement, 45(3), 139–152.
J, Hattie. (2021). Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/>.
Steenbergen-Hu, S. 2016 meta-analysis
Linnea, et al. (2001). Systematic Phonics Instruction Helps Students Learn to Read: Evidence From the National Reading Panel’s Meta-Analysis. Ontario Institute for Studies in Education. Retrieved from <https://www.dyslexie.lu/JDI_02_02_04.pdf>.