A Meta-Analysis of Jolly Phonics

Jolly Phonics is described by its creators as, “a comprehensive programme, based on the proven, fun and multi-sensory synthetic phonics method that gets children reading and writing from an early age. This means that we teach letter sounds as opposed to the alphabet. These 42 letter sounds are phonic building blocks that children, with the right tools, use to decode the English language. When reading a word, they recognise the letters and blend together the respective sounds; when writing a word they identify the sounds and write down the corresponding letters. These skills are called blending and segmenting. These are two of the five skills that children need to master phonics.”

I wanted to evaluate the efficacy of the Jolly Phonics program; however, to the best of my knowledge there existed no prior meta-analysis of the topic. While the NRP meta-analysis did look at Jolly Phonics, they only included 1 study, which found a mean ES of .73. However, this ES was higher than most other effect sizes found for phonics interventions, in the NRP meta-analysis, which was part of what peaked my interest.

 

In order to assess the efficacy of Jolly Phonics further, I conducted my own meta-analysis with the following inclusion criteria: The study had to be on English Language Learning, the study had to include either calculated effect sizes or the raw data for me to calculate effect sizes, and the study had to have a control group. I was able to find 8 studies that met this criteria. Three studies were peer reviewed, 2 studies were Phd theses, 1 study was an unpublished RCT experiment, and 2 studies were government conducted for policy research purposes. 

 

While normally I would exclude non-peer reviewed studies, the research base was so small I included the non-peer reviewed studies. However, the results found in each study were shockingly similar. Indeed I have never previously done a meta-analysis in which the results were so homogenous. The mean ES overall was .96, whereas the mean ES in non peer reviewed studies was 1.02, which is not a statistically significant difference. The country studies showed the lowest results, which is quite typical, as government policy data rarely reflects the results found in clinical studies, likely for fidelity reasons. However, this data was still very significant with a mean ES of .96. 

 

While, many might argue we should exclude the country data, as ministries of education might be more motivated to try and prove a certain result. I actually disagree for several reasons. Firstly, as already mentioned, country data is almost always lower and therefore not likely to exaggerate results. Secondly, the effect sizes found in this country data was very homogenous with the results found in peer reviewed data, albeit slightly more conservative. Thirdly, this country level data not only provides a much greater sample, the authors of the reports were able to give the results for multiple grades, and measurements. Lastly, it's ultimately not clinical results that matter, but practical ones. Examining the effects of changing pedagogy on a policy level gives us far more practical insights into real world effects, than on a clinical level. 

 

Results: 

Studies Included: 

 

Nasrawi, Et, al wrote this study in 2017. The study examined giving 58 grade 1 ESL students, 11.25 hours of Jolly Phonics instruction. This study was peer reviewed. 

 

Callinan, Et, al, wrote this study in 2010 and examined giving 30 kindergarten students Jolly Phonics instruction for 1 year. This study was peer reviewed.

 

Stuart, Et al wrote this study in 1999. This study looked at giving 6 kindergarten classes 60 hours of Jolly Phonics instruction. This study was peer reviewed and included in the NRP meta-analysis. 

 

Crane wrote this Phd thesis in 1999. This study included giving 152 kindergarten students 20 weeks of Jolly Phonics instruction. 

 

Leila Farokhbakht. Wrote this Phd Thesis, in an unspecified year. This study was the only RCT study in the meta-analysis and provided 50 ESL students in grades 6-8, with 45 hours of Jolly Phonics instruction. 

 

N, Katechaiyo, et al, conducted an un peer reviewed study, in an unspecified period of time, for 20 hours of Jolly Phonics instruction on K-2 students. 

 

The Republic of Gambia conducted their own study in 2009, over 2 years on students in grade 1-3. 

 

The government of Nigeria, in 2014,  did their own study on 240 students, in grade 1 classes, for an unspecified amount of time. 

 

Katechaiyo, Et al, conducted a non peer reviewed study in 2014, on 44 k-3 students, for 20 hours. 

 

Discussion: 

I was surprised by these results, specifically in how homogenous they were. Usually, when you conduct a meta-analysis, you find one or two studies with extremely high results and one or two studies with negative results, and the majority of studies with much more moderate results. However, every single study conducted on Jolly Phonics that I found and met my inclusion criteria yielded high to very high results. Indeed the lowest found effect size was .53 and the highest was 1.24, for a meta-analysis this is actually quite a small range. Considering that the effect sizes found were all high to very high, I think it must be concluded that Jolly Phonics is an evidence-based strategy. Moreover, I think this meta-analysis adds to the case that synthetic phonics is superior to other forms of phonics, as the lowest effect size found in this study was higher than the average effect size for phonics overall (.60), as found by Hattie. 

 

Additionally, it is interesting to note that this analysis found high effects sizes for all outcomes, but that the highest outcome was for grade 3. This is particularly surprising as most phonics interventions show diminishing returns after grade 2. This effect size was based on limited data though, so it may be a good place for further research. It is also important to note that one of the highest outcomes was for comprehension, as a common theoretical criticism of phonics is that “phonics is inferior to Balanced Literacy for comprehension outcomes”. This data does not support that Balanced Literacy narrative.  Where the data does seem weakest is actually for phonemic awareness, fluency, and spelling. However, all of these effect sizes are still moderately high, being well above the education study average of .40. 

 

As I recently did a Orton Gillingham (OG)  meta-analysis before this article, readers might feel inclined to compare the two strategies. While, I do not think the comparison is wholly unfair, I would caution that the OG meta-analysis exclusively looked at studies on dyslexic students and at-risk readers. Studies which examine dyslexic students tend to have much lower effect sizes. For example the NRP paper found a .32 ES for phonics interventions for Reading Disabled students, compared to an overall effect size for phonics of .69. The studies included in this meta-analysis looked at exclusively whole class studies, which might explain at least part of why Jolly Phonics outperformed OG.

 

Moreover, Jolly Phonics might actually be considered an OG program, or at the very least similar to an OG program. While the Jolly Phonics website does not describe itself as OG and while the latest OG literature reviews do not include Jolly phonics studies, the approaches do appear at least similar on a surface level. Indeed, multiple teacher websites appear to list Jolly Phonics as an OG strategy. One key difference might be that OG programs are typically either very small group or one to one, where as Jolly Phonics is typically a classroom program.

 

One final note, half of the studies examined were ESL or ELL studies. However, the mean effect size for ELL and non-ELL studies were literally identical. That being said, I think this analysis helps support the case that synthetic phonics helps ELL students. The ELL effect size for this study was 1.02, this is inline with previous ELL phonics research, which shows phonics is a high yield strategy for ELL students. 


Disclaimers:
I have no affiliation with Jolly Phonics, have never taught Jolly Phonics, and have no intention to use Jolly Phonics. The purpose of this article was not to convey my personal opinions of the merits or flaws in Jolly Phonics, but rather to provide as an objectively neutral statistical analysis of the quantitative literature on Jolly Phonics, as possible. 

 

Final Grade: A-: More than 4 studies, with a mean ES above .70

Qualitative Grade: 2/10

The program includes the following evidence-based principles: phonics and direct instruction.

Written by, Nathaniel Hansford

Last Edited 2022-01-28

 

References:

 

Nasrawi, A., & Al-Jamal, D. (2017). The Effect of Using Jolly Phonics on Jordanian First Grade Pupils’ Reading. International Online Journal of Education & Teaching, 4(2), 106–119.

 

Callinan, C., & van der Zee, E. (2010). A comparative study of two methods of synthetic phonics instruction for learning how to read: Jolly Phonics and THRASS. Psychology of Education Review, 34(1), 21–31.

 

M, Stuart. (1999). Getting ready for reading: Early phoneme awareness and phonics teaching improves reading and spelling in inner-city second language learners. British Journal of Psychology. Retrieved from <https://jolly2.s3.amazonaws.com/Research/Getting%20Ready%20for%20Reading.pdf>. 

 

C, Crane, Et, Al. (1999). Improving early language and literacy skills: differential effects of an oral language versus a phonology with reading intervention. University of York. Retrieved from <https://jolly2.s3.amazonaws.com/Research/BowyerCrane%20etal2007proof.pdf>. 

 

L, Farokhbakht. The Effect of Using Multisensory-based Phonics in Teaching Literacy on EFL Young Female/Male Learners' Early Reading Motivation. University of Isfahan. Retrieved from <https://jolly2.s3.amazonaws.com/Research/The%20Effect%20of%20Using%20Multisensory-based%20Phonics%20in%20Teaching%20Literacy%20on%20EFL%20Young%20FemaleMale%20Learners'%20Early%20Reading%20Motivation.pdf>. 


 

N, Katechaiyo, et al. EFFECTS OF JOLLY PHONICS INSTRUCTION FOR PUPIL BOOK 1 ON REDING ABILITY OF THAI EFL YOUNG LEARNERS. Retrieved from <https://jolly2.s3.amazonaws.com/Research/Revealing%20the%20secrets%20of%20remarkable%20improvement%20of%20Thai%20EFL%20young%20learners_Aug.2021.pdf>. 

 

Republic of Gambia. (2009).  IMPACT ASSESSMENT OF INTERVENTIONS ON EARLY GRADE READING ABILITY (EGRA) IN SCHOOLS. Retrieved from <https://www.jollylearning.co.uk/evidence/research/>. 

 

Government of Nigeria. (2014). REPORT ON THE MONITORING EXERCISE FOR THE IMPLEMENTATION OF JOLLY PHONICS APPROACH IN THE FEDERAL CAPITAL TERRITORY, ABUJA. Retrieved from <https://s3.amazonaws.com/jolly2/Research/Jolly+Phonics+in+FCT.pdf>. 

 

N, Katechaiyo, Et al. EFFECTS OF JOLLY PHONICS INSTRUCTION FOR PUPIL BOOK 1 ON READING ABILITY OF THAI EFL YOUNG LEARNERS. Retrieved from <https://jolly2.s3.amazonaws.com/Research/Revealing%20the%20secrets%20of%20remarkable%20improvement%20of%20Thai%20EFL%20young%20learners_Aug.2021.pdf>. 

 

-NRP. (2001). Teaching Children to Read: An Evidence Based Assessment of the Scientific Literature on Reading Instruction. United States Government. Retrieved from <https://www.nichd.nih.gov/sites/default/files/publications/pubs/nrp/Documents/report.pdf>.

 

J, Hattie. (2021). Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/>.