Modality Matters: Benefits of Speech-Based Learning in Learners with Dyslexia

Abstract 

Developmental dyslexia is a common learning disorder that affects reading, writing, and spelling, often leading to academic difficulties. In this study, which was awarded the 2024 Computational Modeling Award by the Cognitive Science Society, researcher Thomas Wilschut aims to address some of these challenges by improving the process of vocabulary learning in dyslexia. He asked two groups—dyslexic learners (n = 57) and typical learners (n = 61)—to complete a memory task using both typing and speech response modalities. Learners with dyslexia performed worse when typing, but this disadvantage disappeared when they responded verbally. At first glance, the results suggested weaker memory performance for learners with dyslexia than for typical learners. However, after inspecting response times, the researchers found the typing-specific disadvantage in learners with dyslexia was mainly due to slower response preparation, rather than memory issues. These results highlight the importance of creating learning systems that account for different response methods to support dyslexic learners more effectively.

Nederlandse samenvatting 

Ontwikkelingsdyslexie is een veelvoorkomende stoornis die invloed heeft op lezen, schrijven en spelling, wat vaak leidt tot problemen met leren. In deze studie, die bekroond werd met de 2024 Computational Modeling Award door de Cognitive Science Society, probeert onderzoeker Thomas Wilschut een aantal van deze uitdagingen aan te pakken door het leren van vocabulair te verbeteren. Hij vroeg twee groepen—dyslectische leerlingen (n = 57) en typische leerlingen (n = 61)—om een leertaak te voltooien met zowel typen als spraak als responsmethoden. Leerlingen met dyslexie presteerden slechter bij het typen, maar dit nadeel verdween wanneer ze konden spreken. Op het eerste gezicht leken de resultaten erop te wijzen dat dyslectische leerlingen een zwakker geheugen hadden. Echter, na analyse van de reactietijden bleek dat het nadeel voornamelijk te wijten was aan trage responstijd en niet aan geheugenproblemen. Deze resultaten demonstreren het belang van leersystemen die rekening houden met verschillende antwoordopties om dyslectische leerlingen effectiever te ondersteunen.

Developmental Dyslexia: a challenge with processing the sounds of language

Dyslexia is more than just trouble with reading—it’s a challenge that affects how the brain processes language. People with dyslexia struggle to connect letters with their sounds, making reading, spelling, and writing difficult (Snowling et al., 2020). Imagine trying to read a book, but the words seem scrambled or take longer to make sense of. You might confuse letters in a word, like reading “cat” as “act” or “stop” as “tops.” Words might also blend together, making it hard to distinguish between them. Dyslexia can also impact other areas, like memory and attention, and can vary from person to person.

Dyslexia affects about 3-7% of the population, making it relatively common (Fletcher et al., 2018). In school settings, where traditional methods rely heavily on reading and writing, dyslexic students can face frustration, anxiety, and even lower self-esteem. They may feel like they’re underperforming simply because their brains process written language differently (Riddick, 2000). For example, a student might understand a lesson when it’s explained verbally but struggle to complete a written assignment on the same topic.

Alleviating dyslexia-related difficulties with model-based personalization 

To help with these challenges, researchers are developing tools that adapt to the unique needs of dyslexic learners. One promising approach that is used in the domain of fact- and vocabulary learning, is to employ adaptive learning applications. Such systems present retrieval practice questions—where the learner is presented with a cue, and asked to retrieve the correct answer from memory—at optimal points in time during a learning session. Some systems use computer simulations of the learners’ memory that track how well someone remembers information over time. These models help to tailor learning sessions based on a student’s performance, helping to improve the long-term retention of the materials. For instance, these systems might notice when a student struggles with a particular word and offer extra practice at the right time to strengthen the memory representation for that word (e.g., Lindsey et al., 2014, Papousek et al., 2014, van Rijn et al., 2009).

Recent studies have explored how different learning methods—such as typing answers versus speaking them out loud—can impact the overall learning efficiency of the adaptive learning systems (Wilschut et al., 2024). In the current study, Wilschut and colleagues explore how these different learning modalities impact learners with developmental dyslexia. There are several reasons to expect that speech-based learning offers more benefits for learners with dyslexia compared to typical learners. Dyslexia often involves difficulties in processing the sounds of language, which affects the ability to decode written text. This means that translating letters and words into their corresponding sounds and meanings can be particularly challenging. However, when information is presented through speech and the learner responds verbally, they don’t need to go through the process of decoding written text. Similarly, speech-based learning doesn’t require correct spelling, which removes another potential barrier. Bypassing these steps could make learning more seamless for individuals with dyslexia. 

Experiment: Speech-Based Learning in Dyslexia

In total, Wilschut and colleagues asked 118 participants to complete two retrieval practice sessions. In the one session, participants were asked to study Swahili vocabulary by typing it’s correct English translation upon the text-based presentation of the Swahili word. In the other session, the Swahili word would be presented both audibly and visually, and participants were asked to respond by speech—much like an in-person rehearsal when preparing for a test. For each repetition of an item, response times and accuracy scores were recorded, and the adaptive learning model described above used these performance metrics to calculate the average speed at which the learners were forgetting the materials. Both learning sessions were followed by a test about all studied answers.

Figure 1: Summary of the results of the experiment. Learners with self-reported diagnoses of dyslexia are shown in purple, typical learners are shown in orange. Dots and dotted lines show group averages. The left panel shows the model-based estimations of the Speed of Forgetting (SoF) during the learning session. The panel on the right shows the number items recalled on a test following the learning sessions. 

The results of the experiment are summarized in Figure 1. The left panel shows the average speed at which the learners forgot the items during the learning session, for the two learning modalities. The results indicate that on average, when typing, learners with dyslexia forgot items quicker than typical learners. When speaking, this difference disappeared and forgetting speeds were similar for typical learners and learners with dyslexia. The right panel shows the number of items that were recalled correctly on the test after the two learning sessions. Here, the same pattern emerges: when typing, there is a clear disadvantage for learners with dyslexia, but this disadvantage disappears when learners can respond by speech.

At first inspection, the lower number of recalled items on the test for learners with dyslexia in the typing condition, might suggest that learners with dyslexia have poorer memory for these types of items. To investigate this, the researchers used a mathematical model to inspect the response times that were recorded during the learning session. They found that the parameters in the model that described the memory components in learning with dyslexia and in typical learners were highly similar. The main difference was in the time it took the learners to prepare a response. In the typing session, learners with dyslexia were much slower than typical learners in reading and processing the Swahili cue and in typing the correct answer, whereas there was no difference in retrieving the item from memory. 

These results indicate that there seems to be nothing wrong with the memory system of dyslexic learners per se, but that they need more time processing the written text and preparing the typed answer. It also explains why the learning disadvantage seems to disappear when learners can respond by speaking rather than by typing. When cues are presented verbally, and when learners can respond verbally, an accurate and quick mapping of letter combinations to their sounds is already provided to the learners by the learning system. This allows dyslexic learners to fully focus on storing and retrieving the information from memory, resulting in a more seamless learning experience.

Conclusions: Speech-Based Adaptive Learning Benefits Learners With Dyslexia

Learners with developmental dyslexia struggle with connecting the sounds of words to their associated letters in printed text, making it difficult for them to read, spell, and write. These issues affect various aspects of education, including the efficient memorization of facts like vocabulary items. Adaptive learning systems have enhanced the efficiency of fact learning by leveraging the benefits of retrieval practice, but these systems are often not well-suited for learners with specific learning disabilities due to their reliance on written text. In the study that is highlighted in this blog, Wilschut and colleagues investigated whether using a speech-based response modality, instead of the traditional typing-based approach, could improve vocabulary learning for dyslexic learners. They found that dyslexic learners memorized fewer words than typical learners when using the typing-based system. However, this difference disappeared when learners used speech-based learning. The results of the study provide a better understanding of the learning challenges linked to dyslexia, showing that the main difficulty lies in the time needed for encoding and producing typed responses rather than in memory processes themselves. This research paves the way for developing learning applications that cater to all learners, including those with specific learning disabilities, who are often underrepresented in educational tools.

Learn more? 

Do you want to learn more about the study discussed in this blog? A more extensive paper discussing the project is freely available at https://escholarship.org/uc/item/57w2x9vr. If you have any further questions or comments, please do not hesitate to contact our researcher Thomas Wilschut, the main author of this work, at thomas@memorylab.nl.

References

Fletcher, J. M., Lyon, G. R., Fuchs, L. S., & Barnes, M. A. (2018). Learning disabilities: From identification to intervention. Guilford Publications. 

Lindsey, R. V., Shroyer, J. D., Pashler, H., & Mozer, M. C. (2014). Improving students’ long-term knowledge retention through personalized review. Psychological science, 25(3), 639–647. 

Papousek, J., Pelánek, R., & Stanislav, V. (2014). Adaptive practice of facts in domains with varied prior knowledge. Educational Data Mining 2014, 6–13. 

Riddick, B. (2000). An examination of the relationship between labelling and stigmatisation with special reference to dyslexia. Disability & Society, 15(4), 653–667. 

Snowling, M., Hulme, C., & Nation, K. (2020). Defining and understanding dyslexia: Past, present and future. Oxford Review of Education, 46(4), 501–513.

Van Rijn, H., van Maanen, L., & van Woudenberg, M. (2009). Passing the test: Improving learning gains by balancing spacing and testing effects. Proceedings of the 9th International Conference of Cognitive Modeling, 2, 7–6. Wilschut, T., Sense, F., & van Rijn, H. (2024). Speaking to remember: Model-based adaptive vocabulary learning using automatic speech recognition. Computer Speech & Language, 84, 101578.

Get in touch with us

Fill out the form

Or send us a message

International Partnership Coordinator assisting you in adaptive learning solutions

Nathan McCabe

International Partnership Coordinator

Partnership & Operations Manager offering adaptive learning solutions

Saar Speijers

Partnership & Operations Manager​

Offerte aanvragen?

Vul het onderstaande formulier in:

Of mail naar:

Aan de slag!

Wilt u meer weten?

Vul het onderstaande formulier in:

Of mail naar: