Students in the Carnegie Mellon University chapter of Design for America (DFA @ CMU) have developed a free mobile app that uses visual cues to help children and adults — particularly those with language difficulties — form meaningful, grammatically correct sentences. The education app, called Sentence Mosaics, uses photos and color-coded parts of speech to prompt users as they construct sentences.

The app is useful for elementary school students and for a wide range of children and adults who are on the autism spectrum, or who have developmental disabilities, hearing disorders, stroke-related language deficits, dementia and other problems with language, said Alyse Baker, a certified speech-language pathologist who has honed the system for decades.

"Alyse has used colored blocks, colored cards and foam tiles attached to a foam board with Velcro as aids, but that's not practical outside the classroom," said Mehar Sawhney, a DFA @ CMU member and a senior majoring in mathematics and physics. "We worked with her to translate her system into an iPad app that can be taken anywhere."

Baker developed Sentence Mosaics based on long-standing research and practice within the field of speech-language pathology. She knew that transforming the system into an app would make sense, but found the prospect daunting.

"When I first looked into developing an app, I was told it would cost $20,000 to $30,000 to write the code," Baker said. "I did not have that much money in my hip pocket, so I knocked on doors for six years until a stroke of luck took me to David Kosbie." Kosbie, an associate teaching professor of computer science at CMU, connected her two years ago with DFA @ CMU, an interdisciplinary group of students who design and engineer solutions for social good.

A group of DFA members, including Sawhney, computer science majors Elizabeth Board and Sashank Gogula, and cognitive science major Elizabeth La, worked with Baker to translate physical aids into digital ones. Jennifer Chou, who graduated last spring with a computer science degree, was also a major contributor to the project.

Photos are key to the system. Users write sentences that describe actions depicted in photos, preferably ones that include the student or show a scene from their everyday life. Board said the app can import photos from an iPad or iPhone.

Baker explained the photos make the subject real to students, which for many is a necessary step before they can write. "Some of my students, if they didn't have a dog of their own, literally could not write a sentence about a dog," she added.

The app is designed to be used with a teacher, parent or other coach, who helps by loading photos into the app and designating a string of colored boxes that signal to the student which parts of speech — nouns, pronouns, verbs, adverbs, adjectives, etc. — are necessary to describe the photo.

"Without the intercession of Dave Kosbie and countless hours of programming by students at Carnegie Mellon, this app would not have become a reality," Baker said.

More information about the app is available on the Sentence Mosaics website. The free iOS app can be downloaded from the App Store.