Blog So how does the Starling actually work?

So how does the Starling actually work?

by Jonathan Simon

Over 30 years of scientific research has shown that the more words spoken to a child between birth and 4 years (when 82% of the brain develops), the more likely they are to reach their intellectual potential. Speaking to a baby is relatively easy (once you get the hang of it), but how do you actually count the words you’re saying?

Enter Starling. Don’t let its cute looks fool you, the Starling’s a pretty advanced little piece of technology. So it’s no surprise that everyone’s first question is always, “How does it work!?”

how to use the Starling  

First, let me explain what the Starling “does.” Place the Starling near your baby and start talking. In real-time, the Starling processes the audio signals from your verbal engagements, turning them into data that will tell you just how many words your baby is being exposed to. Of course, the actual process is much more complex.

Ready for a deeper dive? I’m one of the data scientists who helped pioneer the digital smarts that make the Starling count words. And while counting words sounds as easy as speaking them, it’s not. Imagine a page of text in a book. That’s the written form of how we talk, right? Well, sort of. Spoken speech is much less well defined when it’s viewed in its true form – sound waves. For example, here’s what the spoken phrase, “middle of the night” look like as sound waves…

sound waveform #1  

A sound wave is made of one continuous line that tracks speech. Get it? It’s wavy. The high parts tell you how loud the sound is, and the low parts show how quiet it is. Given this info, I bet you’d be surprised to see where the actual words appear in the waveform.

sound waveform #2  

See? The sounds all sort of blend together. This is why turning waveforms into words to count (even when you know the words you’re looking for) is so hard. But wait – it gets even harder.

Take that fancy digital audio assistant on your smart phone. When you push a button to ask it directions, it’s not processing all your words when you speak them. It’s cheating by just listening for hints of words (called phonemes) and sending them to a gigantic server farm for processing. The Starling doesn’t do that. Without making you push a button before speaking, without recording your interactions, and without needing to be constantly connected to Wi-Fi, the Starling uses sophisticated, proprietary algorithms to pull your words out of the waves, filter them for proximity, remove background noise, calibrate itself to accommodate loud talkers and whisperers, and deliver a word count to your smart phone. All in real-time.

how the Starling works  

So now you have an idea of how powerfully bright our little star truly is. And if someone says, “that’s a lot of trouble just to count words,” you can gently correct them. “It’s not just counting words. It’s helping me give my child the best future possible.”

About the Author

Jonathan Simon

Jonathan Simon is a Data Scientist at VersaMe. In college he majored in math while pursuing interests in psychology and cognitive science. Noticing that working with sophisticated pattern recognition algorithms used in modern data analysis felt like tinkering with a simplified brain, he turned his attention toward data science. Jonathan completed his computer science masters program at Cornell University, where he tailored his coursework around machine learning. When not crunching data, he’s crunching leaves on hikes through nature.

Counting words can change a life

What’s in a Starling Start Number?