The shouting match
Does shouting at a speech recognition system make it work better?
Here's a good example from the front lines - a frustrated human shouting Main menu! at the top of her voice (audio via her sister-in-law). But I bet the recognizer didn't get it. To cut an interesting story unjustly short: speech recognition systems are trained on the acoustic features of "typical" speech, i.e. people talking normally. Shouting distorts those features, not just in terms of higher amplitude, but also in all sorts of other ways, phonetic and prosodic.
So only if the underlying engine had been trained largely on irate speakers would she have had a better shot at recognition by shouting. But sufficient amounts of shouted data are probably not commercially available, and even if they were, an engine trained on that data would produce so many misrecognitions for people with sweet tempers that those people would probably soon turn bad and become dangers to society.
But then again, what's a human to do here? The machine doesn't recognize your polite, restrained voice, even after a courteous repetition or two. You have to try something else, and your only tool is 8 KHz audio and a two- or three-second window. Wait... it's a computer. And when you're training the dog and Rover doesn't get it the first couple of times, shouting actually does work some of the time. Plus you've read somewhere that computers aren't even as smart as dogs, so why not shout even louder?
Or is it more about making the machine suffer? You were gracious enough to grant the system a response to its question - even though it's a machine - and you gave it your clearest articulation, twice, and now it's trying to tell you that you can't even speak your own language! The cheek - got to teach that machine a thing or two. Main menu! That's better. Always good to offload a wad of phonetic and prosodic distortion down the line at high amplitude. One up for the humans!
Or was it a cry for help? Stay with me, now. Frustration is often about the lack of control. Different people have different ways of reacting to machines that don't do what they want them to do (when was the last time you swore at your hardware? Or operating system? ;-) Sometimes it even gets physical. Except you can't get physical with a machine at the other end of the phone line, your options are much more limited. All you've got is your voice, so shouting is the only way to express the anguish of powerlessness. Right?
Enough dodgy psychology. This particular story ends nicely - the company behind this auto-attendant obviously keeps an eye on their data and they even redesigned the system on the basis of feedback from someone's Dad.
And back to the original question: does shouting at a speech recognition system make it work better?
The answer is no, not immediately. It will undoubtedly make it worse (but it may make you feel better). However, the answer may also be yes in the long term. If someone behind the scenes is watching the logs and listening to the audio, you could be helping to make the system better for the next person. (Was "main menu" in the grammar then? Maybe it is now.) Deploying the application doesn't mean that the company's job is done, far from it. If some of the time spent on application design was devoted to closing the feedback loop of user input after initial trials and deployment, there would be a lot less shouting, and a lot more happy customers. There's no data like angry data.
