Larry Osterman's WebLog

Confessions of an Old Fogey
Blog - Title

What happens when audio rendering fails?

What happens when audio rendering fails?

  • Comments 15

Skywing sent me an email earlier today asking me essentially "Why doesn't Windows do a better job of handling the case where the default audio device goes away?"[1]

It's a good question, and one that we've spent a lot of time working on for quite some time (this isn't a new issue for Vista, it's been there since day one, not that it actually matters).

For the vast majority of existing users, this isn't a problem - they only have one audio device on their machine, and thus its a moot question - their default device almost never goes away.  On the other hand, looking forwards, it's going to become a more common scenario because of the prevalence of Bluetooth audio hardware and USB headsets (for example, I'm currently listening to Internet radio over a pair of Bluetooth speakers I purchased the other day).

 

The short answer to Skywing's question is: "It's the responsibility of the application to deal with handling errors".  The audio stack bubbles out the error to the application and lets it figure out how to deal with the problem.

Which then begs the next question: "How do you handle errors so that you can recover from them?"  It turns out that the answer to that question requires a bit of digging into the audio stack.

 

As I've discussed in the past, there are four major mechanisms for accessing the audio functionality in Windows Vista.  They are:

  1. MME - the legacy MME APIs, including waveOutXxx, waveInXxx, and mixerXxxx
  2. DirectSound/DirectShow
  3. Media Foundation (new in Vista)
  4. WASAPI (new in Vista)

For the MME APIs, an application usually accesses the audio stack using the WAVE_MAPPER (or MIDI_MAPPER) pseudo-device.  The nice thing about the WAVE_MAPPER device is that it doesn't matter which device is the current device, it just uses the one chosen as the user's device.  Alternatively, for the MME APIs, you can select a specific device.  For the MME APIs, devices are numbered from 0 to <n>; for appcompat reasons, starting in Windows Server 2003, device 0 is typically the user's default device (there were applications that hard coded device 0 for their output device, which caused issues when you had more than one audio device).

For DirectShow and DirectSound, the call to the DirectSoundCreate API takes a GUID which represents the output or input device, or NULL to represent the default device.  In addition, it also provides a mechanism to address the default voice communications device (DSDEVID_DefaultVoicePlayback).

For Media Foundation and WASAPI, you specify the specific audio endpoint on which you want to render or capture audio in the initialize call (MFCreateAudioRenderer for MF, for WASAPI, you activate an IAudioClient interface on the endpoint).

In either case, once you start streaming to a device, the only mechanism in place when something goes wrong is that an API call fails.  That means that it's up to the application to figure out how to recover from any failure.

For the wave APIs and DSound, you really don't have any choice but to detect the failure close down your local resources and restart streaming - the legacy APIs don't allow you a good mechanism for detecting the cause behind a streaming failure.

For MediaFoundation, MF generates events using its' event generator mechanism to inform an application of interesting events; among the events that can be received is an event indicating that the audio device was removed.  There are other relevant events generated, including events that are generated when the audio service is stopped (which also stops streaming), when the mix format for the audio endpoint is changed, etc.

For WASAPI, a WASAPI client can retrieve the IAudioSessionControl service from an IAudioClient object and can use the IAudioSessionControl::RegisterAudioSessionNotification to register an IAudioSessionEvents interface.  The audio service will call the IAudioSessionEvents::OnSessionDisconnected method when it tears down an audio stream (all these notifications are also passed through to MediaFoundation's event mechanism).

In Windows Vista, there are six different disconnect events generated, and there's a specific set of recovery steps for each of them:

Disconnect Reason Meaning Recovery Steps
DisconnectReasonDeviceRemoval The device used to render the streams on this endpoint has been removed. Stop the stream, re-enumerate the audio endpoints and chose a new endpoint.  If your application is rendering to the default endpoint, just call IMMDeviceEnumerator::GetDefaultAudioEndpoint to determine the new default endpoint.
DisconnectReasonServerShutdown The audio service has been shutdown. Restart the audio service if possible; inform the user of the problem - there's no easy way of recovering from this one automatically.
DisconnectReasonFormatChanged The mix format for the audio engine has changed. Close any existing streams and reopen them in the new format.  Make sure that you rebuild your client side audio graph to ensure that you are generating output in the new mix format
DisconnectReasonSessionLogoff The user has logged off the terminal services session in which the audio session was running Close any existing streams.  It's highly unlikely that this notification will be seen since the operating system tears down the processes for a user when that user logs off
DisconnectReasonSessionDisconnected The user was streaming audio to the console and a TS client connected indicating that the server should redirect audio to the client (or vice versa) Treat this event like you would a DisconnectReasonDeviceRemoval event.
DisconnectReasonExclusiveModeOverride The user has opened an audio stream on the endpoint in exclusive mode.  This force-closes all shared mode streams (you can override this behavior in mmsys.cpl) Close any existing streams, return the error to the user or poll waiting on the endpoint to become available in the future.

 

 

[1] That's not the actual question that he asked, but the answer to his question is included in the answer to my question, so...

  • As a recent purchaser of a set of bluetooth headphones, I frequently run into the situation of loading

  • "For the vast majority of existing users, this isn't a problem - they only have one audio device on their machine, and thus its a moot question - their default device almost never goes away."

    This might be one advantage of OEM versions, even though the purchaser purchases them at retail (usually without an option to refuse) when buying a new PC.  With OEM versions, usually the PC's vendor has undertaken some amount of effort to get a more or less working driver installed.

    But it's still pretty easy to delete that device, accidentally, when doing trial and error to try to debug a problem.  The device might or might not have a driver provided by Microsoft in the Vista DVD, if there's a driver then it might or might not work, and users need to do some trial and error.  For some reason in Vista it's pretty easy to delete Vista's awareness of the device without intending to do so.  In XP it took some deliberate action to delete XP's awareness of a device.  Also in XP it wasn't hard to reverse that action and redetect the device.

  • Norman, I'm not aware of any way of deleting an audio device that doesn't involve the user using the device manager.  If you are aware of one, I'd love to hear about it.

    The only scenario where this matters is when you've got an existing audio device, you add a new audio device (bluetooth headphones, for example), then you remove the new device.  What happens at that point is up to the application.

  • Thanks for the answer, Larry :)

    For purposes of hacking around stupid apps, games in particular, I suppose the most logical place to start is to hook DirectSound and provide a logical abstraction of there never being a loss of the device (when such occurs, recreating the internal, "real" DirectSound objects that the abstraction/hook passes through to for most DSound requests, initializing them as the application initialized the abstraction at start time, and set buffers back up).  I'd imagine there will likely be some glitching in doing all of that, with switching sound buffers midstream without the cooperation of the app, but for the fairly infrequent case of switching audio devices, I'd be willing to live with that.  It's much better than losing sound indefinitely, in any case.

    Aside from WMP, most of the things that I use on a regular basis which are broken like this are likely to be DirectSound.  I'm hoping that WMP can be fixed at some point and save me from having to do similar hackage with whatever API set it uses internally, though :)

  • "I'm not aware of any way of deleting an audio device that doesn't involve the user using the device manager.  If you are aware of one, I'd love to hear about it."

    Taking a glance at a system where that doesn't happen but seeing what menu options pop up, I'll make two vague guesses here.  One is that when the volume mixture applet displayed a device that was disabled due to having no working driver, it was easier to unintentionally delete all knowledge of the device than it was to try to find a working driver.  One is that when the speaker icon had a red X because the sound chip didn't have a working driver, that icon's context menu had something similar.  If I have time to experiment on one PC again then I might be able to reproduce this.

    I have a third vague guess too.  On a PC that I no longer have, the audio system worked fine with Windows Media Player but not with Windows Media Centre.  The dialogs in Windows Media Centre were more confusing.  There was a statement something like "Please reinstall Windows or reinstall Media Centre" which I actually tried doing several times (as an invited beta tester) but had no effect (as I predicted).  There were some other options which didn't help, and I don't remember if this might have been the PC where I ended up entirely deleting the audio without having that intention.  But I no longer have that PC, and its current owner is using it with its XP licence.

  • When audio rendering failed, I felt a great disturbance in the Force, as if millions of voices suddenly cried out in terror and were suddenly silenced. I fear Larry Osterman's ping of death has arrived.

  • Hi Larry,

    You said to Norman:

    "I'm not aware of any way of deleting an audio device that doesn't involve the user using the device manager.  If you are aware of one, I'd love to hear about it."

    I used the contact button on your blog a few days ago as your blog hadn't dealt with the topic yet about Vista behavior regarding unplugged devices.  It seems that some new device drivers shipped with Dells for Sigmatel based devices don't report any devices at all when you query what's connected with waveInGetNumDevs.  If you plug in a 3.5mm jack, suddenly they do report devices.  

    I realize waveInGetNumDevs is an old legacy call, but is that behavior expected on Vista?

    It wouldn't surprise me if these same devices also magically removed the device if you unplugged them.  Unfortunately, these problems don't show up here in testing, only on end-user machines.

    Anthony Wieser

    Wieser Software Ltd

  • Anthony, sorry I must have missed that email.  This is absolutely by design, it's a feature called jack detection - essentially an audio device only becomes available when something's plugged into it.  And yes, the wave device will disappear when it's unplugged.

    But the audio driver isn't removed in that scenario.

  • Thanks for the clarification Larry.

    I guess I misread the original post when I suggested it was removed.  Is there also a notification when the device is unplugged?  What will my App see if that happens?  

  • Anthony: Yes, there is, but it comes from a different mechanism - MMDeviceAPI fires a device state change notification with "Device Unplugged" when you unplug a device.

  • > essentially an audio device only becomes available when something's plugged into it

    Got it.  Then I know which PC it was happening to with me, and it was happening when the external speakers had their AC power turned off.

  • "This is absolutely by design, it's a feature called jack detection - essentially an audio device only becomes available when something's plugged into it."

    I personally find that a really annoying feature. I don't normally have speakers plugged into my computer at work, but sometimes I'm asked to check why a WMP plugin on a webpage is not working (usually someone forgot to upload the .mp3 file!) but if I go to access the page, I just get an error saying there are no audio devices. All I want to see is the "file not found" etc error. I have to go hunting around for a pair of headphones to plug in just so I can get past it...

    Is there anyway to turn that feature off?

  • Dean Harding,

    This issue just reeks of DRM (it checks protected media path). I'm sure Larry or someone else will deny, but he/she better provide a good explanation...

  • Grof: DRM won't affect a rendering stream.

    Dean: No - the jack detection logic exists because we (and our hardware partners) got a stream of complaints from customers who kept on complaining that they didn't hear any output from their computer when the reason was that they hadn't plugged anything into the jack.  

    In all honesty, I suspect that the number of people who WANT to be able to play audio without hearing the audio is relatively small, especially when compared to the set of people who want an error to occur when they attempt to play audio through unplugged speakers.

  • Larry: I guess so :-) I guess I could just buy some $2 headphones and leave them plugged in...

Page 1 of 1 (15 items)