Within Windows

TIP: If you’d like to skip the technical lead up to the conclusion, skip to the last two paragraphs of this post.

While poking around my system, attempting to discover a reason for my long-standing NVIDIA GPU issues (surprised?), I went into panic mode when I discovered the audiodg.exe process having over a million handles open. As the process was a Microsoft-owned process, I thought nothing more of it and restarted the Windows Audio Endpoint Builder service. Problem solved.

… or not. The problem resurfaced when I fired Pandora back up. Counting thousands, it appeared the process was opening approximately twenty handles every second. To what though?

I fired up Process Explorer, reconfigured to show the handles column and clicked on the audiodg.exe process. Process Explorer dumped a huge listing of registry handles, in the bottom pane, that were left open.

I wasn’t convinced Microsoft had missed such a bug so I did some research on the audiodg.exe process. Of course Larry Osterman had an entire post on the matter, which is always nice. Of particular note were the following sentences:

There are two reasons it [audiodg.exe] runs outside of the windows audio service. The first is that there’s 3rd party code that gets loaded into audiodg.exe. Audio hardware vendors have the ability to install custom DSPs (called Audio Processing Objects or APOs) into the audio pipeline… The second reason for using a separate process for the audio engine is DRM.

Ah ha, so I really couldn’t blame audiodg.exe for opening the handles. I needed to see exactly who was responsible for opening these handles…

I stopped the Windows Audio and Windows Audio Endpoint Builder services and fired up Application Verifier and configured it to keep track of audiodg.exe’s handle usage.

After turning down my speakers, I opened WinDbg and invasively attached to the audiodg.exe process. To view the extra information that Application Verifier helped me collect, I issued the !htrace command with one of the handle values I wrote down earlier as a parameter.

Aside from the handle and thread information at top, I was given a stack dump of the functions called resulting in the open handle. Reading from bottom up, I came across a third-party module named SFSAPO64. A quick search on my system resulted in a library within the System32 folder owned by Sonic Focus, Inc., and a quick Google search verified the library was part of my SoundMax driver set.

The driver component appeared to be quite dated so I went to the ASUS support site in an attempt to download newer drivers. After stumbling around for a nice chunk of time, thanks to the pathetically slow server and cumbersome interface, I verified I was using the latest drivers (for this board/chipset combo). Damn.

Before I could jump up and down and claim I found a bug, I had to be sure. I fired up IDA Pro and drilled down into the area WinDbg pointed me to. I found this particularly interesting snippet of assembly (comments in green):

call    cs:RegGetValueW
test    eax, eax
mov     ebx, eax
jnz     short loc_40CAF4 ;// close key and return
mov     eax, [rdi+9FCh]
cmp     [rsp+1B8h+var_174], eax
jnz     short loc_40CAA9 ;// continue and get ParamData value
lea     eax, [rbx+2]
jmp     short loc_40CB01 ;// jump to end of function and return

Very roughly translated, the code probably looked something like this, before compilation (and optimization):

if(RegOpenKey(...) == ERROR_SUCCESS)
{
  if(RegGetValue(...,"ParamDataTime", pcbData) == ERROR_SUCCESS)
  {
    if(pcbData != g_someVariable)
    {
      if(RegGetValue(...,"ParamData", pcbData) == ERROR_SUCCESS)
      {
        /* do something neat */
        RegCloseKey(...);
      }
      else
      {
        RegCloseKey(...);
      }
    }
  }
  else
  {
    RegCloseKey(...);
  }
}

Upon analysis, I verified that if the conditional pcbData != g_someVariable was false, which it was most of the time, the function immediately returned, not calling RegCloseKey(). Oops.

Having confirmed the bug, I then did some research to determine the severity of the issue. Due to how handles are tracked, each process is allowed to open approximately 16 million handles. Handles are derived from a special pool of memory called the paged pool, which is basically a bunch of reserved memory that can be paged out to disk, if free memory is needed for other tasks. In the past, with a similar handle leak, you probably would have exhausted your paged pool (and filled up your page file) before hitting the ~16 million handle limit. Today, however, memory is cheap and abundant therefore hitting that limitation is very feasible.

Assuming I left the system running and playing audio for weeks and reached the open handle limit, the worst case scenario would’ve probably been crashing of the Windows Audio services, muting my sound output, causing me to restart them.

I contacted both Analog Devices Inc. (ADI) and Sonic Focus to alert them of the problem. Two days later (today), I spoke to a Sonic Focus representative confirming the issue and informing me the driver component was already fixed, already in WHQL testing, and that a set of fixed drivers should be available this week. Impressive. Keep an eye on Sonic Focus’ support page for the updated drivers!

NOTE: For non-Blackhawk users, like me, we have to wait a tad longer for Analog Devices Inc., to create a SoundMax bundle specific to our motherboards. I’ll share more information on availability when I receive it.

Until newer drivers are made available, you can do one of two things:

1. Keep an eye on the leak, restart the audio services if need be, or…
2. Disable audio enhancements via the Sound control panel applet (it’s a little checkbox under Signal Enhancements in the Speakers Properties window, Advanced tab).