Within Windows

Microsoft has just finished their investigation into the Windows Phone location data transmission claims originally brought forward by security researcher Samy Kamkar (and later dissected/validated by me). A few statements came out; let’s start by dissecting their press statement:

We have completed our investigation into the Windows Phone’s location service and the unintended sending of Wi-Fi access point and cell tower information.  We’ve posted what we learned and the steps we’ve taken to correct these behaviors on our Location and Privacy FAQs page and our privacy statement found at http://mango.microsoft.com/windowsphone/en-US/howto/wp7/web/location-and-my-privacy.aspx and http://mango.microsoft.com/windowsphone/en-us/privacy.aspx.

Microsoft is committed to user privacy.  As our Privacy Statement explains, the Windows Phone location service uses and stores only limited information about nearby Wi-Fi access points and cell towers, which we use to help provide users with location services more efficiently and effectively.  Most importantly, it does not use or store any information that identifies users or uniquely identifies a device, and Windows Phone does not track users or their devices.

Okay, that was simply a teaser to the new bits added to the privacy policy:

This notice describes unintended behavior in the Windows Phone software involving location services, explains how Microsoft is eliminating that behavior, and reminds you that you can prevent access to location information if you choose to do so.

As described in the Location Services section of this Privacy Statement, the location information stored and used by the Windows Phone location service is limited information about nearby Wi-Fi access points and cell towers that we use to help provide you with location services more efficiently and effectively. It does not include any information that identifies you or uniquely identifies your device and does not allow Microsoft to track you or your device.

The “Windows Phone location service” referred to here is really an entity representative of a few moving cogs – e.g. inference.location.live.net and agps.location.live.net. Take note of the language used here – Microsoft doesn’t discount the fact that PII may exist in the packets on the wire. They simply reaffirm that the data isn’t stored or used. (Seems odd it would be transmitted if unused but eh.) As we have no insight into how these services work, we have to trust Microsoft here. And I think we should.

We have identified an unintended behavior in the Windows Phone 7 software that results in information about nearby Wi-Fi access points and cell towers being periodically sent to Microsoft when using the Camera application, and, for phones that are configured for US-English, when using the phone’s voice command features (such as "Find Pizza"). For the Camera, the software bug results in the behavior even where you have disabled geo-tagging photos in the Camera application.

The Windows Phone 7.5 update eliminates this unintended behavior by the Camera application and voice command feature. After the update, information about nearby Wi-Fi access points and cell towers will be sent when using the Camera application only if you have agreed to tag your photos with location. For voice commands, location information will no longer be requested and information about nearby Wi-Fi access points and cell towers will not be sent to Microsoft when using voice commands.

The language here is a bit confusing. They identified an unintended behavior – singular – but talk to issues in both the Camera and voice command features. I suspect this is because, like I mentioned in a previous post, these modules simply hit a bug specific to the location services code on the phone. Not necessarily application-specific bugs. But hey, they’re fixed now. This behavior aligns perfectly with their U.S. House of Representatives letter now.

We also have identified that the Windows Phone 7.5 update contains an unintended behavior when using the "Me" feature in the People Hub. Each time you access the "Me" feature, information about nearby Wi-Fi access points and cell towers is sent to the Windows Phone location service. The information sent, received and stored by the Windows Phone location service when you use the "Me" feature does not identify you or your individual device. Nevertheless, this behavior is unintended and will be eliminated as part of the next scheduled update to Windows Phone 7.5. After that update, information about nearby Wi-Fi access points and cell towers will be sent only if you have agreed to allow the "Check In" function of the "Me" feature to access and use location information.

Ruh roh! Looks like a worm snuck into the “Mango” shipment. It turns out, the “Me” tile has a similar location leakage problem as well. (This applies to people using the built-in Twitter or check-in functionalities.) Kudos to Microsoft for coming clean here, way before the press spun it out of control.

You will receive a notice on your phone when software updates are available, and you can always disable all access to location information by applications and collection of location information by the Windows Phone location service at any time by going to Settings > Location and toggling the location switch to OFF.

Finished off with an easy workaround.

Last week, I tested location-reporting features present in versions of Windows Phone OS ranging from RTM (Gold) to 7.0.7392. All of the versions exhibited the same behavior – they sent a bunch of location data before an application requested it (which appeared to contradict what Microsoft had said earlier). With the distribution of OS version 7.10.7720.68 (codenamed Mango) today, I went ahead and updated my test Samsung Focus phone.

I have confirmed that Windows Phone “Mango” no longer sends location data prior to being granted permission to do so. The behavior I’m now seeing is perfectly aligned with Microsoft’s letter to the U.S. House of Representatives (Exhibit A, emphasis mine):

[1. User Choice and Control.] Microsoft does not collect information to determine the approximate location of a device unless a user has expressly allowed an application to collect location information. Users that have allowed an application to access location data always have the option to access to location at an application level or they can disable location collection altogether for all applications by disabling the location service feature on their phone.

[2. Observing Location Only When the User Needs It.] Microsoft only collects information to help determine a phone’s approximate location if (a) the user has allowed an application to access and use location data, and (b) that application actually requests the location data. If an application does not request location, Microsoft will not collect location data.

Some questions still buzzing around in my head – was the previous behavior incorrect? What will become of Case 01438? Also curious is that “Mango” went RTM July 26. The case paperwork was filed August 31, 36 days later. Was this bug already known prior to the case?

Before the BUILD conference, I dissected a thin report written by security researcher Samy Kamkar on the topic of Windows Phone and how it handles location data. With BUILD now behind us, I took a moment to test his claims on a legitimate device (Samsung Focus) acquired from my good friend Adam Maras.

Starting with Windows Phone OS 7.0.7004.0, I reset the device and tapped my way through the out-of-box experience, skipping the Live ID configuration. The states of location-sensitive features were as follows:

• Airplane mode: Off
• Wi-Fi: Off
• Bluetooth: Off
• Location: On
• Cellular: SIM error
• Find my phone: Not set up yet
• Feedback: Disabled

I then configured Wi-Fi access and immediately pointed the phone to a proxy server – in this case, my desktop running Fiddler software, which allows me to see packet details in real time. According to Kamkar, launching the Camera application was enough to see the culprit behavior, so I tried it. After launching the app., Fiddler captured location data being sent to and from Microsoft servers, just as Kamkar’s report suggested. Uh oh!

A few packets were sent, one to agps.location.live.net and several to Microsoft’s Location Inference (codenamed Orion) service hosted at inference.location.live.net. Items transmitted include (but aren’t limited to):

• OS Version (7.0.7004.WM7_7.0_Ship(mojobld).20100916-1429)
• Device Information (SAMSUNG/SGH-i917 and SAMSUNG Electronics/SAMSUNG MITs/i917UCJJ1/[digits])
• Wireless access points around me (MAC addresses, power levels)
• Various GUID-based identifiers

In response to these packets was pin-point accurate positioning information –  all before I granted the Camera application access to location data. But let’s think this through – did the Camera application really receive any data? Not likely. More probable is that the Camera application woke up the Location service on the phone. A conversation like this probably occurred:

Camera app: “Hey, I need you to get ready, I’m about to request location data”.
Location service: “Sure thing, boss. While you’re busy, I’ll figure out where I am and cache the results.”

But it doesn’t matter what piece of code is responsible. This behavior appears to contradict Microsoft’s earlier statements to the U.S. House of Representatives (Exhibit A, emphasis mine):

[1. User Choice and Control.] Microsoft does not collect information to determine the approximate location of a device unless a user has expressly allowed an application to collect location information. Users that have allowed an application to access location data always have the option to access to location at an application level or they can disable location collection altogether for all applications by disabling the location service feature on their phone.

[2. Observing Location Only When the User Needs It.] Microsoft only collects information to help determine a phone’s approximate location if (a) the user has allowed an application to access and use location data, and (b) that application actually requests the location data. If an application does not request location, Microsoft will not collect location data.

In my case, the phone determined its exact location via Microsoft services prior to me explicitly allowing such behavior. The question is whether the Microsoft servers in question are in fact collecting data about the phone or simply returning this information with no storage abilities. Only Microsoft can tell us what they’re doing with this information. (Mind you, Microsoft has had some issues in the past with this.)

I re-tested the phone after every update (7008, 7390, and 7392) with no change in behavior. When Mango ships in 1-2 weeks, I’ll test that too. Stay tuned.

Whether it’s to look up an API, find cheap plane tickets, or simply gaze at the beautiful picture du jour, I spend a lot of time on Bing. Kicking it up another notch, the Bing team just announced (and implemented) HTML5 video on the home page and it’s pretty darn slick. It reminds me of Windows DreamScenes, without the crashing.

If you’re one of those unlucky souls that lives in an ignored part of the world, I’ve captured a clip for you to see.

As you probably know by now, I work with Garrett Serack on the CoApp project when time permits. In our adventures to bring open-source software to Windows, we constantly fall into deep pits that require quite the climb back out. In this case, Garrett had a simple piece of code that was making a HTTP HEAD request. While this code executed swiftly in his user account’s context, it was taking ages when running as SYSTEM.

I created a small repro. executable to validate his issue and used psexec –s –I –d cmd.exe to launch an instance of the Command Prompt, in the SYSTEM context.

Stopwatch w = new Stopwatch();
w.Start();

try
{
    var req = WebRequest.Create("http://coapp.org/foo");
    req.Method = "HEAD";
    var res = rq.GetResponse();
}
catch (Exception e)
{
    w.Stop();
}

Console.WriteLine(w.ElapsedMilliseconds);

Sure enough, it was slow. At this point, I knew I needed to profile this code but with the unmanaged/managed boundaries, I didn’t feel like messing around with that. So, instead I did some research on System.Net and debug facilities. It turns out, System.Net is packed full of diagnostics including a Network Tracing feature. Cool!

Following the directions, I enabled Network Tracing for my small test program, gave it another whirl (again, in the SYSTEM context), and dug out the trace log.

System.Net Verbose: 0 : [6932] WebRequest::Create(http://coapp.org/foo)
    DateTime=2011-09-23T01:38:05.2033494Z
System.Net Verbose: 0 : [6932] HttpWebRequest#21454193::HttpWebRequest(http://coapp.org/foo#960239574)
    DateTime=2011-09-23T01:38:05.2033494Z
System.Net Information: 0 : [6932] Current OS installation type is 'Client'.
    DateTime=2011-09-23T01:38:05.2053495Z
System.Net Information: 0 : [6932] RAS supported: True
    DateTime=2011-09-23T01:38:05.2063496Z
System.Net Verbose: 0 : [6932] Exiting HttpWebRequest#21454193::HttpWebRequest()
    DateTime=2011-09-23T01:38:05.2243506Z
System.Net Verbose: 0 : [6932] Exiting WebRequest::Create() 	-> HttpWebRequest#21454193
    DateTime=2011-09-23T01:38:05.2243506Z
System.Net Verbose: 0 : [6932] HttpWebRequest#21454193::GetResponse()
    DateTime=2011-09-23T01:38:05.2243506Z
System.Net Error: 0 : [6932] Can't retrieve proxy settings for Uri 'http://coapp.org/foo'. Error code: 12180.
    DateTime=2011-09-23T01:38:18.7941268Z
[...]

I snipped the output for brevity, but the important line here is the last one. I looked up that 12180 error code and it resolved to ERROR_WINHTTP_AUTODETECTION_FAILED – an error code returned by WinHttpDetectAutoProxyConfigUrl.

A thread stack with the culprit function highlighted (Process Explorer).

To verify this function was the culprit, I fired up Process Explorer then started the repro. executable. As soon as it appeared in the process tree, I right-clicked it and clicked Suspend. I then double-clicked it, navigated to the Threads tab, and checked out the stack for each thread. Bingo. The main thread, in my case, was invoking WinHttpDetectAutoProxyConfig and taking forever.

At this point, I started thinking – where have I seen automatic proxy configuration before? Oh, that’s right. Internet Explorer.

Since the Bronze Age, Internet Explorer has had the option of controlling whether or not your system automatically detected your proxy settings via Web Proxy Auto Discovery (WPAD) (which works over both DNS and DHCP). That means when this automatic detection occurs, you’re making DNS and/or DHCP requests, hence the slow down.

“Automatically detect settings” option present in Internet Explorer

While Internet Explorer had this feature disabled in my user’s context, I suspected it was enabled in the SYSTEM context. Using psexec again I launched Internet Explorer (psexec -s -i -d "%ProgramFiles%\Internet Explorer\iexplore.exe") and navigated to Tools –> Internet Options. Sure enough, it was checked. Blasted!

Unchecking this option took effect immediately and dropped the delay in my repro. executable. Problem solved, at least on my machine.

Unresolved issues/comments

• If I enable this option in both SYSTEM and user contexts, why does this detection take longer in SYSTEM? Winsock initialization?
• We’re looking to move this web request logic back into the user’s context, to take advantage of the auto detection cache. (Users typically run more than one Internet-aware application which means this cache will be primed/valid more often than in the SYSTEM context.)