Windows Core Networking

Final version of Windows Web Services API for Windows XP, Vista, Server 2003 and Server 2008 is now available!

Ari B. Pernick — Fri, 09 Oct 2009 23:32:00 GMT

The final version of the Windows Web Services API for Windows XP, Windows Vista, Windows Server 2003 and Windows Server 2008 is now available. This version corresponds to the final version of the WWSAPI released as part of Windows 7 and Windows Server 2008 R2. To develop for this version of the runtime, you need to install Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1. Please make sure to regenerate all code using the new version of wsutil.exe tool. You will also need to rebuild all source code that used any of the pre-released versions of this API.

This release is available in 25 languages: Arabic, Chinese - Hongkong, Chinese - Simplified, Chinese - Traditional, Czech, Danish, Dutch, English, Finnish, French, German, Greek, Hebrew, Hungarian, Italian, Japanese, Korean, Norwegian , Polish, Portuguese (Brazil), Portuguese (Portugal), Russian, Spanish, Swedish, Turkish. The table below summarizes CPU architectures and the minimal level of Service Packs supported by this release. Make sure to deploy the latest service pack and all critical updates for the version of Windows to computers on which you plan to install this release. To find recent security updates, visit Windows Update.

Windows Version	Service Pack	x86	x64	ia64
Windows XP	SP3	Yes	Yes	No
Windows Server 2003	SP2, R2 SP2	Yes	Yes	Yes
Windows Vista	SP1	Yes	Yes	No
Windows Server 2008	SP1	Yes	Yes	Yes

In contrast to the past pre-released version of this API, this final version release can be used in production code and redistributed with the final versions of the product. Because of this, the final version is only available to companies who agree to terms of Windows Master Redistribution License Agreement (MRLA). To acquire the redistributable installers for this release and a copy of the Windows MRLA for review, please email a formal request to wwsredst@microsoft.com. In your email, please include a brief description of your plans for using this release and the business contact information for your company using which you can be reached at later on. If you have questions about this release, please email them to wwsredst@microsoft.com.

Why didn’t my cached content revalidate to the web server? (IE8 edition)

Ari B. Pernick — Mon, 06 Jul 2009 23:33:50 GMT

For the super eagle eyed developer or Web Server admin, you might have noticed that cache entry validation happens slightly less frequently in IE8. I’m talking about the check to the Web Server (via If-Modified-Since) to see if the cached version of content needs updating or is still current. The short version of why this happens is the way that IE uses WinINet sessions changed to accommodate Loosely Coupled IE and session’s start time is an important factor in evaluating if WinINet needs to issue an If-Modified-Since request.

Digging deeper, we can look at a couple of the factors in the logic IE/WinINet is using to decide to issue an If-Modified-Since request. First is the the Cache Setting: “Check for newer versions of stored pages” in the “Temporary Internet Files and History Settings” as described in KB article 263070 and pictured with this blog entry. These are settings that apply to cached content missing the Expires headers. “Every time I visit the webpage” and “Never” are pretty straightforward. “Every time I start Internet Explorer” and “Automatically” have logic dependant on the the session’s start time. Another factor to note that we have different behavior when using the Back and Forward browser buttons compared to what we do in hyperlink navigation. The Back and Forward behavior also depends on the session’s start time when content is expired or is missing the Expires header.

How does Loosely Coupled IE in IE8 change the session start time? Previously any IE window got a single session in WinINet by the fact it only uses a single process and sharing of the session handle across tabs, with Loosely Coupled IE we had to do work to let multiple processes share session state. This means that new IE8 windows share the same session compared to new IE7 windows which get a new session. Eric Lawrence on the IEBlog talks a bit more about this session sharing behavior. The effect on WinINet is that IE8 sessions last longer on average then they used to and that means potentially less often revalidation of content from the Web Server.

-- Ari Pernick

HTTP Connection Management

Ari B. Pernick — Fri, 08 May 2009 19:29:00 GMT

HTTP is a request/response protocol. You request some resource like the HTML of a webpage and the response comes back with the HTML attached. As each request is sent on a connection, the complete response must be read from the connection before the next response can be read. (There is an optimization where you make more requests on a connection before receiving the full response to the previous requests known as pipelining. However it doesn’t change the need to receive all the data from the previous requests first.)

So if an application is asking for another request to a server while the current one is in progress, what’s a HTTP Client API to do? Well, it can either wait till the connection is available or start a new connection. This decision is mostly controlled by a configurable maximum number of simultaneous connections to a server.

With that in mind, how should the following customer question get answered?

My client application’s performance is getting impacted by WinInet opening new SSL connections to my server rather than reusing the existing connection. How do I fix this?

QoS Traffic Generator Example Usage

Ari B. Pernick — Fri, 08 May 2009 02:47:00 GMT

Update: This tool is no longer available on Microsoft Connect and there is no replacement download site.

In my last post, I announced the availability of QoS Traffic Generator on Microsoft Connect. In order to help you understand the usage of this tool, I have provided some examples of what it can do.

One of the most common tasks is to send traffic from one computer to another. The following example sends UDP traffic at 5 Mbps for 10 seconds.

Source

C:\qos>qostraffic.exe -source -udp -dest 192.168.1.100 -throttle 5000000 
                      -duration 10
Parsing command line...
WSAStartup successful.
Time between each packet (microsec): 2400
Size of each packet: 1472
Statistics sampling interval (msec): 1000
Sending traffic at 5.00 Mbps to 192.168.1.100:9999
Date, Time, Packets received, Bytes received (headers included), Elapsed time (msec), Throughput (Kbps), Bottleneck BW, Available BW, RTT, BBSet, ABSet, RTTSet, ErrorCode
05/07/2009, 23:06:54, 421, 631500, 1014, 4982.249,,,,,,,
05/07/2009, 23:06:55, 422, 633000, 1014, 4994.083,,,,,,,
05/07/2009, 23:06:56, 423, 634500, 1014, 5005.917,,,,,,,
05/07/2009, 23:06:57, 422, 633000, 1014, 4994.083,,,,,,,
05/07/2009, 23:06:58, 423, 634500, 1014, 5005.917,,,,,,,
05/07/2009, 23:06:59, 422, 633000, 1014, 4994.083,,,,,,,
05/07/2009, 23:07:00, 423, 634500, 1014, 5005.917,,,,,,,
05/07/2009, 23:07:01, 422, 633000, 1014, 4994.083,,,,,,,
05/07/2009, 23:07:02, 423, 634500, 1014, 5005.917,,,,,,,
Time has elapsed, or CTRL+C has been pressed.
Stopping Source traffic, waiting 5 sec.
SenderThread is exiting...
SenderThread exited successfully.
WSACleanup successful.

Sink

C:\qos>qostraffic.exe -sink -udp
Parsing command line...
Listening on all IP addresses.
WSAStartup successful.
Waiting for UDP traffic on port 9999, hit CTRL+C to exit...
WSARecvFrom completed.
Date, Time, Packets received, Bytes received (headers included), Elapsed time (msec), Throughput (Kbps), Message
05/07/2009, 23:06:54, 421, 631500, 1014, 4982.249,
05/07/2009, 23:06:55, 422, 633000, 1014, 4994.083,
05/07/2009, 23:06:56, 423, 634500, 1014, 5005.917,
05/07/2009, 23:06:57, 422, 633000, 1014, 4994.083,
05/07/2009, 23:06:58, 423, 634500, 1014, 5005.917,
05/07/2009, 23:06:59, 422, 633000, 1014, 4994.083,
05/07/2009, 23:07:00, 423, 634500, 1014, 5005.917,
05/07/2009, 23:07:01, 422, 633000, 1014, 4994.083,
05/07/2009, 23:07:02, 423, 634500, 1014, 5005.917,

You can see the throughput in Kbps from the last column, along with other information such as timestamp and bytes received. The QoS Traffic Generator can also support multiple TCP connections from one sink. The next example shows two TCP flows, one using qWave and one using TC (used to throttle traffic) sending to the same destination.

Source 1

C:\qos>qostraffic.exe -source -tcp -dest 192.168.1.100 -throttle 1000000 -qwave
Parsing command line...
qWave traffic type: Best effort
qWave flow type: non-adaptive.
qWave LLTD option: system choice.
qWave flow fundamentals: do not display
WSAStartup successful.
qWave started up successfully.
Send interval: not used, packets will be sent as fast as possible
Packets to send per interval: 1
Size of each packet: 1000
Statistics sampling interval (msec): 1000
Sending traffic at 1.00 Mbps to 192.168.1.100:9999
Date, Time, Packets received, Bytes received, Elapsed time (msec), Throughput (Kbps), Bottleneck BW, Available BW, RTT, BBSet, ABSet, RTTSet, ErrorCode
05/07/2009, 23:02:04, 129, 125000, 1014, 986.193,,,,,,,
05/07/2009, 23:02:05, 126, 126000, 1014, 994.083,,,,,,,
05/07/2009, 23:02:06, 125, 125000, 1014, 986.193,,,,,,,
05/07/2009, 23:02:07, 126, 126000, 1014, 994.083,,,,,,,
05/07/2009, 23:02:08, 124, 124000, 1014, 978.304,,,,,,,
05/07/2009, 23:02:09, 127, 127000, 1015, 1000.985,,,,,,,
05/07/2009, 23:02:10, 125, 125000, 1014, 986.193,,,,,,,
05/07/2009, 23:02:11, 126, 126000, 1014, 994.083,,,,,,,
05/07/2009, 23:02:12, 125, 125000, 1014, 986.193,,,,,,,
05/07/2009, 23:02:14, 124, 124000, 1014, 978.304,,,,,,,
05/07/2009, 23:02:15, 126, 126000, 1014, 994.083,,,,,,,
05/07/2009, 23:02:16, 127, 127000, 1014, 1001.972,,,,,,,
05/07/2009, 23:02:17, 125, 125000, 1014, 986.193,,,,,,,
05/07/2009, 23:02:18, 124, 124000, 1014, 978.304,,,,,,,
ERROR: 64 in GetQueuedCompletionStatus: The specified network name is no longer 
available.
SenderThread is exiting...
SenderThread exited successfully.
Stopping Source traffic, waiting 5 sec.
SenderThread exited successfully.
WSACleanup successful.

Source 2

C:\qos>qostraffic.exe -source -tcp -dest 192.168.1.100 -throttle 2000000 -tc 40,4
Parsing command line...
WSAStartup successful.
Send interval: not used, packets will be sent as fast as possible
Packets to send per interval: 1
Size of each packet: 1000
Statistics sampling interval (msec): 1000
Sending traffic at 2.00 Mbps to 192.168.1.100:9999
Date, Time, Packets received, Bytes received, Elapsed time (msec), Throughput (Kbps), Bottleneck BW, Available BW, RTT, BBSet, ABSet, RTTSet, ErrorCode
05/07/2009, 23:01:59, 498, 498000, 1014, 3928.994,,,,,,,
05/07/2009, 23:02:00, 248, 248000, 1014, 1956.607,,,,,,,
05/07/2009, 23:02:01, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:02, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:03, 252, 252000, 1014, 1988.166,,,,,,,
05/07/2009, 23:02:04, 247, 247000, 1014, 1948.718,,,,,,,
05/07/2009, 23:02:05, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:07, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:08, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:09, 247, 247000, 1014, 1948.718,,,,,,,
05/07/2009, 23:02:10, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:11, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:12, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:13, 252, 252000, 1014, 1988.166,,,,,,,
05/07/2009, 23:02:14, 250, 250000, 1014, 1972.387,,,,,,,
05/07/2009, 23:02:15, 252, 252000, 1014, 1988.166,,,,,,,
05/07/2009, 23:02:16, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:17, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:18, 247, 247000, 1014, 1948.718,,,,,,,
ERROR: 64 in GetQueuedCompletionStatus: The specified network name is no longer 
available.
SenderThread is exiting...
SenderThread exited successfully.
Stopping Source traffic, waiting 5 sec.
SenderThread exited successfully.
WSACleanup successful.

Sink

C:\qos>qostraffic.exe -sink -tcp
Parsing command line...
Listening on all IP addresses.
WSAStartup successful.
Waiting for TCP traffic on port 9999, hit CTRL+C to exit...
AcceptEx has successfully completed.
Overlapped AcceptEx signaled.
Connection received: ::ffff:157.59.28.243:56677 --> ::ffff:192.168.1.100:9999
Date, Time, Packets received, Bytes received, Elapsed time (msec), Throughput (Kbps), Message
05/07/2009, 23:01:59, 498, 498000, 1014, 3928.994,
05/07/2009, 23:02:00, 248, 248000, 1014, 1956.607,
05/07/2009, 23:02:01, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:02, 251, 251000, 1014, 1980.276,
Overlapped AcceptEx signaled.
Connection received: ::ffff:157.59.28.243:56678 --> ::ffff:192.168.1.100:9999
Date, Time, Packets received, Bytes received, Elapsed time (msec), Throughput (Kbps), Message
AcceptEx has successfully completed.
05/07/2009, 23:02:03, 252, 252000, 1014, 1988.166,
05/07/2009, 23:02:04, 129, 125000, 1014, 986.193,
05/07/2009, 23:02:04, 247, 247000, 1014, 1948.718,
05/07/2009, 23:02:05, 126, 126000, 1014, 994.083,
05/07/2009, 23:02:05, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:06, 125, 125000, 1014, 986.193,
05/07/2009, 23:02:07, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:07, 126, 126000, 1014, 994.083,
05/07/2009, 23:02:08, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:08, 124, 124000, 1014, 978.304,
05/07/2009, 23:02:09, 247, 247000, 1014, 1948.718,
05/07/2009, 23:02:09, 127, 127000, 1015, 1000.985,
05/07/2009, 23:02:10, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:10, 125, 125000, 1014, 986.193,
05/07/2009, 23:02:11, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:11, 126, 126000, 1014, 994.083,
05/07/2009, 23:02:12, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:12, 125, 125000, 1014, 986.193,
05/07/2009, 23:02:13, 252, 252000, 1014, 1988.166,
05/07/2009, 23:02:14, 124, 124000, 1014, 978.304,
05/07/2009, 23:02:14, 250, 250000, 1014, 1972.387,
05/07/2009, 23:02:15, 126, 126000, 1014, 994.083,
05/07/2009, 23:02:15, 252, 252000, 1014, 1988.166,
05/07/2009, 23:02:16, 127, 127000, 1014, 1001.972,
05/07/2009, 23:02:16, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:17, 125, 125000, 1014, 986.193,
05/07/2009, 23:02:17, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:18, 124, 124000, 1014, 978.304,
05/07/2009, 23:02:18, 247, 247000, 1014, 1948.718,
Event set, thread exiting.
Receive thread exiting.
ReceiverThread exited successfully.
Control Event or Stop() called. TCP receive thread is exiting.
Main receive thread exited successfully.
WSACleanup successful.

The sink prints out statistics from both sources and was stopped using CTRL+C after about 20 seconds. Both traffic sources then exit gracefully. A full list of options is shown below.

C:\qos>qostraffic.exe -?
Parsing command line...
Need to specify -source or -sink flag.

QoS Traffic Generator (Daytona), v. 1.0

Usage:

Sink: qostraffic -sink -tcp/-udp/-tcp_a 
                 [-dest destIP,destPort -local localIP,localPort] ...
Source: qostraffic -source -tcp/-udp/-tcp_a -dest destIP,destPort 
                   -throttle rate
                   [-local localIP,localPort -duration sec] ...

Sink flags:
-tcp/-udp       Listens for TCP/UDP traffic
                default: listens on all addresses, port 9999
-tcp_a          Connects to source and then begins receiving traffic
                Source must also specify -tcp_a flag
                Note: start the source first, then the sink
-dest           OPTIONAL
                Destination address to connect to for -tcp_a
                Ignored for -tcp/-udp cases
-local          OPTIONAL
                Listens on specific local address/port
                default port: 9999
                Note: Can specify different port to listen on with,
                      localPort only.
-qwave          OPTIONAL
                Starts up qWave service.
                Used for adaptive flows from the source.
-noconsole      OPTIONAL
                Suppresses console output
                default is not suppressed
Source flags:
-tcp/-udp       Listens for TCP/UDP traffic
-tcp_a          Accepts connection from sink then sends traffic
                Sink must also specify -tcp_a flag
                Note: start the source first, then the sink
-dest           Send traffic to destination address/port
                default port: 9999
                Note: ignored for -tcp_a case
-throttle       Throttles traffic at specified rate (bits/sec)
-pull           OPTIONAL
                Operates the source in pull mode.
                Note: supported for TCP traffic only.
-local          OPTIONAL
                Send traffic from specified local address/port
                default port: 9999
                Note: for -tcp_a case, this is the local address to 
                      listen on
-duration       OPTIONAL
                Send traffic for specified time (sec)
                default: infinite
-stats          OPTIONAL
                Prints out statistics for specified time (msec)
                default: 1000 msec
                minimum: 200 msec
-qwave          OPTIONAL
                Uses qWave with the specified options (all optional)
                ,, trafficType: traffic type defined by QoS subsystem BE - best effort (default) BCK - background EE - excellent effort AV - audio/video VO - voice CTRL - control adaptive: force use adaptive flow default is nonadaptive lltd: force use LLTD packets default lets system choose LLTD or not flowfund: prints out QueryFlowFundamentals prints bandwidth estimations every 25 ms only supported when using adaptive flows EXAMPLE: -qwave AV,LLTD -server Optional, when used with qWave will prevent doing 
 QOSStartTracking/QOSStopTracking Note: conflicts with -tc flag. -tc OPTIONAL Uses TC with the specified options (all optional) ,<1p> dscp: DSCP value ranging from 0 to 63 default lets system choose 1p: 802.1p tag ranging from 0 to 7 default lets system choose Specifying -1 for either value is equivalent to 
 default. EXAMPLE: -tc 40,4 Note: conflicts with -qwave flag. Must be run as 
 admin. -packet OPTIONAL Performs packet experiments with given options (all 
 required) ,, size - size of each packet in bytes num - number of packets to send interval - time between each send in ms EXAMPLE: -packet 59,50,100 This will send a 59 byte packet every 100 ms, 50 
 times total. Note: packets can be tagged by using -tc options. -log OPTIONAL Logs to the specified file default filename: is "daytona_.txt" -noconsole OPTIONAL Suppresses console output default is not suppressed -localstats OPTIONAL Prints statistics from source default is sink side stats Note: conflicts with packet option, results will be 
 undefined -tolerance OPTIONAL Specify tolerance If no tolerance will assume 2%

Hopefully this post has been helpful in understanding what the QoS Traffic Generator can do. Please send us any comments you have!

-- Jim Liu

WWSAPI security examples setup

Ari B. Pernick — Wed, 06 May 2009 20:44:50 GMT

Just a quick note that over on on haoxu’ s “Lost in History” blog is a posting about how to create and register test SSL certs in http.sys for the Windows Web Services API Security Sample code. This might also be useful for developers using the http.sys apis.

QoS Traffic Generator is now available on Connect

Ari B. Pernick — Fri, 01 May 2009 02:35:00 GMT

Update: This tool is no longer available on Microsoft Connect and there is no replacement download site.

I’m posting on behalf of Jim Liu who is a SDET in Windows Core Networking. -- Ari

The QoS team is pleased to announce the availability of the QoS Traffic Generator on Microsoft Connect. This tool was developed during the Windows 7 development process and has been used extensively to aid in the testing of our QoS technologies. It is a command-line based tool that generates network traffic and utilizes TC/qWave to throttle and mark packets as desired. Some notable features include:

· Ability to send traffic at a specified throughput using Winsock from one machine to another

· Throttling is performed by the application itself or through QoS APIs (Traffic Control or qWave)

· Support TCP and UDP dual-mode sockets

· Provide traffic statistics from either the source machine or destination machine

· Can be built as a static library for use in other applications

We are providing this tool as a demonstration of QoS technologies and how they may be used to produce a novel application. Both 32 and 64-bit binaries are provided along with the source code and instructions for building the static library and the accompanying executable. The application itself has usage information on the available usage options.

To download the package, go to the WNDP site on Microsoft Connect and select Downloads from the left-hand menu. Select QoS Traffic Generator. Please send us any feedback you have about the tool!

-- Jim Liu

Beta versions of Windows Web Services API available

Ari B. Pernick — Tue, 24 Feb 2009 20:40:16 GMT

Windows Web Services API team has just release the Beta version of the API for Windows XP, Windows Vista, Windows Server 2003 and Windows Server 2008. It is available now on the Connect site for Windows Networking. This release contains the same version of the runtime as in Windows 7 Beta. You may find the table that summarizes level of Service Packs and CPU architectures supported by this release and direct links to download pages in this announcement. Please try it out and report any installation and runtime issues by submitting the feedback using the WNDP Connect site. The reference documentation for the API is available from MSDN. Please use this forum to ask WWSAPI team questions about the API and just released installers.

QoS in Windows 7

charwen — Wed, 28 Jan 2009 03:40:00 GMT

Hello all. My name is Charley. I’m the new QoS program manager for Windows Core Networking.

It has been a while since we posted our last article about QoS. We want to assure you that we’re still committed to improving this technology and building new QoS features in Windows. We received many questions and suggestions from you in the past. We hope you continue doing so because they are important to us.

You probably all know that Windows 7 beta is live and available for you to download. You may wonder what is new in Windows 7 regarding QoS. But before getting to that, I’d like to refresh our memory as to what was new in Windows Vista. In Vista, we introduced two major QoS features: qWAVE and Policy based QoS. qWAVE, or Quality Windows Audio Video Experience, is designed to estimate the network bandwidth, intelligently mark the application packets (with proper DSCP values), and interact with the application in the event of network congestion or fluctuations of available bandwidth (so that the application can take appropriate actions). qWAVE APIs aim to simplify the work of application developers developing QoS-enabled applications for a home network. The APIs are documented in the Windows SDK. In contrast, Policy based QoS has a different target audience. It is intended to enable IT administrators to apply QoS to applications (which don’t need to have native supports of QoS), computers, and users in their enterprise network. It is especially beneficial to an organization with branch offices, where the WAN link capacity is limited and users tend to experience unpredictable network delays when accessing files or applications hosted on the main campus (or a different branch office). If you’d like to know more about these features, we’d suggest you read an excellent article written by Joseph Davies for TechNet magazine. Click here to check it out.

So what’s new in Windows 7? The enhancement we’ve made is called URL based QoS. If you’re familiar with Policy based QoS you know that in Vista an IT administrator can create a policy based on the application name, source and/or destination IP addresses, source and/or destination ports, and the protocol (TCP, UDP, or both). We’ve learned since then that many enterprise applications have been, or will be, hosted on web servers and accessed from a browser, so the IT administrators would love to be able to prioritize or control the network traffic from those web-based applications, provided that a convenient configuration is available. To answer their request, we’ve added a new configuration option: you can create a policy based on the URL of an HTTP server. Say, you host all the training videos of your company on an IIS server running Windows 7 (or more precisely, Windows Server 2008 R2). You can then use a URL based QoS policy to throttle the video traffic (to prevent it from overwhelming your corporate network). Or, you host your company’s CRM or ERP applications on an HTTP application server running Windows 7. You can similarly use a URL based QoS policy to prioritize these applications traffic so that even users at a remote branch office always get prompt replies and have smooth UI experience.

Does this sound interesting to you? Please let us know what you think. In the next blog, we’ll detail the configuration and the rules.

Charley

Come See Windows 7 DirectAccess and BranchCache @PDC08 Pavillion

wndpteam — Wed, 29 Oct 2008 21:23:00 GMT

We are demoing Windows 7 Direct Access and Windows 7 Branch Cache at the PDC08 pavillion. We are the second booth to the left when you come in to the Big Room, shaing space with the Windows 7 Sensor folks. Come check it out at talk to us.

-- Ari

Heading to PDC 2008

wndpteam — Wed, 08 Oct 2008 01:44:45 GMT

I’ve been pretty busy Engineering Windows 7 with the rest of the windows networking team and we are really excited to start showing the world some of the work we’ve done at PDC. One of the things we’ve been working on has it’s own session: “Windows 7: Web Services in Native Code” presented by Nikola Dudar. Also those who attend the PDC get Windows 7 bits.

It turns out I’ll be at PDC as well, so let me know if you are heading there and wish to meet up. If you haven’t already registered, act quickly because registration is quickly getting full.

-- Ari Pernick

WinHTTP Questions: About Callbacks

wndpteam — Mon, 03 Mar 2008 20:01:00 GMT

Hello All, I am continuing the "WinHTTP Questions" series with some questions on WinHTTP callbacks.

Is it correct that WinHTTP Callbacks will occur *only* during an in-progress WinHTTP operation? Is it possible that an external event (such as the remote server resetting the underlying tcp connection) result in a callback even when there's no outstanding operation?

No. For external callbacks, WinHTTP will indicate informational callbacks only when there is a pending operation. Essentially, there are two types of callbacks in WinHTTP. Completion callbacks and Informational callbacks.

Completion callbacks are invoked as a completion of a pending API call e.g. read complete is invoked when the read API completes. Similarly, handle-closed callback is invoked in response to the close handle API.
Informational callbacks, on the other hand, are invoked to inform the application about important events seen during the processing of a call.

So, when an async API is invoked and it returns TRUE, the application will receive zero or more informational callbacks and then a completion callback.

When an operation like WinHttpReadData fails immediately (i.e., this call returned FALSE), does it guarantee that user will not receive a STATUS_REQUEST_ERROR callback?

Yes.

Does WinHTTP synchronize WinhttpSetStatusCallback with worker threads? Here is our usage scenario: We are implementing a DLL that get dynamically loaded implements an asynchronous Download function with progress notifications and supports a Cancel call. After Cancel or Download completion our DLL should be able to be safely unloaded. Cancel calls WinHttpSetStatusCallback to unregister notifications, closes the request, connection and sessions handles. However we occasionally still get status callbacks after the DLL is unloaded.

No, WinHTTP does not synchronize WinHttpSetStatusCallback with worker threads. For e.g. If a callback originating in another thread is in progress when an application calls WinHttpSetStatusCallback, the application still receives a callback notification even after WinHttpSetStatusCallback successfully sets the callback function to NULL and returns. The thread will continue to process, so one will need to wait until the current call finishes.

Please also refer to the following links for more on callbacks.

Next time we will go a bit deeper into safe request cancellation.

-- Deepak & Ari

WinHTTP Questions - async close on a sync request

wndpteam — Mon, 25 Feb 2008 18:30:00 GMT

Hello, my name is Deepak and I'm a SDET in serviceability. We handle a bunch of questions from developers using WinHTTP, and thought we might share then in a new posting series, "WinHTTP Questions".

Can I cancel a synchronous WinHttpSendRequest call by closing the request handle from a different thread? Or, are there any requirements that I need to use the asynchronous WinHttpSendRequest if I need the ability to cancel it?

The short answer is don't do that.

Let's think about the implications of this proposed code. Thread 1 starts a synchronous request and Thread 2 comes along an issues a close. We can't synchronize between Thread 1 and Thread 2 by definition because Thread 1's call is synchronous, we either can deterministically call the close before the request call starts or after the call is completed. So to close the handle during the call results in a race condition. If thread 2 attempts to close the connection during the call, the handle passed in to WinHttpSendRequest may become invalid before WinHttp has a chance to work with it. While you might get lucky, you might also non-deterministically get a crash or memory corruption.

-- Deepak and Ari

Ready for Windows Server 2008?

wndpteam — Tue, 08 Jan 2008 02:48:46 GMT

Windows Server 2008 incorporates the completely rewritten TCP/IP stack we shipped in Windows Vista and since this is the first server release with the stack we'd like to ask our readers to load up Windows Server RC1 and see if your applications are ready.

The new features (compared to Windows Server 2003) that may pose potential compatibility issues include: removed filter hooks, receive window auto tuning, dual IP layer architecture for IPv6, compound TCP, ECN support, default strong host model, easier kernel mode programming, extensive protocol offload.

Windows Server 2008 is also the very first Microsoft server product to have the firewall ON by default. Third party applications that are not aware of this behavior may break.

-- Ari and Katarzyna

Happy New Year

wndpteam — Mon, 07 Jan 2008 20:34:00 GMT

Just wanted to drop a quick Happy New Year to our readers and see if there is any topics that you want to hear about. We've been pretty busy with Windows Server 2008, Windows Vista SP1 and future windows releases. With the RTM of Windows Server 2008 and Windows Vista SP1 drawing near, I hope to get some posts on things that are new or have changed, but I'd be happy to hear whatever people are interested in.

-- Ari

Throttling, DSCP, and 802.1p with Traffic Control

wndpteam — Thu, 15 Nov 2007 02:07:39 GMT

In his introductory post about the legacy Traffic Control (TC) API, Gabe discussed the host-based model that TC provides. In this post, we will see how Traffic Control APIs can be used to achieve the following for TCP/IPv4 and UDP/IPv4 traffic sent from a host:

Throttle (rate-limit) outgoing traffic
Add DSCP value in layer-3 (IPv4) header
Indicate that an 802.1p tag value should be added in layer-2 header

At the bottom of the post, we’ve provided a link to the Networking Connect site to download full source and binaries to a tool which implements all of the above functionality.

The following are the steps involved:

1. The first step in the process is to obtain a handle to the Traffic Control subsystem through a call to TcRegisterClient().

2. Next, make a call to TcEnumerateInterfaces() using the registration handle obtained in step #1. This call returns a list of all TC enabled interfaces on the system. Iterate through the list to find the interface(s) on which you want to prioritize and/or throttle outgoing traffic.

3. For each interface of interest, issue a TcOpenInterface() using the pInterfaceName from the corresponding TC_IFC_DESCRIPTOR for that interface from the list returned in step#2. Store the handle returned by TcOpenInterface(); let’s call it the ifchandle.

4. At this point, create a TC Flow and add it to the interface(s) of interest.

a. Create the TC Flow:
A TC flow is a way of describing various QoS characteristics to be applied to a set of packets and is represented by a TC_GEN_FLOW structure. The following code snippet shows how to create a TC Flow given the DSCP value, 802.1p value and the throttle rate.

BOOL CreateFlow(PTC_GEN_FLOW * _ppTcFlowObj, USHORT DSCPValue, USHORT OnePValue, ULONG ThrottleRate)
{
BOOL status = FALSE;
//
// Flow Parameters
//
ULONG TokenRate = QOS_NOT_SPECIFIED;
ULONG TokenBucketSize = QOS_NOT_SPECIFIED;
ULONG PeakBandwidth = QOS_NOT_SPECIFIED;
ULONG Latency = QOS_NOT_SPECIFIED;
ULONG DelayVariation = QOS_NOT_SPECIFIED;
SERVICETYPE ServiceType = SERVICETYPE_BESTEFFORT;
ULONG MaxSduSize=QOS_NOT_SPECIFIED;
ULONG MinimumPolicedSize=QOS_NOT_SPECIFIED;
PVOID pCurrentObject;
PTC_GEN_FLOW _pTcFlowObj = NULL;
int Length = 0;
//
// Calculate the memory size required for the optional TC objects
//
Length += (OnePValue == NOT_SPECIFIED ? 0:sizeof(QOS_TRAFFIC_CLASS)) + (DSCPValue == NOT_SPECIFIED ? 0:sizeof(QOS_DS_CLASS));
//
// Print the Flow parameters
//
printf("Flow Parameters:\n");
DSCPValue == NOT_SPECIFIED ? printf("\tDSCP: *\n"):printf("\tDSCP: %u\n", DSCPValue);
OnePValue == NOT_SPECIFIED ? printf("\t802.1p: *\n"):printf("\t802.1p: %u\n", OnePValue);
ThrottleRate == QOS_NOT_SPECIFIED ? printf("\tThrottleRate: *\n"):printf("\tThrottleRate: %u\n", ThrottleRate);
TokenRate = TokenBucketSize = ThrottleRate;
//
// Allocate the flow descriptor
//
_pTcFlowObj = (PTC_GEN_FLOW)malloc(FIELD_OFFSET(TC_GEN_FLOW, TcObjects) + Length);
if (!_pTcFlowObj)
{
    printf("Flow Allocation Failed\n");
    goto Exit;
}
_pTcFlowObj->SendingFlowspec.TokenRate = TokenRate;
_pTcFlowObj->SendingFlowspec.TokenBucketSize = TokenBucketSize;
_pTcFlowObj->SendingFlowspec.PeakBandwidth = PeakBandwidth;
_pTcFlowObj->SendingFlowspec.Latency = Latency;
_pTcFlowObj->SendingFlowspec.DelayVariation = DelayVariation;
_pTcFlowObj->SendingFlowspec.ServiceType = ServiceType;
_pTcFlowObj->SendingFlowspec.MaxSduSize = MaxSduSize;
_pTcFlowObj->SendingFlowspec.MinimumPolicedSize = MinimumPolicedSize;
//
// Currently TC only supports QoS on the send path
// ReceivingFlowSpec is legacy and ignored
//
memcpy(&(_pTcFlowObj->ReceivingFlowspec), &(_pTcFlowObj->SendingFlowspec), sizeof(_pTcFlowObj->ReceivingFlowspec));
_pTcFlowObj->TcObjectsLength = Length;
//
// Add any requested objects
//
pCurrentObject = (PVOID)_pTcFlowObj->TcObjects;
if(OnePValue != NOT_SPECIFIED)
{
    QOS_TRAFFIC_CLASS *pTClassObject = (QOS_TRAFFIC_CLASS*)pCurrentObject;
    pTClassObject->ObjectHdr.ObjectType = QOS_OBJECT_TRAFFIC_CLASS;
    pTClassObject->ObjectHdr.ObjectLength = sizeof(QOS_TRAFFIC_CLASS);
    pTClassObject->TrafficClass = OnePValue; //802.1p tag to be used
    pCurrentObject = (PVOID)(pTClassObject + 1);
}
if(DSCPValue != NOT_SPECIFIED)
{
    QOS_DS_CLASS *pDSClassObject = (QOS_DS_CLASS*)pCurrentObject;
    pDSClassObject->ObjectHdr.ObjectType = QOS_OBJECT_DS_CLASS;
    pDSClassObject->ObjectHdr.ObjectLength = sizeof(QOS_DS_CLASS);
    pDSClassObject->DSField = DSCPValue; //Services Type
}
DeleteFlow(_ppTcFlowObj);
*_ppTcFlowObj = _pTcFlowObj;
status = TRUE;
Exit:
if(!status)
{
    printf("Flow Creation Failed\n");
    DeleteFlow(&_pTcFlowObj);
}
else
    printf("Flow Creation Succeeded\n");
return status;
}

b. Add the Flow on the interface:
After obtaining a TC_GEN_FLOW structure with the desired characteristics using a function similar to the one above, issue a call to TcAddFlow() with the ifchandle (obtained in step #3) and a pointer to the TC_GEN_FLOW object (obtained in step #4a). Store the handle returned by TcAddFlow();let’s call it the flowhandle.

5. The next step is to create a TC Filter and add it to the TC Flow created above.

a. Create the TC Filter:
A TC Filter is a way of describing which packets to apply the QoS characteristics to. The QoS characteristics defined in the TC_GEN_FLOW will only apply to the packets matching the filter(s) associated with the Flow.
The following code snippet describes how to create a TC Filter given the destination address, the destination port, and the protocol (TCP,UDP or IP).

BOOL CreateFilter(PTC_GEN_FILTER * ppFilter, SOCKADDR_STORAGE Address, USHORT Port, UCHAR ProtocolId)
{

BOOL status = FALSE;
USHORT AddressFamily = Address.ss_family;
PTC_GEN_FILTER pFilter = NULL;
PIP_PATTERN pPattern = NULL;
PIP_PATTERN pMask = NULL;
if(AddressFamily != AF_INET)
    goto Exit;
//
// Allocate memory for the filter
//
pFilter = (PTC_GEN_FILTER)malloc(sizeof (TC_GEN_FILTER));
if(!pFilter)
    goto Exit;
ZeroMemory(pFilter, sizeof(TC_GEN_FILTER));
//
// Allocate memory for the pattern and mask
//
pPattern = (PIP_PATTERN)malloc( sizeof(IP_PATTERN));
pMask = (PIP_PATTERN)malloc( sizeof(IP_PATTERN));
if(!pPattern || !pMask)
    goto Exit;
memset ( pPattern, 0, sizeof(IP_PATTERN) );
pPattern->DstAddr = ((SOCKADDR_IN *)&Address)->sin_addr.s_addr;
pPattern->tcDstPort = htons(Port);
pPattern->ProtocolId = ProtocolId;
memset ( pMask, (ULONG) -1, sizeof(IP_PATTERN) );
//
// Set the source address and port to wildcard
// 0 -> wildcard, 0xFF-> exact match
//
pMask->SrcAddr = 0;
pMask->tcSrcPort = 0;
//
// if the user specified 0 for dest port, dest address or protocol
// set the appropriate mask as wildcard
// 0 -> wildcard, 0xFF-> exact match
//
if(pPattern->tcDstPort == 0)
    pMask->tcDstPort = 0;
if(pPattern->ProtocolId == 0)
    pMask->ProtocolId = 0;
if(pPattern->DstAddr == 0)
    pMask->DstAddr = 0;
pFilter->AddressType = NDIS_PROTOCOL_ID_TCP_IP;
pFilter->PatternSize = sizeof(IP_PATTERN);
pFilter->Pattern = pPattern;
pFilter->Mask = pMask;
//
// Delete any previous instances of the Filter
//
DeleteFilter(ppFilter);
*ppFilter = pFilter;
status = TRUE;
Exit:
if(!status)
{
    printf("Filter Creation Failed\n");
    DeleteFilter(&pFilter);
}
else
    printf("Filter Creation Succeeded\n");
return status;
}

b. Adding the Filter to the TC Flow:
Once a TC Filter structure is obtained using a function similar to the one above, issue a call to TcAddFilter() passing the flowhandle obtained in step #4b and a pointer to the TC_GEN_FILTER structure obtained in step #5a. Store the filter handle returned by TcAddFilter(); let’s call it filterhandle.
You can add multiple filters on the same flow causing different sets of packets matching each filter to get the same QoS characteristics applied to them.

6. At this point, your application is applying QoS on all matching outgoing packets as specified in the TC Filter and TC Flow. Finally, once your purpose is served, make sure you call the respective close calls on all the open TC handles – TcDeletefilter(), TcDeleteFlow(), TcCloseInterface() and TcDeregisterClient().

You can download the full source and binaries of a simple command line tool – tcmonlite, which takes the filter and flow parameters as input, creates TC Flow and Filter, and configures the QoS subsystem with them using the Traffic Control API. All the outgoing traffic on the system matching the filter gets the desired QoS characteristics as long as tcmonlite is running. Go to the Microsoft Connect website, choose Available Connections on the left-hand side of the page, and select Windows Networking from the available connections (bottom half of the page). On the left-hand side of the Windows Networking page, choose Downloads, and select TCMonLite.

This tool can be used in conjunction with the NDIS LWF driver to detect 802.1p tags in the Ethernet header and DSCP in the IP header of packets. Let us know what you think!

-- Hemant Banavar

Introduction to Windows QoS Traffic Control

wndpteam — Tue, 09 Oct 2007 21:05:26 GMT

Disclaimer: Traffic Control (TC) APIs have been marked as deprecated, and will be phased out (eventually removed) when a suitable replacement API is available. No advancements will be made to these APIs (including adding IPv6 support) in their deprecated state; however, application compatibility will be maintained until their eventual removal.

Since the introduction of a QoS platform in Windows 2000, there have been two models for applying prioritization and/or send-rate throttling to TCP/IP and UDP/IP network traffic sent from a Windows PC: host-based and application-based. These terms have been used in QoS documentation such as this recent CableGuy article; however, I acknowledge the meaning of these terms are not immediately obvious.

An application-based model means only the application which owns the socket handle can add/remove/modify a QoS flow for its traffic. Because the application sending data onto the wire (or air) is applying throttling or priority to its own traffic (the connected socket), no elevation of privileges is required. A host-based model means some other process (not the application sending traffic through the socket) on the PC is applying prioritization or throttling to this traffic it doesn't own. Because the process doesn't own the socket handle, elevation of privileges (administrator) is required. While it is certainly possible for the process that owns the socket handle to leverage a host-based model, the added complexity is unnecessary considering socket-based QoS APIs are available for this purpose. A host-based model is a much more complex model than application-based for the following reasons:

The process applying QoS properties to traffic it doesn't own has to run as a service or some other out-of-band means (in typical use cases)
Administrative privileges are required
Because the socket handle is not known, a filter has to be applied to match the traffic of interest, based on: source/destination IPv4 address, source/destination port, and protocol (TCP or UDP)

In Windows, only the Traffic Control (traffic.h/traffic.dll) interface provides programmatic access to a host-based model. There is value gained from this complexity; however. Because this API requires administrative privileges, the caller can specify any arbitrary layer-2 (802.1p) or layer-3 (DSCP) priority value; whereas application-based API models abstract specific priority values with traffic-classes based on established industry standards. It is worth noting that a new policy-based feature has been added to Windows Vista and Windows Server 2008 which enables a host-based model for IT administrators (no coding necessary), which enables significantly richer classification than what TC provides. Policy-based QoS; however, does not provide programmatic access and does not allow for setting layer-2 802.1p tags, only layer-3 DSCP.

Innovation has been focused on application-based APIs such as qWAVE (qos2.h/qwave.dll) to significantly simplify *safely* adding prioritization and throttling to traffic, as well as policy-based mechanisms for host-based needs.

Stay tuned for follow-up posts on how to use TC for adding 802.1p tags to the Ethernet header, DSCP to the IPv4 header, and applying throttling to outgoing traffic.

-- Gabe Frost

WiFi WMM Requirements for Vista Miniport Drivers

wndpteam — Tue, 18 Sep 2007 18:00:31 GMT

A number of partners who author wireless drivers for Vista have asked how they can ensure their WiFi Wireless Multimedia (WMM) implementation is correct, so I thought I'd be explicit about this very important topic. To begin, read the 4-part series WiFi QoS Support in Windows Vista, which describes how Vista internally indicates a WMM Access Category (WMM_AC), how to detect whether an Access Point supports this capability, and how to observe the behavior of prioritized traffic. Next, be sure you explicitly validate your driver does the following:

Confirm the QoS header exists in 802.11 data frames when DSCP and 802.1p are set independently:

DSCP [56, 48, 40, 32, 24, 16, 8, 0]
802.1p [7, 6, 5, 4, 3, 2, 1, 0]

Ensure the miniport indicates NDIS_MAC_OPTION_8021P_PRIORITY in OID_GEN_MAC_OPTIONS
Miniport behavior on receive:

If WMM/11e header is stripped, WMM bit in FrameControlSubtype must be cleared
If WMM/11e header is not stripped, WMM bit in FrameControlSubtype must *not* be cleared
Must *not* strip 802.1Q tag in SNAP header (nwifi.sys will strip if necessary

Miniport behavior on transmit:

Only use NDIS_NET_BUFFER_LIST_8021Q_INFO.WMMInfo field to ascertain correct WMM_AC (*not* UserPriority, or IP DSCP field)
Must not strip 802.1Q tag in SNAP header (may be added by nwifi.sys)

-- Gabe Frost

Detecting 802.1p Priority Tags: Part 3

wndpteam — Sat, 15 Sep 2007 01:39:23 GMT

Parts 1 and 2 of this series discussed how to determine whether an 802.1p tag was added to traffic, and how to modify the NDIS light-weight-filter (LWF) sample driver source code to accomplish this task. We do know that you're all very busy and not everyone is a developer, so we've added to the package: full source code for the complete filter driver, a command-line tool for accessing the filtered packets, and compiled binaries for you non-developers. We also added the ability to validate *both* 802.1p and DSCP (from the IPv4/IPv6 header). This additional package was added to the existing download, so if you haven't already read part-2 of this series, do so now and you'll find download instructions there. For installation instructions, read the README file in the zip archive.

What do you think? Is this approach (source and tools) helpful? We do actively monitor the QoS forum, so let us know how we can improve your understanding of Windows QoS capabilities.

-- Gabe Frost

-- Huge thanks to Hemant Banavar who authored the tools

Detecting 802.1p Priority Tags: Part 2

wndpteam — Fri, 14 Sep 2007 02:43:33 GMT

In Gabe’s last post on detecting 802.1p priority tags, he described at a relatively high-level why it is difficult to detect a priority tag using packet tracing applications, as well as the proper way to determine whether a tag was present in a packet that was sent onto the wire (or air). In this post, I’ll describe how to programmatically access this information by modifying the NDIS Light Weight Filter (LWF) sample driver found in the Windows Driver Kit (WDK). To begin, download and install the WDK and navigate to the LWF sample found in the WinDDK\6000\src\network\ndis\filter directory. The remainder of this post will describe how to modify this sample to gain access to the UserPriority value in the out-of-band (OOB) data of a Net Buffer List (NBL) structure, and whether the miniport driver actually stripped the 1Q tag from the Ethernet header like it was supposed to. Based on what part-1 of this series describes about driver (miniport/LWF) layering and framing details, the modified driver will inspect received packets (meaning the sender added the tag to outgoing traffic).

The source file in the LWF sample of particular interest is filter.c, where we’ll modify the FilterReceiveNetBufferLists() function. FilterReceiveNetBufferLists() is an optional function for filter drivers, which if provided, processes receive indications made by the underlying miniport or filter drivers beneath in the stack. If this handler is NULL, NDIS will skip calling this filter when processing a receive indication and will call the next filter (or protocol driver) above in the stack with a non-NULL FilterReceiveNetBufferLists handler. The remainder of this post goes into detail about which areas of filter.c need to be modified.

To begin, let’s explore how to access the received packets in this function so they can be inspected. The second parameter to the function, NetBufferLists, is a linked list of NetBufferList structures allocated by the underlying driver. Each NetBufferList contains one NetBuffer structure, which represents a received packet. This means, in order to inspect each packet in the NetBufferLists linked list, we need a loop of the following kind within FilterReceiveNetBufferLists():

if (pFilter->TrackReceives)
{
     FILTER_ACQUIRE_LOCK(&pFilter->Lock, DispatchLevel);
     pFilter->OutstandingRcvs += NumberOfNetBufferLists;
     Ref = pFilter->OutstandingRcvs;
     currNbl = NetBufferLists;
     while(currNbl)
     {
          // Call the function to parse the packet in each
          // NetBufferList (one net buffer per NBL )
          inspectNetBuffer(currNbl, pFilter);
          currNbl = currNbl->Next;
     }
     FILTER_LOG_RCV_REF(1, pFilter, NetBufferLists, Ref);
     FILTER_RELEASE_LOCK(&pFilter->Lock, DispatchLevel);
}

In the above code snippet, observe that the loop is run only if pFilter->TrackReceives is TRUE. The idea here is to only inspect packets if the user requests the driver to do receive-side inspections (our focus is on a debugging tool). The TrackReceives flag can be set to FALSE at FilterAttach and can be set to TRUE through an IOCTL. Look at the FilterDeviceIoControl() function in device.c for defining IOCTLs.

Now that we have a pointer to the NetBufferList structure which holds the packet information, let’s explore how to inspect the packet for the 802.1p tag (note you could also inspect the IPv4 or IPv6 header for DSCP here if you really wanted). The inspectNetBuffer function in the above code snippet starts by extracting the NetBuffer pointer from the NetBufferList. Please see the bottom of this post for instructions on where to download the full implementation of this function. Next, if an 802.1p tag is available in the OOB data, it is extracted and stored in the variable called UserPriority as follows:

if (NET_BUFFER_LIST_INFO
(pNetBufferList, Ieee8021QNetBufferListInfo) != 0)
{
Ndis8021QInfo.Value = NET_BUFFER_LIST_INFO(pNetBufferList, Ieee8021QNetBufferListInfo);
UserPriority = (UCHAR)Ndis8021QInfo.TagHeader.UserPriority;
}

At this point, a pointer to the start of the packet is obtained and stored in packetBuffer. The Ethernet header is parsed and RecdUnStrippedPackets (which is initialized to zero at the beginning before starting to track received packets) is incremented if it is found that the Ethernet header still contains the 802.1p tag - indicating the underlying miniport did not strip the tag as per NDIS documentation.

To download the full implementation of inspectNetBuffer(), as used to do the majority of parsing work, go to Microsoft Connect website and login using your passport account (create one if you don’t already have one). Once you have logged in, choose Available Connections on the left-hand side of the page, and select Windows Networking from the available connections (bottom half of the page). On the left-hand side of the Windows Networking page, choose Downloads, and select NDIS LWF Sample With Packet Priority Detection.

-- Hemant Banavar

Detecting 802.1p Priority Tags

wndpteam — Fri, 07 Sep 2007 17:00:00 GMT

Consider a case where a network application calls Windows QoS APIs to add a layer-2 IEEE 802.1Q UserPriority tag (almost always referred to as 802.1p) to outgoing traffic. Ascertaining whether the tag actually got added to an outgoing packet is not as simple as it seems due to the nature of how the Windows network stack is designed, and how framing actually occurs. From an internal implementation perspective, The QoS Packet Scheduler (Pacer.sys in Vista/2008 Server, and Psched.sys in XP/2003 Server) in the network stack merely updates an out-of-band structure (not the actual formed packet) that an 802.1Q UserPriority tag should be added. The specific NDIS structure is NDIS_NET_BUFFER_LIST_8021Q_INFO, which contains member variables for both VlanID and UserPriority, and is passed to the NDIS miniport driver for implementing both priority tagging (UserPriority) and VLAN (VlanId). It is up to the NDIS miniport driver to actually insert the 802.1Q tag into the frame based on these values before transmitting on the wire. A miniport driver will only insert this tag if the feature is supported and enabled in the advanced properties of the NIC driver; typically layer-2 priority tagging is disabled by default.

From a network stack layering perspective, it’s important to understand that Pacer.sys is an NDIS Lightweight Filter (LWF) driver, and will always be inserted above a miniport driver, which will always be the lowest network software in the stack because it communicates directly with the NIC hardware. Also note that network sniffing applications like NetMon and WireShark are also network stack filters, and will always be inserted above the miniport driver. This is important knowledge because it should be clear that taking a network sniff of traffic on the sending PC will never show the tag in a packet (because the tag gets added below the sniffing software). Also, the QoS Packet Scheduler can't know for absolute certainty whether the miniport driver added the tag to the outgoing packet.

What about trying to do a network sniff on the receiving PC? Good question, but also will show the layer-2 tag not present in packets. The reason for this is NDIS developer documentation clearly states that miniport drivers must strip the tag when received, and populate the NDIS_NET_BUFFER_LIST_8021Q_INFO UserPriority and VlanId fields with the values in the tag. This out-of-band structure can then be used by NDIS filter drivers higher up in the stack for implementing these features. The functional reason for stripping the layer-2 tag is because Tcpip.sys will drop any received packet that contains this tag. Therefore, if a misbehaving miniport driver does not strip the tag, the packet will never be received by the user-mode application because it will be dropped internally.

In conclusion:

A network sniffing app on the sending PC will never see a tag
A network sniffing app on the receiving PC will never see a tag

Unless the miniport driver is misbehaving, which will result in dropped packets

Monitoring tagged packets from intermediate network elements (such as a switch) is hard if at all possible

Perhaps a clever SNMP counter could be used, but would depend on the device manufacturer

If you have not came to this conclusion already, the only way to determine whether a layer-2 tag got added on the sending PC and/or received by the receiving PC is to monitor the NDIS_NET_BUFFER_LIST_8021Q_INFO.UserPriority field. Stay tuned for a follow-up post which describes how to do this.

-- Gabe Frost

The Case of Vista Multimedia Playback and Network Throughput

wndpteam — Mon, 27 Aug 2007 20:07:03 GMT

Mark Russinovich has a great post today on the what and how of the network/multimedia vista issue that people have recently been talking about. Amusingly enough a couple people on /. more or less figured it out, but are only modded 3 and lower. Go Figure.

-- Ari

Ask Perf explains how winInet is used

wndpteam — Tue, 21 Aug 2007 20:37:21 GMT

Ask Perf, the blog of the Enterprise Platforms Windows Server Performance Team, is spending some time explaining a bit of how WinInet/WinHTTP and their surrounding components work with each other. Go check it out!

-- Ari

Receive Window Auto-Tuning on Vista.

wndpteam — Fri, 06 Jul 2007 02:02:00 GMT

Hi, my name is Katarzyna and I am the Program Manager within the Internet Protocols team. I have been asked a few times about the Receive Window Auto-Tuning feature on Vista and some associated issues people are having.

One of the many cool new features on Windows Vista, Receive Window Auto-Tuning enables the networking stack to receive data more efficiently than on XP. Auto-Tuning allows the operating system to continually monitor the routing conditions (bandwidth, network delay, application delay) and configure connections (scale the TCP Receiving Window) so as to maximize the network performance.In some high bandwidth, high latency links, we have seen SMB performance improvement up to 20 times!

In every TCP packet there is a "window" field, which informs the receiver how much data the sender can accept back. This window controls the flow by setting a threshold on data kept "in flight" and prevents overwhelming the receiver with data that it cannot accept.

The TCP window field is 16 bits wide, allowing for a maximum window size of 64KB, which used to meet requirements of many older networks. Nowadays, however, network interfaces can handle larger packets and keep more of them in flight at any given time. Thus, a larger TCP window has become necessary; especially on high-speed, high latency networks. To fill such a long, fat pipe and make use of the available bandwidth, the sending system can often require very large windows for good performance.

The solution to this demand is called "window scaling”, described back in 1992 in RFC 1323. It introduces an eight-bit scale factor, which serves as a multiplication factor for the window width. After the factor has been negotiated, window values used by that system on a given connection will be shifted to the left by that scale factor; a window scale of zero, thus, implies no scaling at all, while a scale factor of six implies that window sizes should be shifted six bits, thus multiplied by 2^6 = 64. Now a window greater than 64KB can be easily expressed (e.g., 128KB) by setting the scale factor (e.g., 6) and keeping the window field under the original 16 bits (here, 2048).

The window size included in all packets is modified by the scale factor, which is negotiated once at the very beginning of a TCP connection. The connection requestor suggests window scaling factor in its original SYN packet and if the SYN+ACK packet sent in response contains the option, then this particular value will be used on this connection. The scale factor cannot be changed after the initial setup handshake; remaining data transfers on this connection will implicitly use the negotiated value.

Older routers and firewalls however do not handle window scaling correctly leaving the option in the original SYN packet but setting the connection’s scale factor to zero. Seeing the option on, the receiver responds with its own window scale factor. Believing that its scale factor has been accepted, the initiator scales the window appropriately while the receiver thinks that a scale factor of zero is applied and thus a small window of data should follow. As a result, the communication is slow at best. Sometimes, small window packets are dropped by the routers, essentially breaking the connection.

The resulting slow data transfers or loss of connectivity, users may experience as slow or hung networking applications. Remote Desktop Connection and network file copy are two scenarios particularly hurt by misbehaving routers.

If your connection from a Vista machine appears slow or hung, here are some steps to isolate the cause:

First, make sure that your firewall and router can support window scaling. Some devices from Linksys, Cisco, NetApp, SonicWall, Netgear, Checkpoint, D-Link were reported as having problems with window scaling. (Some of the incompatible devices are given here. You can check with the manufacturer or run the connectivity diagnostic suite (especially, TCP High Performance Test) provided by Microsoft to determine your gateway device’s compliance.
Second, check with the manufacturer if a firmware update has been issued for your device that can fix the problem. Replace the problematic device or update the firmware as suggested by the manufacturer. If the router cannot be replaced or if it the device is remote (e.g., a firewall of your ISP or corporation)
Third, If the problem still persists, you can restrict autotuning by running “netsh interface tcp set global autotuninglevel=restricted” from the command prompt. We have found that restricted mode will often allow some of the benefits of autotuning with a number of problematic devices.
Lastly, if all else fails, in order to disable this feature, run "netsh interface tcp set global autotuninglevel=disabled".
(In order to reenable autotuning, run “netsh interface tcp set global autotuninglevel=normal”.)

Please refer to the following KB articles for more information:

-- Katarzyna

Updated: Broken link to KB 932170
Update 2: Changed the guidance to do restricted before disabled.
Update 3: tunning doesn't have two "n"s. :)
Update 4: no really, tuning doesn't have two "n"s.

OffTopic: Another way to generate XML in PowerShell

wndpteam — Fri, 15 Jun 2007 20:19:27 GMT

I recently needed to generate XML from PowerShell and was disappointed to see the PowerShell blog use the old ASP model of doing text insertion into the middle of a big string. It might be the tester in me, or the security training, but with some unexpected input you can easily end up with malformed XML or even worse maliciously malformed XML. The other extreme would be to use .Net XML APIs to build up the XML doc from scratch, but I didn't like how much redundant code I would need to build the xml header and I didn't find any good sample code in C# to create an XML document from scratch, so I ended up developing a middle approach. Start with a string that has nothing but the xml header and an empty top level element, convert that over to an XMLDocument, use the APIs to add data and then write out the doc to the disk.

For example:

$doc = [xml] "<?xml version=""1.0"" encoding=""utf-8""?><ns:RoleInstance xmlns:ns=""http://namespace.microsoft.com/2007/Whatever""/>"

$elem = $doc.CreateElement("ns:TestBuild")
$elem.SetAttribute("Product", $Product);
$elem.SetAttribute("Lab", $Lab);
$elem.SetAttribute("BuildNumber", $OSBuildNumber);
$elem.SetAttribute("SPBuildNumber", $SPBuildNumber);
$elem.SetAttribute("TimeStamp", $BuildLabString.Split(".")[4]);
$elem.SetAttribute("SKU", $SKU);
$elem.SetAttribute("Language", $SystemLocale.Split("-")[0].Trim());
$elem.SetAttribute("Culture", $SystemLocale.Split("-")[1].Trim());
$elem.SetAttribute("Architecture", $Processor);
$elem.SetAttribute("Type", $Type);
$doc.get_ChildNodes().Item(1).AppendChild($elem) | out-null

$elem = $doc.CreateElement("ns:Implementation");
$elem.SetAttribute("type", "WTTResource");
$elem.SetAttribute("ResourceName", $Name );
$elem.SetAttribute("ResourceId", $ResourceId );
$elem.SetAttribute("ResourceConfigurationId", $Id );
$doc.get_ChildNodes().Item(1).AppendChild($elem) | out-null

$doc.get_ChildNodes().Item(1).SetAttribute("GUID", [GUID]::NewGuid().ToString() );
$doc.save((Join-path $RolePath "RoleInstance.xml"))

This code uses CreateElement and SetAttribute and then associates as a child to the 2nd item in the doc ($doc.get_ChildNodes().Item(1)) which is my RoleInstance tag. By using the XML APIs I can feel confident that the information in each attribute gets properly encoded and I keep my document well formed. The only down side is that it is slightly harder to visualize the structure of the resulting XML.

-- Ari

Update: Bruce over at the powershell blog teaches me a new trick using Domain Specific Languages to do a cleaner job of constructing the XML. Thanks Bruce!

Windows Rally Demo'd at WinHEC

wndpteam — Wed, 16 May 2007 22:52:00 GMT

I noticed that there was a demo of Rally technologies at the WinHEC keynote the other day, so I created a link to part of the keynote with the Rally demo. Enjoy.

-- Ari Pernick