Windows Core Networking

The final version of the Windows Web Services API for Windows XP, Windows Vista, Windows Server 2003 and Windows Server 2008 is now available. This version corresponds to the final version of the WWSAPI released as part of Windows 7 and Windows Server 2008 R2. To develop for this version of the runtime, you need to install Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1. Please make sure to regenerate all code using the new version of wsutil.exe tool. You will also need to rebuild all source code that used any of the pre-released versions of this API.

This release is available in 25 languages: Arabic, Chinese - Hongkong, Chinese - Simplified, Chinese - Traditional, Czech, Danish, Dutch, English, Finnish, French, German, Greek, Hebrew, Hungarian, Italian, Japanese, Korean, Norwegian , Polish, Portuguese (Brazil), Portuguese (Portugal), Russian, Spanish, Swedish, Turkish. The table below summarizes CPU architectures and the minimal level of Service Packs supported by this release. Make sure to deploy the latest service pack and all critical updates for the version of Windows to computers on which you plan to install this release. To find recent security updates, visit Windows Update.

In contrast to the past pre-released version of this API, this final version release can be used in production code and redistributed with the final versions of the product. Because of this, the final version is only available to companies who agree to terms of Windows Master Redistribution License Agreement (MRLA). To acquire the redistributable installers for this release and a copy of the Windows MRLA for review, please email a formal request to wwsredst@microsoft.com. In your email, please include a brief description of your plans for using this release and the business contact information for your company using which you can be reached at later on. If you have questions about this release, please email them to wwsredst@microsoft.com.

For the super eagle eyed developer or Web Server admin, you might have noticed that cache entry validation happens slightly less frequently in IE8. I’m talking about the check to the Web Server (via If-Modified-Since) to see if the cached version of content needs updating or is still current. The short version of why this happens is the way that IE uses WinINet sessions changed to accommodate Loosely Coupled IE and session’s start time is an important factor in evaluating if WinINet needs to issue an If-Modified-Since request.

Digging deeper, we can look at a couple of the factors in the logic IE/WinINet is using to decide to issue an If-Modified-Since request. First is the the Cache Setting: “Check for newer versions of stored pages” in the “Temporary Internet Files and History Settings” as described in KB article 263070 and pictured with this blog entry. These are settings that apply to cached content missing the Expires headers. “Every time I visit the webpage” and “Never” are pretty straightforward. “Every time I start Internet Explorer” and “Automatically” have logic dependant on the the session’s start time. Another factor to note that we have different behavior when using the Back and Forward browser buttons compared to what we do in hyperlink navigation. The Back and Forward behavior also depends on the session’s start time when content is expired or is missing the Expires header.

How does Loosely Coupled IE in IE8 change the session start time? Previously any IE window got a single session in WinINet by the fact it only uses a single process and sharing of the session handle across tabs, with Loosely Coupled IE we had to do work to let multiple processes share session state. This means that new IE8 windows share the same session compared to new IE7 windows which get a new session. Eric Lawrence on the IEBlog talks a bit more about this session sharing behavior. The effect on WinINet is that IE8 sessions last longer on average then they used to and that means potentially less often revalidation of content from the Web Server.

-- Ari Pernick

HTTP is a request/response protocol. You request some resource like the HTML of a webpage and the response comes back with the HTML attached.  As each request is sent on a connection, the complete response must be read from the connection before the next response can be read. (There is an optimization where you make more requests on a connection before receiving the full response to the previous requests known as pipelining. However it doesn’t change the need to receive all the data from the previous requests first.)

So if an application is asking for another request to a server while the current one is in progress, what’s a HTTP Client API to do? Well, it can either wait till the connection is available or start a new connection. This decision is mostly controlled by a configurable maximum number of simultaneous connections to a server.

With that in mind, how should the following customer question get answered?

My client application’s performance is getting impacted by WinInet opening new SSL connections to my server rather than reusing the existing connection. How do I fix this?

In my last post, I announced the availability of QoS Traffic Generator on Microsoft Connect. In order to help you understand the usage of this tool, I have provided some examples of what it can do.

One of the most common tasks is to send traffic from one computer to another. The following example sends UDP traffic at 5 Mbps for 10 seconds.

Source

C:\qos>qostraffic.exe -source -udp -dest 192.168.1.100 -throttle 5000000 
                      -duration 10
Parsing command line...
WSAStartup successful.
Time between each packet (microsec): 2400
Size of each packet: 1472
Statistics sampling interval (msec): 1000
Sending traffic at 5.00 Mbps to 192.168.1.100:9999
Date, Time, Packets received, Bytes received (headers included), Elapsed time (msec), Throughput (Kbps), Bottleneck BW, Available BW, RTT, BBSet, ABSet, RTTSet, ErrorCode
05/07/2009, 23:06:54, 421, 631500, 1014, 4982.249,,,,,,,
05/07/2009, 23:06:55, 422, 633000, 1014, 4994.083,,,,,,,
05/07/2009, 23:06:56, 423, 634500, 1014, 5005.917,,,,,,,
05/07/2009, 23:06:57, 422, 633000, 1014, 4994.083,,,,,,,
05/07/2009, 23:06:58, 423, 634500, 1014, 5005.917,,,,,,,
05/07/2009, 23:06:59, 422, 633000, 1014, 4994.083,,,,,,,
05/07/2009, 23:07:00, 423, 634500, 1014, 5005.917,,,,,,,
05/07/2009, 23:07:01, 422, 633000, 1014, 4994.083,,,,,,,
05/07/2009, 23:07:02, 423, 634500, 1014, 5005.917,,,,,,,
Time has elapsed, or CTRL+C has been pressed.
Stopping Source traffic, waiting 5 sec.
SenderThread is exiting...
SenderThread exited successfully.
WSACleanup successful.

Sink

C:\qos>qostraffic.exe -sink -udp
Parsing command line...
Listening on all IP addresses.
WSAStartup successful.
Waiting for UDP traffic on port 9999, hit CTRL+C to exit...
WSARecvFrom completed.
Date, Time, Packets received, Bytes received (headers included), Elapsed time (msec), Throughput (Kbps), Message
05/07/2009, 23:06:54, 421, 631500, 1014, 4982.249,
05/07/2009, 23:06:55, 422, 633000, 1014, 4994.083,
05/07/2009, 23:06:56, 423, 634500, 1014, 5005.917,
05/07/2009, 23:06:57, 422, 633000, 1014, 4994.083,
05/07/2009, 23:06:58, 423, 634500, 1014, 5005.917,
05/07/2009, 23:06:59, 422, 633000, 1014, 4994.083,
05/07/2009, 23:07:00, 423, 634500, 1014, 5005.917,
05/07/2009, 23:07:01, 422, 633000, 1014, 4994.083,
05/07/2009, 23:07:02, 423, 634500, 1014, 5005.917,

You can see the throughput in Kbps from the last column, along with other information such as timestamp and bytes received. The QoS Traffic Generator can also support multiple TCP connections from one sink. The next example shows two TCP flows, one using qWave and one using TC (used to throttle traffic) sending to the same destination.

Source 1

C:\qos>qostraffic.exe -source -tcp -dest 192.168.1.100 -throttle 1000000 -qwave
Parsing command line...
qWave traffic type: Best effort
qWave flow type: non-adaptive.
qWave LLTD option: system choice.
qWave flow fundamentals: do not display
WSAStartup successful.
qWave started up successfully.
Send interval: not used, packets will be sent as fast as possible
Packets to send per interval: 1
Size of each packet: 1000
Statistics sampling interval (msec): 1000
Sending traffic at 1.00 Mbps to 192.168.1.100:9999
Date, Time, Packets received, Bytes received, Elapsed time (msec), Throughput (Kbps), Bottleneck BW, Available BW, RTT, BBSet, ABSet, RTTSet, ErrorCode
05/07/2009, 23:02:04, 129, 125000, 1014, 986.193,,,,,,,
05/07/2009, 23:02:05, 126, 126000, 1014, 994.083,,,,,,,
05/07/2009, 23:02:06, 125, 125000, 1014, 986.193,,,,,,,
05/07/2009, 23:02:07, 126, 126000, 1014, 994.083,,,,,,,
05/07/2009, 23:02:08, 124, 124000, 1014, 978.304,,,,,,,
05/07/2009, 23:02:09, 127, 127000, 1015, 1000.985,,,,,,,
05/07/2009, 23:02:10, 125, 125000, 1014, 986.193,,,,,,,
05/07/2009, 23:02:11, 126, 126000, 1014, 994.083,,,,,,,
05/07/2009, 23:02:12, 125, 125000, 1014, 986.193,,,,,,,
05/07/2009, 23:02:14, 124, 124000, 1014, 978.304,,,,,,,
05/07/2009, 23:02:15, 126, 126000, 1014, 994.083,,,,,,,
05/07/2009, 23:02:16, 127, 127000, 1014, 1001.972,,,,,,,
05/07/2009, 23:02:17, 125, 125000, 1014, 986.193,,,,,,,
05/07/2009, 23:02:18, 124, 124000, 1014, 978.304,,,,,,,
ERROR: 64 in GetQueuedCompletionStatus: The specified network name is no longer 
available.
SenderThread is exiting...
SenderThread exited successfully.
Stopping Source traffic, waiting 5 sec.
SenderThread exited successfully.
WSACleanup successful.

Source 2

C:\qos>qostraffic.exe -source -tcp -dest 192.168.1.100 -throttle 2000000 -tc 40,4
Parsing command line...
WSAStartup successful.
Send interval: not used, packets will be sent as fast as possible
Packets to send per interval: 1
Size of each packet: 1000
Statistics sampling interval (msec): 1000
Sending traffic at 2.00 Mbps to 192.168.1.100:9999
Date, Time, Packets received, Bytes received, Elapsed time (msec), Throughput (Kbps), Bottleneck BW, Available BW, RTT, BBSet, ABSet, RTTSet, ErrorCode
05/07/2009, 23:01:59, 498, 498000, 1014, 3928.994,,,,,,,
05/07/2009, 23:02:00, 248, 248000, 1014, 1956.607,,,,,,,
05/07/2009, 23:02:01, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:02, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:03, 252, 252000, 1014, 1988.166,,,,,,,
05/07/2009, 23:02:04, 247, 247000, 1014, 1948.718,,,,,,,
05/07/2009, 23:02:05, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:07, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:08, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:09, 247, 247000, 1014, 1948.718,,,,,,,
05/07/2009, 23:02:10, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:11, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:12, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:13, 252, 252000, 1014, 1988.166,,,,,,,
05/07/2009, 23:02:14, 250, 250000, 1014, 1972.387,,,,,,,
05/07/2009, 23:02:15, 252, 252000, 1014, 1988.166,,,,,,,
05/07/2009, 23:02:16, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:17, 251, 251000, 1014, 1980.276,,,,,,,
05/07/2009, 23:02:18, 247, 247000, 1014, 1948.718,,,,,,,
ERROR: 64 in GetQueuedCompletionStatus: The specified network name is no longer 
available.
SenderThread is exiting...
SenderThread exited successfully.
Stopping Source traffic, waiting 5 sec.
SenderThread exited successfully.
WSACleanup successful.

Sink

C:\qos>qostraffic.exe -sink -tcp
Parsing command line...
Listening on all IP addresses.
WSAStartup successful.
Waiting for TCP traffic on port 9999, hit CTRL+C to exit...
AcceptEx has successfully completed.
Overlapped AcceptEx signaled.
Connection received: ::ffff:157.59.28.243:56677 --> ::ffff:192.168.1.100:9999
Date, Time, Packets received, Bytes received, Elapsed time (msec), Throughput (Kbps), Message
05/07/2009, 23:01:59, 498, 498000, 1014, 3928.994,
05/07/2009, 23:02:00, 248, 248000, 1014, 1956.607,
05/07/2009, 23:02:01, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:02, 251, 251000, 1014, 1980.276,
Overlapped AcceptEx signaled.
Connection received: ::ffff:157.59.28.243:56678 --> ::ffff:192.168.1.100:9999
Date, Time, Packets received, Bytes received, Elapsed time (msec), Throughput (Kbps), Message
AcceptEx has successfully completed.
05/07/2009, 23:02:03, 252, 252000, 1014, 1988.166,
05/07/2009, 23:02:04, 129, 125000, 1014, 986.193,
05/07/2009, 23:02:04, 247, 247000, 1014, 1948.718,
05/07/2009, 23:02:05, 126, 126000, 1014, 994.083,
05/07/2009, 23:02:05, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:06, 125, 125000, 1014, 986.193,
05/07/2009, 23:02:07, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:07, 126, 126000, 1014, 994.083,
05/07/2009, 23:02:08, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:08, 124, 124000, 1014, 978.304,
05/07/2009, 23:02:09, 247, 247000, 1014, 1948.718,
05/07/2009, 23:02:09, 127, 127000, 1015, 1000.985,
05/07/2009, 23:02:10, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:10, 125, 125000, 1014, 986.193,
05/07/2009, 23:02:11, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:11, 126, 126000, 1014, 994.083,
05/07/2009, 23:02:12, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:12, 125, 125000, 1014, 986.193,
05/07/2009, 23:02:13, 252, 252000, 1014, 1988.166,
05/07/2009, 23:02:14, 124, 124000, 1014, 978.304,
05/07/2009, 23:02:14, 250, 250000, 1014, 1972.387,
05/07/2009, 23:02:15, 126, 126000, 1014, 994.083,
05/07/2009, 23:02:15, 252, 252000, 1014, 1988.166,
05/07/2009, 23:02:16, 127, 127000, 1014, 1001.972,
05/07/2009, 23:02:16, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:17, 125, 125000, 1014, 986.193,
05/07/2009, 23:02:17, 251, 251000, 1014, 1980.276,
05/07/2009, 23:02:18, 124, 124000, 1014, 978.304,
05/07/2009, 23:02:18, 247, 247000, 1014, 1948.718,
Event set, thread exiting.
Receive thread exiting.
ReceiverThread exited successfully.
Control Event or Stop() called. TCP receive thread is exiting.
Main receive thread exited successfully.
WSACleanup successful.

The sink prints out statistics from both sources and was stopped using CTRL+C after about 20 seconds. Both traffic sources then exit gracefully. A full list of options is shown below.

C:\qos>qostraffic.exe -?
Parsing command line...
Need to specify -source or -sink flag.

QoS Traffic Generator (Daytona), v. 1.0

Usage:

Sink: qostraffic -sink -tcp/-udp/-tcp_a 
                 [-dest destIP,destPort -local localIP,localPort] ...
Source: qostraffic -source -tcp/-udp/-tcp_a -dest destIP,destPort 
                   -throttle rate
                   [-local localIP,localPort -duration sec] ...

Sink flags:
-tcp/-udp       Listens for TCP/UDP traffic
                default: listens on all addresses, port 9999
-tcp_a          Connects to source and then begins receiving traffic
                Source must also specify -tcp_a flag
                Note: start the source first, then the sink
-dest           OPTIONAL
                Destination address to connect to for -tcp_a
                Ignored for -tcp/-udp cases
-local          OPTIONAL
                Listens on specific local address/port
                default port: 9999
                Note: Can specify different port to listen on with,
                      localPort only.
-qwave          OPTIONAL
                Starts up qWave service.
                Used for adaptive flows from the source.
-noconsole      OPTIONAL
                Suppresses console output
                default is not suppressed
Source flags:
-tcp/-udp       Listens for TCP/UDP traffic
-tcp_a          Accepts connection from sink then sends traffic
                Sink must also specify -tcp_a flag
                Note: start the source first, then the sink
-dest           Send traffic to destination address/port
                default port: 9999
                Note: ignored for -tcp_a case
-throttle       Throttles traffic at specified rate (bits/sec)
-pull           OPTIONAL
                Operates the source in pull mode.
                Note: supported for TCP traffic only.
-local          OPTIONAL
                Send traffic from specified local address/port
                default port: 9999
                Note: for -tcp_a case, this is the local address to 
                      listen on
-duration       OPTIONAL
                Send traffic for specified time (sec)
                default: infinite
-stats          OPTIONAL
                Prints out statistics for specified time (msec)
                default: 1000 msec
                minimum: 200 msec
-qwave          OPTIONAL
                Uses qWave with the specified options (all optional)
                ,,
                trafficType: traffic type defined by QoS subsystem
                        BE - best effort (default)
                        BCK - background
                        EE - excellent effort
                        AV - audio/video
                        VO - voice
                        CTRL - control
                adaptive: force use adaptive flow
                        default is nonadaptive
                lltd: force use LLTD packets
                        default lets system choose LLTD or not
                flowfund: prints out QueryFlowFundamentals
                        prints bandwidth estimations every 25 ms
                        only supported when using adaptive flows
                EXAMPLE: -qwave AV,LLTD
-server         Optional, when used with qWave will prevent doing 
                   QOSStartTracking/QOSStopTracking
                Note: conflicts with -tc flag.
-tc             OPTIONAL
                Uses TC with the specified options (all optional)
                ,<1p>
                dscp: DSCP value ranging from 0 to 63
                        default lets system choose
                1p: 802.1p tag ranging from 0 to 7
                        default lets system choose
                Specifying -1 for either value is equivalent to 
                default. EXAMPLE: -tc 40,4
                Note: conflicts with -qwave flag. Must be run as 
                      admin.
-packet         OPTIONAL
                Performs packet experiments with given options (all 
                required)
                ,,
                size - size of each packet in bytes
                num - number of packets to send
                interval - time between each send in ms
                EXAMPLE: -packet 59,50,100
                This will send a 59 byte packet every 100 ms, 50 
                times total.
                Note: packets can be tagged by using -tc options.
-log            OPTIONAL
                Logs to the specified file
                default filename: is "daytona_.txt"
-noconsole      OPTIONAL
                Suppresses console output
                default is not suppressed
-localstats     OPTIONAL
                Prints statistics from source
                default is sink side stats
                Note: conflicts with packet option, results will be 
                undefined
-tolerance      OPTIONAL
                Specify tolerance
                If no tolerance will assume 2%

Hopefully this post has been helpful in understanding what the QoS Traffic Generator can do. Please send us any comments you have!

  -- Jim Liu

Just a quick note that over on on haoxu’ s “Lost in History” blog is a posting about how to create and register test SSL certs in http.sys for the Windows Web Services API Security Sample code. This might also be useful for developers using the http.sys apis.

I’m posting on behalf of Jim Liu who is a SDET in Windows Core Networking. -- Ari

The QoS team is pleased to announce the availability of the QoS Traffic Generator on Microsoft Connect. This tool was developed during the Windows 7 development process and has been used extensively to aid in the testing of our QoS technologies. It is a command-line based tool that generates network traffic and utilizes TC/qWave to throttle and mark packets as desired. Some notable features include:

· Ability to send traffic at a specified throughput using Winsock from one machine to another

· Throttling is performed by the application itself or through QoS APIs (Traffic Control or qWave)

· Support TCP and UDP dual-mode sockets

· Provide traffic statistics from either the source machine or destination machine

· Can be built as a static library for use in other applications

We are providing this tool as a demonstration of QoS technologies and how they may be used to produce a novel application. Both 32 and 64-bit binaries are provided along with the source code and instructions for building the static library and the accompanying executable. The application itself has usage information on the available usage options.

To download the package, go to the WNDP site on Microsoft Connect and select Downloads from the left-hand menu. Select QoS Traffic Generator. Please send us any feedback you have about the tool!

  -- Jim Liu

Windows Web Services API team has just release the Beta version of the API for Windows XP, Windows Vista, Windows Server 2003 and Windows Server 2008. It is available now on the Connect site for Windows Networking. This release contains the same version of the runtime as in Windows 7 Beta. You may find the table that summarizes level of Service Packs and CPU architectures supported by this release and direct links to download pages in this announcement. Please try it out and report any installation and runtime issues by submitting the feedback using the WNDP Connect site. The reference documentation for the API is available from MSDN. Please use this forum to ask WWSAPI team questions about the API and just released installers.

Hello all. My name is Charley. I’m the new QoS program manager for Windows Core Networking.

It has been a while since we posted our last article about QoS. We want to assure you that we’re still committed to improving this technology and building new QoS features in Windows. We received many questions and suggestions from you in the past. We hope you continue doing so because they are important to us.

You probably all know that Windows 7 beta is live and available for you to download. You may wonder what is new in Windows 7 regarding QoS. But before getting to that, I’d like to refresh our memory as to what was new in Windows Vista. In Vista, we introduced two major QoS features: qWAVE and Policy based QoS. qWAVE, or Quality Windows Audio Video Experience, is designed to estimate the network bandwidth, intelligently mark the application packets (with proper DSCP values), and interact with the application in the event of network congestion or fluctuations of available bandwidth (so that the application can take appropriate actions). qWAVE APIs aim to simplify the work of application developers developing QoS-enabled applications for a home network. The APIs are documented in the Windows SDK. In contrast, Policy based QoS has a different target audience. It is intended to enable IT administrators to apply QoS to applications (which don’t need to have native supports of QoS), computers, and users in their enterprise network. It is especially beneficial to an organization with branch offices, where the WAN link capacity is limited and users tend to experience unpredictable network delays when accessing files or applications hosted on the main campus (or a different branch office). If you’d like to know more about these features, we’d suggest you read an excellent article written by Joseph Davies for TechNet magazine. Click here to check it out.

So what’s new in Windows 7? The enhancement we’ve made is called URL based QoS. If you’re familiar with Policy based QoS you know that in Vista an IT administrator can create a policy based on the application name, source and/or destination IP addresses, source and/or destination ports, and the protocol (TCP, UDP, or both). We’ve learned since then that many enterprise applications have been, or will be, hosted on web servers and accessed from a browser, so the IT administrators would love to be able to prioritize or control the network traffic from those web-based applications, provided that a convenient configuration is available. To answer their request, we’ve added a new configuration option: you can create a policy based on the URL of an HTTP server. Say, you host all the training videos of your company on an IIS server running Windows 7 (or more precisely, Windows Server 2008 R2). You can then use a URL based QoS policy to throttle the video traffic (to prevent it from overwhelming your corporate network). Or, you host your company’s CRM or ERP applications on an HTTP application server running Windows 7. You can similarly use a URL based QoS policy to prioritize these applications traffic so that even users at a remote branch office always get prompt replies and have smooth UI experience.

Does this sound interesting to you? Please let us know what you think. In the next blog, we’ll detail the configuration and the rules.

Charley

We are demoing Windows 7 Direct Access and Windows 7 Branch Cache at the PDC08 pavillion. We are the second booth to the left when you come in to the Big Room, shaing space with the Windows 7 Sensor folks. Come check it out at talk to us.

  -- Ari

I’ve been pretty busy Engineering Windows 7 with the rest of the windows networking team and we are really excited to start showing the world some of the work we’ve done at PDC. One of the things we’ve been working on has it’s own session: “Windows 7: Web Services in Native Code” presented by Nikola Dudar. Also those who attend the PDC get Windows 7 bits.

It turns out I’ll be at PDC as well, so let me know if you are heading there and wish to meet up. If you haven’t already registered, act quickly because registration is quickly getting full.

-- Ari Pernick

Hello All, I am continuing the "WinHTTP Questions" series with some questions on WinHTTP callbacks.

Is it correct that WinHTTP Callbacks will occur *only* during an in-progress WinHTTP operation? Is it possible that an external event (such as the remote server resetting the underlying tcp connection) result in a callback even when there's no outstanding operation?

No. For external callbacks, WinHTTP will indicate informational callbacks only when there is a pending operation. Essentially, there are two types of callbacks in WinHTTP. Completion callbacks and Informational callbacks.

• Completion callbacks are invoked as a completion of a pending API call e.g. read complete is invoked when the read API completes. Similarly, handle-closed callback is invoked in response to the close handle API.
• Informational callbacks, on the other hand, are invoked to inform the application about important events seen during the processing of a call.

So, when an async API is invoked and it returns TRUE, the application will receive zero or more informational callbacks and then a completion callback.

When an operation like WinHttpReadData fails immediately (i.e., this call returned FALSE), does it guarantee that user will not receive a STATUS_REQUEST_ERROR callback?

Yes.

Does WinHTTP synchronize WinhttpSetStatusCallback with worker threads? Here is our usage scenario: We are implementing a DLL that get dynamically loaded implements an asynchronous Download function with progress notifications and supports a Cancel call. After Cancel or Download completion our DLL should be able to be safely unloaded. Cancel calls WinHttpSetStatusCallback to unregister notifications, closes the request, connection and sessions handles. However we occasionally still get status callbacks after the DLL is unloaded.

No, WinHTTP does not synchronize WinHttpSetStatusCallback with worker threads. For e.g.  If a  callback originating in another thread is in progress when an application calls WinHttpSetStatusCallback, the application still receives a callback notification even after WinHttpSetStatusCallback successfully sets the callback function to NULL and returns. The thread will continue to process, so one will need to wait until the current call finishes.

Please also refer to the following links for more on callbacks.

• WINHTTP_STATUS_CALLBACK Callback Function (MSDN)
• WinHttpSetStatusCallback Function (MSDN)

Next time we will go a bit deeper into safe request cancellation.

  -- Deepak & Ari

Hello, my name is Deepak and I'm a SDET in serviceability. We handle a bunch of questions from developers using WinHTTP, and thought we might share then in a new posting series, "WinHTTP Questions".

Can I cancel a synchronous WinHttpSendRequest call by closing the request handle from a different thread? Or, are there any requirements that I need to use the asynchronous WinHttpSendRequest if I need the ability to cancel it?

The short answer is don't do that.

Let's think about the implications of this proposed code. Thread 1 starts a synchronous request and Thread 2 comes along an issues a close. We can't synchronize between Thread 1 and Thread 2 by definition because Thread 1's call is synchronous, we either can deterministically call the close before the request call starts or after the call is completed. So to close the handle during the call results in a race condition. If thread 2 attempts to close the connection during the call, the handle passed in to WinHttpSendRequest may become invalid before WinHttp has a chance to work with it. While you might get lucky, you might also non-deterministically get a crash or memory corruption.

  -- Deepak and Ari

Windows Server 2008 incorporates the completely rewritten TCP/IP stack we shipped in Windows Vista and since this is the first server release with the stack we'd like to ask our readers to load up Windows Server RC1 and see if your applications are ready.

The new features (compared to Windows Server 2003) that may pose potential compatibility issues include: removed filter hooks, receive window auto tuning, dual IP layer architecture for IPv6, compound TCP, ECN support, default strong host model, easier kernel mode programming, extensive protocol offload.

Windows Server 2008 is also the very first Microsoft server product to have the firewall ON by default. Third party applications that are not aware of this behavior may break. 

-- Ari and Katarzyna

Just wanted to drop a quick Happy New Year to our readers and see if there is any topics that you want to hear about. We've been pretty busy with Windows Server 2008, Windows Vista SP1 and future windows releases. With the RTM of Windows Server 2008 and Windows Vista SP1 drawing near, I hope to get some posts on things that are new or have changed, but I'd be happy to hear whatever people are interested in.

-- Ari

In his introductory post about the legacy Traffic Control (TC) API, Gabe discussed the host-based model that TC provides. In this post, we will see how Traffic Control APIs can be used to achieve the following for TCP/IPv4 and UDP/IPv4 traffic sent from a host:

• Throttle (rate-limit) outgoing traffic
• Add DSCP value in layer-3 (IPv4) header
• Indicate that an 802.1p tag value should be added in layer-2 header

At the bottom of the post, we’ve provided a link to the Networking Connect site to download full source and binaries to a tool which implements all of the above functionality.

The following are the steps involved:

1. The first step in the process is to obtain a handle to the Traffic Control subsystem through a call to TcRegisterClient().

2. Next, make a call to TcEnumerateInterfaces() using the registration handle obtained in step #1. This call returns a list of all TC enabled interfaces on the system. Iterate through the list to find the interface(s) on which you want to prioritize and/or throttle outgoing traffic.

3. For each interface of interest, issue a TcOpenInterface() using the pInterfaceName from the corresponding TC_IFC_DESCRIPTOR for that interface from the list returned in step#2. Store the handle returned by TcOpenInterface(); let’s call it the ifchandle.

4. At this point, create a TC Flow and add it to the interface(s) of interest.

a. Create the TC Flow:
A TC flow is a way of describing various QoS characteristics to be applied to a set of packets and is represented by a TC_GEN_FLOW structure. The following code snippet shows how to create a TC Flow given the DSCP value, 802.1p value and the throttle rate.

BOOL CreateFlow(PTC_GEN_FLOW * _ppTcFlowObj, USHORT DSCPValue, USHORT OnePValue, ULONG ThrottleRate)
{
  BOOL status = FALSE;

  //
  // Flow Parameters
  //
  ULONG TokenRate = QOS_NOT_SPECIFIED;
  ULONG TokenBucketSize = QOS_NOT_SPECIFIED;
  ULONG PeakBandwidth = QOS_NOT_SPECIFIED;
  ULONG Latency = QOS_NOT_SPECIFIED;
  ULONG DelayVariation = QOS_NOT_SPECIFIED;
  SERVICETYPE ServiceType = SERVICETYPE_BESTEFFORT;
  ULONG MaxSduSize=QOS_NOT_SPECIFIED;
  ULONG MinimumPolicedSize=QOS_NOT_SPECIFIED;
  PVOID pCurrentObject;
  PTC_GEN_FLOW _pTcFlowObj = NULL;
  int Length = 0;

  //
  // Calculate the memory size required for the optional TC objects
  //

  Length += (OnePValue == NOT_SPECIFIED ? 0:sizeof(QOS_TRAFFIC_CLASS)) + (DSCPValue == NOT_SPECIFIED ? 0:sizeof(QOS_DS_CLASS));

  //
  // Print the Flow parameters
  //

  printf("Flow Parameters:\n");
  DSCPValue == NOT_SPECIFIED ? printf("\tDSCP: *\n"):printf("\tDSCP: %u\n", DSCPValue);
  OnePValue == NOT_SPECIFIED ? printf("\t802.1p: *\n"):printf("\t802.1p: %u\n", OnePValue);
  ThrottleRate == QOS_NOT_SPECIFIED ? printf("\tThrottleRate: *\n"):printf("\tThrottleRate: %u\n", ThrottleRate);
  TokenRate = TokenBucketSize = ThrottleRate;

  //
  // Allocate the flow descriptor
  //
  _pTcFlowObj = (PTC_GEN_FLOW)malloc(FIELD_OFFSET(TC_GEN_FLOW, TcObjects) + Length);

  if (!_pTcFlowObj)
  {
    printf("Flow Allocation Failed\n");
    goto Exit;
  }

  _pTcFlowObj->SendingFlowspec.TokenRate = TokenRate;
  _pTcFlowObj->SendingFlowspec.TokenBucketSize = TokenBucketSize;
  _pTcFlowObj->SendingFlowspec.PeakBandwidth = PeakBandwidth;
  _pTcFlowObj->SendingFlowspec.Latency = Latency;
  _pTcFlowObj->SendingFlowspec.DelayVariation = DelayVariation;
  _pTcFlowObj->SendingFlowspec.ServiceType = ServiceType;
  _pTcFlowObj->SendingFlowspec.MaxSduSize = MaxSduSize;
  _pTcFlowObj->SendingFlowspec.MinimumPolicedSize = MinimumPolicedSize;

  //
  // Currently TC only supports QoS on the send path
  // ReceivingFlowSpec is legacy and ignored
  //

  memcpy(&(_pTcFlowObj->ReceivingFlowspec), &(_pTcFlowObj->SendingFlowspec), sizeof(_pTcFlowObj->ReceivingFlowspec));
  _pTcFlowObj->TcObjectsLength = Length;

  //
  // Add any requested objects
  //
  pCurrentObject = (PVOID)_pTcFlowObj->TcObjects;
  if(OnePValue != NOT_SPECIFIED)
  {
    QOS_TRAFFIC_CLASS *pTClassObject = (QOS_TRAFFIC_CLASS*)pCurrentObject;
    pTClassObject->ObjectHdr.ObjectType = QOS_OBJECT_TRAFFIC_CLASS;
    pTClassObject->ObjectHdr.ObjectLength = sizeof(QOS_TRAFFIC_CLASS);
    pTClassObject->TrafficClass = OnePValue; //802.1p tag to be used
    pCurrentObject = (PVOID)(pTClassObject + 1);
  }

  if(DSCPValue != NOT_SPECIFIED)
  {
    QOS_DS_CLASS *pDSClassObject = (QOS_DS_CLASS*)pCurrentObject;
    pDSClassObject->ObjectHdr.ObjectType = QOS_OBJECT_DS_CLASS;
    pDSClassObject->ObjectHdr.ObjectLength = sizeof(QOS_DS_CLASS);
    pDSClassObject->DSField = DSCPValue; //Services Type
  }

  DeleteFlow(_ppTcFlowObj);
  *_ppTcFlowObj = _pTcFlowObj;
  status = TRUE;
  Exit:

  if(!status)
  {
    printf("Flow Creation Failed\n");
    DeleteFlow(&_pTcFlowObj);
  }
  else
    printf("Flow Creation Succeeded\n");

  return status;
}

b. Add the Flow on the interface:
After obtaining a TC_GEN_FLOW structure with the desired characteristics using a function similar to the one above, issue a call to TcAddFlow() with the ifchandle (obtained in step #3) and a pointer to the TC_GEN_FLOW object (obtained in step #4a). Store the handle returned by TcAddFlow();let’s call it the flowhandle.

5. The next step is to create a TC Filter and add it to the TC Flow created above.

a. Create the TC Filter:
A TC Filter is a way of describing which packets to apply the QoS characteristics to. The QoS characteristics defined in the TC_GEN_FLOW will only apply to the packets matching the filter(s) associated with the Flow.
The following code snippet describes how to create a TC Filter given the destination address, the destination port, and the protocol (TCP,UDP or IP).

BOOL CreateFilter(PTC_GEN_FILTER * ppFilter, SOCKADDR_STORAGE Address, USHORT Port, UCHAR ProtocolId)
{

  BOOL status = FALSE;
  USHORT AddressFamily = Address.ss_family;
  PTC_GEN_FILTER pFilter = NULL;
  PIP_PATTERN pPattern = NULL;
  PIP_PATTERN pMask = NULL;

  if(AddressFamily != AF_INET)
    goto Exit;

  //
  // Allocate memory for the filter
  //
  pFilter = (PTC_GEN_FILTER)malloc(sizeof (TC_GEN_FILTER));
  if(!pFilter)
    goto Exit;

  ZeroMemory(pFilter, sizeof(TC_GEN_FILTER));

  //
  // Allocate memory for the pattern and mask
  //
  pPattern = (PIP_PATTERN)malloc( sizeof(IP_PATTERN));
  pMask = (PIP_PATTERN)malloc( sizeof(IP_PATTERN));
  if(!pPattern || !pMask)
    goto Exit;

  memset ( pPattern, 0, sizeof(IP_PATTERN) );
  pPattern->DstAddr = ((SOCKADDR_IN *)&Address)->sin_addr.s_addr;
  pPattern->tcDstPort = htons(Port);
  pPattern->ProtocolId = ProtocolId;
  memset ( pMask, (ULONG) -1, sizeof(IP_PATTERN) );

  //
  // Set the source address and port to wildcard
  // 0 -> wildcard, 0xFF-> exact match
  //
  pMask->SrcAddr = 0;
  pMask->tcSrcPort = 0;

  //
  // if the user specified 0 for dest port, dest address or protocol
  // set the appropriate mask as wildcard
  // 0 -> wildcard, 0xFF-> exact match
  //
  if(pPattern->tcDstPort == 0)
    pMask->tcDstPort = 0;

  if(pPattern->ProtocolId == 0)
    pMask->ProtocolId = 0;

  if(pPattern->DstAddr == 0)
    pMask->DstAddr = 0;

  pFilter->AddressType = NDIS_PROTOCOL_ID_TCP_IP;
  pFilter->PatternSize = sizeof(IP_PATTERN);
  pFilter->Pattern = pPattern;
  pFilter->Mask = pMask;

  //
  // Delete any previous instances of the Filter
  //
  DeleteFilter(ppFilter);
  *ppFilter = pFilter;
  status = TRUE;

  Exit:
  if(!status)
  {
    printf("Filter Creation Failed\n");
    DeleteFilter(&pFilter);
  }
  else
    printf("Filter Creation Succeeded\n");

  return status;
}

b. Adding the Filter to the TC Flow:
Once a TC Filter structure is obtained using a function similar to the one above, issue a call to TcAddFilter() passing the flowhandle obtained in step #4b and a pointer to the TC_GEN_FILTER structure obtained in step #5a. Store the filter handle returned by TcAddFilter(); let’s call it filterhandle.
You can add multiple filters on the same flow causing different sets of packets matching each filter to get the same QoS characteristics applied to them.

6. At this point, your application is applying QoS on all matching outgoing packets as specified in the TC Filter and TC Flow. Finally, once your purpose is served, make sure you call the respective close calls on all the open TC handles – TcDeletefilter(), TcDeleteFlow(), TcCloseInterface() and TcDeregisterClient().

You can download the full source and binaries of a simple command line tool – tcmonlite, which takes the filter and flow parameters as input, creates TC Flow and Filter, and configures the QoS subsystem with them using the Traffic Control API. All the outgoing traffic on the system matching the filter gets the desired QoS characteristics as long as tcmonlite is running. Go to the Microsoft Connect website, choose Available Connections on the left-hand side of the page, and select Windows Networking from the available connections (bottom half of the page). On the left-hand side of the Windows Networking page, choose Downloads, and select TCMonLite.

This tool can be used in conjunction with the NDIS LWF driver to detect 802.1p tags in the Ethernet header and DSCP in the IP header of packets. Let us know what you think!

-- Hemant Banavar