Network Class Library Team (System.Net)

UDP Multicast in Silverlight 4

Aaron Oneal [MSFT] — Wed, 18 Nov 2009 22:40:00 GMT

In Silverlight 4, we have added multicast support. If you’re not familiar with multicast, here’s a quick scenario to explain what it is and why it might be useful to you.

Scenario Overview

Suppose your company provides market data and you need to distribute the same commodity and value records to 50,000 client workstations. It would be quite a load on the server and network infrastructure to do this in the traditional client/server TCP model where all 50,000 clients connect to the server to receive the data feed in round-robin fashion.

Wouldn’t it be great if the server could just send the data once and let the network infrastructure pass it on to all subscribers in an efficient manner? That would take the load off the server since it would only be sending out the data once instead of 50,000 times. Conveniently, that’s precisely how multicast works, and this makes it a useful technique for video and content distribution.

User Datagram Protocol (UDP)

User Datagram Protocol (UDP) supports multicast and is the most common low-level protocol for IP multicast. Because it’s so low-level, there are some key things you should be aware of:

UDP by nature is not “reliable” like TCP. This means messages are not guaranteed to be received in order or at all. There are, however, forms of reliable multicast, such as Pragmatic General Multicast (PGM) which can be implemented on top of UDP.
UDP has no built in IP address spoofing protection. Just because a message indicates it came from a particular IP address, you can’t be sure it did, so your application protocol needs to implement this protection if you need to be sure of the origin of your messages.

Multicast Concepts

Multicast Groups and Addresses

People are most familiar with unicast addresses, which are used to direct packets to a single host. Alternatively, a special multicast address can be used to deliver a packet to a group of destinations; specifically, all nodes which have explicitly joined the multicast group as identified by the multicast address. The 224.0.0.0/4 IPv4 and ff00::/8 IPv6 address ranges have been reserved for identifying multicast groups.

Joining a Multicast Group

When a computer or device wants to join a multicast group, it performs a special exchange with its router using the Internet Group Management Protocol (IGMP) or the Multicast Listener Discovery (MLD) protocol. This lets the router know the computer is interested in receiving traffic from the multicast group. To leverage multicast, it must be supported by the router and not otherwise blocked.

Multicast is often deployed within corporate networks for efficient transmission of video. Be aware that it is common for corporate firewalls to block multicast traffic at the edge so that it cannot traverse the boundary of the corporate network. Many home ISPs enforce similar restrictions.

It’s also worth mentioning that there are multiple versions of IGMP and MLD, but the specifics of negotiation are up to the client OS stack and the first-hop router; it’s not something your application needs to worry about.

Finally, in addition to joining a multicast group, a complementary operation to leave the group is also available.

Multicast Sources

The node or nodes sending data to the group are known as multicast sources. There are two common forms of IP multicast differentiated by the number of sources in the group:

Single Source Multicast / Source Specific Multicast (SSM) – A single, well-known source identified by a unicast IP address will send to the group (one-to-many).
Any Source Multicast (ASM) – Any member of the group may behave as a multicast source and transmit data to the group (many-to-many).

In the case of any source multicast, additional operations are provided to block and unblock specific sources so that messages can be filtered by source address. One of the benefits of SSM is that with a single source known ahead of time, the multicast tree can be built more efficiently when nodes join the group.

New APIs

In correlation to the multicast forms above, two new classes have been introduced to the System.Net.Sockets namespace, UdpSingleSourceMulticastClient and UdpAnySourceMulticastClient. If you’re familiar with Sockets or UdpClient in the .NET Framework, the available operations should look very familiar to you. Here are some brief, simplified samples to demonstrate the basics.

Construction

// Construct a client for the multicast group 224.0.0.1 port 3000
// Note that only ports >= 1024 are supported for Silverlight
var client = new UdpAnySourceMulticastClient(IPAddress.Parse(“224.0.0.1”), 3000);

Join Group

// Join the multicast group
client.BeginJoinGroup((result) => EndJoinGroup(result), null);

Message Receive

// Receive a message
client.BeginReceiveFromGroup(buffer, offset, count,
(result) => { messageLength = EndReceiveFromGroup(result, sourceAddress); }, null);

Leave Group and Dispose

// Leave the group and cleanup connections
client.Dispose();

In a real application, the async calls would be chained together. For a more detailed example showing this, see SilverChat on MSDN Code Gallery.

Browser Security

Because multicast in corporate environments is often blocked at the edge, Silverlight should not be able to circumvent that trust model by allowing code from an untrusted domain to run unchecked within the corporate network where it could potentially receive and forward privileged data. To prevent this, Silverlight requires that a Silverlight Multicast Policy Responder join the multicast group to authorize Silverlight clients. If multicast has been blocked at the corporate edge, then only a service behind the corporate firewall will be able to authorize Silverlight clients, thus maintaining the original network boundary.

You are free to deploy the responder in any way you choose, whether on the same or separate machine as your multicast source.

For details of the policy protocol used and to review a sample implementation, please see the Silverlight Multicast Policy Responder sample also on MSDN Code Gallery.

Conclusion

This article covered how multicast can be used to reduce server and network infrastructure load, the new APIs added in Silverlight 4 to support these features, the special requirements to maintain browser security, and finally, some important points to consider for your application protocols built on UDP. Please let us know if you have questions about these new features. In addition to blog comments, you may also direct inquiries to the forum.

Visual Studio 2010 and .NET Framework 4 Beta 2 Released!

Aaron Oneal [MSFT] — Mon, 19 Oct 2009 17:47:11 GMT

Grab the beta here and please be sure to send us feedback using Connect. Hope you enjoy the release!

New Performance Counters for HttpWebRequest

PeSmith — Sat, 08 Aug 2009 06:56:00 GMT

Performance counters! You can never get enough! And starting with the .NET version 4.0 Beta 2, we in the System.Net team have added six new counters to the old set. But before we begin, some ground rules:

Performance counters have to be enabled in your application’s .config file. I’ve put a sample of my config file at the end of this post.
The new counters need the .NET version 4 Beta 2 runtime.
The counters are now called “.NET CLR Networking 4.0.0.0” (the version number is new). You only get the new performance counters when you run using the .NET version 4 CLR. If your program runs using an older CLR, it will use the older performance counters.
The counters are now initialized on a separate thread. This means that for the first small amount of time, activity isn’t recorded by the performance monitor.

Without any further delay, let’s take a look at the new counters.

The first two performance counters let you know that your application is running smoothly.

1. HttpWebRequests Created/Sec
2. HttpWebRequests Average Lifetime

The ‘Created/Sec’ is just what it says: it’s updated whenever an HttpWebRequest is created. The performance counter takes care of adjusting the raw numbers into a “per sample interval” – just beware that the label says “per second” but it’s really “per performance counter sample time interval”. It turns out that the default sample interval is one second, and the label is much shorter when we just call it “/Sec”.

You can see the effects of this adjustment for yourself – make a program that creates a fixed number of HttpWebRequests per second. When the performance monitor is set to “Sample every 1 seconds”, it gives that amount. Set it to “Sample every 2 seconds” and the number doubles.

But watch out! System.Net will make HttpWebRequest objects for you – in particular, the System.Net automatic proxy detection on Windows XP will create an HttpWebRequest internally to resolve your proxy. System.Net will also make extra objects for tunneling through proxies. It’s even possible for FTP requests to silently make HttpWebRequests. Exactly when we make these extra objects can change from one release to the next.

The average lifetime is a little sneakier. It’s the lifetime of the entire HTTP transaction, starting when the HttpWebRequest is created all the way until the resulting HttpWebResponse is closed. If you ever see this counter stuck at zero even though you’re making requests and getting replies, it’s probably because you haven’t closed your HttpWebResponse! The ‘average’ is the average for the last performance sample interval, so it’s just for the time in the “Sample every ___ seconds” option in the Performance Monitor properties box.

There are two performance counters to look for Queuing issues. The .NET framework doesn’t let you (by default) send lots of requests to a server all at once. Instead, only up to ‘ConnectionLimit’ requests can be active at once (by default, this is two); after that we put the request onto a queue. Thanks to pipelining, a request can be ‘waiting’ without being ‘queued’. This happens when you make a request, and there’s a perfectly good connection to the server ready to be used, but it’s busy with another request. The new request will be pipelined with an earlier request and not queued (but it still has to wait for the earlier request to end).

The connection limit is trickier than you might realize because HttpWebRequests that are bound for different places but which share the same proxy share the same ServicePoint and therefore share the same set of connections.

The Queue performance counters are:
3. HttpWebRequests Queued/Sec
4. HttpWebRequests Average Queue Time

The Queued/Sec counter is updated whenever an item is put onto the queue; the Average Queue Time is updated whenever an item is removed from the queue. This means that it’s possible to see items being added to the queue like crazy even though the average queue time is zero – it could just mean that the server is slow and nothing is coming off of the queue.

Lastly, there are two counters to tell if your requests aren’t completing correctly.
5. HttpWebRequests Aborted/Sec
6. HttpWebRequests Failed/Sec

A request is Aborted when the ‘Abort()’ method is called. The Abort() method is automatically called by System.Net in some cases, all having to do with failures – when the request times out, for example, or sometimes when the request handler code triggers an exception. Your application code can also call Abort() on a request.

The most common reason that a request is Failed is that the final server response is a “could not fulfill your request.” For example, a server reply code of 404 Not Found is a failure. Technically speaking, the failed counter is updated when a request triggers an exception (other than exceptions thrown in your application code). This is a good definition of failed because all of the failed server codes will throw a WebException. Server codes that are not final – for example, a 401 Unauthorized return code will usually result in the System.Net code automatically retrying with proper credentials. In this case, the 401 Unauthorized is not final and does not trigger an exception.

That’s a lot: how about a diagram?
Certainly! Here’s a diagram that our developer created to help understand what all of the timings are. Each of the seven green circles represents one of the six performance counters (there are two ‘5’ items because 5 is the average lifetime, and there are two code paths that will affect that counter).

As promised, here’s a sample of my config file:

The performanceCounters value is documented on MSDN at http://msdn.microsoft.com/en-us/library/ms229151.aspx.

<?xml version="1.0" encoding="utf-8" ?>

<system.net>

</settings>

</system.net>

</configuration>

I also had to add in a “requiredRuntime” to my .config file: <startup><requiredRuntime version="v4.0" /></startup>. Otherwise, my application – which I compiled with an older compiler – wouldn’t use the 4.0 CLR.

What's new in System.Net.Mail

jetucker — Thu, 06 Aug 2009 23:10:00 GMT

What’s new in System.Net.Mail

We’ve made a number of enhancements to our SMTP support for .NET 4.0, mostly in the area of Unicode support and increased standards compliance, which is an important aspect in ensuring that legitimate emails do not get flagged as spam, as well as a few other useful features. In this post I’ll go over these enhancements and what they mean to you as a developer.

Decreased likelihood of being accidentally flagged as spam

Many spammers don’t bother to follow the SMTP protocol correctly in sending their spam, so standards violations tend to be a major factor in calculating spam scores. In .NET 4.0, we’ve made some significant improvements in how we encode headers and fold long lines of text in order to better comply with the SMTP protocol to reduce the likelihood that your legitimate emails will accidentally be caught by an overzealous spam filter. You don’t need to change a thing in your code to take advantage of this since it’s all internal to System.Net.Mail.

Increased Unicode support

You can now include Unicode in your custom headers thanks to a new property to the MailMessage class called HeadersEncoding, which allows you to specify what Unicode encoding that custom headers are using so that they can be sent correctly (the default is UTF-8). HeadersEncoding only affects custom headers that you add; the subject, body, and email address display names can have different encodings that are controlled by other properties in the MailMessage and MailAddress classes.

Clarification on setting header values

While MailMessage allows you direct access to the headers collection some headers being malformed could cause the email message to become corrupted. The following is a list of headers that you should not set via the headers collection; any values you set for these headers will be silently discarded or overwritten when the message is sent (use the appropriate properties of MailMessage to control these headers):

Bcc, Cc, Content-ID, Content-Location, Content-Transfer-Encoding, Content-Type, Date, From, Importance, MIME-Version, Priority, Reply-To, Sender, Subject, To, X-Priority

Note: if you do not specify an X-Sender header, we will create one for you however we will not overwrite an X-Sender that you set.

Multiple Reply-To addresses with the new property ReplyToList

We’ve added another new property to MailMessage called ReplyToList that allows you to add multiple Reply-To addresses to an email. The ReplyTo property has been obsoleted.

Content Disposition time zones

We’ve received some feedback on how ContentDisposition does not allow you to specify time zones in the character formats dictated by RFC 822, such as “GMT” or “PST.” This functionality was actually obsolete by RFC 2822, which replaces RFC 822, and dictates that an email time should always contain the offset for a timezone (e.g. “-0800” or “+0000”). However, we will now accept RFC 822 time zone formats and convert them to their semantically equivalent offsets. Whenever we are unable to determine the correct semantically equivalent offset, it is converted to “-0000” meaning that the time zone is unknown. This affects strings passed to the constructor or Parameters collection of ContentDisposition.

Mail Address formats

We’ve improved our ability to parse email address strings in the MailAddress class. Display names that contain Unicode cases and quoted local parts in the address will now be accepted correctly in most circumstances. If an address does contain an invalid character, the exception message will inform you of what character was invalid. Remember, you should format your email addresses as “displayname” <local@domain>.

Authentication

Some legacy clients do not advertise their AUTH mechanisms correctly. Normally, in the EHLO reply you would expect to see something like AUTH LOGIN NTLM advertising that this server supports LOGIN and NTLM authentication. Some legacy servers will instead reply with AUTH=LOGIN NTLM which is a violation of the RFC, however we have added support to SmtpClient for these legacy servers so authentication will work correctly should you encounter one.

We hope that you find this new and improved functionality useful for your applications and, as always, feel free to drop us a line if you have any feedback on these features or anything else you would like to see!

End-to-end connectivity with NAT traversal

Aaron Oneal [MSFT] — Mon, 27 Jul 2009 21:35:00 GMT

Like street numbers for a house, the Internet was originally designed so that all network devices could be directly addressed. Every connected device was given at least one unique identifier, or IP address, which could be used to route network packets to and from the device. For a while this worked well and devices had end-to-end connectivity.

IPv4 addresses can be used to route to about 4 billion (2³²) devices, but the rapid growth of the Internet quickly exhausted that available real estate. In the late 1980’s, several methods were developed to conserve this rapidly dwindling address space. Network isolation became a key strategy in this effort, and it had a beneficial side-effect – increased security. If an attacker was unable to address a device to directly establish connectivity, then attacking it was more difficult.

Home Routers with NAT

A common means for achieving this network isolation in the home is through an IPv4 router which supports Network Address Translation (NAT). These devices do some useful things for you:

They pick up a single, public IPv4 address from your Internet provider
They hand out IPv4 addresses to devices on your local network, most often through DHCP, drawing from a pool of the IPv4 address space reserved for private networks
They allow all of the devices on your network to share the single, public IPv4 address received from your Internet provider and handle the translation automatically from private to public address in conjunction with port information

The following diagram illustrates this kind of deployment for two private networks both connected to the Internet through a NAT.

With these features also come limitations. Note how both networks share the same private address space, both routers have the same 192.168.1.1 private IP address, and two different computers each have the same 192.168.1.20 private IP address. Devices on each private network can communicate within their own network, but how can they communicate with each other over the Internet?

Internet packets are routed using public addresses, and in the scenario above, each network only has one public IPv4 address. That means all packets go to the home router and it figures out whether to send them on to a device on the home network.

Here are 3 common strategies for achieving this routing between public and private networks:

Automatic translation can occur when a home device makes an outgoing network connection. This allows the NAT to match the destination address and port to the originating local address and port, then translate between them as packets are sent and received.
Manual configuration can be performed by an administrator where a given home router port can be redirected to a private device address and port. This allows communication from the public Internet to be directed to a private device by using the public IPv4 address of the home router in combination with an administrator defined port.
Automatic configuration can be performed by devices or applications behind the home router using UPnP. This allows a device or application to map ports on the router automatically so an administrator doesn’t have to. Most home routers support UPnP today. An unfortunate problem with this approach is that UPnP requires a specific port to be chosen at the time of registration, and this can cause collisions if two callers attempt to register the same port. UPnPv2 and NAT-PMP address this shortcoming.

NAT Traversal & Security Considerations

Because of the security boundary offered by NAT configurations, a key tenant for any traversal technology is to continue to maintain that security boundary for existing applications and services. The technologies discussed in this article adhere to that tenant. For a detailed discussion of this topic, you may refer to this informational on Security Concerns With IP Tunneling.

Impact to Applications

Generally these strategies result in a decent connectivity story, though complicated, and this carries over into application development.

In NAT situations, devices don’t really have end-to-end addressability by default, and the port begins to play an overdeveloped role in routing to your home devices to compensate for private addresses not being publicly reachable.

If your application only makes outgoing connections, the NAT solution will generally handle this transparently for you. The complications occur in applications that not only make but also receive connections – again, very common with peer-to-peer networks and video games.

If your application listens on a particular port for incoming connections and relies strictly on IPv4, you might ordinarily need to resort to one of the following techniques:

Add port mapping support to your application so it can automatically detect when it is behind a home router and configure the router to direct traffic to the application instance and port
Instruct your users to do this manually (which requires far more networking knowledge than is desirable)

A key point and drawback to keep in mind is that there is a 1:1 mapping between the Internet facing port on the router and the private IP and port pair of your local device.

If you want to run a web server from behind your NAT, you might allocate port 80 on the NAT and have it direct traffic to port 80 on Computer 1. If you wanted Computer 2 on the same network to also host a website available on the Internet, a different port on the NAT would need to be allocated, such as port 81 since port 80 is already in use and mapped to Computer 1.

Visitors would then need to use the router’s public IP address combined with a distinct port to reach the appropriate web server. This is not ideal since web visitors may not be accustomed to having to specify a specific, non-default port.

Contention for well-known ports demonstrates just one of the problems that may be encountered when relying on port mapping instead of an IP address with enough specificity to more fully handle addressing.

With that in mind, we’ve made many of these complexities transparent for you in the .NET Framework 4.0.

If you’re using TcpListener or UdpClient, just pass into the constructor IPAddress.IPv6Any, then call AllowNatTraversal with a value of true.

var listener = new TcpListener(IPAddress.IPv6Any, 8000);
listener.AllowNatTraversal(true);
listener.Start();

You’ll notice we have been discussing IPv4, but the example above mentions IPv6. Have no fear, this will work over intermediate IPv4 networks, it just relies on the origin or destination endpoints supporting IPv6 and a couple key technologies which I’ll cover in more detail below. If your application must run on a PC which only has IPv4 installed, a condition that is becoming more rare, then unfortunately this new solution won’t help you right now.

Another point worth mentioning is that applications which wish to listen on all IP addresses today have to set up two listeners, one for IPv4 using IPAddress.Any and one using IPAddress.IPv6Any. We’re investigating ways to take advantage of “dual mode” sockets so you won’t need to worry about tying your application to a specific IP version in the future.

IPv6 Restores Addressability

What if all your network devices had a public IP address which could be used by other devices to directly communicate with them? This would eliminate the need to use NAT for addressing and once again achieve end-to-end connectivity. With IPv6, this is possible.

IPv6 has a significantly larger address space (2¹²⁸ or about 3.4×10³⁸), and with an address space this size, it is once again possible for every device connected to the Internet to be given a unique address.

But, there’s a problem. Much of the world is still using IPv4, so exactly how can your application take advantage of IPv6 addresses today?

Fortunately, a number of transparent solutions have been devised and implemented on platforms and devices so that your application doesn’t need to worry with the specifics so long as it listens on all available IP addresses.

6to4 Tunneling

Although applications generally don’t need to worry about the underlying mechanism used to allocate an IPv6 address, for context, one such solution is 6to4 tunneling.

This solution works great for devices that already have a public IPv4 address. A special range of IPv6 maps to the IPv4 address space, and so with a 6to4 tunnel gateway deployed at the edge, IPv6 connectivity can be automatically enabled.

Windows supports 6to4, so if your computer already has a public IPv4 address, or you take your laptop to a hotspot that assigns it one, it probably also has a public IPv6 address through the 6to4 pseudo adapter.

Of course, in the NAT scenario we’ve been discussing, only the home router has a public IPv4 address. So, to take advantage of 6to4, your router would need to be IPv6 compatible, it would need to be able to assign IPv6 addresses to the local network, and and it would need to have 6to4 tunneling built into it.

Presently, most home routers don’t support this, though it is something a Windows Server can be configured to do as can standard Windows versions supporting Internet Connection Sharing (ICS).

So if a home router holds the one and only public IPv4 address, and 6to4 isn’t an option, how can computers behind the NAT use IPv6?

Teredo Tunneling

Like 6to4, Teredo is another IPv6 transition technology which brings IPv6 connectivity to IPv4 networks. It is described by RFC 4380, is further extended by MS-TERE, and has even been implemented by the open source community as Miredo.

There are a number of technical articles on exactly how Teredo encapsulates IPv6 over IPv4/UDP, which most NATs can forward, so if you’re interested in those details you can find out more from the links at the end of the article.

What’s important to know here is that Windows OS versions starting with Windows XP SP2 and Windows Server 2003 provide a virtual Teredo adapter which can give you a public IPv6 address in the range 2001:0::/32. This address can be used to listen for incoming connections from the Internet and can be provided to IPv6 enabled clients that wish to connect to your listening service.

Teredo and related transition technologies free your application from worrying about how to address a computer behind a home router or NAT since typically all you need to do is connect to it using its IPv6 addresses.

New for .NET 4.0

We’re making some additions in the .NET Framework 4.0 starting with Beta 2 to make these great technologies easier for you to use. The .NET Framework client APIs already support address-based NAT traversal, so the main updates are for listeners.

Listeners

The TcpListener and UdpClient changes were previously mentioned. Use these new methods to toggle whether your application explicitly wants to allow or restrict NAT traversal support.

public class TcpListener
{
public void AllowNatTraversal(bool);
}

public class UdpClient
{
public void AllowNatTraversal(bool);
}

Sockets

If you’re using sockets directly, the interface is a little closer to what Winsock exposes via the IPV6_PROTECTION_LEVEL socket option. We have exposed this on the Socket class as a new method called SetIPProtectionLevel.

public class Socket
{
public void SetIPProtectionLevel
(IPProtectionLevel);
}

public enum IPProtectionLevel
{
    Unspecified = –1,    // platform default
    Unrestricted = 10,   // global with NAT traversal
    EdgeRestricted = 20, // global without NAT traversal
    Restricted = 30,     // site local
}

Set the IPProtectionLevel to Unrestricted prior to Bind to allow clients to connect to your listener deployed behind a NAT. This is what the System.Net listener implementations do when you invoke the previously mentioned AllowNatTraversal method.

EdgeRestricted allows clients to only connect to your listener on IP addresses that aren’t used for NAT traversal (like Teredo).

Restricted only allows intranet connectivity (site and local link).

The actual default setting is Unspecified and left to the underlying platform to determine.

Starting with Windows Vista, this is equivalent to Unrestricted when the Windows Firewall is enabled and an appropriate rule is configured per the instructions below and EdgeRestricted when it is disabled.

This honors that security point mentioned at the beginning of the article. Many IPv4 networks rely on NAT as a limited form of protection. The default setting protects applications from unintentionally exposing themselves to the Internet in NAT scenarios. Instead, applications must explicitly opt-in to NAT traversal using this socket option or by configuring a Windows Firewall rule.

Configuration

To enable you to easily turn these features on for your existing applications, you can also control this through app.config.

<system.net>
<settings>
    <!-- default is platform defined (Unspecified) –>
    <socket ipProtectionLevel="Unrestricted |
      EdgeRestricted | Restricted | Unspecified"/>
</settings>
</system.net>

This setting will affect all listening sockets in an AppDomain.

Address Discovery

It’s not recommended to implement behavior that relies on direct knowledge of IP addresses since this would typically be handled through name resolution, but in some cases, like building peer to peer applications or your own discovery service, it can be useful.

To get the list of addresses on a host, you could do it the traditional way using System.Net.NetworkInformation to enumerate NetworkInterfaces and their addresses, but we’re adding some new methods which make this simpler and also “wake up” Teredo if it hasn’t been used recently.

public class IPGlobalProperties
{
public UnicastIPAddressInformationCollection
    GetUnicastAddresses();

public IAsyncResult BeginGetUnicastAddresses
    (AsyncCallback, object);

public UnicastIPAddressInformationCollection
    EndGetUnicastAddresses(IAsyncResult);
}

This allows you to enumerate addresses as follows.

var addressInfoCollection =
IPGlobalProperties.GetIPGlobalProperties()
.GetUnicastAddresses();

foreach(var addressInfo in addressInfoCollection)
{
Console.WriteLine("Address: {0}",
addressInfo.Address);
}

Once you have acquired this list of addresses, you can give the address list to your clients out of band and they can use Socket.Connect, TcpClient.Connect, or even WebRequest.Create to establish a connection to your service.

For a savvy way to publish your addresses so others can discover them, check out our Peer Name Resolution Protocol (PNRP). Another great discovery mechanism is the Collaboration API. The traditional mechanism is of course to use DNS and the System.Net APIs which accept a host name.

Finally, we’re also adding a convenient property to the IPAddress class so you can tell if an address you’re dealing with is an IPv6 Teredo address.

public class IPAddress
{
public bool IsIPv6Teredo { get; }
}

This property is primarily intended to be used for debugging and test scenarios since you will typically want to listen on all available IP addresses so your application can automatically take advantage of new platform enhancements.

Right now, client support works transparently the whole way up the transport stack, and for listeners we support these new features with Sockets, TcpListener, and UdpClient. We hope to extend this to HttpListener in the future.

Windows Communication Foundation

WCF supports Teredo today for TCP channels when using the NetTcpSection.TeredoEnabled and TcpTransportElement.TeredoEnabled properties.

Firewall Considerations

By default, the Windows Firewall blocks incoming connections. For your listener to be accessible, you will want to create a firewall rule. This is true of any listening application, not just ones that wish to take advantage of NAT traversal. You can do this programmatically, and since adding firewall rules requires UAC elevation, application installation is the best time to do this. Even though the rule can be added at install time, it can be configured to activate only while the application is running.

The firewall can be controlled using COM interop. Add a project reference to FirewallApi.dll. You can then add a rule with the following code.

Guid netFwRuleUuid = new Guid("{2C5BC43E-3369-4C33-AB0C-BE9469677AF4}");
INetFwRule rule = (INetFwRule)Activator.CreateInstance(Type.GetTypeFromCLSID(netFwRuleUuid));

rule.Action = NET_FW_ACTION_.NET_FW_ACTION_ALLOW;
rule.ApplicationName = @"C:\Program Files\My Application\MyApplication.exe";
rule.Description = "My Rule Description";
rule.Direction = NET_FW_RULE_DIRECTION_.NET_FW_RULE_DIR_IN;
rule.EdgeTraversal = true;
rule.Enabled = true;
rule.Grouping = "My Rule Group";
rule.Name = "My Rule Name";
rule.Protocol = (int)ProtocolType.Tcp;

Guid netFwPolicy2Uuid = new Guid("{E2B3C97F-6AE1-41AC-817A-F6F92166D7DD}");
INetFwPolicy2 policy = (INetFwPolicy2)Activator.CreateInstance(Type.GetTypeFromCLSID(netFwPolicy2Uuid));
policy.Rules.Add(rule);

Note that to enable NAT traversal, the EdgeTraversal flag must be set to true since a firewall rule with a setting of false (the default) is used to prevent NAT traversal even when an application sets the IPProtectionLevel for its sockets to Unrestricted.

Starting with Windows 7, although you can still use the approach above, there is more control over the default NAT traversal behavior using the InetFwRule2 interface and NET_FW_EDGE_TRAVERSAL_TYPE.

Third party firewalls may require custom configuration or may prompt the user for permission on first application launch.

Parting Thoughts

With IPv6 and its related transition technologies becoming ubiquitous, now is a great time to take advantage of these capabilities in your applications, and the .NET Framework 4.0 makes it easy to get started.

Additional References

Microsoft IPv6 Website
IPv6 Transition Technologies
TechNet Teredo Overview
TechNet Using IPv6 and Teredo
MSDN Teredo Site
Firewall requirements for coexisting with Teredo
Security Concerns With IP Tunneling

Special thanks to Dave Thaler for his insights and expert feedback.

~ Aaron Oneal | NCL Program Manager

New NCL Features in .NET 4.0 Beta 2

Aaron Oneal [MSFT] — Mon, 20 Jul 2009 18:59:00 GMT

We’re introducing some new features starting with .NET 4.0 Beta 2 that you may find useful. Additional information will be available on MSDN and in subsequent articles. If you have any questions or comments, let us know!

Sockets

DnsEndPoint

This feature was first introduced in Silverlight 2 and it allows you to connect to a listening socket by DNS name, simplifying the connect experience. Now you can take advantage of this in .NET as well.

Before

IPAddress[] IPs = Dns.GetHostAddresses("www.contoso.com");

foreach(IPAddress ip in IPs) {
try {
socket.Connect(new IPEndPoint(ip, port));
break;
}
catch(SocketException) { continue; }

}

After

socket.Connect(new DnsEndPoint("www.contoso.com", port));

IP Version Neutrality

Starting with Windows Vista, the TCP/IP stack has features which allow you to write sockets applications that can transparently handle multiple versions of the Internet Protocol. This works by creating an IPv6 socket and setting the IPv6Only socket option to false. Once set, the IPv6 socket is also able to accept IPv4 traffic. This frees your application from the traditional model of having to create multiple sockets for each IP version.

Socket socket = new Socket(AddressFamily.InterNetworkV6, SocketType.Stream, ProtocolType.Tcp);

socket.SetSocketOption(SocketOptionLevel.IPv6, SocketOptionName.IPv6Only, 0);

socket.Bind(new IPEndPoint(IPAddress.IPv6Any, 8000));
socket.Listen(5);
Socket client = sock.Accept();

SSL Encryption Policy

Traditionally SSL is used both for encryption and for authentication / message integrity. In some cases, however, encryption is not required or desired. This could be for interoperability with other systems or performance reasons. Therefore, you can now change the default encryption policy for SSL. In some circles, this is also referred to as using a “null cipher”.

namespace System.Net.Security
{
public enum EncryptionPolicy
{
    RequireEncryption, // always require encryption
    AllowNoEncryption, // prefer full encryption; allow none if server agrees
    NoEncryption       // require no encryption; fail if server requires it
}
}

This setting can be configured in the following places.

SslStream
ServicePointManager for HttpWebRequest, FtpWebRequest, SmtpClient
Machine.config or app.config

SmtpClient

SSL Configuration

It is now possible to enable explicit SSL in SmtpClient via application config in addition to the runtime option that was already available.

Header Encoding

Prior versions of .NET restricted headers to ASCII encoding. A new MailMessage.HeadersEncoding property will allow for an alternate encoding to be specified for the headers added to the MailMessage.Headers collection.

Multiple ReplyTo Addresses

The MailMessage.ReplyTo property only allows a single address to be added. We’re adding a new collection property called MailMessage.ReplyToList to support multiple addresses.

HttpWebRequest

64-bit Range Header Support

When a client program makes an HTTP request, that request includes a number of headers which determine how the server will respond. One of those headers is the Range header as described in RFC 2616 (obsoletes 2068). The Range header on a request allows a client to request that it only wants to receive some part of the specified range of bytes in an HTTP entity. Servers are not required to support Range header requests.

An example of a range header in the HTTP request is:

Range: bytes=1024-2047

Some of the uses of the range header include:

1. Manually maintaining “QOS” (quality of service) for streaming media. For example, a sophisticated movie viewer program might download the video and audio of a movie separately, downloading the “next” pieces only when needed

2. Manually handling file caching for interrupted downloads

The existing framework includes a set of public HttpWebRequest.AddRange methods as specified on MSDN at http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.addrange.aspx.

Additional overloads supporting Int64 ranges have been added.

Date Header Support

A new HttpWebRequest.Date property has been added to support setting of the HTTP Date header.

Host Header Support

A new HttpWebRequest.Host property has been added to allow configuration of the HTTP Host header. This header can be used during testing and with load balancers, for example, to ensure the TCP connection is established to a particular IP address but that the client can still address the intended web site at the specified HTTP server endpoint. For example:

var request = WebRequest.Create("http://127.0.0.1/") as HttpWebRequest;
request.Host = "contoso.com";
var response = request.GetResponse();

WebRequest Performance Counters

Troubleshooting a server application can be difficult, and one key tool for this is the Windows Performance Monitor. With the addition of new performance counters for WebRequest, this is now easier. The diagram below (click for a larger view) shows the new counters and the points in the lifetime of an HttpWebRequest where they are updated.

NAT Traversal

The term NAT traversal refers to the ability for client devices to address and communicate with listening devices behind a NAT. This turns out to be an incredibly useful thing to do for games, peer-to-peer, and a variety of other applications. A subsequent article will go into full detail about what is available, but here’s a quick preview of one aspect of this feature.

If you’re using TcpListener or UdpClient, just pass into the constructor IPAddress.IPv6Any, then call AllowNatTraversal with a value of true to support NAT traversal in your application.

var listener = new TcpListener(IPAddress.IPv6Any, 8000);
listener.AllowNatTraversal(true);
listener.Start();

Escaped Character Support in Uri

With REST becoming a more popular way of creating web services, and with the key role Uri plays in resource identification, we are providing a way to relax some of the canonicalization performed by our current http(s) scheme parser. If you have encountered issues with special characters such as / or \ being unescaped, you now have the following configuration options at your disposal.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<section name="uri" type="System.Configuration.UriSection, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
</configSections>

<system.net>
    <settings>
      <httpListener unescapeRequestUrl="false" />
    </settings>
</system.net>
</configuration>

This will allow you to turn off this form of escaping in Uri and also in the RequestUrl received from HttpListener. The latter setting will also impact application hosted WCF services using HTTP bindings since those are built on HttpListener.

Bug Fixes

No release would be complete without bug fixes, and we’re addressing plenty of those! The NCL team has fixed over 150 bugs as of this milestone. Our focus during triage was on:

Stability (fix hangs, crashes and improve long-running/data center scenarios such as fix memory leaks)
Performance
RFC compliance with URI, FTP, HTTP, SMTP
Internationalization (better handling of Unicode characters, etc.) in URI, FTP, HTTP, SMTP
IPv6 connectivity
Customer reported bugs through our Connect, MSDN Forums, and other channels

Please try out the new release when available and let us know what you think!

~ NCL Team

Why does Silverlight have a restricted port range for Sockets?

Aaron Oneal [MSFT] — Tue, 23 Jun 2009 22:52:00 GMT

Silverlight restricts the ports of outgoing TCP socket connections to the range 4502 – 4534. Connecting to a different port requires the use of a server-side proxy or port redirector.

One of the most common questions we hear from customers about this is, “Why do you restrict the port range in Silverlight? It doesn’t add any extra security.”

Actually, it does. The short explanation is, it gives network administrators control over their infrastructure by providing a convenient way to distinguish and route Silverlight traffic. For the long answer, read on.

Desktop trust model

When you run an application on your desktop you typically:

Have intentionally downloaded and/or installed the application
Have intentionally executed the application

In the case of a managed corporate environment you:

Have been granted permission by an IT administrator to install and/or execute applications on your PC

In short, by being able to install and explicitly execute an application, you are asserting your trust of that application to not go rummaging around your file system or corporate network, for instance.

Web trust model

The web trust model is different. A web browser is a trusted desktop application, so per above, there are expectations it will not do anything malicious. Since web content can come from anywhere, there are no security guarantees about the intentions of the content provider.

Moreover, the explicitness of application install and execution is not present by design. That is, just by navigating to a website, a number of Silverlight applications could be started – an advertisement playing in the corner of the page, a hidden application with no UI, etc.

None of these Silverlight applications should be able to break out of the “sandbox” trust model without your knowledge, and nothing short of application signing, a domain trust model, prompting, etc. could establish that trust.

We’ve worked hard to keep the experience as unobtrusive as possible by generally avoiding prompting. But even when necessary, such trust models are fragile in nature because there is such a tendency to just click OK when you’re on a trusted site, even if the content came from elsewhere.

User vs. IT admin security decision

The other consideration is whether the decision to trust a website or Silverlight application rests with the web browser user or with the IT administrator. An insecure client on the network can be an entry point to other normally secured systems.

Here is one well-known FTP attack for why a trust decision like this is necessary and why it needs to rest with the IT admin.

The FTP protocol has a PORT command which is typically used to establish connections using “active” FTP. It can also be used to initiate server-to-server transfers.

In a malicious case, the command can be exploited to perform port scanning, or in the case of some active packet filtering devices, to actually open ports in the firewall.

With a desktop FTP client application that the user or IT admin has installed, the behavior of that client is trusted to be benign and these commands are issued at the request of the user. The active packet filter is doing what it was configured to do by opening the necessary ports to allow the connection.

Now imagine if Silverlight were allowed to send those same commands. You visit a website, a hidden application sets up a TCP connection back to its server of origin, and then it sends the PORT command which promptly opens a hole in your firewall. The website then uses this open port to establish a connection back to a victim machine on the internal network. The user never intended this action and therefore the trust model is broken. Moreover, the entire network has been placed at risk since the connection could be to a different computer than the user’s who indicated trust of the application.

Now, such attacks can typically be mitigated through additional configuration and patching, but this class of attack tends to re-surface with various protocols because of the liberties active filters take.

Similar exploits exist for HTTP. You’ll notice that Silverlight, Flash, XmlHttpRequest, etc. block a number of request headers. Now imagine if we allowed TCP connections over port 80 and a malicious application could craft their own HTTP request, effectively bypassing our HTTP implementation which has these checks.

For a detailed look at one such threat, please see this article about CERT’s VU#435052. Security researcher Dan Kaminsky has published a presentation and paper on the subject.

In a corporate environment, IT administrators need to be able to secure their networks against such attacks, so there must be a way for them to retain control of these security decisions and to also be able to distinguish Silverlight traffic from trusted application traffic that might be using similar protocols.

Port ranges and transparent proxy abuse

So how do port ranges help? Well, it’s unlikely your active packet filter scanning FTP port 21 is going to notice if someone attempts to send a maliciously crafted command over port 4502. Moreover, with a clearly identified port range of Silverlight-only traffic, it’s easy to configure such filtering devices to handle that traffic differently and with an appropriate level of trust.

Conclusion

For the time being, we’ve settled on the port range restrictions as the best compromise to maintain security and protect our customers. We understand this makes interoperability and connectivity challenging for some deployments, but we are planning to address that feedback with future work in this area.

I hope this helps to clear up some of the questions out there about this, and if you have more, please let us know. Thanks!

Custom HTTP Authentication Schemes

chrross — Tue, 05 May 2009 21:06:00 GMT

Introduction

My name is Chris Ross and I work as a developer for Microsoft’s .NET Framework networking components. As part of the Network Class Library (NCL) team I get lots of networking questions from other developers. This post resulted from my research into a question about using custom HTTP authentication schemes. The custom scheme in question was Google’s GoogleLogin scheme, so I’ll use it as an example to show how System.Net.WebRequest and System.Net.WebClient can work with custom schemes. Fist though, I should explain how standard HTTP authentication works.

Standard HTTP Authentication

It has become increasingly important to secure online information from unauthorized users. But once secured, how do clients figure out how to login? Well, the HTTP protocol utilizes a back and forth negotiation pattern to let the client know that something is secured and what type of credentials they must submit to gain access.

A common HTTP authentication pattern goes as follows:

WebRequest.Credentials = new NetworkCredential(username, password);
WebRequest sent (without Credentials)	To	Secured web page
Error 401 – List of login schemes supported (Basic, Digest…)	From	Secured web page
WebRequest +Credentials (formatted according to the selected login scheme) sent	To	Secured web page
200 – Success – Here is your secured content	From	Secured web page

The best part is that .NET handles this back and forth for you under the hood, so all you end up seeing is the content you asked for. The .NET Framework (WebClient & WebRequest) has built in support for Basic, Digest, NTLM, Kerberos, and Negotiate authentication schemes.

Here is an example on how you would use standard authentication schemes with WebRequest:

public static void NormalHTTPAuthExample(String username, String password)

{

NetworkCredential creds = new NetworkCredential(username, password);

WebRequest request = WebRequest.Create("http://www.contoso.com/");

request.Credentials = creds;

// Send the request and process the response

WebResponse response = request.GetResponse();

StreamReader responseStreamReader =

new StreamReader(response.GetResponseStream());

String result = responseStreamReader.ReadToEnd();

responseStreamReader.Close();

Console.WriteLine(result);

}

WebRequest Authentication Practices

WebRequest takes great care with the credentials you provided. Many of these precautions are to help prevent your username and password from being exposed to unauthorized servers. Two features related to our example are CredentialCache and PreAuthenticate.

WebRequest by default facilitates automatic request redirection from site to site. However if a server you have logged into decides to redirect you elsewhere, WebRequest will remove its Credentials property before automatically redirecting to prevent your username and password from being exposed. If you are aware that redirects will take place, you can use the CredentialCache class to manage what servers are allowed access to your credentials. A CredentialCache pairs credentials with the URIs and authentication schemes they’re allowed to be used with. When you have created and configured a CredentialCache object, you then assign it to the WebRequest.Credentials property. You can see this in the next code sample. A CredentialCache will not be removed from the Credentials property when redirecting because WebRequest knows where you will allow your credentials sent. You may also reuse your cache by assigning it to subsequent requests. All of this also applies to WebClient.Credentials, as it uses WebRequest for its underlying operations.

A second WebRequest property that can be useful with HTTP authentication is PreAuthenticate. Without it, WebRequest will not include your credentials on a request until it has first received a challenge from the server. Setting this property is optional but will help boost performance on secured sites as follows. After you’ve successfully authenticated once, this flag causes your credentials to be included in the Authorization header automatically on subsequent requests and redirects that match the same URI path at the folder level. This way the server does not have to issue a challenge for authentication on each page. WebClient does not expose this property.

Here is a code sample that shows both of these properties:

public static void RedirectHTTPAuthExample(String username, String password)

{

NetworkCredential creds = new NetworkCredential(username, password);

CredentialCache credCache = new CredentialCache();

// Cached credentials can only be used when requested by

// specific URLs and authorization schemes

credCache.Add(new Uri("http://www.contoso.com/"), "Basic", creds);

WebRequest request = WebRequest.Create("http://www.contoso.com/");

// Must be a cache, basic credentials are cleared on redirect

request.Credentials = credCache;

request.PreAuthenticate = true; // More efficient, but optional

WebResponse response = request.GetResponse();

StreamReader responseStreamReader =

new StreamReader(response.GetResponseStream());

String result = responseStreamReader.ReadToEnd();

responseStreamReader.Close();

Console.WriteLine(result);

}

Custom HTTP Authentication Schemes

What if the sever you’re requesting information from does not utilize one of the standard authentication schemes? Take for an example Google’s custom GoogleLogin authentication scheme. I’m no expert on Google’s APIs, but its differences from standard HTTP authentication schemes make for a good example. GoogleLogin requires first making a request to a completely different login server for an Auth token, then adding that token to every subsequent request’s Authorization header. You can still do this with .NET, but because GoogleLogin is a custom scheme you might have to handle some of the back and forth yourself, such as turning off automatic redirects so you can be sure to include the Auth token on each request.

For example, you would first make a request to the login server for your user token. Then you’d add that token to a second request to the Calendar server. The Calendar server responds with a redirect to your specific calendar at a session specific url. You then issue one final request (again including the Auth token) to that session specific url for your calendar data.

WebRequest (with user name and password) sent	To	Login server
Auth token	From	Login server
WebRequest + Auth token	To	Secured Calendar server
302 – Redirect to session specific URL	From	Secured Calendar server
WebRequest + Auth token sent to new URL	To	Your Secured Calendar
200 – Success – Here is your secured content	From	Your Secured Calendar

While this approach can work well enough to use once, I find it unwieldy to make three different requests where only one would normally be required, especially if you needed to do it often.

Luckily .NET provides a better answer to this sort of problem: Custom Authentication Modules. You can add your own modules to the list of supported authentication schemes for WebClient & WebRequest, and let .NET juggle the authentication, redirection, proxies, and other complications for you. Below is an example of how to do this for Google’s custom GoogleLogin scheme.

How to use a custom module with WebRequest/WebClient

How do we use WebRequest and a custom authentication module to log in to Google services? Here is a sample:

public static void GoogleAuthManagerExample(String username, String password)

{

AuthenticationManager.Register(new GoogleLoginClient());

// Now 'GoogleLogin' is a recognized authentication scheme

GoogleCredentials creds = // user@gmail.com

new GoogleCredentials(username, password, "HOSTED_OR_GOOGLE");

CredentialCache credCache = new CredentialCache();

// Cached credentials can only be used when requested by

// specific URLs and authorization schemes

credCache.Add(new Uri(

"http://www.google.com/calendar/feeds/default/private/"),

"GoogleLogin", creds);

WebRequest request = WebRequest.Create(

"http://www.google.com/calendar/feeds/default/private/full?q=null+item");

// Must be a cache, basic credentials are cleared on redirect

request.Credentials = credCache;

request.PreAuthenticate = true; // More efficient, but optional

WebResponse response = request.GetResponse();

StreamReader responseStreamReader =

new StreamReader(response.GetResponseStream());

String result = responseStreamReader.ReadToEnd();

responseStreamReader.Close();

Console.WriteLine(result);

// Erase cached auth token unless you'll use it again soon.

creds.PrevAuth = null;

}

The only meaningful difference between this code block and the previous authentication example I showed is the registration of the GoogleLoginClient. The GoogleLoginClient is a custom authentication module that WebRequest can refer to for handling the special symantics of Google’s API. As I described before, the alternative is to include much of the folowing Google API code inline, as well as managing all of your own redirection. I definitely prefer the custom modules, so lets look how to implement this custom authentication module.

How to write a custom authentication module for GoogleLogin

The first minor element is a storage place for Google’s Auth token and login parameters. Caching the token is ok so long as you’re careful where you put it and you know when you need to clear it.

public class GoogleCredentials : NetworkCredential {

private Authorization prevAuth = null;

public Authorization PrevAuth { // Cached login token

get { return prevAuth; }

set { prevAuth = value; }

}

private String accountType;

public String AccountType {

get { return accountType; }

// Validate "GOOGLE","HOSTED", or "HOSTED_OR_GOOGLE"

set { accountType = value; }

}

public GoogleCredentials(String user, String pswd,

String accountType)

: base(user, pswd) {

this.AccountType = accountType;

}

Then we need to implement the IAuthenticationModule interface. When we are challenged for GoogleLogin authorization this will do the actual request to Google’s login server and fetch us an Auth token.

public class GoogleLoginClient : IAuthenticationModule {

internal const string AuthType = "GoogleLogin"; // Scheme identifier

internal static string AuthServer

= "https://www.google.com/accounts/ClientLogin";

public String ServiceString = "cl"; // Calendar

public String Source = "MSTest"; // Our program name

public Authorization Authenticate(string challenge,

WebRequest webRequest, ICredentials credentials) {

// Careful, if your challenge contains more than one

// authorization scheme, this one might not be first

// in the list. Also ignore parameter names and quoted

// parameter values. See RFC 2617 Section 1.2

// ie: Basic, Digest nonce=122352354,

// realm="www.GoogleLoginDirections.com/help",

// GoogleLogin realm="http://login.google.com/",Ntlm

if (!challenge.Contains(AuthType)

/* && ContainsNotInQuotes(challenege,AuthType)

* && MoreValidation(challenege,AuthType) */) {

return null;

}

return Login(webRequest, credentials);

}

public Authorization PreAuthenticate(WebRequest webRequest,

ICredentials credentials) {

return Login(webRequest, credentials);

}

public bool CanPreAuthenticate {

// Some schemes do not support PreAuthentication

get { return true; }

}

public string AuthenticationType {

get { return AuthType; }

}

private Authorization Login(WebRequest webRequest,

ICredentials credentials) {

// Do we have credentials for this site?

NetworkCredential NC = credentials.GetCredential(

webRequest.RequestUri, AuthType);

GoogleCredentials gcreds = NC as GoogleCredentials;

if (gcreds == null)

return null; // none or wrong type of credentials

if (gcreds.PrevAuth != null)

return gcreds.PrevAuth; // Cached from last login

ICredentialPolicy policy =

AuthenticationManager.CredentialPolicy;

if (policy != null && !policy.ShouldSendCredential(

webRequest.RequestUri, webRequest, NC, this))

return null;

WebRequest client = WebRequest.Create(AuthServer);

client.ContentType = "application/x-www-form-urlencoded";

client.Method = "POST";

// Custom authentication string:

// http://code.google.com/apis/accounts/docs/AuthForInstalledApps.html

String requestParams = "accountType=" + gcreds.AccountType

+ "&Email=" + gcreds.UserName + "&Passwd=" + gcreds.Password

+ "&service=" + ServiceString + "&source=" + Source;

byte[] bytes = Encoding.UTF8.GetBytes(requestParams);

// Google's API says that the custom authentication string

// goes in the body of this request. This is unusual.

Stream requestStream = client.GetRequestStream();

requestStream.Write(bytes, 0, bytes.Length);

requestStream.Close();

// The Auth token comes in the response body. Also unusual.

WebResponse response = client.GetResponse();

StreamReader responseStreamReader =

new StreamReader(response.GetResponseStream());

String result = responseStreamReader.ReadToEnd();

responseStreamReader.Close();

String authToken = ""; // Parse out the Auth token

String[] tokens = result.Split(new String[] { "\n" },

StringSplitOptions.None);

foreach (String token in tokens) {

if (token.StartsWith("Auth=",

StringComparison.OrdinalIgnoreCase)) {

authToken = token;

break;

}

if (authToken == "")

throw new WebException("GoogleLogin authentication failed");

// Assemble the Authorization header and cache it

gcreds.PrevAuth = new Authorization(AuthType + " " + authToken);

return gcreds.PrevAuth;

}

And you’re done. Now any time your application is challenged for GoogleLogin authentication, WebRequest can just refer to this new module automatically.

Conclusions

HTTP authentication can take many forms, and .NET includes support for several standard schemes. When these are not sufficient, it is easy to add custom schemes for many other services with minimal changes to your existing code.

Notes & references

- WebClient: http://msdn.microsoft.com/en-us/library/system.net.webclient.aspx

- WebRequest: http://msdn.microsoft.com/en-us/library/system.net.webrequest.aspx

- IAthenticationModule: http://msdn.microsoft.com/en-us/library/system.net.iauthenticationmodule.aspx

- Register a new custom module in the App.Config file without modifying existing code: http://msdn.microsoft.com/en-us/library/y9b82x09.aspx

- CredentialCache: http://msdn.microsoft.com/en-us/library/system.net.credentialcache.aspx

- GoogleLogin API: http://code.google.com/apis/accounts/docs/AuthForInstalledApps.html

- RFC 2617 – HTTP Basic & Digest authentication - http://www.ietf.org/rfc/rfc2617.txt

- RFC 2616 – HTTP 1.1 - http://www.ietf.org/rfc/rfc2616.txt

- VB example: http://support.microsoft.com/default.aspx/kb/331501

~Chris Ross

System.Net

Functional Areas of the Network Class Library

Aaron Oneal [MSFT] — Thu, 26 Mar 2009 01:51:28 GMT

If you are wondering exactly what is in the NCL, the diagram below should help. It’s a high-level roadmap of the key feature areas and components.

As you can see, the NCL is a roll up of the System.Net.* namespaces and the URI related classes in System.

Networking features include sockets, an HTTP client, an HTTP listener, an SMTP client, network adapter information, and peer-to-peer connectivity. Future articles will cover these areas in more detail.

Would you like to see more features added? Add a suggestion using Microsoft Connect and tell us more about your scenarios! Be sure to mention NCL/System.Net so it can be routed to our team.

Code Patterns for Online/Offline Network Detection

PeSmith — Tue, 24 Mar 2009 03:17:00 GMT

In the last entry I showed you why you should use the NetworkChange.NetworkAddressChanged API to control when to check if your application is online or not. If you didn’t read that article, go do it now – it will help you understand why online detection is hard, and why the network detection code is made the way it is.

As with all code, there are certain code patterns you can use to organize your online detection code. Using the patterns means you are more likely to write good code that works well; ignoring the patterns means you are more likely to write hard-to-maintain code that doesn’t quite work well enough. What are the good code patterns for online detection? It turns out that there are at least three. Like any good blogger, I’ve got clever names for all them:

A is for Ask the User.

B is for Before you Connect, Validate

C is for Constantly Check Results

Ask the User is good because sometimes users have information that you do not – for example, that an upstream router is back on line, or that they have entered (into another window) some token or bit of authorization. They also know that they are on an automatically-dialed connection, and that they pay by the minute, and that even if your program could connect, they really don’t want it to connect right now. (There are always stories nowadays of people who accidently turned on “international roaming” and were charged thousands of dollars of fees)

Before you Connect, Validate is simple: you check a known file on your server before any other activity. You have to check the file when your application starts and when you get a callback from NetworkAddressChanged. Do not forget that you have to check the contents of the file, not just the status code! There are several tricks to make sure that the file is truly from your server and not from a cache. First, it helps to add in a query on the URL that points to the file. The query should be different each time (a GUID, perhaps, or a random number), and the file should include the query parameter. That way you can be fairly confident that there aren’t any caches trying to help you when you really don’t want them to.

Constantly Check Results means that you can try to download a file or resource any time – but that each time you do, check the results to make sure that it’s really a file from your server. After getting a valid response that isn’t from your server, you must be in something like the “Airport Coffee Shop” case – someone is returning data as if they are your server, but it’s not. In this case, your application might start queuing up all of the network requests and then retry them when the network might be available (which, as always, you know because you get a callback from NetworkAddressChanged).

Each of these methods has advantages and disadvantages. Asking the user all the time, for example, is a big thing to ask. However, please always consider this on a setup or preferences page. Some users really do have to pay for bandwidth and will not be pleased with any application that goes onto the network without permission.

The before you connect, validate code pattern is simple, but requires that you put a known file on your server. It’s also a bit slower to detect network availability because it will also check before allowing an actual file to be transmitted. Lastly, the network might change faster than the code responds, resulting in bad data being accepted as good data.

The constantly check results pattern is more complicated to implement, and requires that you be able to validate each possible result – and that may not be possible. On the other hand, it’s recovers faster and generates less network traffic.

MSDN documentation for the NetworkChange.NetworkAddressChanged event is at http://msdn.microsoft.com/en-us/library/system.net.networkinformation.networkchange.networkaddresschanged.aspx

I gave a talk at Mix '09 about network detection and reachability: http://videos.visitmix.com/MIX09/T78M (also look at the slide deck—it has notes that go into more depth than the presentation). The talk is for Silverlight 3 users, but it applies to both the full framework and the Silverlight framework.

Online/Offline Network Detection with the .NET Framework

PeSmith — Tue, 17 Mar 2009 11:47:00 GMT

Imagine this: you have been working on your great New World Wide Web program. Your very first customer tries it out on their new notebook computer -- perhaps one of those very cute tiny notebooks that people are buying everywhere. And then you get your first report back: “I tried it while walking down the street. Every time I walked by a coffee shop, it failed horribly”. This blog entry will tell you how to use NetworkChange.NetworkAddressChanged to solve your problem.

You have just been hit by the biggest problem in the world of occasionally-connected computing! Your application was fooled into thinking it was talking to your server – that it was online – that it could do something useful. In reality, all it was talking to was the coffee shop, and all the coffee shop was saying was, “please enter the network access code on your receipt to enable networking at this location.” What you need now is online detection, and what you most need to know about online detection is:

You are online if you can connect to your server and you have validated the response.

You might be fooled into looking at the various .Net and native APIs that return a simple boolean that says if you are online or not. Ignore those APIs! They do not provide the answer you need. Why not? Well, there is an entire list of reasons:

1. They mostly say if you are on “a” network, not on “a useful” network or even the internet. Merely being connected to a hub with power is enough for many of them to consider you to be connected.

2. From my personal, hard-fought experience, network detection is an area where some hardware and drivers just do not work reliably. I have even seen a case where configuring a network only worked with a Wizard – manual configuration resulting in a working network but broken online detection code.

3. Many places—for example, the coffee shops and airports – will advertise network connectivity to your computer. But until you enter a special code in your browser (or perform some other action, possibly involving money), the only data that will ever be returned is a request to enter that special code.

4. Some people are on automatic dial-up connection where they pay by the minute. The more modern version of this is the fancy cell-network connections where some data plans will charge by the kilobyte. People with these sorts of connections will be “on line” but will not appreciate your application using the network.

Instead, you have to write your own method that will try to connect to your server, and then check the answer. But wait! When is a good time to check? Well, when your application starts for sure. Other than that – hook up your code to NetworkChange.NetworkAddressChanged. Just make sure that if you are connecting to “your server”, but it is returning nonsense (like, “Welcome to the Airport Coffee Shop...”) then you have to throw in a timer and periodically retry.

One last hint about NetworkAddressChanged– it returns more events than you might think. While testing code for these entries I saw that plugging and unplugging the network cable on my laptop resulted in multiple events being triggered.

MSDN documentation for the NetworkChange.NetworkAddressChanged event is at http://msdn.microsoft.com/en-us/library/system.net.networkinformation.networkchange.networkaddresschanged.aspx

Next up: code patterns to make detection easier.

FtpWebRequest compile with extended comments

mariya — Sat, 26 Jul 2008 11:12:00 GMT

This is a current compile of the team's existing blogs on FtpWebRequest. I am going to update it periodically with new blogs and links to interesting forum questions regarding FtpWebRequest. If you find intersting topics or have ideas for new topics that would benefit our customers do not hesitate to contact us.

MSDN documentation:
http://msdn.microsoft.com/en-us/library/system.net.ftpwebrequest.aspx

FtpWebRequest Overivew: Here you will find the overview of the FtpWebRequest programming model in .Net Framework 2.0 and 3.5
http://blogs.msdn.com/adarshk/archive/2004/09/13/229069.aspx

Sending FTP commands with FtpWebRequest
http://blogs.msdn.com/mariya/archive/2008/07/26/ftpwebrequest-commands-and-how-they-work.aspx

Ftp over SSL with FtpWebRequest
This is a full article on how FtpWebRequest implements FTP over SSL (FTPS) it works, how to validate the server certificate, how to assign your own certificate. Includes samples
Note: FTPS is different form SFTP. Currently FtpWebRequest does not support SFTP
http://blogs.msdn.com/adarshk/archive/2005/04/22/410925.aspx

Sample code for parsing the response returned by FtpWebRequest for ListDirectoryDetails method
http://blogs.msdn.com/adarshk/archive/2004/09/15/230177.aspx

How to resume broken file download with FtpWebRequest
http://blogs.msdn.com/adarshk/archive/2004/12/01/273362.aspx

How to change to the root directory with FtpWebRequest
This question has been posted on the forums several times. The short answer is: yes, you can change to the root directory if your server allows you to. Find how in the blog below
http://blogs.msdn.com/mariya/archive/2006/03/06/544523.aspx

WebException message vs. Response.StatusDescription on FtpWebRequest
http://blogs.msdn.com/adarshk/archive/2005/05/04/414524.aspx

FtpWebRequest: Does the slash matter?http://blogs.msdn.com/mariya/archive/2006/06/15/632768.aspx

How to troubleshoot your System.Net code

mariya — Sat, 26 Jul 2008 06:19:00 GMT

Sytem.Net tracing
One of the easiest ways is to use the built in System.Net tracing. It is a matter of copiying a config file to your machine and re-running your application. Here is a very detailed article on how tracing works and how to enable it.
http://blogs.msdn.com/dgorti/archive/2005/09/18/471003.aspx

NetMon capture
The other way is to sniff the network traffic using NetMon, Ethereal, or etc. Here is how you can do it with NetMon
http://blogs.msdn.com/dgorti/archive/2005/10/29/486887.aspx

Welcome to the Network Class Library (NCL) Team Blog

Aaron Oneal [MSFT] — Tue, 22 Jul 2008 03:33:00 GMT

Delivering your managed networking experience.

There is a chance that you have never heard about the NCL team and yet you’re using our components every day (most likely, right now while you’re reading this). How could this be?

When you opened this page to read our blog, here is what happened under the covers: The browser sent an HTTP web request and the server answered by sending back an HTTP web response; when those calls are executed using managed code, then you’re using our components. In fact, every time you download a file, access secure resources (via SSL), or access any other network resources using managed code (any .NET language), you’re using our features. If you’re working with ASP.NET and web services in IIS, without realizing it, you’re using our features in almost every call that you make. In addition, if you’re writing or using Silverlight rich internet applications, you’re benefiting from our features (check out http://www.silverlight.net/ to see how cool Silverlight is).

Who are we and what do we do?

We are developers, testers, program managers, and document writers who work together to give you an unmatched managed networking experience. We own the System.Net namespace(s), the System.Uri class, and core networking components for Silverlight and Windows Live Mesh.

Why our technology?

Because our technology is simpler, more secure, and more powerful. We have dozens of classes that make it easy to perform a variety of tasks, including sending email, serving up a web page, retrieving a web page, creating a secure connection, and even gathering information about the networking capabilities of the computer your code is running on.

Sending data over a network can be as simple as instantiating a TcpClient and connecting to a URI. You can have your networking code up and running in minutes. Need more control over your network stream than that? Our layered approach allows flexibility; open a socket directly so that you can implement your protocols the way you need them. The only limit is your own creativity.

Furthermore, because you’re in a managed environment, you have all the power of that environment at your disposal. Your code will work on both 32 and 64 bit platforms, we handle interoperability with COM and native libraries, and we even manage the memory for you. With our technology, you can spend less time figuring out how to deliver solutions and more time actually delivering them.

You can find more detailed technical information in our MSDN library documentation and community forum, and you can contact us using the e-mail link above. For feature requests and bug submissions, please use the Microsoft Connect website.

Welcome to our blog and have fun reading our posts!

- NCL Team

Network Class Library Team (System.Net)

UDP Multicast in Silverlight 4

Scenario Overview

User Datagram Protocol (UDP)

Multicast Concepts

New APIs

Browser Security

Conclusion

Visual Studio 2010 and .NET Framework 4 Beta 2 Released!

New Performance Counters for HttpWebRequest

What's new in System.Net.Mail

End-to-end connectivity with NAT traversal

Home Routers with NAT

NAT Traversal & Security Considerations

Impact to Applications

IPv6 Restores Addressability

6to4 Tunneling

Teredo Tunneling

New for .NET 4.0

Listeners

Sockets

Configuration

Address Discovery

Windows Communication Foundation

Firewall Considerations

Parting Thoughts

Additional References

New NCL Features in .NET 4.0 Beta 2

Sockets

DnsEndPoint

IP Version Neutrality

SSL Encryption Policy

SmtpClient

SSL Configuration

Header Encoding

Multiple ReplyTo Addresses

HttpWebRequest

64-bit Range Header Support

Date Header Support

Host Header Support

WebRequest Performance Counters

NAT Traversal

Escaped Character Support in Uri

Bug Fixes

Why does Silverlight have a restricted port range for Sockets?

Desktop trust model

Web trust model

User vs. IT admin security decision

Port ranges and transparent proxy abuse

Other solutions

Conclusion

Custom HTTP Authentication Schemes

Introduction

Standard HTTP Authentication

WebRequest Authentication Practices

Custom HTTP Authentication Schemes

How to use a custom module with WebRequest/WebClient

How to write a custom authentication module for GoogleLogin

Conclusions

Notes & references

Functional Areas of the Network Class Library

Code Patterns for Online/Offline Network Detection

Online/Offline Network Detection with the .NET Framework

FtpWebRequest compile with extended comments

How to troubleshoot your System.Net code

Welcome to the Network Class Library (NCL) Team Blog