Ravi Rao's Blog

Get Visual Studio 2005 Express for Free!

ravirao — Wed, 09 Nov 2005 08:57:00 GMT

Great deal!

http://msdn.microsoft.com/vstudio/express/default.aspx

PDC '05 videos are now online

ravirao — Sun, 30 Oct 2005 12:30:00 GMT

All talks are available at http://microsoft.sitestream.com/PDC05/

People Near Me Talk: http://microsoft.sitestream.com/PDC05/COM/COM319.htm#nopreload=1&autostart=1

Peer to Peer Talk: http://microsoft.sitestream.com/PDC05/COM/COM311.htm#nopreload=1&autostart=1

FAQ

ravirao — Sun, 30 Oct 2005 07:37:00 GMT

I'll try to keep this FAQ updated on the general questions asked on the p2p infrastructure:

1. How scalable is the p2p framework?

* PNRP is the serverless name resolution protocol, and is scalable to the size of the internet.

* Graphing/Grouping: These are protocols intended for peer to peer group activities, and are scalable to the thousands.

* Peer Channel (a part of the Windows Communication Foundation): These protocols intended for multiparty messaging are scalable to the tens of thousands and likely more (still testing)

2. How much bandwidth do these protocols consume?

The protocols in general are intended to consume far less traffic than any application using the framework will consume. We are in the process of testing bandwidth consumption, and will be happy to share numbers in a little time.

3. How much time is spent before a node connects to a network?

* PNRP - should be less than 10 seconds

* Graphing/Grouping/Peer Channel - this really depends on the mechanism used to find a node in the mesh to connect to. If PNRP is used as the mechanism of finding such a node, then it will take a couple of seconds more. The actual connection establishment mechanism is fairly quick.

4. Where is Teredo server currently setup?

Microsoft hosts a public Teredo server that is available for all applications to automatically use. To learn more on Teredo, take a look at http://www.microsoft.com/technet/prodtechnol/winxppro/maintain/teredo.mspx

5. What are the typical topologies formed for say when 30 nodes gets connected in a multiparty activity?

Take a look at the white paper http://www.microsoft.com/technet/prodtechnol/winxppro/deploy/p2pintro.mspx, which shows what a graph looks like.

6. Reliability. When a message sent by a node does not receive a response, does it really means node isn't? Numbers of hops may be pretty large and hence chance of message loss is pretty high. Is it true?

* PNRP - if a response is not received, then almost certainly, the node is either not accessible because it is behind a corporate firewall for instance, or it is just not online

* Graphing/Grouping/Peer Channel - if a message is not received, then there are some possibilities (all of which should be very rare) - the mesh is partitioned and a message consequently could not get through, some neighbor was just not able to forward the message on, or the local stack itself did not deliver the message to the application. All of these should be very rare, however, if your application requires reliability guarantees (100% of the time), then you may have to implement a simple protocol on top of the peer channel for this purpose.

7. Replicated data store. Is it built on SQL Server, XML or something else? How dynamics are orchastrated? What happens when A syncs with B and C, B syncs with D and then C syncs with D? Would there be any ADO.Net provider to access this data store? Are there any events raised as data changes? Is there any built in features to maintain versions of data?

* The data store is a very simple store. It does allow for use of XML attributes in records that you can use for searching for relevant data. The synchronization protocol is based both on timestamp and version number, and is based on the last writer wins principle. However, you can enforce security in the group so as to ensure that no one else can clobber your records. There are no ADO.Net providers for this data store. There are record events raised any time data is added/deleted/modified.

8. Real time streaming capabilities. Can a node, for example, transmit the video content to thousands of other nodes without reaching bandwidth bottlenecks.

* A group and a graph cannot be used directly for streaming. You would use the group/graph as a mechanism to get the topology, and then construct a streaming overlay on top. You can always use the underlying graph for sending less real-time sensitve data such as chat, collateral for meetings etc.

* The peer channel (which is showing incredibly high throughput numbers) could be usable in streaming scenarios. We are currently evaluating that. If you are interested, let me know and we can discuss.

9. Distributed computing. Is there any built-in features to let node execure code on other nodes remotely (P2P version of RPC)?

* The data you send in messages to the mesh could contain anything. Your application which installed on all machines can quite easily send some data (which is code) to execute on other machines. However, our infrastructure natively does not enable these scenarios yet.

10. How does WCF/Indigo compare with Bonjour/Rendezvous?

* WCF enables messaging peer to peer, client server, amongst servers etc. It implements a large set of WS-* protocols for interop with other web services protocols/infrastructures as well. Bonjour is just a small piece of messaging from Apple that focuses on finding devices on the same subnet. It is likely similar to one protocol, WS-Discovery which enables discovery of services on the network. It is designed to be a scalable architecture that can be bridged across subnets using proxies. It is shipping as part of Windows Vista. Another piece built on WS-Discovery that Bonjour may be close to is People Near Me, which enables discovery of people located on the same subnet. Bonjour does not use PNRP! Neither does WS Discovery. The two are not compatible with each other.

Adrian Moore's managed projects

ravirao — Sun, 30 Oct 2005 07:33:00 GMT

Some cool work by Adrian Moore on managed wrappers and project built on the peer to peer collaboration framework.

http://www.codeproject.com/csharp/PeerNameResolution.asp

I did a People Near Me Podcast

ravirao — Sun, 30 Oct 2005 07:13:00 GMT

http://ch1.learningfeeds.com/

Let me know what you think!

Peer to Peer Whitepapers suggestions?

ravirao — Sat, 08 Oct 2005 21:00:00 GMT

Any suggestions on the kinds of whitepapers you would like to see on the peer to peer collaboration space? We have technologies for serverless name resolution, multiparty communication meshes, people near me and application invite, amongst others.

We already have an introductory white paper on the website. Have you read it? What are your thoughts on it, and suggestions for improvement?

Also, let us know what other white papers you think we should have.

Event name suggestions?

ravirao — Sat, 08 Oct 2005 08:55:00 GMT

So, we are trying to figure out a good name for an event for the following:

The mechanism by which you access published capabilities and objects from a person near you is by calling the PeerCollabRefreshEndpointData() API. This API effectively refreshes the local cache with whatever data is being published by the endpoint. Since this is a network operation, it is asynchronous, and has an event for when the operation completes. We are trying to figure out a good name for this operation.

Any thoughts, on what sounds like a good name? Each event is associated with a structure containing the data that was returned. Please give suggestions!

PEER_EVENT_ENDPOINT_DATA_REFRESHED

PEER_EVENT_ENDPOINT_DATA_CHANGED

PEER_EVENT_ENDPOINT_DATA_AVAILABLE

Any other ideas?

API model for People Near Me vs Internet/Contacts presence

ravirao — Mon, 03 Oct 2005 09:16:00 GMT

The model for presence revolves around contacts. You have a bunch of contacts, and you want to get their presence when you sign in, and if they signed-in after you did, then you get their presence at that point of time. That is, the programming model is a subscription model. The infrastructure automatically subscribes to the presence, capabilities and objects of a user's contacts. All API calls are simply local at that point (for instance, getting presence, capabilities, objects).

With People Near Me however, the model is a polling model. Since there can be a very large number of people on the same subnet, we do not automatically subscribe to everyone's capabilities and objects. Instead, we expose a method (RequestPublishedItems) for the developer to get the published items (capabilities, objects etc) on demand. Once this API is called, then all operations become local after that much like it is with presence. So, when dealing with people near me, this is an additional API to call for the people you are interested in getting more info from. Now, the API ends up keeping data for future callers as well. Consequently, if another application were to call PeerCollabEnumObjects, it will return the objects returned in a prior call to RequestPublishedItems.

Some APIs, by design, do not work for People Near Me. For instance, setting the endpoint name does not work, because it is automatically set to the machine name. If you want to get the endpoint name, you get that via the peer_collab_endpoints_near_me event. I'll keep this post updated with other APIs that will not work for one signin option vs another.

The Peer Channel in Windows Communication Foundation (aka Indigo)

ravirao — Tue, 27 Sep 2005 05:58:00 GMT

At this point, our only managed offering is the peer channel. The peer channel provides multiparty messaging in a mesh. It is an implementation of the graphing/grouping algorithms in Windows Communication Foundation (WCF), and allows you to use the familiar (assuming you are familiar with the WCF programming model) WCF programming model to send transient messages (think multicast) to the mesh. The messages should be small, because the mesh has a certain amount of redundancy to ensure integrity in the face of node transience. It is very straightforward to use if you are familiar with WCF. If not, there are lots of bloggers on WCF who I'm sure will love to help you out.

We are seeing staggering (from my perspective anyway :)) performance numbers right now, with throughputs of ~1000 messages per second in small meshes...I think it is a bug in our measurements, but we'll know more soon :) However, if the numbers are right, it is a testament to just how performant the stack really is.

If you are planning on using WCF, and are interested in multiparty communication, consider using the peer channel.

How do I learn the GUID of the application to send an invite to

ravirao — Mon, 26 Sep 2005 23:29:00 GMT

Applications can publish their availability to people near me. When you call PeerCollabRegisterCapability(), you can specify a scope which is the subnet and information about that application is now available to people near me. Now, each capability is associated with a GUID, a name, description, a path and a scope. If the capability is registered in the pnm scope, then our infrastructure will publish out some information about the application (name, guid, but not the path).

The client that wishes to send an invite calls RequestPublishedItems(), and can then enumerate the capabilities, to get the GUID of the application to send the invite to.

What does "Signing In" mean anyway?

ravirao — Mon, 26 Sep 2005 19:05:00 GMT

Our infrastructure exposes APIs to signin...PeerCollabSignin(). There are two primary aspects of signing in:

1. Signing in to People Near Me (PNM): Signing into PNM means that you are publishing a nickname (which the user entered when our infrastructure was used for the first time) to the people on the same subnet. In addition, you can enumerate the people who have opted in to PNM as well. Applications can also choose to make capabilities and any objects available as well.

2. Signing in to "Internet": The notion of signing into the Internet is actually a little different from signing into PNM. The idea with signing in to the Internet is that you have the ability to see the presence of your contacts, not of everyone on the Internet :) Consequently, signing into the Internet requires that you have contacts, so that you can get their presence. However, just because you have added someone as a contact does not mean that you can get that person's presence. The contact on the other end actually needs to have approved you viewing their presence.

If you look at the PEER_CONTACT structure, there are two fields - fWatch and PEER_WATCHER_PERMISSIONS. You need to add a person as a contact, once you do that you can get a contact structure from it. The fWatch field in this structure is used to tell our infrastructure to get that particular contact's presence. The PEER_WATCHER_PERMISSIONS structure is used to authorize our infrastructure to release my presence to that contact.

For instance, if Adrian wants to view my presence, then he will mark the fWatch field on my contact to true. However, he will only get my presence after I mark the PEER_WATCHER_PERMISSIONS field as "Allow" on his contact in my list. Similarly, if I want to get Adrian's presence, I mark the fWatch field on Adrian's contact. If Adrian does not want me to see his presence, he would mark the PEER_WATCHER_PERMISSIONS field as "Deny" on my contact. Now, Adrian can watch my presence, but I cannot watch Adrian's presence. This helps in building up a rich presence system.

The last aspect is...how do we exchange contacts. Contacts are fully integrated with the Windows Address Book (WAB). Just navigate into the WAB, and you will find your own contact that contains the nickname you had entered about yourself. Put this in email, and send it to your friend. Your friend can now save this in the WAB, and he has your contact info. Your friend would do exactly the same as well.

The same can be done programmatically as well.

To summarize, signing into PNM will publish your nickname to people on the same subnet. Signing into the Internet will get presence of your contacts (who have authorized you to get their presence), and will publish your presence to your contacts as well (who you have authorized to see your presence).

Terminology used in the peer to peer SDK

ravirao — Mon, 19 Sep 2005 21:55:00 GMT

I'll keep this post updated with new names/terms as we put them in our SDK. Most of our terms had to be prefixed with the word "Peer" because the goal of the SDK (according to the SDK team) is to keep the documentation as *clear* as possible. Not sure we are achieving that goal here :)

PNRP - Peer Name Resolution Protocol - allows for registering and resolving a name in a secure serverless manner. When you are resolving a name, the protocol assumes that you already know the name you are looking for. It is a "name resolution" protocol, not a "name discovery" protocol. In the context of PNRP, other terms you will see get used are peer names. From the API, you register and resolve for peer names.

Graphing/Grouping/Peer Channel - all these technologies refer to the notion of being able to do multiparty messaging on top of a mesh. Our infrastructure creates the mesh on top of which developers can call APIs to send messages to all users who are connected to it. There's just lots of different names for the same technology. There are some subtle differences however. Graphing and grouping are win32 native APIs intended for creating a replicated *persisted* store and synchronization. Specifically, graphing lets you plug in your own security model and name resolution service, while grouping comes with a predefined name resolution service (PNRP), and a security model. Peer Channel is meant for multiparty messaging. Messages are sent to everyone, and the application can choose whether they wish to save these messages or not. Peer Channel is part of the Windows Communication Foundation (Indigo) stack.

Identity - The user identity, which most commonly refers to the default identity that is created on user logon. It has a cert associated with it which is generated locally. This is also called the "Me contact".

GMC - Group Membership Certificate - this is the identity of a user in a specific group. It is linked to the identity defined above. Basically, this is a group certificate issued to the identity defined above.

Contact - The "Me Contact" is the same as Identity described above. If you add my Me contact to your contact list (in the Windows Address Book), then I am your Contact. The word "Peer contact" also is used to describe this. In some places, we also seem to use the word "Local peer".

Capabilities - This just refers to a collaborative application you are writing. Other terms used are "Application, Peer Application".

PNM - People Near Me Collaboration. Discover people near you and interact with them. PNM is a "Name Discovery" protocol, that is, you discover the name of the people you want to interact with. Since it is on the local subnet, it also ends up giving the associated IP address of the endpoint (person) you want to interact with.

Q. How do PNM, PNRP and Mesh relate to each other.

A. Think of it this way. You discover the people located near you using PNM. Information about the person you discover is a peer name. This peer name can be resolved using PNRP to an IP address even if the person is not located nearby anymore. You and the person near you or on the Internet can participate in a group activity (such as chatting or photosharing or gaming) by using one of the mesh technologies.

What is this collaboration host icon that shows up on my tray...

ravirao — Mon, 19 Sep 2005 21:49:00 GMT

If you are developing to the peer to peer collaboration APIs you may see a tray icon appear when you first use the APIs. This is the collaboration host (the process name is p2phost.exe, and it is a user mode process). p2phost.exe is responsible for listening for incoming invitations, and for popping up UI asking the user to authorize the launch of an application. In addition, you would be able to opt in and out of people near me publication.

Questions asked during the PDC talk on developing P2P applications

ravirao — Fri, 16 Sep 2005 19:07:00 GMT

Q. Is there a concept of an owner/administrator of a mesh?
A. Yes, you can have roles in a mesh.

Q. Is there a way to kick someone out of a mesh?
A: Currently, the only way to kick someone out of a mesh is to create another mesh without this member. Clearly, this may work with small groups, but not with larger groups. With larger groups, we currently do not have a means to revoke a person from a mesh.

Q: What are the differnet optimizations that are done on the mesh, and how do these work?
A: Meshes are dynamically shaped for optimizing data delivery to all nodes, and meshes are very difficult to get partitioned. If they do get partitioned, some of our offerings have ways in which these can be repaired. To read on details on how the mesh algorithms work at a high level, please take a look at this white paper http://www.microsoft.com/technet/prodtechnol/winxppro/deploy/p2pintro.mspx under the section of graphing. Please note that we use the words graphing, grouping, peer channel and mesh interchangeably just to keep things confusing :)

Q: Is there a way in which data sent in the mesh can be signed so that it is not tampered?
A: Yes, data sent in the mesh can be signed and encrypted.

Q: Are there any message patterns/protocols being standardized on top of the mesh?
A: WCF (aka Indigo) provides a means by which services/protocols can be composed on top of existing channels. I don't have information on whether any of these message patterns built on top of the peer channel would get standardized or not at this stage.

Q: Are there constraints to the clients on how much data can be sent?
A: There are limits on how much data can be flooded around to prevent obvious DoS attacks.

Q: What are the plans with standards body alignment?
A: At this stage, I don't have anything concrete to tell you all. Please stay tuned in this space.

Q: When would I use grouping vs the peer channel?
A: Grouping is a win32 API that enables multiparty replication/synchronization on a small scale (in the hundreds). If you have the need to maintain a history for what has happened in a mesh, then the grouping technology would suit you best. Peer Channel on the other hand is a multiparty messaging system (with no backing store for the data sent) that is intended for larger scale scenarios (where number of participants is in the thousands). Clearly, you could use either technology depending on what your needs are. If you want data replication in the peer channel, you would have to maintain your own store. If you want messaging in grouping, you would need to send short lifetime records that are replicated. It depends on the scenario. If you want managed API, and are already building a WCF application, then the peer channel may be the way to go. If you already have a win32 application that you are enhancing, then grouping could be the way to go. We are providing different options for you to pick from.

Q: How do the contact certs in question get deployed on the various machines?
A: This is out of band of our infrastructure, and admittedly, is a little hard. We are making it as simple as emailing the Me contact which is located in the WAB, but it is still a manual step outside of our infrastructure. Do you think this is actually that hard? I am interested in hearing your feedback on how we could make this easier if this is a hard problem.

Some useful links

ravirao — Fri, 16 Sep 2005 04:21:00 GMT

Here are some useful links on P2P:

http://www.microsoft.com/p2p (uber website, badly in need of updating)

http://www.microsoft.com/technet/prodtechnol/winxppro/deploy/p2pintro.mspx (White paper)

http://windowssdk.msdn.microsoft.com/library/default.asp?url=/library/en-us/P2PSDK/p2p/collaboration_api_reference.asp (Windows Vista p2p collaboration APIs)

peerfb@microsoft.com (email alias for questions)

http://www.thoughtpost.com/p2p.aspx (Managed Wrappers for PNRP, Graphing and Grouping)

http://mooresmind.blogspot.com (Managed wrappers from Adrian Moore on the P2P Collaboration Infrastructure)

http://www.codeproject.com/csharp/PeerNameResolution.asp (Managed project built on p2p collaboration infrastructure by Adrian Moore)