Home
Live Mesh - Site Home - MSDN Blogs

Live Mesh

Here, There, Everywhere...

  • Live Mesh

    Heads up: required update to Live Mesh coming next week

    • 19 Comments

    With PDC just around the corner, we will be releasing the next update to Live Mesh, and it’s a major one. In fact, this is the most significant update to Live Mesh since we opened as a technology preview back in April. We know that many of you are going to be excited to see all the new features and fixes this release will bring (and there are a lot of them, many in response to your requests.)  In the spirit of openness with you, our technology preview customers, we’d like to take a little time today to talk about what the update experience is going to be like—and why.

    For all the previous updates to Live Mesh that we’ve released since the tech preview began in April, customers were notified (via a little pop up above the Live Mesh notifier) that updates were available. When you were ready to update the Live Mesh software on your device, you clicked on that notification and the process began. If you chose not to update right away, though, that was okay—Live Mesh continued to work uninterrupted. With this update, however, things are going to be a little different. Specifically, all Live Mesh users will have to update their Live Mesh software to continue using the product. This time, you won’t be able to wait to update Live Mesh—in order to keep using it, you’ll have to update the software, and until you do that synchronization and Live Remote Desktop will not be available.  There is a fairly straightforward reason for this: we are upgrading our software and services to support an open beta, and to handle that scale we need to get all of our existing users onto these new systems quickly.  For this particular release, we are forgoing backwards compatibility in favor of making the new set of features as broadly available as possible.  Once you see how much Live Mesh has grown, we think you’ll agree that this was the right tradeoff!

    We hope the advance warning reduces the amount of interruption this upgrade might cause. We will, of course, let you know before we begin to actually deploy the new update—we just wanted to make sure all our valued early adopters were “in the know” right from the start.

    Stay tuned for lots of Live Mesh and Live Services news from PDC over the next week, and thank you to all of you for the great feedback you’ve provided over the last six months. Live Mesh wouldn’t be where it is today without all of you!

    Technorati Tags:
  • Live Mesh

    Behind Live Mesh: The Pub-Sub System

    • 6 Comments

    Hey there! I’m Viraj Mody, a developer on the Live Mesh services team. I bet some of you are curious about how the Live Mesh software running on your devices detects when others add files to your shared Live Folders, quickly changes icon colors when devices go from offline to online, or updates your Live Mesh News Feeds as changes occur. This post will give you some insight into the facilities we’ve built in the Live Mesh back-end that enable client applications to do all of the above and much more.

    In order to be responsive and perform well, Live Mesh, like most other software + services systems, requires that the cloud (services) be able to send out-of-band messages to clients (software). Most actions initiated by the client (MOE and Live Desktop) are triggered as a result of these messages sent from the Live Mesh services cloud to that specific client. These could be messages informing the client that items in the cloud have changed, that another client wants to initiate a peer-to-peer connection to this client, and so on. Such out-of-band messages that the cloud sends to clients are called Notifications. In order to keep the overall system performing well and to safeguard the privacy of clients, it is important that a client only receives Notifications for items it is interested in and has permissions to access. Each client has the ability to tell the cloud what items it is interested in monitoring. This expression of a client’s interest in changes to specific information in the cloud is referred to as a Subscription. Together, these Subscriptions and Notifications form the building blocks of Live Mesh’s Pub-Sub System.

    In this post, I’ll go over the different back-end services that comprise the Pub-Sub System and provide an overview of how the system works end-to-end. In many ways, the Pub-Sub System can be understood by comparing it with a ‘secure and smart postal service’, where the post office only sends you mail that you’re interested in receiving and that the sender says you have permission to receive. A spam-free postal system – wouldn’t that be awesome!

    Establishing a Communication Channel

    For the Pub-Sub system to be effective there must be some way for the cloud to identify and communicate with each client. Live Mesh exposes facilities that allow clients to create a Queue which is uniquely associated with that particular client and assigns a unique name that is used to identify this queue. When creating a Subscription to a specific item, the client passes along its unique Queue name which allows Notifications to be delivered to that Queue. Going back to the postal system analogy, this Queue is like the mail box outside your home, and the unique name of the Queue is like your mailing address. Similar to how the postal system ensures that mail addressed to you only ends up in your mail box, Live Mesh services ensure that Notifications for a given client end up only in that client’s Queue.

    The client is responsible for retrieving Notifications from its Queue. By separating delivery of Notifications from the act of retrieving them, the architecture enables various transport types to be used to retrieve Notifications from a Queue. Currently, the Live Mesh service cloud exposes two transports for retrieving these Notifications – HTTP and TCP. Given the pervasiveness and ease of use of these two transports, these were the first types we decided to support. As our service offering and scope grows, we can enable more transport types without major changes to the back-end architecture. Using the HTTP resource model exposed by the Live Mesh services, clients can choose to periodically fetch Notifications from their Queue. Using TCP enables clients to establish a long-lived connection to the Live Mesh service cloud so that Notifications can be pushed to clients as soon as they arrive. Different clients have different behavior, requirements and constraints – by providing various transport types we can enable several classes of clients and applications to leverage Live Mesh’s Pub-Sub system. Of course, these communications are encrypted and only the owner of a Queue can retrieve Notifications from it. It’s important to note that as it currently stands, the Queue and related services only support unidirectional messaging. Messages from clients to the Live Mesh service cloud do not flow via the Queue. Also, as of writing this post, the Live Mesh client available for Tech Preview doesn’t yet leverage the TCP transport solution – it will in a future release.

    Besides change Notifications, the client’s Queue is also populated with a special type of Notification when another client wishes to initiate a peer-to-peer connection. Only authorized clients are allowed to send such peer-to-peer invites. The purpose of the peer-to-peer connection might be for file exchange and sync, for Live Remote connections, etc. Details about our peer-to-peer design can be found in this Channel 9 video.

    All Queues and Queue-related information are managed by the Queue Service. Like other services described later in this post, the Queue service is a transient-state service and is built to scale horizontally. I’ll touch on both these characteristics of the services later.

    Creating Subscriptions

    Once a client has created its Queue, it can create Subscriptions to items in the Live Mesh cloud that it wants to stay current on by providing its Queue name along with the Subscription. Typically, each Live Mesh device creates Subscriptions for user/device presence, for each Live Folder’s various feeds (Membership, Contents, etc), for the news feed, and so on. In order to create a Subscription for a specific item, the client must provide proof that it has privileges to access the item. Once these Subscriptions have been created, the system ensures that the client will be informed when any of the items it has subscribed to are changed – each Notification that is delivered into the client’s Queue contains information about which specific resources has changed, in addition to other potentially interesting information about the changed resource.

    Subscriptions created by clients are held in the PubSub Service. It is responsible for maintaining information about which client is interested in what item and for fanning out Notifications to the right clients when an item changes. You can think of it as the central post office of the ‘secure and smart postal system’ which acts as the one location for collecting all mail and then routing it onwards to the right destination. Just like the Queue Service, the PubSub Service is a transient-state service and is built to scale horizontally.

    End-to-end Flow

    Once a client has created a Queue and all its Subscriptions, it doesn’t aggressively keep refreshing information from the cloud. If using the HTTP transport to talk to its Queue, the client periodically polls the Queue to check for any Notifications that may have arrived. In cases where it’s using the TCP transport, the client is waiting for a Notification to be pushed down to it over the TCP channel. When an item changes in the cloud (let’s say, a file was added in a folder that the client has expressed interest in knowing about), the service responsible for that item (in this case, the storage service) informs the PubSub Service that the item has changed. The PubSub Service, which has been keeping track of all clients who are interested in that particular item, drops an appropriate change Notification in each interested client’s Queue. As soon as clients retrieve these Notifications (or as soon as the Notification is pushed to them) they can react. In the case of a new file being added, they might choose to begin initiating a peer-to-peer connection in order to sync the new file.

    In our ‘secure and smart postal system’ analogy, this is equivalent to a magazine publisher informing the post office that a new edition is available, and the post office dropping a letter in each subscriber’s mail box informing them that a new edition is available. Optimally, the magazine publisher could also deliver one copy of the latest edition to the post office and the post office could be smart enough to create the right number of replicas and deliver a copy to each subscriber’s mail box. Here’s where the ‘smart’ in ‘smart and secure’ could come in!

    image

    Pub-Sub System end-to-end flow

    Transient State Services and Scale-out

    As I mentioned previously, all services that comprise the Live Mesh Pub-Sub System are transient-state services. Queues created by clients, Notifications that are delivered to specific Queues and Subscriptions representing a client’s interest in a particular resource are only ever held in memory and never persisted to any kind of store. As you might probably guess, performance was one of the biggest motivators for this design. Pub-Sub is characterized by short-lived rapidly changing data and data that needs to be readily available. Short-lived and changing because Queues, Subscriptions and Notifications are, by nature, transient – once a Notification is delivered to its intended recipient, it’s of no use; once a client is offline or doesn’t care about receiving Notifications, its Queue is of no use; Subscriptions could come as and go as application state changes. Data must be readily available because subscriber lists can often be huge, so retrieving them from persistent stores can introduce latencies that only increase as the service grows in size. Holding these in memory allows reads and writes to be processed very fast. Since they live in memory, both Queues and Subscriptions have lifetimes associated with them – clients must perform certain actions (some explicit, some implicit) to keep Queues and Subscriptions ‘alive’ and prevent associated resources from being reclaimed by the server. Given the initial wave of Live Mesh experiences and applications, having these be transient-state services definitely helps ensure high throughput and low latency. Of course, the fact we chose to implement the system as a transient state system is an implementation detail – as the product evolves and use cases changes, there might be reasons to prefer some kind of hybrid approach.

    The system is also designed to enable horizontal scale-out. As the system needs more space to hold Queues and Subscriptions, we can bring up new instances of these services to increase capacity. Using a scheme based on consistent hashing, the Live Mesh services cloud guarantees that there is ever only one specific server instance that can ‘manage’ a given Queue or Subscription. The system also enables routing of messages for specific Queues and Subscriptions to the correct current manager. As new service instances come online and others go offline, the system automatically re-balances the distribution of Queues and Subscriptions to the currently available servers such that every Queue or Subscription is managed by one and only one server instance.

    One of the obvious concerns with the system being implemented in-memory only is data recovery – when servers go down because of hardware/software issues, are re-booted, or otherwise need to be reset, all data resident in memory on those servers is also gone. For Queue Service instances, this implies that Queues belonging to several clients and potentially interesting Notifications in those Queues might be gone. For the PubSub Service, several subscriber lists might be lost when a server loses state. This is a problem we spent a huge amount of time addressing and designing for, and probably deserves a post of its own at some future time. A short summary of the solution is that in cases where one or several Queue and/or PubSub Servers go down, the system is able to detect exactly what happened and take remedial action to restore state in the cloud in cooperation with clients (because clients were the original source for all the transient data that was resident on those servers before they lost state).

    Parting Notes

    As the future scope of the Live Mesh experience, services and platform evolves (for instance, potentially allowing third-party services to subscribe to items in Live Mesh, enabling aggregators to leverage Live Mesh Pub-Sub, etc), the current Pub-Sub System architecture will hopefully provide a good scalable foundation which we can leverage to rapidly increase capabilities of the service.

    We’re working on exposing Pub-Sub capabilities via the Live Mesh SDK when it becomes available so that developers can leverage the system in innovative ways to build responsive applications. Be sure to visit the Live Mesh Developer page and join the developer waiting list for announcements around the Live Mesh SDK when it’s available.

    I hope this post gives a little more insight about how things work behind Live Mesh. Be sure to install Live Mesh on all your devices, give us feedback and report any issues you see!

    Technorati Tags:
  • Live Mesh

    Service Update

    • 8 Comments

    We're doing some server maintenance over the next couple days to update some of our configuration information.  During this time you may experience Live Mesh service outages, including Live Mesh folders that are not synchronizing, or an inability to establish Live Mesh Remote Desktop connections to your devices. We'll update here once we've completed this work.  Thanks for your patience as we get everything patched up.

    UPDATE: We've completed the work mentioned above.  Some users have reported issues with their devices appearing in the offline state and not available for Live Remote Desktop connections.  To bring your device back to a healthy state, you need to exit and restart the Live Mesh client software, and then also stop and restart the Remote Desktop services (either net stop/start wlcrasvc, or just reboot).  Step by step directions are at http://social.microsoft.com/Forums/en/LiveMesh/threads/ under the "Live Remote Desktop Issue" thread.

    Quick background: our updates this week involved a regularly scheduled update of some of the certificates we use to identify devices in the Mesh and to encrypt traffic sent to or through our servers.  We've discovered some cases where the client software doesn't pick up all the right updates unless you follow the particular order above (restart the client software/notifier first, then restart the remote desktop service.  As always, we appreciate your help in tracking down these kinds of issues in our Tech Preview.

    Technorati Tags:
  • Live Mesh

    The Live Platform at PDC

    • 1 Comments

    We've had several posts here talking about Live Mesh as a platform, and we've said that we'll tell more of that story at our Professional Developer's Conference in October.  The session list for PDC is now mostly complete, so you can start to get a sense of what we'll have to say about our platform:

    Live Platform: New Developer Services and APIs
    Live Platform: Building Mesh Applications
    Live Platform: Mesh Services Architecture Deep Dive
    Live Platform: Architecture and Advanced Programming Techniques
    Live Platform: Building Applications with Social Data
    Live Platform: FeedSync and Mesh Synchronization Services
    Live Platform: Notifications, Awareness, and Communications
    Live Platform: The Future of the Device Mesh

    We're working on a couple more sessions as well that we hope to unveil once the conference starts.  And our VP, David Treadwell, is confirmed as a keynote speaker.

    They'll be a bunch of folks from the Live Platform team spending time in the community areas at PDC, and we're looking forward to your feedback and suggestions, and hearing what you plan to build on top of our platform.

    Technorati Tags:
  • Live Mesh

    Got conflicts or missing files? An update...

    • 9 Comments

    Last week, we posted about an issue that might cause your Live Mesh Folders to either appear to have missing files, or have unexpectedly large numbers of conflicts.  We mentioned that we were testing a fix and we’d update you as soon as possible…and then it got quiet here for a bit.  The good news is that we’ve made a lot of progress since then, so here’s an update.

    The root cause of the issue (clients not synching properly after encountering too many tombstones in a row) is fixed.  We actually put this fix into production early last week, and then watched to make sure tombstones were no longer causing any issues.  It’s been a week now, and we’re feeling good that we have this particular tombstone issue fixed.  We grudgingly admit that we are not perfect, and there might be other bugs left in our Tech Preview that cause incorrect conflicts or synch behavior – keep reporting issues that you see, and we’ll keep fixing them.

    Of course we also want to help users who previously hit this bug and have folders that are in a confused state.  If you were hitting the issue where folders were only partially synchronizing, you should have already seen this fixed – the first time your client software contacted the server with the fix in production, the server should have returned the complete file list.

    If you hit the issue where a folder had a large number of conflicts, we have good news for you here as well.  This particular tombstoning issue affects folders in a very distinct way, since it ends up creating conflicting items whose names and file contents are identical.  Conveniently, this also means that it’s fairly easy to automate clean up of such a folder.  Since we can tell that there are no real conflicts (that is, the files are in fact identical as determined by hashing file contents), we can remove any conflicts and just keep one copy of each file.  The net result is that we’ve written a tool which can identify folders affected by this particular bug, and then automatically clean them up.  The tool runs entirely in our datacenter, it never actually accesses file contents (we create/store the hash on upload), and it never exposes any of the filename data outside the datacenter.  We can ask the tool to give us back a list of LiveIDs of users who have affected folders, and then we can tell the tool to go fix all the folders that a given LiveID owns (if you create a folder, Live Mesh treats you as the owner.)

    This tool should fix many of the conflicts created by this tombstone issue, but not all of them.  If your folders have been out of synch, and you’ve been making changes to files on multiple PCs, well then you have actually generated a real conflict (two different copies of the file exist), and it will take your attention to decide which copy to keep. 

    Here’s the announcement about the tool that we’re posting to the forums with details on how to get your affected folders cleaned up:

    Last week we posted a sticky in our forums containing the details of a recently discovered issue which has resulted in some Live Mesh folders being only partially synchronized. To better serve our customers, we have analyzed our storage service to identify folders that might have been impacted. If our analysis shows that you have a folder that might have been affected by this issue, resulting in either a large number of conflicts associated with your folders or fewer than expected files, you will be receiving an e-mail from us within the next three days asking whether you would like us to perform an automated clean up tool on our servers that will resolve spurious conflicts and retrieve missing files. Further details about this process will be included in that e-mail. Note that while this tool can greatly assist those with large numbers of affected folders, we cannot guarantee that all files and folders will be restored to their original state.

    We will also soon be posting steps for performing a manual recovery to help those of you who have a small number of affected folders.

    Finally, please remember that Live Mesh is currently a Tech Preview and that you should always backup any data stored in Live Mesh.

    UPDATE: We have posted instructions for how to manually clean up affected folders here.

    Technorati Tags:
  • Live Mesh

    Got conflicts or missing files? Read this (and don't uninstall for now)

    • 14 Comments

    A few users have reported either files missing when they view Live Mesh folders on their computer (but appearing as expected on their Live Desktop) or their computer showing all files but reporting many more conflicts than expected (and their Live Desktop showing duplicate copies of any file with a conflict.) We’ve spent the weekend investigating this, and we have a few details below. But the key news is this:

    • Live Mesh is not deleting or losing files. Even if they don’t all appear in Live Mesh folders on your computer, they are all present in a hidden state, as well as on your Live Desktop if the folder is synchronized there.
    • Uninstalling the Live Mesh software from a PC will delete the files that are in a hidden state on that PC. If the folder is not synchronized on any other computer or your Live Desktop, then uninstalling will result in the hidden files no longer being available.
    • If you stopped synchronizing a folder on your PC, then re-synchronized it, that folder is likely to generate some number of false conflicts.

    We're testing a fix for this now and will deploy it as soon as we're confident it’s ready. In the meantime, if you have a Live Mesh folder that appears to be missing files (that is, synchronization appears to be complete but expected files are still not present) or one that has an unexpectedly large number of conflicts, in order to prevent data loss, do not uninstall the Live Mesh software. In addition, do not re-synchronize any folders you had previously stopped synchronizing.

     

    Why is this happening? A technical explanation

     

    In the interest of being open with you, our beloved and valued technology preview testers, we’re happy to provide more detail on what’s causing this behavior. The key to a good synchronization algorithm is understanding the history of all the files in a folder. You want to make sure that any client, even one that hasn’t synchronized in a good long while, can get an accurate list of what changed and what needs to be brought up to date. Part of a complete folder history is keeping track of files or folders that have been deleted, so that if a client comes online and tries to sync a file that doesn’t exist in the up-to-date folder list, we can tell whether it’s a new file or a file someone deleted a while back but is still on this particular client because the file deletion was never synchronized to it. We use the industry jargon tombstone for these deleted files.

     

    What we discovered recently is an interesting edge case around tombstones. Live Mesh performs its synchronization in batches of roughly 50 files at a time. The client asks for the first 50 changes, processes those, then asks for the next 50, and so on. If, through sheer chance, the next batch of 50 items is made up entirely of tombstones—that is, it’s a list of 50 files that have all been deleted since this particular client last synchronized—the storage server gets a little confused. It returns that list of tombstones and then concludes that synchronization has ended. If there are additional file changes, they never get sent to the client. So this can lead to two patterns:

    • If you’re synchronizing a folder for the first time, and that folder has this pattern of 50 tombstones in a row, then you get only a partial sync. Any files the server returned before this block of 50 tombstones show up just fine, but any files after that won’t be synchronized to the client.
    • If you have first stopped and then restarted synchronization for a particular folder, it gets a little more complicated. In this scenario, when synchronization resumes, the client ought to just quickly see which of the files have changed and get everything up to date. But if a client hits this 50-tombstone-in-a-row bug, the Live Mesh service ends up confused. The client might have tens or even hundreds of files that you and I know are up to date and part of the folder, but the server failed to tell the client about them. So the client has no choice but to conclude that you have a bunch of new files, and that’s what it tells the server. The server, of course, recognizes that files with those names already exist, and so it mistakenly thinks the client has just created a bunch of new files that unfortunately have the exact same names as files already in the folder. Result: sync conflicts, and potentially lots of them.

    Now, conflicts are an inevitable part of any synchronization system, and so both the server and client are built to understand what a conflict is and to store any conflicting files in a separate holding area, where they remain until the user decides how to resolve the conflicts. While they are in the holding area, the client software might not display the files as being part of the folder, while the Live Desktop instead represents the conflict by showing both files (yes, this is something we are already working on improving and making consistent ;). So the files are all in your mesh; it’s just not obvious where they are or how to access them.

     

    The one catch is that the client holding area is stored in the Live Mesh folder for temporary application files. This directory is something we remove when the Live Mesh software is uninstalled (also something we are already working on improving—either warning you that uninstalling will remove any files in a conflicted state or just copying all conflicts somewhere else where they won’t get removed on uninstall). So if you somehow end up in a state where the folder is present only on one client, uninstalling the Live Mesh software from that client may remove all files in a conflicted state. That’s bad, we know it’s bad, and that’s why we suggest you don’t uninstall until we fix this issue.

     

    Thanks as always for your continued participation and feedback. Ferreting out issues like this is exactly what our technical preview is designed to do, and so while it’s sometimes a bit painful, we appreciate your help and your patience as we work to get the issue fixed.

     

    Technorati Tags: Live Mesh

  • Live Mesh

    Service update: new build available (0.9.3103.14)

    • 12 Comments

    We are pleased to announce that our next update is now available. The new build contains a crucial Live Mesh folder synchronization fix – details below. By default your Live Mesh software will automatically update itself within 24 hours of the new build being posted, but you can always right-click the Notifier icon in the system tray and choose to force an immediate update.

    Fix:

    ·         Fixed issue where deleting a Live Mesh folder on one device, or stopping the folder from synchronizing on that device, would cause the folder to stop synchronizing on all devices.

    If any of your Live Mesh folders have been affected by this issue, please use the following steps to re-establish stable sync relationships for those folders:

    1.       Update the Live Mesh software on all of your device to the newest version.

    2.       If any affected folders are currently set to synchronize with your mesh, please stop those folders from synchronizing on each of your devices.

    a.       Right-click on the folder and select Live Mesh Options, then click Change Sync Settings

    b.       For each device, under Synchronize files, select Never with this device.

    3.       Once you have removed the affected folders from your mesh, you can then add them back in again by right-clicking the folder and selecting Add folder to your Live Mesh.

    Windows Vista User Account Control (UAC) reminder

    While we no longer require that UAC be enabled when running Live Mesh on Vista SP1, if you initially installed Live Mesh with UAC enabled, and then disabled it, you might have some issues upgrading. Please see the “NOTICE: Updating Live Mesh and User Account Control (UAC)” Announcement on the Live Mesh forums for further information.

    Thank you for helping us test Live Mesh!

    Technorati Tags: LiveMesh

  • Live Mesh

    Service update: new build coming today (0.9.3103.13)

    • 26 Comments

    We are pleased to announce that our next update will be available later today is now available. The new build contains a number of reliability and performance improvements; we've highlighted the key fixes below. By default your Live Mesh software will automatically update itself within 24 hours of the new build being posted, but you can always right-click the Notifier icon in the system tray and choose to force an immediate update.

    Fixes:

    • Increased the reliability of Live Mesh folder behavior in Windows Explorer.
    • Fixed a problem with the ‘Save As’ dialog in Internet Explorer when downloading video files from the Live Desktop.
    • Enabled syncing of files greater than 2GB in size.
    • Greatly increased the speed of P2P synchronization.
    • Implemented a number of reliability fixes for backend services.
    • And, of course, a bunch of additional general performance improvements.

    Windows Vista User Account Control (UAC) reminder

    While we no longer require that UAC be enabled when running Live Mesh on Vista SP1, if you initially installed Live Mesh with UAC enabled, and then disabled it, you might have some issues upgrading. Please see the “NOTICE: Updating Live Mesh and User Account Control (UAC)” Announcement on the Live Mesh forums for further information.

    And as always, thank you for all the great feedback!

     

    Technorati Tags: LiveMesh

  • Live Mesh

    Live Mesh Expansion

    • 25 Comments

    Amit Mital here – I run the Live Mesh & Developer Platform team. At Web 2.0 in April we announced Live Mesh and opened it up for people to try out our platform experience -- the first evidence of what scenarios our platform is capable of enabling.

    We said we’d tell you more about Live Mesh and give you access to the platform in the future, allowing you to build your own experiences on top of Live Mesh.  En route to opening up the developer platform we’ve been systematically updating and expanding the tech preview to help us scale out the underlying technology (see behind the mesh). We’ve enjoyed watching our service handling the load, and we want more!

    Today we are again increasing the number of available slots in the Live Mesh Technology Preview, and expanding the list of countries that will allow sign-up without a wait list to include Canada, India and Ireland! (although you still need to run with an English locale for now.) This is in addition to the availability we’ve previously announced for USA, UK, Australia and New Zealand (see coverage map).

    The experience you see today is just tip of the iceberg!  To see what the Live Mesh platform is capable of, check out the Channel 9 videos, and try out the user experience.  We’re eager for you help to keep pushing our scale, and your feedback!

    We do still have a maximum limit on the number of users we’ll allow into the Technology Preview, but as long as we’re below the limit, anyone in the countries above can sign-up today with no delays or wait list.

    Technorati Tags:
  • Live Mesh

    The potential of Live Mesh is limitless...

    • 21 Comments

    …but today’s implementation is not ;)  Our underlying platform is designed to enable a wide range of scenarios, at broad Internet scale. We closely watch the feedback from our Technology Preview users along with the health and performance of the service, using it to continually improve the system and ramp up our scale.

    As we go through this process, we’re hearing your questions about how many folders and files users can put in Live Mesh today. Here’s a quick summary of the upper bounds we’re testing against ourselves at the moment – we’ll expect these numbers to keep going up with subsequent service updates. [UPDATE: these are not limits that are hard coded into the system.  They are boundary guidelines we've established based on internal and external feedback, and we're confident that anything under these boundaries will work well for all users.  Because they are not hard coded limits, you can exceed these boundaries, and at that point your mileage may vary.  Feel free to certify and share your own success stories...and stay tuned as we will definitely increase these boundaries/guidelines as we complete further tuning and testing.]

    • Size we've currently tested for contents in a single Live Folder: 10GB 17GB 40GB (go Mesh test team go!) (of course there’s still the 5GB quota which limits how much you can synchronize with the Live Desktop)
    • Individual file size: 2GB
    • Items (file or folders) per Live Folder: 100,000
    • Members per Live Folder: 200
    • Number of Live Folders per user: 200
    • Number of devices per user: 100
    Technorati Tags:
  • Live Mesh

    Service update: new build coming today (0.9.3103.9)

    • 31 Comments

    We are pleased to announce the release of our next update, which will be is now available later today. We’re making Live Mesh available to even more people, and removing the limit on the number of invites that you have. We’ve also greatly streamlined the sign-up process. And of course we’ve made a number of general performance improvements as well. There are more details below. By default your Live Mesh client will automatically update itself within 24 hours of the new build being posted, but you can always right-click the Notifier icon in the system tray and choose to force an immediate update.

    What's new

    • More capacity: We’ve again raised the limit on the total number of customers we’ll allow to access the Live Mesh Tech Preview. Get ‘em while they’re hot!
    • Expanded availability: If your country/region is the United States, United Kingdom, Australia, or New Zealand you can sign up for Live Mesh directly, with no waiting list, by visiting www.mesh.com. We’re limiting the Tech Preview to these countries/regions for now because we want to make sure that all customers have a great experience with Live Mesh, and we’re still in the process of testing the experience for other countries/regions.
    • More invites: There is no longer any limit on the number of invitations that Live Mesh customers have for inviting others to the Tech Preview. Invite as many people as you like! (as long as they're in one of the countries/regions mentioned above, of course.)
    • Simplified waiting list: Though we’re not quite ready to accept customers into the Tech Preview from countries/regions other than the United States, United Kingdom, Australia, and New Zealand, we have greatly simplified the waiting list process for everyone else. You no longer need to sign up through Microsoft Connect. Instead, simply sign in with your Live ID on www.mesh.com, click Sign Up, and you’ll automatically be notified once Live Mesh is available in your country/region.
    • Performance tuning: We’ve continued making a number of back-end performance enhancements, including tuning the P2P synchronization channel.

    Windows Vista User Account Control (UAC) reminder

    While we no longer require that UAC be enabled when running Live Mesh on Vista SP1, if you initially installed Live Mesh with UAC enabled, and then disabled it, you might have some issues upgrading. Please see the “NOTICE: Updating Lie Mesh and User Account Control (UAC)” Announcement on the Live Mesh forums for further information.

    As always, we are continuing to improve Live Mesh with all the great feedback we’ve been receiving from our customers. Keep it coming!

    Technorati Tags:
  • Live Mesh

    Minor service update today

    • 10 Comments

    We will be rolling out a tune-up for the Live Desktop later today, July 21. There will be no update to the Live Mesh software.  You may notice a slowdown in service or some display issues while the servers update.  If you are still seeing these problems after the 21st, please report them on the forums or through Microsoft Connect.

    Technorati Tags:
  • Live Mesh

    Live Mesh still in Tech Preview, but ready for more users

    • 16 Comments

    Our forum announcement from yesterday got a lot of people talking (more).  We just updated the forum post to clarify exactly what's going on, but I want to explain here as well.  This week we did two things:

    1. Doubled the maximum number of users we'll allow to access the Live Mesh Technology Preview.
    2. Simplified the Tech Preview sign-up process.  We've removed the requirement to sign up via Microsoft Connect, so that instead you can sign up directly from www.mesh.com.

    We'll continue to listen to your feedback (keep it coming!), add features, and open up for more Live Mesh users over time.  And we'll have plenty to say to developers about the platform at PDC.

    Technorati Tags:
  • Live Mesh

    Service update: new build, new features coming today (0.9.3103.2)

    • 38 Comments

    We are delighted to announce the release of our next major update, which will be available later today is live as of now. We’ve added some new features by popular demand, as well as tuned up our performance. We’ve got the juicy details further below.  By default your Live Mesh client will automatically update itself within 24 hours of the new build being posted, but you can always right click the Notifier icon in the system tray and choose to force an immediate update.  We'll update this blog post once the update is available.

    The team has also been hard at work on our developer platform. We are on track to share that work with the world at PDC ’08 (Oct 27-30 in Los Angeles.) You can see a few of our sessions on the agenda page, including “Live Platform: New Developer Services and APIs”, “Live Platform: Building Mesh Applications” and “Live Platform: Mesh Services Architecture Deep Dive”. We’ve seen a lot of interest in our platform experience – the “app” that is the face of Live Mesh today. But Live Mesh is fundamentally a developer platform, and our goal is to enable a wide range of applications that build on our relationship directory and communications and synchronization infrastructure. If you haven’t seen Ori’s Channel 9 video on the platform, take a look to see what we’re getting at.

    New Features

    · Sync Live Mesh folders peer-to-peer only, excluding your Live Desktop. This will make it possible to synch files across your devices (or with devices of other members of a Live Folder) without consuming any storage space or quota in the Live Mesh cloud storage service. For completeness sake, though, we should mention that it’s not a 100% P2P synch – to optimize the synch experience we still use the cloud to store the authoritative metadata for the folder (for example, the file list and change history) and to broker encrypted P2P connections between clients. But the files themselves will not be stored in the cloud. Also, note that only the creator of a Live Mesh folder will have permission to change the cloud synch settings, since all the contents of a Live Mesh folder (no matter which user uploaded them) get charged against the quota of the folder’s creator.  We've got some details on how peer communications and synch work up today on Channel 9.

    · File conflicts management on the Live Desktop. Pretty much speaks for itself. The same ability to view and resolve conflicting edits to a file that you’ve had in the Windows client is now available on the Live Desktop. Actually, we think it’s an even better experience, since we took the opportunity to streamline the UX when we implemented it for Live Desktop (we’ll bring the same improvements to the client in the future.)

    · Added news events for New Live Mesh folder and Delete Live Mesh folder. We’re continuing to tune the news feed to provide the right set of information you need to keep track of what’s happening across your mesh.

    Fixes

    · Improved performance when syncing files peer-to-peer. David and Trevor get into some of the details of where we found room for improvement in today's video.

    · Better coalescing for news events. In non-geek speak, that means we’ll do a better job showing you only one news event when a user makes a bunch of changes in quick succession.

    · News performance and scale improvements. Faster and better, new and improved, it slices and dices!

    Windows Vista User Account Control (UAC) Update

    In our last client release, we removed the requirement for UAC to be enabled when running on Vista SP1.  If you originally installed the Live Mesh software with UAC enabled, then got this update and disabled UAC, you might have some issues upgrading.  We'll have an announcement on the forum shortly with details of how to fix.

    Last, a technicality for those keeping score at home -- the version number referenced in the title, 0.9.3103.2, is what you'll see in the client software.  The version number in Live Desktop will  be 0.9.3103.3, since we did a little extra work there after the client release was finalized.  Don't worry about the mismatch, this is expected and is the configuration we tested and validated before releasing.

    2008-07-14 Update: Excluding Live Desktop from synching a folder is only possible from the Live Desktop itself in this release, we're working on making it an option you can control from any device in your mesh.  But for now, the steps to exclude your Live Mesh Folder from synching with Live Desktop are:

    1. Create a Live Mesh Folder
    2. If you created it from the Live Desktop
      1. Choose to synchronize it with at least one device
      2. Wait until the folder is completely synchronized on that device
    3. Go to the Live Desktop
    4. Right click on the folder and choose “Change sync settings”
    5. Click the drop down arrow for the Live Desktop and choose “Never with this device”

    As mentioned above, keep in mind that only the creator of the folder can do this, and once the folder creator disables Live Desktop synch the files will not be available via Live Desktop for any other members of the folder.

    Technorati Tags:
  • Live Mesh

    Service update: new build released (2815.17)

    • 20 Comments

    We're continually working to fix bugs reported by our Tech Preview users, as well as implement popular feature requests (hello UAC ;). We pushed a new update out last night (version 0.9.2815.17) that includes some key fixes noted below. You'll be automatically notified of the update by your Live Mesh client software, or you can right click the Notifier icon in the system tray and choose "Update Live Mesh" to force an immediate upgrade.

    Thank you for all your great feedback, and keep it coming!  You can submit (and view others’) feedback and bugs here on the Microsoft Connect website.  Read and participate in the forums for current discussions about fixes, problems, and ways people are incorporating Live Mesh into their daily routines.  Send us private feedback using our online contact form.

    Fixes

    • Live Mesh is now available in all English-speaking countries (not just the U.S.)
    • Removed the User Account Control (UAC) requirement when installing and using Live Mesh with Windows Vista SP1 (as promised)
    • Index for Desktop Search now works with Live Mesh folders
    • Fixed bug where an underscore in a Hotmail account name returned an “Invalid Hotmail Address” error
    • Fixed bug with Silverlight 2 Beta 2 failing to load in Silverlight Media View
    • Fixed bug where the notifier tooltip incorrectly indicated that Live Mesh Remote Desktop was unavailable for a computer running in non-admin mode
    • Fixed bug where the Live Mesh folder icon was not displayed in the e-mail inviting someone to share a folder
    • Fixed one of the bugs that caused Live Mesh to fail to start
    • Fixed problem with Live Mesh returning errors when waking from sleep/hibernate
    Technorati Tags:
  • Live Mesh

    Behind Live Mesh: Authorization and encryption

    • 20 Comments

    Hi, I am Nikolai Smolyanskiy, a developer on the Accounts team. I work on the Account service that takes care of account management, and user/device authentication and authorization in Live Mesh.

    There were many questions asked by early adopters about how their data is transmitted and stored in Live Mesh, and how access is controlled. In this post I’ll talk about the security and authorization architecture behind Live Mesh, so that you understand the amount of care we take and feel better about trusting your data to the Mesh. This post can also be found here.

    Here is the diagram that illustrates all communications between user devices and Live Mesh cloud services and encryption/security mechanisms used in these communication channels:

    meshauth

    Live Mesh security is rooted at the authentication provider (Windows Live ID, aka Microsoft Passport is our provider today) which is used for initial user and device authentication. Once a user or a device is authenticated and a corresponding authentication token is obtained, the Live Mesh client passes this token to the Live Mesh Account service to access the root of the user’s mesh and to get the initial set of Live Mesh tickets. These tickets are used for further Mesh operations on other resources that this root is pointing to. All communications with the Live Mesh cloud services are done via HTTPS / SSL, so 3rd parties cannot intercept and read client-server communication.

    All user (or device) related resources in Live Mesh are organized in a RESTful manner, i.e. they form a graph where each node is identified by a unique URL and represents a given resource. Nodes contain resource metadata and links to other resources. Mesh operations are essentially CRUD operations on the nodes of the user tree or nodes of other user trees if those users shared any data. Live Mesh cloud services check access rights in each operation by inspecting passed tickets and authorizing access only if a correct set of tickets is passed. Tickets can be obtained from the Account service or from responses to previous cloud operations.

    Live Mesh authorization tickets are standard SAML tickets. They are digitally signed with the Live Mesh private key to prevent spoofing and they expire after a limited lifetime. Some tickets are used to just authenticate users or devices, other tickets contain authorization information about user/device rights. Cloud services inspect each resource request and authorize access only if it contains valid tickets (correctly signed and not expired) and these tickets specify that the requestor indeed has access to the requested resource. For example, a device X can initiate P2P data synchronization with device Y only if it presents a ticket that is correctly signed by Live Mesh and contains a record saying that both device X and Y are claimed by the same user OR if it contains a record saying that X and Y have the same Live Mesh Folder mapped on them (in the case that the devices are claimed by different users that are members of this Live Mesh Folder). Tickets are passed to the cloud services in the Authorization header using HTTPS to prevent replay attacks.

    Each device in Live Mesh (computers, PDAs, mobile phones) has a unique private key that is generated during Live Mesh installation and used to authenticate the device in P2P communications with other devices. When a P2P communication is being established between two devices, they first use asymmetric encryption (RSA algorithm) to exchange encryption keys and then use symmetric encryption (AES with 128 bit key) to transfer data/files over TCP/IP. The RSA exchange guards against leaking symmetric encryption keys. AES encryption protects actual data from prying eyes. Live Mesh also uses a keyed message authentication code (HMAC) to verify the integrity of the data exchanged on a P2P channel.

    If there is no direct connection between two devices (for example, if one device is behind a firewall), then the cloud communication relay located in the Microsoft data center is used to forward data packets from one device to another. All the traffic is encrypted in the same way as in the case with direct P2P link, i.e. first keys are exchanged with RSA and then traffic is encrypted with AES. The cloud relay cannot decrypt/read user data, since encryption keys are exchanged with the use of asymmetric encryption (RSA).

    Live Mesh cloud services help devices find each other and establish communications. They cannot read synchronized user data/files relayed through the cloud, except for the case when user files are synchronized with the cloud storage (i.e. Live Desktop). At the moment, the limited Tech Preview of Live Mesh synchronizes your files not only between your devices, but also with your cloud storage (which you can access via Live Desktop) until you reach your storage quota (5GB as of today). So your files and metadata that describes them are stored in the Microsoft datacenter. They are protected by strong access control mechanisms, but the data is not stored in encrypted form. After the storage quota has been reached, all files are synchronized only P2P and not stored in the cloud (only metadata is stored in the datacenter). In the future, Live Mesh will allow users to selectively choose which files or Live Mesh Folders they want to synchronize with the cloud. If you choose to synchronize your data/files between your devices only, Live Mesh will not store your files in the cloud and will only store metadata that lets the service to operate.

    Technorati Tags:
  • Live Mesh

    Service update: New build released

    • 20 Comments

    We've been listening to your feedback here and on the forum, and made a number of bug fixes and changes to better suit you. The update will be published later today. You may notice a slowdown in service or some display issues (missing icons, folders slow or not synchronizing) while the servers update. If you are still seeing these problems after the 20th, please report them!

    To install the new version

    · Use Live Mesh! New updates will automatically apply when you log in.

    Tell us what you think

    · Read and participate in the forums for current discussions about fixes, problems, and ways people are incorporating Live Mesh into their daily routines.

    Invite your friends and family

    · Share a folder with your family or friends so they can experience Live Mesh, too.

    What we fixed

    News

    · News not appearing until a second person has logged in.

    Language settings

    · Fixed date format and removed region requirement. You still need to set your OS language to English.

    · User names with non-US-ASCIIcharacters are now supported.

    Notifier

    · Fixed device appearing offline in the notifier but showing online on your Live Desktop.

    · Fixed synchronization failure when the network is unavailable.

    Explorer

    · Fixed the Windows Explorer crash when folders were closed

    · Fixed the Windows XP “Roaming Desktop Icons” issue with Live Mesh Remote Desktop.

    · Fixed the mesh bar not always opening when you open a Live Mesh folder

    Live Desktop

    · Fixed the “I’m interested in” in your Windows Live ID profile to be correctly viewed. No longer will you be “searching for love and romance” when you really wanted “Enterprise Networking.”

    Live Mesh

    · Fixed logs not being collected when your Windows XP login name has a space in it

    · Fixed SilverLight version error with the Beta release

    · Fixed SilverLight Play and Pause buttons being swapped around

    · Fixed login names with a single character domain (livemesh @ m.com) not working correctly

    Thank you for your time and participation!

    Technorati Tags:
  • Live Mesh

    Service update: email invitation outage

    • 0 Comments

    Yesterday (May 12th) we experienced an email service outage from approximately 12:30pm to 8:00pm PST.  If you invited anyone to share a folder during this time, they likely did not receive the invitation email.  We don't yet have support for re-sending an invitation, so for now the best option is to remove any impacted users and then re-invite them.

    To re-invite a user:

    1. Delete the pending invitation
      1. Sign in to Live Mesh at http://www.mesh.com
      2. Click on the Desktop button to view the Live Desktop
      3. Locate your folder on the Live Desktop and double click it
      4. On the right hand side of the folder view you will see the Live Mesh bar, click on Members at the bottom of the bar
      5. Confirm that the person you invited is still listed as being in the Pending state
      6. Click on Edit
      7. Select the person and then click Remove
      8. Now click OK
    2. Re-invite the member
      1. While still in the Live Desktop view of the folder that you want to invite the member to, click on Add
      2. Type the email address of the person you would like to invite to the folder
      3. Click OK

    Thank you for your understanding and for participating in the Live Mesh Tech Preview to help us make a better product.

    Technorati Tags:
  • Live Mesh

    Behind Live Mesh: What is MOE?

    • 20 Comments

    Howdy!  This is Richard Chung, a client developer for Live Mesh. You can find me blogging sporadically as DevDuck over on red hot place.  I was one of the first developers on the Live Mesh team—I’ve been working on this stuff for over 2 years now and have been itching to talk about it!

    By now, you’ve read all about Live Mesh.  Hopefully, some of it makes sense to you—you might even be able to envision how it can dramatically change the way you manage data across your devices, share and interact with your friends and family, and collaborate with peers or partners!  If not, that’s ok.  The whole concept of Live Mesh will become clearer in the months ahead, especially as more  applications and experiences are created on top of the platform.

    Software + Services

    When you were first introduced to Live Mesh, you probably played with the Live Desktop.  It’s pretty snazzy.  Maybe you even uploaded a few files too.  Hey, it’s a cool service!  You can store stuff in a cloud somewhere and access it anywhere using a webpage.  Great!

    As I look at the statistics on the service though, I notice that a significant portion of our users have stopped here.  This pains me, as there’s a whole lot more you can do with Live Mesh.  Didn’t you hear all the hoopla about Software + Services?  Ever wonder, “Where’s the software?”

    You might have noticed that on the device ring there’s a big orange button with a white ‘+’ sign.  The magic happens when you click that big orange button and opt to “add a device” to your mesh.  Click “Install” and you’ll download and install a very small program onto your machine.  Once set up, you may notice a process called MOE.exe running on your computer.  What exactly is MOE?

    MOE: Mesh Operating Environment

    MOE is an acronym for the “Mesh Operating Environment.”  MOE is a service composition runtime that provides a unified view of all Live Mesh services and a simple way for applications to interact with Live Mesh.  MOE is everywhere—it’s on all devices in your mesh (as “client MOE”), and it’s in the cloud (“cloud MOE”).  The two are quite similar; they provide a symmetrical and consistent programming model between client and cloud.  If you’re a developer, you’ll be able to take advantage of these flexible points of entry  to Live Mesh in the way that’s most natural for you.

    That’s a lot to digest.  I won’t go into the developer story today; you’ll see more of that in the months ahead.  For the purposes of this post, I’ll only focus on client MOE.  Client MOE is at the heart of every device connected to your mesh, be it your PC or Mac or mobile.

    The goals for client MOE are simple.  Here are some of them:

    · To allow seamless access to a user’s mesh and all the data in it;

    · To abstract away common functionality, like synchronization protocols;

    · To allow existing applications to participate in a user’s mesh, by optionally leveraging the local file system;

    · To provide a seamless online/offline experience (also known as “occasionally connected”) by synchronizing and caching metadata from the cloud;

    · To update seamlessly, providing new features and fixes in the platform; and

    · To use minimal system resources, running in the background all the time.

    You’ll notice that the idea of “seamlessness” really runs through these tenets.  Obviously, since what we just released is a “Tech Preview,” we have only just started towards achieving these goals today.  Rest assured that we will get better!

    Let me elaborate on a couple of these points.

    Online vs. Offline

    In today’s world, many applications just don’t work if they’re not connected to the Internet.  It’s much easier to write programs that assume connectivity to a service, than not.  But why should it be so hard?  MOE abstracts all this behavior for you.  It caches the metadata that’s in your mesh and automatically synchronizes it, so that talking to MOE is essentially like talking to the cloud.  And talking to MOE will always work, regardless of whether or not you have Internet connectivity.  MOE will do the heavy lifting for you by synchronizing changes you’ve made locally, whether you are online or offline, with your mesh (when you have internet connectivity).

    Today’s Applications

    As high as our hopes are for Live Mesh, we realize that there isn’t anything today that fully showcases the true power and potential of the mesh yet.  This is one of the reasons we’ve built simple file synchronization capability as a rich experience on top of MOE.  This way, any existing application that writes files to disk (virtually all applications) will suddenly be able to leverage Live Mesh, without *any* changes to the application.

    For example, say you use Windows Live Writer to collaboratively write blog posts, but it often takes several iterations before posting.  You can now save your blog drafts into a Live Mesh folder, and suddenly those drafts are everywhere—on each of your devices and those with whom you share the folder.  An idea for a post pops into your head on the bus ride to work?  Start a post on the bus, add content to it at work, have others read it over and revise it on their own devices, add your final touches, and post at home that very evening!  All without even thinking about e-mailing attachments back and forth.

    Working on Live Mesh

    I’ve been working on Live Mesh for over two years now.  Prior to this I was a developer on the Windows Live Messenger service.  When I first started on Live Mesh, I thought it would be a piece of cake.  After working on huge scalable services that support millions of users, how hard could writing a little lightweight client be?  It turns out that it’s quite hard.  Synchronization may appear to be an easy problem (and in fact, there are a ton of apps out there that do simple file sync already), but it’s actually really tough to solve it in a *generic* fashion so that other applications can be built on top of it.  That’s what we’ve strived for.  (We’ll have a future post on how Live Mesh uses FeedSync underneath the covers, and what types of extensibility and other advantages that provides.  Stay tuned!)

    Here, There, Everywhere

    Live Mesh is about to become your new best friend.  The one you can’t live without and is always in the background to take care of things for you.

    So what are you waiting for?  Download and install Live Mesh on all your devices today!  You’re missing out if you don’t.  And be sure to give us feedback and report any issues you see!

     

    Technorati Tags:
  • Live Mesh

    Blogs from the team

    • 3 Comments

    Several members of our team have begun posting about Live Mesh on their own personal blogs -- if you're looking for more perspective on Live Mesh, or to learn more about the team, here are a few pointers, with tantalizing excerpts!

     

    Mike Zintel (Product Unit Manager, Mesh and Storage Platform): Why Sync?

    For several decades after the invention of the computer, the dominant constraint was processor cost. While mainframes had, in relative terms, impressive storage and local I/O capacity, they were primarily designed to do useful work on every clock cycle. Indeed the humans who were running around mounting and unmounting tapes according to instructions in a job control stream were trying to keep the processor from ever executing a single very expensive NOP.

    It was very Tron. [...]

    It’s tempting to assume that web client server wins everything. But I don’t think so.

    I love the web. I’m using it now. But I still spend more time using Windows applications. At the moment I’m listening to very high quality glitch free music being streamed from my hard drive, through an iTunes client and into my headphones. I have a few folders open where I did some photo editing with Photoshop earlier today. I’m using Office to author this blog and enjoying best in class spelling and grammar checking. Messenger is open. I routinely use Premier, Lightroom, Dreamweaver, Nero, and various scanning programs.

    The industry’s newest, coolest, ease of use standard setting, cleverest mobile browser ever powered device, the iPhone, has a local application for text messaging, calendar, photo album, camera, YouTube, stocks, maps, weather, clock, calculator, notes, mail, music, the phone itself and of course, the shell. The reason for this is that local applications can have experiences that are highly tuned to the device capabilities and the preferences of the customers using them.

    Of course what is really happening it that the industry is moving toward the best of both worlds. Many of the local applications I describe above are invisibility connected to the web. This is nature of what we’ve been calling Software + Services.

     

    Roman Batoukov (Software Development Engineer): What's in your mesh?

    There are multiple perspectives to look at Live Mesh. [...]  First, I view the Mesh as a collaboration tool between individuals. [...]

    Second, I view the Mesh as a set of Devices people own or use. [...]

    Every time you add Device to a mesh, you create a new sync endpoint. You can then chose what subset of your data  you want to sync with the particular device by mapping Mesh Objects (such as Live folders) to the device.

     

    Archishmat Gore (Software Development Engineer): Live Mesh! - the appeal to a guy like me

    [...] that is what Mesh is at heart - the few guiding principles around which software and services were built. Rather than focus on "What will this service do to keep it extensible?", the Mesh team focused on "How can we represent anything and everything so that anyone can read it anywhere in the world?". The subtle difference here is the pivot point - for some, the pivot is the service, it is the hard absolute that cannot change. Everything else must be written around it to accommodate it somehow. Mesh has universally-readable data representation as it's pivot - services and software were written around it. To put it another way, the software+services implementations are a natural consequence of the design, rather than interoperability being the consequence of the implementations. Enough abstract philosophy, let's look at how these principles rendered themselves in practice

    Technorati Tags:
  • Live Mesh

    Behind Live Mesh: Live Desktop architecture

    • 12 Comments

    As the unveiling of Live Mesh is a tech preview of the platform, similarly the Live Desktop release is only a glimpse into what will be available via the web portal. The promise is a rich cross-browser web experience that requires no installations, but will offer an enhanced look-and-feel if plugins are installed. I'd like to take a moment to describe the architecture of this Live Mesh platform experience and illustrate how the design will enable it to grow into a much richer experience. Let me warn you that this is intended to be a detailed developer-focused post. The key architectural features are the generic presentation framework; Live Mesh communication layer; Live Desktop windowing system; and C# to JavaScript compilation with Script#.

    The generic presentation framework can be easily reused and extended to rapidly create performant data-driven applications. Its design resembles that of Windows Presentation Foundation in that there exist two clearly separated trees, one logical and one visual. In addition to being a natural separation for JavaScript and HTML, this has made code reuse and the addition of new features significantly faster and more seamless as logical controls can be customized with different styles for different UI components. The notable base classes in the logical tree are the UIElement and UIGroup.

    The UIElement is the base class and has a one-to-one relationship with a DOM Element, handles input events, supports command routing, and participates in databinding. The UIElement performs memory management by removing its associated visual element from the DOM and detaching its event listeners when disposed. Command routing offers a convenient way of bubbling events through the logical tree to notify parent containers that an event has occurred. Databinding offers a very flexible way to present data as a UIElement may listen for specific property change events and update its associated visual element when these occur. In our framework, Binding objects are reusable and are composed of a source data property, a target UIElement, and a Converter which contains the logic for rendering data.

    The UIGroup is a collection of UIElements and is the base class of all elements that participate in the rendering pass by the layout host. For expensive rendering events that involve more than small DOM manipulations, the UI Group will be added to the queue of items to be rendered and then its OnRender method will be called to perform the visual changes. It is worth mentioning the ItemsControl, which derives from UIGroup and represents a collection of data. The ItemsControl is instantiated with a shared visual template that is applied to render each item.

    The visual styles are defined in resource dictionaries, which are reusable html string snippets sent down in JavaScript files. DOM elements are instantiated from these strings, keyed by id. This offers performance benefits as each snippet is sent over the wire exactly once and cached by the browser.

    Let's look at a short example of how this framework is used to render the Icon View of files within a folder explorer on the Live Desktop. The files are represented by a ListBox, which derives from ItemsControl and extends it by maintaining a selected item. The style for the visual template for each item is defined in the resource dictionary as:

     

    <div id=""$spaceIconItem""> 
        <img id=""iconItemImg"" /> 
        <div id=""folderViewNameCellLabel""></div> 
    </div>

     

    The image represents an icon specific to mime type and the label represents the name of that file. When the visual template is executed to render an individual item, it sets up databinding for each of these DOM Elements:

    image.SetBinding( 
       "Src",  CommonBindings.MimeTypeToMediumIconOneTimeOneWay); 
    label.SetBinding( 
       "Text", CommonBindings.NameOneWay);

     

    The image source databinding will be executed only once, and will set the src property of the img to be that of the appropriate icon url. The label text will be updated whenever the name of the file changes and will modify the InnerText of the div. The ListBox will be added to the rendering queue of the layout host whenever the collection of files changes, for example by navigating into a new folder, and will instantiate each of its DOM children by invoking the above visual template.

    For completeness in discussing the presentation framework, it is important to note that each collection of data in memory is stored in an ObservableCollection, which maintains a list of items and fires unique change events when items are added, items are removed, or the entire collection changes. This enables an ItemsControl to intelligently decide whether to render an individual data item or the entire collection.

    The Live Mesh communication layer updates the ObservableCollections by performing incremental data requests on corresponding data feeds on a recurring interval. This layer also performs the http operations that result from user actions. It maintains a queue of operations and associated callbacks. It understands the Live Mesh cloud API and provides a layer of abstraction between app-specific transactions and the semantics of the Live Mesh platform.

    With all of the above infrastructure in place, it was straightforward to build a windowing system into the Live Desktop. The list of windows is maintained as an ObservableCollection. There are three different templates applied to render each window depending on whether it is in the taskbar, on the desktop, or in the window toggle control (shift+tab to see this view). Activating a window simply means setting the ActiveWindow property true, as the css class of the DOM Element is bound to that property. Opening or closing a window will fire a granular change event from the ObservableCollection that the ItemsControl listens for and will respond to by adding or removing the individual item from view.

    The architectural complexity and heavy usage of inheritance described above were greatly facilitated by Script#, a C# to JavaScript compiler developed by Nikhil Kothari. The strong IDE support from Visual Studio and compile-time syntax checking that this offers have been crucial as the Live Desktop codebase has grown.

    What all this means is that the foundation has been laid to add many more exciting features to your Live Desktop. When you view it today, you should see a visually appealing view of a desktop that works well cross-browser, performs responsively once loaded (and will soon load faster), and enables you to traverse your cloud. You might also think to yourself "I wish I could upload more than one file at once" or "I wish this SilverLight media view wasn't one-size fits all and only worked on a subset of my data". The good news is that these features and many more are planned and that the Live Desktop architecture will facilitate adding these new components rapidly. While we have a clear set of features that we think will make this offering more compelling, we are eager to hear ideas from you.

    - Alex Himel, developer for Live Desktop

    Technorati Tags:
  • Live Mesh

    Live Mesh service upgrade

    • 1 Comments

    We started deploying an update a few hours ago which is causing some temporary website issues. You might notice icons or buttons missing when visiting http://www.mesh.com or accessing your Live Desktop. These issues will disappear as soon as we’ve completed the servicing process.

    If you were using Live Mesh between 12:30pm PST and 2:30 pm PST time and are experiencing these issues, deleting your temporary internet files should fix things back up. Here's how to do that using Internet Explorer and Firefox.

    Today’s update includes:

    · Performance and reliability fixes based on Tech Preview customer feedback.

    · Fixes to the Live Desktop Silverlight player to address common customer reported errors.

    If you are still experiencing issues, please report them to the Live Mesh Forum and we will investigate shortly.

    Thank you for your participation and feedback,

    The Live Mesh Team

     [Update 6:15pm PST: the upgrade has completed]

  • Live Mesh

    Behind Live Mesh: How we run cloud services

    • 10 Comments

    A quick self-introduction: I’m Alex Mallet, one of the development leads on the Live Mesh project. I’ve been at Microsoft since ’97, except for an abortive [but instructive] side trip to graduate school in an attempt to get a PhD in computational biology. Just about all of my time has been spent working on distributed systems, of gradually increasing scale – I started out working on IIS, moved to Application Center 2000, worked on our P2P API toolkit and finally ended up on the Live Mesh team about a year and a half ago.  On Live Mesh, my team and I are responsible for making sure our datacenter services are easy to deploy and manage, and for providing common functionality needed by our cloud services. So, on the heels of the previous blog posts that have introduced the “big picture” view, I thought I’d give you a bit more insight into some of the details of the “services” part of “Software + Services”, by talking about our services that run in the cloud.

    Our general philosophy when building our cloud services was to adhere to the tenets of Recovery-Oriented Computing (ROC): programs will crash, hardware will fail, and they will do so regularly, so your system should be prepared to deal with these failures. While it’s easy to espouse these principles in theory, the obvious next question is how to turn them into practice, and here we were aided by a great “best practices” survey paper written by James Hamilton, namely “On Designing and Deploying Internet-Scale Services”.  I won’t claim that we managed to do everything that’s in his paper [we’re only at the Tech Preview stage, after all J], but I think we’ve done a decent job so far, and are heading in the right direction overall.

    Enough philosophy, on to some more detail.

    From a functionality perspective, our cloud services can be grouped into four buckets: dealing with feed and data synchronization, providing authentication and authorization, maintaining and fanning out the system’s transient state [like the various notifications provided in the Live Mesh Bar], and the connectivity services for synchronization and remote desktop access to work across any network topology. Sliced along the “state” axis, we have stateless front-end services, back-end services that maintain in-memory state, and persistent storage layers that handle both structured and unstructured data. From a scaling perspective, our plan is to scale out, not up. Thus, we’ve invested in making sure that we have as many stateless services as possible, as well as having facilities that allow us to partition our state [both persistent and transient] across multiple machines, and reconfigure these partitions as necessary. Overall, we have close to 20 different services, with each service consisting of multiple, redundant instances of a particular bit of code, striped across several racks of machines in the datacenter – in keeping with the ROC assumptions, our goal is to be resilient to multiple hardware and software failures.                

    Our front-end services are accessible [only] via HTTPS – all of the traffic that flows in and out of our system is encrypted. Our back-end services use a mixture of HTTPS and custom protocols layered on top of TCP. The vast majority of the services are written in C#, with the only exceptions being services that needed deep integration with Windows functionality that isn’t [easily] accessible to an application written in managed code.

    All of our services sit on top of a runtime library that contains facilities commonly needed by each service: process lifetime management, HTTP and TCP listeners, a debug logging facility, a work queue facility, APIs to generate monitoring data like performance counters, etc. This common runtime also contains debugging, testing and monitoring hooks; for example, we have the ability to inject random delays and failures into our HTTP pipeline, which allows us to test our failure monitors and the overall response of the system to slow and failing services.

    Building a full-scale datacenter deployment and management system is a huge undertaking, so we chose the lazy smart route and went with an existing, battle-tested system, namely the Autopilot framework, which was developed, and is being used, by the Windows Live Search team to manage their tens of thousands of datacenter machines [we’re not at that scale yet, but we hope to be, with your help J]. We use Autopilot to manage our code and data deployments, for [some of our] failure monitoring and self-healing, and to give us insight into the current state of our datacenter machines and services.

    On the monitoring front, we actually monitor the system at several levels – via simple Autopilot-style watchdogs, with more extensive tests called “runners”, by hitting our service from various points outside our datacenter, and also using a variety of tools that scan our logs for error messages, highlight machines that appear to be having problems, look for crashing services etc. Of course, all these monitors are still somewhat untested – I’m sure we’ll be making lots of tweaks, and adding new tools over the coming weeks and months as we start having to troubleshoot and keep a real live system up and running. J

    Ok, I think that’s enough for one post. If this is a topic of interest to you, and you’d like more detail on some of the stuff I’ve talked about, please leave suggestions and questions in the comments, and I’ll address them in follow-up posts.

    And, of course, don’t forget to sign up for Live Mesh and give us feedback !

    Technorati Tags:
  • Live Mesh

    Get Mesh (and why we require UAC)

    • 21 Comments

    I’m John Macintyre, the Group Program Manager for Live Mesh Client Platform & Runtime (it fits on a business card if you use 6 point type).  I wanted to talk about how Live Mesh client install works, and address the questions we’ve seen about why User Account Control (UAC) must be enabled on Windows Vista in order for Live Mesh to work.

    The design challenge we faced with Live Mesh client deployment was to make it simple to install, while providing rich client experiences that are deeply integrated with the device experience i.e. today your Windows Shell, tomorrow your Mac, Mobile device etc. 

    It’s worth taking a quick look at the client footprint before we cover how it is deployed. The Live Mesh client can be separated into two distinct components:

    1. Mesh Operating Environment (MOE) – this is the client version of our services composition runtime, a cohesive programming model for interacting with the mesh.  On the client, it is responsible for data synchronization, cloud interaction, P2P interaction and handling requests from applications.
    2. Live Mesh Client – this is the set of core experiences that are built on the runtime to deliver experiences around Live Mesh Folders and Live Mesh Remote Desktop.  

    There is a further subcomponent breakdown but at a conceptual level you can think of the client as a runtime engine with an experience layered on top of it.  Today the client is deployed as a single unit but in subsequent refreshes we will enable other configuration and distribution scenarios.

    We’ve tried to keep the client deployment model as seamless and simple as possible.  It can be broken into three separate stages …

    1) Add Device …

    The Live Mesh client is deployed through the Devices page of www.mesh.com.  Clicking on “Add Device” will initiate the download of the Live Mesh installer.  The first thing that you may notice is the installer is relatively small.  The initial install of the Live Mesh client is actually just a bootstrap component that reaches out to a server and retrieves the latest version of the client.  This provides the benefit of small download from the browser and an always up-to-date client footprint.

    2) Run LiveMesh.exe …

    Once the bootstrap installer has been downloaded, users click on ‘run’ to launch the installer.  Live Mesh installs on the system as “per user” which means that installation only applies to the current Windows user. So by default, installing on Vista does not require administrator privileges, and no UAC prompt is shown.   The runtime client binaries are deployed to the user’s profile under application data and all COM objects are registered for the current user.  This means that installations are not shared across Windows accounts.  Another interesting aspect of install is that the user is not presented with a terms of use or confirmation dialog since this was already done through service sign-up.  We hope this simplifies the overall flow and experience around client deployment.

    Why does Live Mesh require UAC?

    At this point it’s worth talking about UAC and why the Live Mesh client requires that UAC be turned on. If you’ve tried to install Live Mesh on a system with UAC disabled you’ll have found that the installer blocks on this configuration.  There’s a technical reason why we impose this restriction:  In Vista RTM with UAC off, COM does not read the per-user hive, meaning COM objects registered per-user do not work.  Since Live Mesh installs per-user without elevation it is subject to this restriction.  To prevent users from installing on unsupported configurations we added a block in our installer when UAC is off.  Fortunately there’s light at the end of the tunnel for users that run with UAC disabled.  Windows Vista SP1 has a change that allows per-user COM when UAC is disabled.  With the release of SP1, we have begun validating that Live Mesh fully functions with UAC disabled, but haven't yet finished our testing.  Expect this configuration to be supported in a subsequent refresh.   As you can probably guess, this also means that even after you install, UAC must remain enabled for Live Mesh to work correctly.  If you turn UAC off, we won’t be able to find any of our COM objects.  

    3) Add Device …

    After installing the Live Mesh client, users are prompted to sign-in with their Live ID.  This Live ID is used to link the device to your personal mesh.  By “device” we generally mean Windows user account.  Multiple Live IDs can independently use the same Windows account (similar to Messenger).  Once the device is part of your mesh you can initiate sharing (or synchronization) with your other devices, the cloud or other users.
    Vista users will see an additional option on the Add Device dialog.  This option is to enable features that require administrator privileges including system level Live Mesh Remote Desktop (accessing the system while it is locked or an account is not logged in) and peer to peer file transfer.  By default this option is selected and users will be asked to elevate to proceed with the device claim.  If the user is not an administrator they can disable this option.

    Once the user goes through this client deployment flow they are up and running with their mesh-enabled device.  The runtime and experience components will continue to update themselves as new versions are available.  As much as possible we try to do this without affecting the overall experience or disturbing the user.

    Technorati Tags:
  • Live Mesh

    Web 2.0 keynote video posted

    • 5 Comments

    The Web 2.0 folks have posted a recording of Amit's keynote from Wednesday night.  It's a 10-minute "higher order bit" session.  Check it out at http://blip.tv/file/854328 .

     

    Technorati Tags:
Page 2 of 3 (55 items) 123