LucidWorks Search on Windows Azure delivers a high-performance search service based on Apache Lucene/Solr open source indexing and search technology. This service enables quick and easy provisioning of Lucene/Solr search functionality on Windows Azure without any need to manage and operate Lucene/Solr servers, and it supports pre-built connectors for various types of enterprise data, structured data, unstructured data and web sites.
In June, we shared an overview of the LucidWorks Search service for Windows Azure. For this post, the first in a series, we’ll cover a few of the concepts you need to know to get the most out of the LucidWorks search service on Windows Azure. In future posts we’ll show you how to set up a LucidWorks service on Windows Azure and demonstrate how to integrate search with Web sites, unstructured data and structured data.
Options for Developers
Developers can add search to their existing Web Sites, or create a new Windows Azure Web site with search as a central function. For example, in future posts in this series, we’ll create a simple Windows Azure web site that will use the LucidWorks search service to index and search the contents of other Web sites. Then we’ll enable search from the same demo Web site against a set of unstructured data and MySQL structured data in other locations.
Overview: Documents, Fields, and Collections
LucidWorks creates an index of unstructured and structured data. Any individual item that is indexed and/or searched is called a Document. Documents can be a row in a structured data source or a file in an unstructured data source, or anything else that Solr/Lucene understands.
An individual item in a Document is called a Field. Same concept – fields can be columns of data in a structured source or a word in an unstructured source, or anything in between. Fields are generally atomic, in other words they cannot be broken down into smaller items.
LucidWorks calls groups of Documents that can be managed and searched independently of each other Collections. Searching, by default is on one collection at a time, but of course programmatically a developer can create search functionality that returns results for more than one Collection.
Security via Collections and Filters
Collections are a great way to restrict a group of users, controlled by access to Windows Azure Web sites and by LucidWorks. In addition, LucidWorks Admins can create Filters inside a Collection. User identity can be integrated with an existing LDAP directory, or managed programmatically via API.
LucidWorks additional Features
LucidWorks adds value to Solr/Lucene with some very useful UI enhancements that can be enabled without programming.
Persistent Queries and Alerts, Auto-complete, spellcheck and similar terms.
Users can create their own persistent queries. Search terms are automatically monitored and Alerts are delivered to a specified email address using the Name of the alert as the subject line. You can also specify how often the persistent query should check for new data and how often alerts are generated.
Search term Typeahead can be enabled via LucidWorks’ auto-complete functionality. Auto-complete tracks the characters the user has already entered and displays terms that start with those characters.
When results re displayed, LucidWorks can spell-check queries and offer alternative terms based on similar spellings of words and synonyms in the query.
Stopwords
Search engines use Stopwords to remove common words from queries and query indexes like “a”, “and”, or “for” that add no value to searches. LucidWorks has an editable list of Stopwords that is a great start to increase search relevance.
Increasing Relevance with Click Scoring
Click scoring tracks common queries and query results and tracks which results are most often selected against query terms and scores relevance based on the comparison results. Results with a higher relevance are placed higher in search result rankings, based on user activity.
LucidWorks on Windows Azure – Easy Deployment
The best part of LucidWorks is how easily Enterprise Search can be added as a service. In our next LucidWorks blog post we’ll cover how to quickly get up and running with Enterprise search by adding a LucidWorks service to an existing Windows Azure Web site.
Portability and interoperability of virtualization technologies across platforms using Linux and Windows virtual machines are important to Microsoft and to our customers.
To that end, System Center VMM continues to gain valuable interoperability and portability experience using Open Virtualization Format (OVF) with their OVF Export/Import tool and partners such as Citrix and VMware.
For more information, see System Center's most recent post from Cheng Wei and that of Citrix's technical architect Shishir Pardikar.
Monica MartinSenior Program ManagerMicrosoft Open Technologies, Inc.
Here’s the latest about Node.js on Windows. Last week, working closely with the Node.js core team, we checked into the open source Node.js master branch the code to add support for ETW and Performance Counters on Windows. These new features will be included in the new V0.10 when it is released. You can download the source code now and build Node.js on your machine if you want to try out the new functionality right away.
Developers need advanced debugging and performance monitoring tools. After working to assure that Node.js can run on Windows, our focus has been to provide instrumentation features that developers can use to monitor the execution of Node applications on Windows. For Windows developers this means having the ability to collect Event Tracing for Windows ® (ETW) data and use Performance Counters to monitor application behavior at runtime. ETW is a general-purpose, high-speed tracing facility provided by the Windows operating system. To learn more about ETW, see the MSDN article Improve Debugging And Performance Tuning With ETW.
With ETW, Node developers can monitor the execution of node applications and collect data on key metrics to investigate and performance and other issues. One typical scenario for ETW is profiling the execution of the application to determine which functions are most expensive (i.e. the functions where the application spends the most time). Those functions are the ones developers should focus on in order to improve the overall performance of the application.
In Node.js we added the following ETW events, representing some of the most interesting metrics to determine the health of the application while it is running in production:
For Node.js ETW events we also added some additional information about the JavaScript track trace at the time the ETW event was generated. This is important information that the developer can use to determine what code has been executed when the event was generated.
Most Node developers are familiar with Flamegraphs, which are a simple graphical representation of where time is spent during application execution. The following is an example of a Flamegraph generated using ETW.
For Windows developers we built the ETWFlamegrapth tool (based on Node.js) that can parse etl files, the log files that Windows generates when ETW events are collected. The tool can convert the etl file to a format that can be used with the Flamegraph tool that Brendan Gregg created.
To generate a Flamegraph using Brendan’s tool, you need to follow the simple instructions listed in the ETWFlamegraph project page on Github. Most of the steps involve processing the ETW files so that symbols and other information are aggregated into a single file that can be used with the Flamegraph tool.
ETW relies on a set of tools that are not installed by default. You’ll either need to install Visual Studio (for instance, Visual Studio 2012 installs the ETW tools by default) or you need to install the latest version of the Windows SDK tools. For Windows 7 the SDK can be found here.
To capture stack traces:
To extract the stack for process node.exe and fold the stacks into perf.csv.fold, this includes all information about function names that will be shown in the Framegraph.
node etlfold.js perf.csv node.exe. (etlfold.js is the file found in the ETWFlamegraph project on GitHub).
Then run the flamegraph script (requires perl) to generate the svg output:
flamegraph.pl perf.csv.fold > perf.svg
If the Node ETW events for JavaScript symbols are available then the procedure becomes the following.
The remaining steps are the same as in the previous example.
Note: for more advanced scenarios where you may want to have stack traces that include the Node.js core code executed at the time the event is generated, you need to include node.pdb (the debugging information file) in the symbol path so the ETW tools can resolve and include them in the Framegraph.
In addition to ETW, we also added Performance Counters (PerfCounters). Like ETW, Performance counters can be used to monitor critical metrics at runtime, the main differences being that they provide aggregated data and Windows provides a great tool to display them. The easiest way to work with PerfCounters is to use the Performance monitor console but PerfCounters are also used by System Center and other data center management applications. With PerfCounters a Node application can be monitored by those management applications, which are widely used for instrumentation of large cloud and enterprise-based applications.
In Node.js we added the following performance counters, which mimic very closely the ETW events:
All Node.js performance counters are registered in the system so they show up in the Performance Monitor console.
While the application is running, it’s easy to see what is happening through the Performance Monitor console:
The Performance Monitor console can also display performance data in a tabular form:
Collecting live performance data at runtime is an important capability for any production environment. With these new features we have given Node.js developers the ability to use a wide range of tools that are commonly used in the Windows platform to ensure an easier transition from development to production.
More on this topic very soon, stay tuned.
Claudio Caldato Principal Program Manager Lead Microsoft Open Technologies, Inc.
From:
Erik Meijer, Partner Architect, Microsoft Corp.
Claudio Caldato, Principal Program Manager Lead, Microsoft Open Technologies, Inc.
There is much more to cloud computing than running isolated virtual machines, yet writing distributed systems is still too hard. Today we are making progress towards easier cloud computing as ActorFX joins the Microsoft Open Technologies Hub and announces its first, open source release. The goal for ActorFx is to provide a non-prescriptive, language-independent model of dynamic distributed objects, delivering a framework and infrastructure atop which highly available data structures and other logical entities can be implemented.
ActorFx is based on the idea of the Actor Model developed by Carl Hewitt, and further contextualized to managing data in the cloud by Erik Meijer in his paper that is the basis for the ActorFx project − you can also watch Erik and Carl discussing the Actor model in this Channel9 video.
What follows is a quick high-level overview of some of the basic ideas behind ActorFx. Follow our project on CodePlex to learn where we are heading and how it will help when writing the new generation of cloud applications.
At a high level, an actor is simply a stateful service implemented via the IActor interface. That service maintains some durable state, and that state is accessible to actor logic via an IActorState interface, which is essentially a key-value store.
There are a couple of unique advantages to this simple design:
The essence of the ActorFx model is captured in two interfaces: IActor and IActorState.
IActorState is the interface through which actor logic accesses the persistent data associated with an actor, it is the interface implemented by the “this” pointer.
public interface IActorState { void Set(string key, object value); object Get(string key); bool TryGet(string key, out object value); void Remove(string key); Task Flush(); // "Commit" }
By design, the interface is an abstract key-value store. The Set, Get, TryGet and Remove methods are all similar to what you might find in any Dictionary-type class, or a JavaScript object. The Flush() method allows for transaction-like semantics in the actor logic; by convention, all side-effecting IActorState operations (i.e., Set and Remove) are stored in a local side-effect buffer until Flush() is called, at which time they are committed to the durable store (if the implementation of IActorState implements that).
An ActorFx actor can be thought of as a highly available service, and IActor serves as the computational interface for that service. In its purest form, IActor would have a single “eval” method:
public interface IActor { object Eval(Func<IActorState, object[], object> function, object[] parameters); }
That is, the caller requests that the actor evaluate a delegate, accompanied by caller-specified parameters represented as .NET objects, against an IActorState object representing a persistent data store. The Eval call eventually returns an object representing the result of the evaluation.
Those familiar with object-oriented programming should be able to see a parallel here. In OOP, an instance method call is equivalent to a static method call into which you pass the “this” pointer. In the C# sample below, for example, Method1 and Method2 are equivalent in terms of functionality:
class SomeClass { int _someMemberField; public void Method1(int num) { _someMemberField += num; } public static void Method2(SomeClass thisPtr, int num) { thisPtr._someMemberField += num; } }
Similarly, the function passed to the IActor.Eval method takes an IActorState argument that can conceptually be thought of as the “this” pointer for the actor. So actor methods (described below) can be thought of as instance methods for the actor.
In practice, passing delegates to actors can be tedious and error-prone. Therefore, the IActor interface calls methods using reflection, and allows for transmitting assemblies to the actor:
public interface IActor { string CallMethod(string methodName, string[] parameters); bool AddAssembly(string assemblyName, byte[] assemblyBytes); }
Though the Eval method is still an integral part of the actor implementation, it is no longer part of the actor interface (at least for our initial release). Instead, it has been replaced in the interface by two methods:
There are two ways to define actor methods:
(1) Define the methods directly in the actor service, “baking them in” to the service.
myStateProvider.Set( "SayHello", (Func<IActorState, object[], object>) delegate(IActorState astate, object[] parameters) { return "Hello!"; });
(2) Define the methods on the client side.
[ActorMethod] public static object SayHello(IActorState state, object[] parameters) { return "Hello!"; }
You would then transport them to the actor “on-the-fly” via the actor’s AddAssembly call.
All actor methods must have identical signatures (except for the method name):
Additionally, actor methods defined on the client side and transported to the actor via AddAssembly must be decorated with the “ActorMethod” attribute, and must be declared as public and static.
We wanted to be able to provide subscription and publication support for actors, so we added these methods to the IActor interface:
public interface IActor { string CallMethod(string clientId, int clientSequenceNumber, string methodName, string[] parameters); bool AddAssembly(string assemblyName, byte[] assemblyBytes); void Subscribe(string eventType); void Unsubscribe(string eventType); void UnsubscribeAll(); }
As can be seen, event types are coded as strings. An event type might be something like “Collection.ElementAdded” or “Service.Shutdown”. Event notifications are received through the FabricActorClient.
Each actor can define its own events, event names and event payload formats. And the pub/sub feature is opt-in; it is perfectly fine for an actor to not support any events.
If you wanted your actor to support counter semantics, you could implement an actor method as follows:
[ActorMethod] public static object IncrementCounter(IActorState state, object[] parameters) { // Grab the parameter var amountToIncrement = (int)parameters[0]; // Grab the current counter value int count = 0; // default on first call object temp; if (state.TryGet("_count", out temp)) count = (int)temp; // Increment the counter count += amountToIncrement; // Store and return the new value state.Set("_count", count); return count; }
Initially, the state for the actor would be empty.
After an IncrementCounter call with parameters[0] set to 5, the actor’s state would look like this:
Key
Value
“_count”
5
After another IncrementCounter call with parameters[0] set to -2, the actor’s state would look like this:
3
Pretty simple, right? Let’s try something a little more complicated.
For a slightly more complicated example, let’s consider how we would implement a stack in terms of actor methods. The code would be as follows:
[ActorMethod] public static object Push(IActorState state, object[] parameters) { // Grab the object to push var pushObj = parameters[0]; // Grab the current size of the stack int stackSize = 0; // default on first call object temp; if (state.TryGet("_stackSize", out temp)) stackSize = (int)temp; // Store the newly pushed value var newKeyName = "_item" + stackSize; var newStackSize = stackSize + 1; state.Set(newKeyName, pushObj); state.Set("_stackSize", newStackSize); // Return the new stack size return newStackSize; } [ActorMethod] public static object Pop(IActorState state, object[] parameters) { // No parameters to grab // Grab the current size of the stack int stackSize = 0; // default on first call object temp; if (state.TryGet("_stackSize", out temp)) stackSize = (int)temp; // Throw on attempt to pop from empty stack if (stackSize == 0) throw new InvalidOperationException( "Attempted to pop from an empty stack"); // Remove the popped value, update the stack size int newStackSize = stackSize - 1; var targetKeyName = "_item" + newStackSize; var retrievedObject = state.Get(targetKeyName); state.Remove(targetKeyName); state.Set("_stackSize", newStackSize); // Return the popped object return retrievedObject; } [ActorMethod] public static object Size(IActorState state, object[] parameters) { // Grab the current size of the stack, return it int stackSize = 0; // default on first call object temp; if (state.TryGet("_stackSize", out temp)) stackSize = (int)temp; return stackSize; }
To summarize, the actor would contain the following items in its state:
After the items “foo”, “bar” and “spam” had been pushed onto the stack, in that order, the actor’s state would look like this:
“_stackSize”
“_item0”
“foo”
“_item1”
“bar”
“_item2”
“spam”
A pop operation would yield the string “spam”, and leave the actor’s state looking like this:
2
Once you have actors up and running in the Actor Runtime, you can connect to those actors and manipulate them via use of the FabricActorClient. This is the FabricActorClient’s interface:
public class FabricActorClient { public FabricActorClient(Uri fabricUri, Uri actorUri, bool useGateway); public bool AddAssembly(string assemblyName, byte[] assemblyBytes, bool replaceAllVersions = true); public Object CallMethod(string methodName, object[] parameters); public IDisposable Subscribe(string eventType, IObserver<string> eventObserver); }
When constructing a FabricActorClient, you need to provide three parameters:
The AddAssembly method allows you to transport an assembly to the actor. Typically that assembly would contain actor methods, effectively add behavior to or changing the existing behavior of the actor. Take note that the “replaceAllVersions” parameter is ignored.
What’s next?
This is only the beginning of a journey. The code we are releasing today is an initial basic framework that can be used to build a richer set of functionalities that will make ActorFx a valuable solution for storing and processing data on the cloud. For now, we are starting with a playground for developers who want to explore how this new approach to data storage and management on the cloud can become a new way to see old problems. We will keep you posted on this blog and you are of course more than welcome to follow our Open Source projects on our MSOpenTech CodePlex page. See you there!
At Microsoft Open Technologies, Inc., we’re happy to share the news that Single Sign-on of Drupal Web sites hosted on Windows Azure with Windows Live IDs and / or Google IDs is now available. Users can now log in to your Drupal site using Windows Azure's WS-Federation-based login system with their Windows Live or Google ID. Simple Web Tokens (SWT) are supported and SAML 2.0 support is currently planned but not yet available.
Setup and configuration is easy via your Windows Azure account administrator UI. Setup details are available via the Drupal project sandbox here. Full details of setup are here.
Under the hood, WS-Federation is used to identify and authenticate users and identity providers. WS-Federation extends WS-Trust to provide a flexible Federated Identity architecture with clean separation between trust mechanisms (In this windows Live and Google), security token formats (In this case SWT), and the protocol for obtaining tokens.
The Windows Azure Authentication module acts as a relying party application to authenticate users. When downloaded, configured and enabled on your Drupal Web site, the module:
-Makes a request via the Drupal Web site for supported identity providers -Displays a list of supported identity providers with Authentication links -Provides return URL for authentication, parsing and validating the returned SWT -Logs the user in or directs the user to register
-Makes a request via the Drupal Web site for supported identity providers
-Displays a list of supported identity providers with Authentication links
-Provides return URL for authentication, parsing and validating the returned SWT
-Logs the user in or directs the user to register
Adalberto Foresti, Principal Program Manager, Microsoft Open Technologies, Inc. Scott Blomquist, Senior Development Engineer, Microsoft Open Technologies, Inc.
It’s great to see that the W3C Pointer Events Working Group has expanded its membership and published the first working draft last week in the process to standardize a single input model across all types of devices. To further contribute to the technical discussions, today Microsoft Open Technologies, Inc., published an early open source HTML5 Labs Pointer Events prototype of the W3C Working Draft for WebKit. We want to work with the WebKit developer community to enhance this prototype. Over time, we want this prototype to implement all the features that will be defined by the W3C Working Group’s Pointer Events specification. The prototype will help with interoperability testing with Internet Explorer.
The Web today is fragmented into sites designed for only one type of input. The goal of a Pointer Events standard is to help Web developers to only need to code to one pointer input model across all types of devices and to have that code work across multiple browsers. Google, Microsoft, Mozilla, Nokia and Zynga are among the industry members working to solve this problem in the W3C Pointer Events WG.
Microsoft submitted the Pointer Events specification to the W3C just three months ago. The working group is using Microsoft’s Member submission as a starting point for the specification, which is based on the APIs available today in IE10 on Windows 8 and Windows Phone 8.
Our team developed this Pointer Events prototype of the W3C Working Draft for WebKit as a starting point for testing interoperability between Internet Explorer and WebKit in this space. As we have done in the past on HTML5 Labs, the prototype intends to inform discussions and provide information grounded on implementation experience. Please provide feedback on this initial implementation in the comments of this blog and in the WebKit mailing lists. We also would love to get some advice on how/when to submit this patch to the main WebKit trunk.
Overall, we believe that we are on a solid path forward in this standardization process. In a short time, we have a productive working group, a first W3C Working Draft specification, and an early proof of concept for WebKit that should provide valuable insights. We’re looking forward to working closely with the community to develop this open source code in WebKit so we can start testing interoperability with Internet Explorer.
In June, we shared an overview of the LucidWorks Search service for Windows Azure, and in our first post in this series we provided more detail on features and benefits. For this post, we’ll start with the main feature of LucidWorks – quickly creating a LucidWorks instance by selecting LucidWorks from the Azure Marketplace and adding it to an existing Azure Instance. It takes a few clicks and a few minutes.
LucidWorks Search is listed under applications in the Windows Azure Marketplace. To set up a new instance of LucidWorks on Windows Azure, just click on the Learn More button:
That takes you to the LucidWorks Account Signup Page. From here, you select a plan, based on the type of storage being used and the number of documents to index. There are currently four plans available: Micro, which has no monthly fee, Small and Medium, which have pre-set fees, and Large, which is negotiated directly with LucidWorks based on several parameters. All of the account levels have fees for overages, and the option to move to the next tier is always available via the account page.
The plans are differentiated on document limits in indexes, the number of queries that can be performed per month, the frequency that indexes are updated, and index targets. Index targets are the types of content that can be indexed – for a Micro, only Websites can be indexed, for small and large, files, RDBMS, and XML content can also be indexed. For large instances ODBC data drivers can be used to make content available to indexes.
Once the plan is selected, enter your information, including Billing Information:
Once the payment is processed (Or in the case of Micro, no payment), a new instance is generated and you’re redirected to an account page, and invited to start building collections!
In the next part of the series we’ll cover setting up collections in more detail, for now let’s cover the account settings and configuration. Here’s the main screen for collections:
The first thing you see is the Access URL options. You can access your collections via Solr or REST API, and here’s where you get the predefined URL for either. When you drill down into the collections you see a status screen first:
This shows you the index size and stats about modification, queries per second, and updates per second, displayable by the last hour, day or week. This screen is also where you can see the most popular queries.
If you were managing external data sources, here’s where you configure them, via the Manage Data Sources button.
From here you can select a new data source from the drop-down. The list in this drop-down is as of this writing, and may change over time – check here for more information on currently supported data sources.
The Indexing Settings are the next thing to manage in your LucidWorks on Azure account. Here’s the Indexing UI:
De-duplication manages how duplicate documents are handled. (As we discussed in our first post, any individual item that is indexed and/or searched is called a document.) Off ignores duplicates, Tag identifies duplicates with a unique tag, and Overwrite replaces duplicate documents with new documents when they are indexed. Remember that de-duplication only applies to the indexes of data, not the data itself – only the indexed reference to the document is de-duplicated – so duplicates will still exist in the source data even if data in the indexes has been de-duplicated. Duplicates are determined based on key fields that you set in the fields editing UI.
Default Field Type is used for setting the type of data for fields whose type LucidWorks cannot determine using its built-in algorithms.
Auto-commit and Auto-soft commit settings determine when the index will be updated. Max time is how long to wait before committing, and max docs is how many documents are collected before a commit. Soft commits are used for real time searching, while regular commits manage the disk-stored indexes.
Activities manage the configuration of indexes, suggested autocomplete entries, and user result click logging.
Full documentation of indexing settings can be found here.
Field Settings allow configuration of each field in the index. Fields displayed below are automatically defined by data extraction and have been indexed:
Field types defined by LucidWorks have been optimized for most types of content, and should not generally be changed. The other settings need to be configured once the index has run and defined your fields:
For example, a URL field would be a good candidate for de-duplication, and you may want to index it for autocomplete as well. You can also indicate on Field Settings whether you want to display URLs in search results. Here is full documentation of Field Settings.
Dynamic Fields are almost the same as fields, but are created or modified when the index is created. For example, adding a value before or after a field value, or adding one or more fields together to form a single value.
Field Types is where you add custom field types in addition to the default field types created by your LucidWorks installation.
Schedules is where you add and view schedules for indexing.
Querying Settings is where you can edit the configuration for how queries are conducted:
The Default Sort sets results to be sorted by relevance, date, or random.
There are four Query Parsers available out of the Box for LucidWorks; a custom LucidWorks parser, as well as standard Lucene, dismax and extended dismax. More information on the details of each parser is available here.
Unsupervised feedback resubmits the query using the top 5 results of the initial query to improve results.
This is also where you configure the rest of your more familiar query behavior, like where stop words will be used, auto complete, and other settings, the full details of which are here.
In the next post in the series, we’ll demonstrate setting up a custom Web site that integrated LucidWorks Search, and the configuration settings we use to optimize search for that site. After that, in future posts we’ll discuss tips and tricks for working with specific types of data in Lucidworks.
Brian Benz Senior Technical Evangelist Microsoft Open Technologies, Inc.
Back in November Microsoft Open Technologies, Inc. announced that Advanced Message Queuing Protocol (AMQP) 1.0 was approved as an OASIS Standard. AMQP 1.0 enables interoperability using wire-level messaging between compliant clients and brokers. Applications can achieve full-fidelity message exchange between components built in multiple languages and frameworks and running on different operating systems.
Today we’re happy to share the news that the Apache Qpid Proton C AMQP library has been updated to support Windows. Proton C also includes bindings for several interpreted languages including PHP, Python, Ruby and Perl, all of which can now be used on Windows.
UPDATE 04/18/2013 - The following paragraph was in error. The Windows Proton libraries do not work with the latest preview release of service bus due to lack of SSL support. We apologize for the error.
These Proton clients can be used in conjunction with Windows Azure Service Bus, which introduced support for AMQP 1.0 as a preview feature last October, with GA planned later this year. Applications can use AMQP to access the queuing and publish/subscribe brokered messaging features. Service Bus is a multi-protocol service, so in addition to AMQP, applications can also use REST/HTTP to access Service Bus from any platform.
For more information, check out the official OASIS site, this developer’s guide, and downloads(the 0.4 version supports Windows) for Qpid Proton. MS Open Tech was one of many contributors to this project, and we appreciate all the work that the community is doing to help developers take full advantage of AMQP across many different languages, frameworks, and platforms.
From: Gabriel Montenegro Principal Software Development Engineer, Microsoft Corporation
Andrei Popov Senior Software Development Engineer, Microsoft Corporation
Brian Raymor Senior Program Manager, Microsoft Open Technologies, Inc.
Rob Trace Senior Program Manager Lead, Microsoft Corporation
We wanted to give our readers an update on the standardization of the next version of the Hypertext Transfer Protocol, HTTP/2.0, based on our recent industry standards meeting.
Representatives from Microsoft Corporation and Microsoft Open Technologies, Inc., recently attended the Internet Engineering Task Force 86 meetings in Orlando to make progress on the first in a series of experimental implementations of HTTP/2.0 (see our earlier blog for details).
Much of this HTTPBIS Working Group meeting focused on presentations on header compression, which is one of the big open issues that must be resolved for the first experimental implementation of HTTP/2.0.
Martin Thomson (HTTP/2.0 co-editor) collected and presented a number of pending specification issues for discussion and rough consensus - little things that I would like to change in HTTP/2.0 that I don’t feel I have the authority to change without working group feedback.
Gabriel Montenegro shared a presentation on Known startup state for a simpler and more robust HTTP 2.0 that reduces the complexity of HTTP/2.0 implementations by ensuring that the protocol starts in a known state for both the client and server.
At the Transport Layer Security Working Group (TLS WG) meeting, this group reviewed proposals for application protocol negotiation requested by HTTPBIS for HTTP/2.0 negotiation. Andrei Popov presented the Application Layer Protocol Negotiation Extension (ALPN) – one of the proposals under consideration, co-authored with Stephan Friedl (Cisco). After much discussion and a straw poll, there was rough consensus to adopt ALPN. Eric Rescorla (TLS co-chair) sent a Confirming Consensus for ALPN message to the TLS mailing list to encourage additional discussion from IETF members who had not attended the meeting.
It was exciting to see the progress and tone of the discussions that you can see reflected in the transcriptions below:
Mark Nottingham (HTTPBIS chair) also suggested that HTTPBIS continue meeting on a frequent schedule to make progress on the first HTTP/2.0 experimental implementation with future interim meetings proposed before and after IETF 87 in Berlin:
Representatives from Microsoft Corporation and Microsoft Open Technologies, Inc. plan on participating in these meetings and encourage the community to also attend and become more involved in defining the next generation of HTTP at the IETF.
In June, we shared an overview of the LucidWorks Search service for Windows Azure, and in our first post in this series we provided more detail on features and benefits. In December we covered the main features of LucidWorks Search, but today Microsoft Open Technologies, Inc, is happy to share with you a few new data sources that are Available in LucidWorks Search on Windows Azure, and a new easier way to sign up for LucidWorks Search Service on Windows Azure.
LucidWorks Search is still listed under applications in the Windows Azure Marketplace, and from there you can create an account via the LucidWorks Account Signup Page. But getting started is now even easier as we’ve integrated LucidWorks’ service with the Windows Azure Store, so you can now set up an instance on Windows Azure By clicking 0n the Store option in the Windows Azure Dashboard:
Next, you’ll be prompted to choose an Add-on from a list. Select LucidWorks Search. The next screen invites you Personalize your new Add-On:
At this point, all you have to do is enter a new Name for your LucidWorks Search Add-on and the region you want your instance to be located in.
Right now the only option for signup via the Windows Azure Store is the Micro level, which is great for getting started. Should you exceed the limits of the Micro level, you can also sign up for other enterprise-level accounts from the LucidWorks Dashboard using the LucidWorks account that is automatically created when you sign up via the Window Azure Store.
Along with the Windows Azure Store integration, we also released LucidWorks Search support for Windows Azure SQL Databases, Windows Azure Blobs, and windows Azure Table Storage. All are available via the LucidWorks Search Dashboard under Indexing > Data Sources:
Windows Azure Blobs provide a way to store large amounts of unstructured, binary data, such as video, audio, and images, including streaming content such as video or audio. There are two types of blob storage available, block blobs and page blobs. Block blobs are optimized for streaming and referenced by a unique block ID. Page blobs are optimized for random access and composed of pages that are referenced by offsets from the beginning of the blob. More information on Windows Azure blobs can be found here.
Windows Azure Table storage is a collection of non-relational structured data. Unlike tables in a database, there is no schema that enforces a certain set of values on all the rows within a table. Windows Azure Storage tables are more like rows within a spreadsheet application such as Excel than rows within a database such as SQL Server. Each row can contain a different number of columns, and of different data types, than the other rows in the same table. You can find more information on Windows Azure Table storage here.
Windows Azure SQL Databases are similar to an on-premise instance of SQL Server, but not the same. Windows Azure SQL Databases expose a tabular data stream (TDS) interface for Transact-SQL-based database access, so they can be used the same way you use on-premise SQL Server.
However, there are some very important differences for administration. Windows Azure SQL Database abstracts the logical administration from the physical administration. That means that you continue to administer databases, logins, users, and roles, but Windows Azure manages the physical hardware and networking to ensure enterprise-class availability, scalability, security, and self-healing. More information on Windows Azure SQL Databases is available here.
To set up a new Azure SQL Database as a Data source, select Database as your data source option under Indexing > Data Sources.
There are a few tips for setting up a Windows Azure SQL Database as a data source for LucidWorks that you need to know. First of all, copy the URL for your Database from the JDBC connection strings in your Windows Azure Dashboard, using this format:
jdbc:sqlserver://<WindowsAzureSQLDBURL>:1433/<databaseName>
Next, select the SQL Server JDBC driver as the Driver for your Windows Azure SQL Database. You also have to include at least one SQL SELECT statement that includes an id column in the result. The id column is used at the Document identifier in LucidWorks search, and relates each row returned by the SELECT statement as fields in that Document. Have a look at my first post in this series for more information on how LucidWorks works with Documents, Fields, and Collections to return search results.
When done your Data Source configuration should look something like the sample here:
Next there are two additional options for setting up SELECT statements to work with your database. The Delta SQL Query uses the primary key to compare new records in the database with existing Documents in the LucidWorks Search Index, and only indexes the new or updated rows. Nested Queries allow you to set up one-to many relationships in the source Windows Azure SQL Database to include multiple rows of data in a single LucidWorks index Document, based on the primary key. Full instructions on setting up these queries as well as other options can be found in the LucidWorks help documentation here.
These are just the latest new features to help you easily and quickly set up LucidWorks Search service on Windows Azure, and there are more on the way. Get started with your own LucidWorks Search solution by signing up via the Windows Azure Store, and let us know what you think!
Continuous integration (CI), where software teams integrate their work continuously into frequent builds in an Agile environment, has been around for a relatively long time. Tools for managing the CI process have been around too, and have been gaining in popularity in the last few years, as the CI process becomes more complicated and the benefits of CI become more obvious. CI tools can be used in conjunction with existing SCM version control tools to manage today’s complex build, test and deployment processes that SCM tools and processes don’t cover completely on their own.
Jenkins is a popular open source CI tool, with many installations and extensions, as well as strong community commitment. For this reason Microsoft Open Technologies, Inc. has released a Jenkins plugin for using Windows Azure’s Blob Storage service as a repository of build artifacts.
Using our Jenkins plugin can improve your CI process by using the Windows Azure Storage plugin to manage artifact storage in a Windows Azure Blob. Choosing the Windows Azure Blob service to store your build artifacts ensures that you have all the resources you need each time a build is required, all in a safe and reliable yet centralized location, with configurable access permissions. This takes a load off on-premise network bandwidth and storage, and improves continuous build performance.
We’ve also open-sourced our plugin to share with the community. Source code for the plugin is available on Github here.
The plugin works with any Jenkins CI installation. VM Depot, MS Open Tech’s community-driven repository of Linux Virtual Machines, also has several preconfigured Linux and Jenkins Virtual Machines ready to quickly get Jenkins up and running in a Windows Azure Linux VM. For more information on setting up VM Depot Virtual Machines on Windows Azure, follow this link.
It’s also easy to set up a custom instance of Jenkins on a customized Windows Azure Virtual Machine. Here are some great resources to get started.
For source code versioning and repository management, Jenkins on Windows Azure can use the built-in CVS or Subversion instances that are downloaded with Jenkins, or you can connect to any code management repository source that a plugin exists for, including Team Foundation Server (via the Jenkins TFS plugin), or the GitHub plugin.
Once you have a code repository and a Jenkins instance set up, you’re ready to configure Jenkins for build management and deployment. We’ve created a detailed tutorial here on how to set up and use the plugin.
To install the plugin, go to Manage Jenkins > Manage Plugins, select the Available Plugins tab and select the Windows Azure Storage Plugin from the Artifact Uploaders Category.
After selecting Install without Restart, you should see a confirmation screen like this one when done:
After the plugin is installed, the first step you should take is to configure one or more Windows Azure storage accounts for Jenkins to use. You do that using Jenkins’ Configure System page, in the Windows Azure Storage Account Configuration section:
After you have configured your storage account(s), you can start adding this new Post-Build action to your jobs: Upload artifacts to Windows Azure Blob Storage:
Selecting and configuring this option will enable you to work with your artifacts using Azure Blob Storage services, which helps with management and speed of integration. For more information on the configuration options, please refer to our tutorial.
We’re excited to be participating in the Jenkins ecosystem to enable build artifacts to be stored in Windows Azure storage. As always, we’re looking for ways to make it easier for developers to interact with Windows Azure services in any way we can, so if you have suggestion on what we can do to improve interoperability between Jenkins and Windows Azure, let us know!
From the Rx and ActorFx team: Claudio Caldato, Principal Program Manager Lead, MS Open Tech Erik Meijer, Partner Architect, Microsoft Corp. Brian Grunkemeyer, Senior Software Development Engineer, MS Open Tech Hub Joe Hoag, Senior Software Development Engineer, MS Open Tech Hub
Today Microsoft Open Technologies, Inc., is releasing updates to improve two cloud programming projects from our MS Open Tech Hub: Rx and ActorFx .
Reactive Extension (Rx) is a programming model that allows developers to use a common interface for writing applications that interact with diverse data sources, like stock quotes, Tweets, computer events, and Web service requests. Since Rx was open-sourced by MS Open Tech in November, 2012, it has become an important under-the-hood component of several high-availability multi-platform applications, including NetFlix and GitHub.
Rx 2.1 is available now via the Rx CodePlex project and includes support for Windows Phone 8, various bug fixes and contributions from the community.
ActorFx provides a non-prescriptive, language-independent model of dynamic distributed objects for highly available data structures and other logical entities via a standardized framework and infrastructure. ActorFx is based on the idea of the mathematical Actor Model, which was adapted by Microsoft’s Eric Meijer for cloud data management.
ActorFx V0.2 is available now at the CodePlex ActorFx project, originally open sourced in December 2012. The most significant new feature in our early prototype is Actor-to-Actor communication.
The Hub engineering program has been a great place to collaborate on these projects, as these assignments give us the agility and resources to work with the community. Stay tuned for more updates soon!
More good news for Reactive Extensions (Rx).
Just yesterday, we told you about improvements we’ve made to two Microsoft Open Technologies, Inc., releases: Rx and ActorFx, and mentioned that Netflix was already reaping the benefits of Rx.
To top it off, on the same day, Netflix announced a Java implementation of Rx, RxJava, was now available in the Netflix Github repository. That’s great news to hear, especially given how Ben Christensen and Jafar Husain outlined on the Netflix Tech blog that their goal is to “stay close to the original Rx.NET implementation” and that “all contracts of Rx should be the same.”
Netflix also contributed a great series of interactive exercises for learning Microsoft's Reactive Extensions (Rx) Library for JavaScript as well as some fundamentals for functional programming techniques.
Rx as implemented in RxJava is part of the solution Netflix has developed for improving the processing of 2+ billion incoming requests a day for millions of customers around the world.
To summarize, here’s a great quote from Ben Christensen on the Netflix Tech Blog about Rx:
“Functional reactive programming with RxJava has enabled Netflix developers to leverage server-side concurrency without the typical thread-safety and synchronization concerns. The API service layer implementation has control over concurrency primitives, which enables us to pursue system performance improvements without fear of breaking client code.”
Interoperability in server and cloud space has found even more evidence with the release announcement of Open Virtualization Format (OVF) 2.0 standard. We congratulate DMTF for this new milestone, a further proof that customers and industry partners care deeply about interoperability and we are proud of our participation to advance this initiative.
Browsing the OVF 2.0 standards specification, it is evident the industry is aligning around common scenarios and it comes as a pleasant surprise how some of those emerging scenarios have been driving our own thinking in the direction for System Center.
Microsoft has collaborated closely with Distributed Management Task Force (DMTF) and our industry partners to ensure OVF provides improved capabilities for virtualization and cloud interoperability scenarios to the benefit of customers.
OVF 2.0 and DMTF are making progress on key emerging patterns for portability of virtual machines and systems, and it’s nice to see OVF being driven by the very same emerging use cases we have been analyzing with our System Center VMM customers such as shared Hyper-V host clusters, encryption for credential management and virtual machine boot order management (not to mention network virtualization, placement groups and multi-hypervisor support).
Portability in the cloud and interoperability of virtualization technologies across platforms using Linux and Windows virtual machines continues to be important to Microsoft and to our customers and are increasingly becoming key industry trends. We continue to assess and improve interoperability for core scenarios using the SC 2012 VMM. We also believe moving in this direction will provide great benefit to our customer and partner eco-system, as well as bring real-world experience to our participation with OVF in DMTF.
See the overview for further details and other enhancements in System Center 2012 VMM.
Mark GaylerSenior Technical EvangelistMicrosoft Open Technologies, Inc.
XML was first published as a W3C Recommendation on 10 February 1998.
I would have never dreamt, 15 years ago, that we would be so successful in our dream of exchanging information freely between different platforms and now across devices and clouds. For me, this has been the beginning of the Openness revolution. I truly believe that that the strength of XML is its inherent unique capability of representing homogenously documents and data: those scenarios and capabilities will be even more important for the next 15 years.
Vive XML and to its bright future!
Jean Paoli President, Microsoft Open Technologies, Inc.Co-Creator, XML 1.0 @ W3C
From: Asir Vedamuthu Selvasingh, Principal Program Manager Microsoft Open Technologies, Inc.
Adalberto Foresti, Principal Program Manager Microsoft Open Technologies, Inc.
Developers can start building multi-input websites and apps with greater confidence that an emerging industry standard will enable building a single website targeting multiple devices and platforms.
Only three months after its creation, the W3C Pointer Events Working Group has announced that Pointer Events has reached “Last Call Working Draft” status and is considered feature complete by the Working Group. The W3C Pointer Events Working Group has been hard at work over the last few months to standardize a single device input model – mouse, pen and touch – across multiple browsers. Congratulations to the W3C Pointer Events Working Group!
Microsoft Open Technologies, Inc. (MS Open Tech), and the Microsoft Corp. Internet Explorer teams have been working with our colleagues across the industry, engaging developers to test and provide feedback on the specification, and incorporating all the received feedback into this Last Call Working Draft.
“Last Call Working Draft” means that members of the Working Group, including representatives from Google, jQuery Foundation, KAIST, Microsoft, Mozilla, Nokia, Opera, Zynga, and others, consider that this specification has satisfied all the technical requirements outlined in the Working Group Charter. The working group intends to advance the specification to implementation after this Last Call review.
What’s cool is that you can go build websites using Pointer Events today. The Working Group is using Microsoft’s member submission as a starting point for the specification, which is based on the APIs available today in IE10 on Windows 8 and Windows Phone 8.
If you are building your apps using Pointer Events and testing these apps on various browsers, you should try out the hand.js polyfill developed by David Catuhe from Microsoft France. Check out a demo that uses hand.js - universal virtual joystick. We expect that native implementations for WebKit-based browsers will follow shortly.
To demonstrate cross-browser interoperability for Pointer Events, MS Open Tech developed a Pointer Events prototype for WebKit on HTML5 Labs and submitted the patch to the WebKit community. Today MS Open Tech posted an updated version of the patch on HTML5 Labs and on the WebKit issue tracker, incorporating community feedback received on the previous version. Working with the WebKit community, MS Open Tech will continue updating this prototype to implement the latest draft of the specification.
Recently, MS Open Tech hosted an HTML5 Labs Test Jam event on Feb. 11 to share an early preview of the new prototype and collect feedback, and the browser community has been playing with the prototype as noted in a blog post by our friends at AppendTo. AppendTo shares Chromium builds for OSx and for Windows that integrate the Pointer Events patch by MS Open Tech.
If you are attending W3Conf this week in San Francisco, you don’t want to miss “Pointing Forward” at 3:00 pm on Thursday, February 21, presented by Jacob Rossi, program manager for Internet Explorer and co-editor of the W3C Pointer Events specification. You also can watch a live stream of his conference presentation on the W3Conf site and UStream, or later on video on demand.
And, you can learn more by checking out the Pointer Events Primer on WebPlatform.org, developed by Rob Dolin, senior program manager at MS Open Tech. The primer provides guidance on how to use Pointer Events in ways similar to mouse events, and how to access and use additional attributes such as pointer type, button(s) pressed, touch size and pen tilt. The primer is a great resource if you are migrating your code from handling mouse, to consistently handling input from mouse, pen and touch.
We’ve been happy to share the great progress for the W3C Pointer Events emerging standard. Stay tuned for more updates as we work together on this open standard that can further enable natural and simple computing interfaces on the Web.
As part of the HTTP/2.0 effort, the industry is collaborating to reinforce Internet communication security in the IETF Transport Layer Security Working Group (TLS WG). Two security experts from Cisco and Microsoft Corp. have submitted ALPN-01 (Application Layer Protocol Negotiation), a safer and simpler application protocol negotiation approach, backed up by a new HTML5 Labs HTTP/2.0 prototype by Microsoft Open Technologies, Inc., incorporating an initial implementation of ALPN-01.
Stephan Friedl (Cisco) and Andrei Popov (Microsoft Corp.) co-authored the ALPN-01 Internet draft that is under discussion among the TLS WG mailing lists. This is in response to discussions at the IETF 85 meeting in Atlanta where the IETF TLS WG received a request from the HTTPBIS Working Group for “a mechanism that allows clients and servers to negotiate the particular application protocol to use once the session is established." Currently, there are two proposals:
The new ALPN-01 (Application Layer Protocol Negotiation) Internet draft proposes a protocol negotiation in accordance with established TLS architecture with the following benefits:
Thanks to these benefits, and because of its stricter adherence to established TLS design principles, ALPN represents the best choice to address the requirements articulated by the HTTBIS working group for HTTP/2.0 protocol negotiation.
Our HTML5 Labs prototype is the first implementation that is based on the ALPN-01 Internet draft. It is an evolution of earlier prototypes that couples a modified command-line C# client with a basic HTTP/2.0 server. We plan to further develop it in the coming weeks, and we look forward to your feedback both on the TLS WG mailing list and through Html5 Labs. We will gladly apply changes to the draft as well whenever applicable.
Go ahead and download the MS Open Tech HTTP/2.0 prototype using ALPN from HTML5 Labs! And please share your thoughts on this post below.
HTTP, the Hypertext Transfer Protocol, is one of the most important protocols for the Internet, and we’re pleased to report progress on the next generation HTTP/2.0 as we recently returned from our interim HTTPbis Working Group meeting in Tokyo (HTTPbis is the HTTP Working Group in the Internet Engineering Task Force).
Our industry standards community reached preliminary agreements on the next steps for the first in a series of experimental implementations of HTTP/2.0 that will improve the performance for how every application and service on the Web communicates today.
Progress on Negotiation and Flow Control
In our previous post - Sharing proposals for negotiation and flow control for HTTP/2.0 at IETF 85 – we shared our positions on Negotiation and Flow Control and outlined future plans to make progress in these areas.
After final review at the interim HTTP/2.0 meeting, we’re pleased to announce that HTTP 2.0 Negotiation that Microsoft co-authored with Exceliance and Orange and HTTP 2.0 Principles for Flow Control that Microsoft co-authored with Ericsson and Google were incorporated into the latest HTTP/2 base draft.
Implementation Draft Specification
The most important outcome of the interim meeting in Tokyo was the recommendation to create a HTTP/2.0 “Implementation Draft Specification” based on the set of features that have achieved rough consensus in the HTTPBIS working group at this time. There was strong agreement among the attendees with this direction and commitment to implementing this draft specification when available.
The implementation draft is targeted for March with another HTTP/2.0 interim meeting proposed between June-September where interoperability testing can occur.
The full proposal is available here. Many of these features are dependent on the rapid execution of their related action items.
The proposed feature list includes significant changes to:
The intent is to converge on the details using the IETF HTTPBIS mailing list and then implement and validate the subsequent implementation draft. And then repeat the process based on our experience and new understanding – as Mark Nottingham (IETF HTTPBIS chair) has clarified:
Note that we are NOT yet firmly choosing any particular path; rather, we're working on proposals in code as well as text, based upon discussion to date. As such, we're likely to have several such implementation drafts that progressively refine the approach we're taking. I.e., we don't have to agree that the above is what we want HTTP/2.0 to look like -- only that it's interesting to make these changes now, so that we can test them.
Looking Ahead
We are pleased with the direction of the HTTPBIS working group and are looking forward to interoperability testing with our HTML5 Labs HTTP/2.0 prototype.
Based on the action items from the interim meeting in Tokyo, there is already active discussion on the IETF HTTPBIS mailing list as more detailed proposals are prepared and shared with the working group. We encourage the community to openly and actively contribute to the mailing list and strongly consider prototyping the implementation draft when available.
We are looking forward to further discussions at the IETF 86 HTTPBIS meeting on March 15 in Orlando where we continue our goal to help ensure, along with our IETF colleagues, that HTTP/ 2.0 meets the needs of the broader Internet community.
Gabriel Montenegro and Brian Raymor
Microsoft Open Technologies, Inc will be at ApacheCon in Portland next week and we hope to see you there. We’re sponsoring the Hackathon on Monday, and come see my session on options for implementing CouchDB on Windows Azure on Thursday. Other than that we’ll be around all week, so if you see one of us stop and say hi!
Furthering the goal of bridging Microsoft and non-Microsoft technologies, Microsoft Open Technologies, Inc. developed the Android SDK for Windows Azure Mobile Services that is being announced today by Scott Guthrie on his blog.
Windows Azure Mobile Services was created to make it easier for developers to build engaging and dynamic mobile apps that scale. By using Mobile Services, developers are not only able to connect their applications to a scalable and secure backend hosted in Windows Azure, but also store data in the cloud, authenticate users and send push notifications.
The Android SDK lets you connect your favorite Android phone or tablet (Android 2.2+) to a cloud backend and deliver push notifications via Google Cloud Messaging. It also allows you to authenticate your users via their Google, Facebook, Twitter, or Microsoft credentials. To enable this, the MS Open Tech engineering team delivered the following key features:
The SDK is available on GitHub under the Apache 2.0 license and community contributions are very welcome.
You can learn more about the new SDK reading Scott’s blog, and the getting started tutorial and come back soon as we are working on more samples/demos/tutorials.
The success of the recent Screen Actors Guild (SAG) Awards ceremony was buoyed by the move of its Drupal-based website hosted on internal Linux servers to one hosted on Windows Azure.
The SAG Awards site is a highly visible, high-traffic website running on Drupal. Hosting it on Azure provides a scalable, public cloud environment for SAG team. They can tune up or down the compute and storage requirements according to expected website loads, thereby getting a more scalable, manageable and cost-effective solution for running their site.
SAG also gets the benefits of PaaS – no need to manage the operating system patches, virtual machine images, network topology etc. This is particularly useful for SAG as the site has stable traffic for nine months, but which spikes for the three months from when award nominations open to the night of the event itself.
The SAG Awards site was previously hosted on internal Linux boxes. In previous years, performance was negatively impacted by site outages and slow performance during peak-usage days, with SAG having to consistently upgrade their hardware to meet demand for those days. That upgraded hardware was then not optimally used during the rest of the year.
The usage pattern for the SAG Awards site fluctuates, but spikes between November and February when the site is used for SAG award nominations in early November to the actual announcement of nominations in in mid-December. Peak usage is on the night of the awards ceremony where multiple uploads of pictures, news articles, and site visits happen.
What is even more impressive is that both visits and page views almost doubled on the night of the event. In 2011, some 222,816 people visited the site and 434,743 pages were viewed, while this year there were some 325,303 site visits and 789,310 page views, reflecting the stability and performance of the site on Windows Azure.
Microsoft started working with the SAG Awards team in May 2011, when their CIO Erin Griffin joined the Interoperability Executive Council (IEC) - founded by Microsoft in 2006 with a goal of identifying the industry’s greatest areas of need and to work together to create solutions - and attended a council meeting.
In September Mike Story, SAG’s chief architect, attended an IEC work stream meeting and asked for Microsoft’s support in porting the site to Azure. The Business Platform Division’s Customer Experience (CAT) team, the Interoperability group and Windows Azure all started working with SAG in early October and, on December 20, 2011, the site went live on Windows Azure.
“We moved to Windows Azure after looking at the services it offered,” said Erin Griffin, CIO at SAG. “Understanding the best usage scenario for us took time and effort, but with help from Microsoft, we successfully moved our site to Windows Azure and the biggest traffic day for us went off with flying colors.”
This is just one real world outcome from the IEC, which has counseled Microsoft on many interoperability topics and introduced a number of real world scenarios for discussion. The IEC, working together with Microsoft, has developed a number of solutions for these scenarios, with this one for the SAG Awards being the latest.
Curt Peterson, Microsoft’s Principal Group Program Manager, BPD Customer Experience, notes that the success of Sunday’s SAG Awards ceremony underscores how Windows Azure is a scalable, open Cloud platform ready for production use. “We are committed to making it easier for all our customers to use cloud computing on their terms with Windows Azure,” he says.
I am excited to share some great news about how we are opening up the SQL Server data platform even further with expanded interoperability support through new tools that allow customers to modernize their infrastructure while maximizing existing investments and extending virtually any data anywhere.
The SQL Server team today introduced several tools that enable interoperability with SQL Server 2012.
These tools help developers to build secure, highly available and high performance applications for SQL Server in .NET, C/C++, Java and PHP, on-premises and in the cloud.
These new tools include a Microsoft SQL Server 2012 Native Client, a SQL Server ODBC Driver for Linux, backward compatibility with ADO.Net and the Microsoft JDBC Driver 4.0 and PHP Driver 3.0.
You can find more information on all this goodness on the SQL Server blog here.
BuildNewGames.com, a new site to make building web games easier for developers using HTML5, CSS3 and JavaScript, is now live!
Along with a new partnership with Bocoup, Microsoft announced @ JSConf the launch of this new site.
You can read the post from Justin Garret, Senior Product Manager in the IE team, announcing the partnership and the new site launch.
Over the next few months, the site will feature 50 tutorials ranging from the coding basics of games all the way to how to make money across a range of platforms. Follow @buildnewgames or @IE for the latest.
Developers want to be able to write code that works reliably in all modern browsers, including ie10/9, Chrome and Firefox, along with mobile browsers, resulting in a complex test matrix and higher development costs. Through standards bodies leadership and practical learning, Microsoft wants to help Web developers have an easier time targeting various browsers at once, allowing them to concentrate on innovating and delivering an outstanding Web and gaming experience to final users.
BuildNewGames.com already features technical articles on Animation, Compositing, Graphics, Mobile, SVG, Sprites, Tools, WebSockets.
Developing games is becoming lots of fun again!
MS Open Tech is pleased to announce a new series of videos on Channel 9 that covers MongoDB topics for developers working on Windows Azure and Windows. Each video in the series features insights from one of the MongoDB experts at 10gen, the leader in MongoDB development, support, training and consulting.
The first three videos in the series have been posted, and more are coming soon. Here’s what has been covered in the first videos in the series …
MongoDB Overview with Jared Rosoff provides a high-level overview of the approach that MongoDB takes for delivering highly scalable read and write operations. If you’re entirely new to MongoDB, this is the place to start. MongoDB is one of many database platforms that are often grouped together as “NoSQL databases,” but each NoSQL database has its own unique philosophy and personality. In this video, you’ll get a feel for MongoDB’s personality.
MongoDB Replica Sets with Sridhar Nanjundeswaran covers the key concept at the heart of MongoDB scalability: replica sets, which are groups of MongoDB servers that can provide high availability and performance even in the face of failures at the network and hardware level. MongoDB replica sets are easy to set up and deploy, and Sridhar sets up a simple replica set from scratch and then shows how it gracefully handles various failover scenarios
MongoDB C#/.NET Driver with Robert Stam is a hands-on look at how to do common database operations in C# through use of the C#/.NET driver from 10gen. Robert is the developer of the driver, and in this video he shows how to create, read, update and delete documents in MongoDB collections.
MS Open Tech has been working closely with 10gen to improve the MongoDB experience on Windows Azure, and we’re working together on a variety of new initiatives to continue on that path. Future videos will cover the results of that work, as well as advanced topics related to the current videos (for example, Linq support in the C#/.NET driver) and other topics of interest to developers who are working with MongoDB on Windows Azure.
Stay tuned, and if there are MongoDB/Azure topics you’d be interested in seeing covered in this series please let us know!
I’m pleased to announce the availability of the Windows Azure Plugin for Eclipse with Java (by Microsoft Open Technologies), June 2012 CTP.
This has been the most ambitious and technically complex update we’ve had, focusing on improving the ease of creating projects, deploying apps to the cloud, and simplifying developers’ programmatic access to various services provided by Windows Azure. This update also includes a set of other enhancements driven by user feedback.
These are the main additions:
- New Windows Azure Deployment Project wizard – enables you to select your JDK, Java server, and Java apps right in the improved wizard UI. The list of out-of-the-box server configurations to choose from now includes Tomcat 6, Tomcat 7, GlassFish OSE 3, Jetty 7, Jetty 8, JBoss 6, and JBoss 7 (stand-alone), and it is user-customizable. (This UI improvement is an alternative to dragging and dropping compressed files and copying over startup scripts, which was previously the main approach. That method still works fine but will likely be preferred only for more advanced scenarios now.)
- Server Configuration role property page – enables you to easily switch the servers and applications associated with your deployment after you create the project, as part of the Role Propertiesdialog box.
- “Publish to cloud” wizard – an easy way to deploy your project to the Windows Azure cloud directly from Eclipse, automating all the heavy lifting of fetching credentials, signing in, uploading, and so on. (This is a contribution from our Java partner GigaSpaces Technologies Ltd.)
- Widows Azure Toolbar – provides easy access to several commonly used actions: Run in emulator, Reset emulator, Create cloud package, New Windows Azure Project, Publish to Windows Azure cloud, Unpublish.
- Componentsproperty page makes it easier for advanced users to set up project dependencies between individual Windows Azure roles in the project and other external resources such as Java application projects, as well as to describe their deployment logic.
- Package for Windows Azure Libraries for Java (by Microsoft Open Technologies, Inc) – consists of all the JAR files needed for programming the Windows Azure APIs, including the Windows Azure Libraries fo Java. It is installed by default when you install the main plugin. You add a reference to just this one Eclipse library from your Java project. You can now also easily embed the entire library in your WAR file at the same time with just a single check box (no need to configure the deployment assembly separately). This package is for users who do not use Maven and would rather not have to download all the JAR files on their own.
- Instance input endpoint configuration UI– helps enable remote debugging and JMX diagnostics for specific compute instances running in the cloud in scenarios with multi-instance deployments. Users can do this by configuring this new type of Windows Azure endpoint. (Previously, remote debugging could be made to work reliably only for single-instance deployments.)
- Windows Azure Access Control Services Filter (by Microsoft Open Technologies, Inc) – enables your Java application to seamlessly take advantage of Windows Azure Active Directory Access Control (ACS) authentication using various identity providers (such as Google, Live.com, and Yahoo). You don’t have to write authentication logic yourself, just configure a few options and let the filter do the heavy lifting of enabling users to sign in using ACS. Then just write the code that gives users access to resources based on their identity, as returned to your app by the filter inside the Request object.
Martin Sawicki
Principal Program Manager
Microsoft Open Technologies, Inc.
A subsidiary of Microsoft Corporation
We’re pleased to announce that GigaSpaces Technologies Ltd, an established leader in helping enterprises move their Java applications to the cloud, has joined Microsoft Open Technologies, Inc. and Persistent Systems Ltd. in the the development work behind the latest version of the Windows Azure Plugin for Eclipse with Java (by Microsoft Open Technologies) - June 2012 CTP.
GigaSpaces has contributed the “Publish to cloud”wizard to the plugin, enabling Java developers to easily deploy their projects to the Windows Azure cloud directly from within Eclipse, thus eliminating the need for manual uploads via the Windows Azure portal. GigaSpaces has also contributed other new capabilities for Java developers, including:
You can read more about the latest plugin update here, and you can learn how to use the “Publish to cloud” feature here.
Known for its industry-leading scalable application platforms, GigaSpaces Technologies is the creator of Cloudify, an innovative Open PaaS stack solution that enables on-boarding of mission-critical and big-data applications to the cloud without any code or architectural changes. Cloudify's recipe-based approach provides the flexibility and control required to manage the deployment, scaling, management and high availability of all the tiers of your application. Hundreds of tier-1 organizations worldwide use GigaSpaces technology to enhance IT efficiency and performance, among which are Fortune Global 500 enterprises and ISVs, from many industries spanning financial services, e-commerce, Telco, healthcare, and more.
Thanks to the team at GigaSpaces for all they’ve done to help streamline and improve the Windows Azure development and deployment experience for Java developers! We look forward to continued collaboration with Gigaspaces in the future.
Yesterday, Bill Laing of the Windows Azure team announced support for virtual machines running the Windows Server operating system as well as Linux distros such as Ubuntu, CentOS, and OpenSUSE. Now you can run existing Linux payloads on Windows Azure virtual machines, with no need to change any of your code. This capability makes Windows Azure a great platform for IaaS deployment of applications that run on Windows or Linux servers. You can find more information about what's new in Windows Azure on Scott Guthrie's blog post today and the MeetWindowsAzure event that he'll be kicking off this afternoon.
There are two ways to work with the new virtual machine and web site capabilities of Windows Azure: through the management portal, or at the command line. This article covers the concepts behind the command-line tool, but those who prefer to use a GUI can also provision web sites and deploy Windows or Linux virtual machines from the Windows Azure portal. An easy-to-use GUI takes you through every step of the process.
Many developers prefer the power and flexibility of command-line tools, however, which can be automated via a scripting language. If you’re working exclusively on Windows machines, the Windows PowerShell cmdlets are your best option, but for mixed environments, the Windows Azure command-line tool for Mac and Linux provides a consistent experience across Linux, Mac OS, and Windows desktops.
Installation of the command line tool is very simple. If you’re working on a Mac OS X machine, you can use the Mac installer, and for Windows or Linux you’ll just need to install the latest version of Node.js and then type this command.
npm install azure --global
That will install the Windows Azure SDK for Node.js, which includes the command-line tool. Alternatively, you can download the command line tools or the Windows PowerShell cmdlets from this download page.
To verify that you have the tool installed and ready to use, type the command azure --help and you‘ll see the output shown to the right. This screen tells you which version of the tool you’re using, and how to get information about each of the commands.
The first thing to understand is the basic structure of the commands. In general terms, you type azure followed by a topic (what you’re working with), a verb (what to do), and various optional parameters to provide additional information. Here’s a diagram that provides a general framework for understanding the command-line syntax.
Some commands have other required command-line parameters in addition to what’s shown here. For more information about specific command syntax, see the reference documentation.
The command-line tool allows you to provision new web sites and virtual machines, and that activity needs to be associated with a Windows Azure subscription. So before you start using the tool, you’ll need to download a publish settings file from the Windows Azure portal and then import it as a local configuration setting. For more information about how to do this, see the how-to guide How to use the Windows Azure Command-Line Tools for Mac and Linux, which also covers the basics of deploying web sites and virtual machines.
Let’s take a look at some of the other things you can do with the command-line tool …
Locations and affinity groups. When you deploy a virtual machine, you must tell Windows Azure the location where you’d like for your virtual machine to be deployed – North Central US, for example. The azure vm location list command provides a list of available locations that you can use.
You can also use an affinity group to specify the location. You can create your own affinity groups (here’s how) and then use an affinity group instead of a location when you deploy a virtual machine, cloud service, or storage account. The use of an affinity group tells Windows Azure “please host these services as close together as possible,” with a goal of reducing network latency. The azure account affinity-group list command lists your available affinity groups.
Cloning a customized virtual machine. After you’ve deployed a virtual machine and customized it by installing and configuring software via SSH or other means, you may want to deploy additional instances of that virtual machine that will include your customizations. To do this, stop the virtual machine and use the vm capture <vm-name> <target-image-name> command to capture a cloned copy of it. Then you can deploy new instances of your customized virtual machine through the vm create command.
Virtual machine data disks. When you deploy a virtual machine, you may want to attach a separate data disk, which is a .vhd file in Windows Azure blob storage that provides additional storage for a virtual machine. The azure vm disk command provides options for creating data disks and attaching them to virtual machines. Use the azure help vm disk command to list the available options.
Virtual machine endpoints. When you deploy multiple instances of a virtual machine, you need to set up port mapping between the virtual machines and the load balancer. The load balancer uses an internal IP address to route traffic to each virtual machine, and these mappings are defined through the azure vm endpoint create command. In a blog post later this month, we’ll take a hands-on look at the details of configuring multiple virtual machines behind a load balancer.
Windows Azure cloud services. Although the main focus of the command-line tool is working with virtual machines and web sites (IaaS scenarios), it can also be used to view the cloud services that you have deployed through web roles and worker roles. The azure service list command lists your cloud services, and the azure service delete command deletes a cloud service.
Working with Linux virtual machines. The command-line tool supports both Linux and Windows operating systems for deployment on virtual machines, and for most of the commands there is no difference between working with Windows and working with Linux. Some differences are inherent in the operating system itself, however. For example, Windows uses RDP whereas Linux uses SSH. The article An Introduction to Linux on Windows Azure provides an overview of what you need to know to take full advantage of Linux virtual machines on Windows Azure.
Write custom service management tools and workflows in Node.js. You can provision and manage virtual machines from your own code, through the new iassClient module that provides access to the service management API from Node.js. For more information, see the reference documentation.
As you can see, the Windows Azure command-line tool for Mac and Linux opens a whole new world of possibilities for developers. Working from a Linux or Mac desktop, you can now deploy and manage virtual machines and web sites on Windows Azure. You can also migrate an existing Linux application to Windows Azure without changing a line of code, and then begin taking advantage of powerful Windows Azure services at any time. It’s all about developer choice: your choice of client operating system, server operating system, programming language, frameworks, and tools – all supported by Windows Azure!
Doug MahughSenior Technical EvangelistMicrosoft Open Technologies, Inc. A subsidiary of Microsoft Corporation
Last week’s Windows Azure release delivered a host of new services for developers, ranging from hybrid cloud capabilities and Linux virtual machine support to OSS technologies delivered as a service from many vendors. Gianugo Rabellino covered the high-level view of all the exciting new offerings, and in this post I’d like to take a closer look at a service that’s likely to become very popular: LucidWorks Cloud for Windows Azure.
Lucid Imagination, the leading experts in Lucene/Solr technology, has packaged their LucidWorks Enterprise search service in a cloud-friendly way that requires only four quick and simple steps: select a plan, sign up, log in, and start using it. LucidWorks Enterprise is based on Apache Solr, the open-source search platform from the Apache Lucene project, and it includes a variety of enhancements from the search experts at Lucid that make it easy to use Lucene/Solr functionality while preserving the purity of the open source code base and open APIs. There’s a comprehensive REST API for integrating it into your applications and services, and you get all of the functionality that has made Solr and LucidWorks Enterprise so popular: high-performance indexing for a wide range of data sources, flexible searching and faceting, and user-oriented features like auto-complete, spell-checking, and click scoring.
As covered on the Lucid Imagination web site, there are four levels of service available for LucidWorks Cloud: Micro, Small, Medium and Large. Pick the level that meets your needs, sign up for the service, and you’re ready to start creating collections and searching your content. You can currently search content in web sites, Windows shares, Microsoft SharePoint sites, FTP, and other sources, with Windows Azure blob storage support coming soon. You can even index and search your data from Hadoop if desired. All index data is stored on Windows Azure drives, which offer high availability and reliability, and the Lucid dev operations engineering team can provide expert support for your LucidWorks Cloud environment.
If you’re new to Solr, check out the free white paper available for download from the Lucid web site, which covers the basics of LucidWorks Enterprise and shows how to use the indexing and searching functionality through the LucidWorks dashboard. Most developers will want to study the API and integrate search tightly into their own software, but you can learn all of the key concepts through the dashboard UI without writing a single line of code.
One concept worth pointing out here is that Solr isn’t just about searching web sites and HTTP documents. Sure, it does a great job of that, but it can also index content stored in database tables, local file systems, and other sources. There is also an XML-based Solr document format that you can use for importing data directly into the Solr engine, giving developers flexibility for indexing any type of content from any source.
This new service from Lucid Imagination is great for those who want to get up and running quickly, but there are also developers who will want to take responsibility for all of the details and host Solr or LucidWorks Enterprise themselves. You can download LucidWorks Enterprise and install it, or you can take advantage of the simple Solr installer for Windows Azure that helps you deploy your own Solr instances as Windows Azure cloud services.
As you can see, there are many options for getting up and running with Solr and LucidWorks. For a simple overview of how easy it is to start using the new LucidWorks Cloud service, check out this Getting Started video that covers how to create a collection, index a web site, and then search that website using the Lucidworks Cloud dashboard. Lucid continues to evolve and invest in supporting the most popular Solr clients, so there will surely be more good news for Lucene/Solr users going forward.
In a future blog post, we’ll be covering how to use LucidWorks Cloud with popular content management systems such as WordPress and Drupal.
Justin Garret from the IE team published a post on the Exploring IE blog late last week that explains how web standards not only help reduce the cost of development and the complexity of testing across browsers and devices, but also achieves noticeable performance benefits.
Justin’s post describes results of performance tests run on sites before and after upgrading them to web standards, demonstrating an average of 30% better page load time in IE10. Developers can also read in Justin’s post recommendation on how to upgrade their sites.
Microsoft’s commitment to web standards involves not just implementing all the stable specifications edited by W3C, but also in actively participating in the definition of these standards by co-chairing the W3C HTML Working Group, proposing specs, sharing early implementations of the spec proposals on HTML5 Labs, demonstrating early implementations in the upcoming products on IE test drive. These are then delivered in release products after getting feedback from developers and users from HTML5 Labs and IE Test drive.
We at Microsoft Open Technologies, Inc. actively participate in the open standards process, are deeply engaged with the industry in the W3C, are developing and publishing prototypes in collaboration with the IE team, and engage with the community to gather input and feedback.
Check out Justin's post on the Exploring IE blog to learn more about how to make your site faster using web standards.
I know there was a flurry of news at the recent TechEd 2012 conference in Orlando, so I wanted to point you to a piece of Interoperability goodness that might have gone unnoticed in the mix: the release of the System Center 2012 – Virtual Machine Manager (VMM) OVF Export/Importtool.
I recently chatted with my colleague Monica Martin, who is involved in the DTMF work around OVF for MS Open Tech. She gave me a lot of insight into the tool, which uses the Distributed Management Task Force (DMTF)’s Open Virtualization Format (OVF 1.1) standard, enables Interoperability between System Center 2012 Virtual Machine Manager (VMM) and VMware vCenter and Citrix Xen Server.
The tool allows Microsoft’s System Center 2012 VMM users to import and export a virtual machine in an OVF 1.1 format from VMware’s vCenter and Citrix’s Xen Server.
The OVF Import/Export tool is a set of cmdlets for use with VMM. Use of OVF promotes portability and interoperability of a virtual machine across Microsoft, VMware and Citrix hypervisors. We’ve gained valuable implementation experience with Citrix and VMware using OVF and successfully tested with vCenter and Xen Server.
Adding OVF and the OVF Import/Export tool to the cache of advanced infrastructure, configuration and service management capabilities to SC 2012 is another milestone in Microsoft’s plans to deliver ongoing value to our customers and partners.
The Open Virtualization Format (OVF) is an open standard for packaging and distribution of virtual appliances to run in virtual machines developed in the Distributed Management Task Force (DMTF), Inc. Microsoft and other industry partners are focused on the development of OVF. Microsoft has been involved in OVF development from the outset.
OVF 1.1 is an international standard important to customers and partners, who are looking for strategies to effectively enable and speed their on-ramp of virtualization technologies in an interoperable way.
This is another example of how Microsoft is committed to interoperability and openness in the products and services we provide, including our multi-hypervisor and standard-based storage management features in SC 2012.
We have now taken this even further with the release of the System Center 2012 – Virtual Machine Manager (VMM) OVF Export/Import tool, which can be downloaded from the Microsoft Download Center. More information can be found on TechNet.
For more information about this tool and other System Center products and solutions, please visit the System Center website.
Good news for all you PHP developers out there: I am happy to share with you the availability of Windows Azure SDK for PHP, which provides PHP-based access to the functionality exposed via the REST API in Windows Azure Service Bus. The SDK is available as open source and you can download it here.
This is an early step as we continue to make Windows Azure a great cloud platform for many languages, including .NET, Java, and PHP. If you’re using Windows Azure Service Bus from PHP, please let us know your feedback on how this SDK is working for you and how we can improve them. Your feedback is very important to us!
You may refer to Windows Azure PHP Developer Center for related information.
Openness and interoperability are important to Microsoft, our customers, partners, and developers. We believe this SDK will enable PHP applications to more easily connect to Windows Azure making it easier for applications written on any platform to interoperate with one another through Windows Azure.
Thanks,Ram JeyaramanSenior Program ManagerMicrosoft Open Technologies, Inc.
Symfony, the popular open source web application framework for PHP developers, is now even easier to use on Windows Azure thanks to Benjamin Eberlei’s Azure Distribution Bundle project. You can find the source code and documentation on the project’s GitHub repo.
Symfony is a model-view-controller (MVC) framework that takes advantage of other open-source projects including Doctrine (ORM and database abstraction layer), PHP Data Objects (PDO), the PHPUnit unit testing framework, Twig template engine, and others. It eliminates common repetitive coding tasks so that PHP developers can build robust web apps quickly.
Symfony and Windows Azure are a powerful combination for building highly scalable PHP applications and services, and the Azure Distribution Bundle is a free set of tools, code, and documentation that makes it very easy to work with Symfony on Windows Azure. It includes functionality for streamlining the development experience, as well as tools to simplify deployment to Windows Azure.
Features that help streamline the Symfony development experience for Windows Azure include changes to allow use of the Symfony Sandbox on Windows Azure, functionality for distributed session management, and a REST API that gives Symfony developers access to Windows Azure services using the tools they already know best. On the deployment side, the Azure Distribution Bundle adds some new commands that are specific to Windows Azure to Symfony’s PHP app/console that make it easier to deploy Symfony applications to Windows Azure:
Benjamin Eberlei, lead developer on the project, has posted a quick-start video that shows how to install and work with the Azure Distribution Bundle. His video takes you through prerequisites, installation, and deployment of a simple sample application that takes advantage of the SQL Database Federations sharding capability built into the SQL Database feature of Windows Azure:
Whether you’re a Symfony developer already, or a PHP developer looking to get started on Windows Azure, you’ll find the Azure Distribution Bundle to be easy to use and flexible enough for a wide variety of applications and architectures. Download the package today – it includes all of the documentation and scaffolding you’ll need to get started. If you have ideas for making Symfony development on Windows Azure even easier, you can join the project and make contributions to the source code, or you can provide feedback through the project site or right here.
Symfony and Doctrine are often used in combination, as shown in the sample application mentioned above. For more information about working with Doctrine on Windows Azure, see the blog post Doctrine supports SQL Database Federations for massive scalability on Windows Azure.
Symfony and Doctrine have a rich history in the open source and PHP communities, and we’re looking forward to continuing our work with these communities to make Windows Azure a big part of the Symfony/Doctrine story going forward!
Doug MahughSenior Technical EvangelistMicrosoft Open Technologies, Inc.
Symfony and Doctrine are a popular combination for PHP developers, and now you can take full advantage of these open source frameworks on Windows Azure. We covered in a separate post the basics of getting started with Symfony on Windows Azure, and in this post we’ll take a look at Doctrine’s support for sharding via SQL Database Federations, which is the result of ongoing collaboration between Microsoft Open Technologies and members of the Symfony/Doctrine community.
My colleague Ram Jeyaraman covered in a blog post last December the availability of the SQL Database Federations specification. This specification covers a set of commands for managing federations as objects in a database. Just as you can use SQL commands to create a table or a stored procedure within a database, the SQL Database Federations spec covers how to create, use, or alter federations with simple commands such as CREATE FEDERATION, USE FEDERATION, or ALTER FEDERATION.
If you’ve never worked with federations before, the concept is actually quite simple. Your database is partitioned into a set of federation members, each of which contains a set of related data (typically group by a range of values for a specified federation distribution key):
This architecture can provide for massive scalability in the data tier of an application, because each federation member only handles a subset of the traffic and new federation members can be added at any time to increase capacity. And with the approach used by SQL Database Federations, developers don’t need to keep track of how the database is partitioned (sharded) across the federation members – the developer just needs to do a USE FEDERATION command and the data layer handles those details without any need to complicate the application code with sharding logic.
You can find a detailed explanation of sharding in the SQL Database Federations specification, which is a free download covered by the Microsoft Open Specification Promise. Questions or feedback on the specification are welcome on the MSDN forum for SQL Database.
The Doctrine Project is a set of open-source libraries that help ease database development and persistence logic for PHP developers. Doctrine includes a database abstraction layer (DBAL), object relational mapping (ORM) layer, and related services and APIs.
As of version 2.3 the Doctrine DBAL includes support for sharding, including a custom implementation of SQL Database Federations that’s ready to use with SQL Databases in Windows Azure. Instead of having to create Federations and schema separately, Doctrine does it all in one step. Furthermore, the combination of Symfony and Doctrine gives PHP developers seamless access to blob storage, Windows Azure Tables, Windows Azure queues, and other Windows Azure services.
The online documentation on the Doctrine site shows how easy it is to instantiate a ShardManager interface (the Doctrine API for sharding functionality) for a SQL Database:
The Doctrine site also has an end-to-end tutorial on how to do Doctrine sharding on Windows Azure, which covers creation of a federation, inserting data, repartitioning the federation members, and querying the data.
Doctrine’s sharding support gives PHP developers a simple option for building massively scalable applications and services on Windows Azure. You get the ease and flexibility of Doctrine Query Language (DQL) combined with the performance and durability of SQL Databases on Windows Azure, as well as access to Windows Azure services such as blob storage, table storage, queues, and others.
Microsoft Open Technologies, Inc. has just released an update to the open source HTTP Speed+Mobility Prototype that it first announced in early May to the developer community. This update implements the latest changes made by Microsoft to the HTTP Speed+Mobility proposal to the IETF httbis workgroup on June 15, 2012.
As Jean Paoli and Sandeep Singhal had articulated in their blog post back in March, the HTTPbis working group in the Internet Engineering Task Force (IETF) has approved a new charter to define HTTP “2.0” to address performance limitations with HTTP. The original HTTP Speed+Mobility proposal was the first contribution made by Microsoft toward that goal.
The updated proposal reaffirms the guiding principles of HTTP Speed+Mobility. Specifically, in our view any successful update to the HTTP protocol will have to:
We would like to thank the community for your interest in our proposal and for providing valuable feedback on the initial prototype implementation. We made several notable enhancements to the proposal, which the new version of the prototype now implements:
Collectively, these changes make the HTTP Speed+Mobility protocol both better integrated with the existing RFCs it builds upon, and at the same time, simpler to implement and debug.
As always, we encourage you to download the prototype, try it out, inspect the source code, and give us your feedback. We look forward to your contributions, as well as to constructive discussions about the next version of HTTP at the upcoming IETF meetings!
Adalberto Foresti Senior Program Manager Microsoft Open Technologies, Inc. A subsidiary of Microsoft Corporation
Do you need to build a high-availability web application or service? One that can scale out quickly in response to fluctuating demand? Need to do complex queries against schema-free collections of rich objects? If you answer yes to any of those questions, MongoDB on Windows Azure is an approach you’ll want to look at closely.
People have been using MongoDB on Windows Azure for some time (for example), but recently the setup, deployment, and development experience has been streamlined by the release of the MongoDB Installer for Windows Azure. It’s now easier than ever to get started with MongoDB on Windows Azure!
MongoDB
MongoDB is a very popular NoSQL database that stores data in collections of BSON (binary JSON) objects. It is very easy to learn if you have JavaScript (or Node.js) experience, featuring a JavaScript interpreter shell for administrating databases, JSON syntax for data updates and queries, and JavaScript-based map/reduce operations on the server. It is also known for a simple but flexible replication architecture based on replica sets, as well as sharding capabilities for load balancing and high availability. MongoDB is used in many high-volume web sites including Craigslist, FourSquare, Shutterfly, The New York Times, MTV, and others.
If you’re new to MongoDB, the best way to get started is to jump right in and start playing with it. Follow the instructions for your operating system from the list of Quickstart guides on MongoDB.org, and within a couple of minutes you’ll have a live MongoDB installation ready to use on your local machine. Then you can go through the MongoDB.org tutorial to learn the basics of creating databases and collections, inserting and updating documents, querying your data, and other common operations.
MongoDB Installer for Windows Azure
The MongoDB Installer for Windows Azure is a command-line tool (Windows PowerShell script) that automates the provisioning and deployment of MongoDB replica sets on Windows Azure virtual machines. You just need to specify a few options such as the number of nodes and the DNS prefix, and the installer will provision virtual machines, deploy MongoDB to them, and configure a replica set.
Once you have a replica set deployed, you’re ready to build your application or service. The tutorial How to deploy a PHP application using MongoDB on Windows Azure takes you through the steps involved for a simple demo app, including the details of configuring and deploying your application as a cloud service in Windows Azure. If you’re a PHP developer who is new to MongoDB, you may want to also check out the MongoDB tutorialon php.net.
Developer Choice
MongoDB is also supported by a wide array of programming languages, as you can see on the Drivers page of MongoDB.org. The example above is PHP-based, but if you’re a Node.js developer you can find a the tutorial Node.js Web Application with Storage on MongoDB over on the Developer Center, and for .NET developers looking to take advantage of MongoDB (either on Windows Azure or Windows), be sure to register for the free July 19 webinar that will cover the latest features of the MongoDB .NET driver in detail.
The team here at Microsoft Open Technologies is looking forward to working closely with 10gen to continue to improve the MongoDB developer experience on Windows Azure going forward. We’ll keep you updated here as that collaboration continues!
If you need a highly scalable data layer for your cloud service or application running on Windows Azure, the Cloudant Data Layer for Windows Azure may be a great fit. This service, which was announced in preview mode in June and is now in beta, delivers Cloudant’s “database as a service” offering on Windows Azure.
From Cloudant’s data layer you’ll get rich support for data replication and synchronization scenarios such as online/offline data access for mobile device support, a RESTful Apache CouchDB-compatible API, and powerful features including full-text search, geo-location, federated analytics, schema-less document collections, and many others. And perhaps the greatest benefit of all is what you don’t get with Cloudant’s approach: you’ll have no responsibility for provisioning, deploying, or managing your data layer. The experts at Cloudant take care of those details, while you stay focused on building applications and cloud services that use the data layer.
You can do your development in any of the many languages supported on Windows Azure, such as .NET, Node.JS, Java, PHP, or Python. In addition, you’ll get the benefits of Windows Azure’s CDN (Content Delivery Network) for low-latency data access in diverse locations. Cloudant pushes your data to data centers all around the globe, keeping it close to the people and services who need to consume it.
For a free trial of the Cloudant Data Layer for Windows Azure, create a new account on the signup page and select “Lagoon” as your data center location.
For an example of how to use the Cloudant Data Layer, see the tutorial “Using the Cloudant Data Layer for Windows Azure,” which takes you through the steps needed to set up an account, create a database, configure access permissions, and develop a simple PHP-based photo album application that uses the database to store text and images:
The sample app uses the SAG for CouchDB library for simple data access. SAG works against any Apache CouchDB database, as well as Cloudant’s CouchDB-compatible API for the data layer.
My colleague Olivier Bloch has provided another great example of using existing CouchDB libraries to simplify development when using the Cloudant Data Layer. In this video, he demonstrates how to put a nice Windows 8 design front end on top of the photo album demo app:
This example takes advantage of the couch.js library available from the Apache CouchDB project, as well as the GridApp template that comes with Visual Studio 2012. Olivier shows how to quickly create the app running against a local CouchDB installation, then by simply changing the connection string the app is running live against the Cloudant data layer running on Windows Azure.
The Cloudant data layer is a great example of the new types of capabilities – and developer opportunities – that have been created by Windows Azure’s support for Linux virtual machines. As Sam Bisbee noted in Cloudant’s announcement of the service, “The addition of Linux-based virtual machines made it possible for us to offer the Cloudant Data Layer service on Azure.”
If you’re looking for a way to quickly build apps and services on top of a scalable high-performance data layer, check out what the Cloudant Data Layer for Windows Azure has to offer!
Doug Mahugh Senior Technical Evangelist Microsoft Open Technologies, Inc.
Jason Zander blogged about new releases of Microsoft’s developer tools today – tools that include many contributions from the open source community with the MS Open Tech Hub on CodePlex.
The OSS community helped build out the source code for ASP.NET MVC 4, Web API, Web Pages 2 and Entity Framework 5 – key components in the new releases of Visual Studio 2012, Team Foundation Server 2012, and .NET Framework 4.5. Through CodePlex, developers outside Microsoft submitted patches and code contributions that the MS Open Tech Hub development team reviewed for potential inclusion in these products. I described this process in more detail last month, More of Microsoft’s App Development Tools Goes Open Source.
Today’s news had an additional cool factor. As Jason highlighted in his blog, “Developing great apps for Windows 8 is an important goal of this release. Therefore, in coordination with today’s developer tools releases, you’ll notice that the final version of Windows 8 has released to the web as well.”
There is a ton of great resources on these tools that you can check out and download today. The ASP.net website is a great place to start. I also recommend my friend Scott Hanselman’s new videos.
Microsoft’s partner-centric approach has been with the company since the very beginning. Today’s milestone shows that all developers can contribute to and benefit from Microsoft’s open platforms in the future.
Gianugo Rabellino Senior Director Open Source Communities Microsoft Open Technologies, Inc. A subsidiary of Microsoft Corporation
Microsoft at DrupalCon is becoming a tradition. After having partnered closely with the Drupal community to make Drupal available on Windows, Microsoft teams are continuing this engagement and are eager to meet Drupal developers in Munich next week.
If you are going to the event, don’t miss the various panels and sessions Microsoft attendees will participate:
And of course, you should stop by the booth to say hi!
If you’re not in Germany and can’t attend DrupalCon, we encourage you to follow @Gracefr and @Brian_Swan on Twitter. They’ll provide insights on the cool things happening there. Don’t miss Brian’s blog that offers great technical information on Drupal on Windows Azure and other related topics: a must read!
Gearing up for back to school, the Microsoft Open Technologies Inc. team has been busy updating the Windows Azure Plugin for Eclipse with Java.
This August 2012 Preview update includes some feedback-driven usability enhancements in existing features along with number additional bug fixes since the July 2012 Preview. The principal enhancements are the following:
You can learn more about the plugin on the Windows Azure Dev Center.
To find out how to install, go here.
Martin Sawicki Principal Program Manager Microsoft Open Technologies, Inc. A subsidiary of Microsoft Corporation
Today I’m happy to announce the open source release of the npm.net library. This is the same library that the WebMatrix team used to implement the NPM package discovery feature as explained in Justin’s blog. The library gives developers using managed code access to NPM commands to, for instance, deploy or update node.js modules on a client machine.
We are releasing the source code of the library today so developers that are interested in building automation tools or any other sort of integration between node.js and .NET can leverage some of the work we have done for the WebMatrix team.
Claudio CaldatoPrincipal Program ManagerMicrosoft Open Technologies, Inc.
Henrik Frystyk Nielsen Principal Architect, Microsoft Open Technologies, Inc.
Gabriel Montenegro Principal Software Development Engineer, Microsoft Corporation
We just came back from the IETF meeting in Vancouver, where the HTTP working group was meeting to decide on the way forward for HTTP/2.0. We are very happy with the discussions and overall outcomes as reflected in the meeting minutes and as summarized by the Chair, Mark Nottingham. At the meeting, the working group clarified the direction for HTTP/2.0 and began to draft a new charter. The group agreed that seven key areas need deep, data-driven discussion as part of the HTTP/2.0 specification process, and the resulting standard will not be backward compatible with any existing proposals (SPDY, HTTP Speed+Mobility, and Network-Friendly HTTP Upgrade). The charter calls for a proposed completion date for the standard of November 2014. In other words, while we are excited about where we are, it is clear that we are just at the beginning of the process toward HTTP 2.0.
The meeting outlined clearly the need for discussions and consensus over seven key technical areas such as Compression, Mandatory TLS, and Client Pull/Server Push. This list of issues is aligned with the position that Microsoft’s Henrik Frystyk Nielsen outlined in an earlier message to the HTTP discussion list (see excerpts below). Overall, we believe there needs to be robust discussions about how we bring together the best elements of the current SPDY, HTTP Speed+Mobility, and Network-Friendly HTTP Upgrade proposals.
Area
Opinion that seems to prevail
1. Compression
SPDY or Friendly
2. Multiplexing
SPDY
3. Mandatory TLS
Speed+Mobility
4. Negotiation
Friendly or Speed+Mobility
5. Client Pull/Server Push
Speed+Mobillity
6. Flow Control
7. WebSockets
We are particularly gratified to see this language in the proposed charter:
“It is expected that HTTP/2.0 will: * Substantially and measurably improve end-user perceived latency in most cases, over HTTP/1.1 using TCP.”
This supports Microsoft’s position that the HTTP update must be data-driven to ensure that it provides the desired benefits for users. . The SPDY proposal has done a good job of raising awareness of the opportunities to improve Web performance.
To compare the performance of SPDY with HTTP 1.1 we have run tests comparing download times of several public web sites using a controlled tested study. The test uses publically available software run with mostly default configurations while applying all the currently available optimizations to HTTP 1.1. You can find a preliminary report on the test results here: http://research.microsoft.com/apps/pubs/?id=170059. The results mirror other data (http://www.guypo.com/technical/not-as-spdy-as-you-thought) that indicate mixed results with SPDY performance.
Our results indicate almost equal performance between SPDY and HTTP 1.1 when one applies all the known optimizations to HTTP 1.1. SPDY's performance improvements are not consistent and significant. We will continue our testing, and we welcome others to publish their results so that HTTP 2.0 can choose the best changes and deliver the best possible performance and scalability improvements compared to HTTP 1.1.
We discussed those results in Vancouver and it was great to see the interest that this research received from the community on the IETF mailing list and on Twitter.
In light of the discussions and the proposed charter, HTTP2.0 will undoubtedly not be backward compatible with any of the current proposals (SPDY, Speed+Mobility, Friendly); in fact, we expect that it might differ in substantial ways from each of these proposals. Consequently, we caution implementers against embracing unstable versions of the specification too eagerly. The proposed charter calls for an IETF standard by November 2014.
We are happy that the working group decided, for practical reasons, to use the text from http://datatracker.ietf.org/doc/draft-mbelshe-httpbis-spdy/ as a starting point. The discussions around the previously cited seven design elements will deeply modify this text . As the Chair wrote, “It’s important to understand that SPDY isn’t being adopted as HTTP/2.0” . This is in line with the Microsoft approach: Our HTTP Speed+Mobility proposal starts from both the Google SPDY protocol (a separate submission to the IETF for this discussion) and the work the industry has done around WebSockets, and the main departures from SPDY are to address the needs of mobile devices and applications.
We’re excited for the web to get faster, more stable, and more capable. HTTP/2.0 is an important part of that progress, and we look forward to an HTTP/2.0 that meets the needs of the entire web, including browsers, apps, and mobile devices.
Henrik Frystyk Nielsen, Gabriel Montenegro and Rob Trace
“Dear All,
We remain committed to the HTTP/2.0 standards process and look forward to seeing many of you this week at the IETF meeting in Vancouver to continue the discussion. In the spirit of open discussion, we wanted to share some observations in advance of the meeting and share the latest progress from prototyping and testing.
There are currently three different proposals that the group is working through:
* SPDY (http://tools.ietf.org/html/draft-mbelshe-httpbis-spdy), * HTTP Speed+Mobility (http://tools.ietf.org/html/draft-montenegro-httpbis-speed-mobility), * Network-Friendly HTTP Upgrade (http://tools.ietf.org/html/draft-tarreau-httpbis-network-friendly).
The good news is that everyone involved wants to make the Web faster, more scalable, more secure, and more mobile-friendly, and each proposal has benefits in different areas that the discussion can choose from.
--- A Genuinely Faster Web ---
The SPDY proposal has been great for raising awareness of Web performance. It takes a "clean slate" approach to improving HTTP.
To compare the performance of SPDY with HTTP/1.1 we have run tests comparing download times of several public web sites using a controlled tested study. The test uses publically available software run with mostly default configurations while applying all the currently available optimizations to HTTP/1.1. You can find a preliminary report on the test results here: http://research.microsoft.com/apps/pubs/?id=170059. The results mirror other data (http://www.guypo.com/technical/not-as-spdy-as-you-thought) that indicate mixed results with SPDY performance.
Our results indicate almost equal performance between SPDY and HTTP/1.1 when one applies all the known optimizations to HTTP/1.1. SPDY's performance improvements are not consistent and significant. We will continue our testing, and we welcome others to publish their results so that HTTP/2.0 can choose the best changes and deliver the best possible performance and scalability improvements compared to HTTP/1.1.
--- Taking the Best from Each ---
Speed is one of several areas of improvement. Currently, there's no clear consensus that any one of the proposals is the clear choice or even starting point for HTTP/2.0 (based on our reading the Expressions of Interest and discussions on this mailing list. A good example of this is the vigorous discussion around mandating TLS encryption (http://tools.ietf.org/html/rfc5246) for HTTP/2.0.
We think a good approach for HTTP/2.0 is to take the best solution for each of these areas from each of the proposals. This approach helps us focus the discussion for each area of the protocol. Of course, this approach would still allow the standard to benefit from the extensive knowledge gained from implementing existing proposals.
We believe that the group can converge on consensus in the following areas, based on our reading of the Expressions of Interest, by starting from the different proposals.
------------------|------------------ Area | Opinion that | seems to prevail ------------------|------------------ 1. Compression | SPDY or Friendly ------------------|------------------ 2. Multiplexing | SPDY ------------------|------------------ 3. Mandatory TLS | Speed+Mobility ------------------|------------------ 4. Negotiation | Friendly or | Speed+Mobility ------------------|------------------ 5. Client Pull/ | Speed+Mobility Server Push | ------------------|------------------ 6. Flow Control | SPDY ------------------|------------------ 7. WebSockets | Speed+Mobility ------------------|------------------
Below, we discuss each HTTP/2.0 element and the current consensus that appears to be forming within the Working Group.
Compression is simple to conceptualize and implement, and it is important. Proxies and other boxes in the middle on today's Web often face problems with it. The HTTP/2.0 discussion has been rich but with little consensus.
Though some studies suggest that SPDY's header compression approach shows promise, other studies show this compression to be prohibitively onerous for intermediary devices. More information here would help us make sure we're making the Web faster and better.
Also, an entire segment of implementers are not interested in compression as defined in SPDY. That's a challenge because the latest strawman for the working group charter (http://lists.w3.org/Archives/Public/ietf-http-wg/2012JulSep/0784.html) states that the "resulting specification(s) are expected to be meet these goals for common existing deployments of HTTP; in particular, ... intermediation (by proxies, Corporate firewalls, 'reverse' proxies and Content Delivery Networks)."
We think the SPDY or Friendly proposals is a good starting point for progress.
All three proposals define similar multiplexing models. We haven't had substantial discussion on the differences. This lack of discussion suggests that there is rough consensus around the SPDY framing for multiplexing.
We think that the SPDY proposal is a good starting point here and best captures the current consensus.
3. Mandating Always On TLS
There is definitely no consensus to mandate TLS for all Web communication, but some major implementers have stated they will not to adopt HTTP/2.0 unless the working group supports a "TLS is mandatory" position. A very preliminary note from the chair (http://lists.w3.org/Archives/Public/ietf-http-wg/2012JulSep/0601.html) states that there is a lack of consensus for mandating TLS.
We think the Speed+Mobility proposal is a good starting point here as it provides options to turn TLS on (or not).
Only two of the proposals actually discuss how different endpoints agree to use HTTP/2.0.
(The SPDY proposal does not specify a negotiation method. Current prototype implementations use the TLS-NPN (http://tools.ietf.org/html/draft-agl-tls-nextprotoneg) extension. While the other proposals use HTTP Upgrade to negotiate HTTP/2.0, some parties have expressed non-support for this method as well.)
We think either of the Friendly or Speed+Mobility proposals is a good starting point because they are the only ones that have any language in this respect.
5. Client Pull and Server Push
There are tradeoffs between a server push model and a client pull model. The main question is how to improve performance while respecting bandwidth and client caches.
Server Push has not had the same level of implementation and experimentation as the other features in SPDY. More information here would help us make sure we're making the Web faster and better.
We think the Speed+Mobility proposal is a good starting point here, suggesting that this issue may be better served in a separate document rather than tied to the core HTTP/2.0 protocol.
There has only been limited discussion in the HTTPbis working group on flow control. Flow Control offers a lot of opportunity make the Web faster as well as to break it; for example, implementations need to figure out how to optimize for opposing goals (like throughput and responsiveness) at the same time.
The current version of the SPDY proposal specifies a flow control message with many settings are that are not well-defined. The Speed+Mobilty proposal has a simplified flow control model based on certain assumptions. More experimentation and information here would help us make sure we're making the Web faster and better.
We think that the SPDY proposal is a good starting point here.
We see support for aligning HTTP/2.0 with a future version of WebSockets, as suggested in the introduction of the Speed+Mobility proposal.
--- Moving forward ---
We're excited for the Web to get faster, more stable, and more capable, and HTTP/2.0 is an important part of that.
We believe that bringing together the best elements of the current SPDY, HTTP Speed+Mobility, and Network-Friendly HTTP Upgrade proposals is the best approach to make that happen.
Based on the discussions on the HTTPbis mailing list, we've suggested which proposals make the most sense to start from for each of the areas that HTTP/2.0 is addressing. Each of these areas needs more prototyping and experimentation and data. We're looking forward to the discussion this week.
Sincerely,
Henrik Frystyk Nielsen
Principal Architect, Microsoft Open Technologies, Inc.
Gabriel Montenegro
Principal Software Development Engineer, Microsoft Corporation
Rob Trace
Senior Program Manager Lead, Microsoft Corporation
Adalberto Foresti
Senior Program Manager, Microsoft Open Technologies, Inc.”
It was great to see everyone at OSCON last week! The MS Open Tech team had a fun and productive week meeting new people, reconnecting with old friends, learning about the latest OSS trends, and playing with the amazing 82" Perceptive Pixel touch screen at our booth. Julian Cash took over 4000 photos of visitors to the booth, and if you were one of the lucky people who spent time making creative photos with him, stay tuned. We'll post an update shortly when all of the photos have been uploaded to his web site.
If you weren't able to attend OSCON this year, you can find speaker slides and videos on the OSCON web site. Those videos are also a great resource for those who attended the conference -- for example, I've just finished watching Laurie Petrycki's interview with Alex Payne about Scala's interesting combination of functional and object-oriented programming language constructs.
There are two interviews with our team's leader Gianugo Rabellino that are available on YouTube and well worth watching to better understand the work we're doing with open source communities:
Thanks to all the hard-working event organizers, exhibitors, sponsors, and attendees who made OSCON such a well-run and successful show!
Citrix, IBM, Microsoft, Progress Software, SAP AG, and WSO2 have submitted a proposal to OASIS to begin the formal standardization process for OData. You can find all the details here, and OData architect Pablo Castro also provides some context for this announcement over on the OData.org blog. It’s an exciting time for the OData community!
OData is a REST-based web protocol for querying and updating data, and it’s built on standardized technologies such as HTTP, Atom/XML, and JSON. If you’re not already familiar with OData, the OData.org web site is the best place to learn more.
Many organizations are already working with OData, and it has proven to be a useful and flexible technology for enabling interoperability between disparate data sources, applications, services, and clients. Chris Woodruff has a blog post this week that lists many OData implementations, and as he explained in a post last week, “By having data that is easy to consume and understand organizations can allow their customers and partners (via the developers that build the solutions using one or more of the available OData libraries) to leverage the value of curated data that the organization owns.” Many organizations are already pursuing that vision – as Ralf Handl of SAP AG told us at a recent OData meetup, “my job is relatively simple: I want to put OData into all of our products.”
We support OData in many Microsoft products and services, and the list is growing longer all the time. This includes OData consumers such as Microsoft Excel (via the free PowerPivot add-in) as well as OData producers such as Microsoft SharePoint, Microsoft SQL Server Reporting Services, and Microsoft Dynamics CRM. Windows Server supports OData, and Windows Azure provides OData support in many areas, including Windows Azure Storage Table Service, Windows Azure Marketplace, and ACS Management Service. We’re also making many Microsoft data sources available in OData format. For example:
A variety of OSS technologies can benefit from OData support, and our team has delivered tools to make it easy for OSS developers to expose data as OData from a variety of platforms. Earlier this year we announced Open Source OData Tools for MySQL and PHP Developers, including the OData Producer Library for PHP and the OData Connector for MySQL. We’re continuing to work closely with various OSS communities on OData support, and we’ll be releasing information soon on new ways to provide OData feeds from popular OSS frameworks and applications.
OData’s query syntax is straightforward from a developer’s perspective. For example, here’s a query that you can use in any browser to return the count of the number of products in the sample Northwind database OData feed on OData.org:
http://services.odata.org/Northwind/Northwind.svc/Products/$count
In a typical application, that query would be generated behind the scenes, and the returned result would be rendered in a nicely formatted manner as appropriate for the particular application.
To enable those sorts of scenarios, developers need OData support for the languages, framework, and tools that they’re already using. Many developer tools already offer OData support. Here are a few examples:
As you can see, the OData ecosystem is growing, and awareness of OData is growing with it. At the OData meetup earlier this year, we heard from many people who are finding innovative ways to use OData in their organizations to improve customer service, enable new scenarios, and increase efficiency. Anant Jhingran of APIgee stated in his presentation at the meetup that “if data isn’t your core business, then you should give it away.” It was a provocative statement, and for those who share that philosophy, OData is a great tool for making it easier to share data.
If you’re interested in implementing OData or contributing to the OData standard, now’s the time to get involved. You can work with the odata.org community to help drive awareness and share implementation experiences, or join the OASIS OData technical committee (OData TC) to contribute to the standard. The OData TC will be a vibrant and diverse group of people – just like the community who got us here today – working together to open up data sources in a standardized way. As Pablo stated in his blog post, the main value of OData is not any particular design choice, but the fact that enough people agree to the same pattern, thus removing friction from sharing data across independent producers and consumers. The first TC call will be in late July, so there’s still plenty of time to get involved if you’d like to be part of the team that will be helping OData evolve.
Congratulations to everyone who has worked so hard to get OData to this important step on the journey to standardization! We’re looking forward to working with the community to develop OData into a formal standard through OASIS.
Hey OSCON friends, drum roll please … our Microsoft-sponsored photographer and joyful open source geek extraordinaire Julian Cash has posted your photos from our booth, along with a fun video, on his JC Event Photo OSCON Event page and his Facebook page. Find the photo you love from among his shots of you, give it a right-click, “save picture as…” in your favorite format, and it’s all yours. Copy the photo to your favorite social media sites and send a copy to Mom – I did!
On behalf of the MS Open Tech evangelism team, thanks to everyone who spent time with us at OSCON. I blogged earlier about the new friends we made and the cool conversations we all had about emerging technologies, but I think it’s time to simply say that a picture is a worth a thousand words …
Clockwise from top right: Gianugo Rabellino, “Grazie!”— Olivier Bloch, “Merci!” — Doug Mahugh and Robin Bender Ginn, “Thank you! Thank you!”
Jean Paoli, President, Microsoft Open Technologies, Inc.
Michael Champion, Senior Program Manager, Microsoft Open Technologies, Inc.
We are thrilled to share the news that the W3C announced the alpha release of Web Platform Docs. Adobe, Facebook, Google, HP, Microsoft, Mozilla, Nokia and Opera are among the stewards of the project. Together, we worked with the W3C on creating this wiki-styled site and contributed thousands of web documentation articles.
W3C’s Web Platform Docs is a community site designed to be a comprehensive and authoritative resource for developers to help them build modern web applications that will work across browsers and devices, and share their own expertise, which will further the goal of web platform interoperability and same markup.
Currently, developers need to do a lot of research about what technologies work on which platforms when building websites and applications with HTML5, CSS and other open web standards. It’s costly and inefficient for them to spend precious hours consulting multiple resources to understand how to employ web technologies in a way that functions across browsers, operating systems and devices. W3C’s Web Platform Docs addresses these issues by offering a single “go-to” source for web developer documentation, and providing a site that the community can continually edit and improve.
Microsoft Open Technologies, Inc., represented by Michael Champion, and the Microsoft Internet Explorer team represented by Eliot Graff, have been involved from the very inception of the project as we strongly believe this community site is key in the journey to an interoperable web platform and same markup.
As an initial contribution, Microsoft donated more than 3,200 topics from MSDN and will continue to add content moving forward. This is an open community – web developers can get an account at webplatform.org to make their own contribution – fill in gaps, correct errors, and flesh out the documentation with sample code to explain how to use the web platform to its full potential.
So what does this mean for you, the developer?
You will save time and resources, knowing you can consult with confidence a community-curated site to learn about standards, innovations and best practices including:
W3C’s Web Platform Docs is an open site where anyone can become a member and contribute. Microsoft and the other founding stewards helped boot up the wiki (and will continue to contribute new content), but YOU, the developer community, own the site. W3C convened the community and will administer webplatform.org in the future, but you don’t have to join W3C to participate in this effort.
All materials on W3C’s Web Platform Docs are freely available and licensed to foster sharing and reuse.
Begin simplifying your web development and check out W3C’s Web Platform Docs today. Better still, sign up for an account, find a topic of interest, and contribute your expertise!
Two years ago we shared our view on Interoperability Elements of a Cloud Platform. Back then we talked to customers and developers and came out with an overview of an open and interoperable cloud, based on four distinct elements: Data Portability, Standards, Ease of Migration and Deployment, and Developer Choice. Since then, we have been laser focused on the quest for an interoperable and flexible cloud platform that would enable heterogeneous workloads.
Windows Azure is committed to openness across the entire application stack, with service APIs and service management APIs exposed as RESTful endpoints that can be used from any language or runtime, key services such as Caching, Service Bus, and Identity that can be hosted either on-premises or in the cloud, and open source SDKs for popular languages that give developers a choice of tools for building cloud-based applications and services.
In this blog post I’ll recap some of the most important news of the last year in each of these areas. As I mentioned in a blog postearlier this year, when a journey reaches an important milestone it’s good to look back and think about the road so far. We’ve come even farther down that road now, and here are many technical examples of what has been accomplished.
Data Portability
When customers create data in an on-premises application, they have a high level of confidence that they have control over the data stored in the on-premise environment. Customers should have a comparable level of control over their data when they are using cloud platforms. Here are some examples of how Windows Azure supports Data Portability:
Standards
Cloud platforms should reuse existing and commonly used standards when it makes sense to do so. If existing standards are not sufficient, new standards may be created. Here are some of the ways we’re working to support standards for cloud computing:
Ease of Migration and Deployment
Cloud platforms should provide a secure migration path that preserves existing investments and enable co-existence between on-premise software and cloud services. Here are some examples of ease of migration and deployment on Windows Azure:
Cloud platforms should enable developer choice in tools, languages and runtimes to facilitate the development of interoperable customer solutions. This approach will also broaden the community of developers that write for a given cloud platform and therefore enhance the quality of services that the platform will offer to customers. Here are some of the ways that Windows Azure is delivering on developer choice:
It’s exciting to see how far we’ve come, and we still have much to do as well. The Interoperability Elements of a Cloud Platform originally came out of discussions with customers, partners, and developers about what they need from an interoperable cloud, and we’re continuing those discussions going forward, and we will continue to deliver on these important elements!
Gianugo RabellinoSenior Director, Open Source CommunitiesMicrosoft Open Technologies, Inc.
MS Open Tech Announces Intent to Implement in Blink While Continuing WebKit Implementation
Asir Vedamuthu Selvasingh, Principal Program Manager Microsoft Open Technologies, Inc.
The W3C Pointer Events emerging standard continues to gain traction, advancing support for interoperable mouse, touch, and pen interactions across the web. Today, Microsoft Open Technologies, Inc. (MS Open Tech) made the first step towards interoperable support for Pointer Events in Blink by submitting a formal Intent to Implement to enable our engineering team to actively collaborate and work toward a positive adoption of Pointer Events by the Blink developer community.
Earlier this year, MS Open Tech published a Pointer Events prototype for WebKit on HTML5 Labs and submitted the patch to the WebKit developer forum. To help even more developers adopt the Pointer Events technology, we plan to continue our collaboration with the WebKit community.
Pointer Events makes it easier to support many browsers and devices by saving Web developers from writing unique code for each input type. Today people interact with Web content on a range of devices – phones, tablets, PCs, even the living room TV. Pointer Events unifies how you code for point, click and touch across these devices. The input model is based on the APIs already available in IE10 on Windows 8 and Windows Phone 8 and you can start building websites incorporating point, click, and touch today.
Pointer Events continues to receive positive feedback from the developer community -- many are already embracing it as a unified model for cross-browser multi-modal input:
The jQuery team has been watching the progress of Pointer Events and participating in the W3C standardization process as it moves to Candidate Recommendation. Congratulations to the W3C working group for their hard work in getting to the CR stage. We see Pointer Events as a great way to unify the haphazard models that exist today for mouse, touch, and other pointer devices. Our team looks forward to supporting web developer use of Pointer Events as implementations begin to arrive.
Dave Methvin, President jQuery Foundation
Pointer Events is a great way to add cross-browser support for multi-modal input from mouse, pointer, and touch. With Pointer Events reaching Candidate Recommendation stage, it is time all mobile html5 application developers paid attention. The best part; you can start using it today….
Jesse MacFadyen, Adobe Developer, Cordova Committer
Pointer Events Advances to W3C Candidate Recommendation (CR)
As announced last week by the W3C and shared by our colleagues in IE, the W3C has now published the Pointer Events specification as a Candidate Recommendation, an important step toward a standard. This progression from Last Call Working Draft to Candidate Recommendation is a mark of the effective collaboration among Microsoft Corp., Google, Mozilla, Opera, Nokia, jQuery, and others to help sites take advantage of new interactive devices for the Web.
‘Candidate Recommendation’ indicates that the W3C considers the specification widely reviewed and satisfying the Working Group’s technical requirements. It signals a call for additional implementation experience to inform the group.
MS Open Tech and the Microsoft Internet Explorer teams will continue to work with our colleagues across the industry, engaging developers to test and provide feedback on the specification to W3C.
Learn more about Pointer Events on Web Platform Docs
As you start building, migrating, or testing your apps using Pointer Events on various browser platforms, you should check out the resources available on the Pointer Events Wiki at Web Platform Docs:
So much progress -- Pointer Events enables you to build for the future of the Web, today. So when someone invents the next big input breakthrough (such as Tom Cruise’s crime lab from Minority Report or Tony Stark’s holographic CAD console from Iron Man), you’re already on your way to supporting new input features.
Jump in, have fun with the demos, join the discussion at #PointerEvents and update your site with the cool capabilities of Pointer Events. Point. Click. Touch.
Thanks to all our friends who sent us anniversary greetings and also those who joined us in the Valley last week to celebrate. It was great to reflect on the many open source and open standards projects with community and industry leaders in the house.
The future looks bright as we look to collaborate on more openness projects with you!
From: Martin Sawicki – Principal Program Manager – MS Open Tech Brian Benz – Sr. Technical Evangelist – MS Open Tech
Incorporating developer feedback and keeping pace with Windows Azure’s ongoing evolution, Microsoft Open Technologies, Inc., has released a new update to the Windows Azure Toolkit for Eclipse. This latest version – the May 2013 Preview (or v2.0.0) – includes a number of new improvements we hope will further simplify your Java work in the Windows Azure cloud. This major update accompanies the release of the Windows Azure SDK v2.0.
This new option automatically uploads the selected JDK and Web Application Server, when needed, to a selected Windows Azure storage account and deploys these components from that account. This commonly requested feature can greatly enhance the ease and efficiency of deploying the JDK and server components, as well as help with team development scenarios, by avoiding the need to embed these components in the deployment package, or the hassle of manual uploads. Our “Hello World” tutorial walks you through these steps in detail. Here is a screenshot from the updated project creation wizard showing this feature in action:
You may be wondering how Windows Azure knows which storage accounts to display in the drop-down list. The answer is another new feature – centralized storage account tracking. In order to reference storage accounts more easily across the various features that rely on storage, like caching and, in this case, the JDK or the Web Application server component deployment, you can register existing storage accounts names and endpoints in your current Eclipse workspace under Preferences > Windows Azure > Storage Accounts:
As usual, you can import the information from your subscription’s publish settings file to provide the storage account discovery magic.
Here’s an example of the centralized storage account tracking for caching as well. The drop down shows you where you would specify which storage account to store credentials for the cache configuration in a Windows Azure role. This way you no longer have to enter the access key manually.
For this release we’ve also streamlined the previous two-step remote access setup into one step. In the “Publish to Cloud” wizard, type in a user name and password to enable remote access, or leave it blank to keep remote access disabled.
By default, Eclipse will use the sample certificate for encrypting your remote access credentials in the Windows Azure configuration file. If you’d rather use your own certificate, choose the Advanced… link.
The latest SDK distribution of the Windows Azure Libraries for Java are packaged along with their open-source dependencies as an Eclipse library and referred to as the Package for Windows Azure Libraries for Java (by MS Open Tech). If your Java code needs to take advantage of the Windows Azure API, just add this library to the build path of your Java project (not the Windows Azure project) and all the needed libraries will be automatically referenced and included in your WAR file.
We’ve also fixed sticky sessions on Windows Server 2012 (previously limited to Windows 2008). And we’ve made some package publish performance improvements that can make the upload portion of the deployment process up to twice as fast as in previous releases, especially helpful if you’re not using the “deploy from download” options, but still embedding large components in the deployment package itself. We’ve also made a number of bug fixes, including some reported issues related to deploying Tomcat and Jetty on Windows 2012 from a cloud download.
Here are the complete instructions to download and install the Windows Azure Plugin for Eclipse with Java, as well as updated documentation.
As always, let us know how the latest release works for you and how you like the new features! To send feedback or questions, just use MSDN Forums or Stack Overflow.
I was fortunate to be invited to speak on behalf of MS Open Tech at last weekend’s LinuxFest Northwest in Bellingham, WA. This was a local event with a wide variety of developers and tech enthusiasts who gathered at Bellingham Technical College to participate in a broad spectrum of presentations, demonstrations, and labs.
My presentation Microsoft, Linux and the Open Source Community was part of the Developing a Community Track at LinuxFest so, given Microsoft Open Technologies, Inc. had just celebrated our One Year Anniversary, I took this opportunity to demonstrate some of the MS Open Tech projects that are enabling the open source community to benefit from new interoperability technology initiatives:
Linux on Windows Azure – Just prior to LinuxFest, the Azure team announced general availability of Windows Azure Infrastructure-as-a-Service. Windows Azure Infrastructure Services enable you to deploy and run durable VMs in the cloud. As well as Windows Server options, the built-in image gallery of VM templates includes Linux images for Ubuntu, CentOS, and SUSE Linux distributions. During my presentation session, I used Windows Azure Web Sites to create a new cloud-based WordPress site including a MySQL instance.
VM Depot – Built on the capabilities of Linux on Windows Azure, VM Depot is a cloud-based catalog of more than 200 open source Linux virtual machine images for Windows Azure contributed by the community. Developed by MS Open Tech, on VM Depot the community can build, deploy and share their favorite Linux configurations and other freely downloadable images, create custom open source stacks, and work with others to build new architectures for the cloud that leverage the openness and flexibility of the Windows Azure platform.
OData – OData is an open data protocol jointly developed by Microsoft, IBM, SAP, Citrix, and other industry partners and currently undergoing standardization via OASIS. We have recently revamped the http://odata.org website and encourage community contributions to develop consumer and producer services using OData as highlighted in the Ecosystem subsection. My demonstration showed how OData can be used by the community to publish and access open government data using the DataLab open source code on GitHub.
Pointer Events - Pointer events is an emerging standard developed by the W3C to define a single device input model – mouse, pen and touch – across multiple browsers. Microsoft contributed the initial specification and is working to demonstrate cross browser interoperability for Pointer Events. MS Open Tech developed an open source Pointer Events prototype for WebKit on HTML5 Labs and submitted the patch to the WebKit community. We encourage the developer community to learn more about Pointer Events on Web Platform Docs and join the #PointerEvents discussion.
I would like to thank the LinuxFest organizers for the opportunity to participate. Events like LinuxFest are an ideal way for us to share the work we do at MS Open Tech with the open source community and seek feedback on our efforts. I look forward to the next opportunity.
Every CIO today has mobile VPN access sitting very high on the priority list and knows how mobile connectivity to corporate resources is an important yet complicate matter. In a quest to solve the VPN issue in a simple way, Microsoft Open Technologies, Inc. created AppGateway, an application of VPN access for Android and iOS based on the preview of a Windows Azure Application Gateway service meant to help mobile users be more productive on the go. With this app, developed in collaboration with the Windows Azure Active Directory team, Android and iOS users can experience easy connectivity to web sites behind the corporate firewall, leveraging Windows Azure authentication services.
For obvious security reasons, connecting to resources that are behind a firewall usually requires a fairly complex infrastructure such as VPN (Virtual Private Network). The AppGateway demo app is designed to make the process simpler yet highly secured. The mobile app connects to a service on Windows Azure that acts as the proxy to an agent that is running inside the network behind the firewall. Using the Windows Azure Authentication service, the proxy can establish a trusted connection to the agent so that the application on the mobile device can browse web sites that would not be normally accessible outside the corporate network.
The demo app is available on the Apple App Store, GooglePlay and Amazon marketplaces at the following links:
For more information on the Windows Azure Application Gateway service preview, visit http://appgportal.cloudapp.net/.
In our best tradition, we also released the source code of the demo app. You can find the Android application code here.
For detailed instructions and a quick ‘get started; guide is available here.
We look forward for your feedback, comments and suggestions.
In early 2012 Microsoft chose Tel Aviv, as the location for its first start-up accelerator. When the team at Microsoft Research and Development, Israel asked if MS Open technologies, Inc. could speak about open source on Windows Azure at their Cloud Day event I immediately jumped at the chance since I’m fascinated by how open source can help accelerate innovation. It seemed to me that this event would attract the kinds of people who could show me a thing or two about innovation on the cloud.
The Cloud Day event was designed to provide an opportunity for members of the local start-up community to explore the latest & hottest trends in cloud computing and to gain insights on how to get the most value out of the cloud. Speakers included representatives of start-ups at varying stages of development, Venture Capitalists, Cloud Consultants and Microsoft Open Technologies.
My session focused on using open source software to get the most from the cloud. I discussed how Microsoft Open Technologies, Inc. work with projects and communities in order to ensure Windows Azure is an ideal platform for open source solutions. This was illustrated with a number of examples and case studies, such as how Ascribe Ltd transformed healthcare using open source big data solutions thanks to Microsoft’s contributions to Apache Hadoop and how Teletica.com used Azure and open source to manage a massive surge in web traffic. I also demonstrated how, using VM Depot makes it possible for anyone to build a video sharing site during their coffee break (more on that in a later post).
For me though what I had to say was not the important part, it was what the attendees had to say that interested me. After my session I was able to talk to quite a few people who were both keen to tell me about their start-up and to learn how they can make the most of open source software. Almost everyone I spoke to demonstrated a hunger, energy and determination that was nothing short of impressive. It's nowonder that the Microsoft Accelerator here has such a high success rate.
By the end of the day it was clear that open source is an important part of the start-up ecosystem in Israel and I, along with the rest of Microsoft Open Technologies, look forward to continuing to support the brilliant team at Microsoft Israel R&D a they continue to provide support for local innovation and business development.
Our team is making steady progress on building a solid framework for our new open source computing model for the cloud ActorFx, and we’re happy to report that we just released version 0.40 for this ActorFx project on CodePlex.
ActorFx provides a non-prescriptive, language-independent model of dynamic distributed objects for highly available data structures and other logical entities via a standardized framework and infrastructure.
Since Microsoft Open Technologies, Inc., announced that ActorFx joined the MS Open Tech Hub in December, we’ve been hard at work at adding new features in regular releases.
For this release we focused on adding some interesting features that enriches the framework and makes it suitable for a wider range of scenarios.
We are already working on the V0.50 release. Let us know if there is any feature you would like to be added to the ActorFx project and, as always, comments and feedback are welcome.
The ActorFx team:
Brian Grunkemeyer, Senior Software Engineer, Microsoft Open Technologies Hub
Joe Hoag, Senior Software Engineer, Microsoft Open Technologies Hub