Hello Windows Azure Marketplace users,
We’re back with another exciting set of announcements! In the past 4 months, we’ve not only supported the launch of a new storefront, but also made great strides in making our service more resilient, added a feature to help users guard against interruptions to service, added a huge portfolio of data services and app services, while continuing to improve the user experience on the portal and from Office products.
If you are a Developer, you will be very excited to know that we launched the Azure Store, which is a marketplace for data services and app services on the Windows Azure Portal, in October 2012. Want to learn more? Check out the keynote from Build 2012!
If you are an Analyst or an Information Worker, you will be interested in knowing that we improved the experience of using Windows Azure Marketplace from Excel PowerPivot, and we helped with the launch of Data Explorer, which is a data aggregation and shaping tool from SQL Information Services. Download Data Explorer and start playing with it right away!
Here are a few snapshots from Data Explorer:
Here are a few snapshots with the improved experience from Excel PowerPivot:
We have also continued to improve the user experience on the Windows Azure Marketplace Portal. We added a sitemap, which is a one-stop shop access point to all the important resources on the websites. With this feature, discoverability of resources on the website was greatly improved. We also made an improvement to the data visualization tool on our portal, Service Explorer, by replacing it with the more metro-looking Query Builder, to unify the data exploration experience found in Excel and PowerPivot. With this release, we also added support for download to PowerPivot 2010 and PowerPivot 2013, to target a broader set of customers.
We also added a feature in which users can opt-in to Windows Azure Marketplace automatically refilling their subscriptions when the subscription balances are low – we got a lot of feedback from our users that there ought to be a way to automatically re-subscribe to a subscription when the subscription balance runs out, and we heard the feedback, and we released the feature called Auto Refill, that did exactly that. Hmm, too good to be true? Read more about Auto Refill.
Here is a snapshot of the sitemap:
Here is a snapshot of the new data visualization tool on the Marketplace Portal:
Here are a few snapshots of Auto-Refill:
We have also released a good list of content in the past 5 months, and one of the interesting offerings was the Synonyms API that we made available through the Windows Azure Marketplace. The Synonyms API returns alternate references to real world entities like products, people, locations, and more.
For instance, the API intelligently matches the following real world entities to commonly used synonyms:
Try out this cool API for yourself right here.
We also released a ton of great content by Dun & Bradstreet, Digital Folio and our other key publishers, and here’s a short list:
To get a full list of data services, please click here and to get a full list of all the applications available through the Windows Azure Marketplace, please click here.
Thank you,
Windows Azure Marketplace Team
Here we go again, everybody... More content!!
We’re very happy to publish our first offering from China!
According to Chanjet: “In short, from the operational indicators for the SME (small and medium-sized enterprise) segment, we see that the impact of external factors outweighs the internal factors, and this is causing increasing operational risk for the Chinese SME. The key focus areas for the SME in order to minimize their operational risk, include: capital risk, receivable risk and inventory risk.” Did you know this? If you’re like me (I didn’t know this), then you, too, can benefit from some quality time poking around Chanjet’s China Small & Medium Enterprises Management & Operation KPI Data offering.
Our friends at LexisNexis have published Legal Communities. This offering provides access to the largest collection of Legal Blogs written by leading legal professionals, more than 1,600 Legal Podcasts featuring legal luminaries, information about Top Cases and Emerging Issues. The communities enable you to keep current with the latest Legal News, Issues and Trends. I recommend adding this to your daily blog reading regimen – right after your daily stop to the DataMarket blog.
And last, but certainly not least, check out of Sao Paulo, Brazil, courtesy of Sanborn!
Sanborn’s Building Model Service contains some 66 of the largest metropolitan urban areas across the United States, Mexico, South America, Canada, and Europe. This spatial database is comprised of photogrammetric extracted, from high resolution aerial photography, accurate and precise property-and-address-based building footprint geospatial building footprint vectors. Building footprints are concurrent with an accurate 3D building “Massing” model, allow for advanced modeling and analysis of urban buildings and structures, blocks and neighborhoods.
Enjoy new content!
The Windows Azure DataMarket Team
Hello everyone! It’s that time again… We’re happy to announce our latest content release!
Have you ever wondered if there are any environmental hazards around your house? If so, we’ve got the solution! Environmental Data Resources, Inc. is publishing their Environmental Hazard Rank offering. The EDR Environmental Hazard Ranking System depicts the relative environmental health of any U.S. ZIP code based on an advanced analysis of its environmental issues. The results are then aggregated by ZIP code to provide you with a rank so you can see how the ZIP code you're interested in stacks up.
It’s Trivia Time! Today’s question: Is the buying power of US dollar greater today than it was in 1976? And today’s Bonus Question: was the buying power of US dollar greater in 1976 than it was in 1923? If you’re sitting there scratching your head thinking “Hmm, these are good questions, Mr. Azure DataMarket Blog Writer Man”, then fear not, MetricMash has got you covered. They’re publishing their U.S. Consumer Price Index - 1913 to Current offering. This offering provides the changes in the prices paid by consumers for over 375 goods and services in major expenditure groups – such as food, housing, apparel, transportation, medical care and education cost. The CPI can be used to measure inflation and adjust the real value of wages, salaries and pensions for regulating prices and for calculating the real rate of return on investments. And, speaking of buying power, MetricMash is offering a free trial on this offering to make it easy for developers to use this information inside their apps.
- Mr. Azure DataMarket Blog Writer Man
Streamlined access to local weather information promises to empower product development ranging from consumer experience such as www.myweather.com, which provides real-time radar and a seamless worldwide satellite sector, to complex machine learning algorithms. If you’re interested in learning more about this topic, don’t miss the upcoming webcast, “Using Windows Azure DataMarket to Explore the Power of Local Weather”, which will take place Tuesday, June 14, 2011 8:00-9:00 am, PDT.
During this webcast, presenters Ben Zimmerman, Weather Central Dr. of Business Development and Elisa Flasko, Microsoft Program Manager, will explore how you can use local weather data to create compelling applications and revolutionize business intelligence. This session will also focus on Weather Central’s differentiating science, the ease of data integration offered through Windows Azure Marketplace DataMarket, and share several use case applications.
Click here to learn more and to register.
Today we are very excited to announce, in partnership with OnTerra Systems, the launch of the new DataMarket Map App in the Bing Map App Gallery!
The application allows you to use your existing DataMarket subscription to access and visualize layers of geospatial DataMarket data on Bing Maps. If latitude and longitude don’t exist in the data offer, then the Bing Maps geocoder will geocode on the fly.
Don’t have a DataMarket account yet? Well, you’ll definitely want to check it out and sign up, but there is some sample crime and demographic data available to give an example of how the app works.
-The DataMarket Team
Hi! I’m Jovana, and I’m currently interning on the DataMarket team. I come from sunny Western Australia, where I’ve almost finished a degree in Computer Science and Mechatronics Engineering. When I came here I noticed that there wasn’t too much available in the way of tutorials for users who wanted to use DataMarket data in a project, but weren’t C# programmers. I’d written a total of one function in C# before coming here, so I’d definitely classify myself in that category. The languages I’m most familiar with are PHP, Python and Java, so over the next few weeks I’ll do a series of posts giving a basic introduction to consuming data from DataMarket using these languages. I’ll refer to the 2006 – 2008 Crime in the United States (Data.gov) dataset for these posts, which is free to subscribe to, and allows unlimited transactions.
In this post I’ll outline two methods for using PHP to query DataMarket; using the PHP OData SDK, and using cURL to read and then parse the xml data feed. For either method, you’ll firstly need to subscribe to a dataset, and make a note your DataMarket account key. Your account key can be found by clicking “My Data” or “My Account” near the top of the DataMarket webpage, then choosing “Account Keys” in the sidebar.
DataMarket uses the OData protocol to query data, a relatively new format released under the Microsoft Open Specification Promise. One of the ways to query DataMarket with PHP is to use the PHP OData SDK, developed by Persistent Systems Ltd. This is freely available from CodePlex, however unfortunately there seems to be little developer activity on the project since its release in March 2010, and users report that they need to do some source code modifications to get it to work on Unix systems. Setting up the SDK also involves making some basic changes to the PHP configuration file, potentially a problem on some hosted web servers.
A word of warning: not all DataMarket datasets can be queried with the PHP OData SDK! DataMarket datasets can have one of two query types, fixed or flexible. To check which type a particular set is, click on the “Details” tab in the dataset description page. The SDK only supports datasets with flexible queries. Another way to check is to take a look at the feed’s metadata. Copy the service URL, also found under the “Details” tab into your browser’s address bar and add $metadata after the trailing slash. Some browsers have trouble rendering the metadata; if you get an error, save the page and open it up in notepad. Look for the tab containing <schema xmlns=”…”> (There will probably be other attributes, such as namespace, in this tab). The PHP OData SDK will only work with metadata documents specifying their schema xmlns ending in one of “/2007/05/edm”, “/2006/04/edm” or “/2008/09/edm”.
The PHP OData SDK comes with a PHP utility to generate a proxy class for a given OData feed. The file it generates is essentially a PHP model of the feed. The command to generate the file is
php PHPDataSvcUtil.php /uri=[Dataset’s service URL]
/out=[Name out output file]
Once generated, check that the output file was created successfully. The file should contain at least one full class definition. Below is a snippet of the class generated for the Data.gov Crime dataset. The full class is around 340 lines long.
/** * Function returns DataServiceQuery reference for * the entityset CityCrime * @return DataServiceQuery */ public function CityCrime() { $this->_CityCrime->ClearAllOptions(); return $this->_CityCrime; }
/**
* Function returns DataServiceQuery reference for
* the entityset CityCrime
* @return DataServiceQuery
*/
public function CityCrime()
{
$this->_CityCrime->ClearAllOptions();
return $this->_CityCrime;
}
With the hardest part complete, you are now ready to start consuming data! Insert a reference to the proxy class at the top of your PHP document.
require_once "datagovCrimesContainer.php";
Now you are ready to load the proxy. You’ll also need to pass in your account key for authentication.
$key = [Your Account Key];
$context = new datagovCrimesContainer();
$context->Credential = new WindowsCredential("key", $key);
The next step is to construct and run the query. There are a number of query functions available; these are documented with examples in the user guide. Keep in mind that queries can’t always be filtered by any of the parameters– for this particular dataset we can specify ROWID, State, City and Year. The valid input parameters can be found under the dataset’s “Details” tab. Note that some datasets have mandatory input parameters.
try { $query = $context->CityCrime() ->Filter("State eq 'Washington' and Year eq 2007"); $result = $query->Execute(); } catch (DataServiceRequestException $e) { echo "Error: " . $e->Response->getError(); } $crimes = $result->Result;
try
$query = $context->CityCrime()
->Filter("State eq 'Washington' and Year eq 2007");
$result = $query->Execute();
catch (DataServiceRequestException $e)
echo "Error: " . $e->Response->getError();
$crimes = $result->Result;
(If you get a warning message from cURL that isn’t relevant to the current environment, try adding @ in front of $query to suppress warnings.)
In this example we’ll construct a table to display some of the result data.
echo “<table>”; foreach ($crimes as $row) { echo "<tr><td>" . htmlspecialchars($row->City) . "</td>"; echo "<td>" . htmlspecialchars($row->Population) . "</td>"; echo "<td>" . htmlspecialchars($row->Arson) . "</td></tr>"; } echo "</table>";
echo “<table>”;
foreach ($crimes as $row)
echo "<tr><td>" . htmlspecialchars($row->City) . "</td>";
echo "<td>" . htmlspecialchars($row->Population) . "</td>";
echo "<td>" . htmlspecialchars($row->Arson) . "</td></tr>";
echo "</table>";
DataMarket will return up to 100 results for each query, so if you expect more than 100 results you’ll need to execute several queries. We simply need to wrap the execute command in some logic to determine whether all results have been returned yet.
$nextCityToken = null; while(($nextCityToken = $result->GetContinuation()) != null) { $result = @$context->Execute($nextCityToken); $crimes = array_merge($crimes, $result->Result); }
$nextCityToken = null;
while(($nextCityToken = $result->GetContinuation()) != null)
$result = @$context->Execute($nextCityToken);
$crimes = array_merge($crimes, $result->Result);
The documentation provided with the SDK outlines a few other available query options, such as sorting. Some users have reported bugs arising if certain options are used together, so be sure to test that your results are what you expect.
If the PHP OData SDK isn’t suitable for your purpose, another option is to assemble the URL to the data you are after, then send a request for it using cURL and parse the XML result. DataMarket’s built in query explorer can help you out here – add any required parameters to the fields on the left, then click on the blue arrow to show the URL that corresponds to the query. Remember that any ampersands or other special characters will need to be escaped.
We use cURL to request the XML feed that corresponds to the query URL from DataMarket. Although there are a number of options that can be set, the following are all that is required for requests to DataMarket.
$ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $queryUrl); curl_setopt($ch, CURLOPT_USERPWD, ":" . $key); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); $response = curl_exec($ch); curl_close($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $queryUrl);
curl_setopt($ch, CURLOPT_USERPWD, ":" . $key);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
$response = curl_exec($ch);
curl_close($ch);
The $response variable now contains the XML result for the query.
Before using the data, you’ll need to parse the XML. Because each XML feed is different, each dataset needs a parser tailored especially to it. There are a number of methods of putting together a parser, the example below uses xml_parser.
The first step is to create a new class to model each row in the result data.
class CityCrime { var $City; var $Population; var $Arson; public function __construct() { } }
class CityCrime
var $City;
var $Population;
var $Arson;
public function __construct()
I’m also going to wrap the all the parser functions in a class of their own. This function will be called with the query uri and account key. Firstly I’ll give it some class variables to store the data that has been parsed.
class CrimeParser { var $entries = array(); var $count = 0; var $currentTag = ""; var $key = ""; var $uri = ""; public function __construct($key, $uri) { $this->key = $key; $this->uri = $uri; } }
class CrimeParser
var $entries = array();
var $count = 0;
var $currentTag = "";
var $key = "";
var $uri = "";
public function __construct($key, $uri)
$this->key = $key;
$this->uri = $uri;
The parser requires OpenTag and CloseTag functions to specify what should happen when it reaches an open tag or close tag in the XML. In this case, we append or remove the tag name from the $currentTag string.
private function OpenTag($xmlParser, $data) { $this->currentTag .= "/$data"; } private function CloseTag($xmlParser, $data) { $tagKey = strrpos($this->currentTag, '/'); $this->currentTag = substr($this->currentTag, 0, $tagKey); }
private function OpenTag($xmlParser, $data)
$this->currentTag .= "/$data";
private function CloseTag($xmlParser, $data)
$tagKey = strrpos($this->currentTag, '/');
$this->currentTag = substr($this->currentTag, 0, $tagKey);
Now we are ready to write a handler function. Firstly declare the tags of all the keys that you wish to store. One method of finding the tags is to run the code using a basic handler function that simply prints out all tags as they are encountered.
private function DataHandler($xmlParser, $data) { switch($this->currentTag){ default: print "$this->currentTag <br/>"; break; } }
private function DataHandler($xmlParser, $data)
switch($this->currentTag){
default:
print "$this->currentTag <br/>";
break;
The switch statement in the handler needs a case for each key. We also need to let it know when it reaches a new object – from running the code with the previous handler, I knew that the properties for each row started and finished with the tag /FEED/ENTRY/CONTENT, so I’ll add a class variable to keep track of when the handler comes across that tag – every second time it comes across it I know that the result row has been fully processed.
var $contentOpen = false; const rowKey = '/FEED/ENTRY/CONTENT'; const cityKey = '/FEED/ENTRY/CONTENT/M:PROPERTIES/D:CITY'; const populationKey = '/FEED/ENTRY/CONTENT/M:PROPERTIES/D:POPULATION'; const arsonKey = '/FEED/ENTRY/CONTENT/M:PROPERTIES/D:ARSON'; private function DataHandler($xmlParser, $data) { switch(strtoupper($this->currentTag)){ case strtoupper(self::rowKey): if ($this->contentOpen) { $this->count++; $this->contentOpen = false; } else { $this->entries[$this->count] = new CityCrime(); $this->contentOpen = true; } break; case strtoupper(self::cityKey): $this->entries[$this->count]->City = $data; break; case strtoupper(self::populationKey): $this->entries[$this->count]->Population = $data; break; case strtoupper(self::arsonKey): $this->entries[$this->count]->Arson = $data; break; default: break; } }
var $contentOpen = false;
const rowKey = '/FEED/ENTRY/CONTENT';
const cityKey = '/FEED/ENTRY/CONTENT/M:PROPERTIES/D:CITY';
const populationKey = '/FEED/ENTRY/CONTENT/M:PROPERTIES/D:POPULATION';
const arsonKey = '/FEED/ENTRY/CONTENT/M:PROPERTIES/D:ARSON';
switch(strtoupper($this->currentTag)){
case strtoupper(self::rowKey):
if ($this->contentOpen)
$this->count++;
$this->contentOpen = false;
else
$this->entries[$this->count] = new CityCrime();
$this->contentOpen = true;
case strtoupper(self::cityKey):
$this->entries[$this->count]->City = $data;
case strtoupper(self::populationKey):
$this->entries[$this->count]->Population = $data;
case strtoupper(self::arsonKey):
$this->entries[$this->count]->Arson = $data;
Now we create the parser, and parse the result from the cURL query.
$xmlParser = xml_parser_create(); xml_set_element_handler($xmlParser, "self::OpenTag","self::CloseTag"); xml_set_character_data_handler($xmlParser, "self::DataHandler"); if(!(xml_parse($xmlParser, $xml))) { die("Error on line " . xml_get_current_line_number($xmlParser)); } xml_parser_free($xmlParser);
$xmlParser = xml_parser_create();
xml_set_element_handler($xmlParser, "self::OpenTag","self::CloseTag");
xml_set_character_data_handler($xmlParser, "self::DataHandler");
if(!(xml_parse($xmlParser, $xml)))
die("Error on line " . xml_get_current_line_number($xmlParser));
xml_parser_free($xmlParser);
After the call to xml_parse, the $entries will be populated. A table of the data can now be printed using the same foreach code as the SDK example, or manipulated in any way you see fit.
The two methods of consuming data from DataMarket with PHP both have their strengths and weaknesses. A proxy class generated from the OData SDK is very easy to add to existing code, but setting up the library can be tedious, and there is not much support available for it. Using cURL and parsing the xml provides slightly more flexibility, but requires much more coding to set up.
Since it only requires an URL and an Account key, opening the connection to DataMarket is very straightforward, whichever method is chosen. If the dataset you’re connecting to is free, I suggest opening Service Explorer and trying out various queries to get a feel for the data. Both methods shown above will result in the dataset’s conversion to an associative array, from which data can be manipulated using any of the PHP functions available.
At this stage, if you want to access a flexible query dataset, and are able to modify your PHP configuration file, the PHP OData SDK is a good tool for accessing OData feeds. However, if you want access to a fixed query dataset, or are unable to modify the configuration file, using cURL and parsing the result is straightforward enough to still be a valid option.
- Jovana Taylor
Announcing the release of the DataMarket Service Update 2! With this release, we’re rolling out a little something for everyone. First up: check out our shiny new Service Explorer:
The offering query UI has evolved into more of an all-purpose toolbox. Now you can create data visualizations right there in your browser. And if you want to do more with the data – but don’t feel like whipping up some code – now you can download your query results in XLS, CSV and XML formats (you already know about our DataMarket Add-In for Excel, don’t you? The CTP2 of the Add-In, shipped today, will no longer require Account key and you can simply use your Windows Live ID credentials).
Have you found yourself exploring the marketplace and found an offering you wanted to share – maybe some historic weather observations from WeatherBug (is it really normal to have this much rain in Seattle?), or nutrition information from Gregg London Consulting (hmm, better not have the second helping of dessert after all)? Now you can tell all of your friends about it with our new Facebook Like Button.
For all you developers out there, we’ve made it easier for you to bring valuable data into your apps by enabling Free Trials! This functionality allows you to subscribe to offerings for free, if the content publisher has chosen to publish a trial, so you can wrap your brain around the data and the best way to pop it into your app. Dun & Bradstreet, Boundary Solutions, MetricMash and StrikeIron helped us launch this new feature by creating a bunch of free trials – be sure to check out their free stuff.
Did you want to your application to access the dataset on behalf of the user? Now you can with OAuth v2 integration in SU2! Surely enough, the user has to actually approve the usage as part of the consent flow (enjoy the snapshot below).
Hello everyone outside the United States! We haven’t forgotten about you. With this release we’re introducing the first phase of our international expansion. If you’re in Australia, Austria, Canada, France, Germany, Italy, Spain and the United Kingdom, now you can subscribe to commercial DataMarket offerings – paying for them in your local currency! Thank you MetricMash and StrikeIron for helping us launch this important expansion by making your offerings available to people outside the US.PS – If you’re not in one of the countries listed above, fear not, we still haven’t forgotten about you…
And last, but certainly not least, we’re also releasing offerings from three new content publishers: Digital Trowel, InfoChimps and MetricMash. Welcome to the DataMarket, guys! And, best of all… all three have published some sort of free service so you can get started with their stuff at no cost!
The DataMarket Team
On May 24 at 8 am PST, join our upcoming MSDN webcast “Leveraging DataMarket to create Cloud-Powered Digital Dashboards. Milos Glisic, Director of Development, ComponentArt Inc., and Christian Liensberger, Program Manager, Microsoft Corporation will showcase ComponentArt’s Data Visualization technology to create interactive, web-based digital dashboards in Microsoft Silverlight 4 and mobile dashboards on Windows Phone 7 using Windows Azure DataMarket.
Click here to register
The DataMarket team will be at 3 of the upcoming Startup Weekends, April 15-17 in New York City, May 6-8 in San Francisco and June 10-12 in Seattle.
If you haven’t heard about it, Startup Weekend is an intense 54 hour event focusing on building a web or mobile application which could form the basis of a credible business over the course of a weekend. The weekend brings together people with different skill sets - primarily developers, UX/UI designers and business people - to build applications and develop a commercial case around them.
We are very excited to be a part of these great events, come join us and learn more about DataMarket. You can find details on these and all of the upcoming events at http://startupweekend.org/.
Today’s content announcement is a little different that our previous monthly Content Updates. As we continue to work on making the best data offers available to you through DataMarket, we are in the process of shortening the time between our content releases. Beginning today, you will see us release smaller updates on a much more frequent basis, allowing you to get your hands on the data that much sooner!
Today’s update brings with it a couple of exciting new public and commercial offerings.
World Life Networks
Microsoft
As always, we will continue to post our blog whenever new data becomes available on DataMarket… so be sure to check the site regularly to learn the latest.
- The DataMarket Team
Today, the Windows Azure DataMarket team’s own Moe Khosravy participated in a fireside chat at the sold out GigaOm Structure: Big Data 2011 event. Part of GigaOm’s Structure series, which brings together industry leaders across the cloud computing and Internet infrastructure industry, the first annual Big Data event will feature 40 representatives from companies across the cloud, data analytics, marketplace and storage industry to discuss the recent explosion in virtual data. Specifically, the conference will explore the uptick in interest around the “big data” trend as it relates to three areas, including capacity, insights and monetization.
You can learn more about Moe’s fireside chat titled, “Cash For Your Data? The Future For Datamarkets” where he discussed the future of data marketplaces alongside Flip Kromer, CTO of InfoChimps, a data marketplace provider, at http://gigaom.com/2011/03/23/moe-khosravy-microsoft-flip-kromer-infochimps/
Interested in learning more? Checkout our upcoming webcast on [UPDATED] Wed Apr 6, 2011 at 8:00 AM PST, Leverage ParcelAtlas data on DataMarket to create Innovative Solutions.
[Updated 3/23/2011] Register at: https://www.eventbuilder.com/microsoft/event_desc.asp?p_event=k11a33z2&requireprofile=false
During this webcast you will learn:
Meet the Presenters:
Dennis H. Klein, GISP, past PE, past AICP, Boundary Solutions
Dennis started his GIS career in 1971 as a practicing planner who was esri's first private sector customer, the Bretton Wood, Mt Washington Development Program. After fifteen years in the planning trenches, Dennis went full time GIS, opening up Facility Mapping Systems in 1986, offering AutoCAD based GIS software for municipal and utilities. Shipped 3,000 systems, including hundreds to counties that started full function digital parcel maps early on. After nine years, sold to EaglePoint. Started next project. Building on experience of providing software to counties, now assembling, normalizing and deploying a national parcel layer online as ParcelAtlas, now available in MarketPlace.
Sudhir Hasbe, Senior Product Manager ; SQL Azure and Middleware, Microsoft
Sudhir Hasbe is a Sr. Product Manager at Microsoft focusing on SQL Azure and Middleware. Mr. Hasbe is responsible for DataMarket, Microsoft's information marketplace. Mr. Hasbe is responsible for building ISV partner ecosystem around Windows Azure Platform. Mr. Hasbe has passion for incubation businesses and has worked at few startup businesses prior to Microsoft. Sudhir Hasbe has rich background in ERP, EAI and Supply chain management space. Prior to Microsoft Mr. Hasbe has worked with more than 55 companies in 10 countries over 10 years implementing and integrating ERP systems with various supply chain and data collection solutions.
As part of the Real World DataMarket series, we talked to Ayush Yash Shrestha, the Professional Services Manager at Dundas Data Visualization, about how his company is taking advantage of DataMarket (a part of Windows Azure Marketplace) to enhance its flagship Dundas Dashboard business intelligence software.
MSDN: Can you tell us about Dundas Data Visualization and the services you provide? Shrestha: Dundas produces data visualization and dashboard solutions that are used by some of the largest companies in the world. Dundas Dashboard is a one-stop shop for bringing in data from various systems, building business models, and finding key performance indicators (KPIs). Through a simple drag-and-drop interface, a fairly sophisticated business user—even without development experience—can turn raw data into a useful dashboard in about four steps. It’s self-service business intelligence (BI).
MSDN: What business challenges did Dundas and its customers face before it recently began using DataMarket? Shrestha: We wanted to help our customers contextualize their internal BI data by comparing it with data from outside the enterprise. A KPI without proper context is meaningless. You can use your BI system to gather internal data and determine if you’re meeting your goals, but if you don’t have the ability to combine the data with outside information, then your goals may not be contextually sound. Traditionally, buying this information from multiple providers and using multiple data feeds with different formats has been cumbersome and expensive, with a lot of IT support requirements.
MSDN: How does DataMarket help you address the need to cost-effectively contextualize BI data? Shrestha: We’re able to bring in reliable, trusted public domain and premium commercial data into Dashboard without a lot of development time and costs. This is possible because DataMarket takes a standardized approach, with a REST-based model for consuming services and data views that are offered in tabular form or as OData feeds. Without having a standard in place, we’d be forced to write all kinds of web services and retrieval mechanisms to get data from the right places and put it into the proper format for processing.
MSDN: What are the biggest benefits to Dundas customers by incorporating DataMarket into Dashboard? Shrestha: Our customers just create a single DataMarket account, and they’re ready to get the specific data that they need. This will result in significant savings, especially for companies that would otherwise make several large business-to-business data purchases. Also, the data provided through DataMarket is vetted. Without a service such as DataMarket in place, there is a risk of businesses adding inaccurate data to their dashboards. With DataMarket being such a reliable source, we’re able to provide meaningful context that few other dashboard providers offer at this time. Finally, we can build new dashboards for a customer once, and they can be easily improved as the DataMarket data catalog continues to gain depth. We can integrate additional KPI context into existing dashboards with a few mouse clicks, versus the old approach of reengineering the system.
MSDN: And what are the biggest benefits to Dundas? Shrestha: The most important are added value, competitive edge, and new business opportunities. If a customer is comparing Dundas Dashboard with a competing product that does not use DataMarket, we immediately have an edge thanks to the vast additional resources that DataMarket provides. Also, customers often expect us to offer analysis services in addition to pure data visualization, which in the past we did not have the resources to provide. With DataMarket, we’re now in a position to offer this additional level of service, which will help us expand our customer base and increase revenue.
Read the full case study at:
http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseStudyID=4000009285
To read more about DataMarket, visit: https://datamarket.azure.com
As part of the Real-World DataMarket series, we talked to Dan Malks, Vice President for App Platform Engineering at JackBe, about how his company is taking advantage of DataMarket (a part of Windows Azure Marketplace) to power Presto, the company’s flagship Real-Time Intelligence (RTI) solution.
MSDN: Tell us about JackBe. What services do you offer and what is your mission? Malks: JackBe focuses on delivering true real-time intelligence through our Presto solution. Every business needs to be able to bring data together quickly from a variety of internal and external sources to make real-time decisions. Our mission is to help businesses make smarter decisions faster, and Presto is the fastest way for organizations to gain RTI, which we facilitate by delivering portable web Apps that can be used when and where our customers need them.
MSDN: What differentiates Presto from other business intelligence or dashboard solutions? Malks: The Presto concept is similar to business intelligence and dashboarding, but Presto is more dynamic and user-friendly. Organizations can use Presto Apps for a variety of RTI goals and tasks, such as monitoring key performance indicators, operations management, situational awareness, departmental and organizational transparency and accountability, geospatial awareness, as well as implementing 360-degree operational dashboards. And these are just examples; the platform is highly flexible and customizable.
MSDN: What prompted JackBe to start looking at cloud technology? Malks: The business value proposition of the cloud is now well established, letting organizations spin up scalable solutions quickly, without requiring the traditional internal investment in IT expertise, server provisioning, and so on. But this also makes the cloud a valuable platform for information. As our customers and their applications move to the cloud, they want us to help them connect their cloud data to their on-premise data, and to data from other cloud sources such as DataMarket.
MSDN: Did you consider any cloud platforms besides the Windows Azure platform? Malks: Yes—two major competitors. Google App Engine was too limited from a programming language perspective. Neither Google nor Amazon had anything like DataMarket. The fact that Windows Azure is evolving so quickly, and the amount and types of information in DataMarket are constantly expanding, make the platform ideal for us and our customers.
MSDN: How does DataMarket help you address the RTI needs of your customers? Malks: The information in DataMarket is especially valuable because of its richness, diversity, and accuracy. Today, many decision makers spend half their time just looking for the right data, even before than can act on it. DataMarket speeds the decision-making process by providing a central place where they can find whatever data they need for different purposes, saving time and increasing efficiency.
MSDN: As a Microsoft partner, what’s the primary business value that Presto adds to DataMarket? Malks: Presto brings data together from many internal and external sources and combines it into a single feed that is easily customizable to fit a specific business purpose, which makes a lot of business sense. By bringing Presto and DataMarket together, some of the traditional technical challenges of managing multiple data sources—IT work to handle multiple feeds, formats, and so on—aren’t challenges anymore. As a result, for customers, development becomes easier, they get their real-time intelligence faster, and their business benefits are achieved much quicker.
Read the full story at:
http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseStudyID=4000009252
To learn more, visit: https://datamarket.azure.com
Announcing the release of DataMarket Service Update 1 & the February Content Update. Service Update 1 introduces a number of new improvements for our Content Providers including reporting and offer wind down features, while the February Content Updates brings with it a number of exciting new public and commercial offerings.
Take a look at some of the great new data we have:
Boundary Solutions, Inc.
Government of the United Kingdom
Gregg London Consulting
StockViz
United Nations
As we post more datasets to the site we will be sure to update you here on our blog… and be sure to check the site regularly to learn more about the latest data offerings.
Happy New Year! Today we are announcing the release of a number a exciting new offerings on DataMarket – both public and commercial! Take a look at some of the great new data we have:
As part of the Real World DataMarket series, we talked to Ellie Fields, Director of Product Marketing at Tableau Software, about using DataMarket, part of the Windows Azure Marketplace, to add premium content as a data source option in its data visualization software. Here’s what she had to say:
MSDN: Can you tell us more about Tableau Software and the products you offer?
Fields: Tableau Software offers “rapid-fire business intelligence.” Tableau has proprietary technology developed at Stanford University that enables users to drag and drop data from data sources to quickly transform text into rich data visualizations.
MSDN: What were the biggest challenges that Tableau Software faced prior to adopting DataMarket?
Fields: Data is proliferating everywhere, but it can be difficult and time-consuming for customers to find, purchase, and format data to augment their existing data sources. It’s like the Wild West of data out there, with data available everywhere and in any imaginable format. We’re always looking to improve our service offerings, but it’s a fine line—we want to deliver valuable services, but we don’t want to take our focus away from what we do best, which is data visualization.
MSDN: Can you describe how Tableau Software is using DataMarket to help tame the Wild West of data?
Fields: After the 5.0 release of Tableau Software, one of our developers coded a basic integration into DataMarket during one of our “hackathons.” We were so impressed with the opportunity to access premium content via DataMarket that we decided to include DataMarket as a data source option in our products. Now, when customers use Tableau, they see DataMarket as a data source option. They simply provide their DataMarket account key for authentication and then find the data sets they want to use. Customers can import the data into Tableau and combine that information with their own corporate data for deep business intelligence.
Figure 1: When using Tableau, customers see DataMarket as a data source option.
MSDN: What makes your solution unique and how does DataMarket play a role in that unique quality?
Fields: Unlike other data visualization software, Tableau Software gives customers the ability to simply drag and drop data to create business intelligence. Using DataMarket supports that same idea by offering premium content that is already structured and formatted, and easily available. Customers don’t have to spend valuable time seeking out and formatting the data before integrating it with their own data for rich visualizations. For example, they can add population data to their sales data to assess regions for growth.
MSDN: What kinds of benefits is Tableau Software realizing with DataMarket?
Fields: We have been able to add a valuable service for our customers—the data sets that customers can access in DataMarket are of tremendous value for creating very rich business intelligence. What’s really great is that we were able to very quickly and easily add access to DataMarket in the Tableau line of products, thanks to the DataMarket API, and did so while maintaining laser-sharp focus on our core business.
Read the full story at: www.microsoft.com/casestudies/casestudy.aspx?casestudyid=4000009036
As part of the Real World DataMarket series, we talked to Joseph Albahari, developer of LINQPad, about his product’s support for DataMarket, a part of Windows Azure Marketplace. Here’s what he had to say:
MSDN: How did you come to develop LINQPad?
Albahari: When Microsoft first announced LINQ, I came to realize that it was one of the best technologies that Microsoft had produced. LINQ is a significantly more productive querying language than traditional SQL. It’s simpler, tidier, higher-level, type-safe, and its queries are composable. Not having to worry about lower-level details when I write my queries is a big win. I was so impressed with LINQ that I built a tool—LINQPad—to take advantage of it. LINQPad brings the same interactive experience to LINQ that SQL Server Management Studio brings to SQL: you simply point LINQPad to a database, write a query, and hit Run. After writing LINQPad, I realized that I’d in fact written something broader than a LINQ tool—I had written a code scratchpad that executed not only LINQ queries, but any C# or Microsoft Visual Basic code snippet! LINQPad complements the Microsoft Visual Studio development system nicely: you write, test, and tune queries and code snippets interactively in LINQPad, and then paste working code right into a Visual Studio project.
MSDN: What’s been the reaction of the developer community?
Albahari: Developers have responded enthusiastically. Over the past three years, LINQPad has been downloaded around 250,000 times and has probably tens of thousands of active users.
MSDN: Tell me what brought you to support cloud-based data in LINQPad?
Albahari: Data used to exist mostly in databases on the same network. But now data has found a new home: in the cloud. LINQPad has supported Microsoft SQL Azure for some time, but there was a demand for OData [Open Data Protocol]-based services. OData is a protocol for querying and updating data across the Internet. The great thing about LINQ is that the same querying language works with a multitude of providers. So, if you know LINQ, you can write OData queries—the provider takes the same LINQ query and translates it into an OData URI [Uniform Resource Identifier] instead of SQL.
MSDN: Why did you decide to support Windows Azure Marketplace DataMarket in particular?
Albahari: Having a centralized portal for OData services makes a lot of sense. In particular, it offers the potential for unified billing for premium data sources. I anticipate a growing user base, so I want to offer these people an interactive querying tool.
MSDN: What’s the basis of DataMarket support in LINQPad?
Albahari: One of the keys to supporting DataMarket in LINQPad is OData. I’d already built OData support into LINQPad by leveraging OData support within the Microsoft .NET Framework. This made it easy to write a DataMarket driver that gives developers LINQPad support for DataMarket “out of the box.” To use that support, developers choose DataMarket as a data source for LINQPad. The LINQPad software also gives them the option to sign up for DataMarket, to subscribe to datasets, and to browse for their account keys to the Microsoft service. Developers can then compose LINQ queries and execute them against DataMarket, with the results appearing immediately in LINQPad. It’s that easy.
Read the full story at: www.microsoft.com/casestudies/casestudy.aspx?casestudyid=4000008920
DataMarket recently announced Section 508 compliance and the intent to secure certification for DataMarket to demonstrate compliance with the Federal Information Security Management Act (FISMA).
On Friday, we published the new DataMarket for Government page which highlights these as well as how DataMarket, enables governments to achieve their goals of transparency, participation and collaboration, streamlining the process of publishing data, and making it easier for constituents to access the data from the applications they currently use. The key attributes of DataMarket that make it particularly suitable for government agencies are that it provides a Scalable and Rich Data Ecosystem built on a scalable Windows Azure platform, provides an easy mechanism to federate data (fold in semantic web concepts, federation API concepts).
At Microsoft, we are committed to developing products that are accessible to everyone. We take a strategic approach to accessibility by focusing on integrating accessibility into product planning, research and development, product development, and testing. As such, we are very excited to announce that DataMarket is Section 508 compliant and that the Windows Azure Marketplace DataMarket v1.0 Voluntary Product Accessibility Template (VPAT) is now available on the Microsoft Section 508 VPATs site.
At this time, we are also announcing our intent to secure certification for DataMarket to demonstrate compliance with the Federal Information Security Management Act (FISMA). FISMA establishes responsibility and accountability for the security of all federal agency information systems and defines security requirements that must be met by all US Federal government information systems.
As part of the Real World DataMarket series, we talked to Greg Kirkorsky, Senior Vice President for the Americas at STATS, about using DataMarket (a part of Windows Azure Marketplace) to support his company’s entry into the global consumer market. Here’s what he had to say:
MSDN: Let’s start by talking about STATS.
Kirkorsky: Sure. STATS provides real-time sports data, historical information, and turnkey fantasy operations to sports organizations and media companies of all types. If you’re holding a baseball card, looking up sports data on a website, or watching the electronic crawl across the bottom of your TV screen during a sports report, you’re probably looking at information that originated with STATS.
MSDN: What motivated you to look at DataMarket?
Kirkorsky: STATS is the leading sports data provider in its business-to-business markets, but we lacked a presence in the global consumer market. We saw this as an area of vast potential growth. But we also saw it as a high-risk market with massive costs of entry. We have global data centers, but we anticipated needing to build them out considerably to support anticipated growth. We’d then have a major marketing expense to bring consumers to our data. It would also be costly to build and maintain a pricing and invoicing infrastructure. We wanted to enter the consumer market—but we wanted to mitigate the risks while reducing the costs.
MSDN: Tell us about working with Microsoft to maximize your participation in DataMarket.
Kirkorsky: We worked with Microsoft to define the sports data sets that we intended to expose and the data structures for supporting that information. We chose data sets on game-by-game and year-to-date statistics, as well as live scores, for baseball and football. Microsoft provided guidance on using the pricing and invoicing infrastructure that is built into DataMarket.
MSDN: How are you hosting your DataMarket data?
Kirkorsky: Although we have our own data centers, we chose to host the databases for DataMarket in Microsoft SQL Azure. It gave us complete flexibility to make the best technical decisions regarding our data. It also provided the greatest ease of implementation for DataMarket. We know that with SQL Azure, we’re getting guaranteed scalability, reliability, and performance—factors that are essential for a smooth entry into a new market.
MSDN: How do you expect consumers to use your data on DataMarket?
Kirkorsky: Anyone with a spreadsheet can link up with our data—but that’s just the beginning. Consumers can use STATS data with any Microsoft graphing, charting, or visualization tools they already have. They can use any new data tools that Microsoft makes available for DataMarket customers. We’re building our own visualization products and we could make those available to customers through DataMarket. There is also the potential for mash-ups with other content providers on DataMarket, and for partnerships with independent software developers who want to build STATS data into their applications.
MSDN: Sum up the benefits from your participation in DataMarket—both actual and projected.
Kirkorsky: We’ve extended our reach to vast numbers of new customers around the world and gained a way to enter a new market while minimizing our investment. We estimate we’ve reduced our investment cost by U.S.$1 million over what we would have spent to build our own consumer infrastructure. Also, we’ve avoided the need to create and manage our own billing infrastructure and fulfillment service.
Read the full story at: www.microsoft.com/casestudies/casestudy.aspx?casestudyid=4000008815
To read more about DataMarket, visit:
https://datamarket.azure.com
As part of the Real World DataMarket series, we talked to Dean Stoecker, President at Alteryx, about using DataMarket (part of the Windows Azure Marketplace), to offer customers faster time-to-market, more flexibility, and less work when using Alteryx business intelligence (BI) solutions. Here’s what he had to say:
MSDN: Tell us about Alteryx and the services you offer.
Stoecker: Alteryx makes geographic business intelligence software to help companies in many industries make sense of vast amounts of data. Retailers, for example, use our software to combine and analyze demographic, traffic, income, and other data to determine where to open new stores. Our software is easier to use than many traditional BI software programs, so more people in an organization can use it.
MSDN: What prompted you to look into DataMarket?
Stoecker: We’ve done a good job of delivering our software to business users over the web as software-as-a-service (SaaS). But we wanted to help developers incorporate our technology into their solutions and get to market faster. Also, we had to anticipate the data sets that these developers might need. Customers wanted more control over how they consumed our application.
MSDN: How has DataMarket helped?
Stoecker: We can list individual APIs [application programming interfaces] on DataMarket for purchase—an API for store location functionality, for example. Instead of developing all the code needed to create geolocation functionality in an application, a developer simply picks up our API from DataMarket and snaps it into his or her application.
MSDN: So you’re packaging your software in smaller, more easily consumable chunks?
Stoecker: Essentially, yes. We’re offering our technology as small, specific services that developers can use as needed to save themselves some development time. They also do not have to license and manage third-party data sets. They can pull data from DataMarket and let Microsoft take care of licensing and management.
MSDN: What are you offering on DataMarket?
Stoecker: We have five APIs on DataMarket so far, all providing geolocation-related services. One, for example, helps retail businesses determine new store locations. From DataMarket, customers can pull U.S. Census data, consumer expenditure data, geospatial data, traffic data, and other data sets. They can then integrate these data sets and the Alteryx APIs into their own applications, often cutting development time from weeks to hours.
MSDN: Would you have been able to sell your APIs without a framework like DataMarket?
Stoecker: That’s another good point. We had long wanted to offer our technology as pure web services, but building an e-commerce-enabled data community is an expensive proposition—in the hundreds of thousands of dollars. By using DataMarket, we let Microsoft assume that expense. We get the benefit of posting our offerings on an online marketplace without the burden of maintaining the e-commerce infrastructure.
MSDN: So, how would you summarize the benefits of integrating DataMarket into your business model?
Stoecker: By incorporating DataMarket into our BI solutions and business model, we can give our customers more flexibility and faster time-to-market, reduce our development work and costs, and open up more opportunity to grow our business. Customers can make their own choices, and you can’t put a dollar figure on the value of giving customers the ability to build what they want.
Read the full story at: www.microsoft.com/casestudies/casestudy.aspx?casestudyid=4000008818
As part of the Real World DataMarket series, we talked to Fabrice Marguerie, Owner of metaSapiens, about his plans to align his data querying tool, Sesame Data Browser with DataMarket (a part of Windows Azure Marketplace) to gain access to a global developer customer base. Here’s our conversation:
MSDN: Can you tell us about metaSapiens and your flagship product?
Marguerie: I’m a developer and I own metaSapiens, the company that offers Sesame Data Browser. This is a tool that makes it easier for people to browse and query databases. Using Sesame, developers can easily preview data without having to write SQL queries by hand.
MSDN: What were the biggest challenges that metaSapiens faced prior to discovering DataMarket?
Marguerie: Well, I had a product that I felt would be of interest to developers the world over, but I’m the sole owner of metaSapiens and so I needed an inexpensive, efficient way to expose Sesame to a large potential customer base, developers like myself. I looked at several online data markets as a potential forum to promote Sesame, but none met my expectations. They often forced developers to use different formats, or they didn’t have robust data sets. If I was going to align Sesame Data Browser to any online information marketplace, it needed to support OData and meet my expectations for one-stop shopping for high-quality data.
MSDN: Can you describe how metaSapiens is using DataMarket to help build your business on a global scale?
Marguerie: As a developer, I could see the attraction of DataMarket, so I reasoned it would be attractive to my potential market. DataMarket offers users standard OData feeds and consistent, REST-based OData application programming interfaces, and it’s interoperable with Microsoft Visual Studio development tools. Because of these features, I believe that DataMarket will attract top-notch development talent, the kind of customers I want for Sesame Data Browser. Instant access to new customers around the world is an invaluable business opportunity.
MSDN: Do you think you’ll be able to reach more customers with DataMarket?
Marguerie: There a millions of information workers who use Microsoft Office and they can also take advantage of DataMarket by subscribing to data sets to integrate information into their documents, spreadsheets, and presentations. I plan to take advantage of this potential market by selling Sesame as data-browsing software for end users. The more business people who are browsing data sets in DataMarket, the better I can build that part of my business. There’s nothing like the opportunity to reach millions of Microsoft Office users!
MSDN: What other benefits is metaSapiens realizing with DataMarket?
Marguerie: Well, I’ll be able to market my product to a global audience with minimal investments on my part, which is invaluable for a sole owner of an IT company. And I’ll have the opportunity to diversify Sesame Data Browser to work with Microsoft SQL Azure. I have been working with Microsoft development tools and the Microsoft .NET Framework for long enough to know that DataMarket will provide a great environment in which to grow my business.
http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseStudyID=4000008716
Today’s update to DataMarket completes the roll out of paid data offerings that we announced at PDC last month, across all US-based DataMarket accounts. What does this mean? All US-based accounts are now able to purchase data offerings on DataMarket. All you need is a credit card!
This update also introduces a number of great new data offerings for you to explore, including:
Checkout out this great session December 1st to learn more about building innovative new applications with DataMarket!
Session Title: DataMarket: Enabling ISVs to build innovative applications across industry verticals
Date: December 1, 8 am-9am PST
Session Abstract: This webcast will provide an overview of DataMarket and the tools and resources available in DataMarket to help ISVs build new and innovative applications, reduce time to market and cost. DataMarket also enables ISVs to access data as a service.
ISVs across industry verticals can leverage datasets from partners like D&B, Lexis Nexis, Stats, Weather Central, US Governments, UN, EEA to build innovative applications. DataMarket includes data across different categories like demographic, environmental, weather, sports, location based services etc.
Presenter: Sudhir Hasbe, Sr Product Manager, DataMarket
Registration link:
Partner: https://training.partner.microsoft.com/learning/app/management/LMS_ActDetails.aspx?UserMode=0&ActivityId=719815