Cum Grano Salis

How to seamlessly embed data from Excel Services spreadsheets into Word

2009-11-09T17:00:00Z

One of the fun things about working on the programmability aspects of Excel Services is realizing just how many interesting scenarios exist that you never thought of originally. I got an IM from Dany Hoter asking for help on an issue he ran into with one of his customers. I have been working with Dany for about 10 years now and he always has interesting questions and ideas. This time around, the problem revolved around a complicated word document that contained links to Excel data – the word document would update and grab data from the workbook via automation (OLE embedding essentially). For a number of reasons, Dany wanted to remove the dependency on the Excel client and was wondering how to go about it. Eran from the Excel Services team suggested looking into fields and that’s how Dany landed in my IM window.

This is where the fun part begins. In Word, one can insert fields into the document that have certain functionality. The most used ones are probably the document Index, page numbers and Date/Time. One of those fields is an IncludeText field which can take text from various sources and put it inside the field. To add such a field, you use the “Quick Parts” drop-down in the “Insert” ribbon tab:

Once selected, you are presented with a slew of options – the one you want is the “IncludeText” field:

When the IncludeText field is selected, the user is presented with two more panes – the file name or URL from which to take the data (that’s in the middle) and the options of the field (that’s on the right).

In the URL field, we supply an Excel REST URL that points to our desired cell – this should be a REST URL that’s presented in the Atom format – that way we can traverse the data and get back what we want. In my post about getting range data from Excel Services, there’s an example for such a link. Here’s what the link looks like for our case here:

http://tsdlt2/_vti_bin/ExcelRest.aspx/docs/Bugs.xlsx/Model/Ranges('PersonalBugCount')?$format=atom

As you can see, we are going to a spreadsheet called “Bugs.xlsx” and asking for the range called “PersonalBugCount”. The parameter $format is set to atom to instruct Excel Services to return an XML representing the data. The result (removed some elements for brevity) looks like this:

<?xml version="1.0" encoding="utf-8"?>

<title type="text">AverageBugCount</title>

<x:range name="AverageBugCount">

<x:row>

<x:c>

<x:v>1.3</x:v>

<x:fv>1.3</x:fv>

</x:c>

</x:row>

</x:range>

</content>

</entry>

We now need to set two more fields in the dialog – the namespace mappings and the XPath expression – luckily, for the case where you want a single value, both of these are the same and require no modification.

Namespace Mapping is used in conjunction with the XPath expression – the XPath expression needs to know what aliases to use for what namespaces. In this case, we have two namespaces we need to use:

http://www.w3.org/2005/Atom – this is the namespace for the ATOM scheme.

http://schemas.microsoft.com/office/2008/07/excelservices/rest – this is the namespace for the Excel REST data scheme.

Both of these will need to have aliases associated with them (the aliases are not really that important). The setting that should be entered in the “Namespace mapping” field is then:

This goes in the “Namespace Mappings” field:

xmlns:a="http://www.w3.org/2005/Atom" xmlns:x="http://schemas.microsoft.com/office/2008/07/excelservices/rest"

The XPath needs to point to the “fv” element that’s in the XML and so it should be set to:

This goes in the “XPath Expression” field:

/a:entry/a:content/x:range/x:row/x:c/x:fv

And that’s pretty much it! Once you hit OK, the dialog will close and you should see the value from the workbook. Here’s text from a word document with fields toggled to see the codes:

And here’s the text as it appears when the fields are shown as text:

The highlighted areas are the values that came from Excel Services. The cool part is that when the workbook updates and the word document is updated, this data will update as well, showing the new values!

Now, another nifty aspect of this is that the same mechanism can be used to embed richer data – like HTML. If you will recall, at the bottom of this post I showed how one can access Ranges from Excel and get back HTML fragments. Guess what – Word knows how to embed those as well. Using the same mechanism, we can enter a field and point it to a REST URL returning an HTML fragment. In this case, we do not need the namespace mappings or the XPath expression – just the URL (in this example a URL to a range that looks like a score-card):

http://tsdlt2/_vti_bin/ExcelRest.aspx/Docs/Bugs.xlsx/Model/Ranges('PersonalScoreCard')

The URL resolves to the following HTML:


	Shaharp:	10
	Average:	1.3

When embedded in the workbook by using the IncludeText field, this is what it looks like:

The same thing applies as before – if the data of the workbook changes and the scorecard data and formatting changes, the next time the Word document will be updated, this text and formatting will also be updated.

Finally, if you look just above the IncludeText field, you will see an IncludePicture field – using that allows you to embed live images from Excel Services within the workbook – in this example, the URL we will embed is the chart that shows the progress made on the bugs and the goals:

To summarize – as you can see, you can create richer and self-updating documents in word by using fields that connect to Excel data and bring back plain text (the first example with the Atom) or formatted text (the second example with the HTML fragment) or charts (last example). These can be updated at will just like any other field in Excel and guarantee that your document will contain data that is up to date and is synchronized with the source – no more checking to see if the data was copied properly etc.

Bringing it all back home – using advanced REST functionality with the Excel Services Gadget

2009-11-06T17:00:00Z

In the past two weeks I detailed both the new REST functionality in Excel Services as well as the Excel Services Gadget (which will hopefully be available for download soon). In the previous post I showed how you can use URL parameters to REST URLs to modify the result you get back. In this post, I will show how you can create an incredibly compelling story for users by using both that and the gadget that I introduced earlier this week.

To start, I have a workbook that contains information about my team bug counts. On top of everything else inside that workbook, it also contains a cell where, when an alias (email) is entered, pars of the workbook recalculate to show the bug trend/counts for that specific person:

If you click on the image, you will see the cell called “Alias” (the one that currently has “shaharp” inside it). the green table you see contains information specific to the Alias cell (so, if you change the cell to another value, the data will update accordingly). On top of that we also have a table of links in the workbook:

For each person there are two links – one under the alias of the person and the other called “Home Site”. Here are the formulas for both:

Under the alias, the formula is: =HYPERLINK("http://tsdlt2/_vti_bin/ExcelRest.aspx/Docs/TeamBugs.xlsx/Model/Charts('PersonalBugs')?Ranges('Alias')=" & A1, A1)

Under the Home Site the formula is:=HYPERLINK("http://tsdlt2/PersonalPages/" & A1 & ".aspx", "Home Site")

As you can see, the first formula resolves to a hyperlink to a REST URL where the object shown is the same “PersonalBugs” chart but where for each person, a value is set into the “Alias” cell. In this way, a single workbook/chart can serve multiple purposes. Here, we pass in the alias to a cell and then get the chart that corresponds to that person.

Here’s how this works. First, you can see that the link show up in the gadget itself:

Now, remember, there are two links here. The Home Site one is just a run of the mill link and when clicked, will open up in your default browser:

As you can see, the link corresponds to an alias which then opens up in the browser according to the formula.

The interesting part is what we can do with the other link though. The gadget recognizes that the link is actually a REST link – and so it decides that it can actually display it as part of the gadget in a flyout:

Once the gadget loses it’s focus, or you click inside it, the flyout will collapse back into the gadget.

And that’s it for this week. This week I tried to show you some interesting ways of using the new Excel Services REST APIs and some of the new and exciting opportunities it gives developers and users. There was a post about the “essentials” of REST in Excel Services – a kind of “cheat sheet” for all the things REST can do. There were two posts about the Excel Services Gadget which showed the concept and then a post that explained the more advanced stuff REST can do by passing parameters into the workbook via the URL.

Next week will show the actual code running behind the gadget. Have a good weekend!

Advanced Excel Services REST API capabilities – passing parameters to a spreadsheet

2009-11-05T17:00:00Z

So far in the posts discussing Excel Services REST, I showed how easy it was to gain direct access to elements within workbooks. One immediately sees the potential of really releasing the data and the visualization that’s locked within Excel and making it easily available on the web. The really advanced stuff though is a little more subtle! With the Excel Services REST APIs, you can also pass in parameters into the spreadsheet, allowing you not only to leverage the data and the visualization, but also to leverage the logic that’s behind them. These things are always explained best with an example. Take a workbook I used in a talk at the SharePoint Conference 2009 – a workbook that calculates mortality chances per age group, adjusted to life-style. So, for example, if you smoke 5 packets a day, guess what – you have more chances of dying at 50 than if you didn’t.

Here’s part of the workbook:

The table contains a bunch of parameters (each a named cell) which when changed affect the curve and peak of the chart. The chart shows on the Y axis the chances of death and on the X axis the age for said chance. In this case, when all the parameters are empty (zero), you can see that the peak for the chances is around age 70 with a 30% chance to kick the barrel there. So how do we leverage this? Lets start simple – the URL for the chart is the following (as discussed in the first post I made about REST):

http://tsdlt2/_vti_bin/ExcelRest.aspx/Docs/MortalityChances.xlsx/Model/Charts('ChancesChart')

And when navigated to, simply produces the chart that’s in the workbook:

Now, lets tell Excel Services to put a value in one of the parameters and recalculate before it returns the actual result – this is done by using the “Ranges” URL parameter:

http://tsdlt2/_vti_bin/ExcelRest.aspx/Docs/MortalityChances.xlsx/Model/Charts('ChancesChart')?Ranges(‘Packets’)=10

When used in this manner, the following steps will take place:

1. Excel Services will find the cell named “Packets” and place the value 10 in it.
2. It will make sure a recalc takes place so that the workbook will represent the actual changes requested.
3. The chart image will be sent back to the user.

Here’s the result:

As you can see, the peak shifted and now the estimate peaks at 30% still, but with the age being about 60! You can of course add more parameters – lets say the person also has a history of heart conditions. We can simply pass another parameter:

http://tsdlt2/_vti_bin/ExcelRest.aspx/Docs/MortalityChances.xlsx/Model/Charts('ChancesChart')?Ranges(‘Packets’)=10&Ranges(‘FamHeart’)=1

And the effect on the chart:

You can easily see how this mechanism can be leveraged to get much more out of your workbook! Not only can you get all the data and all the visual capabilities of the workbook, but REST also lets the logic shine through and be leveraged by the user/developer. And this of course works with all the other REST requests that are supported by Excel Services – it is not limited to charts.

In the next post, I will show how the Gadget can be used in conjunction to other REST links (including REST links with parameters) to allow for even better reuse of Excel workbook data and logic.

Showing Excel ranges in the Excel Services Gadget

2009-11-04T10:27:00Z

The last post I wrote, I showed the Excel Services gadget and how it can be a great tool for bringing your Excel data right to the desktop. In this short post, I will show some of the other capabilities it has – namely, the ability to show ranges on top of charts.

In the settings page of the gadget, you have the ability to choose not only charts, but also ranges, tables and PivotTables. The workbook I have contains a simple color coded “score card” which contains the number of bugs I have and the average number – I can choose that score-card to show when the gadget is minimized – in some cases, it can be a better indication than the chart shown in the last post. The actual workbook that contains the range is relatively simple – here’s what that specific range looks like:

Here’s what the settings page looks like – notice that I still want to see the chart showing my stats when the gadget is maximized:

Once you hit OK and the gadget refreshes, you will see the score card showing in the gadget itself:

And just like in the previous post – clicking on the “Maximize” button will show the chart.

Here’s a short video that shows exactly how this works and what it looks like:

Enjoy!!

Introducing the Excel Services Windows 7 Gadget

2009-11-03T17:00:00Z

The past few posts have shown some of the basic REST functionality you can expect to get from the new Excel Services that will ship as part of SharePoint 2010. Before going into some of the more advanced features (well.. feature) of Excel Services REST, I thought I would show you a Windows Gadget we implemented for internal use. I will make the source code available either through the MSDN Code Library or through this blog in the very near future.

Now, gadgets are not hard to write, but they do require some knowledge of HTML and JavaScript. This Gadget is an absolutely great tool for people who have any sort of spreadsheet that contains any type of data that periodically changes and which is usually used as a sort of tracking application – instead of having to write the gadget yourself, you can just use the Excel workbook. You can think of any type of workbook – from the workbook that tracks your portfolio, to the one that displays information about your warehouse stock, to the one that tracks your project progress. This data is usually “locked” inside spreadsheets and one needs to manually open the workbook and actively look in. This gadget opens up an opportunity to use virtually any piece of information that resides within such workbooks and make it easily accessible right on your desktop – the gadget will make sure to periodically update the data so that with a simple glance, you can see exactly what’s going on!

Once installed, you add the gadget just like you would any other – run the “Desktop Gadget Gallery” in Windows 7 and double-click the Excel icon:

Once added to the desktop, you need to properly configure your gadget to show whatever information you want. In this example, we will show information about bug counts. For starters, you click the little wrench icon – that will get you to the settings screen:

Next, you type the name of the URL for the workbook in the “Workbook” field and hit Refresh. The Gadget will go and use REST discovery to figure out what elements exist in the workbook:

As you can see, the gadget lets you choose from any of the entities available in the workbook. For now, lets choose the “PersonalBugs” – it is what I like seeing on my desktop, so I can closely monitor my status.

As you can see, the chart is not really designed to be used in such a small space – clicking on the little arrow will “undock” the gadget (a term inherited from Vista) and will show the chart at full size:

Now, this looks much better. It can be even better though – if you noticed, in the first screenshot of the “Settings” screen, there are two combo boxes – one for the actual object you want to see and the other for the thumbnail. The nice thing is that you can set up the gadget to show you one element when it’s “minimized” and one when it’s “maximized”. In this example, we conveniently have a chart called “MiniPersonalChart” which contains the chart in a smaller format. Here is how you set the two objects up in the gadget:

Now, when the gadget is “minimized”, it will show the chart that’s better suited for smaller space (it has no header, legend or axis):

That’s it for this post! Next post will show how the gadget can show not only charts, but also show ranges!

Excel Services REST APIs – the basics

2009-11-02T17:00:00Z

Last week I posted a number of pieces about the various ways the new Excel Services REST APIs allows developers to leverage data and elements from within workbooks by giving direct URLs to them. This post contains details about the available capabilities of the REST APIs – essentially a list of all the entities that are available for deep-linking.

All the relative URLs presented are based off the base REST URL which I explained in the very first post – the base URL essentially follows the following template:

http://[server]/_vti_bin/ExcelRest.aspx/[DocLib]/[File.Xlsx]

The following table shows all the accessible resources off this base URL:

Resource	Format	Example	Notes
/Model	atom (default)		This returns an ATOM feed with the resources supported by the workbook (Ranges, Charts, PivotTables and Tables).
/Model	workbook	/Model?$format=workbook	The binary workbook (the xlsx/xlsb/xlsm) itself.
/Model/Ranges	atom (default)	/Model/Ranges /Model/Ranges?$format=atom	An ATOM feed listing all the named ranges in the workbook.
/Model/Ranges('[Name]')	html (default)	/Model/Ranges('MyRange') /Model/RangeS('MyRange')?$format=html	An HTML fragment for the requested range.
/Model/Ranges('[Name]')	atom	/Model/Ranges('MyRange')?$format=atom	An ATOM entry containing an XML representation of the data within the range.
/Model/Charts	atom (default)	/Model/Charts /Model/Charts?$format=atom	An ATOM feed listing all the charts in the workbook.
/Model/Charts('[Name]')	image (default)	/Model/Charts('MyChart') /Model/Charts('MyChart')?$format=image	An image (PNG form) of the chart.
/Model/Tables	atom (default)	/Model/Tables /Model/Tables?$format=atom	An ATOM feed listing all the available tables in the workbook.
/Model/Tables('[Name]')	html (default)	/Model/Tables('MyTable') /Model/Tables('MyTable')?$format=html	An HTML fragment for the requested table.
/Model/Tables('[Name]')	atom	/Model/Tables('MyTable')?$format=atom	An ATOM entry containing an XML representation of the data within the table.
/Model/PivotTables	atom (default)	/Model/PivotTables /Model/PivotTables?$format=atom	An ATOM feed listing all the available PivotTables in the workbook.
/Model/PivotTables('[Name]')	html (default)	/Model/PivotTables('MyPT') /Model/PivotTables('MyPT)?$format=html	An HTML fragment for the requested PivotTable.
/Model/PivotTables('[Name]')	atom	/Model/PivotTables('MyPT')?$format=atom	An ATOM entry containing an XML representation of the data within the PivotTables

As you can see, there are a number of elements inside workbooks that are addressable via a URL. For example, the table I pasted in here actually resides within Excel Services. This means that it is addressable via the /Model/Tables(‘[Name]’) rows. The following example shows this very same table accessed in IE by a direct link:

The name of the table is “Table1” – because the HTML format is the default one when accessing tables, that means that one does not need to specify it to get HTML back.

That’s it for now. The next post will show you a solution written using REST which will allow end users to put refreshable data from Excel Services on their desktop via a relatively simple Windows 7 Gadget.

What’s coming to Cum Grano Salis this week..

2009-11-02T16:00:00Z

Just thought I’d drop a quick note as to what to expect to see this week..

Today there will be a post about the basics of REST – essentially a reference manual to all the basic capabilities of the API.

After that, there will be two posts about the Excel Services Gadget I showed in SharePoint conference 2009.

Towards the end of the week there will be one post about the advanced capabilities REST has and then, the final post of the week will tie it all together and show how to leverage the heck out of your workbooks by using the Excel Services Gadget.

Most of the posts next week (week of the 9th) will show the inner working of the Excel Services Gadget.

Reason #4871 why I hate DHTML

2009-11-01T06:36:00Z

Margins. In script.

To set? Use element.style.marginTop.

To get in runtime? Use element.topMargin.

Arghhhhhhh! It's like DHTML is forcefuly trying to make me hate it.

Getting ranges via Excel Services REST as well as Charts and Discovery

2009-10-30T16:00:00Z

The previous post (and the one before) showed how the Excel Services REST API allows developers to discover information about a workbook as well as get back images of live charts from an Excel Workbook.

It is also quite possible to get other types of data – specifically ranges from the spreadsheet. Now, Excel Services supports two mechanisms for getting ranges through REST. The first is used mainly to allow applications to get to the raw-data of the workbook (i.e. Raw numbers/values from a sheet) and the other is to get HTML fragments for use inside a browser. In future posts, I will show how you can leverage these elements inside a page to create a seamless experience, but for now, I will just show the simplest use for both.

I will use the same workbook shown in the previous post – the one containing bug status information. In this case however, we will look at a different element in there – a named range called “PersonalScoreCard”. Using the discovery mechanism described in the previous post, if you go to the “Ranges” atom feed on the server, you land on a page that shows all the named ranges currently in the workbook (there’s only the one in this example):

Clicking on the name of the range will navigate to this URL: http://tsdlt2/_vti_bin/ExcelRest.aspx/Docs/Bugs.xlsx/Model/Ranges('PersonalScoreCard')?$format=atom

Inside IE, the result looks like an error though:

If you look closely, you will see that it’s just that IE does not know how to show a single-entry atom feed item. Viewing the source of the document will show the XML the feed item contains:

<?xml version="1.0" encoding="utf-8"?>

<title type="text">PersonalScoreCard</title>

<id>http://tsdlt2/_vti_bin/ExcelRest.aspx/Docs/Bugs.xlsx/Model/Ranges('PersonalScoreCard')</id>

</author>

<x:range name="PersonalScoreCard">

<x:row>

<x:c>

<x:fv>Shaharp:</x:fv>

</x:c>

<x:c>

<x:v>10</x:v>

<x:fv>10</x:fv>

</x:c>

</x:row>

<x:row>

<x:c>

<x:fv>Average:</x:fv>

</x:c>

<x:c>

<x:v>1.3333333333333333</x:v>

<x:fv>1.333333333</x:fv>

</x:c>

</x:row>

</x:range>

</content>

</entry>

As you can see, the feed item contains XML representing the data inside the range. Here’s the quick and dirty on the xml elements (though most of them are pretty self explanatory):

<range> : The container of the returned range.

<row> : Each row in the range that was requested is represented by one of these.

<c> : Each cell in a row is represented by this.

<fv> : Formatted Value – this is the value as it is formatted by Excel. If the value is of type string in the workbook, this element will be the only one under <c>.

<v> : In the case where the value in the cell is not a string but a number, the Value element will contain that information.

In this way it’s very easy to get data out of an Excel range and use it in your application.

But that’s not all. If you will notice, the final part of the URL contains a parameter called $format which is set to Atom. This parameter can also take the value of html – when that’s the case, the URL will return an HTML fragment rather than an atom feed:

http://tsdlt2/_vti_bin/ExcelRest.aspx/Docs/Bugs.xlsx/Model/Ranges('PersonalScoreCard')?$format=html

If you were to change the URL in IE, here’s the result:

This HTML can either be directly consumed in an IFRAME, or used in JavaScript to create a more seamless experience (I will show examples of this in later posts).

Next post will recap the basics of the Excel Services REST API and what resources are available through it.

Discovery via the Excel Services REST APIs

2009-10-29T16:00:00Z

In my previous post, I showed what embedding a chart from Excel Services inside a blog looks like. In this post, we will take a step back and explore the discovery mechanisms built into the Excel Services REST APIs.

Discovery allows developers and users to explore the content of the workbook either manually or programmatically by supplying ATOM feeds that contain information the elements that reside a specific workbook. Starting out with discovery is pretty easy. If you will recall, the link we used to access the chart in the previous post looked a like this:

http://tssrva/_vti_bin/ExcelRest.aspx/sites/sp2009/shared documents/Bugs.xlsx/Model/Charts(‘PersonalBugs’)

To start seeing how discovery works, simply chop off all that Charts business and you are left with:

http://tssrva/_vti_bin/ExcelRest.aspx/sites/sp2009/shared documents/Bugs.xlsx/Model

This is your anchor for starting discovery. Pointing your browser to this URL will show up the four resource collections we currently support:

You will notice that you get access to four resource collections: Ranges, Charts, Tables and PivotTables.

Clicking on the Charts link will bring to yet another ATOM feed – this one listing all the charts available in the workbook:

As you can see, there are two charts on the workbook – one called “PersonalBugs” (which was used in the previous post) and one called “TeamBugs”. Clicking on “TeamBugs” will navigate to the actual chart URL:

Using the ATOM feed provided by the REST APIs allows for an incredibly easy way of getting to the data you care about. The flip side of this is that since these are just ATOM feeds, they are readable by machine as well! So, in the previous screenshot, if you ask FireFox to give you the source of the document, you get a bunch of XML:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>

<title type="text">Charts</title>

<id>http://tssrva/_vti_bin/ExcelRest.aspx/sites/sp2009/shared%20documents/Bugs.xlsx/Model/Charts</id>

</author>

<entry>

<title>PersonalBugs</title>

<id>http://tssrva/_vti_bin/ExcelRest.aspx/sites/sp2009/shared%20documents/Bugs.xlsx/Model/Charts('PersonalBugs')</id>

</author>

</entry>

… [Removed the rest of the feed]

As you can see, the feed contains easily traversable elements that will allow any piece of code to go through and discover what elements exist in the workbook – each ATOM entry corresponds to a Chart that can be accessed.

This same mechanism applies to discovering PivotTables, Tables and Ranges. In the next post, I will show some of the capabilities revolving around those.

So what does REST on Excel Services look like???

2009-10-28T16:00:00Z

In my first post about Excel Services and REST, I showed how to construct a simple REST URL pointing directly to a PNG in a workbook. This post was supposed to be about the discovery built into the Excel Services REST API. However, I got a couple of questions about what my previous post actually looks like visually when used inside SharePoint 2010 and so I decided to post about that today instead. People like shiny screenshots.. :)

In the previous post, I gave the following example of a REST URL:

http://server/_vti_bin/ExcelRest.aspx/Shared Documents/Bugs.xlsx/Model/Charts(‘PersonalBugChart’)

In this post, I will show how a similar URL is used to post a chart about my alleged bug status into a SharePoint blog and have the chart be tied directly to the Excel file in which it resides.

First off – here’s what the chart looks like in Excel:

If you click the image you will see that the chart is called “PersonalBugs”. The actual location of the file is at “http://tssrva/sites/sp2009/shared documents/bugs.xlsx”. That means that the link to the chart inside Excel would be:

http://tssrva/_vti_bin/ExcelRest.aspx/sites/sp2009/shared documents/Bugs.xlsx/Model/Charts(‘PersonalBugs’)

The next step would be to create a blog post in SharePoint – when creating the blog entry, you can choose the “Insert” tab from the ribbon and then choose “From Address”:

Once you click address, all you need to do is put in the URL to the chart:

Once you hit OK, you will immediately see the chart appear within the blog post editor:

Hitting the Submit button will post the blog entry:

The real neat part then becomes that as the workbook changes (due to external data being refreshed, the spreadsheet itself being modified and saved etc), everybody will see the changes directly on the blog the next time they come in!

I hope the shiny screenshots made everything clearer.

Next up: Discovery!!

Welcome to the new Excel Services!

2009-10-27T16:00:00Z

Now that Microsoft SharePoint Conference 2009 is done, I have some more time to start posting about all the nice things we have done with SharePoint 2010 – especially with Excel Services. In the coming months, I will post about Excel Services, concentrating on some of the new capabilities we enabled via our new extensibility mechanisms.

Since the new REST capabilities we built into Excel Services are close to my heart, I will start discussing them first.

Now, if you know what REST is, you will probably aching to comment that what I am going to discuss is not really REST. There are quite a few parts missing and some things that may not be exactly up to the “standard” (as much as it is one, which is not much). If it’s easier for you, you can also call it “URL facilitated access to spreadsheet component elements”. I find REST quicker to say.

The new API allows you to access parts of the workbook directly through a URL. The first part of the URL is a representation of the workbook location. The second part is the path to the requested element inside the workbook. In practice, there’s also a “marker” which is our entrypoint (an aspx page).

To start off with a simple example, say there’s a workbook called bugs.xlsx which lives in a document library called “Shared Documents”. Say that workbook contains a chart called “PersonalBugChart”. You can get direct access to the chart via a simple URL:

http://server/_vti_bin/ExcelRest.aspx/Shared Documents/Bugs.xlsx/Model/Charts(‘PersonalBugChart’)

Each Excel Services REST URL is built out of three parts:

Marker path: That’s the “http://server/_vti_bin/ExcelRest.aspx”. To access REST you always need to preface it with this.

File path: That’s the “/Shared Documents/Bugs.xlsx” part. This is the file that contains the element you are interested in.

Element path: That’s “/Model/Charts(‘PersonalBugChart’)” part. This is the path inside the workbook to the element you request.

That’s it. The result of this URL is a PNG stream which would be the chart that is named PersonalBugChart. When the workbook updates, the REST path will return the new chart – this is a completely live URL. Placing this URL inside an img tag on a page will place the image on the page:

If you have SharePoint 2010 Enterprise, try it with your own Excel workbooks! It’s as easy as that! Change the chart type, the contents, anything, and the next time the chart is requested, you will see the new content.

In the next post, we will discuss how you can access ranges both as data (via XML) and as HTML fragments.

UlsViewer released

2009-10-14T04:00:00Z

The folks in Office Operations released the Uls Viewer tool – it allows users of SharePoint to easily view and analyze ULS logs produced by the system (though not initially designed for user consumption, people seem to have been using the logs pretty much since we shipped). This also allows users of Excel Services to gain some insight as to what’s going on inside the product as it’s running.

http://code.msdn.microsoft.com/ULSViewer

This is a fun moment for me – the tool was originally written by my team as we were tired of using [insert favorite text editor here] to try and understand the logs we produced. Being the new kid on the block in Office 12, we were using the ULS with a gusto (I think we are probably still the team that use logging the most in SharePoint) and we liked the fact that we could track everything the server was doing. At some point it became too much, so we sat down and wrote a tool that would allow us to view, dice, slice, analyze and otherwise BI the heck out of our logs.

During O14, the tool was pretty much taken over by the ULS folks who have fixed many of the bugs we had in the original tool. On top of that, they thought it would be useful for users to have and asked us if we were okay with releasing it publicly. Once all the profanity was removed from the tool and some of the bugs squashed (not all of them, as you will notice in the Filter dialog box and other places), the tool was apparently deemed worthy of “shipping” under the MSDN code library.

Both my team and the ULS team are looking to fix more bugs and make the tool more efficient memory-wise – if you have any other suggestions, please be sure to let us know through the tool page above, or by commenting on this blog.

Optimizing life – Life in Async

2009-07-01T20:43:09Z

I was going to get coffee today at work, minding my own business, when someone from the test-team asked me if the order I was doing things when preparing coffee was intentional. Absentmindedly, I explained the reasoning behind the order of things and how I reached this way of doing things and belatedly realized that I may have gone too far in my explanation – I think it was the look of pity in his eyes that clued me in.

To explain, first you need to understand what our kitchenette looks like and the various steps in making coffee:

Here are the various things we are seeing here:

1) Access – that’s where I typically come to the kitchenette from.
2) The coffee machine
3) Counter
4) Cups
5) Lids
6) Fridge (Where milk is).

Our coffee machines basically give you an Americano – no milk. The way I take my cup’o’joe is to do 12 oz of machine generated coffee (into a 16 oz cup) and fill another 2-3 oz of milk – and then the cup needs to be lidded as well. Of course, there are a number of variations on how to make such a coffee with this layout.

Making Coffee – the naive way:

Step 1: Go to position (4) – grab a cup.
Step 2: Go to position (1) – place cup at machine.
Step 3: Press buttons - Wait for coffee to be made.
Step 4: Take milk from fridge (6)
Step 5: Fill cup with milk (possibly at position (3))
Step 6: Discard milk carton or, if still full, place back in fridge.
Step 7: Go to (5) and grab a lid.
Step 8: Place lid on cup.

And we are done. To understand how this can be optimized we first need to figure out how much each part of the coffee making process takes:

Making coffee (the machine phase): ~2 minutes (constant).
Getting milk (from position (1)): On average, takes about 20 seconds. Depending on traffic, can easily take as long as 1:00 minutes if there is a lot of traffic and even longer if you consider the possibility of people talking to you while you are making the coffee.
Getting a cup: Usually takes about 5 seconds from position 1 – can take longer due to traffic and chat. Rarely takes more than 30 seconds and requires little concentration.
Filling cup with milk: Takes about 5 secodns.
Discarding/Returning milk: Takes about 10 seconds. Worst case (traffic etc) can take up to a minute.
Grabbing a lid: 5 seconds from position (1).
Lidding the cup: 5 seconds at position (3).

So.. Translating this into pseudo code, this is what we get:

function NaiveCoffeeMaking()
{
WalkTo(4);
GetCup(); // 10s
WalkTo(2);
PlaceCupInMachine(); // 1s
MakeCoffee(); // 120s
GrabCupFromMachine(); // 1s
WalkTo(6);
GetMilk(); // 20s
PourMilk(); // 5s
DiscardOrReturnMilk(); // 10s
WalkTo(5);
GrabLid(); // 5s (probably closer to 3s in this case)
WalkTo(3);
LidCup(); // 5s
}

Phew.. Okay.. So we have that. Now lets calculate how much time this takes (best case):177 seconds!

The thing to consider though is that the machine phase can be asynchronous – that is to say – while the machine is making coffee, the CPU (me) is waiting Idle. The question is, how can we use that to optimize the time? For our pseudo code then, we will now separate MakeCoffee into two functions: BeginMakeCoffee() which essentially means pressing the buttons on the machine and immediately return to the caller and EndMakeCoffee() which completes the operation (and if not done, blocks until done). And so, the most optimized version of how to make coffee is:

function FullyOptimizedCoffeeMaking()
{
WalkTo(2);
BeginMakeCoffee(); // 120s – but returns immediately
WalkTo(4);
GrabCup(); // 5s
WalkTo(6);
GetMilk(); // 20s
PourMilk(); // 5s
DiscardOrReturnMilk(); // 10s
WalkTo(5);
GrabLid(); // 5s (probably closer to 3s in this case)
WalkTo(2);
PlaceCupInMachine();
EndMakeCoffee();
LidCup(); // 5s
}

Notice that when we call BeginMakeCoffee(), we immediately return and continue processing – going through the phases of getting the milk, preparing the cup and only then, when done, placing it in the machine. So, in this case, best case time would be: 125 seconds. That’s 120 seconds that it takes to make the coffee plus the 5 that it takes to lid the cup. The rest of the time is taken up in parallel to making the coffee.

But there’s a problem! BeginMakeCoffee() and EndMakeCoffee() do not work exactly like a machine async operation. Generally speaking, when you issue an async operation, the sub-system will take care to store the result for you when it’s done and wait patiently for you to call the second part of the operation (EndMakeCoffee) before it returns the result. A coffee machine does not work that way. When the coffee is ready, it’s coming out – whether there’s something to store it or not. That presents some problems for the FullyOptimizedCoffeeMaking() function! What happens if instead of talking about Best case, we were to talk about worst case. The GetCup() can sometimes take 30 seconds. GetMilk can sometimes take 120 seconds. As well as some of the other things that make take longer under high-stress cases. What happens then? Well, what happens is that the coffee starts pouring and there’s nowhere for it to go, so it gets wasted. Sacrilege!!

In essence, we need something that’s both optimized AND safe. Since life does not have convenient temporary memory (where the coffee would have been stored) or sync objects (where the act of pouring the coffee could have been locked and deferred until a cup is to be placed underneath the nozzle), it means we need to work around those limitations:

function OptimizedAndSafeCoffeeMaking()
{
WalkTo(2);
BeginMakeCoffee(); // 120s – but returns immediately
WalkTo(4);
GrabCup(); // 5s
WalkTo(2);
PlaceCupInMachine();
WalkTo(6);
GetMilk(); // 20s
WalkTo(2);
PourMilk(); // While cup is in the machine – 10s
WalkTo(6)
DiscardOrReturnMilk(); // 15s
WalkTo(5);
GrabLid(); // 5s (probably closer to 3s in this case)
WalkTo(2);
EndMakeCoffee();
LidCup(); // 5s
}

The CPU (me) now needs to work more – you will notice that there is more walking around to accommodate the new placement of the cup (it can no longer be carried around – one needs to walk to it to pour the milk and then back to the fridge to discard or put back). The time this now takes is still 125 seconds - 120s to make the coffee and prepare the cup and another 5s at the end when the cup is lidded. Switching the lines around and doing more leg work though has bought us a lot more safeness.

Note that this is still not perfect – if the 5s it takes to get a cup and place it in the machine would balloon from 5s to 120s, we would be right back in our original position of losing our coffee. However, that is far less likely to happen in this case. If you want to eliminate risk completely, simply switch BeginMakeCoffee() with GrabCup() and PlaceCupInMachine(). You will lose 5 seconds (making the total time 130 seconds), but you are then completely safe.

So that’s what I was trying to explain to the tester when that look came to his face. The thing is, I am sure everyone does this. But people probably do this unconsciously or at least in a less ~~obsessive compulsive~~ analytic manner than a software developer would.

Windows 7 RC – Media Streaming (“Stream” button in Windows Media Player)

2009-05-07T23:28:49Z

I found out about this feature a bit ago and.. Wow.. I am so impressed. The set-up procedure is a little convoluted, but once you do it, the thing just works.

Here’s the idea: Allow your identity (your Windows Live account, basically) to stream your media accross the interwebs. If my home computer has my entire outdated music library on it, I don’t need to copy the entire thing to my laptops to access it – I just make sure to associate my Windows Login user to my Live account user and voila – I can stream media from my house when I work!

Step by step:

You can associate your live-id to your windows user account by using the control panel, but Windows Media Player gives you a shortcut. In the Stream drop-down, choose “Allow Internet access to home media”:

Then, you will be presented with a dialog containing the options you have. You should click “Link online ID”:

Once you click that, WMP will bring you to a window that allows you to add an online ID provider. Click the “Add an online…” button:

Once clicked, you will be presented with providers. Click the “Windows Live” one – that will again redirect you to a download from Microsoft. Choose the correct architecture (x86 or x64) and download/install the package. Once you did that, going back to the “Link online IDs” dialog will look like this:

You now need to select the “Link online ID” on the WindowsLiveID highlighted item. That will ask you to log into Windows Live with your Live-ID. Once that’s done, the window will again change to show the linked account:

If you go back to Windows Media Player, you should choose the”Allow internet access to home media” option on the open dialog box (same as the second image in this post). Once clicked, the dialog should close and you should land back inside Windows Media Player. But now, you will have your home library available to choose from:

You can now play your music to your heart conent. Apparently, this also supports streaming of video. The quality seems kind’a random – I have seen it be very good (indistinguishable from the original) and pretty bad. I guess it takes your bandwidth into account:

Enjoy!!