Excel Programming

Custom Task Panes

2008-08-14T01:29:00Z

A Custom Task Pane (CTP) is a user interface component in Office which is used to provide a non-modal window. Some of Excel's features, such as the PivotTable field list and Reseach tool, use CTPs.

CTPs are exposed to customer extensions via the Office extensibility COM libraries. Any Excel developer can develop their own CTP and have it work just like the built-in CTPs.

Despite what some people think, you do not need VSTO to build a CTP.

In this post I will develop an Excel feature called: annotations. Annotations (as implemented here) are basically free-from textual comments that can be added to a workbook. They are added into the file but are not displayed on the grid. You can use them to store notes or commentary in the workbook. They are displayed and edited in a Custom Task Pane as shown below.

The annotations are stored within the CustomXMLParts collection of the workbook. CustomXMLParts are a new feature in Excel 2007 which enable Excel developers to store their own XML within a workbook. I will cover CustomXMLParts in my next post.

What are Custom Task Panes?

A CTP is a simple window that contains an ActiveX control. Excel manages the CTP window (creates it, destroys it, handles its window messages etc.) and the ActiveX control provides all the features. The CTP is really just a container, but a container that is nicely integrated into Excel. A CTP can be docked inside the main Excel window or can float. The user can resize it, move it and close it and all of this is handled for us by Excel.

Building a Custom Task Pane

Creating a CTP itself is very simple. CTP functionality is provided via a COM interface called ICustomTaskPaneConsumer. From a .NET point of view, this interface is implemented in Microsoft.Office.Core. When Excel loads our COM addin it performs a QueryInterface call to see whether or not our addin implements the ICustomTaskPaneConsumer interface. If it does, Excel calls the ICustomTaskPaneConsumer.CTPFactoryAvailable method. It is in this method that our addin creates the CTP.

Let's take a look at the code.

public class Connect : Object, Extensibility.IDTExtensibility2, IRibbonExtensibility, ICustomTaskPaneConsumer {

public void CTPFactoryAvailable(ICTPFactory CTPFactoryInst) {

_ctpFactory = CTPFactoryInst;

CustomTaskPane ctp = _ctpFactory.CreateCTP("ExcelExtensions.AnnotationsCtrl", "Annotations", Missing.Value);

_annotationsExt = new AnnotationsExtension(ctp, _application);

}

The Connect class implements the ICustomTaskPaneConsumer interface. Its implementation of the CTPFactoryAvailable method does two things. Firstly, it stores the ICTPFactory object passed to it by Excel and secondly it creates a CTP using the ICTPFactory.CreateCTP method. The first parameter to the CreateCTP method is the ProgId of the ActiveX control that the CTP is to host. The CTP will create an instance of this ActiveX control and place it within the CTP window.

CreateCTP returns a CustomTaskPane object which we store for later use. The CustomTaskPane object is how our addin can interact with the CTP window itself. In the example above, we store the CustomTaskPane object inside an AnnotationsExtension object but we can actually store it anywhere we like. The AnnotationsExtension object is a class inside our addin - the details of this class will be covered in the next post - for now, all we need to know is that it is a conceptual wrapper of our CTP.

So, the basic workflow in creating a Custom Task Pane is as follows:

Displaying a Custom Task Pane

So far we have created a CTP. We have a reference to it and the CTP knows which ActiveX control it is hosting. However, we need to explcitly display the CTP in order for it to be visible.

We do this via the CustomTaskPane.Visible property. Setting this property to true or false will display or hide the CTP (notice that the CTP is hidden - not destroyed).

This makes sense because we'd typically want to show or hide our CTP based on some user action, like a user clicking a button on the Ribbon. In this example we'll add an Annotations button onto the Review tab of the Ribbon.

<toggleButton id="AnnotationsButton" onAction="OnAnnotationsClicked" label="Annotations"

imageMso="ExchangeFolder" supertip="Add new and view existing annotations." size="large"/>

</group>

</tab>

We then implement the OnAnnotationsClicked event handler in our COM addin.

public void OnAnnotationsClicked(IRibbonControl Control, bool IsPressed) {

_annotationsExt.Visible = IsPressed;

}

Because we added a toggleButton control to the Ribbon, our event handler gets passed a boolean which indicates whether the toggleButton is pressed or not. All we need to do is set the Visible property of the CTP equal to this value.

Writing the ActiveX Control

We now have enough code to create and display a CTP when a button on the Ribbon is pressed. The final bit (and the harder bit) is writing the ActiveX control ... this is the thing that actually implements our custom features.

In this example, our ActiveX control reads and write annotations to the CustomXMLParts collection of the active workbook. I'll be covering the CustomXMLParts collection, how it is stored in the new Office XML file formats and how the ActiveX control works next time.

What's great is that the CTP can host any ActiveX control. So even though the CTP itself is native code, we can write our ActiveX control in .NET code and it all works.

Summary

To create a Custom Task Pane using managed code:

Add a reference to Microsoft.Office.Core;
Implement the ICustomTaskPaneConsumer interface on your addin class (i.e., the Connect class);
In your implementation of ICustomTaskPaneConsumer.CTPFactoryAvailable call CreateCTP on the supplied ICTPFactory object;
In the call to CreateCTP, specify the ProgId of the ActiveX control you want to host in the CTP;
Store the CustomTaskPane object returned from CreateCTP and toggle its Visible property to show and hide the CTP as and when required;

Links

The following link gives more information on creating Custom Task Panes:

http://msdn.microsoft.com/en-us/library/aa942864.aspx

Next Time ...

We'll take a look at how the Annotations ActiveX control uses the CustomXMLParts collection to store the annotations entered by the user.

How to code a .NET RefEdit Control

2008-06-12T06:02:00Z

One of the controls frequently used by Excel is the reference edit box. This control is used to gather a range address from the user. The user specifies the range either by typing in its address or by using the mouse and clicking on the actual cells they want to use.

Here's a screenshot of the inbuilt reference edit box:

Whenever the user is required to specify a range, using the reference edit box is my preferred way to gather the address from the user because it enables (what I call) visual selection of ranges. Range addresses can be tedious to work with whereas visually clicking on the cells you want is a much easier and more efficient way of working.

Custom solutions can also use visual range selection because Excel ships a control called the RefEdit control. This control is implemented in REFEDIT.DLL located in the Office folder (on my laptop this is: C:\PROGRAM FILES\MICROSOFT OFFICE\OFFICE12) and mimics the inbuilt reference edit control.

Well that's the good news ... now the bad: the REFEDIT.DLL cannot be used from managed code. In fact, it can only be used from VBA. Years ago, that wasn't a problem because most solutions were VBA based. However, these days we live in a .NET world and more and more solutions are being written in managed code.

So, what are the options if we want to use the RefEdit control from managed code?

The way I see it, there are three options. Firstly, we could write an XLA containing a VBA userform on which we place the RefEdit control. We would then have to link the managed code and the VBA userform via macros somehow. The downsides of this option are that we create a mixed code solution i.e., we have some VBA and some managed code. This may or may not be an issue for us.

The second option is to buy a commercial RefEdit control that can be used by .NET WinForms. Actually, I don't know if such a thing exists but let's assume that it does.

The third option is to write a managed code RefEdit control from scratch. This option is probably the most work and might be quite hard. However, I decided to give it a go...

In this post, I present a managed code RefEdit control written in C#. I have implemented the control as a .NET UserControl which can be added into the Visual Studio toolbox and dragged and dropped onto any WinForm just like any other .NET control.

The control I present does not behave exactly like Excel's reference edit boxes but I think it behaves close enough that most people wouldn't notice the difference.

Here's a screenshot of the control in action:

This example uses the managed code addin I presented in a previous post (here) but the control can be used on any WinForm.

You are free to download the control and use it in your solutions. It is totally free (in all senses of the word) and you can take the code and modify it without restriction. All I ask is that you let me know if you improve it or find any bugs so that I can incorporate those improvements for the benefit of the community.

Note: The RefEdit control also uses the ExcelUtility class I published previously in this post.

How to code a RefEdit Control in C#

Let's first think a bit about the behaviour of the RefEdit control. It does some fancy things in very subtle ways. It's one of those wonderful UI concepts that we use everyday without even thinking about its complexities.

When we click on the button in the RefEdit control, the window on which the RefEdit control resides is minimised and all controls on the window are not displayed. Clicking on the button again restores the window to its original state.

The other feature to note is how the visual selection works. When the text box in the RefEdit control has focus, the address of the selected cells in Excel is displayed. Even if multiple, discrete ranges are selected the addresses are displayed. The user can even change sheets and select ranges on different sheets. This all takes place while the window containing the RefEdit control is still visible but is in its minimised state and all the time the addresses of all the selected ranges are displayed in the RefEdit control.

These two features are the killer features of the RefEdit control. If we can code these features then we are well on our way to our goal of having a managed code RefEdit control.

So, let's take a look at how we do this.

First, we need to have a function that can minimise its parent window and hide all controls on the window except the RefEdit control. Also, this function needs to remember the window's previous state and restore the window to that state once the user has selected their range.

protected void ResizeParent() {

Form parentForm = (Form)Parent;

foreach (Control c in parentForm.Controls)

c.Visible = _state.ParentMinimised;

Visible = true;

if (!_state.ParentMinimised) {

_state.ParentClientSize = parentForm.ClientSize;

_state.PrevX = Left;

_state.PrevY = Top;

_state.Anchor = Anchor;

Anchor = AnchorStyles.None;

parentForm.ClientSize = new Size(Width, Height);

Left = 0;

Top = 0;

_state.ParentPrevBorder = parentForm.FormBorderStyle;

parentForm.FormBorderStyle = FormBorderStyle.FixedDialog;

_state.ParentControlBox = parentForm.ControlBox;

parentForm.ControlBox = false;

}

else {

parentForm.ClientSize = _state.ParentClientSize;

Anchor = _state.Anchor;

Left = _state.PrevX;

Top = _state.PrevY;

parentForm.FormBorderStyle = _state.ParentPrevBorder;

parentForm.ControlBox = _state.ParentControlBox;

}

Notice that the first line of code casts the control's Parent to a Form object. This is our first limitation. If this cast fails then our control won't work. Why would the cast fail? It would fail if the parent is not a Form. This means that our RefEdit control's direct parent must be a Form. In other words, we can't place our RefEdit control within another control (such as a group box); it must be placed directly on a Form.

The _state object is an instance of the following struct:

struct RefEditState {

public Size ParentClientSize;

public bool ParentMinimised;

public int PrevX;

public int PrevY;

public FormBorderStyle ParentPrevBorder;

public bool ParentControlBox;

public AnchorStyles Anchor;

public Excel.Worksheet Sheet;

public bool UseSheetAddress;

}

We use this struct to store state information while we are minimising the parent window. Any property of the parent window that we change during the minimising we first save in the _state object. If the window is to be minimised (this happens when the user clicks on the button in the RefEdit control), all controls except the RefEdit control are hidden and the window is resized so that it is fits snuggly around the RefEdit control. That is, the window is made to look like this:

When the window is to be restored (when the user clicks on the RefEdit button again) we re-apply the settings stored in _state.

Updating the RefEdit Control with the Selected Range

The RefEdit control now resizes and restores its parent window whenever the user clicks on the button. Now we need to display the address of the ranges the user selects in Excel, while the user is selecting them.

So, if the user clicks $A$1 the RefEdit should display $A$1 in its text box. If the user selects the range $A$1:$B$3 the RefEdit should display $A$1:$B$3. Also, we want to support selecting multiple ranges. So, if the user selects $A$1 and $F$3 the RefEdit control should display $A$1,$F$3.

We detect which ranges the user is selecting by sinking the SheetSelectionChange event on the Excel.Application object.

private void SinkEvents() {

_con.ExcelApp.SheetSelectionChange += new Excel.AppEvents_SheetSelectionChangeEventHandler(ExcelEvent_SheetSelectionChange);

_con.ExcelApp.SheetActivate += new Excel.AppEvents_SheetActivateEventHandler(ExcelEvent_SheetActivate);

}

However, we don't want to always sink these events. We only want to sink them while the RefEdit is active. By active I mean the user has either clicked inside the RefEdit's text box or has clicked the RefEdit's button. In other words, we only sink these events while the user is using the RefEdit control.

We trap the Enter event of the RefEdit's address textbox and sink the Excel events whenever this event fires. We also save the active sheet in the _state object.

void _addressTxt_Enter(object sender, EventArgs e) {

SinkEvents();

if (_state.Sheet == null)

_state.Sheet = ExcelUtility.ApplicationInvoke<Excel.Worksheet>(_con.ExcelApp, "ActiveSheet");

}

We detach from the events when the stops using the RefEdit control.

private void UnsinkEvents() {

_con.ExcelApp.SheetSelectionChange -= ExcelEvent_SheetSelectionChange;

_con.ExcelApp.SheetActivate -= ExcelEvent_SheetActivate;

}

private void _addressTxt_Leave(object sender, EventArgs e) {

UnsinkEvents();

}

Notice that we also sink the SheetActivate event. We do this because the RefEdit control allows users to select ranges on any sheet; not just the sheet that was active at the time the RefEdit was displayed. Therefore, if the user selects a range on a different sheet we need to include the sheet name in the address of that range. For example, if Sheet1 is active when the RefEdit control is displayed but the user selects $A$1 on Sheet2 then the RefEdit control needs to display 'Sheet2'!$A$1 ... not $A$1.

One way of solving this problem is to make the RefEdit control always include sheet names in range addresses but this makes for long, messy addresses and Excel's reference edit boxes don't behave like that ... so neither should our RefEdit control.

Thus, we only want to include sheet names in the addresses if we have to. If we don't have to then we should omit them.

And that's why we sink the SheetActivate event.

Here's the code that runs when the SheetActivate event and the SheetSelectionChange event fire:

void ExcelEvent_SheetSelectionChange(object Sh, Excel.Range Target) {

string address = "";

try {

Excel.Worksheet sheet = (Excel.Worksheet)Sh;

if (_state.UseSheetAddress) {

foreach (Object obj in Target.Areas) {

if (address.Length > 0)

address += ",";

Excel.Range rng = (Excel.Range)obj;

address += "'" + rng.Worksheet.Name + "'!";

address += ExcelUtility.RangeInvoke<string>(rng, "Address", true, true, Excel.XlReferenceStyle.xlA1);

}

else {

address += ExcelUtility.RangeInvoke<string>(Target, "Address", true, true, Excel.XlReferenceStyle.xlA1);

}

_addressTxt.Text = address;

}

catch (Exception) {

}

void ExcelEvent_SheetActivate(object Sh) {

Excel.Worksheet sheet = (Excel.Worksheet)Sh;

if (!_state.UseSheetAddress)

_state.UseSheetAddress = sheet.Name != _state.Sheet.Name;

}

As the user selects ranges Excel fires the SheetSelectionChange event. If the range that was selected is not on the sheet that was active when the RefEdit control was displayed we append the sheet name to the address, otherwise we don't.

Using the RefEdit Control on a WinForm

The RefEdit control is a normal .NET UserControl so it appears in the Visual Studio toolbox like every other UserControl. You can drag and drop it onto a WinForm as required.

However, when we display a window which contains a RefEdit control we have to do so in a certain, non-standard way in order for the RefEdit control to function properly.

In order for the user to be able to select ranges in Excel while the window containing a RefEdit control is visible, the window must be non-modal. To display a non-modal window we normally invoke its Show()method. We still do that, but we have to invoke the version of Show()that takes a native window handle as a parameter. This handle is the handle of the native window which is the parent window of the non-modal window. When we display a non-modal window in Excel we need to make sure that we tell the non-modal window that Excel is its parent window. This ensures that the window behaves properly (for example, when the user alt-tabs to another application).

The overloaded Show() method we use actually takes an IWin32Window. IWin32Window interface is defined in the .NET FCL and is used to wrap a native HWND. We have to create a class that implements the IWin32Window interface and supply an instance of that class to the Show() method of the window containing a RefEdit control.

The following code defines a class, NativeWindowWrapper, that implements the IWin32Window interface.

class NativeWindowWrapper : IWin32Window {

IntPtr _handle;

public NativeWindowWrapper(int Hwnd) {

_handle = new IntPtr(Hwnd);

}

public IntPtr Handle {

get { return _handle; }

}

We use this class when we display a window containing a RefEdit control. We pass an instance of this class into the Show method of the window containing the RefEdit control.

In the following example we are using the ColumnAnalysisDisplayWindow discussed in a previous blog post ... but the idea is obvious: replace ColumnAnalysisDisplayWindow with your own window.

protected void DisplayStatsWindow() {

ColumnAnalysisDisplayWindow window = new ColumnAnalysisDisplayWindow();

window.Show(new NativeWindowWrapper(_connection.ExcelApp.Hwnd));

}

Summary

In this post we've talked about how to code a RefEdit .NET control that mimics the behaviour of Excel's inbuilt reference edit box. I'm not claiming this is absolutely the best implementation we could produce, but I think it's not bad and is fairly easy to do.

There are (no doubt) bugs in the code and situations where the control doesn't work that well. As I encounter and fix these prolbems I'll post updated versions of the control. If you encounter a problem, feel free to drop me a line and tell me about it.

The following points summarise this post:

The RefEdit control mimcs Excel's reference edit controls by enabling the user to visually select ranges;

The control does not behave exactly like Excel's reference edit boxes but is fairly close;

The control is implemented as a .NET UserControl;
The control requires:

A reference to the Excel 2007 PIA;
The ExcelUtility class defined in my ExcelExtensions COM addin (see below);

Windows that contain an instance of the RefEdit control have to be displayed non-modally;

And Excel's HWND must be specified as the native owner of the containing window;

Downloading the source code

I have updated my ExcelExtensions addin to include the RefEdit control.

All the source code is included in the ExcelExtensions download found here.

There are two downloads on the Downloads tab. These are:

ExcelExtensionsSrc.zip
ExcelExtensions.msi

ExcelExtensionsSrc.zip contains the complete source code and ExcelExtensions.msi is the set up program.

Upcoming Topics

2008-05-28T03:39:00Z

I know that posts have been few and far between (ok ..non-existant) this month. I hope this isn't something that will be repeated ... this month has just been crazy busy with deadlines (I'm sure you know that feeling...)

Anyway, I have a healthy backlog of topics that I want to blog about over the summer:

Multi-threaded UDFs in Excel 2007;
RefEdit control for COM addins written in managed code;
C API enhancements in Excel 2007;
PivotTable stuff

I get lots of questions about PivotTable programming so there is plenty I want to talk about here;

Ribbon customisation;

I'm at TechEd next week and am hoping that the flight time there will give me time to get started on some of these posts.

Feel free to drop me a line if there is something you would like covered.

Busy Times ... and a Note about my TechEd Sessions

2008-04-26T02:04:00Z

Busy times at Microsoft has meant that I've been slower than usual in posting recently. My current expectation is to be over my heavy workload by about mid-June. I'll continue to post in the meantime; but at a slower pace.

TechEd is being held in Orlando in June this year. There are a lot of very interesting sessions being held and the key note will be done by Bill Gates. If you can make it, I'd recommend coming along. You can find out more about TechEd here.

I have two speaking slots at this year's show: one in the Developer track and one in the IT Professional track.

In the Developer session I'll be talking about the improvements we made to XLLs in Excel 2007. In the IT Professional session I'll be talking about the Business Intelligence capabilities of Excel 2007 and Excel Services.

Writing Custom Excel Worksheet Functions in C#

2008-04-07T14:00:00Z

Writing our own, custom worksheet functions is a great way to extend Excel. Before Excel 2002 we developed custom functions by either writing them using VBA inside an XLA or by using C/C++ inside an XLL. Excel 2002 introduced a new type of addin called an automation addin. An automation addin enables Excel to call functions on COM objects from cells on a worksheet. In other words, it enabled us to call COM functions just as if they were normal, built-in Excel functions.

This opened up the world of custom worksheet functions to COM developers. So, now we can now write custom functions in any COM language; including C#.

In this article we'll write a custom worksheet function in C# called UNIQUEVALUES.

What will our Custom Function Do?

The UNIQUEVALUES function will return an array of the unique values in a specified range. For example, if the range A1:A5 contains the values Red, Yellow, Red, Blue, Yellow, then {=UNIQUEVALUES(A1:A5)} will return the array {Red, Yellow, Blue}.

The function will also be able to cope with a mixture of numerical data and text data. So, if A1:A5 contains the values: Red, 1, 1, Red then {=UNIQUEVALUES(A1:A5)} will return the array {Red, 1}.

Because UNIQUEVALUES is a worksheet function that references the source data, whenever the values in the source range change the function automatically extracts the new unique values from the range. So we know that the values returned from the function are always up-to-date (once the worksheet has calculated).

Notice that the UNIQUEVALUES function returns an array. This means that we array enter the funtion by pressing Ctrl-Shift-Enter rather than just Enter when we type the function into the worksheet.

Coding the UNIQUEVALUES Function

There are two tasks we need to do in order to make a C# class's methods be callable as worksheet functions:

Define our worksheet functions in the way Excel expects;
Add the Programmable key to the registry for our class's CLSID;

The first task seems intriguing: Define our worksheet functions in the way Excel expects. What does that mean?

Well, this is where we have to remember that Excel is speaking to our automation addin via COM. So Excel expects to be able to use COM techniques to discover and access our functions. Once we know what these COM techniques are, we can make sure that our C# class interoperates with COM in the appropriate way.

When an automation addin is loaded, Excel needs to discover which functions that addin supports. For example, our automation addin will have a UNIQUEVALUES function that we want to use in worksheets. How does Excel discover that our addin supports that function and how does it discover information about the parameters of the function?

Excel discovers this information using a COM interface called ITypeInfo.

ITypeInfo is an interface that is used to access metadata about a COM type. It is similar in concept to .NET reflection, but is very different in implemenation.

Excel uses ITypeInfo to discover the names and the parameter details of the functions exposed by our automation addin. This is how Excel knows to call our addin whenever it comes across a call to UNIQUEVALUES.

But Excel will only query the default interface of our COM class. So we need to make sure that the functions we want to expose as worksheet functions are defined on the default interface of our class.

There are two ways of doing this in C#:

By specifying the ClassInterfaceType.AutoDual attribute on our C# class;
By defining our worksheet functions on a dedicated interface and using the ComDefaultInterface attribute to make that interface the default interface of our class;

To understand what the ClassInterfaceType.AutoDual attribute value does to our COM interface, we need to understand a little more about COM interfaces.

In COM, the IDispatch interface enables clients to dynamically call your functions at runtime. Instead of the compiler checking that functions exist and that the parameters types match at compile-time, IDispatch enables clients to do so at runtime. This enables components to be extremely loosely coupled. This type of function call is termed: late-bound.

Support of IDispatch is optional. COM classes do not have to support it. However, by default, C# classes support IDispatch. In fact (unless we specify otherwise) the default interface created for a C# class is a dispatch interface.

Why do we need to care about this? Well, since the default interface is a dispatch interface our worksheet functions are not explicitly defined on the interface and are therefore not discoverable by Excel. So, even though our C# class may have a public function named UNIQUEVALUES Excel would not be able to call it.

A dual interface allows a dispatch interface to explicitly define custom functions in addition to those already defined by IDispatch. The ClassInterfaceType.AutoDual attribute turns our COM interface into a dual interface. This effectively takes all public functions defined on our class (including those inherited from base classes) and explicitly defines them on our COM interface which in turn makes them discoverable by Excel.

Thus, an easy way of getting our C# class to define Excel worksheet functions is to simply mark the class with ClassInterfaceType.AutoDual.

However, there is a downside to doing this. All public functions on our class are then available as worksheet functions, including those inherited from System.Object. This isn't ideal. It would be better if we could have more control over what gets exposed to Excel and make sure that only the functions that are intended to be worksheet functions are actually discoverable by Excel. This is the reason I prefer not to use the ClassInterfaceType.AutoDual attribute.

(There is also a problem with COM versioning when using ClassInterfaceType.AutoDual - but we won't cover that here).

Instead, I prefer to define the worksheet functions on a dedicated interface and have my C# class implement that interface.

So, let's define our worksheet function interface. We'll call it IFunctions.

public interface IFunctions {

object[,] UNIQUEVALUES(Excel.Range TargetRange);

}

Notice that the UNIQUEVALUES function returns a 2-d array of objects. This array will contain the unique values extracted from the specified Excel.Range object.

Next, we define our functions class and have it implement the IFunctions interface. We also use the default attribute to specify that we want the IFunctions interface to be the class's default COM interface.

[ComDefaultInterface(typeof(IFunctions))]

public class Functions : IFunctions {

public object[,] UNIQUEVALUES(Excel.Range TargetRange) {

object[,] values = TargetRange.get_Value(System.Reflection.Missing.Value) as object[,];

List<object> unqVals = new List<object>();

foreach (object obj in values) {

if (!unqVals.Contains(obj))

unqVals.Add(obj);

}

object[,] resVals = new object[unqVals.Count, 1];

for (int idx = 0; idx < resVals.Length; ++idx)

resVals[idx, 0] = unqVals[idx];

return resVals;

}

The algorithm for extracting the unique values is as follows (I confess that this may not be the optimal algorithm).

We extract the values in the Range object into a 2-d array of object. Because the array contains object, we can cope with both textual data and numerical data. Next, we iterate over the array and store each value that hasn't already occurred in a List<object>. We then allocate a new 2-d array of object and copy the unique values from the List<object> into the new array. Finally, we return the new array to Excel.

When we return an array from a UDF Excel handles serialising the array into the worksheet in the appropriate way (as long as the user array-entered the cell formula). This means that the dimensions of the array matter. The first dimension of the array is mapped to columns. So if we return an object[3] then Excel interprets this as "three columns containing one row each" and places the three values adjacent to each other (i.e., each one in a different column). If we return an object[1,3]then Excel interprets this as "one column containing three rows" and places each value beneath each other (i.e., each one in a different row).

So, it is very easy to return an array to Excel. All we have to do is set the correct array dimensions and let Excel handle copying the values from the to the appropriate cells in the worksheet.

We'll talk more about how arrays are used in Excel worksheet functions later in the article.

If we stop coding here, we'll find that Excel still can't use our class. There is one final thing we need to do: set the class's Programmable key in the registry.

The Programmable key indicates that a COM class exposes type information for its default interface. The key is specified under the COM class's HKEY_CLASSES_ROOT\CLSID\{Guid} key (where {Guid} is the guid of the COM class).

There is a convenient way to set this registry key from C#. There exist two attributes called ComRegisterFunctionAttribute and ComUnregisterFunctionAttribute. These attributes are used to mark which functions COM should call during the COM registration/unregistration process. So all we need to do is write a function that inserts the Programmable key during registration and removes it during unregistration.

[ComRegisterFunctionAttribute]

public static void RegisterFunction(Type type) {

Registry.ClassesRoot.CreateSubKey(GetSubKeyName(type));

}

[ComUnregisterFunctionAttribute]

public static void UnregisterFunction(Type type) {

Registry.ClassesRoot.DeleteSubKey(GetSubKeyName(type), false);

}

private static string GetSubKeyName(Type type) {

string s = @"CLSID\{" + type.GUID.ToString().ToUpper() + @"}\Programmable";

return s;

}

(Notice that the registration functions are static).

Now we've finished coding the Functions class and the UNIQUEVALUES function. We can now use from an Excel worksheet just like any other custom worksheet function.

Using UNIQUEVALUES in a Worksheet

The first to do is to install our Functions class as an automation addin. This enables Excel to call our custom UDFs.

To install as an automation addin, we bring up the usual Addin Manager via Excel Options->Add-Ins->Manage Excel Add-ins. From this dialogue we click the Automation button and select ExcelExtensions.Functions from the list.

To use the function, we highlight the cells which we want the unique values written into, type the =UNIQUEVALUES( ... ) where ... is the range from which we want to extract the values and then press ctrl-shift-enter.

Note that we must press ctrl-shift-enter to tell Excel that this function returns an array. Excel automatically serialises the values returned from the function into the array of highlighted cells. If the number of values returned from the function is less than the number of cells we highlighted, Excel places #N/A in each of the extra cells.

Consider the following example.

Notice that there are only four unique values in the range D3:D9. However, because we highlighted 7 cells (i.e., E3:E9) Excel has placed #N/A in each of the 3 additional cells.

If we highlight too few cells (say, 2 cells) then Excel will place the first to values returned from the function in those cells.

This is normal array function behaviour in Excel. Notice that we didn't have to write anything in our code to do this; Excel does this for us automatically.

If the data in D3:D9 changes, Excel fires UNIQUEVALUES and our function recalculates the new unique values.

In the following screenshot, we change the first value from Red to Yellow and the unique values automatically update:

The UNIQUEVALUES function can also be used with Excel's native functions that work with arrays. Two examples of such functions are COUNT and INDEX.

Both COUNT and INDEX can accept arrays as parameters. So we can use these functions on the array returned by UNIQUEVALUES.

The formula =COUNT(UNIQUEVALUES(D3:D9)) returns the number of values in the array. The formula =INDEX(UNIQUEVALUES(D3:D9), 1, 1) returns the first value in the array; and =INDEX(UNIQUEVALUES(D3:D9), 2, 1) returns the second; and so on ... (remember that UNIQUEVALUES returns an array one column wide).

Summary

The functions that we want to expose as custom worksheet functions need to be defined on our class's default interface;

Defining our custom worksheet functions on a dedicated interface and using ComDefaultInterface to make that interface the default is one way of doing this;

Our COM class must define the Programmable registry key under its CLSID key;
We can return multiple values from a custom UDF by returning them in an array;
Excel handles the serialisation of the array into the worksheet;

Download

I have added the UNIQUEVALUES function into my ExcelExtensions add-in. This can be downloaded from the MSDN Code Gallery by clicking here.

There are two downloads which can be downloaded from the Releases tab: ExcelExtensionsSrc.zip and ExcelExtensions.msi.

ExcelExtensionsSrc.zip contains the complete source code and ExcelExtensions.msi is the set up program.

Over time, I'll be adding additional feature to the addin.

Using Statistics in Business Intelligence

2008-04-04T19:27:00Z

One other area I work in is Business Intelligence. These days BI is an important technology to an increasing number of companies. There are many great BI tools out there, one of which is (of course) Excel.

One of the things that makes Excel such a useful BI tool is its richness of functionality. It has so many existing features that can be used in the BI area - one of which is its statistical analysis functions.

I posted an article over on the Excel team blog illustrating one way we can use Excel's statistical functions to extract useful information from business data.

If you're interested in statistical analysis or BI in Excel, you may find the article interesting. Click here to view the article.

OOXML wins ISO Vote

2008-04-01T19:49:00Z

Office Open XML (OOXML), the XML file formats used by Office 2007, today won the ISO vote which paves the way for it becoming an ISO standard.

This is very exciting news indeed for Excel programmability. Coupled with the new Open XML SDK that we announced last week, programming against Excel's native file format has never been as open. I think this opens up quite a lot of exciting programmability scenarios, some of which I'll cover on this blog over time.

More information can be found at the following links:

Open XML SDK Roadmap Announced

2008-03-20T07:06:00Z

The new XML file formats in Office 2007 open the door to lots of exciting programmability solutions. For example, the files can be created and consumed without automating Excel.exe. This really helps server-side solutions. It also aids batch processing of Excel files and cross-platform business solutions that exchange Office files.

To enable easy access to the files, we provided the .NET Packaging API. But this is a low-level API which does not have any application awareness. For example, it doesn't understand Excel ranges or tables; it deals with the raw XML elements and relations.

But we are working on improving this situation.

The Open XML SDK is a .NET library which will provide a higher-level API for the Open XML files. It will have application awareness and will greatly benefit developer productivity.

Version 1.0 will be released in May 2008. This will be an augmentation of last year's CTP. However, there won't really be anything in v1.0 for Excel developers. The Excel support will come later this year.

Here are some links which contain more information on the SDK:

Analysing Column Data using C# (... with a little Custom Drawing thrown in for fun)

2008-03-11T01:35:00Z

When presented with a table of data in Excel, sometimes it is useful to be able to learn some quick facts about it. One quick fact which is often useful is the distribution of values within a particular column. For example, if we have a column called Region it is useful to know which regions occur more often than others. Perhaps 50% of the column is North America, 30% Europe and 19% Pacific. Excel has all the tools we need to do this but we just need to tie them together in a way that suits us.

Additionally, it would be nice if we displayed this information graphically; perhaps using some form of simple histogram to compliment the numerical values and provide a tiny bit of visualisation.

In this post, we'll write a C# addin that does such a thing.

Here's a screenshot of the addin we'll build.

Getting Started

We won't cover the basics of getting up-and-running with C# Excel addins in this post. I have already covered that briefly here. Instead, we'll fast forward to the stage where we've already got our basic addin project and now we're ready to add the fun bits.

Note: This addin uses the ExcelUtility class I published previously in this post.

Designing the Custom Ribbon

We'll add a custom tab to the Ribbon called "Excel Extensions". On this tab, we'll place a button titled "Analyse Column" inside a group titled "Table Extensions".

Here's our Ribbon XML:

<?xml version="1.0" encoding="utf-8" ?>

<tabs>

<button id="ColumnAnalysis" onAction="OnColumnAnalysisClicked" label="Analyse Column" getImage="GetColumnAnalysisButtonImage" getEnabled="GetColumnAnalysisEnabled"

supertip="Analyse a table column and view statistical details about the values in the column."

size="large"/>

</group>

</tab>

</tabs>

</ribbon>

</customUI>

We'll store this in an XML file as part of our solution. Then we'll link the XML file into our assembly at compile time as an embedded resource i.e., the XML is compiled into our assembly. This means that we don't have to distribute the XML file with our addin and we use the .NET Assembly class to access the XML resource at runtime.

Here's our implemention of the IRibbonExtensibility.GetCustomUI method:

public string GetCustomUI(string RibbonID) {

Assembly assembly = Assembly.GetExecutingAssembly();

using (System.IO.Stream stream = assembly.GetManifestResourceStream ("ExcelExtensions.ExcelExtensionsRibbon.xml")) {

return new System.IO.StreamReader(stream).ReadToEnd();

}

It would be nice if our Analyse Column button was enabled only when the user has clicked inside a table. To do this, we need to interact with the Ribbon API a little more.

In our XML, we specify a getEnabled="GetColumnAnalysisEnabled" attribute on the button. This tells the Ribbon that whenever it needs to draw our button it should first call the GetColumnAnalysisEnabled method. This method will return a true/false value indicating whether the button should be displayed as enabled or not.

The Ribbon will call GetColumnAnalysisEnabled when it first displays our custom tab. But it won't call it again. So, how do we toggle the state of our button dynamically at runtime?

When our addin is first loaded, we are given a reference to the main Ribbon instance. This is an object of type IRibbonUI. One of the methods on this interface is Invalidate. When we call this method, the Ribbon invokes all of our getEnabled functions. It is our responsibility to make sure that these functions return the correct true/false value.

This seems a little weird to begin with. It is a side affect of the declarative nature of the Ribbon API; that is, we are not given pointers or references to the Ribbon controls at runtime so we need some callback approach to enable us to change the controls' appearances at runtime.

The Ribbon doesn't know anything about our application logic so it doesn't know when we want to enable/disable buttons or dynamically change the Ribbon. To avoid constantly calling into all addins, the Ribbon declares methods by which all addins can call into it and force it to redraw part or all of their custom Ribbon tabs.

Because we want to enable/disable our button based on the cell that the user selects, we want the Ribbon to redraw our button every time the sheet selection changes. When the user selects a cell, we want to force the Ribbon to invoke our GetColumnAnalysisEnabled method. In this method we will determine whether or not the user clicked inside a table and return true/false appropriately.

First, let's sink the SheetSelectionChange event and invalidate the Ribbon whenever SheetSelectionChange fires.

protected void SinkExcelEvents() {

_application.SheetSelectionChange += new Excel.AppEvents_SheetSelectionChangeEventHandler(_application_SheetSelectionChange);

}

void _application_SheetSelectionChange(object Sh, Excel.Range Target) {

_ribbonUI.Invalidate();

}

Next, we implement the GetColumnAnalysisEnabled method. This method returns true if the selected cell is inside a table and false otherwise.

public bool GetColumnAnalysisEnabled(IRibbonControl Control) {

return ColumnAnalysisExtension.EnableUI(this);

}

public static bool EnableUI(Connect Connection) {

Excel.ListObjects listObjects = ((Excel.Worksheet)Connection.ExcelApp.ActiveSheet).ListObjects;

Excel.Range activeCell = Connection.ExcelApp.ActiveCell;

foreach (Excel.ListObject listObj in listObjects) {

if (listObj.SourceType == Excel.XlListObjectSourceType.xlSrcRange) {

List<object> param = new List<object>();

Excel.Range intersection = ExcelUtility.ApplicationInvoke<Excel.Range>(Connection.ExcelApp, "Intersect", activeCell, listObj.Range);

if (intersection != null){

return true;

}

return false;

}

The GetColumnAnalysisEnabled method is defined on our Connect class (the class that implements IDTExtensibility2 and IRibbonExtensibility). But we've seperated the implementation of the column analysis extension into a different class called ColumnAnalysisExtension.

So, the Ribbon invokes Connect.GetColumnAnalysisEnabled which in turn invokes ColumnAnalysisExtension.EnableUI.

We determine whether or not the active cell is in a table by looping through all the tables in the book and calculating the intersection of each table's range with the active cell. If the intersection is non null then the active cell is in the table's range; otherwise, it's not.

So, when the user clicks inside a table our button is enabled and when they click outside a table the button is disabled.

Calculating the Distribution of Values in a Column

There's no point writing our own code to count the occurances of values in an Excel range when we can just get Excel to do it. So, we'll make use of the existing COUNTA and COUNTIF worksheet functions to perform the actual calculations.

For each value in the column we'll gather two values: the number of occurances and the percentage of occurances. We'll store this data in a Dictionary<string, List<double>> object.

Therefore, we need to write a method that does the following:

Count the number of rows in the column by evaluating =COUNTA(<column range>);
Determine all the unique values in the column;
For each unique value:

evaluate =COUNTIF(<column range>, <value>) to calculate the number of occurances of the value in the column;
Store the number of occurances and the percentage of occurances in our dictionary object;

Here's the code:

protected void CalculateStats() {

string eval = "=COUNTA(" + ExcelUtility.RangeInvoke<string>(_columnRange, "Address",

null, null, Excel.XlReferenceStyle.xlA1) + ")";

double totalCount = (double)_connection.ExcelApp.Evaluate(eval);

_stats = new Dictionary<string, List<double>>();

foreach (Excel.Range cell in _columnRange.Cells) {

try {

string val = cell.Value2.ToString();

if (!_stats.ContainsKey(val)) {

_stats.Add(val, new List<double>());

}

catch (Exception) {

}

foreach (KeyValuePair<string, List<double>> entry in _stats) {

eval = "=COUNTIF(" + ExcelUtility.RangeInvoke<string>(_columnRange, "Address",

null, null, Excel.XlReferenceStyle.xlA1) + ", \"" + entry.Key + "\")";

double count = (double)_connection.ExcelApp.Evaluate(eval);

entry.Value.Add(count);

entry.Value.Add(count / totalCount);

}

Displaying the Data using a DataGridView Control and Custom Drawn Cells

The next stage is to display the data to the user. For this, we'll write a WinForm which contains a DataGridView control. In the DataGridView we'll list each value along with the value's statistics. However, inserting rows of data into a DataGridView control isn't that interesting so we'll not discuss that here. Instead we'll concentrate on the nicer bit: using custom drawn DataGridView cells.

One of the big advantages of using .NET to write Excel addins is that we have access to all the .NET framework class library (FCL). This brings about some exciting possibilities for Excel addins, one of which is using WinForms to build good looking and modern user interfaces.

The DataGridView control has a really nice feature which makes it very easy for us to perform custom drawing for cells in the control. It allows us to define a custom column class which the DataGridView will use as if it was an inbuilt column class.

Then we define our custom cell class. This is the class which defines the cells in our custom column. The cell class extends one of the inbuilt DataGridViewXXXCell classes (such as DataGridViewTextBoxCell). All we have to do is override the Paint method of the base class and perform the custom drawing that we want to do.

Custom drawing doesn't get much easier than this!

Let's remind ourselves what we want to draw.

We want to draw a rectangle whose size represents the percentage value of the item. So, North America has a value of 50.8% so we want the rectangle to fill (horizontally) 50.8% of the cell. In other words, we want something similar to conditional formatting in Excel, but in a DataGridView control.

Notice that we also want to display the value itself as a number and we still want the cell borders and alternating background colours to be drawn.

The first step, is to define our custom column class. We'll call this DataGridViewBarChartColumn and its definition is as follows:

public class DataGridViewBarChartColumn : DataGridViewColumn {

public DataGridViewBarChartColumn() {

this.CellTemplate = new DataGridViewBarChartCell();

this.ReadOnly = true;

}

The CellTemplate property specifies the class which defines the cells that occur in the column. In this case its DataGridViewBarChartCell. This is our custom cell class and it is in this class that we do the custom drawing.

The DataGridViewBarChartCell definition is as follows.

class DataGridViewBarChartCell : DataGridViewTextBoxCell {

protected Color _barColor = Color.FromArgb(50, 0, 0, 0);

protected override void Paint(Graphics graphics,

Rectangle clipBounds,

Rectangle cellBounds,

int rowIndex,

DataGridViewElementStates elementState,

object value,

object formattedValue,

string errorText,

DataGridViewCellStyle cellStyle,

DataGridViewAdvancedBorderStyle advancedBorderStyle,

DataGridViewPaintParts paintParts) {

base.Paint(graphics,

clipBounds,

cellBounds,

rowIndex,

elementState,

value,

formattedValue,

errorText,

cellStyle,

advancedBorderStyle,

paintParts);

float val = 0.0F;

try {

val = Convert.ToSingle(value);

val /= 100.0F;

}

catch (InvalidCastException InvalidCastEx) {

return;

}

RectangleF barRect = cellBounds;

barRect.Width *= val;

using (SolidBrush brush = new SolidBrush(_barColor)) {

graphics.FillRectangle(brush, barRect);

}

DataGridViewBarChartCell is a subclass of the inbuilt DataGridViewTextBoxCell class. Extending the inbuilt class means that all we have to do is override the paint behaviour of the DataGridViewTextBoxCell class and let the DataGridViewTextBoxCell class handle everything else.

In the Paint method you can see that we are automatically provided with a Graphics object and a bounding rectangle in which to paint. All we need to do is calculate the width of the rectangle and then call Graphics.FillRectangle.

Finally, notice that the first thing we do in the Paint method is call base.Paint. This means we don't have to care about drawing the cell's value or the background or the borders because the base class will do that for us. All we do is then draw on top of what the base class draws. If we didn't call base.Paint then the cells would only contain the filled rectangle and nothing else.

Hooking it all Together

To make the DataGridView control use our custom column class, we set the type of the appropriate column in the DataGridView control to DataGridViewBarChartColumn.

The final stage is to display our window when the user clicks the button in the Ribbon. Looking back at our Ribbon XML we defined the onAction callback for the button as onAction="OnColumnAnalysisClicked". This means that the Ribbon will call the OnColumnAnalysisClicked method on our addin when the user clicks the button, so, all we need to do is define the OnColumnAnalysisClicked method and make it public.

public void OnColumnAnalysisClicked(IRibbonControl Control) {

ColumnAnalysisExtension ext = new ColumnAnalysisExtension(this);

}

In this method we are creating a new ColumnAnalysisExtension object. The constructor of this class simply calls our CalculateStats method and then displays the window containing the DataGridView control (it does one or two other things too).

public ColumnAnalysisExtension(Connect Connection) {

_connection = Connection;

BuildColumnRange();

if (_columnRange != null) {

CalculateStats();

DisplayStatsWindow();

}

Summary

Use the IRibbonUI.Invalidate method to force the Ribbon to redraw your custom Ribbon controls when you need their state to change;
To avoid having to ship your Ribbon XML, compile it as an embedded resource into your assembly;
Using .NET to write user interfaces offers exciting UI possibilities, such as easily-to-code custom drawn controls;
To custom draw cells in the DataGridView control, subclass the DataGridViewColumn class and one of the inbuilt DataGridViewXXXCell classes;
The code for the custom drawing should override the Paint method of the custom cell's base class;

Download the Excel Extensions Addin

The Excel Extensions addin and full source code can be downloaded from the MSDN Code Gallery here.

There are two downloads which can be downloaded from the Releases tab. There are: ExcelExtensionsSrc.zip and ExcelExtensions.msi.

ExcelExtensionsSrc.zip contains the complete source code and ExcelExtensions.msi is the set up program.

Over time, I'll be adding additional feature to the addin.

Using Custom Functions in Dynamic Ranges

2008-02-19T20:33:00Z

Named ranges are great. They enable us to assign a name to a collection of cells and use that name in code and worksheet functions. However, named ranges tend to be static. That is, they tend to represent a fixed set of cells, such as, A1:E100. But sometimes it would be useful to have the range's set of cells be dynamic. Instead of the range representing a fixed set of cells, we would like to dynamically determine which cells are included in the range based on some custom logic.

Excel already lets us do this. It allows us to embed functions inside the definition of a named range and have the functions determine on-the-fly which cells are included in the range. But it's not just native Excel functions (like IF, OFFSET, etc.) that we can embed into a named range defintion. We can also embed our own, custom functions; we can write a custom function that uses our own application logic to dynamically determine at calculation time which cells are in a named range.

This gives us the capability to point the named range at different cells without actually changing the named range's definition.

In this posting, I'll talk about dynamic ranges and embedding custom functions into named range defintions.

Getting Started with Dynamic Ranges

Let's take a simple example. Let's say that we have a column of numbers in the range 'Example 1'!$C$1:$C$100 and we want to find the cells in this range that are higher than some value, X = 34,999.

Using dynamic ranges, we can create a named range (called, say, Range 1) with a definition of:

=GreaterThan('Example 1'!$C$1:$C$100, 34999)

We also define a UDF called GreaterThan as:

Public Function GreaterThan(Rng As Range, Limit As Long) As Range
    Dim Cell As Range
    Dim ResultRange As Range

    For Each Cell In Rng
        If Cell.Value2 > Limit Then
            If ResultRange Is Nothing Then
                Set ResultRange = Cell
            Else
                Set ResultRange = Application.Union(ResultRange, Cell)
            End If
        End If
    Next
    Set GreaterThan = ResultRange
End Function

The UDF iterates over the source range testing each cell. If the cell meets our criteria (in this case its value being greater than 34999) then it is added into a temporary range object. Once the code has iterated over all the cells, the temporary range is returned to the caller. This range represents the set of cells the named range now refers to.

(Note: we're not coping with any runtime errors in this code ... which we should do in a proper version).

We can use the named range just like any other named range and Excel will call our UDF as and when it needs to. So, we could calculate the average of our range by using normal worksheet functions, such as:

=AVERAGE(Range1)

We can visually confirm the range that our UDF returned using the Name Manager. If we open up the Name Manager and select Range 1 and then click inside the 'Refers to:' textbox, Excel invokes our UDF and highlights the cells on the worksheet that our UDF returned.

Recalculating the Range when Data Changes

Excel knows that our dynamic range depends on $C$1:$C$100. This is because we pass $C$1:$C$100 to the UDF call in the named range definition.

This enables Excel to properly calculate our named range during worksheet calculations.

In our example, the cell $E$3 (which contains the formula =AVERAGE(Range1)) depends on Range 1 and Range 1 depends on $C$1:$C$100. Therefore, if $C$1:$C$100 changes, Excel knows that it needs to recalculate Range 1 in order to recalculate $E$3.

So, if we change the value of $C$1 from 40000 to 20000 Excel knows that it needs to recalculate $E$3 and, hence, invokes our UDF. Our UDF determines which cells Range 1 refers to and deems that $C$1 is no longer part of Range 1 (because its value is no longer > 34999); thus the value of $E$3 changes, as shown below.

We can also confirm that Range1 is comprised of different cells by inspecting Range1's address in the VBA Immediate window (this also illustrates that we can use the dynamic named range in VBA just like we can use a 'normal' named range).

A Note about Interacting with the Calc Tree

If our UDF didn't declare its source data range as a parameter then Excel would not be able to call it at the appropriate times during recalculation.

For example, if our UDF simply iterated through $C$1:$C$100 without declaring this range as a parameter then, when $C$1:$C$100 is changed, $E$3 wouldn't be recalculated.

This is because Excel would not know that $E$3 depends on $C$1:$C$100 because that dependency would be hidden inside the UDF.

There may circumstances when we don't care about this. But it is worth remembering that Excel needs to know about any worksheet dependencies our UDF has in order for our dynamic range to interact with the calculation tree properly.

Making More Use of Dynamic Ranges

Our GreaterThan UDF is about as basic as could be, but, it illustrates the technique. It doesn't, however, take much imagination to see how this technique could be used in more sophisticated ways.

For example, inside our UDF we could make use of external data, such as our company's databases, and use the logic and/or data stored in them to determine the contents of the dynamic range in Excel.

One example could be that we have a list of dates in our worksheet and we want to highlight the dates on which somebody downloaded a trial version of our product from our web site. This would make use of the same technique described here, except, in the loop in our UDF, we would make a call to the database to determine whether that cell should or should not be included in the range.

In other words, using UDFs in dynamic ranges enables us to incorporate external data and external components into our range definitions.

Another use is: search. We could write a fairly sophisticated custom, search feature that is built on top of dynamic ranges and UDFs.

Concerns About Execution Speed

I'm afraid I've left the bad news until the end...

Implementing dynamic ranges exactly as I have described here could hand you serious performance issues.

To illustrate this, let's change the definition of Range 1 from:

=GreaterThan('Example 1'!$C$1:$C$100, 34999)

to:

=GreaterThan('Example 1'!$C:$C, 34999).

Now, we are checking for cells greater than 34999 in the entire C column. In Excel 2007, that's 1,048,576 cells. On my PC (which is a 2.66 GHz dual core with 3GB RAM - not that dual core helps here anyway), calculating the range once takes 3 seconds. If we referenced Range 1 in only a handful of places in our workbook or VBA, our workbook would become fairly unusable fairly quickly.

It would be interesting to see if implementing our UDF in an XLL would improve the speed substantially - (something I'll look into).

Also, if our UDF did fancier things (like calling into an external database) then you can just imagine what would happen when we're iterating over 1,048,576 cells ....

So, it's worth stating that we should exercise some common sense and some judgement when using dynamic ranges. However, if we restrict our usage to suitably sized source ranges, dynamic ranges are a useful tool indeed (where suitably sized is something we each have to define for ourselves).

Summary

Named range definitions can contain native and custom functions;
A custom function used as a named range definition should return the range that it determines the named range represents;
To interact properly with Excel's calculation tree, all worksheet dependencies (such as the source range) should be included as parameters to the custom function;
Dynamic ranges can be used to build some fairly useful and sophisticated extensions;
Speed when iterating over large ranges is not good and we should employ careful judgment when using dynamic ranges on potentially large source ranges;

You can download the workbook I used for this post in the list of attachments below.

Ribbon Customisation Links

2008-02-14T23:00:00Z

I thought it would be useful to provide a list of some good articles on customising the Office ribbon:

Excel Event Handling in C#

2008-02-13T03:03:00Z

After I posted about getting up-and-running with managed code and Excel, I realised that I was really writing a brief introduction to coding a C# Excel addin. Then I realised that there are two more areas that need to be covered before we would be ready to start building a proper Excel addin in C# (and by proper I mean one that can be used by real Excel users to do real stuff). These are:

Customising the ribbon;
Trapping Excel events;

The ribbon is an Office-wide feature and has been covered in detail in other blogs and MSDN articles, so we won't cover it in detail here. A good article about the ribbon can be found here.

Instead, we'll focus on event handling in C# and how it compares to VBA.

Excel Event Overview

Excel events are COM events. They are defined in COM interfaces exported from Excel.exe. When we trap these events from VBA we're not really aware of the underlying mechanisms that enable Excel to callback into our code. The sinking and dispatching just happens automagically for us..

There are two broad parts of event handling: the event source; and the event handler. In the case of Excel, the event source is an instance of the Excel class that defines the event, such as a Workbook instance or a Worksheet instance. The event handler (or subscriber as some people call it) is the custom code (i.e., our code) that is invoked by the event source when the event happens.

The event source publishes a defined event signature which all event handlers must adhere to. As long as a handler adheres to this signature, the event source can invoke it when required.

Let's take an example: the Workbook.Open event.

When we use the VBA object browser to look at the definition of the Workbook class we can see that the event is called Open() and has a special lightning bolt symbol next to it indicating that it is an event.

What the object browser did was look at the COM type library metadata for the Workbook class. As well as containing all the methods and properties of the Workbook class the type library also contains the published event signatures. The event signatures have special attributes applied to them. To see these, we use the OleView.exe tool to generate IDL from Excel.exe (note that this is a part of COM generally well hidden from Visual Basic. If you code COM components in C++ then this is should be very familiar to you).

Notice that the Workbook class has two interfaces: _Workbook and WorkbookEvents. Notice also that the WorkbookEvents interface has an attribute named: source. This tells COM that the WorkbookEvents interface is an event source for the Workbook class.

COM has its way of handling events and .NET has its and, through the magic of COM interop, we can tie these two disparate programming models together. This means that we can have a COM event source and a .NET event handler.

However, the code we need to write to handle events in C# is very different from what we are used to writing in VBA. When we are coding in VBA, most of the event mechanism and underlying framework is hidden from us ... but not so in C#.

Delegates, Events and Excel

To C#, a COM event looks very much like a .NET event. This greatly simplifies things for .NET developers. However, if you are a VBA Excel developer moving to C#, the event mechanism in .NET seems complex and doing something simple, like handling the Workbook.Open event, can look like a mystery.

The first step to demystifying this is to understand what a delegate is.

A delegate is a special type of .NET class used in the .NET event mechanism. Think of them as being wrappers of function pointers. In COM an event is defined by its function signature whereas, in .NET, it is defined by a delegate but they both play the same role: they both define the contract between the event source and the event handler.

Let's take an example.

Let's say we are defining an event in our C# class called OnClick. We have to define the signature of the event. In C# this means defining a delegate and declaring the event to be of that (delegate) type.

Here's an example delegate and event definition:

public delegate void OnClickDelegate(string Title);

public event OnClickDelegate OnClick;

If we took out the delegate keyword, OnClickDelegate would look just like a function definition. Having the keyword delegate there wraps the function definition up into a type safe class. We then declare our OnClick event as being of type OnClickDelegate. This is us saying: we want an event called OnClick that has the signature defined by the OnClickDelegate thus, all event handlers have to have the same signature as OnClickDelegate.

At first, this seems overly complex ... but it's not really. It just takes a little getting used to (especially if you are from a VB background). If you ever coded in C/C++ and were used to passing around function pointers, this seems a wonderful step forward. Actually, it's one of my favourite parts of .NET :o)

So, what's this got to do with Excel programming?

Well, if you want to be able to handle Excel events in C# you have to understand delegates.

Handling Excel Events

So, let's take a look at what the C# code for handling Excel events looks like. For example, how do we handle the SheetSelectionChange event of the Application class?

_application.SheetSelectionChange += new Excel.AppEvents_SheetSelectionChangeEventHandler(_application_SheetSelectionChange);

void _application_SheetSelectionChange(object Sh, Excel.Range Target) {

//event handler code here

}

The first line creates a delegate of type Excel.AppEvents_SheetSelectionChangeEventHandler and adds to to the application object's SheetSelectionChange event. In the constructor of the delegate we pass the address of the function that the delegate encapsulates, in this case it is the _application_SheetSelectionChange function but it could be any function that has the same signature as the AppEvents_SheetSelectionChangeEventHandler delegate.

This is a subtle but important point, especially if you are used to coding VBA. In VBA we cannot specify the name of our event handler functions because the name indicates which event is being handled on which object. But in C# the event handler name is not important; what is important is the signature of the function. So, although our event handler is called _application_SheetSelectionChange which looks exactly like the name we'd use in VBA, we are free to change its name to anything we like as long as it still returns void and takes an object and Excel.Range as parameters.

So, to summarise: every time we want to handle an Excel event, we have to create an instance of the corresponding delegate and supply the address of a compatible function which the delegate encapsulates. This function is invoked by Excel when the event occurs.

But how do we know which delegate to create? In the example above, we created an instance of AppEvents_SheetSelectionChangeEventHandler but how did we know that and where do we go to find out the delegate types?

Excel's Delegate Names and COM Event Sources

When it comes to knowing which delegate type to use, Visual Studio comes to our rescue. Visual Studio detects that we are assigning a new delegate to an event and offers to auto-complete; generating both the delegate creation code and also an event handler function which complies to the delegate's signature.

So, consuming events is typically a case of typing <object name>.<event name> += <press tab twice to accept auto complete's delegate creation and function generation>.

But, its worth taking a deeper look at Excel's delegate names as it sheds some light on the underlying COM event interfaces.

If we look at the Application class's RCW (the ApplicationClass) in the Visual Studio Object Browser we see that it implements a number of interfaces.

Notice that one of the interfaces is called AppEvents_Event. Remember that these interfaces are proxies/wrappers of COM interfaces, so, let's now look at the COM definition of the Application CoClass using OleView.exe.

The Application CoClass implements the AppEvents interface. Also, the IDL tells us the the AppEvents COM interface is an event source interface for the Application CoClass. So, it turns out that the events for the Application class are actually all defined in one COM interface: the AppEvents interface. This is also something that is hidden from us when we are coding in VBA.

If we take a look at the definition of the COM AppEvents interface and the .NET AppEvents_Event interface we will see that they each define the Application events in their respective ways.

So, the AppEvents_Event interface in the PIA wraps up the AppEvents COM interface; the ApplicationClass in the PIA implements the AppEvent_Event interface and the Application CoClass in COM implements the AppEvents COM interface.

Given this we can now understand how the delegate names are generated, thus:

<COM Interface Name>_<EventName>EventHandler

i.e. AppEvents_SheetSelectionChangeEventHandler

Summary

Hopefully, this has laid out in clear terms out how we handle Excel events in C#. I have stayed away from talking about the specifics of how COM events actually work but this is a subject covered in detail on MSDN. What's more important to us, from an Excel point of view, is understanding the Excel event interfaces and how to use them from C#.

The key points to take away are:

Excel events are COM events;
Delegates are strongly typed wrappers of function pointers;
.NET events use delegates to invoke event handling functions;
The name of the event handling function is not important - it is the signature of the function that is important;
Excel's COM classes have COM event interfaces and these map to the .NET RCWs in the Excel PIA;

The Woes of Excel's Optional Parameters and C# ... a (potential) Workaround

2008-02-06T22:55:00Z

When coding in C# with Excel, it doesn't take long before you encounter the dreaded 'optional parameter' issue. If you have ever copied-and-pasted numerous System.Reflection.Missing.Value statements then you've probably wondered a few things.

Firstly, what is this and why is it happening? Secondly, how do I get round it?

In this post, I'll offer answers to those questions.

What's the Problem with Optional Parameters?

As you know, some of Excel's APIs (such as Application.Intersect or Range.Sort) use optional parameters . When you are coding in VBA you hardly notice this but it's a whole different story in C#.

As I briefly covered in my introduction to managed programming and Excel, when you use a managed language to code against Excel, Excel's API is exposed as a collection of Runtime Callable Wrappers (RCWs). These are managed classes that have the same methods and properties as the COM classes they wrap. When you call Application.Intersect from C# you are really calling Application.Intersect on the RCW. The RCW then internally marshals the call to the real, COM Application.Intersect.

In other words, the RCWs expose managed wrappers of the COM methods.

However, the methods and properties that have optional parameters in COM are not optional in C#. As a C# programmer, you have to supply a value for every single parameter, regardless of whether or not it is optional in COM.

At first glance, this seems like not such a big deal. That is, however, until you realise that some methods (like Application.Intersect) take 30 parameters; 28 of which are optional!

Suddenly, having to specify a value for every single parameter becomes tedious and messy and litters your code with statements such as:

_application.Intersect(

rng1,

rng2,

System.Reflection.Missing.Value,

System.Reflection.Missing.Value);

Is this a .NET Problem?

No. The CLR supports optional parameters. This is a C# problem (Incidentally, VB.NET doesn't have this issue).

To underline this point, let's leave C# for a moment and jump into what managed code is compiled to: MSIL. We'll disassemble Excel's RCWs and take a look at the MSIL they contain. This gives us a language agnostic view of what the RCW is really defined as.

The following screenshot shows the MSIL of the Application.Intersect method.

What to notice here is that the first two parameters are of type Microsoft.Office.Interop.Excel.Range but that the rest are of type object and they are also decorated with the [opt] attribute.

When you call this method from C# you are forced to specify a value for every parameter because C# (not .NET) doesn't support optional parameters.

It's worth noting that this is also not an Excel problem. Any COM library that uses optional parameters will have the same problem when accessed from C#.

So, what can we do about this?

The obvious answer is: 'don't use C# use VB.NET instead' but this isn't really an option for those programmers who (for other reasons) may want to use C#.

Therefore, is there anything we can do in C# to make the story a little better?

Working Around the Problem

The most common workaround I know of is to create wrapper classes of the RCWs. For example, we could write a wrapper of the Microsoft.Office.Interop.Excel.ApplicationClass class with more friendly C# function signatures. Internally, our wrapper would still have to make the call to the RCW with a value for all parameters but our wrapper would expose a nicer programming interface which we would use throughout our program. This means that the messy call to the RCW happens in just one place in our code rather than in multiple.

The big problem with this workaround is that the Application class has quite a lot of methods and properties and duplicating them all is time consuming. Also, we'd have to do the same thing for the Range class and the Workbook class and every other (Excel) class that we use.

We could, instead, just wrap up the functions that we care about. So, instead of wrapping every function we only wrap the the Excel functions we use in our program. This is a much smaller task and the is the route most people (I think) have taken. But it's still not ideal and it's not very generic.

Therefore, is there another option?

Using Reflection to Dynamically Invoke Functions at Runtime

.NET assemblies have lots of metadata in them which can be accessed at runtime via a technology known as Reflection. Reflection, in .NET, is hugely powerful and is a massive subject in its own right. But one of the things it enables us to do is to invoke methods by name at runtime.

When our code is compiled, the compiler ensures that we have used the correct function signatures and types. If we haven't, the compiler emits an error. Reflection in .NET allows us to discover function signatures and parameter types at runtime. We can then go ahead and invoke these functions 'adhoc' at runtime; bypassing the type checking and validation normally done by the compiler. If we use the wrong function signature or the wrong data type we'll get a runtime error rather than a compile-time error.

Let's take our example of calling Application.Intersect. The C# compiler requires us to specify a value for all 30 parameters. If we don't do this, the compilation fails. There's no way round this ... unless we don't actually tell the compiler that we are invoking the Intersect method.

In the remainder of this post, I'll introduce a C# class I've written that uses .NET reflection. The class invokes methods and properties of Excel RCWs dynamically at runtime, thus, hiding from the compiler what we are really doing and removing the need for us to specify a value for all optional parameters.

The ExcelUtility Class

This class is very small and is mostly defined in about 30 lines of code. However, in these few lines I make use of reflection and generics and my explanation assumes you have some basic knowledge of these two topics.

The important method in the ExcelUtility class is the Invoke method. This method contains the code which discovers function signatures at runtime and handles the invocation of them.

public static TRet Invoke<TRet>(object Obj, Type ObjType, string MethodName, params object[] CallerParams) where TRet : class

The MethodName parameter is, well, the name of the method we want to call on the RCW. For example, if we want to call Application.Intersect then we set MethodName to the string "Intersect".

ObjType is the Type object of the class which we are invoking. The framework contains a class called Type which represents the metadata of data types. Every class has a Type which can be accessed at runtime. For the Excel Application class (well, the RCW) we get its Type object using typeof(Microsoft.Office.Interop.Excel.ApplicationClass).

Obj is the actual object which we want to invoke i.e. the Application instance.

Finally, the CallerParams array contains the parameters that the caller wants to pass into the function call. So, in the case of calling Application.Intersect, we would supply the range objects we want to intersect.

Notice that Invoke is a generic method. The generic parameter, TRet, specifies the type of the object to return. Invoke will attempt to cast the result of the dynamic function call to this type. So, if the method we are invoking returns a string then we specify string as the generic parameter i.e. string val = Invoke<string>( ... )

Let's now look at the method's code.

public static TRet Invoke<TRet>(object Obj, Type ObjType, string MethodName, params object[] CallerParams) where TRet : class{

MemberInfo[] memberInfo = ObjType.GetMember(MethodName, BindingFlags.Public | BindingFlags.IgnoreCase | BindingFlags.Instance);

BindingFlags memberTypeFlag;

int paramCount = 0;

switch (memberInfo[0].MemberType) {

default:

return null;

case MemberTypes.Method:

MethodInfo methodInfo = ObjType.GetMethod(MethodName, BindingFlags.Public | BindingFlags.IgnoreCase | BindingFlags.Instance);

paramCount = methodInfo.GetParameters().Length;

memberTypeFlag = BindingFlags.InvokeMethod;

break;

case MemberTypes.Property:

PropertyInfo propInfo = ObjType.GetProperty(MethodName, BindingFlags.Public | BindingFlags.IgnoreCase | BindingFlags.Instance);

paramCount = propInfo.GetIndexParameters().Length;

memberTypeFlag = BindingFlags.GetProperty;

break;

}

object[] paramArray = new object[paramCount];

for (int i = 0; i < paramCount; ++i) {

paramArray[i] = Missing.Value;

}

if (CallerParams != null)

CallerParams.CopyTo(paramArray, 0);

TRet ret = Obj.GetType().InvokeMember(MethodName, memberTypeFlag | BindingFlags.Instance | BindingFlags.Public, null, Obj, paramArray) as TRet;

return ret;

}

The basic idea is this. First, get the metadata for the member specified in MethodName and then store how many parameters the member takes. We also use a bitmap (memberTypeFlag) to store whether the member is a method or a property (this is used later).

Next, we create an array of objects. the size of the array is equal to the number of parameters the method/property needs. We also initialise every object in that array to Missing.Value. So if the method takes 30 parameters paramArray will have 30 objects in it, each set to Missing.Value.

We then copy the parameters in CallerParams into paramArray . So, if CallerParams had two objects in it, paramArray would have 30 objects with the first two being the same as those specified in CallerParams (i.e. the parameters we care about) and the remaining 28 being set to Missing.Value.

Finally, we invoke the method/property passing in paramArray as its parameters and casting the result to an object of type TRet and returning it to the caller.

So to summarise, Invoke takes an object reference and a name of a method. It then figures out how many parameters the method expects and ensures that it supplies any missing parameters not specified by the caller. That is, the caller only needs to supply the parameters it cares about; Invoke will fill-in-the-blanks.

Using ExcelUtility

The Invoke method will call methods on any class. But using it directly means that we have to constantly specify which class we want to use. So, to make our lives a little simpler, I have added wrapper functions that can be used with the Application and Range classes.

These functions are all one-liners and simply reduce the number of parameters we need to pass when using the ExcelUtility in our code.

Here's the wrapper function for invoking the Application class (the Range function wrapper can be obviously deduced).

public static TRet ApplicationInvoke<TRet>(object Obj, string MethodName, params object[] Params) where TRet : class {

return Invoke<TRet>(Obj, typeof(Excel.ApplicationClass), MethodName, Params);

}

So, let's take a look at how the ExcelUtility class changes the landscape.

Using the RCWs directly, to invoke Application.Intersect for two ranges (rng1 and rng2) we have to write:

Excel.Range rng = _application.Intersect(

rng1,

rng2,

System.Reflection.Missing.Value,

System.Reflection.Missing.Value);

With the ExcelUtility class, this changes to:

Excel.Range rng = ExcelUtility.ApplicationInvoke<Excel.Range>(_application, "Intersect", rng1, rng2);

(where _application is our instance of Excel's application object).

Quite a difference...

Great ... but not so great ...

On the surface, it looks like a nice solution. But there are downsides to this workaround that become apparent once you start using it for real.

First of all ... things start looking not so nice again when you invoke methods/properties with a non-optional parameter in the middle of a list of optional ones. For example, the Range.Address property has 5 parameters. The first two are optional, as are the last two, but the third parameter is non-optional.

Directly invoking the RCW to get Range.Address means writing:

string addr = _columnRange.get_Address(

System.Reflection.Missing.Value,

Microsoft.Office.Interop.Excel.XlReferenceStyle.xlA1,

System.Reflection.Missing.Value,

System.Reflection.Missing.Value

);

and using ExcelUtility we do:

addr = ExcelUtility.RangeInvoke<string>(_columnRange, "Address", null, null, Microsoft.Office.Interop.Excel.XlReferenceStyle.xlA1);

I'd say that wasn't much of an improvement. It might even be worse.

(Note that we specify the property name "Address" in the call to RangeInvoke where as the RCW uses a get_Address method. This is because C# does not support properties with parameters - but that's another topic).

Second problem ... we only find out about coding errors at runtime. Because we are hiding the call to the RCW from the compiler we won't get any compilation errors if we use the wrong data types or incorrectly spell a function name. If, for example, we try to invoke the "Addres s" property - our code will compile. It will only be at runtime that we get an error (and even then the error message might not be that useful in figuring out our mistake). This means that we need to already know (or have looked up) the signatures of all the Excel APIs we are calling.

Third problem ... when the method we are calling on the RCW does not have optional parameters, calling the RCW directly is nicer. So, ExcelUtility isn't always a good replacement for calling the RCW directly. It's more like something that should be used when there are optional parameters.

Summary

What I've presented here is an idea for how we can workaround the problem of optional parameters in Excel's API when coding in C#. I should point out that it's just my idea and isn't a Microsoft recommended solution. Also, I have not exhaustively tested this code. By that I mean I haven't tested whether this works with every single method and property in the Excel API so there may be (and likely are) cases where this fails.

These are the key points:

ExcelUtility uses reflection to discover function signatures at runtime;
The Invoke method fills-in-the-blanks; we only have to specify the parameters we care about (most of the time);
Because we don't tell the compiler which methods we are invoking, we only get told of errors at runtime;
To call a method or a property on the Application class, use the ApplicationInvoke method;
To call a method or a property on the Range class, use the RangeInvoke method;
The main method is Invoke which can be used (in theory) to call methods on any class;

Download the Source Code

I have published ExcelUtility.cs to the MSDN Code Gallery and it can be downloaded by clicking on the link below (go to the Releases tab to download).

Click here to download

Creating Custom Calculated Members and Sets in OLAP PivotTables

2008-02-06T22:45:00Z

If you are using PivotTables against an OLAP data source you may find it useful to be able to create your own custom, calculated measures and sets. These calculations are client-side meaning they exist inside Excel and you don't have to change the server-side OLAP cube.

Excel does not enable you to do this using the user interface, however, you can do this via the API.

Allan Folting (our PivotTable expert) has published a post showing how to do this using VBA over in the team's blog.

Click here to read his post.

MSDN Code Gallery

2008-02-05T23:02:00Z

We have recently launched a code sharing community web site called MSDN Code Gallery.

MSDN Code Gallery hosts code snippets and entire projects that have been shared with the community. You can publish your own code or search for and download other peoples' code.

There are lots of nice features on the web site. For example, when you publish code, the site enables issue tracking, discussions, comments and RSS feeds pertaining to your release.

The basic format is that you create resource pages. These pages each have descriptions, versioning etc. and contain releases. Each release is a file. It can be a code snippet or documentation or any other useful file. You search the gallery using tags or using the embedded Live search. Also, the web site automatically enforces the Microsoft Public License on all the source code downloads.

All in all, it seems like a great idea, so, I'll be publishing all my source code to the MSDN Code Gallery.

If you're interested in publishing your own code, you can find out how to do it here.

As an example, the resource page for my post on filtering a PivotTable using cell contents has been published here. The workbook can be downloaded from the Releases tab.