Welcome to MSDN Blogs Sign in | Join | Help

Excel 2007 investments in UDFs #1

For the next few posts, we have a “guest post” from Danny Khen, a program manager on the Excel Services team.  Danny is going to talk about UDFs in Excel and Excel Services.  Enjoy.

UDFs are user-defined worksheet functions – custom functions that you create to supplement Excel’s set of intrinsic worksheet functions. UDFs are used to create calculation libraries, or to import data into Excel sheets in custom ways.

In Excel 2007, we’ve made a number of key investments around UDFs. They revolve around two main areas: allowing UDFs to take advantage of important Excel improvements, and extending UDF-based Excel solutions to the server side with Excel Services.

Updated XLLs

A while back Dave mentioned that we updated XLLs (Excel’s addins based on the C-API) to give developers access to new Excel functionality. As that post explained, one of the common things that developers do in XLLs is to implement UDFs. We wanted to make sure that XLL authors can make use of some great new features of Excel 2007 itself in the UDFs they create. To recap, XLLs have support in Excel 2007 for:

  • The bigger grid
  • More function arguments
  • Multi-threaded calculation

There are many more detailed about those improvements in that other post.

Server-side UDFs

Dave has also posted a number of entries about Excel Services – the new feature in the Office SharePoint Server 2007 that enables calculation, display, and exploration of Excel workbooks on the server. These posts cover many aspects of Excel Services.

Much like Excel’s ability to be extended by writing UDFs in Excel addins, Excel Services also has an extensibility mechanism for writing UDFs. I’d like to use the rest of this post to explain and demonstrate Excel Services UDFs. In a follow-up couple of posts, I will show how you can create Excel solutions that use UDFs and can run both on a client machine using Excel 2007 and in a server environment using Excel Services.

They’re managed

Server-side UDFs are implemented as methods .NET 2.0 assemblies. That is to say, Excel Services directly supports only managed code UDFs. Existing native function libraries and Excel UDFs can be used with Excel Services by “wrapping” them with the new style of server managed UDFs; I will show how in the follow-up posts.

But why did we actually “go managed”? Excellent question. Managed code for enterprise-level solutions is becoming more and more popular, because of the many advantages that .NET code has to offer. Robustness and security are among the important advantages. Some of you may already be engaged in developing managed UDFs or other types of managed solutions. With Excel Services, we focused specifically on server stability, and we felt that using .NET as the basis for our extensibility would be the right thing to do in this respect.

They’re part of a V1 feature

In Office SharePoint Server 2007 we introduce the first implementation of Excel Services and of its extensibility mechanism. To start with, we simply had to prioritize the support for various features in Excel Services (even regardless of UDFs), and not everything made it for this first version. Moreover, we know that going forward we will be doing a lot of thinking around managed interfaces and extensibility for Excel – both on the client side and on the server. We want to make sure that customers' initial investment in managed UDFs is secured, and that at the same time we are not constrained in any way when we design the best possible infrastructure in the future. For those reasons, there are certain restrictions with the first version of server UDFs:

  • No Excel OM: the entire interface with the Excel sheet is done thru the UDF call signature; arguments are passed into the function from the Excel formula, and return values are passed back into the formula.
  • More restrictive than Excel addins with
    • Type conversion and supported data types.
    • Error handling: all exceptions thrown by the UDF code are returned into the Excel sheet as #VALUE errors.
  • Simple load / runtime behavior: for example, all UDFs currently run together with Excel Services code in the same .NET application domain.

We believe that lots of useful solutions can be created under these restrictions; in fact we see many existing Excel UDF-based solutions that could comply.

They need to be thread-safe

Excel Services is a server feature, and its calculation engine runs as a multi-threaded backend service. We have to require (and assume) that all UDFs are thread safe.

Security

A couple of things to mention about the way a server administrator can control the security of Excel Services UDFs.

First, Excel Services will not load and run just any old method in any old .NET assembly. An admin needs to register the assembly on a server list of trusted UDF assemblies.

On top of that, since these are .NET assemblies, an admin can make use of .NET’s CAS (Code Access Security) infrastructure to restrict UDF access to resources. For example, if a UDF package is simply a collection of math calculation functions, an admin can turn off its ability to access web services, external data, native code etc – and make the server environment safer for everyone.

So what does this thing look like?

Very simple, actually. All you need to know about (assuming you speak some .NET dialect) is two new attributes. Both of them are defined in the Microsoft.Office.Excel.Server.Udf namespace, and you need to reference an assembly that is shipped with Excel Services (Microsoft.Office.Excel.Server.Udf.dll) to get them.

The two attributes are UdfClass, which you use to mark a class where server UDFs are defined, and UdfMethod, with which you mark each individual public method to be considered a server UDF. The UdfMethod also has a boolean property – IsVolatile – used to declare the UDF as volatile, if you want the server to call it each time it recalculates the workbook, regardless of any change in its dependencies. The default is false, which means that the method is non-volatile; it gets called only when a value changes somewhere down the dependency chain of the formula that calls the UDF.

That’s really it. A typical UDF class will look something like this:

using Microsoft.Office.Excel.Server.Udf;

namespace YourNamespace
{
    [UdfClass]

    public class YourClass
    {
       
[UdfMethod]
        public <return-type> NonVolatileMethod(<arguments>)
        {
  ...
        }
        [UdfMethod(IsVolatile=true)]
        public <return-type> VolatileMethod(<arguments>)
        {
  ...
        }
    }
}

Show us the goods!

Well, alright… Attached to this blog post is a fully developed code sample. It shows a UDF package by the name of WishExcelHad (because it implements a couple of functions that we wish Excel had…). The package has two method to manipulate text strings with words. The first method, WehWordM, takes the string, a required word position, and a delimiter string, and returns the word in that position after parsing the text with the given delimiter. The other method, WehWordcountM, takes a string and a delimiter, and returns the number of words in that string, when it is parsed with the given delimiter.

You can use the attached Excel workbook as an example that calls these two methods in formulas. You’ll need to save it to Excel Services as an XLSX file.

Why the weird “M” as the suffix for those method names? It designates “Managed” – these UDFs are implemented purely with managed code, and are intended to run with Excel Services. Stay tuned to the blog and look for the following posts, where I will show versions of the same UDFs, that can run across Excel 2007 and Excel Services.

Published Wednesday, May 03, 2006 6:14 AM by David Gainer

Comments

# re: Excel 2007 investments in UDFs #1

Wednesday, May 03, 2006 12:31 PM by Andrew Walaszek
Does an Excel Undo mechanism exist for the implementation of managed UDFs? This also applies for any operations performed by a third-party Add-In.

Regards,
Andrew

# re: Excel 2007 investments in UDFs #1

Wednesday, May 03, 2006 6:23 PM by Harlan Grove
A question I didn't ask in the original blog post on XLLs and UDFs: did you increase the limit on string argument and string return value length to/from XLL UDFs? One of the most frequent hassles with XLL UDFs for string processing is the 255 char limit.

# re: Excel 2007 investments in UDFs #1

Wednesday, May 03, 2006 6:58 PM by Harlan Grove
With regard to the wonderful sample udfs, these have been dealt with ad nauseum in the Excel newsgroups.

WehWordM: return n_th word in s delimited by d
=MID(s,FIND(CHAR(127),SUBSTITUTE(d&n,
d,CHAR(127),4)),FIND(CHAR(127),
SUBSTITUTE(s&d,d,CHAR(127),4))
-FIND(CHAR(127),SUBSTITUTE(d&s,
d,CHAR(127),4)))

WehWordCountM: return # words in s delimited by d
=1+(LEN(s)-LEN(SUBSTITUTE(s,d,"")))/LEN(d)

Not good examples of "functions that we wish Excel had."

Any chance you could show examples of udfs that *CAN'T* be performed by built-in functions alone (nontrivial udfs)? Maybe something that only needs the arguments, like a generalized string concatenation udf that would concatenate all items in all its arguments into a single string? And don't let noncommutivity stop you: Excel's NPV function seems to be able to iterate through ranges and arrays in a specified order, so generalized concatenation should iterate the same way.

# re: Excel 2007 investments in UDFs #1

Wednesday, May 03, 2006 8:34 PM by a user
I think I understand the reasons for the UDF framework, and for deprecating VBA. But I do not understand why it is called "User Defined Function" when this is clearly, expressly, and exclusively for Developers.

What happens to the customer base when Users are deprecated?

# re: Excel 2007 investments in UDFs #1

Wednesday, May 03, 2006 10:00 PM by Colin Banfield
<<WehWordM: return n_th word in s delimited by d
=MID(s,FIND(CHAR(127),SUBSTITUTE(d&n,
d,CHAR(127),4)),FIND(CHAR(127),
SUBSTITUTE(s&d,d,CHAR(127),4))
-FIND(CHAR(127),SUBSTITUTE(d&s,
d,CHAR(127),4))) <<

Harlan, why would the average user (or even a seasoned user for that matter) prefer to use a formula with thirteen functions (assuming that he/she can figure it out in the first place) over a single function with two arguments?  You have to remember, perhaps 90% of all Excel users are not as smart or as masochistic as Harlan Grove.

# re: Excel 2007 investments in UDFs #1

Thursday, May 04, 2006 1:02 PM by Harlan Grove
Colin,

How many current Excel users could recognize the udf code as written in C#? Who is the audience for this particular blog posting?

You could write a udf for everything that requires 3 or more built-in function calls. Would that make sense? How many more functions should Excel have? What would be the most useful additional functions?

Alternatively, which is easier: remembering how to write longish formulas or remembering a +1000 functions?

My point was that there are things that can *ALREADY* be done using built-in functions alone. It'd be better to show examples of new udfs that do things that *CAN'T* be done by udfs alone. A generalized concatenation udf would have shown whether the new XLLs could accept variable numbers of arguments, detect whether any of those arguments were arrays, and if so show how to iterate over them. A udf wrapper around the split method of the string class isn't interesting or illuminating, not to mention there have been newsgroup postings showing VBA udf wrappers around VBA's split function which return the array of tokens.

Also, it'd be a good thing to design udfs to match up with Excel's capabilities. When it comes to functions, that means supporting use in array formulas. WehWordM and WehWordCountM should accept arrays of strings as 1st arg or arrays of positions as 2nd arg and return array results. A modest modification to my long formula can do that. The udfs linked to by the blog posting can't. Again, for the intended audience for this blog posting, wouldn't it have been useful to show how (whether) new XLL udfs return array results? If those udfs can only return scalars, claiming it provides V1 features is an overstatement.

# re: Excel 2007 investments in UDFs #1

Thursday, May 04, 2006 4:55 PM by Danny Khen
Thanks everyone for the feedback and questions! Some answers (in order of appearance):

Andrew: I was discussing server-side UDFs, and we don't actually have editing capabilities on the server, so I am not sure what kind of Undo mechanism you are talking about. Please drop me an email thru the "Email" link at the top and we can discuss.

Harlan: Your comments and the discussion that followed are interesting, however I was not trying to demonstrate the implementation of any particular feature. The point here is to show how to create managed UDFs for Excel Services. UDFs as their name implies are all about your own extensibility needs, whatever they may be; we're just providing the framework and defining the interface.

The server UDFs can indeed return an array and be called in an array formula. You'd need to define your return value as object[] or System.Double[,] etc. We will have full reference documentation as part of the Office SharePoint Server 2007 SDK, which is due to be published with Office Beta2 as far as I know.

A user: "User-Defined Functions" are just an industry term. They refer to functions defined by developer users, as opposed to end-users. We did not deprecate VBA in Excel; however it is not supported on Excel Services, since we believe that managed extensibility is the way to go for a server product (and we will be extending what we support in this area in future releases).

We're going to have a follow-up post on this blog to show how UDFs in existing XLL addins can be used on the server thru managed wrappers - stay tuned.

# re: Excel 2007 investments in UDFs #1

Thursday, May 04, 2006 5:04 PM by Danny Khen
Harlan, I missed your first question about the limit on string argument and string return value length.
Short answer - we did increase the limit. A bit longer answer: We added Unicode string types, and for those, you can pass strings up to 65535 characters long. Of course, depending on the purpose of the string, the actual limit may be smaller; e.g. if you are setting a string into a cell, the Excel 2007 limit on cell string length is 32767 chars, so anything longer will be truncated.

# re: Excel 2007 investments in UDFs #1

Friday, May 05, 2006 6:38 PM by Colin Banfield
Hi Harlan,

<<How many current Excel users could recognize the udf code as written in C#? Who is the audience for this particular blog posting?<<

Oh, I was thinking of the convenience of the UDF to the end user, who woundn't know or care what code the UDF is written in.

<<You could write a udf for everything that requires 3 or more built-in function calls. Would that make sense? How many more functions should Excel have? What would be the most useful additional functions?<<

There are few if any ATP functions (for example) that can't be emulated with basic math functions.  The ATP functions are popular among users nonetheless.

<<My point was that there are things that can *ALREADY* be done using built-in functions alone. It'd be better to show examples of new udfs that do things that *CAN'T* be done by udfs alone. A generalized concatenation udf would have shown whether the new XLLs could accept variable numbers of arguments, detect whether any of those arguments were arrays, and if so show how to iterate over them. A udf wrapper around the split method of the string class isn't interesting or illuminating, not to mention there have been newsgroup postings showing VBA udf wrappers around VBA's split function which return the array of tokens.<<

No argument here, when viewed from the developer's perspective.

<<Also, it'd be a good thing to design udfs to match up with Excel's capabilities. When it comes to functions, that means supporting use in array formulas. WehWordM and WehWordCountM should accept arrays of strings as 1st arg or arrays of positions as 2nd arg and return array results. A modest modification to my long formula can do that. The udfs linked to by the blog posting can't. Again, for the intended audience for this blog posting, wouldn't it have been useful to show how (whether) new XLL udfs return array results? If those udfs can only return scalars, claiming it provides V1 features is an overstatement. <<

Well, I try to avoid array formulas wherever possible but I have to admit, there are times where they're darned convemient to use.  Personally, I'd like to examples of XLL UDFs that support 3D ranges.




# re: Excel 2007 investments in UDFs #1

Friday, May 05, 2006 7:50 PM by Danny Khen
Ranges / array formulas:
In his blog, Shahar had a post with sample code for UDFs that receive and return both 1- and 2- D arrays. These can be used in array formulas.
Here's the link:
http://blogs.msdn.com/cumgranosalis/archive/2006/04/11/WebServiceAsyncUdfs.aspx

# re: Excel 2007 investments in UDFs #1

Monday, May 08, 2006 1:30 PM by Harlan Grove
Colin,

I'd also like udf examples that use 3D references, but that'd require a fundamental, nontrivial addition to Excel's object model since there are no such objects available through XL11.

# re: Excel 2007 investments in UDFs #1

Monday, May 08, 2006 2:54 PM by Harlan Grove
Colin,

Goofed. I was thinking in terms of VBA udfs, and there are no 3D references there. Since it appears that udf support is moving away from VBA, there may never be VBA support for 3D references. But XLL udfs through XL11 accept 3D references, Longre's MOREFUNC.XLL's THREED udf being the principal example I know of.

# Making Excel Services UDFs Work in Excel 2007

Friday, September 01, 2006 1:32 PM by Microsoft Excel 2007 (nee Excel 12)
Shahar Prish, one of the developers on the Excel Services team, has recently posted a few entries on...

# Excel su Sharepoint Server come client server

Friday, September 22, 2006 8:55 AM by Tecnologie .NET (Dotnet)
Excel Sharepoint UDF server side

# Excel su Sharepoint Server 2007 come client server

Friday, November 03, 2006 2:25 AM by Tecnologie .NET (Dotnet)

Excel Sharepoint UDF server side

# Excel su Sharepoint Server 2007 come client server

Friday, November 10, 2006 2:51 AM by Tecnologie .NET (Dotnet)

Excel-Sharepoint-Services-UDF Serverside Excel

# Christian Stich on Excel Services: Financial option valuation implemented with Monte Carlo simulation using multithreaded User Defined Functions (UDFs)

Tuesday, August 14, 2007 11:25 PM by Microsoft Excel

Today’s author: Christian Stich, a program manager on the Excel Services team who likes to combine his

# Christian Stich on Excel Services: Financial option valuation implemented with Monte Carlo simulation using multithreaded User Defined Functions (UDFs)

Tuesday, August 14, 2007 11:37 PM by Noticias externas

Today’s author: Christian Stich, a program manager on the Excel Services team who likes to combine his

# Building an Asset Tracking Application in Excel Services – Part 3b of 5

Wednesday, July 02, 2008 1:01 PM by Microsoft Excel

Today's author, Dan Parish, continues his discussion on building an Excel Services solution. In Part

# Building an Asset Tracking Application in Excel Services ??? Part 3b of 5, Microsoft Office developer blog

# iterate formula in excel

Wednesday, July 16, 2008 9:14 AM by iterate formula in excel

# Building an Asset Tracking Application in Excel Services ?????????????? Part 3b of 5, Microsoft Office developer blog

# UDFs in Microsoft Excel 2007 services &laquo; Knowledgebase

# Microsoft Excel Excel 2007 investments in UDFs 1 | debt consolidator

New Comments to this post are disabled
 
Page view tracker