Eric White's Blog

Leaving Microsoft, Blog Moving to EricWhite.com

Eric White - Independant Consultant — Wed, 19 Jan 2011 16:40:00 GMT

Due to personal obligations (moving away from Seattle) it is with sadness that I’m announcing that I’m leaving Microsoft. My last day at Microsoft will be January 21. This will be the last post on this blog. I will be commencing to blog at EricWhite.com/blog. Here is what I'm planning for the near future on my new blog:

This blog is inactive.
New blog: EricWhite.com/blog

Blog TOCI want to explore some simple and powerful ways to build parameterized server-side document generation systems. I’m envisioning a blog post series that will be fun to write. The code that I’ll be releasing on my blog will be under a license similar to code published on my MSDN blog. You can alter the code if necessary, and embed it in your projects as you wish.
I plan to explore the SharePoint server object model in detail, writing lots of code samples, as a way of filling in gaps in my knowledge. I’ll be blogging those snippets.
I will be completing the blog post series on writing a recursive descent parser for SpreadsheetML formulas.
While the blog map that I maintain on this blog will be frozen, I’ll maintain a new blog map on ericwhite.com that encompasses content on this blog as well as my new blog.

I want to say thanks to all the folks with whom I’ve worked. Microsoft is filled with brilliant, hardworking people, and it is an honor to have worked with you. While here, I have had the chance to work on some of the coolest technology in the world: LINQ to XML, Open XML, Office, and SharePoint. You guys rock!

Over the years, I have regularly talked with many of you (both inside and outside of Microsoft) about Open XML – explaining markup and suggesting approaches to the design of solutions. I really enjoy sharing what I’ve learned, so please continue to contact me with questions via email, or if you want to chat by phone, I’m happy to do so by appointment. I also enjoy discussing Office or SharePoint development. Contact me at eric at ericwhite.com.

Connect with me on LinkedIn, or follow me on twitter.

Using the OData Rest API for CRUD Operations on a SharePoint List

Eric White - Independant Consultant — Fri, 17 Dec 2010 21:06:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

SharePoint 2010 exposes list data via OData. This post contains four super-small code snippets that show how to Create, Read, Update, and Delete items in a SharePoint list using the OData Rest API.

This post is one in a series on using the OData REST API to access SharePoint 2010 list data.

These snippets work as written with the 2010 Information Worker Demonstration and Evaluation Virtual Machine. That VM is a great way to try out SharePoint 2010 development. Also see How to Install and Activate the IW Demo/Evaluation Hyper-V Machine.

See the first post in this series, Getting Started using the OData REST API to Query a SharePoint List, for detailed instructions on how to build an application that uses OData to query a list. These snippets use the list that I describe how to build in that post.

Query a List

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using Gears.Data;

class Program
{
    static void Main(string[] args)
    {
        // query
        TeamSiteDataContext dc =
            new TeamSiteDataContext(new Uri("http://intranet/_vti_bin/listdata.svc"));
        dc.Credentials = CredentialCache.DefaultNetworkCredentials;
        var result = from d in dc.Inventory
                     select new
                     {
                         Title = d.Title,
                         Description = d.Description,
                         Cost = d.Cost,
                     };
        foreach (var d in result)
            Console.WriteLine(d);
    }
}

Create an Item

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using Gears.Data;

class Program
{
    static void Main(string[] args)
    {
        // create item
        TeamSiteDataContext dc =
            new TeamSiteDataContext(new Uri("http://intranet/_vti_bin/listdata.svc"));
        dc.Credentials = CredentialCache.DefaultNetworkCredentials;
        InventoryItem newItem = new InventoryItem();
        newItem.Title = "Boat";
        newItem.Description = "Little Yellow Boat";
        newItem.Cost = 300;
        dc.AddToInventory(newItem);
        dc.SaveChanges();
    }
}

Update an Item

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using Gears.Data;

class Program
{
    static void Main(string[] args)
    {
        // update item
        TeamSiteDataContext dc =
            new TeamSiteDataContext(new Uri("http://intranet/_vti_bin/listdata.svc"));
        dc.Credentials = CredentialCache.DefaultNetworkCredentials;
        InventoryItem item = dc.Inventory
            .Where(i => i.Title == "Car")
            .FirstOrDefault();
        item.Title = "Car";
        item.Description = "Super Fast Car";
        item.Cost = 500;
        dc.UpdateObject(item);
        dc.SaveChanges();
    }
}

Delete an Item

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using Gears.Data;

class Program
{
    static void Main(string[] args)
    {
        // delete item
        TeamSiteDataContext dc =
            new TeamSiteDataContext(new Uri("http://intranet/_vti_bin/listdata.svc"));
        dc.Credentials = CredentialCache.DefaultNetworkCredentials;
        InventoryItem item = dc.Inventory
            .Where(i => i.Title == "Car")
            .FirstOrDefault();
        dc.DeleteObject(item);
        dc.SaveChanges();
    }
}

Comparing Excel Automation to Excel Services

Eric White - Independant Consultant — Thu, 16 Dec 2010 22:56:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

In a previous post, I compared and contrasted Word Automation to Word Automation Services. There is a similar contrast between Excel Automation and Excel Services, although there is less overlap in functionality when compared to the two approaches for Word.

This post is one in a series on Microsoft Office 2010 application development. These posts will be published in the future as part of an MSDN article. As usual, after the MSDN article is published, I’ll place pointers in the blog posts to the article on MSDN.

Excel Automation

Excel Automation is a technology where you can write a .NET application that uses the Excel client executable to accomplish some task such as creating a spreadsheet, querying a spreadsheet for content, or modifying the content of a spreadsheet. Its roots are COM – in addition to using .NET to link to the Primary Interop Assemblies, you can use C++ or VB6 (or any other technology that can consume COM) to automate the Excel client application. In other words, you write a program that runs Excel. There is a rich object model that you can use to manipulate workbook, worksheets, ranges, cells, comments, etc. This technology has been around for years. If you are an experienced Excel developer, then you have probably used Excel Automation at one time or another.

The user operates a .NET managed application that in turn runs the Excel 2007 or Excel 2010 client.

To make this completely clear, the following small console application starts Excel, creates a workbook and worksheet, sets the value of a cell, saves it, and then causes the Excel application to quit. You need to add a reference to the Excel 2010 PIA. You find this on the COM tab.

Note that the following sample uses the feature C# 4.0 that allows you to omit arguments that have default values.

using Excel = Microsoft.Office.Interop.Excel;

class Program
{
    static void Main(string[] args)
    {
        Excel.Application app = new Excel.Application();
        app.Visible = true;
        Excel.Workbook workbook = app.Workbooks.Add();
        Excel.Worksheet sheet = workbook.ActiveSheet;
        sheet.Cells[1, 1] = "Hello World!";
        workbook.SaveAs(@"E:\Temp\Test.xlsx");
        ((Microsoft.Office.Interop.Excel._Application)app).Quit();
    }
}

This scenario, where you are programmatically generating spreadsheets, is where you should be using the Open XML SDK instead of Excel automation.

However, there are scenarios where we recommend that instead of using the Open XML SDK, you should automate Excel in some fashion. If you are altering or modifying the workbook (other than simple setting of cell values), then you should automate Excel. Altering spreadsheets using the Open XML SDK is involved. Inserting a row or column using the Open XML SDK would involve parsing formulas and modifying the parsed formulas, a fairly involved operation.

In addition, just as with Word, many customers find it is advantageous to integrate Excel into their application, or to extend Excel to augment their applications functionality.

Excel Services

Excel Services is a service application that enables you to load, calculate, and display Microsoft Excel workbooks on Microsoft SharePoint Server 2010. Excel Services was first introduced in Microsoft Office SharePoint Server 2007.

By using Excel Services, you can reuse and share Excel workbooks on SharePoint Server 2010 portals and dashboards. For example, financial analysts, business planners, or engineers can create content in Excel and share it with others by using a SharePoint Server 2010 portal and dashboard—without writing custom code. You can control what data is displayed, and you can maintain a single version of your Excel workbook.

You cannot use Excel Services to create new workbooks or to edit existing workbooks. To author a workbook for use with Excel Services, you can use the Open XML SDK, Microsoft Office Excel 2007, or Microsoft Excel 2010.

There are four primary interfaces for Excel Services:

An Excel Web Access Web Part, which enables you to view and interact with a live workbook by using a browser
Excel Web Services for programmatic access
An ECMAScript (JavaScript, JScript) object model for automating and customizing, and to drive the Excel Web Access control and help build more compelling, integrated solutions
A Representational State Transfer (REST) API for accessing workbook parts directly through a URL

See the Excel Services Overview for more information.

To Summarize

You can use Excel Automation or the Open XML SDK to create workbooks.

You can use Excel Automation, the Open XML SDK, or Excel Services to retrieve contents of workbooks.

You can use Excel Automation to alter workbooks, with the exception that in some cases you can use the Open XML SDK to set values of cells.

Microsoft Word 2010 Developer Building Blocks

Eric White - Independant Consultant — Wed, 15 Dec 2010 20:16:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

Extending Microsoft Word 2010 is one of the most effective ways to augment content and enable richer collaboration around documents. There are many ways to extend Microsoft Word 2010 and integrate user-friendly and powerful document editing into your application. Extending Word is the most common way to develop for Microsoft Office.

In this series of posts, I'll be publishing a post on each of the Office client applications, listing the developer building blocks for it. I put the various developer building blocks into one of four categories:

· User Interface Extensions – You use these building blocks to create custom ribbons, custom user interfaces, extend content menus, and more.

· Canvas Extensions – Available only in Microsoft Word and Microsoft Excel , canvas extensions are those extensibility points that enable you to add controls to the surface of a document or spreadsheet. You can add controls to the surface of a document to create more interactivity for users.

· Programmability Extensions – In combination with user interface and canvas extensions, you use programmability extensions to build customizations and add-ins.

· Supported for Previous Versions – listed for completeness, instead of using these legacy technologies, Microsoft recommends using newer approaches for programmability using managed add-ins, customizations, and user interface extensions.

In addition to extending the Word user interface on client computers, developers take advantage of services on servers such as Word Automation Services or WCF services to build distributed applications with rich functionality. Using the Open XML SDK on servers enables you to develop enterprise-quality bulk document generation applications that perform well and are robust.

Developer Center: Word Developer Center

Technical Articles: Word 2010 Technical Articles

Technical Articles: Word 2007 Technical Articles

Videos: Word Visual How Tos

User Interface Extensions

Building Block	Description
Action Pane	Extend the user interface by creating a customizable pane for document-level solutions.
Backstage	Customize the backstage view enables you to display information about documents to users.
Context Menu	Customize the context menu to make custom features easy to use.
Custom Task Pane	Extend the user interface by creating a customizable pane for application-level solutions.
Document Information Panel	Enable users to enter metadata about a document.
Ribbon	Customize the ribbon to augment the editing experience of the document and make the features of your add-in more discoverable.

Canvas Extensions

Building Block	Description
Content Controls	Use content controls to give structure to documents.
User Controls	Add Windows Forms or WPF controls to the surface of a document to build interactive documents.

Programmability Extensions

Building Block	Description
Application Level Add-Ins	Develop a managed add-in that users can use to interact with Word documents. An installed solution can operate on any open document.
Automation	Use automation to build an executable (external to Word) that can control Word, including starting Word, setting and getting ranges of text, saving the document, and more.
AltChunk	Use the altChunk functionality of Open XML to merge content from other Open XML documents, HTML documents, and more into a new document.
Business Data Connectivity Object Model	Connect to BCS data through SharePoint.
Custom XML Parts	Embed and associate XML documents in a word-processing document.
Document Level Customization	Develop a managed add-in that users can use to interact with Word documents. The solution is attached to a specific document or template.
Local Database	Include a database file, such as a SQL Server Express (.mdf) file or a Microsoft Office Access (.mdb) file, in your Visual Studio Tools for Office solution.
Open XML SDK	Use the Open XML SDK to generate Word documents. In addition to generating documents, the Open XML SDK enables developers to make extensive modifications to a document such as replacing the contents of a paragraph.
Visual Basic for Applications	Use Visual Basic for Applications (unmanaged code) that is connected to and persisted with a specific document.

Supported for Previous Versions

Building Block	Description
ActiveX Controls on Word Document	Use ActiveX controls on the surface of a document to create interactive documents.
ActiveX Controls on User Form	Use ActiveX controls on a user form to build interactive solutions.
COM Extensions	Use unmanaged code to create a COM based add-in.
Command Bars	Customize command bars to enable users to carry out actions. Instead of using command bars, we recommend extending the ribbon.
Smart Documents	Use XML to define custom task pane content that automatically synchronizes help, instructions, and programmable controls with the user's position in the document.

Action Pane

An actions pane is a customizable task pane that is attached to a specific Microsoft Office Word document in a document-level solution. It is hosted inside the Office task pane along with other built-in task panes. This real-estate can be docked to the right, bottom, left, or right of the document or spreadsheet. It can also be a floating window. You can use Windows Forms controls or WPF controls in an action pane.

SDK: Actions Pane Overview

How Do I: Use WPF Controls Inside an Excel Actions Pane?

MSDN: Features Available by Office Application and Project Type

Blog Post: Understanding the Difference between Custom Task Panes and Action Panes

ActiveX Controls on Word Document

You can add controls to your documents to create interactive documents, such as online forms. Instead of using ActiveX controls, Microsoft recommends using Windows Forms or WPF controls.

MSDN: Using ActiveX Controls on Word Documents

ActiveX Controls on User Form

You can use ActiveX controls to create user forms. Instead of building user forms using ActiveX, Microsoft recommends building forms using Windows Forms or WPF.

MSDN: Effectively Using ActiveX Form Controls in Microsoft Word

MSDN: OLE Programmatic Identifiers

AltChunk

The altChunk functionality of the Open XML file formats enables easy merging of documents. You can merge content from multiple sources (other Open XML documents, HTML, plain text, and more) into a single document. After using the Open XML SDK to set up the document that imports alternative content, if you want to convert the document so that the new content is transformed to typical Open XML WordprocessingML, you need to open and save the document using Word 2010. Alternatively, you can use Word Automation Services to process the document and import the alternative content.

Blog Post: How to Use altChunk for Document Assembly

Blog Post: Comparison of altChunk to the DocumentBuilder Class

Blog Post: Formats Supported for altChunk

Blog Post: Inserting Content That Contains Images Using altChunk

Technical Article: Developing with SharePoint 2010 Word Automation Services

Application Level Add-Ins

This type of add-in’s functionality is available regardless of which document, spreadsheet, or presentation is opened. An example of this variety is a department-wide or corporate-wide application that enables some level of functionality that every employee of the department or corporation needs to access on a regular basis. As an example, Lexis for Microsoft Office is an Office customization targeting legal firms. Users need to execute the same code for every document, so it is implemented as an application-level managed add-in.

MSDN: Office Development in Visual Studio

MSDN: Architecture of Application-Level Add-Ins

MSDN: Programming Application-Level Add-Ins

MSDN: Managing Task Panes in Multiple Word and InfoPath Documents

MSDN: Managing Custom Task Panes in Multiple Application Windows

MSDN: Expose an object in an application-level add-in to other Microsoft Office solutions.

SDK: Microsoft.Office.Interop.Word Namespace

Blog Post: Understanding the Three Approaches to Office Development

Automation

Word Automation is a technology where you can write a .NET application that uses the Word client executable to accomplish some task such as creating a document, querying a document for content, or modifying the content of a document. Its roots are COM – in addition to using .NET to link to the Primary Interop Assemblies, you can use C++ or VB6 (or any other technology that can consume COM) to automate the Word client application. In other words, you write a program that runs Word. There is a rich object model that you can use to manipulate paragraphs, ranges, comments, etc. This technology has been around for years. If you are an experienced Office developer who needed to generate a large number of documents (prior to the introduction of the Open XML SDK), then you have probably used Word Automation at one time or another.

The user operates a .NET managed application that in turn runs the Word 2007 or Word 2010 client.

Instead of generating word-processing documents using Word Automation, we recommend using the Open XML SDK together with Word Automation Services. The Open XML SDK and Word Automation Services are ‘server hardened’.

Blog Post: What is the Difference between ‘Word Automation’ and ‘Word Automation Services’?

Backstage

Clicking the File tab takes you to the customizable Microsoft Office Backstage view. Backstage view helps you discover and use the features that fall outside the authoring features on the Ribbon. Ribbon, Mini toolbar and galleries all help you work in your documents; backstage view helps you work with your documents.

The following screen clipping shows the customizable backstage view.

Backstage View Developer Center

Technical Article: Introduction to the Office 2010 Backstage View for Developers

Technical Article: What's New for Developers in Office 2010

Technical Article: Customizing the Office 2010 Backstage View for Developers

Business Data Connectivity Object Model

Business Connectivity Services (BCS) brings line-of-business data into SharePoint 2010. SharePoint 2010 makes this data available to the Office client applications through the Business Data Connectivity Object Model (BDC OM). The following diagram shows how the Office clients (J) access BCS data through the BDC client runtime (K).

Business Connectivity Services Resource Center

TechNet: Business Connectivity Services overview

MSDN: Understanding Business Connectivity Services Client Cache to Optimize Your Solutions

MSDN: When Is the Cache Refreshed?

Blog: Overview of BCS Advanced Code-Based Solutions

COM Extensions

You can build a Component Object Model (COM) Office add-in using C++, VB6, or other languages that can consume COM libraries. Microsoft does not recommend using COM extensions for new development. Microsoft does not recommend building COM based add-ins using C# or VB.NET. Instead, you should build an application-level add-in or a document-level customization using Visual Studio.

MSDN: Creating COM Add-ins in Office Developer

MSDN: Building COM Add-ins for the Visual Basic Editor

Office Online: Enable or disable add-ins in Office programs

Command Bars

The command bars object model exposes objects, collections, properties, and methods that you can use to show, hide, and modify existing command bars and command bar controls, and create new ones. In addition, you can specify a Microsoft® Visual Basic® for Applications (VBA) procedure to run when a user clicks a command bar button or to respond to events triggered by a command bar or command bar control. Instead of using command bars, Microsoft recommends extending the ribbon.

MSDN: Manipulating Command Bars and Command Bar Controls with VBA Code

How To Use Automation to Create Office Command Bars and Controls with Visual Basic .NET

Content Controls

Content controls are bounded and potentially labeled regions in a document that serve as containers for specific types of content. Using content controls in Word 2007 and Word 2010, you can create document templates with rich, structured blocks of content.

Word Content Controls Resource Center

How to: Add Content Controls to Word Documents

Walkthrough: Binding Content Controls to Custom XML Parts

Video: How Do I: Use Word Content Controls?

Visual How-To: Mapping Word 2007 Content Controls to Custom XML Using the XMLMapping Object

Context Menu

Customizing the context menu provides a convenient way for users to access functionality for your application-level add-in or document-level customization.

Technical Article: Customizing Context Menus in Office 2010

Custom Task Pane

Custom task panes consist of screen real-estate that co-exists with the open document or spreadsheet within the Office client. This real-estate can be docked to the right, bottom, left, or right of the document or spreadsheet. It can also be a floating window. You can use Windows Forms controls or WPF controls in an action pane.

MSDN: Custom Task Panes Overview

MSDN: Managing Custom Task Panes in Multiple Application Windows

Blog Post: Understanding the Difference between Custom Task Panes and Action Panes

Custom XML Parts

Custom XML parts enable you to store arbitrary XML of your own design in the package of a document. The most common use for custom XML parts is to associate data in the parts with content controls in the document. However, you can use custom XML parts for any purpose that you like. For example, your managed add-in may need to keep some data for some important functionality. It can keep that data in a custom XML part, so that the data always accompanies the document.

Walkthrough: Binding Content Controls to Custom XML Parts

Video How to: Binding Content Controls to Custom XML Parts

Blog Post: Creating Data-Bound Content Controls using the Open XML SDK and LINQ to XML

Document Level Customization

Using document-level customizations,.NET managed assemblies (code-signed for security purposes) are attached directly to each document, so if you send the document to a new user, they can open the document and use the custom functionality without explicitly installing an add-in. For example, a tax analysis department needs to systematically analyze documents for their tax implications and then communicate that analysis back to the document author. The result of that analysis is associated with a specific document, so it makes sense to build a document-level customization that manages the process of assessing and optimizing the tax implications of that document. A typical deployment might include an application-level add-in that is used by the tax analysis firm that produces a document that is customized with a document-level customization.

MSDN: Developing Document-Level Customizations for Word

MSDN: Architecture of Document-Level Customizations

MSDN: Cached Data in Document-Level Customizations

How Do I: Attach or Detach a VSTO Assembly from a Word Document?

Document Information Panel

Often in document management systems, users are asked—usually at the point when they are saving their documents—to provide metadata (properties) for their documents. Document information panels enable users to specify all the properties on a document at once, in one place, at any point when they are working with that document.

SDK: Document Information Panel Overview

Local Database

You can include a database file, such as a SQL Server Express (.mdf) file or a Microsoft Office Access (.mdb) file, in your Visual Studio Tools for Office solution. This enables end users to maintain a local database in situations where maintaining a centralized database is not required, for example in a inventory solution where the data is embedded inside the document, and used on a single computer.

MSDN: Using Local Database Files in Office Solutions Overview

Open XML SDK

You can work directly with the Open XML file formats (DOCX, XLSX, and PPTX files) using the Open XML SDK. When you use the Open XML SDK, you are not directly customizing Microsoft Office, but are instead directly generating or modifying Office documents. The bulk generation of documents using the Open XML SDK is much faster than automating Word.

Open XML Developer Center

Blog: Eric White’s Blog

Blog: Zeyad Rajabi’s Open XML SDK Blog Map

Ribbon

The key benefit of customizing the ribbon is that it makes features of your application more discoverable. Users can see tabs that you add to the ribbon. They click on the tab, and then see buttons and other controls that help them understand the capabilities of your Office application.

MSDN: Ribbon Overview

How to: Get Started Customizing the Ribbon

Walkthrough: Creating a Custom Tab by Using the Ribbon Designer

Walkthrough: Creating a Custom Tab by Using Ribbon XML

Smart Documents

Developers can create Word or Excel Smart Documents augmented with programmable pieces that help users create or modify them. Reusing boilerplate text, implementing business rules for how a document should be formatted or completed, and routing a document through an approval process are all procedures that Office Smart Documents can simplify.

Instead of using Smart Documents, we recommend building application-level or document-level managed add-ins.

MSDN Magazine: Create Word and Excel Smart Documents with C++ and XML

SDK: Smart Documents SDK

SDK: Understanding XML Expansion Packs

Office Online: About smart documents

User Controls

You can add Windows Forms or WPF controls to the surface of a document, enabling richer interaction in the document for users. You can bind controls to data.

MSDN: Windows Forms Controls on Office Documents Overview

MSDN: Limitations of Windows Forms Controls on Office Documents

MSDN: Using Windows Forms Controls on Word Documents

MSDN: Binding Data to Controls in Office Solutions

How to: Populate Documents with Data from a Database

VisualBasicforApplications

Visual Basic for Applications (VBA) is an implementation of Microsoft's event-driven programming language Visual Basic 6. Most Microsoft Office applications include an integrated editing and debugging environment for VBA. VBA enables developers to build user defined functions and automate processes that access the Word document and customize the user interface. Applications built with the 64-bit version of Office 2010 can reference larger address spaces, and therefore provide the opportunity to use more physical memory than ever, potentially reducing the overhead spent moving data in and out of physical memory.

Technical Article: Getting Started with VBA in Office 2010

Technical Article: Getting Started with VBA in Word 2010

MSDN: Signing a VBA Project

MSDN: Word Object Model Overview

Office online: Digitally sign a macro project

Technical Article: Compatibility Between the 32-bit and 64-bit Versions of Office 2010

MSDN: Interoperability between VBA and Visual Studio Tools for the Office System (3.0)

How to Install and Activate the IW Demo/Evaluation Hyper-V Machine

Eric White - Independant Consultant — Mon, 13 Dec 2010 15:19:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

The 2010 Information Worker Demonstration and Evaluation Virtual Machine is the easiest way to evaluate SharePoint development. The VM comes complete with an installed, working copy of SharePoint 2010, Visual Studio, Office 2010, and much, much more. The installation instructions mention that you can connect the virtual machine to the internet and activate it for 180 days. However, it had been long ago enough since I last performed this process that I had forgotten a couple of details, and had one of those scratching-my-head experiences, so I’m noting the steps to do this. Actually, I had intended to write this info down the last time I forgot how to activate the VM, but I neglected to, and so was doomed to repeat my experience.

While I am documenting the key points on activating, I’m also going to note some of my favorite ways to configure these machines – my personal preferences.

1. Enable virtualization technology in the BIOS. Sometimes the setting can be difficult to find – it was under Security on one of my desktop machines. On my Lenovo laptop, it is pretty obvious.

If you don’t remember to enable this, while following the IW Demo VM installation instructions, you will receive the error:

An error occurred while attempting to start the selected virtual machine(s).

‘2010-7a’ could not initialize.

The virtual machine could not be started because the hypervisor is not running.

If you have forgotten to enable it, and then receive the error, and then enable it, then remember that you have to physically power-off the CPU after changing the BIOS settings, not just reboot, otherwise the BIOS setting change isn’t in effect. That little tidbit was paid for with an embarrassing amount of time. :)

2. For running virtual machines, I prefer to boot to an install of Windows Server 2008 R2 that has as little as possible installed on it. I install the Hyper-V role, and if a laptop, I install the Wireless LAN Service, and that is about all. If the default display drivers work well enough, I don’t bother installing specialized display drivers. I actually prefer not to. I would never host IIS on a machine that is hosting Hyper-V VMs. The more memory you preserve in the host operating system, the more memory is available to various configurations of hosted VMs, some of which are tight in 8GB. I would never attempt to run on a machine with less than 8GB. I always make my laptop dual-boot.

When you initially set-up Hyper-V on the host computer, you are given an option of creating a network adapter for the wired internet port. You need to do this because you will be temporarily connecting the virtual machine to the internet to activate.

3. Download the various RAR files, expand them, and follow the procedures in the Virtual Machine Setup Guide that accompanies the IW Demo/Eval VM. To try your hand at development, you only need to install and run Virtual Machine “A”. Once it is running, you can log in, write and compile code, and extend SharePoint in a variety of ways. The instructions include the following note:

Activation of the operating system in the virtual machine is OPTIONAL. Please do NOT perform the following steps unless your initial 10 day evaluation period expires.

Because it was said so emphatically, I feel obligated to repeat it.

4. At this point, because you have followed the procedures in the VM installation instructions, if you open the Virtual Network Manager in Hyper-V Manager, you will see two networks. You added the internal network while following the VM installation procedures, and the other network is there because you responded positively to including it when installing the Hyper-V role.

5. While the VM is shut down, open the Settings for the Demo VM. You will see there is only the one network adapter.

6. Click Add Hardware, click Network Adapter, and click Add. Add a network adapter for the Local Area Connection.

Because you are adding the network adapter for the wired network (not wireless), even if you have a wireless connection on your laptop, you need to physically connect the computer to a network that has access to the internet.

7. Start the VM and let it boot. After logging into the VM, you should be able to browse the internet. You can then activate Windows Server 2008 R2 as usual. Click Start, right-click Computer, and then click Properties. Windows will inform you that the Windows license is valid for 180 days. Shut down the VM.

8. Open the settings for the Demo VM, and remove the network adapter for the Local Area Connection. The VM is not designed to be connected to the internet, and will not operate properly while connected.

Finally, Tristan Watkins has a great post on Optimizing the SharePoint 2010 IW Demo VM.

Getting Started using the OData REST API to Query a SharePoint List

Eric White - Independant Consultant — Thu, 09 Dec 2010 22:39:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

SharePoint 2010 exposes list data via OData. I’m currently working on an article around SharePoint and OData. As part of this effort, at various points, I’ll blog some of my intermediate samples. This post details the minimum number of steps to query a SharePoint list using OData.

This post is one in a series on using the OData REST API to access SharePoint 2010 list data.

By far, the easiest way to get started with SharePoint development is to use the 2010 Information Worker Demonstration and Evaluation Virtual Machine that runs using Hyper-V. The instructions in this and future related posts will be for that virtual machine. If you have setup your own development environment with a different machine name, it is easy enough to adjust these procedures to work with your own servers.

Procedure: Query a SharePoint list using OData

1. Download, extract, and boot the virtual machine. The download includes detailed instructions.

2. Log into the virtual machine. Use the username brads. The password is pass@word1.

3. Create a new list. Name the list ‘Inventory’. Create a two new columns: Description, and Cost. Make the type of the Description column be ‘Single line of text’. Make the type of the Cost column be ‘Currency’. Enter a couple of rows of sample data.

After completing this task, my list looked like this:

4. Start Visual Studio 2010 in the virtual machine.

5. Create a new project. Click File -> New -> Project. Select a directory for the project. Set the name of the project to Gears. The New Project dialog box will look something like this:

6. Right click on the References node in the Solution Explorer window, and click Add Service Reference. Enter http://intranet/_vti_bin/listdata.svc for the address. Change the namespace to Data. The Add Service Reference dialog box will look like this:

Click OK.

7. Copy and paste the following code into Program.cs:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using Gears.Data;

namespace Gears
{
    class Program
    {
        static void Main(string[] args)
        {
            TeamSiteDataContext dc =
                new TeamSiteDataContext(new Uri("http://intranet/_vti_bin/listdata.svc"));
            dc.Credentials = CredentialCache.DefaultNetworkCredentials;
            var result = from d in dc.Inventory
                         select new
                       {
                             Title = d.Title,
                             Description = d.Description,
                             Cost = d.Cost,
                         };
            foreach (var d in result)
                Console.WriteLine(d);
        }
    }
}

8. Press F5 to run. If everything worked, you will see something like the following:

Understanding the Difference between Custom Task Panes and Action Panes

Eric White - Independant Consultant — Thu, 09 Dec 2010 17:56:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

To the user, custom task panes and action panes have an identical appearance. Custom task panes and action panes consist of some screen real-estate that co-exists with the open document or spreadsheet within the Office client. This real-estate can be docked to the right, bottom, left, or right of the document or spreadsheet. It can also be a floating window.

The following screen clipping shows a custom task pane for an application-level add-in.

The following shows an action pane associated with a spreadsheet.

As you can see, custom task panes and action panes look very similar. There is one key difference. Custom task panes are associated with an application-level add-in in Excel, InfoPath, Outlook, PowerPoint, and Word. Action panes are associated with a document-level customization in Word and Excel only. With an application-level add-in, if the custom task pane is visible, it is visible regardless of which document or spreadsheet is the active one. With a document-level customization, the action pane is visible only if the customized document or spreadsheet is active.

Note that it is more common to build application-level add-ins and to use custom task panes. If you are building a document-level add-in, make sure that your scenario requires the additional complexity of attaching a code-signed assembly to documents and/or spreadsheets.

A note on nomenclature: As delivered, the Office client applications make use of task panes for a variety of purposes. The styles window (which you have probably used frequently) is a task pane. Most often, you see the styles task pane as a floating window, however, you are free to dock it to one side of the application. When you create a task pane for a managed add-in, it is a custom task pane.

There are a few final points to make about the difference between application-level add-ins and document-level add-ins.

With application-level add-ins, it is significantly easier to operate on all open documents or workbooks if you need to. Document-level add-ins, in contrast, are designed to work with one specific document. If you need to process or programmatically examine any or all open documents or spreadsheets, you should use an application-level add-in.

The root object when working with application-level add-ins is an Application object. In the case of Word, it is an object that implements the Microsoft.Office.Interop.Word.Application interface. For Excel, it is Microsoft.Office.Interop.Excel.Application.

The root object when working with document-level customizations is a document or spreadsheet. For Word, the root object implements the Microsoft.Office.Interop.Word.Document interface. For Excel, it is Microsoft.Office.Interop.Excel.Workbook.

Understanding the Architecture of Office 2010 Managed Add-Ins and Customizations

Eric White - Independant Consultant — Wed, 08 Dec 2010 15:52:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

The most important technology for building Office applications is that of managed customizations (including application-level managed add-ins, and document-level customizations). Understanding the architecture and capabilities of managed customizations enables you to place the various developer technologies in context. It is important to know which building blocks are integral to managed customizations, which building blocks provide the foundation for managed customizations, and which building blocks commonly are used in conjunction with managed customizations to provide users with the necessary functionality.

As I mentioned in Overview of Office 2010 Application Development, managed customizations consists of .NET classes that wrap the underlying COM libraries. Because add-ins wrap COM libraries, there are some differences between these class hierarchies when compared to other .NET class libraries.

You rarely instantiate objects. Instead, in your code in a managed add-in, you are passed an already created object graph that represents the underlying document or spreadsheet. You then navigate from one object to another in the object graph. Instead of explicitly instantiating new objects, you often request the framework to create one. In the post, What is the Difference between ‘Word Automation’ and ‘Word Automation Services’?, the code example shows requesting the paragraphs collection to create a new paragraph. This is, in effect, a factory method that creates a paragraph object.
In some cases, the programming interface does not follow the naming conventions for .NET.

Other than these differences, while the class hierarchies for each of the various applications are extensive, they are straightforward and easy to use. For example, in the Word object model, you commonly work with the Application, Document, Paragraph, and Range classes. See the Word Object Model Overview for more information. In the Excel object model, you commonly work with the Application, Workbook, Worksheet, and Range classes. See the Excel Object Model Overview for more information.

You can build managed add-ins for Word, Excel, Outlook, PowerPoint, Project, Visio, and InfoPath. For the other applications, you need to use COM or other developer technologies to extend them.

One key difference between writing COM interfaces and writing a managed add-in is that when building applications using the COM interfaces, you could place ActiveX controls in the document or spreadsheet. When writing a managed add-in, you use host items and host controls, which extend the underlying COM interface with some amount of additional functionality, a few more events, and the ability to use .NET Windows Forms controls on the surface of the document. See Host Items and Host Controls Overview for more information.

When you create a new Office 2010 project in Visual Studio, you must pick from one of a variety of project templates. Following are the project templates for C#. VB.NET, of course, has the same set of templates.

The following table details the varieties of add-ins:

Project Template Name	Description
Excel 2010 Add-in	Application-level add-in
Excel 2010 Template	Document-level customization that creates and customizes an Excel template
Excel 2010 Workbook	Document-level customization that creates and customizes an Excel workbook
InfoPath 2010 Add-in	Application-level add-in
Outlook 2010 Add-in	Application-level add-in
PowerPoint 2010 Add-in	Application-level add-in
Project 2010 Add-in	Application-level add-in
Visio 2010 Add-in	Application-level add-in
Word 2010 Add-in	Application-level add-in
Word 2010 Document	Document-level customization that creates and customizes a Word document
Word 2010 Template	Document-level customization that creates and customizes a Word template

The difference between a document-level add-in that creates a template and a document-level add-in that creates a document or workbook is that when the user uses the add-in to create a template (either a document or spreadsheet), the user opens a template to create a local copy as a new document. They cannot modify the original template unless they are given special permissions. See Office Project Templates Overview and Document vs. Template Solutions for more information.

SDK: Features Available by Office Application and Project Type

Developer Center: VSTO Developer Center

MSDN: Office Solutions Development Overview

What is the Difference between ‘Word Automation’ and ‘Word Automation Services’?

Eric White - Independant Consultant — Tue, 07 Dec 2010 14:42:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

‘Word Automation’ and ‘Word Automation Services’ are two developer building blocks that have similar names, and have overlapping functionality, but are very different pieces of technology. I received some email about the post Understanding the Three Approaches to Office Development using VSTO where readers were confused about this, so here is the scoop.

Word Automation

The user operates a .NET managed application that in turn runs the Word 2007 or Word 2010 client.

To make this completely clear, the following small console application starts Word, adds a paragraph, saves it, and then causes the Word application to quit. You need to add a reference to the Word 2010 PIA. You find this on the COM tab.

Note that the following sample uses the feature C# 4.0 that allows you to omit arguments that have default values.

using Word = Microsoft.Office.Interop.Word;

class Program
{
    static void Main(string[] args)
    {
        Word.Application app = new Word.Application();
        app.Visible = true;
        Word.Document doc = app.Documents.Add();
        Word.Range range = doc.Content;
        Word.Paragraph paragraph = range.Paragraphs.Add();
        paragraph.Range.Text = "Hello World!";
       object saveChanges = true;
        doc.SaveAs2(@"E:\Temp\Test.docx");
        ((Microsoft.Office.Interop.Word._Application)app).Quit();
    }
}

This scenario, where you are programmatically generating word-processing documents, is where you should be using the Open XML SDK instead of Word automation.

Word Automation Services

Word Automation Services is a service that runs under SharePoint Server 2010 (both Standard and Enterprise) that can convert documents to and from various formats, and can update content inside Word documents, such as calculated fields (like the table-of-contents). Word Automation Services is specifically designed to complement the Open XML SDK for server-side scenarios. Some tasks, such as repagination, updating fields, and importing altChunk elements are difficult to accomplish when using the Open XML SDK, so if your scenario requires those areas of functionality, it’s great to use the Open XML SDK together with Word Automation Services. Both the Open XML SDK and Word Automation Services are ‘server hardened’. You can build systems that generate and process hundreds or thousands of documents per day.

To use Word Automation Services, you must write code that runs on the SharePoint server; you cannot directly access Word Automation Services from a client computer. The Managed Client Object Model does not include a programming interface to Word Automation Services. See Developing with SharePoint 2010 Word Automation Services for more information.

However, there is an easy way to work around the server-side limitation of Word Automation Services:

Write a WCF service that runs on the SharePoint server computer that exposes the functionality that you need in your client application.
Write your client application to consume that WCF service, integrating Word Automation Services functionality into the client application.

You might have a scenario where you want to generate a PDF or XPS document based on user input in the client application. To implement this, you need to expose functions in the WCF service that can take a Word document as input, and returns a PDF or XPS document. You can then use the Open XML SDK to generate an Open XML word-processing document, and then use your WCF service to do the conversion.

Excel Automation vs Excel Services

Excel has the same possibilities. You can write a program that automates the Excel client application to accomplish certain scenarios. Alternatively, you can use Excel Services (part of SharePoint Server 2010 Enterprise) to implement functionality.

In contrast to Word, there are scenarios where using Excel automation is the only way to accomplish specific tasks. If you need to modify a spreadsheet, such as inserting or deleting a row or column, then the appropriate technology may be to automate the Excel client application. Inserting a row or column using the Open XML SDK would involve parsing formulas and modifying the parsed formulas, a fairly involved operation.

Understanding the Three Approaches to Office Development using VSTO

Eric White - Independant Consultant — Thu, 02 Dec 2010 16:07:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

When using Visual Studio Tools for Office (VSTO), there are three basic approaches to Office development:

Application-Level Managed Add-In
Document-Level Customization
Office Automation

Under the covers, the Office client applications use COM to expose their functionality. One of the uses of COM in Office is to develop an add-in, which is code that the Office client application loads and runs as the user operates the Office client application. There are specific mechanisms that Office uses for add-ins, including placing DLLs in appropriate places, configuring code access security to enable running the code, and creating registry entries so that the Office client applications know of the existence of the add-in.

Visual Studio Tools for Office layers a .NET managed programming interface on top of the COM interfaces. This is immensely valuable – the Microsoft developers who built VSTO take care of many issues around building COM applications, and free us up to develop the functionality that meets our customers’ needs. The .NET run-time is a managed run-time; hence the name of an add-in built using VSTO is a managed add-in. The .NET assemblies that are layered over COM are called the Office Primary Interop Assemblies (often called PIAs).

There are two approaches to building a customization of Office.

Application-level add-in – This type of add-in’s functionality is available regardless of which document, spreadsheet, or presentation is opened. An example of this variety is a department-wide or corporate-wide application that enables some level of functionality that every employee of the department or corporation needs to access on a regular basis. As an example, Lexis for Microsoft Office is an Office customization targeting legal firms. Users need to execute the same code for every document, so it is implemented as an application-level managed add-in.
Document-level customization – This type of functionality is part of the document. In this scenario, .NET managed assemblies (code-signed for security purposes) are attached directly to each document, so if you send the document to a new user, they can open the document and use the add-in’s functionality without explicitly installing an add-in. For example, a tax analysis department needs to systematically analyze documents for their tax implications and then communicate that analysis back to the document author. The result of that analysis is associated with a specific document, so it makes sense to build a document-level add-in that manages the process of assessing and optimizing the tax implications of that document. Actually, in this case you might have an application-level add-in that is used by the tax analysis firm that produces a document that is customized with a document-level add-in.

In another case, a large consulting and accounting firm produces spreadsheets that implement sophisticated calculations that would be difficult to implement directly using Excel formulas. They use a document-level add-in so that they can send spreadsheets to associates in other firms or to their customer, and those recipients can take advantage of the code in the document-level add-in. Note that with Office 2007 and Office 2010, document-level customizations are available only for Word and Excel.

The third approach to use the COM interfaces (both directly and through the PIAs) is Office Automation. You can develop an automation application that runs an Office client application to perform some specific task. Your application may have the appearance of a traditional Windows application. It may even be a simple console application. When you run this application, under the covers, the application runs the Office application (such as Word 2010 or Excel 2010) and uses it to accomplish the desired task.

Applications that use Office Automation are not intended for unattended server-side automation. The Office client applications are, at their core, user applications. They may put up dialog boxes and require user input. The issues around server-side Office client automation are well-known. Instead of using Office automation, the modern approach is to use the Open XML SDK combined with Word Automation Services. The Open XML SDK and Word Automation Services are server-hardened tools. You can accomplish many word-processing scenarios using them. The resulting application will be much more reliable, and will have better performance than if you build it using Office automation. There are very few word-processing scenarios where it is better to automate Office than to use the Open XML SDK and Word Automation Services. Note that while the name of Word Automation Services has the word 'Automation' in it, Word Automation Services is not automating Office on the server. Word Automation Services is a service that comes with SharePoint Server 2010. It is a service that you can use in combination with the Open XML SDK to accomplish word-processing scenarios.

In contrast, there are many Excel scenarios where we recommend using Office automation (although not unattended server-side automation). Modification of an Excel spreadsheet is fairly involved. For instance, if you insert a row, you probably want to update all formulas that reference cells below the inserted row. This is not trivial, and would rely on a formula parser and a fair amount of logic. Instead, for many scenarios, it is better to use Excel automation. Another approach to accomplishing some scenarios is to use Excel Services, which is a service that comes with SharePoint Server 2010 Enterprise.

Regardless of whether you are building an application-level add-in or a document-level customization, your extensions fall into one of two categories:

Your extension may enhance the user interface. For example, you may develop an extension that adds a new tab to the ribbon, and exposes functionality through an action pane or custom task pane (more on these shortly).
Your extension may directly manipulate open documents, spreadsheets, or presentations. For example, you may develop an application that inserts a pre-formatted table with data retrieved from a corporate database.

Typically, a application-level managed add-in or document-level customization will do both of these – it extends the user interface, and when the user operates the user interface, it manipulates the open document in some fashion.

In the next post, I'm going to discuss some of the notable points about Office development using VSTO, and compare that to more conventional C# / VB.NET development.

Office Application Scenarios

Eric White - Independant Consultant — Tue, 30 Nov 2010 14:47:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

Many developers identify and document developer scenarios while designing software before writing functional specifications. Sometimes this process consists of the construction of ‘story boards’ that show how the user moves from point A to point B to accomplish a specific task. Sometimes you may do formal use-case analysis. You may even mock up a scenario using PowerPoint or a modeling tool. In most development organizations, you write a formal specification that outlines and details the scenario.

There are a few general types of scenarios.

Scenarios that retrieve external data from a line-of-business system and surface that data for users. In these scenarios, you typically will generate documents, spreadsheets, or presentations from that data. For example, you may provide an approach for sales people to pull sales projections and results from a server to create a status update PowerPoint presentation for a sales meeting.
Scenarios that make it easier (or perhaps simply possible) to accomplish a specific task. You may have a mathematical optimizer written in FORTRAN that performs some heavy-duty number crunching, and integrates the results into a spreadsheet or word document.
Scenarios that augment the experience for a user. You may build a system that retrieves reference material automatically for specific content in a document or spreadsheet.
Scenarios around document generation. You may generate hundreds or thousands of Open XML document based on data stored in SharePoint lists or external systems, and then convert to PDF and print them.
Scenarios around process and procedure. Scenarios can be as simple as a document approval process, or can be as involved as shepherding a RFP through generating a proposal to signing of a contract.

These are a few examples of varieties of scenarios, but, of course, there are many more. You implement each developer scenario by using one or more Office developer building blocks. Of course, as systems designers, you must determine your own scenarios. Then, your task is to be familiar with the available building blocks that you can use to implement your scenario. In many cases, you have more than one option; each option has different characteristics, and one option might suit more than another for a particular scenario.

The building blocks vary for each Office application. Word and Excel have the highest number of building blocks, which makes sense due to their heavy usage and large number of scenarios where you want to extend them. In this article, I list the Office developer building blocks for each of the Office applications. I explain the use case for each building block, and provide links to further information to help you get started.

Developers typically extend a single Office application at a time. You may need to extend Word 2010 to enable certain content authoring scenarios. You may need to extend Excel 2010 to enable high-performance calculations that would be difficult to implement using just Excel. You may need to enable sales people to build PowerPoint presentations that are directly targeted towards the specific customer they are seeing that day. For each of these specific scenarios, you typically extend a single Office client application. There are some scenarios where you need to work with word-processing documents, spreadsheets, and presentations simultaneously, such as the scenario presented in Assembling Documents on SharePoint 2010 Sites by Merging Content from Excel, PowerPoint, and Word, but for most scenarios you extend only one or another of the Office client applications.

Development of an Office application is singularly different from other types of development. From the developer’s point of view, the Office clients comprise a framework – a large framework with a remarkable set of capabilities, but first and foremost the Office clients are end-user applications. This impacts our development in interesting ways.

Overview of Office 2010 Application Development

Eric White - Independant Consultant — Mon, 29 Nov 2010 15:37:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

You can use a variety of approaches to extend Microsoft Office 2010. There are multiple ways to extend each of the Microsoft Office applications’ user interfaces and programmatically work with the underlying documents, spreadsheets, or presentations. The best way to accomplish your desired objective is not always obvious. This article provides a high-level overview of Office 2010 as a development platform, and lists the Office Developer Building Blocks, which are the technologies that you use to extend Microsoft Office.

I am a developer relatively new to Office client development. It often is most useful for me to write about a technology immediately or soon after I learn it – mentioning those aspects which stood out to me, and placing technologies in context to other developer technologies that I’ve used.

When you consider Office development, one of the complicating factors is that Office has been around for many, many years, and the developer approach toward Office development has evolved over those years. It is pretty amazing that many applications that were written years ago using developer technologies of those days continue to operate correctly with Office 2010. However, our job as developers is to know what the new developer approaches are, and to migrate from the old to the new. There are good reasons why the Microsoft Office division releases new developer technologies and de-emphasizes older technologies. It is a tribute to Office development team’s efforts that many of those old developer technologies still work on Office 2010, but except where absolutely necessary, for new development, we should use the new approaches. While you should only use the recommended approaches for new development, it is valuable and educational to list every Office developer building block. This is one of the goals of this series of blog posts.

A Short History of Office Development

WordBasic was introduced to Microsoft Word for Windows and Word for OS/2 in 1989. I believe that it was the first Microsoft Office technology that enabled developers to manipulate documents programmatically. A few years after that, Microsoft released Visual Basic for Applications, bringing new capabilities and a more normalized library. COM-based interfaces followed, bringing Office development to C and Visual Basic. Visual Studio Tools for Office (which are layered on top of the COM-based interfaces) followed, enabling developers to create managed add-ins.

Visual Basic for Applications (VBA)

Many savvy users use VBA by recording a macro, then using the VBA editor to modify the macros. This is a purely pragmatic developer experience. Those users are concerned with getting the job done in as fast a manner as possible. When using VBA in that fashion, VBA is not so much a development environment as an extended, super-powerful character-based user interface to Office documents. There are companies that have developed extensive applications that used VBA as a development platform, but modern Office applications use Visual Studio Tools for Office to build managed add-ins. However, there are scenarios where securely-signed VBA code meets the user’s needs better than any other approach.

Extending Office by using COM

The COM based interfaces for Office automation were real development platforms. It was possible to build robust applications using COM and ActiveX, but if you used COM incorrectly, it was possible to build applications that were NOT robust, to say the least. There were a variety of problems caused by multiple versions of dynamic link libraries. This approach to application development was not for the faint of heart, but a detail-oriented developer could put together some cool functionality. If you were a software vendor who built products that provided significant value on top of the Office applications, you could build great products, but the developers on your staff needed to be top-notch.

Managed Add-Ins using Visual Studio Tools for Office (VSTO)

The release of Visual Studio Tools for Office (VSTO), along with the Primary Interoperability Assemblies (PIA) significantly made it easier to develop. The PIAs are specially coded .NET assemblies that provide a managed class library that wraps the underlying COM interfaces. It is so much easier to build robust C# and VB.NET application than applications written in C / C++ using COM. In development, frictional issues are those that you must deal with due to the underlying platform. They add no value to your application, but are necessary for developing reliable software. I’ve developed using both C / COM and C# / VB.NET, and can attest that when developing C / COM applications, you spend more time dealing with frictional issues than you spend when developing using the .NET languages. When you build an add-in for Office using VSTO, it is called a managed add-in. I will examine managed add-ins in more detail after I describe what an Office application is.

Note: VSTO has been around since Visual Studio 2003 for document-level projects. There were two versions of VSTO for Visual Studio 2005: VSTO 2005 and VSTO 2005 SE. See Features Available by Product Combination for the differences in features. See this topic for the features in Visual Studio 2008, and this topic for features in Visual Studio 2010.

What is an Office Application

To summarize, Office 2010 consists of a number of applications (in alphabetical order, Access, Excel, InfoPath, Lync, OneNote, Outlook, PowerPoint, Project, Visio, Word), each of which can be extended in specific ways.

Typically, an Office application uses a back-end system that enables collaboration between users or richer interaction with documents, spreadsheets, and presentations. An application could connect to a high-performance installation of SQL Server or to web services; your managed add-in saves and retrieves data from the SQL server or the web services. Alternatively, an Office application can be an extension of an existing line-of-business system – pulling data from that system, and generating documents or spreadsheets based on information stored in that line-of-business system.

With the release of Office 2010 and SharePoint 2010, the most effective way to enable collaboration for users of your Office application is to base the application on SharePoint 2010. SharePoint 2010’s integration with the Office client makes it easier to build features and implement scenarios that make it seamless to operate your Office application from either the Office client or from a browser.

In a separate article, SharePoint Developer Building Blocks: Technologies for Creating SharePoint Applications, I enumerate the developer building blocks for SharePoint 2010. Using the SharePoint developer building blocks in combination with the Office developer building blocks provides developers with a powerful set of tools for building enterprise applications.

In the next post, I'm going to examine Office development scenarios.

Office/SharePoint Building Blocks and Developer Stories

Eric White - Independant Consultant — Tue, 23 Nov 2010 16:44:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

For the last few months, it has been my assigned job to think and write about Office and SharePoint development from a very high level. I haven’t written any code (sadly). Instead, I’ve been simply involved in a survey, first, of SharePoint 2010, followed by the Office 2010 client applications. The first result of my efforts (as well as the efforts of many other individuals) was SharePoint Developer Building Blocks: Technologies for Creating SharePoint Applications. For a variety of reasons, I delayed publishing anything until that article was written and reviewed by appropriate people. But I’m going to take a different tack with Office client development. Over the next 2-3 weeks, I’m going to first blog my thoughts on Office client development from an overview level, and then cover each of the Office clients in a separate post. As with the SharePoint article, I’m going to focus much more on what you can do, and why you want to do it, and less on how to do it. I’ll link to other videos, SDK documentation, Visual How-To articles, etc. that explain the ‘how’.

So why the term ‘building blocks’? I suppose the answer is that we haven’t found a better term.

From my limited perspective on the bottom of the heap, inside the Microsoft Office division, as new version of the products are being developed, the folks in charge have various meetings where they introduce new features in SharePoint or Office, and the common term that is bandied around is ‘developer story’, as in, ‘What are the developer stories for Silverlight and SharePoint’, or ‘What’s the developer story for AJAX and SharePoint?’ Then, the job of writers and others is to ‘tell that story’.

The term ‘developer story’ probably doesn't make sense to developers who are not inside the Microsoft bubble here in Redmond, but to me, the term ‘developer story’ is basically synonymous with ‘developer building block’.

One of the problems is that there isn’t any one thing that comprises building blocks. In some cases, you code some declarative XML. In others, you write C#/VB.NET code that uses a library. In still others, you use SharePoint designer to put workflows together (workflows are essentially a declarative programming language). Therefore, we couldn’t use the term library, language, programming interface, or any of the other terms that we use in taxonomies of developer technologies. The same issue applies with Office client development. There are specialized languages (VBA, Access Data Macros), COM libraries, .NET libraries that are layered on top of the COM libraries, declarative approaches, and more. So building block is the term we’ll use.

When I’ve been examining first SharePoint, and then the Office client applications, sometimes it is a judgment call about whether some particular developer technology is listed as a building block. My litmus test for this is ‘what do developers need to know about?’ To a certain extent, my judgment has been colored by my personal experience in coming up to speed as a SharePoint/Office developer. If something confused me while learning about it, then I felt I needed to pick building blocks to prevent (if possible) others from having the same confusion.

In the next blog post, I’m going to write about the varieties of applications you can build with Office. I’ll also define what an ‘Office client developer scenario’ is.

As I've been making my way through the vast subject of Microsoft Office client development, I've had many 'cool, I didn't know you could do that' experiences. The best part of writing this series of blog posts is that it is going to be FUN to share those experiences.

Easy SharePoint 2010 Setup for Developers on Windows 7

Eric White - Independant Consultant — Tue, 02 Nov 2010 18:16:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

Chris Johnson and Paul Stubbs have released some PowerShell scripts that make it easy to setup SharePoint 2010 on a Windows 7 developer workstation. These scripts will install and configure the prerequisites to get up and running with SharePoint development. All of the source is provided, so you can customize it. Out of the box it will help you install:

• SharePoint Server 2010 + pre-requisites (Standalone)

• Visual Studio 2010 Ultimate Edition

• Silverlight 4 Tools for Visual Studio

• Expression Studio 4 Ultimate

• Open XML SDK

• Visual Studio SDK

• Visual Studio SharePoint Power Tools

• Office 2010 Professional Plus

• SharePoint Designer 2010

• Visio 2010

Chris Johnson has a walk-through guide, as well as details on where to download.

SharePoint Developer Building Blocks: Technologies for Creating SharePoint Applications

Eric White - Independant Consultant — Tue, 26 Oct 2010 14:20:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

You can use a variety of languages, frameworks, object models, and tools to build scalable distributed web applications on the SharePoint 2010 development platform. It can be difficult for a developer new to SharePoint to know what to use to accomplish various scenarios, or even what is possible. This article provides a high-level overview of SharePoint as a development platform, compares and contrasts SharePoint development to other types of development, and lists the SharePoint Developer Building Blocks, which are the technologies that you use to build a SharePoint application. Perhaps the biggest challenge for you as a SharePoint developer is to understand what your choices are for building blocks, and to select the correct options that meet your architectural and design goals.

This is the MSDN article that I’ve been literally obsessing about for the last few weeks, to be published sometime in the near future. After it is published on MSDN, I’ll update this post with a link to the MSDN article. AJ May and Dallas Tester contributed to this article. It was technically reviewed by Mike Morten, Mary Lee, Iouri Simernitski, Dallas Tester, Siew Moi Khor, Erika Ehrli Cabral, and Jim Corbin.

I wrote this article with myself as the prospective reader (before learning about SharePoint 2010).

Introduction
What is a SharePoint Application?
Comparing and Contrasting SharePoint Development with other Forms of Development
Building Scalable .NET Web Applications
SharePoint Development vs. Database Application Development
SharePoint Development compared to Traditional Rich Client Development
Languages and Platforms
SharePoint Application Development Terms
Varieties of SharePoint Applications
SharePoint Capabilities
SharePoint Developer Scenarios
SharePoint Developer Building Blocks
Developer Dashboard
Accessibility
SharePoint Development Tools
Setting up a Development Environment
Application Lifecycle Management (ALM)
Conclusion
Additional Resources

Introduction

From a high level, it is useful to examine SharePoint development from a number of different viewpoints, such as:

· How SharePoint development compares to and contrasts with other forms of development:

o   Scalable loosely coupled ASP.NET application development
o   Database application development
o   Traditional rich client development

· What languages and frameworks must developers know?

· How do document formats such as Open XML affect your application design? What new scenarios are possible?

This article presumes that the decision to use the SharePoint development platform has been already made. This is the position that many developers are in. In some cases, developers are assigned to a project that uses SharePoint as a platform. In others, developers are proactively developing their skills.

From a high level, the SharePoint 2010 developer architecture consists of a number of layers of technology. It is an ASP.NET application that is built on SQL Server, and runs on Windows Server. The following diagram shows how a number of the more critical building blocks use underlying systems to provide necessary functionality.

Figure 1: Diagram of key building blocks and underlying systems

It can be a bit daunting to see the breadth of technologies that a SharePoint developer must know. When I jumped into SharePoint development, I started from a spot similar to many other developers who enter SharePoint development. I was a competent developer with in-depth experience in a few areas (in my case, C/C++/C#/.NET/LINQ/SQL/XML/Open XML), shallower experience in a number of areas, and a cursory knowledge of others.

I had to deepen my knowledge in the following technologies:

· ASP.NET

· ASP.NET Themes and Skins

· Html and CSS

· JavaScript

· ASP.NET Web Parts

· IIS

While I understood the architecture, as a developer I needed to learn the details of the following technologies almost from scratch.

· The SharePoint object model

· The SharePoint managed client object model

· Silverlight and XAML

· Workflows

· WCF and web services

· SharePoint REST APIs

· Excel Services

· Windows Identity Foundation and claims-based identity

Each of these technologies requires some amount of study. The basic concepts are not difficult, but learning enough to be proficient can be a lengthy process.

In this article, I’m going to list some of my thoughts about various aspects of SharePoint development. I’m going to correlate aspects of SharePoint development to previous experiences. What was similar to previous development environments that I’ve worked in? What was different? I’m also going to discuss how I think of SharePoint development. What comprises a SharePoint application? What are my options for building one? What is the big picture, and where do I need to drill down when implementing a particular scenario? My goal is to outline the key aspects that you need to know in order to be effective at SharePoint development.

What is a SharePoint Application?

To summarize, SharePoint Server is a system with the following capabilities:

· Sites – Web Sites both inside and outside the firewall.

· Communities – Collaboration through wikis, blogs, and more.

· Content – Enterprise Content Management (ECM) – Web Content Management (WCM) – Publishing Systems.

· Search – SharePoint includes a search engine that you can customize for your needs.

· Insights – Business Intelligence (BI).

· Composite Applications – includes data access, user interface, processes, and infrastructure.

SharePoint is a scalable architecture that can be appropriate for a few users, or 100,000 employees of a large corporation. For web sites outside of the firewall, SharePoint for Internet Services allows you to build scalable, flexible web sites that you can maintain with the ease of maintaining a SharePoint site.

In most cases, when you build a SharePoint application, you are adding additional capabilities to the core functionality. You may want to streamline the user interface for some important users. You may want to provide some custom capabilities in several Web parts for a particular vertical market such as construction, law, or manufacturing.

In a few cases, you may build a stand-alone application that makes programmatic use of SharePoint’s capabilities. The end user may never see the SharePoint user interface, or even be aware that the application is developed using SharePoint as part of its infrastructure.

Customization can be light or deep. Your entire customization may consist of a single web part that enables some scenario in your group or team. Or you may be building a complex commercial application that has data flow and processes. You may have integration (deep or shallow) with external Customer Relationship Management (CRM) or Enterprise Resource Planning (ERP) systems.

SharePoint’s integration with the Microsoft Office 2010 system is important. You can take advantage of this integration to build solutions such that no matter where your users are – in an Office client application or in a web browser – the features of your application are discoverable and convenient for them. At various places in this article, I’ll discuss where SharePoint development intersects with Office 2010 client development.

Comparing and Contrasting SharePoint Development with other Forms of Development

Building Scalable .NET Web Applications

You can look at SharePoint development from the perspective of a developer who builds highly scalable, loosely coupled web applications that run on large server farms. These applications need to be capable of handling hundreds or thousands of page views per minute.

At its core, SharePoint is an ASP.NET application that runs on IIS and can have multiple web front end machines that handle load balancing. SQL Server provides the integrity, scalability, robustness, and security of your data and documents that you store in your SharePoint sites. Following are some important ways that scalability affects SharePoint development.

· API Design - Scalability drives certain characteristics of SharePoint’s programming interfaces. It is easier to grasp SharePoint’s programming interfaces when you understand that they are designed to facilitate scalability. For example, the SharePoint 2010 Managed Client Object Model, while very similar in abstractions to the SharePoint server object model, is more complex because it allows you to explicitly control when you retrieve data or content from the server.

· Solution Design - Scalability influences how you design a solution built on SharePoint. You need to avoid designs that cause unnecessary computation or querying activity on the server. You need to write applications that do not consume more resources than they should. For instance, this means appropriately using Collaborative Application Markup Language (CAML) and LINQ to SharePoint to query for list items.

· Best Practices - Scalability is behind certain programming approaches and issues that are ‘best practices’ of SharePoint development. For example, certain objects in the SharePoint Object Model have unmanaged data associated with them, so you need to understand and follow the rules around object disposal. Similarly, there are several best practices to keep in mind when working with large lists in SharePoint. If you don’t follow those rules, you can negatively impact the farm. See Best Practices with SharePoint Server for more information. See also Best Practices: Using Disposable Windows SharePoint Services Objects and Dispose Patterns by Example. You can Automate SharePoint Dispose() code reviews with SPDisposeCheck.

These are issues that are identical or similar to the issues that developers face who are building highly scalable web applications. I heard a story about a SharePoint developer who wrote code to iterate over all documents in his site collections at regular intervals and assemble information to be presented in a tree control. This worked fine in his test environment, but the code design presented a performance problem with real-world quantities of documents and list items.

Scalability affects solution design in two distinct ways.

· Building distributable applications - Build solutions that will work properly when deployed on multiple web front ends (WFE). For example, you could build a little Create/Retrieve/Update/Delete (CRUD) web service for Business Connectivity Services (BCS) that stores data in a local XML file, but that would not work when deployed on a load-balancing farm.

· Building applications that perform well – For example, unless you know for certain that a list will contain few enough list items, don’t iterate over them using the object model; use LINQ to SharePoint instead, and give SharePoint a chance to optimize.

For more information about how SharePoint is similar to and different from ASP.NET development, see Glide Path for the ASP.NET Developer. See also SharePoint 2010 Development for ASP.NET Developers.

Best Practices with ASP.NET Developers contains guidance that will help you avoid pitfalls that can adversely impact performance, including guidance around object disposal, event receivers, large folders and lists, and code performance optimization.

SharePoint Development vs. Database Application Development

Database application development is one of the good vantage points for viewing SharePoint development. Custom lists in a SharePoint site have much in common with database tables. You can define columns in lists, with rich metadata about those columns. Further, SharePoint lists can effectively have ‘foreign keys’ defined, so that you can model more interesting scenarios that contain related data. SharePoint provides cascade and restrict behavior for delete operations. You can create these lists programmatically or declaratively, or you can write programs that work with lists that users have defined. These lists can be visible or hidden. You can use the security features of SharePoint to restrict access.

One aspect of SharePoint that is a direct parallel to SQL databases is that you use a non-procedural declarative query language to retrieve data. However, instead of using SQL, you use LINQ to SharePoint or CAML, which you write using XML.

SharePoint’s integration with database technology is deep. You can use BCS to consume databases, web services, and just about any data source. These data sources are represented as an External Content Type (ECT).

An interesting characteristic of the data capabilities of SharePoint is that it doesn’t have transactional guarantees. You can’t, for instance, insert a list item in one table, and update a list item in another table, and be guaranteed that neither or both will take place. SharePoint is not intended for use as a platform to implement transactional systems. Instead, such transactional systems should be implemented in external databases that can provide the appropriate guarantees. That data can then be brought into SharePoint using BCS. As you design your SharePoint application, you must take this characteristic into account.

A notable difference between the data capabilities of SharePoint and traditional database development is that SharePoint lists (comparable to database tables) are not necessarily rectangular. In SharePoint, a content type defines the fields that comprise a list item (comparable to a row) in a list. This can be thought of as the schema for a list item. A SharePoint list can contain list items of more than one content type. The following figure depicts a SharePoint list that contains items of two content types: LABOR, and MATERIAL.

This impacts how you design and develop applications that consume list items and the fields in list items. If you require rectangular data, you can define lists such that they can contain only one content type. If you allow non-rectangular data, when you iterate through list items, as necessary, you need to check for the content type and alter the behavior of your code accordingly.

Another interesting characteristic – due to internal implementation details (not least of which is the point made in the previous paragraph), querying SharePoint using LINQ to SharePoint or CAML is quite a bit slower than querying tables using SQL. You do not want to create a design where some list unintentionally grows to 50,000 or 100,000 items, and the user is free to attempt to bring the list up in a window in a browser. SharePoint 2010 contains throttling features that prevent such designs from taking down the farm, but the experience to the user would be random and undesirable. There are ways to handle lists with huge numbers of items. You have to handle this issue intentionally. For more information, see Handling Large Folders and Lists.

SharePoint is built upon SQL Server. Document libraries and SharePoint lists are stored in a database. You see this whenever you see architectural diagrams of a SharePoint installation. The actual SharePoint database isn’t particularly relevant to you as a SharePoint developer – you never directly access this database. You always use the programming interfaces to alter sites, lists, and document libraries. However, you can use that same SQL Server installation to host a database that you consume directly or via BCS, so it can be a part of the infrastructure that you use to build a SharePoint application.

Access Services allows you, with limitations, to publish Access databases to a SharePoint site. This opens up interesting possibilities for allowing SharePoint users to share data in a convenient form.

Client-Side Data Access Technologies

You can programmatically access SharePoint data from client computers. When accessing data from client computers, SharePoint uses networking technologies to send the request to SharePoint Foundation or SharePoint Server and to retrieve the results of the request from SharePoint.

Client-Side BCS Object Model	Enables online and offline access to data in ECTs. You can access this object model from a Microsoft Office Word, Excel, or PowerPoint managed add-in. This is sometimes the most powerful and convenient way to access data that is external to SharePoint in an application.
Custom WCF Services	Enables high-performance designs where you host custom WCF services on the SharePoint server. The WCF service can access data and documents through the server object model, and allow the client to access the data and documents through the service that you implement.
Excel Services ECMAScript / JavaScript Browser Object Model	Enables you to customize, automate, and drive the Excel Web Access Web Part control on a page
Excel Services REST API	Enables you to access workbook parts or elements directly through a URL.
Excel Services Web Services	Enables applications to calculate, set, and extract values from workbooks, and to refresh external data connections.
ECMAScript / JavaScript Client Object Model	Enables JavaScript / ECMAScript programs to access SharePoint data. This programming interface is a variant of the Managed Client Object Model.
Managed Client Object Model	Enables .NET applications (including stand-alone applications, Silverlight applications, and ASP .NET applications) to access SharePoint data. This framework is similar to the server object model, with additional semantics that allow you to control network traffic to and from the server. It is important to know and use CAML when using the client object model.
REST / OData Data Services	Provides a RESTful programming interface to lists, list items, and more.
SharePoint Web Services	Provides Web service methods that you can use to work remotely with a deployment of SharePoint Foundation.
Web-based Distributed Authoring and Versioning (WebDAV)	Consists of a set of methods, headers, and content-types that augment HTTP/1.1 to allow for the creation and management of resource collections. SharePoint support for WebDAV allows you to upload and download documents. WebDAV (RFC 4918) is a working document of the Internet Engineering Task Force (IETF).

Server-Side Data Access Technologies

When writing code that runs directly on the server, regardless of whether you are building a sandboxed solution or farm solution, you can specify your data queries in either CAML or using LINQ to SharePoint. CAML returns weakly-typed SPListItem objects, whereas LINQ to SharePoint returns strongly typed objects that are defined in generated code.

CAML	XML based non-procedural query language.
LINQ to SharePoint	Language-Integrated Query

In addition to using CAML or LINQ to SharePoint to query for a set of items, you can use the server object model to directly iterate through items in a list or library. However, you should not do this unless you need to process all items in the list, and you know how many items are in the list. I often design applications to be data driven. I will design a list to contain some specific data that drives behavior. My design might be such that I always know that there are a certain number of records, and that I need to process all records in order to drive my application behavior. In this case, I would iterate directly through the items without querying using CAML or LINQ to SharePoint. Note that this same approach does not work when using the Managed Client Object Model. With the Managed Client Object Model, you must supply a CAML query when retrieving a collection of list items.

SharePoint Development compared to Traditional Rich Client Development

In a keynote speech, Steve Ballmer, CEO of Microsoft Corporation, called SharePoint Server ‘an operating system for the enterprise.’ SharePoint has much in common with an operating system:

· SharePoint has storage. Document libraries that contain hierarchies of folders are remarkably similar to files in directories in a file system.

· SharePoint has a programmable user interface.

· As with an operating system, you can write code that runs at varying levels of privilege. With SharePoint development, you can write code that needs to run with farm administrator privileges. You can write code that runs in a sandboxed solution, and you can write ECMAScript / JavaScript that runs in the client browser.

· As with an operating system, you can write services, which provide necessary infrastructure for building SharePoint applications with complex dynamics.

Scale, once again, is the biggest difference. Scale drives the programmability characteristics of SharePoint. You can write software that runs with great performance across hundreds of servers, providing an entire enterprise with a consistent experience that increases collaboration and productivity. Dealing with the issues of scalability is the price of admittance when moving from traditional application development to SharePoint development.

Languages and Platforms

One question that is at the top of all new SharePoint developer’s minds is the following: just how much do I need to learn? It is interesting to list out the languages and platforms that you need to know when doing SharePoint development. The first item in the list below is the most important. Beyond that, it depends on what you’re doing. It is not necessary to master every technology in the following list. You need to learn only what is applicable to your current task.

· C# / VB.NET / .NET Framework – You use the .NET platform in one form or another when building full-trust farm applications, sandboxed site-collection level solutions, or within a Silverlight application. You will be a more effective developer if you understand the various LINQ technologies: LINQ to XML, LINQ to Entities, LINQ to SharePoint, and LINQ to SQL.

· ASP.NET – SharePoint is an ASP.NET application hosted in IIS. It makes use of master pages and application pages, which allow you to standardize page look and feel. It also makes use of Web Part pages and Web parts, which allow users to configure their own pages. Web parts can be connected to each other so that you can create parent/child data editing capabilities or web mashup applications.

· HTML/CSS – There are a number of places in SharePoint development where it is helpful to be able to write and understand HTML and CSS.

· ECMAScript / JavaScript and the HTML Document Object Model (DOM) – ECMAScript / JavaScript is a powerful tool in your tool-chest. You can use the ECMAScript / JavaScript client object model or the OData REST API from ECMAScript / JavaScript to get at data and documents in your sites. JQuery and AJAX allow you to add rich interactivity.

· Silverlight/XAML – Silverlight is a development platform for creating engaging, interactive applications for many screens across the Web, desktop, and mobile devices. You can use the managed client object model from within the Silverlight code to read and write site data and documents. Expression Blend is an effective tool for designing Silverlight Web parts for SharePoint.

· XML – You need to know how to code a valid XML document. Regarding developing with XML, my personal recommendation is to learn and master LINQ to XML. It is a more effective way of working with XML. Knowing XSLT can be useful – you can manually write custom XSLT list views, although the primary purpose of XSLT list views is to allow you to apply interesting formatting to a list using SharePoint Designer. In other words, XSLT is more often used as the output of tooling rather than as a commonly used developer tool.

SharePoint Application Development Terms

To make it easier to talk about SharePoint application development, I’m going to define four terms (taken from the SharePoint marketing sites). I’m defining them here so that I can discuss them more easily through the remainder of the article.

SharePoint application – A SharePoint application consists of a number of customizations that taken together fills a particular market or customer need. An application consists of some combination of custom Web parts, workflows, event handlers, content types, pre-defined lists, or any of the other technologies that you use to enhance SharePoint functionality.

SharePoint capabilities – SharePoint capabilities are the key areas of SharePoint functionality. Developers build on these areas of functionality to put together a solution where the whole is more than the sum of the parts. These capabilities are in the areas of Sites, Communities, Composites, Content, Insights, and Search.

SharePoint developer scenarios – These are activities that a developer wants to enable for SharePoint users. For example, a developer may want to allow the user to generate several hundred Word documents based on data stored in a SharePoint list, generate PDF or XPS files from those documents, and to print them or send them by email. To accomplish this scenario, you will use a number of developer building blocks. One possible implementation would be to use the Open XML SDK in a Web Part that is implemented as a sandboxed solution, then use SharePoint 2010 Word Services to convert from DOCX to PDF.

SharePoint developer building blocks – These are the technologies in SharePoint that you use to do your development. Examples are Web parts, workflows, event handlers, data, and services. When you build an application, you develop using the building blocks so that the parts all work together.

Varieties of SharePoint Applications

What are the varieties of applications that SharePoint developers build? This is similar to asking the question about the varieties of applications that you can build on an operating system. Developers are creating new uses for SharePoint that cover a wide range of scenarios. Here are a few varieties of applications.

· Vertical Markets – Vendors customize SharePoint for a variety of markets such as real-estate management, resort and entertainment management, or equipment rental.

· Customized team applications – Sometimes a custom application can help a department of a large organization meet their goals in a more organized and visible manner. A testing organization could build a system to maintain a library of testing instruments and resources.

· Content Publishing Systems – These systems can be as simple as a site or site collection that is maintained by a single author. They can be as complex as a technical document generation system used by hundreds of writers, developers, and managers.

· Accounting Systems – While Microsoft does not recommend that transactional systems be implemented in SharePoint, bringing data into SharePoint from external accounting systems significantly augments the user experience.

· Management Information Systems – Some automated management systems do not have the same transactional requirements as accounting systems. You can maintain lists of customers, products, assets, and resources using SharePoint out-of-the-box, and when extended programmatically, you can provide as sophisticated of an experience as you want.

· Sales and Marketing – Many aspects of market and sales systems resemble management information systems. Integration of SharePoint with CRM systems such as Microsoft Dynamics results in powerful hybrid applications. In addition, mobile access to SharePoint sites provides road warriors with tools to interact with data necessary to do their job.

· Project management – The integration of Microsoft Project Server 2010 with SharePoint Server Enterprise provides project and portfolio management to help organizations prioritize investments, align resources and execute projects. You can develop for Project Server using its API.

While many examples of SharePoint applications fit into one of these categories, this list is certainly not complete. I’m certain that someone somewhere has written a game that runs on SharePoint that takes advantage of the collaboration features. I also would guess that someone has built a system that controls computer automated manufacturing systems using SharePoint lists, event handlers, etc. Vendors serving other vertical markets such as computer aided design or architecture will invent their own variety of enhancements of SharePoint.

SharePoint is a general purpose platform. The types of applications that you can build are nearly as varied as the types of applications that you can build for the Windows operating system.

SharePoint Capabilities

You can find a lot of information about SharePoint capabilities from the Microsoft SharePoint 2010 Product Overview. The capabilities include the following:

· Sites – You can develop and manage all of your business web sites using SharePoint 2010. SharePoint 2010 provides a full set of tools that users can use to create any kind of site. SharePoint provides infrastructure that simplifies site management.

· Communities – SharePoint 2010 provides collaboration tools that anyone can use to share ideas, find people and expertise, and locate business information. SharePoint as delivered has many collaboration capabilities, but extending SharePoint in interesting ways can make collaboration proactive instead of reactive. Integrating presence into SharePoint applications using Office Communications Server (OCS) makes it easier for colleagues to contact each other.

· Content – Publishing organizations come in a vast variety of forms – from international press organizations to a small department that is responsible for publishing documentation for a product. Organizations can use SharePoint 2010 to manage content and use Word 2010 to edit content. I further divide the content capabilities into the following three sub-categories.

· Publishing Systems – You can use SharePoint Server 2010 to manage a comprehensive publishing system. You may deliver the final product in PDF, XPS, or some other form. Open XML is an especially powerful technology that you can use to build publishing systems. You may need to write a sophisticated transformation from Open XML to other forms such as XHtml.

· Web content management – Whether using SharePoint for Internet Services to build externally facing web sites, or using SharePoint for building wikis and pages inside the firewall, SharePoint allows users to build out rich content with a minimum of effort.

· Enterprise content management – Many organizations have a vast amount of ‘organizational experience’, often in the form of documents that are managed with revisions, approval processes, and metadata.

· Search – Customizing the search experience can be a core component of making the information come alive for your users. You can customize to allow SharePoint’s search engine to search through custom documents. You can customize search to allow it to search through ECTs.

· Insights – A recent survey asked CEOs of very large companies which issues were ‘top-of-mind’ for them. BI was mentioned most often as the top priority. Excel Services and the ability to build dashboards are important for building powerful, flexible BI systems. PerformancePoint services for SharePoint provide another option to deliver customizable BI dashboards.

· Composites – Create innovative solutions that combine customer user interfaces, data access, business logic, and services. You can document and standardize business processes using workflows.

SharePoint Developer Scenarios

There are even more varieties of scenarios than there are varieties of applications. However, there are a few general types of scenarios.

· Scenarios that provide better user interfaces to existing SharePoint data. For instance, you may want to display calculated values that are based on data in several SharePoint lists.

· Scenarios that involve pulling in external data from external systems. You may be connecting to a modern ERP or CRM system, or you may be connecting to a proprietary database that is implemented on UNIX using an emulator of ancient hardware architecture.

· Scenarios around document generation. You may generate hundreds or thousands of Open XML document based on data stored in SharePoint lists or external systems, then convert to PDF, etc.

· Scenarios around process and procedure. Scenarios can be as simple as a document approval process, or can be as involved as shepherding a RFP through generating a proposal to signing of a contract.

You implement each developer scenario by using one or more SharePoint developer building blocks. Of course, as systems designers, you must determine your own scenarios. Then, your task is to be familiar with the available building blocks that you can use to implement your scenario. In many cases, you have more than one option; each option has different characteristics, and one option might suit more than another for a particular scenario. To a large extent, you probably already know the direction you need to take. A high-end marketing and design site or a luxury goods web site may from the outset plan to use Silverlight wherever necessary, whereas a part-time developer for a team site might be interested in whatever will get the job done in the fastest and easiest way.

SharePoint Developer Building Blocks

At this point, I’ve discussed what a SharePoint application is. I’ve also identified the several SharePoint capabilities that people find so valuable. I’ve discussed developer scenarios, and provided a few examples of types of scenarios. Next, I’ll cover the SharePoint developer building blocks. What do you, as a developer, actually develop when building a SharePoint application?

Important Building Blocks

While the following table is presented in alphabetical order, not all building blocks have the same importance to you. This section lists a few of the building blocks deserve to be called out separately.

· First and foremost, you need to learn about the server object model. You will use this programming interface when you are building Web parts, workflows that incorporate custom actions, event receivers, and more. Further, the managed client object model is based on the server object model. Much of what you learn about the server object model applies to the client-side object model.

· Web parts are the most common way to extend SharePoint. Learning how to build a Web Part is a great way to started in SharePoint development.

· Workflows enable you to codify and standardize business processes, and are one of the essential tools for implementing certain scenarios. If you extend workflows using code, you use the server object model.

· BCS makes data from outside SharePoint available to your SharePoint application.

· Understanding CAML and LINQ to SharePoint are key to building good SharePoint applications.

· To you as a developer, in addition to being end-user features of SharePoint Foundation, document libraries and lists are programming resources. Often, when designing a software system, you need to store configuration information that drives behavior of the application. You can store such information in hidden lists, and restrict access to users as necessary. This is most appropriate when you need to enable non-developers to maintain the application configuration. Another approach is to place configuration XML files in a hidden, access restricted document library.

The following table lists the SharePoint developer building blocks that you can use to build a SharePoint application.

Figure 2 – SharePoint 2010 Developer Building Blocks

Building Block	Description
Access Services	Publish Access databases to SharePoint
Administration	Programmatically administer SharePoint Foundation and Server
Alerts	Send email or SMS
ASPX Pages	Build Web pages with .NET code-behind
BCS	Bring external data into SharePoint
BCS Client-Side API	Access BCS data from client computers
CAML	Declarative query language expressed as XML
Column, Field Types	Define domain definitions for data in lists and libraries.
Content Types	Define prototypes for list items.
Custom Action	Create links, toolbar buttons, menu items
Custom Views	Define view for lists and document libraries
Dashboards and KPIs	Web part for reports, charts, and KPIs
Document Libraries	Storage for documents or files of any type
Document Metadata	Sophisticated document categorization
ECMAScript Client Object Model	Access SharePoint data from ECMAScript / JavaScript
Event Receivers	Provide behavior when SharePoint lists or libraries are updated
Excel Services	Load, calculate, and display Excel workbooks
Excel Services ECMAScript Interface	Access the Excel Web Access Web Part control from ECMAScript
Excel Services REST API	Access workbook parts or elements directly through a URL
Excel Services User Defined Functions (UDF)	Write high-performance functions for use in server-side spreadsheets
Excel Web Services	Access Excel Services programmatically using Web services
Features	Unit of deployment
Health Rules	Codify how you monitor health in a large installation
InfoPath Forms Services	Make forms to be filled out in a browser
LINQ to SharePoint	Use LINQ to query SharePoint lists and document libraries
Lists	User editable storage of data
Managed Client Object Model	Access SharePoint data from client computers
Mobile Pages, Controls, and Adaptors	Brings SharePoint data to phones
OData REST APIs	Provides REST access to SharePoint data using the OData proposed standard.
Office Client Managed Add-Ins	Enhance Office clients to use SharePoint data and documents
Open XML SDK	Use Open XML for document generation and transformation
Performance Point Services	Build dashboards, scorecards, reports, and key performance indicators (KPIs)
PowerPivot	Create business intelligence solutions
Sandboxed Solutions	The Sandbox infrastructure isolates and shuts down bad code, preventing it from taking down the farm
Search Customization	Enable search through varied content, including ECTs.
Secure Store Service	Securely access legacy systems
Security	Ensure that sensitive information is available only to authorized users
Server Object Model	Server-side .NET API for accessing data and content
Server Ribbon	Customize the SharePoint ribbon
Service Application Architecture	Create shared services that can be consumed by multiple SharePoint applications
SharePoint Dialog Framework	JavaScript programming interface for putting up dialog boxes and other client-side user interface elements
SharePoint Themes	Apply custom branding to a SharePoint site or site collection
SharePoint Web Services	Access SharePoint data through Web services
Silverlight Web Part	Create user interfaces that are better looking and easier to use
Site Definition	Create, configure, and provision SharePoint sites
Site Pages	User customizable pages that conserve resources
Timer Jobs	Execute code at specific times or intervals
Variations	Tailor sites for different cultures, markets, and languages
Visio Services	Share and view Microsoft Visio drawings
Visual Studio SharePoint Tools Extensibility	Extend Visual Studio to enable developers to create custom solutions more quickly
WebDAV	Upload and download files efficiently
Web Parts	Web parts and Web part pages enable users to customize their user interface. Building Web parts is the most common way to extend SharePoint.
Web Services	Key component of building scalable, distributed, and loosely coupled SharePoint applications.
Windows Identity Foundation	Provides claims-based identity services based on trusted providers
Word Automation Services	Provides services to convert documents to and from various file formats
Workflows	Automate business processes using tasks and forms

Access Services

You can use Access Services in Microsoft SharePoint Server 2010 to edit, update, and create linked Microsoft Access 2010 databases by using an Internet browser or the Access client. You can use SQL Server Reporting Services to produce formatted reports.

Video: The Access Show: Access 2010 demo of Access Services and web databases

SDK: Access Services administration

SDK: Using Access Services with SQL Reporting Services

Administration

The classes of the Microsoft.SharePoint.Administration namespace provide developers with tools for programmatically deploying, migrating, maintaining, and administering their SharePoint sites and SharePoint-based solutions.

SDK: SharePoint Foundation Administration

SDK: Microsoft.SharePoint.Administration Namespace

SDK: The Administrative Object Model of Microsoft SharePoint Foundation

Alerts

Alerts provide a framework for automatically sending email or SMS notifications to SharePoint users. Alerts can be attached to lists or list items. Microsoft SharePoint Foundation provides a library of alert templates for common scenarios.

SDK: Building Block: Alerts

SDK: Alerts in SharePoint Foundation

SDK: Alerts Overview

SDK: Alert Object Model

ASP.NET Application Pages

To build a custom user interface with specific application logic (written in C# or VB.NET in code-behind), a developer can use ASP.NET application pages (also called ASPX pages), which are a special variety of ASP.NET pages.

Your ASPX page will need to integrate with an ASP.NET master page. The relationship between master pages and ASPX pages enable you to provide a consistent user interface across SharePoint sites and site collections.

Because they are traditional ASPX pages that can contain code-behind and be precompiled, they offer some performance advantages over Site Pages.

You cannot use application pages to host Web part zones or Web parts.

One characteristic of ASPX pages is that you have a number of options about where you place Silverlight controls. For certain users, you can exclusively provide a Silverlight interface to large parts of the site, yet site maintainers can use SharePoint’s user interface.

SDK: Master Pages

SDK: Sites and Pages

SDK: SharePoint Page Types

MSDN: Creating Application Pages for SharePoint

SDK: Building Block: Pages and User Interface

SDK: Cascading Style Sheets Class Usage in SharePoint Foundation

Business Connectivity Services (BCS)

BCS provides infrastructure so that you can define a few methods (with specific prototypes) and bring data into SharePoint in such a way that users can view and modify that data directly from SharePoint. External tables are called External Content Types (ECTs), and sometimes entities. You can use ECTs as entities (as in entity/relationship theory).

There are three basic BCS mechanisms that you can use to bring external data into SharePoint.

· You can connect to a SQL database. SQL Server is supported out-of-the-box. With a bit of effort, you can connect to Oracle or other varieties of database servers.

· You can consume a Web Service that exposes a set of methods the follow specific patterns for the method prototypes.

· You can write .NET managed code (C# or VB) to connect to any variety of data. There are two ways that you can extend through code:

· You can write a .NET Assembly Connector to allow users to create ECTs to a set of predefined schemas. The schemas can’t change without recompiling and redeploying.

· If you are building upon an underlying system that contains its own schema mechanism and definitions, you can write a Custom Connector that uses the underlying schema definition and allows users to create ECTs for any table in the underlying schema. This is, of course, more work than writing a .NET Assembly Connector.

External lists and external data columns are part of built-in user interface functionality that allows you to show and maintain data in ECTs. You can create external lists either using the Web interface, SharePoint Designer, or Visual Studio 2010.

You can expose your ECTs to the search engine, so that users can find data in them. By default you display data in a profile page. You can customize the page, or customize what happens when a user clicks on an item in the search results. For example, you could use a non-SharePoint website to display and maintain the item.

SDK: Business Connectivity Services Overview

SDK: Microsoft Business Connectivity Services

SDK: Business Connectivity Services: How-tos and Walkthroughs

SDK: Differences Between Using the .NET Assembly Connector and Writing a Custom Connector

How to: Create External Lists in SharePoint

How to: Create an External Content Type Based on a SQL Server Table

How to: Connect to an Oracle Database Using Business Connectivity Services

How to: Create a SharePoint List with External Data Columns

Visual how-to: Creating SharePoint 2010 Web Parts That Can Read and Write Data to External Data Sources

Walkthrough: Creating an External List in SharePoint by Using Business Data

Video and hands-on lab: Accessing External Data with Business Connectivity Services in SharePoint 2010

Blog: Walkthrough of creating a SharePoint 2010 external list using Visual Studio 2010

Blog: Searching External Data in SharePoint 2010 Using Business Connectivity Services

Business Connectivity Services Client-Side API

Using the BCS Client-Side API, you can create applications that have CRUD functionality for an ECT. When building Microsoft Office managed add-ins, a convenient way to present the user interface is via task or action panes. This allows your users to interact with BCS data while continuing to work in their document or spreadsheet.

BCS Client-Side API provides a means for client applications to access data from BCS. Further, you can configure BCS so that the data can be taken off-line. When the client computer comes online again, changes are synchronized through BCS, using conflict resolution if necessary. BCS Client-Side API is most often used from within an Office 2010 client managed add-in, although it can be used from any .NET application.

You can take the data offline so that your users can continue to review and modify the data in a cache. If an offline user modifies data in such a way that it conflicts with modifications made by another user, you can write code to resolve those conflicts, and present appropriate options to your users.

SDK: Creating Advanced Code-Based Solutions Using Business Connectivity Services

How to: Show External Data in Word Using Content Controls

Blog: Overview of BCS Advanced Code-Based Solutions

Collaborative Application Markup Language (CAML)

Collaborative Application Markup Language (CAML) is an XML based query language that allows you to specify filters, sort order, and more. You can specify exactly which fields in each item you want to retrieve. This allows you to control the size of the query results.

While there are certain circumstances where you can use LINQ to SharePoint instead of CAML for querying, CAML is still relevant in SharePoint 2010. You use CAML when querying using the managed client object model. When using the managed client object model, if you filter list items using LINQ instead of CAML, in the underlying system, the filtering happens after all items have been returned. This can result in using more CPU and memory resources on the server than necessary.

SDK: Collaborative Application Markup Language Core Schemas

SDK: Introduction to Collaborative Application Markup Language (CAML)

Column and Field Types

Column and field types allow you to define rich metadata for standardized fields and columns in SharePoint lists. You can define a site column (in a site column gallery) of a specific data type that you can use throughout your site. This is similar to defining domains in database schema design. It allows you to ensure that columns in several lists use the same value space.

SDK: Custom Field Types

SDK: Site Columns

Content Types

You can define a content type that is a prototype of a list item. If you are using content types for a list, you can define that the list contain only items of one content type, or you can define that it can contain items of one of several content types.

SDK: Content Types

SDK: Introduction to Content Types

Custom Actions

A custom action represents a link, toolbar button, menu item, or any control that can be added to a toolbar or menu that appears in the UI. You define custom actions by using a custom action element within a Feature definition file. You can bind custom actions to a list type, content type, file type, or programmatic identifier (ProgID).

SDK: Custom Action Element Type

How to: Modify the User Interface Using Custom Actions

Custom Views

Defining custom views for SharePoint lists is a key tool in your toolkit. You can programmatically define custom views for specific users or scenarios so that data is presented in the most convenient form. A common use of custom views is to define the format of data for exporting to a spreadsheet for further processing.

Note: A View in this context is different from a view in the context of SQL Server. In SQL Server, a view is a database construct that defines a view into a table. In this context, a view is a visible table presented in a page so that users can see some specific subset of the data in a SharePoint list.

Office online: Create or change a view

SDK: SPViewCollection Class

SDK: SP.View Class

SDK: SP.ViewCollection Class

Dashboards and KPIs

A dashboard is a solution for information workers that consolidates personal, team, corporate, or external information, and provides single-click access to analytical and collaborative tools. It brings an integrated view of a company's data resources to an individual's desktop, providing access to key business information.

Getting Started: Introduction to the Business Intelligence Center

Office Online: Getting started with dashboards

Office Online: KPIs and Status Indicators

SDK: Development Scenarios with PerformancePoint Services

Document Libraries

SharePoint document libraries are special forms of SharePoint lists. They contain features such as the capability to keep major and minor versions of documents. They can restrict access to documents based on identity. You can define any number of columns that contain auxiliary information about each document in the library.

A very important feature of document libraries is that you can bind columnar data to metadata in the document. When you check the document out of the library, the metadata is copied to the document, where it can be maintained by the user. When the document is checked back into the document library, the updated metadata is copied back into the fields in the item in the document library.

Document libraries can contain files of any variety. Document libraries can contain nested folders – this makes a document library resemble a file system.

SDK: Files and Documents

Document Metadata

Document metadata is custom data that SharePoint stores for each document in a document library. Your users can specify this data using the SharePoint Web user interface or using Microsoft Word. You can programmatically set this data in an event handler as documents are added to a document library. Your users can then filter and find documents using this data. In addition, you can access this metadata in a SharePoint workflow, and alter workflow behavior based on metadata. You typically define document metadata using Content Types. Workflows can interact with this data.

A document information panel is a form that is displayed within the client application, and which contains fields for the document metadata. Document information panels enable users to enter important metadata about a file anytime they want, without having to leave the client application.

SDK: Metadata Navigation and Filtering

SDK: Document Information Panel Overview

ECMAScript / JavaScript Client Object Model

The ECMAScript / JavaScript Client Object Model is a variant of the managed client object model that you can use from ECMAScript / JavaScript. You use the same methods and data types. The same programming model applies. As with the managed client object model, you control when you send network traffic to and from the server.

SDK: Managed Client Object Model

SDK: Differences Between Managed and ECMAScript Object Models

Event Receivers

Event receivers enable you to write event handlers that are called when items are added, deleted, or modified in a SharePoint document library or list, as well as at other times. Event handlers can be deployed either as a full-trust farm solution, or as a sandboxed solution.

Video: How Do I: Create Event Receivers for SharePoint 2010 in Visual Studio 2010

SDK: Building Block: Event Handling

SDK: Events in SharePoint Foundation 2010

SDK: Introduction to Events in SharePoint Foundation 2010

SDK: Walkthrough: Add Feature Event Receivers

SDK: SPEventReceiverDefinition Class

Excel Services

Excel Services enables users to load, calculate, and display Excel workbooks on Microsoft Office SharePoint Server 2010. There are four primary programmatic interfaces for Excel Services.

· Excel Web Access is a viewer page and an Excel Services Web Part that you can add to any Web parts page in SharePoint Server 2010. Excel Web Access renders (in other words, creates the HTML for) live Excel workbooks on a Web page, and enables the user to interact with those workbooks.

· Excel Web Services for programmatic access

· An JavaScript object model for automating and customizing, and to drive the Excel Web Access control and help build more compelling, integrated solutions

· A Representational State Transfer (REST) API for accessing workbook parts directly through a URL

To further customize spreadsheets, you can extend Excel Calculation Services by creating user-defined functions (UDFs).

Office Online: Excel Services

SDK: Excel Web Access

SDK: Excel Web Services

SDK: Excel Services ECMAScript (JavaScript, JScript)

SDK: Excel Services REST API

SDK: Getting Started with Excel Services

SDK: Excel Services Development Roadmap

Excel Services ECMAScript Interface

The ECMAScript object model in Excel Services enables you to write code to automate, customize, and interact with the Excel Web Access Web Part control on a page. By using the ECMAScript object model, you can build mashups and other integrated solutions that interact with Excel Web Access Web Part controls. It also enables you to add more capabilities to your workbooks.

SDK: Excel Services ECMAScript (JavaScript, JScript)

Excel Services REST API

The Excel Services REST API provides programmatic access to workbook parts or elements directly through a URL.

SDK: Excel Services REST API

Excel Services User Defined Functions (UDF)

Excel Services User Defined Functions (UDF) enables you to write high-performance functions in C or C++ for use in server-side spreadsheets.

SDK: Excel Services User Defined Functions

Excel Web Services

You can use Excel Web services to access Excel Services programmatically.

SDK: Excel Web Services

Features

Features are a unit of deployment for SharePoint. Deploying a Feature in a large farm ensures that the artifacts in the Features are installed on all servers. Features can contain templates, pages, list definitions, event handlers, workflows, JavaScript source files, CSS files, and more. Features can contain Feature Receivers, which are event handlers that run when the Feature is installed, uninstalled, activated, deactivated, or upgraded.

SDK: Using Features

SDK: Feature Upgrade Overview

SDK: Walkthrough: Add Feature Event Receivers

Health Rules

SharePoint Health Analyzer monitors the health of SharePoint Foundation throughout a server farm by applying a set of health rules. A health rule is executable code that is deployed in a compiled assembly at the server farm level and registered with SharePoint Health Analyzer. When a rule is registered, a timer job is created to run the rule. An item for the rule is then added to the Health Analyzer Rule list under Review rule definitions in the Monitoring section of Central Administration.

SDK: Building Block: Health Rules

SDK: SharePoint Health Analyzer

InfoPath Forms Services

InfoPath Forms Services is a server technology that makes it possible for people to fill out forms in a Web browser. By managing forms in a central location, you can streamline business processes and improve relations with customers, partners, and suppliers.

Developer Center: InfoPath Developer Center

Office Online: Getting started with InfoPath 2010

TechNet: Plan InfoPath Forms Services

LINQ to SharePoint

LINQ to SharePoint is a LINQ query provider that enables you to write queries more naturally than when using CAML. It is most useful when using the SharePoint server object model.

One important characteristic of LINQ to SharePoint is that it uses an approach of generated code so that your code uses a strongly-typed object model for accessing the data returned by a query.

For developers interested in functional programming, LINQ to SharePoint serves as a data source for doing pure functional transformations.

SDK: Managing Data with LINQ to SharePoint

Video and hands-on lab: Accessing SharePoint 2010 Data with Server-Side APIs

Blog: Query Composition using Functional Programming Techniques in C# 3.0

Lists

Custom lists in a SharePoint site provide a place to store data. Using SharePoint lists to contain data that you access programmatically is a very important building block for developers. In the section, SharePoint Development vs. Database Application Development, I describe how you can use SharePoint lists for parts of your database.

SDK: Building Block: Lists and Document Libraries

SDK: SharePoint List Data Model

Managed Client Object Model

The Managed Client Object Model provides access to SharePoint data from external .NET applications. This API is sometimes called the client-side object model (CSOM).

The managed client object model has three basic uses:

· Managed .NET application – You can use the managed client object model from most .NET applications, including rich client applications running on a user’s computer, or as part of a .NET server application. You could write a WPF real-estate management application that runs on a client computer and pulls information from a SharePoint server using the managed client object model. You could also enhance a traditional ASP.NET application to pull data using the client object model.

· Silverlight Web Part – Code within a Silverlight control can use the client object model.

· HTML Web Part – You can use a JavaScript version of the client object model. It has exactly the same semantics, except that you use a JavaScript programming interface.

Alert: When you are using the managed client object model to query for items in a list, you should always form your queries using CAML, not LINQ. If you use LINQ, there is a performance hit that is relational to the total number of items in the list. In contrast, when using the server object model, we recommend using LINQ to SharePoint instead of CAML.

A common activity for client-side code is to upload and download documents and files to and from SharePoint document libraries. In the managed client object model, the ClientOM.File.OpenBinaryDirect method and the ClientOM.File.SaveBinaryDirect method use WebDAV for uploading and downloading files. Without building your own custom WCF service, WebDAV is the most efficient way to upload and download files.

SDK: Managed Client Object Model

Technical Article: Using the SharePoint Foundation 2010 Managed Client Object Model

Technical Article: Using the SharePoint Foundation 2010 Managed Client Object Model with the Open XML SDK 2.0

Video and hands-on lab: Accessing SharePoint 2010 Data with Client-Side APIs

Mobile Pages, Controls, and Adaptors

SharePoint Foundation provides a set of mobile pages and mobile controls that is separate from the pages and controls that you see from desktop and laptop computers. Mobile browsers can access certain Web parts by means of mobile Web Part adapters.

SDK: Building Block: Mobile Pages, Controls, and Adapters

OData REST APIs

SharePoint 2010 publishes data as resources that are addressable by URIs using the OData protocol. Data is accessed and changed by using standard HTTP verbs of GET, PUT, POST, and DELETE. OData uses the entity-relationship conventions of the Entity Data Model to expose resources as sets of entities that are related by associations.

SDK: SharePoint Foundation REST Interface

Office Client Managed Add-Ins

Office Client managed add-ins are the way that you can extend the Office client applications and integrate them more deeply with SharePoint Foundation and SharePoint Server. Building Office client managed add-ins is an effective way to make the features of your SharePoint application discoverable and easy to use for end users. Deep integration of SharePoint data with Word documents or Excel spreadsheets results in an application where you have only one version of your critical data, helping to prevent decisions based on stale data.

You can access SharePoint external content types through the BCS client-side API. You can access data and documents stored in SharePoint lists and documents using the managed client object model.

Developer Center: Office Developers Center

Open XML SDK

Most SharePoint users make extensive use of document libraries, and most documents in document libraries are stored in the Open XML file format. Prior to the advent of Open XML, it was difficult to write code that directly accessed the contents of documents, but with the Open XML SDK, you have server-hardened tools to query and transform Open XML document. Using Open XML from within a SharePoint application opens up new scenarios. You can generate large numbers of documents, and automatically convert those documents from one format to another. You can query the contents of existing documents and bring that content and data to your users in new and interesting ways. For example, a product design team might use a specific document template for specifications, and each section of the specification may follow a convention that reports the state of the specification. You could then write an application to process all specifications for a project, and generate a report for management that shows the overall state of the specification writing process.

Combining the power of the Open XML SDK with the power of Word Services, you can implement a large number of server-side scenarios with high performance.

Prior to the advent of Open XML, customers sometimes attempted to solve server-side document processing needs using Office automation. Automating Microsoft Office applications on servers presents a set of well-known issues. The solution is to use SharePoint 2010 and the Open XML SDK.

A common pattern for document generation in SharePoint 2010 is to generate them into a SharePoint document library. Then, if necessary, you can use Word Automation Services to update fields in the document, or to import altChunk content.

Developer Center: Open XML Formats Resource Center

External link: ECMA 376-2

External link: ISO/IEC 29500-2

KB article: Considerations for server-side Automation of Office

Blog: Eric White's Blog

Blog: Brian Jones’ Blog

Blog: How to Use altChunk for Document Assembly

PerformancePoint Services

PerformancePoint Services in Microsoft SharePoint Server 2010 is a performance management service that you can use to monitor and analyze your business. It provides tools for building dashboards, scorecards, reports, and key performance indicators (KPIs). Integrated analytics help employees move quickly from monitoring information to analyzing it and, when appropriate, sharing it throughout the organization.

SDK: Development Scenarios with PerformancePoint Services

PowerPivot

PowerPivot gives users the power to create compelling self-service BI solutions, facilitates sharing and collaboration on user-generated BI solutions, and enables IT organizations to increase operational efficiencies through Microsoft SQL Server 2008 R2-based management tools.

MSDN: PowerPivot Overview

External: PowerPivot.com

Sandboxed Solutions

Sandboxed solutions are one approach to solving scalability problems when SharePoint users need to have specialized solutions built. If every developer knew all best practices, followed them religiously, and always wrote perfect code, this wouldn’t be a problem. However, code is not perfect, and bugs in code that runs with farm administrator rights can bring down the farm. With sandboxed solutions, there is an infrastructure that watches code for thrown exceptions, and monitors memory and CPU use, among other things. The misbehaving code gets stopped, and not the farm.

SDK: Sandboxed Solutions

How to: Creating Sandboxed Workflow Actions.

Video and hands-on lab: Sandboxed Solutions for Web Parts in SharePoint 2010

Search Customization

You can customize search to index new and different types of content, including ECTs.

SDK: SharePoint Enterprise Search

Technical Article: Searching External Data in SharePoint 2010 Using Business Connectivity Services

Secure Store Service

Secure Store Service is a claims-aware authorization service that provides storage and mapping of credentials such as account names and passwords. It enables you to securely store data that provides credentials required for connecting to external systems and associating those credentials to a specific identity or group of identities.

SDK: Secure Store Service

How to: Use Secure Store Service to Connect to an External System

Security

Security features of SharePoint Foundation 2010, and the underlying ASP.NET and .NET Framework, help give you protection from unauthorized changes to your data, whether deliberate or inadvertent, and help ensure that sensitive information is available only to those authorized to see it.

SDK: SharePoint Foundation Security

Server Object Model

The SharePoint server object model allows you to write programs to access SharePoint lists and document libraries, site collections, sites, and a lot more. Although the object model is very large, it is not inherently difficult.

When you are developing for SharePoint, things can become somewhat complex. You may be developing a Web part, an event receiver, or code that runs as part of a workflow. In all of these scenarios, you will be writing code that uses the server object model. This means that your edit/compile/execute cycle will be more complex. With each compilation, you may need to reinstall a SharePoint Feature, cycle Internet Information Services, or connect to another process for debugging. Dealing with those issues while at the same time learning how the server object model works is not desirable. Instead, while I was learning the server object model, I built small console applications that used the server object model. I executed those console applications directly on the server. When writing more complex code that would eventually end up in a Web part, I factored the code so that the code that used the server object model was isolated in one or a few classes in my application. I factored code that used the Open XML SDK into its own set of classes. This gave me a quick edit/compile/execute cycle for both the Open XML code and the server object model code. Then, after all constituent parts are developed, it is easy to build and deploy a Web part that uses the debugged code.

Code that uses the SharePoint server object model can run with farm administrator privileges or as a sandboxed solution. With sandboxed solutions, you have access to a subset of the API that is available to code that is running with farm administrator rights.

The SharePoint server object model can be divided into three hierarchies:

· The Content Hierarchy of Microsoft SharePoint Foundation discusses the various classes that represent publishable items of data, such as list items. There are also classes that represent nested containers of data, such as lists, content databases, Web sites, collections of Web sites, and groupings of site collections called Web applications.

· The Physical Objects Hierarchy of Microsoft SharePoint Foundation discusses the various classes that represent physical entities, such as servers and files, and containers of such entities, such as farms and folders.

· The Services Hierarchy of Microsoft SharePoint Foundation discusses classes that represent Web services, Windows services, other types of services, and instances of services.

SDK: Server and Site Architecture: Object Model Overview

SDK: Essential SharePoint Programming Tasks

SDK: Using the SharePoint Foundation Server-Side Object Model

SDK: Basic Object Model Tasks

SDK: Digital Asset Management Programming Model

SDK: Document Management Programming Model

SDK: Records Management Programming Model

SDK: Metadata and Taxonomy Programming Model

SDK: Publishing Programming Model

SDK: Sandboxed Solutions

Video and hands-on lab: Accessing SharePoint 2010 Data with Server-Side APIs

Server Ribbon

Customizing the SharePoint 2010 Server ribbon allows you to give an intuitive and powerful user interface to your SharePoint application. The key point behind customizing the ribbon is to make features and capabilities of your custom SharePoint application discoverable to your users. Customizing the ribbon is analogous to customizing the ribbon for the Office client applications.

SDK: Building Block: Server Ribbon

SDK: Server Ribbon in SharePoint Foundation

SDK: Architecture of the Server Ribbon

SDK: Customizing the Server Ribbon

Video and hands-on lab: Creating Dialogs and Ribbon Controls for SharePoint 2010

Service Application Architecture

The service application architecture enables you to create robust and scalable shared services that can be consumed by multiple SharePoint applications.

SDK: Service Application Framework

SDK: Service Application Framework Architecture

Video: SharePoint 2010 Service Application Architecture

Video: Creating Custom Service Applications

SharePoint Dialog Framework

When building web parts or application pages, you may need to put up a dialog box of one sort or another.

SDK: Creating a Page for a Modal Dialog Box

Video: SharePoint 2010 UI Features

Blog: How to Display a SharePoint Dialog from Ribbon Button and Get Selected Item Context

Video and hands-on lab: Creating Dialogs and Ribbon Controls for SharePoint 2010

SharePoint Themes

Themes, which are important when branding a SharePoint site, provide a quick and easy way to apply colors and fonts to sites in SharePoint 2010. A change in theme cause the color of most page elements, such as background images, text, and hyperlinks, to change. The fonts used for some page elements, such as titles, also changes.

TechNet: SharePoint Themes

SharePoint Web Services

SharePoint has long provided a set of Web services to enable client interaction with SharePoint deployments. Some of these are specifically designed for data retrieval and others include some data retrieval services. The APIs of these data-providing Web services are still supported for backward compatibility and interoperability with Web service clients. For reasons of both performance and ease, we recommend that you use either the client object model or the ADO.NET Data Services Framework to query SharePoint data.

SDK: SharePoint Foundation 2010 Web Services

SDK: Lists Web service

SDK: Webs Web service

SDK: WebSvcsites Web service

SDK: SiteData Web service

SDK: DspSts Web service

Interoperability documentation: SharePoint Products and Technologies Protocols

Instead of using Web services, see the following:

SDK: SharePoint Foundation REST Interface

SDK: Managed Client Object Model

Technical Article: Using the SharePoint Foundation 2010 Managed Client Object Model

Technical Article: Using the SharePoint Foundation 2010 Managed Client Object Model with the Open XML SDK 2.0

Video and hands-on lab: Accessing SharePoint 2010 Data with Client-Side APIs

Silverlight Web Part

Silverlight Web parts are an important tool for building the best user interfaces. There are times when nothing less than Silverlight will do.

Code Sample: Silverlight List Viewer

Video and hands-on lab: Creating Silverlight User Interfaces for SharePoint 2010 Solutions

Site Definitions

A site definition is a set of files that includes a master XML configuration file that is stored on all front-end Web servers. A site definition provides the basic blueprint for how sites look, what lists they include, their default navigational structures, and so on.

SDK: Site Types: WebTemplates and Site Definitions

Site Pages

Site Pages provide an efficient means (in terms of both storage and CPU time) to allow for hundreds of thousands of Web pages where many are the same and a few are customized. Unlike a traditional ASP.NET application, a physical ASPX page does not need to exist for every site page on the server. SharePoint treats any site page that is not customized as a virtual file, which can allow a small number of files to appear in many places. This is one of the many aspects of SharePoint that gives it scalability. Without this capability, you would need far too many compiled ASPX pages for a large site. Management of those pages would be unwieldy.

SDK: Sites and Pages

SDK: SharePoint Page Types

Blog: Creating a SharePoint Site Page With Code-Behind Using Visual Studio 2010

Timer Jobs

Like a CRON job, a timer job is code that runs at a specific time. An application may regularly need to process data in a SharePoint list, and perhaps modify data in an external system. The developer can write a timer job for this that will run at 1am every morning.

SDK: Timer job reference

Variations

Variations enable you to tailor sites for different cultures, markets, and languages. It is a feature that facilitates serving different content to different audiences. Variations can help serve different content based on the language of the user. In addition, variations can serve different content based on end-user devices, or any arbitrary distinction that you choose, such as brand or level of user expertise.

SDK: Variations and Multiple Languages Sites

Blog: Variations in SharePoint 2010

Visio Services

Visio Services is a service application that enables users to share and view Microsoft Visio Web drawings. The service also enables data-connected Microsoft Visio 2010 Web drawings to be refreshed and updated from various data sources.

SDK: Visio Services overview

Visual Studio SharePoint Tools Extensibility

The SharePoint tools in Visual Studio 2010 meet the requirements of many application development scenarios. However, you might discover cases where they do not provide functionality that you or other developers require. In these cases, you can extend the SharePoint tools to create the functionality that you need.

SDK: Visual Studio SharePoint Tools Extensibility

Video: Getting Started with SharePoint 2010 Developer Tools in Visual Studio 2010

Web-based Distributed Authoring and Versioning (WebDAV)

A common activity for client-side code is to upload and download documents and files. You may need to retrieve documents from a SharePoint document library to the client computer, or upload documents from the client computer to a SharePoint document library. You can use WebDAV directly, but an easy way to use WebDAV is to use the managed client object model. The ClientOM.File.OpenBinaryDirect method and the ClientOM.File.SaveBinaryDirect method in the client object model use WebDAV for uploading and downloading files. Without building your own custom WCF service, WebDAV is the most efficient way to upload and download files. Note that WebDAV is limited to accessing files, not data.

External link: RFC 4918

Web Parts

SharePoint is an ASP.NET application. It inherits a number of important aspects of ASP.NET, including the notions of web part pages, and web parts. The classes that represent SharePoint Web parts derive from the appropriate ASP.NET classes.

Building SharePoint web parts accounts for more SharePoint developer hours than any other category (perhaps more than the rest put together). Today, more than ever, users expect good looking, intuitive, and powerful user interfaces, and developing web parts is often the most effective approach.

Web parts can communicate with other web parts. For example, they can be linked to enable parent/child form/list user interfaces when maintaining data.

AJAX Web parts are a special category. When your optimal user interface requires it, you can take advantage of AJAX architecture.

Silverlight Web parts are another special category. There are times when nothing less than Silverlight will do. I think that Silverlight web parts in SharePoint are the future.

Web parts can be built with code-behind, either in sandboxed or full-trust mode.

On the client side, web parts can communicate with the SharePoint server using the Managed Client Object Model, REST APIs, or web services. WCF is an option from within a Silverlight web part.

While developers can build custom Web parts, SharePoint 2010 provides a number of build-in Web parts that can be useful in putting together a custom solution:

Some of the more common SharePoint built-in Web parts are:

List View Web Part	SDK: List Views Blog: SharePoint 2010 List View Blog Series Office online: About the List View Web Part
Image Web Part	Office online: Image Web Part
Site Users Web Part	Office online: Site Users Web Part
Chart Web Parts	Office online: Introduction to the Chart Web Part

SDK: Web Parts in SharePoint Foundation

SDK: Microsoft.SharePoint.WebPartPages Namespace

SDK: Building Block: Web Parts

SDK: Connect Web Parts in SharePoint 2010

SDK: JS Grid Control

How to: Create a Basic JS Grid

Video and hands-on lab: Getting Started Building Web Parts in SharePoint 2010

Video and hands-on lab: Building Blocks for Web Part Development in SharePoint 2010

Video and hands-on lab: Creating Silverlight User Interfaces for SharePoint 2010 Solutions

Video and hands-on lab: Sandboxed Solutions for Web Parts in SharePoint 2010

Video: Creating SharePoint 2010 Web Parts That Can Read and Write Data to External Data Sources

Video: Advanced Web Part Development in SharePoint 2010

SDK: Cascading Style Sheets Class Usage in SharePoint Foundation

Code Sample: Silverlight List Viewer

Web Services

When you are building large distributed applications, best practice is to loosely couple them, using ASMX web services or WCF services to do the coupling. You need to apply this same technique when building large scale SharePoint applications.

As an example, Operational Research often requires deployment of a sophisticated mathematical optimizer to arrive at an optimal or near optimal solution to some problem. This should be deployed on dedicated hardware, and exposed via Web services.

In some cases, you may want to build client applications that make use of sophisticated server-side functionality that scales to the maximum extent. You may want to reduce extraneous network traffic to an absolute minimum. In these cases, the best solution often is to host a WCF service on your SharePoint server. Code in this service can use the server object model with full farm privileges, and supply the necessary functionality for the client application. As appropriate, when building these types of services, you need to do attack surface analysis and threat modeling.

Developer Center: WCF Developers Center

Blog: Getting Started Building a WCF Web Service

Windows Identity Foundation

The claims-based identity model for Microsoft SharePoint Foundation 2010 and Microsoft SharePoint Server 2010 is built upon Windows Identity Foundation (WIF). Features of claims-based identity include:

· Authentication across users of Windows-based systems and systems that are not Windows-based.

· Multiple authentication types.

· Stronger real-time authentication.

· A wider set of principal types.

· Delegation of user identity between applications.

When you build claims-aware applications, the user presents an identity to your application as a set of claims. One claim could be the user’s name, another might be an email address. The idea is that an external identity system is configured to give your application all the information that it needs about the user with each request, along with cryptographic assurance that the identity data that your application receives comes from a trusted source. Under this model, single sign-on is much easier to achieve.

Developer Center: SharePoint 2010 Security Resource Center

Developer Center: Windows Identity Foundation

Blog: Consuming a Claims-Enabled WCF Web Service as a SharePoint 2010 External Content Type

Word Automation Services

There are some tasks that are difficult when using the Open XML SDK, such as repagination, conversion to other document formats such as PDF, or updating of field codes or the table of contents, fields, and other dynamic content in documents. Word Automation Services is a feature of SharePoint 2010 that can help in these scenarios. It is a shared service that provides unattended, server-side conversion of documents into other formats, and some other important pieces of functionality. It was designed from the outset to work on servers and can process high volumes of documents in a reliable and predictable manner.

SDK: Word Automation Services

Technical Article: Developing with SharePoint 2010 Word Automation Services

Workflows

SharePoint workflows allow you to codify business process. Workflow logic can be very simple or quite elaborate. End users can use workflows to build no-code business solutions. Developers can enhance workflows with C# code that is called at appropriate times. Workflows can be deployed either as a full-trust farm solution, or as a sandboxed solution. Examples of workflow use include claims processing, content approval, or purchasing processes.

SDK: Workflows in SharePoint Foundation

SDK: Microsoft.SharePoint.Workflow

Walkthrough: Use SharePoint Full-Trust Workflow Activities with Business Connectivity Services.

Video and hands-on lab: Developing Business Processes with SharePoint 2010 Workflows

Developer Dashboard

Developing SharePoint applications can be a fairly complex undertaking. If a page or a Web part does not perform well, it can be difficult to determine what is causing the delay or consuming resources. SharePoint 2010 provides a developer dashboard that you can turn on to view performance metrics related to the page that is loading.

You can turn on the developer dashboard by running the following Stsadm command:

stsadm -o setproperty -pn developer-dashboard -pv OnDemand

SDK: SharePoint 2010 Development for ASP.NET Developers

SDK: Using the Developer Dashboard

Accessibility

Microsoft adopted the Web Content Accessibility Guidelines 2.0 (WCAG 2.0) for SharePoint 2010, and set a goal for Level AA. Becoming a W3C recommendation on December 11th, 2008, WCAG 2.0 defines the expectations of and the techniques deployed in well-built, accessible Web sites.

External link: Web Content Accessibility Guidelines 2.0

Blog: Accessibility and SharePoint 2010

SharePoint Development Tools

There are a number of tools that you use to build SharePoint 2010 applications.

Visual Studio 2010	Visual Studio 2010 is the primary tool for professional developers on SharePoint 2010. It contains a variety of pre-configured project types to make it as easy as possible to write code for your scenario.
SharePoint Designer 2010	SharePoint Designer 2010 enables you to create no-code SharePoint solutions that encompass a variety of common scenarios, from collaborative sites and web publishing to Line-Of-Business data integration, business intelligence solutions, and human workflows. Developers can use SharePoint Designer 2010 to get a quick start on SharePoint development projects.
Visio 2010	Visio 2010 enables you and subject matter experts to author SharePoint workflows using drag-and-drop. You can then import them into SharePoint Designer 2010 for further editing, and then execute them in SharePoint 2010.

SDK: SharePoint Foundation Development Tools

MSDN: SharePoint Development in Visual Studio

MSDN Magazine: Visual Studio 2010 Tools for SharePoint Development

Video: Getting Started with SharePoint 2010 Developer Tools in Visual Studio 2010

Download: Microsoft SharePoint Designer 2010 (32 bit)

Blog: SharePoint Workflow Authoring in Visio Premium 2010

Blog: Visual Studio SharePoint Development Blog

Setting up a Development Environment

There are several ways to set up a SharePoint 2010 development environment.

· My favorite approach is to use Windows Server 2008 R2, and set up my development machine using Hyper-V. Immediately after installing everything, I take a snapshot. This allows me to revert back to this clean state if necessary, although in my experience, it rarely is. For the best experience, place the virtual hard drive on a drive other than where the host operating system resides.

· You can install on Windows Server 2008 (or R2) that is running directly on the server, not in a Hyper-V VM. This can be an effective approach, although it loses the ability to revert to a snapshot.

· You can install on Windows 7 64-bit or Windows Vista 64-bit. Some developers work for companies where they are prevented by policy from running server operating systems, so this can be a good solution. In addition, if I were starting a long development project that would take several months to complete, I would install SharePoint 2010 on my laptop so that I could do development at opportune times, yet retain the ease of use of an end-user operating system.

SDK: Setting up a SharePoint 2010 Development Environment.

Blog post: SharePoint development environments

Application Lifecycle Management (ALM)

Application Lifecycle Management is the coordination of all aspects of software engineering—including the formulation and communication of business and technical requirements, code design and architecture, project tracking, change management, coding, testing, debugging, and release management—by using tools that facilitate and track collaboration among and within work teams.

Resource Center: Application Lifecycle Management Resource Center

Video and hands-on lab: Development Life Cycle for SharePoint 2010

Video: Improving Software Development Across the Lifecycle with Visual Studio 2010

Conclusion

With SharePoint Foundation 2010 and SharePoint Server 2010, SharePoint has evolved into a comprehensive development platform. While many SharePoint applications are simple, SharePoint applications can very complex, using a large number of building blocks, all working together in concert. Careful design of your application is necessary to ensure that your application is robust and can scale, and that the user interface of your application meets the needs of your users.

Additional Resources

Code samples: SharePoint Server 2010 Code Samples

Code samples: SharePoint Foundation 2010 Code Samples

SDK: Best Practices with SharePoint Server

Articles: SharePoint 2010 Technical Articles

Videos: SharePoint 2010 Videos | SharePoint Conference 2009

Video and hands-on lab: What Developers Need to Know About SharePoint 2010

Blog: SharePoint Team Blog

Blog: Microsoft SharePoint Developer Documentation Team Blog

MSDN: Developer Getting Started Training

MSDN: Advanced Developer Training

Microsoft Training and Certification for SharePoint 2010

Building a Simple Recursive Descent Parser (Completed Simple Parser)

Eric White - Independant Consultant — Fri, 24 Sep 2010 04:34:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

In this post, I complete the recursive descent parser that will parse the simple grammar that I presented previously in this series. In this post, I’ll examine in detail the NospaceExpression class, which needs to implement operator precedence.

Note: I have to apologize for the delay in writing this post. First summer vacation took some time, and then I became involved in a project of writing two papers – one that summarizes SharePoint application development, and another that summarizes Office Client application development. I’ve been literally obsessing about every possible way that you as a developer can extend SharePoint Server 2010, and every way that you can extend the Office 2010 client applications. The SharePoint paper is nearing publication. The Office Client paper will be along in a bit. While obsessing, it has been difficult for me to mentally switch tracks back to this series. But here we go again…

This post is one in a series on using LINQ to write a recursive-descent parser for SpreadsheetML formulas. You can find the complete list of posts here.

The NospaceExpression Class

Here is the entire implementation of the NospaceExpression class. I explained all of the code that has a gray background in the previous post, Building a Simple Recursive Descent Parser (continued).

public static NospaceExpression Produce(IEnumerable<Symbol> symbols)
{
    // nospace-expression = open-parenthesis expression close-parenthesis
    //         / numerical-constant
    //         / prefix-operator expression
    //         / expression infix-operator expression

    if (!symbols.Any())
        return null;

    if (symbols.First() is OpenParenthesis && symbols.Last() is CloseParenthesis)
    {
        Expression e = Expression.Produce(symbols.Skip(1).SkipLast(1));
        if (e != null)
            return new NospaceExpression(new OpenParenthesis(), e, new CloseParenthesis());
    }

    // expression, infix-operator, expression
    var z = symbols.Rollup(0, (t, d) =>
    {
        if (t is OpenParenthesis)
            return d + 1;
        if (t is CloseParenthesis)
            return d - 1;
        return d;
    });
    var symbolsWithIndex = symbols.Select((s, i) => new
    {
        Symbol = s,
        Index = i,
    });
    var z2 = symbolsWithIndex.Zip(z, (v1, v2) => new
    {
        SymbolWithIndex = v1,
        Depth = v2,
    });
    var operatorList = z2
        .Where(x => x.Depth == 0 &&
            x.SymbolWithIndex.Index != 0 &&
            InfixOperator.Produce(x.SymbolWithIndex.Symbol) != null)
        .ToList();
    if (operatorList.Any())
    {
        int minPrecedence = operatorList
            .Select(o2 => OperatorPrecedence[o2.SymbolWithIndex.Symbol.ToString()]).Min();
        var op = operatorList
            .Last(o2 => OperatorPrecedence[o2.SymbolWithIndex.Symbol.ToString()] == minPrecedence);
        if (op != null)
        {
            var expressionTokenList1 = symbols.TakeWhile(t => t != op.SymbolWithIndex.Symbol);
            Expression e1 = Expression.Produce(expressionTokenList1);
            if (e1 == null)
                throw new ParserException("Invalid expression");
            var expressionTokenList2 = symbols
                .SkipWhile(t => t != op.SymbolWithIndex.Symbol).Skip(1);
            Expression e2 = Expression.Produce(expressionTokenList2);
            if (e2 == null)
                throw new ParserException("Invalid expression");
            InfixOperator io = new InfixOperator(op.SymbolWithIndex.Symbol);
            return new NospaceExpression(e1, io, e2);
        }
    }

    NumericalConstant n = NumericalConstant.Produce(symbols);
    if (n != null)
        return new NospaceExpression(n);

    PrefixOperator p = PrefixOperator.Produce(symbols.FirstOrDefault());
    if (p != null)
    {
        Expression e = Expression.Produce(symbols.Skip(1));
        if (e != null)
            return new NospaceExpression(p, e);
    }

    return null;
}

Let’s work through the new code. Let’s consider how the new code behaves when parsing the following formula:

"1+((2+3)*4)^5"

When determining if an expression has an infix operator, we specifically need to ignore operators that are in parentheses. Those operators will have their chance to be recognized when NospaceExpression.Produce sees that an expression starts with an open parenthesis and ends with a close parenthesis, as processed by this code:

if (!symbols.Any())
    return null;

if (symbols.First() is OpenParenthesis && symbols.Last() is CloseParenthesis)
{
    Expression e = Expression.Produce(symbols.Skip(1).SkipLast(1));
    if (e != null)
        return new NospaceExpression(new OpenParenthesis(), e, new CloseParenthesis());
}

So we need to ignore operators that are inside parentheses. The easiest way to determine if an operator is inside of parentheses is to use the Rollup extension method, as follows:

// expression, infix-operator, expression
var z = symbols.Rollup(0, (t, d) =>
{
    if (t is OpenParenthesis)
        return d + 1;
    if (t is CloseParenthesis)
        return d - 1;
    return d;
});

This use of the Rollup extension method returns a new collection of integers that has the same number of elements in it as the source collection. The integer is the depth of parentheses at that point. In the following snippet, I add a bit of code to print the items in the collection, and then exit:

// expression, infix-operator, expression
var z = symbols.Rollup(0, (t, d) =>
{
    if (t is OpenParenthesis)
        return d + 1;
    if (t is CloseParenthesis)
        return d - 1;
    return d;
});
foreach (var item in z)
    Console.WriteLine(item);
Environment.Exit(0);

When I run this example for the test formula "1+((2+3)*4)^5", I see:
0
0
1
2
2
2
2
1
1
1
0
0
0

When looking for an infix operator, I can ignore all operators where the depth is not zero.

Later on, I’m going to need the index of the symbol of the infix operator that we find. The next projection uses symbols as its source, and projects an anonymous type that includes the symbol and its index:

var symbolsWithIndex = symbols.Select((s, i) => new
{
    Symbol = s,
    Index = i,
});
foreach (var item in symbolsWithIndex)
    Console.WriteLine(item);
Environment.Exit(0);

When I run this for the test formula, I see:

{ Symbol = 1, Index = 0 }
{ Symbol = +, Index = 1 }
{ Symbol = (, Index = 2 }
{ Symbol = (, Index = 3 }
{ Symbol = 2, Index = 4 }
{ Symbol = +, Index = 5 }
{ Symbol = 3, Index = 6 }
{ Symbol = ), Index = 7 }
{ Symbol = *, Index = 8 }
{ Symbol = 4, Index = 9 }
{ Symbol = ), Index = 10 }
{ Symbol = ^, Index = 11 }
{ Symbol = 5, Index = 12 }

And next, use the Zip extension method to zip together the last two queries to create a list of items that contains:

· The symbol

· Its index

· Its parenthesis depth

This produces:

{ SymbolWithIndex = { Symbol = 1, Index = 0 }, Depth = 0 }
{ SymbolWithIndex = { Symbol = +, Index = 1 }, Depth = 0 }
{ SymbolWithIndex = { Symbol = (, Index = 2 }, Depth = 1 }
{ SymbolWithIndex = { Symbol = (, Index = 3 }, Depth = 2 }
{ SymbolWithIndex = { Symbol = 2, Index = 4 }, Depth = 2 }
{ SymbolWithIndex = { Symbol = +, Index = 5 }, Depth = 2 }
{ SymbolWithIndex = { Symbol = 3, Index = 6 }, Depth = 2 }
{ SymbolWithIndex = { Symbol = ), Index = 7 }, Depth = 1 }
{ SymbolWithIndex = { Symbol = *, Index = 8 }, Depth = 1 }
{ SymbolWithIndex = { Symbol = 4, Index = 9 }, Depth = 1 }
{ SymbolWithIndex = { Symbol = ), Index = 10 }, Depth = 0 }
{ SymbolWithIndex = { Symbol = ^, Index = 11 }, Depth = 0 }
{ SymbolWithIndex = { Symbol = 5, Index = 12 }, Depth = 0 }

Next, we can look for any infix operators where the Depth == 0. In addition, we filter out operators that are found in the very first position, because that will not be an infix operator, but instead a prefix operator.

This produces a list of the two possible operators that are part of an expression that uses an infix operator:

{ SymbolWithIndex = { Symbol = +, Index = 1 }, Depth = 0 }
{ SymbolWithIndex = { Symbol = ^, Index = 11 }, Depth = 0 }

Next, of the operators that could possibly be the infix operator for this expression, we find the minimum precedence of any operators. When implementing a top-down parser, we process operators with the least precedence before processing operators of higher precedence. When looking at the expression tree, operators with the least precedence are nearer to the root of the tree. Operators with the highest precedence will be closer to the leaves, so that they will be evaluated before the operators with lower precedence. If this isn’t clear, I think that it will be when you see the entire parse tree. In this particular case, we need to find the + operator, not the ^ operator, as + has lower precedence.

To enable processing of precedence of operators, I defined a small dictionary of operators and their precedence:

public static Dictionary<string, int> OperatorPrecedence = new Dictionary<string, int>()
{
    { "^", 3 },
    { "*", 2 },
    { "/", 2 },
    { "+", 1 },
    { "-", 1 },
};

In the following listing, the highlighted line finds the minimum precedence of the operators that are in play.

// expression, infix-operator, expression
var z = symbols.Rollup(0, (t, d) =>
{
    if (t is OpenParenthesis)
        return d + 1;
    if (t is CloseParenthesis)
        return d - 1;
    return d;
});
var symbolsWithIndex = symbols.Select((s, i) => new
{
    Symbol = s,
    Index = i,
});
var z2 = symbolsWithIndex.Zip(z, (v1, v2) => new
{
    SymbolWithIndex = v1,
    Depth = v2,
});
var operatorList = z2
    .Where(x => x.Depth == 0 &&
        x.SymbolWithIndex.Index != 0 &&
        InfixOperator.Produce(x.SymbolWithIndex.Symbol) != null)
    .ToList();
if (operatorList.Any())
{
    int minPrecedence = operatorList
        .Select(o2 => OperatorPrecedence[o2.SymbolWithIndex.Symbol.ToString()]).Min();
    var op = operatorList
        .Last(o2 => OperatorPrecedence[o2.SymbolWithIndex.Symbol.ToString()] == minPrecedence);
    if (op != null)
    {
        var expressionTokenList1 = symbols.TakeWhile(t => t != op.SymbolWithIndex.Symbol);
        Expression e1 = Expression.Produce(expressionTokenList1);
        if (e1 == null)
            throw new ParserException("Invalid expression");
        var expressionTokenList2 = symbols
            .SkipWhile(t => t != op.SymbolWithIndex.Symbol).Skip(1);
        Expression e2 = Expression.Produce(expressionTokenList2);
        if (e2 == null)
            throw new ParserException("Invalid expression");
        InfixOperator io = new InfixOperator(op.SymbolWithIndex.Symbol);
        return new NospaceExpression(e1, io, e2);
    }
}

And finally, we find the infix operator:

If we found an operator, then we can new-up (highlighted in green) an instance of NoSpaceExpression, passing the symbols before and after the infix operator to Expression.Produce (highlighted in cyan).

After parsing the sample expression, “1+((2+3)*4)^5”, the parser produces the following parse tree:

Formula >1+((2+3)*4)^5<
Expression >1+((2+3)*4)^5<
    NospaceExpression >1+((2+3)*4)^5<
      Expression >1<
        NospaceExpression >1<
          NumericalConstant >1<
            SignificandPart >1<
              WholeNumberPart >1<
                DigitSequence >1<
                  DecimalDigit >1<
      InfixOperator >+<      <= the + operator has lower precedence, so is nearer to the root of the tree, and will be evaluated last
        Plus >+<
      Expression >((2+3)*4)^5<
        NospaceExpression >((2+3)*4)^5<
          Expression >((2+3)*4)<
            NospaceExpression >((2+3)*4)<
              OpenParenthesis >(<
              Expression >(2+3)*4<
                NospaceExpression >(2+3)*4<
                  Expression >(2+3)<
                    NospaceExpression >(2+3)<
                      OpenParenthesis >(<
                      Expression >2+3<
                        NospaceExpression >2+3<
                          Expression >2<
                            NospaceExpression >2<
                              NumericalConstant >2<
                                SignificandPart >2<
                                  WholeNumberPart >2<
                                    DigitSequence >2<
                                      DecimalDigit >2<
                          InfixOperator >+<
                            Plus >+<
                          Expression >3<
                            NospaceExpression >3<
                              NumericalConstant >3<
                                SignificandPart >3<
                                  WholeNumberPart >3<
                                    DigitSequence >3<
                                      DecimalDigit >3<
                      CloseParenthesis >)<
                  InfixOperator >*<
                    Asterisk >*<
                  Expression >4<
                    NospaceExpression >4<
                      NumericalConstant >4<
                        SignificandPart >4<
                          WholeNumberPart >4<
                            DigitSequence >4<
                              DecimalDigit >4<
              CloseParenthesis >)<
          InfixOperator >^<   <= the ^ operator has higher precedence, so is closer to leaves of the tree, so will be evaluated first
            Caret >^<
          Expression >5<
            NospaceExpression >5<
              NumericalConstant >5<
                SignificandPart >5<
                  WholeNumberPart >5<
                    DigitSequence >5<
                      DecimalDigit >5<

Following is the complete listing of the simple parser. While I haven’t discussed every class in the parser, I’ve introduced every technique that I’ve used in writing the Produce methods for each class. To run this example, you can simply copy/paste this example into Visual Studio and run it. There are no dependencies.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace SimpleParser
{
    public abstract class Symbol
    {
        public List<Symbol> ConstituentSymbols { get; set; }
        public override string ToString()
        {
            string s = ConstituentSymbols.Select(ct => ct.ToString()).StringConcatenate();
            return s;
        }
        public Symbol(params Object[] symbols)
        {
            List<Symbol> ls = new List<Symbol>();
            foreach (var item in symbols)
            {
                if (item is Symbol)
                    ls.Add((Symbol)item);
                else if (item is IEnumerable<Symbol>)
                    foreach (var item2 in (IEnumerable<Symbol>)item)
                        ls.Add(item2);
                else
                    // If this error is thrown, the parser is coded incorrectly.
                    throw new ParserException("Internal error");
            }
            ConstituentSymbols = ls;
        }
        public Symbol() { }
    }

    public class Formula : Symbol
    {
        public static Formula Produce(IEnumerable<Symbol> symbols)
        {
            // formula = expression

            Expression e = Expression.Produce(symbols);
            return e == null ? null : new Formula(e);
        }
        public Formula(params Object[] symbols) : base(symbols) { }
    }

    public class Expression : Symbol
    {
        public static Expression Produce(IEnumerable<Symbol> symbols)
        {
            // expression = *whitespace nospace-expression *whitespace

            int whiteSpaceBefore = symbols.TakeWhile(s => s is WhiteSpace).Count();
            int whiteSpaceAfter = symbols.Reverse().TakeWhile(s => s is WhiteSpace).Count();
            IEnumerable<Symbol> noSpaceSymbolList = symbols
                .Skip(whiteSpaceBefore)
                .SkipLast(whiteSpaceAfter)
                .ToList();
            NospaceExpression n = NospaceExpression.Produce(noSpaceSymbolList);
            if (n != null)
                return new Expression(
                    Enumerable.Repeat(new WhiteSpace(), whiteSpaceBefore),
                    n,
                    Enumerable.Repeat(new WhiteSpace(), whiteSpaceAfter));
            return null;
        }

        public Expression (params Object[] symbols) : base(symbols) { }
    }

    public class NospaceExpression : Symbol
    {
        public static Dictionary<string, int> OperatorPrecedence = new Dictionary<string, int>()
        {
            { "^", 3 },
            { "*", 2 },
            { "/", 2 },
            { "+", 1 },
            { "-", 1 },
        };

        public static NospaceExpression Produce(IEnumerable<Symbol> symbols)
        {
            // nospace-expression = open-parenthesis expression close-parenthesis
            //         / numerical-constant
            //         / prefix-operator expression
            //         / expression infix-operator expression

            if (!symbols.Any())
                return null;

            if (symbols.First() is OpenParenthesis && symbols.Last() is CloseParenthesis)
            {
                Expression e = Expression.Produce(symbols.Skip(1).SkipLast(1));
                if (e != null)
                    return new NospaceExpression(new OpenParenthesis(), e, new CloseParenthesis());
            }

            // expression, infix-operator, expression
            var z = symbols.Rollup(0, (t, d) =>
            {
                if (t is OpenParenthesis)
                    return d + 1;
                if (t is CloseParenthesis)
                    return d - 1;
                return d;
            });
            var symbolsWithIndex = symbols.Select((s, i) => new
            {
                Symbol = s,
                Index = i,
            });
            var z2 = symbolsWithIndex.Zip(z, (v1, v2) => new
            {
                SymbolWithIndex = v1,
                Depth = v2,
            });
            var operatorList = z2
                .Where(x => x.Depth == 0 &&
                    x.SymbolWithIndex.Index != 0 &&
                    InfixOperator.Produce(x.SymbolWithIndex.Symbol) != null)
                .ToList();
            if (operatorList.Any())
            {
                int minPrecedence = operatorList
                    .Select(o2 => OperatorPrecedence[o2.SymbolWithIndex.Symbol.ToString()]).Min();
                var op = operatorList
                    .Last(o2 => OperatorPrecedence[o2.SymbolWithIndex.Symbol.ToString()] == minPrecedence);
                if (op != null)
                {
                    var expressionTokenList1 = symbols.TakeWhile(t => t != op.SymbolWithIndex.Symbol);
                    Expression e1 = Expression.Produce(expressionTokenList1);
                    if (e1 == null)
                        throw new ParserException("Invalid expression");
                    var expressionTokenList2 = symbols
                        .SkipWhile(t => t != op.SymbolWithIndex.Symbol).Skip(1);
                    Expression e2 = Expression.Produce(expressionTokenList2);
                    if (e2 == null)
                        throw new ParserException("Invalid expression");
                    InfixOperator io = new InfixOperator(op.SymbolWithIndex.Symbol);
                    return new NospaceExpression(e1, io, e2);
                }
            }

            NumericalConstant n = NumericalConstant.Produce(symbols);
            if (n != null)
                return new NospaceExpression(n);

            PrefixOperator p = PrefixOperator.Produce(symbols.FirstOrDefault());
            if (p != null)
            {
                Expression e = Expression.Produce(symbols.Skip(1));
                if (e != null)
                    return new NospaceExpression(p, e);
            }

            return null;
        }

        public NospaceExpression(params Object[] symbols) : base(symbols) { }
    }

    public class NumericalConstant : Symbol
    {
        public static NumericalConstant Produce(IEnumerable<Symbol> symbols)
        {
            // numerical-constant = [neg-sign] significand-part

            SignificandPart s = SignificandPart.Produce(symbols);
            if (s != null)
                return new NumericalConstant(s);
            NegSign n = NegSign.Produce(symbols.First());
            if (n != null)
            {
                SignificandPart s2 = SignificandPart.Produce(symbols.Skip(1));
                if (s2 != null)
                    return new NumericalConstant(n, s2);
            }
            return null;
        }
        public NumericalConstant(params Object[] symbols) : base(symbols) { }
    }

    public class SignificandPart : Symbol
    {
        public static SignificandPart Produce(IEnumerable<Symbol> symbols)
        {
            // significand-part = whole-number-part [fractional-part] / fractional-part

            FractionalPart f;
            f = FractionalPart.Produce(symbols);
            if (f != null)
                return new SignificandPart(f);
            IEnumerable<Symbol> s = null;
            WholeNumberPart w = WholeNumberPart.Produce(symbols, out s);
            if (w != null)
            {
                if (!s.Any())
                    return new SignificandPart(w);
                f = FractionalPart.Produce(s);
                if (f != null)
                    return new SignificandPart(w, f);
            }
            return null;
        }
        public SignificandPart(params Object[] symbols) : base(symbols) { }
    }

    public class WholeNumberPart : Symbol
    {
        public static WholeNumberPart Produce(IEnumerable<Symbol> symbols,
            out IEnumerable<Symbol> symbolsToProcess)
        {
            // whole-number-part = digit-sequence

            IEnumerable<Symbol> s = null;
            DigitSequence d = DigitSequence.Produce(symbols, out s);
            if (d != null)
            {
                symbolsToProcess = s;
                return new WholeNumberPart(d);
            }
            symbolsToProcess = null;
            return null;
        }
        public WholeNumberPart(params Object[] symbols) : base(symbols) { }
    }

    public class FractionalPart : Symbol
    {
        public static FractionalPart Produce(IEnumerable<Symbol> symbols)
        {
            // fractional-part = full-stop digit-sequence

            if (!symbols.Any())
                return null;
            if (symbols.First() is FullStop)
            {
                IEnumerable<Symbol> s = null;
                DigitSequence d = DigitSequence.Produce(symbols.Skip(1), out s);
                if (d == null || s.Any())
                    return null;
                return new FractionalPart(new FullStop(), d);
            }
            return null;
        }
        public FractionalPart(params Object[] symbols) : base(symbols) { }
    }

    public class DigitSequence : Symbol
    {
        public static DigitSequence Produce(IEnumerable<Symbol> symbols,
            out IEnumerable<Symbol> symbolsToProcess)
        {
            // digit-sequence = 1*decimal-digit

            IEnumerable<Symbol> digits = symbols.TakeWhile(s => s is DecimalDigit);
            if (digits.Any())
            {
                symbolsToProcess = symbols.Skip(digits.Count());
                return new DigitSequence(digits);
            }
            symbolsToProcess = null;
            return null;
        }
        public DigitSequence(params Object[] symbols) : base(symbols) { }
    }

    public class NegSign : Symbol
    {
        public static NegSign Produce(Symbol symbol)
        {
            // neg-sign = minus

            if (symbol is Minus)
                return new NegSign(symbol);
            return null;
        }
        public NegSign(params Object[] symbols) : base(symbols) { }
    }

    public class PrefixOperator : Symbol
    {
        public static PrefixOperator Produce(Symbol symbol)
        {
            // prefix-operator = plus / minus

            if (symbol is Plus || symbol is Minus)
                return new PrefixOperator(symbol);
            return null;
        }
        public PrefixOperator(params Object[] symbols) : base(symbols) { }
    }

    public class InfixOperator : Symbol
    {
        public static InfixOperator Produce(Symbol symbol)
        {
            // infix-operator = caret / asterisk / forward-slash / plus / minus

            if (symbol is Plus || symbol is Minus || symbol is Asterisk || symbol is ForwardSlash
                || symbol is Caret)
                return new InfixOperator(symbol);
            return null;
        }
        public InfixOperator(params Object[] symbols) : base(symbols) { }
    }

    public class DecimalDigit : Symbol
    {
        private string CharacterValue;
        public override string ToString() { return CharacterValue; }
        public DecimalDigit(char c) { CharacterValue = c.ToString(); }
    }

    public class WhiteSpace : Symbol
    {
        public override string ToString() { return " "; }
        public WhiteSpace() { }
    }

    public class Plus : Symbol
    {
        public override string ToString() { return "+"; }
        public Plus() { }
    }

    public class Minus : Symbol
    {
        public override string ToString() { return "-"; }
        public Minus() { }
    }

    public class Asterisk : Symbol
    {
        public override string ToString() { return "*"; }
        public Asterisk() { }
    }

    public class ForwardSlash : Symbol
    {
        public override string ToString() { return "/"; }
        public ForwardSlash() { }
    }

    public class Caret : Symbol
    {
        public override string ToString() { return "^"; }
        public Caret() { }
    }

    public class FullStop : Symbol
    {
        public override string ToString() { return "."; }
        public FullStop() { }
    }

    public class OpenParenthesis : Symbol
    {
        public override string ToString() { return "("; }
        public OpenParenthesis() { }
    }

    public class CloseParenthesis : Symbol
    {
        public override string ToString() { return ")"; }
        public CloseParenthesis() { }
    }

    public class SimpleFormulaParser
    {
        public static Symbol ParseFormula(string s)
        {
            IEnumerable<Symbol> symbols = s.Select(c =>
            {
                switch (c)
                {
                    case '0':
                    case '1':
                    case '2':
                    case '3':
                    case '4':
                    case '5':
                    case '6':
                    case '7':
                    case '8':
                    case '9':
                        return new DecimalDigit(c);
                    case ' ':
                        return new WhiteSpace();
                    case '+':
                        return new Plus();
                    case '-':
                        return new Minus();
                    case '*':
                        return new Asterisk();
                    case '/':
                        return new ForwardSlash();
                    case '^':
                        return new Caret();
                    case '.':
                        return new FullStop();
                    case '(':
                        return new OpenParenthesis();
                    case ')':
                        return new CloseParenthesis();
                    default:
                        return (Symbol)null;
                }
            });
#if false
            if (symbols.Any())
            {
                Console.WriteLine("Terminal Symbols");
                Console.WriteLine("================");
                foreach (var terminal in symbols)
                    Console.WriteLine("{0} >{1}<", terminal.GetType().Name.ToString(),
                        terminal.ToString());
                Console.WriteLine();
            }
#endif
            Formula formula = Formula.Produce(symbols);
            if (formula == null)
                throw new ParserException("Invalid formula");
            return formula;
        }

        public static void DumpSymbolRecursive(StringBuilder sb, Symbol symbol, int depth)
        {
            sb.Append(string.Format("{0}{1} >{2}<",
                "".PadRight(depth * 2),
                symbol.GetType().Name.ToString(),
                symbol.ToString())).Append(Environment.NewLine);
            if (symbol.ConstituentSymbols != null)
                foreach (var childSymbol in symbol.ConstituentSymbols)
                    DumpSymbolRecursive(sb, childSymbol, depth + 1);
        }
    }

    class Program
    {
        static void Main(string[] args)
        {
            string[] sampleValidFormulas = new[] {
                "1+((2+3)*4)^5",
                "1+2-3*4/5^6",
                "(1+2)/3",
                " (1+3) ",
                "-123",
                "1+2*(-3)",
                "1+2*( - 3)",
                "12.34",
                ".34",
                "-123+456",
                " ( 123 + 456 ) ",
                "-.34",
                "-12.34",
                "-(123+456)",
            };

            string[] sampleInvalidFormulas = new[] {
                "-(123+)",
                "-(*123)",
                "*123",
                "*123a",
                "1.",
                "--1",
            };

            StringBuilder sb = new StringBuilder();
            foreach (var formula in sampleValidFormulas)
            {
                Symbol f = SimpleFormulaParser.ParseFormula(formula);
                SimpleFormulaParser.DumpSymbolRecursive(sb, f, 0);
                sb.Append("==================================" + Environment.NewLine);
            }
            foreach (var formula in sampleInvalidFormulas)
            {
                bool exceptionThrown = false;
                try
                {
                    Symbol f = SimpleFormulaParser.ParseFormula(formula);
                }
                catch (ParserException e)
                {
                    exceptionThrown = true;
                    sb.Append(String.Format("Parsing >{0}< Exception: {1}", formula, e.Message) +
                        Environment.NewLine);
                }
                if (!exceptionThrown)
                    sb.Append(String.Format("Parsing >{0}< Should have thrown exception, but did not",
                        formula) + Environment.NewLine);
            }
            Console.WriteLine(sb.ToString());
        }
    }

    public class ParserException : Exception
    {
        public ParserException(string message) : base(message) { }
    }

    public static class MyExtensions
    {
        public static IEnumerable<T> SkipLast<T>(this IEnumerable<T> source,
            int count)
        {
            Queue<T> saveList = new Queue<T>();
            int saved = 0;
            foreach (T item in source)
            {
                if (saved < count)
                {
                    saveList.Enqueue(item);
                    ++saved;
                    continue;
                }
                saveList.Enqueue(item);
                yield return saveList.Dequeue();
            }
            yield break;
        }

        public static string StringConcatenate(this IEnumerable<string> source)
        {
            StringBuilder sb = new StringBuilder();
            foreach (string s in source)
                sb.Append(s);
            return sb.ToString();
        }

        public static string StringConcatenate<T>(
            this IEnumerable<T> source,
            Func<T, string> projectionFunc)
        {
            return source.Aggregate(new StringBuilder(),
                (s, i) => s.Append(projectionFunc(i)),
                s => s.ToString());
        }

        public static IEnumerable<TResult> Rollup<TSource, TResult>(
            this IEnumerable<TSource> source,
            TResult seed,
            Func<TSource, TResult, TResult> projection)
        {
            TResult nextSeed = seed;
            foreach (TSource src in source)
            {
                TResult projectedValue = projection(src, nextSeed);
                nextSeed = projectedValue;
                yield return projectedValue;
            }
        }
    }
}

Building a Simple Recursive Descent Parser (continued)

Eric White - Independant Consultant — Tue, 03 Aug 2010 18:31:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

In this post, I further enhance the recursive descent parser that will parse the simple grammar that I presented previously in this series. I’ll examine some more classes that represent symbols in that grammar. The classes I present in this post further show how it’s possible to write LINQ code that closely parallels the grammar.

This post is one in a series on using LINQ to write a recursive-descent parser for SpreadsheetML formulas. You can find the complete list of posts here.

In this version of the parser, I’ll enhance the parser to a certain extent, but it still will not be a full implementation of the simple grammar that I defined. In particular, I’m not going to implement infix operators until the next post. Infix operators are a bit more involved, particularly because we need to implement operator precedence. It is not a lot of code, but I use a few useful little LINQ tricks that deserve explanations.

The NospaceExpression Class

In the last version of the parser, I implemented a ‘dummy’ NospaceExpression class. For development and testing purposes, any sequence of symbols passed to that implementation of the NospaceExpression.Produce method were considered to make up a valid nospace-expression symbol.

In this post, I’ll simplify the grammar for the NospaceExpression symbol to the following, eliminating infix operators:

// nospace-expression = open-parenthesis expression close-parenthesis
//         / numerical-constant
//         / prefix-operator expression
~~//         / expression infix-operator expression~~ (not implemented yet)

Here is this implementation of the NospaceExpression class:

public class NospaceExpression : Symbol
{
    public static NospaceExpression Produce(IEnumerable<Symbol> symbols)
    {
        // nospace-expression = open-parenthesis expression close-parenthesis
        //         / numerical-constant
        //         / prefix-operator expression
        ~~//         / expression infix-operator expression~~ (not implemented yet)

        if (!symbols.Any())
            return null;

        // nospace-expression = open-parenthesis expression close-parenthesis
        if (symbols.First() is OpenParenthesis && symbols.Last() is CloseParenthesis)
        {
            Expression e = Expression.Produce(symbols.Skip(1).SkipLast(1));
            if (e != null)
                return new NospaceExpression(new OpenParenthesis(), e, new CloseParenthesis());
        }

        // numerical-constant
        NumericalConstant n = NumericalConstant.Produce(symbols);
        if (n != null)
            return new NospaceExpression(n);

        // prefix-operator expression
        PrefixOperator p = PrefixOperator.Produce(symbols.FirstOrDefault());
        if (p != null)
        {
            Expression e = Expression.Produce(symbols.Skip(1));
            if (e != null)
                return new NospaceExpression(p, e);
        }

        return null;
    }

    public NospaceExpression(params Object[] symbols) : base(symbols) { }
}

Yellow code: The nospace-expression symbol is made up of three possible sets of constituent tokens. It could be made up of an open parenthesis, an expression, and a close parenthesis.

Green code: It could be made up of a numerical constant.

Cyan code: It could be made up of a prefix operator followed by a NospaceExpression.

If it is none of those three, then the Produce method returns null.

You can see that the C# / LINQ code to process the constituent symbols and produce a symbol has a direct correlation to the grammar.

The NumericalConstant Class

The next class I’ll examine in this post is the NumericalConstant class. One interesting characteristic of the grammar associated with this class is that it contains an optional symbol in the grammar, indicated by the square brackets:

// numerical-constant = [neg-sign] significand-part

This is described in the post on the Augmented Backus-Naur Form Grammar, which discusses the various notations that we need to pay attention to.

The NumericalConstant.Produce method first attempts to produce a SignificandPart symbol using the complete list of constituent symbols that was passed to it. If successful, the method can produce a NumericalConstant object with one constituent symbol, the SignificandPart object.

If SignificandPart fails to produce a symbol, then this method asks the NegSign.Produce method to produce a NegSign object, passing the first symbol in the collection to it. If that was successful, then the method asks the SignificandPart class to produce a symbol using a collection of all symbols except the first one (using the Skip extension method).

public class NumericalConstant : Symbol
{
    public static NumericalConstant Produce(IEnumerable<Symbol> symbols)
    {
        // numerical-constant = [neg-sign] significand-part

        SignificandPart s = SignificandPart.Produce(symbols);
        if (s != null)
            return new NumericalConstant(s);
        NegSign n = NegSign.Produce(symbols.First());
        if (n != null)
        {
            SignificandPart s2 = SignificandPart.Produce(symbols.Skip(1));
            if (s2 != null)
                return new NumericalConstant(n, s2);
        }
        return null;
    }
    public NumericalConstant(params Object[] symbols) : base(symbols) { }
}

One key point about the NegSign.Produce method: it will be asked to produce a symbol only for a single symbol, which will be a Minus symbol, so for simplicity, its Produce method takes a singleton Symbol instead of a collection. Here is the NegSign class:

public class NegSign : Symbol
{
    public static NegSign Produce(Symbol symbol)
    {
        // neg-sign = minus

        if (symbol is Minus)
            return new NegSign(symbol);
        return null;
    }
    public NegSign(params Object[] symbols) : base(symbols) { }
}

The SignificandPart Class

As a last example in this post, let’s take a look at the significand-part symbol, which is comprised of one of:

A fractional part
A whole-number-part followed by an optional fractional-part

// significand-part = fractional-part / whole-number-part [fractional-part]

To process this grammar, the first thing to do is to ask the FractionalPart class to produce a symbol, passing all of the symbols that were passed to SignificandPart.Produce. If the FractionalPart class can produce that symbol, then the method can return a SignificandPart object.

If the FractionalPart class can’t produce a symbol, then SignificandPart.Produce needs to ask the WholeNumberPart class to produce a symbol. However, if there is a fractional part following the WholeNumberPart, WholeNumberPart.Produce will not consume all of the constituent symbols. Therefore, WholeNumberPart.Produce has an additional out argument, which will contain any unprocessed symbols. If the unprocessed symbols collection is empty, then there is no fractional part, and SignificandPart can produce its symbol (which has no fractional part).

However, if the unprocessed symbols collection is not empty, then SignificandPart.Produce will ask FractionalPart.Produce to produce a FractionalPart with the unprocessed symbols, and if it does, then SignificandPart.Produce can produce its symbol.

This sounds more complicated than it is. Here is the implementation of the SignificandPart class:

public class SignificandPart : Symbol
{
    public static SignificandPart Produce(IEnumerable<Symbol> symbols)
    {
        // significand-part = fractional-part / whole-number-part [fractional-part]

        FractionalPart f;
        f = FractionalPart.Produce(symbols);
        if (f != null)
            return new SignificandPart(f);
        IEnumerable<Symbol> s = null;
        WholeNumberPart w = WholeNumberPart.Produce(symbols, out s);
        if (w != null)
        {
            if (!s.Any())
                return new SignificandPart(w);
            f = FractionalPart.Produce(s);
            if (f != null)
                return new SignificandPart(w, f);
        }
        return null;
    }
    public SignificandPart(params Object[] symbols) : base(symbols) { }
}

And here is the WholeNumberPart class, which potentially will not consume all of the symbols that are passed to its Produce method.

public class WholeNumberPart : Symbol
{
    public static WholeNumberPart Produce(IEnumerable<Symbol> symbols,
        out IEnumerable<Symbol> symbolsToProcess)
    {
        // whole-number-part = digit-sequence

        IEnumerable<Symbol> unprocessedSymbols = null;
        DigitSequence d = DigitSequence.Produce(symbols, out unprocessedSymbols);
        if (d != null)
        {
            symbolsToProcess = unprocessedSymbols;
            return new WholeNumberPart(d);
        }
        symbolsToProcess = null;
        return null;
    }
    public WholeNumberPart(params Object[] symbols) : base(symbols) { }
}

It, in turn, is comprised of a DigitSequence symbol, which is interesting in that it provides another example of how the LINQ code parallels the grammar. A DigitSequence is comprised of one or more DecimalDigit symbols (which are terminals). The LINQ code essentially says: query for the first n DecimalDigit symbols in the sequence. If there are any, then return a DigitSequence symbol, and set the out parameter to the unprocessed symbols. If there aren’t any, then return null.

public class DigitSequence : Symbol
{
    public static DigitSequence Produce(IEnumerable<Symbol> symbols,
      out IEnumerable<Symbol> symbolsToProcess)
    {
        // digit-sequence = 1*decimal-digit

        IEnumerable<Symbol> digits = symbols.TakeWhile(s => s is DecimalDigit);
        if (digits.Any())
        {
            symbolsToProcess = symbols.Skip(digits.Count());
            return new DigitSequence(digits);
        }
        symbolsToProcess = null;
        return null;
    }
    public DigitSequence(params Object[] symbols) : base(symbols) { }
}

In the next post, I’ll dive into processing infix operators, and how we can write succinct code to implement operator precedence.

Building a Simple Recursive Descent Parser

Eric White - Independant Consultant — Fri, 30 Jul 2010 13:54:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

In this post, I present the start of a recursive descent parser that will parse the simple grammar that I presented previously in this series. I’ll point out some key features of the code so that it is easy to see how the code works.

This post is one in a series on using LINQ to write a recursive-descent parser for SpreadsheetML formulas. You can find the complete list of posts here.

The Symbol Class

We need to write a class for each symbol (production) in the grammar. Most of these classes have a method named Produce that given a collection of Symbol objects, produces an instance of that symbol. Each Produce method in the various classes is free to return null if the collection that is passed to the method can’t produce that symbol.

We need to define an abstract base class for each of these classes. Most instances of classes that derive from the abstract base class will contain a list of constituent symbols, so the abstract base class includes a public property, ConstituentSymbols, of type List<Symbol>.

The ToString method for each of these classes returns the string representation of the symbol. If a symbol is comprised of a list of constituent symbols (i.e. is not a terminal), then the ToString method returns the concatenated strings of the constituent symbols.

There is a constructor for Symbol that takes a params array of Object. Each element in the params array can either be an instance of Symbol, or an object that implements IEnumerable<Symbol>. Those are the only valid types in the params array. The constructor throws an internal ParserException if anything other than one of those types is passed as an argument. I explained this idiom in the previous post in this series, Creating a Collection from Singletons and Collections using LINQ. This method also uses the StringConcatenate extension method, which I discussed in this topic in the LINQ to XML documentation.

public abstract class Symbol
{
    public List<Symbol> ConstituentSymbols { get; set; }
    public override string ToString() {
        string s = ConstituentSymbols.Select(ct => ct.ToString()).StringConcatenate();
        return s;
    }
    public Symbol(params Object[] symbols)
    {
        List<Symbol> ls = new List<Symbol>();
        foreach (var item in symbols)
        {
            if (item is Symbol)
                ls.Add((Symbol)item);
            else if (item is IEnumerable<Symbol>)
                foreach (var item2 in (IEnumerable<Symbol>)item)
                    ls.Add(item2);
            else
                // If this error is thrown, the parser is coded incorrectly.
                throw new ParserException("Internal error");
        }
        ConstituentSymbols = ls;
    }
    public Symbol() { }
}

The OpenParenthesis Class

There are two varieties of symbols – terminal, and non-terminal. I discussed both types of symbols in the post on the Augmented Backus-Naur Form Grammar. A subclass of the Symbol class for terminal symbols is super simple. It contains an override of the ToString method, which returns the string of the terminal. Due to the semantics of C#, it also needs to include a constructor with no parameters:

public class OpenParenthesis : Symbol
{
public override string ToString() { return "("; }
public OpenParenthesis() { }
}

All of the terminals except for DecimalDigit look like the OpenParenthesis class. DecimalDigit is similar, except that its constructor takes a character, and the ToString method returns that character. To make the code as succinct as possible, digits are stored as strings instead of characters.

public class DecimalDigit : Symbol
{
    private string CharacterValue;
    public override string ToString() { return CharacterValue; }
    public DecimalDigit(char c) { CharacterValue = c.ToString(); }
}

The Formula Class

A derived class for non-terminal symbols is also pretty simple. There is one public method, Produce, which takes a collection of Symbol objects, and then produces an instance of the class if it can; otherwise it returns null.

Recursive descent parsers are ‘top-down’ parsers, so it makes sense to define the Formula class first. If you examine the simple grammar that I defined, you can see that a formula is comprised of an expression, so the Formula.Produce method simply passes on the collection of Symbol objects to the Expression.Produce method. Again, due to the semantics of C#, it is necessary to define the constructor that takes a params array of Object, but it doesn’t need to do anything other than call the constructor in the base class.

class Formula : Symbol
{
    public static Formula Produce(IEnumerable<Symbol> symbols)
    {
        // formula = expression

        Expression e = Expression.Produce(symbols);
        return e == null ? null : new Formula(e);
    }
    public Formula(params Object[] symbols) : base(symbols) { }
}

The Expression Class

The Expression class is more interesting. The grammar production for Expression is as follows:

expression = *whitespace nospace-expression *whitespace

This means that an expression consists of zero or more WhiteSpace symbols, followed by one and only one NospaceExpression, followed by zero or more WhiteSpace symbols. Using LINQ, it is super easy to count the WhiteSpace symbols at the beginning and end of the collection of symbols. The Expression.Produce method can then pass the symbols in the middle (between the white space) to NospaceExpression.Produce. If that method returns a NospaceExpression object, then the method can instantiate an Expression object and return it. The Expression.Process method makes use of the SkipLast extension method. The Expression class looks like this:

public class Expression : Symbol
{
    public static Expression Produce(IEnumerable<Symbol> symbols)
    {
        // expression = *whitespace nospace-expression *whitespace

        int whiteSpaceBefore = symbols.TakeWhile(s => s is WhiteSpace).Count();
        int whiteSpaceAfter = symbols.Reverse().TakeWhile(s => s is WhiteSpace).Count();
        IEnumerable<Symbol> noSpaceSymbolList = symbols
            .Skip(whiteSpaceBefore)
            .SkipLast(whiteSpaceAfter)
            .ToList();
        NospaceExpression n = NospaceExpression.Produce(noSpaceSymbolList);
        if (n != null)
            return new Expression(
                Enumerable.Repeat(new WhiteSpace(), whiteSpaceBefore),
                n,
                Enumerable.Repeat(new WhiteSpace(), whiteSpaceAfter));
        return null;
    }

    public Expression (params Object[] symbols) : base(symbols) { }
}

I follow the same pattern for every class – the grammar is included in a comment, followed by a bit of LINQ code to produce an instance of the class, returning null if necessary.

Let’s look at the code to instantiate the Expression object:

return new Expression(
    Enumerable.Repeat(new WhiteSpace(), whiteSpaceBefore),
    n,
    Enumerable.Repeat(new WhiteSpace(), whiteSpaceAfter));

The arguments in yellow are collections of Symbol. The parameter n, which is of type NospaceExpression, is a singleton, so we’re taking advantage of the ability to pass either singletons or collections to the Expression constructor.

The NospaceExpression Class

To keep this first example as simple as possible, we’ll implement a dummy NospaceExpression class. We’ll pretend that any collection of symbols is a valid NospaceExpression symbol. In the next post, I’ll examine how we need to code the NospaceExpression class, as well as classes for some of the other symbols.

public class NospaceExpression : Symbol
{
    public static NospaceExpression Produce(IEnumerable<Symbol> symbols)
    {
        return new NospaceExpression(symbols);
    }

    public NospaceExpression(params Object[] symbols) : base(symbols) { }
}

Projecting a Collection of Symbols from a String

One aspect of the approach that I took is that I first projected a collection of terminal Symbol objects from the collection of characters that make up the string being parsed. The string class implements IEnumerable<char> so we can do a simple select, as follows:

IEnumerable<Symbol> symbols = s.Select(c =>
{
    switch (c)
    {
        case '0':
        case '1':
        case '2':
        case '3':
        case '4':
        case '5':
        case '6':
        case '7':
        case '8':
        case '9':
            return new DecimalDigit(c);
        case ' ':
            return new WhiteSpace();
        case '+':
            return new Plus();
        case '-':
            return new Minus();
        case '*':
            return new Asterisk();
        case '/':
            return new ForwardSlash();
        case '^':
            return new Caret();
        case '.':
            return new FullStop();
        case '(':
            return new OpenParenthesis();
        case ')':
            return new CloseParenthesis();
        default:
            return (Symbol)null;
    }
});

If we parse the formula " (1+3) ", and dump out the terminal symbols, we see this:

Terminal Symbols
================
WhiteSpace > <
OpenParenthesis >(<
DecimalDigit >1<
Plus >+<
DecimalDigit >3<
CloseParenthesis >)<
WhiteSpace > <
WhiteSpace > <

By first transforming the string into a collection of terminals, it allows us to write more expressive code. It is easier to read this code:

If (t is WhiteSpace)

It is a bit harder to read this:

If (t is Character && t.ToString() == " " )

The DumpSymbolRecursive Method

As you can see, Symbol objects form a hierarchical tree – each symbol (except for terminals) has constituent symbols. This allows us to write a recursive method to dump symbols to a StringBuilder object.

public static void DumpSymbolRecursive(StringBuilder sb, Symbol symbol, int depth)
{
    sb.Append(string.Format("{0}{1} >{2}<",
        "".PadRight(depth * 2),
        symbol.GetType().Name.ToString(),
        symbol.ToString())).Append(Environment.NewLine);
    if (symbol.ConstituentSymbols != null)
        foreach (var childSymbol in symbol.ConstituentSymbols)
            DumpSymbolRecursive(sb, childSymbol, depth + 1);
}

If we parse the formula “ (1+3) ” and dump it, we see:

Formula > (1+3) <
Expression > (1+3) <
    WhiteSpace > <
    NospaceExpression >(1+3)<
      OpenParenthesis >(<
      DecimalDigit >1<
      Plus >+<
      DecimalDigit >3<
      CloseParenthesis >)<
    WhiteSpace > <
    WhiteSpace > <

We can see the instances of the Formula, Expression, and NospaceExpression classes, as well as the various terminals that make up the expression.

Complete Listing

Here is the complete listing of the first version of the SimpleFormulaParser example. You can simply cut and paste this code into Visual Studio, and run it. It doesn’t have any dependencies. It transforms a formula to a collection of terminals, prints the terminals, and then uses the Formula, Expression, and NospaceExpression classes to produce a parse tree (which is incomplete because NospaceExpression is a dummy implementation).

In the next post in this series, I’ll continue to develop this simple parser.

Table Markup in Open XML SpreadsheetML

Eric White - Independant Consultant — Wed, 21 Jul 2010 13:59:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

Tables are a feature in Open XML spreadsheets that allow you to interact with data more like a database. You can select a region and convert it to a table. Most Excel aficionados are familiar with this feature. It is very useful.

In a previous post, Using LINQ to Query Excel Tables, I presented some code that allows you to write LINQ queries over Excel tables. The code that retrieves rows from the table is somewhat involved. To be blunt, in that code, I got it wrong. The code reported incorrect values for cells if you reordered, deleted, or inserted columns in the table. I’ve now corrected the code and replaced the attachment to that post, which is a zip file that contains the code, as well as sample spreadsheets.

This post presents a summary of the Open XML SpreadsheetML markup that you use to define or query a table. I’ll show the markup for the very simple table in the above screen-clipping.

We find the definitions for the table in a table part (following is a screen-clippings of the Visual Studio OPC Package Editor, which is an invaluable tool for exploring and editing Open XML documents):

The following listing shows the markup in that part. You can see that there are three columns. The tableColumn elements are in the same order as they occur in the table. The ref attribute of the table element contains the range that the table occupies.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<table xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main"
       id="1"
       name="Table1"
       displayName="Table1"
       ref="A1:C3"
       totalsRowShown="0">
<autoFilter ref="A1:C3"/>
<tableColumns count="3">
    <tableColumn id="1"
                 name="Name"/>
    <tableColumn id="4"
                 name="Gender"/>
    <tableColumn id="2"
                 name="Age"/>
</tableColumns>
<tableStyleInfo name="TableStyleMedium2"
                  showFirstColumn="0"
                  showLastColumn="0"
                  showRowStripes="1"
                  showColumnStripes="0"/>
</table>

The worksheet, sheet1, contains a relationship to the table part. This is how we know that the table is in sheet1.

The following listing contains the worksheet sheet1. The markup for the worksheet is basically the same as it would be for a sheet that didn’t contain a table.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main"
           xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships"
           xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
           mc:Ignorable="x14ac"
           xmlns:x14ac="http://schemas.microsoft.com/office/spreadsheetml/2009/9/ac">
<dimension ref="A1:C3"/>
<sheetViews>
    <sheetView tabSelected="1"
               workbookViewId="0">
      <selection activeCell="E12"
                 sqref="E12"/>
    </sheetView>
</sheetViews>
<sheetFormatPr defaultRowHeight="15"
                 x14ac:dyDescent="0.25"/>
<cols>
    <col min="1"
         max="1"
         width="11"
         customWidth="1"/>
    <col min="2"
         max="2"
         width="13"
         customWidth="1"/>
    <col min="4"
         max="4"
         width="9.85546875"
         customWidth="1"/>
</cols>
<sheetData>
    <row r="1"
         spans="1:3"
         x14ac:dyDescent="0.25">
      <c r="A1"
         t="s"> 
        <v>0</v>
      </c>
      <c r="B1"
         t="s">
        <v>2</v>
      </c>
      <c r="C1"
         t="s">
        <v>1</v>
      </c>
    </row>
    <row r="2"
         spans="1:3"
         x14ac:dyDescent="0.25">
      <c r="A2"
         t="s"> 
        <v>3</v>
      </c>
      <c r="B2"
         t="s">
        <v>4</v>
      </c>
      <c r="C2">
        <v>50</v> 
      </c>
    </row>
    <row r="3"
         spans="1:3"
         x14ac:dyDescent="0.25">
      <c r="A3"
         t="s">
        <v>5</v>
      </c>
      <c r="B3"
         t="s">
        <v>6</v>
      </c>
      <c r="C3">
        <v>45</v>
      </c>
    </row>
</sheetData>
<pageMargins left="0.7"
               right="0.7"
               top="0.75"
               bottom="0.75"
               header="0.3"
               footer="0.3"/>
<tableParts count="1">
    <tablePart r:id="rId1"/>
</tableParts>
</worksheet>

The following listing shows the shared string table. The values for cells that have type = string (the attribute t == “s”) are indexes into this string table.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<sst xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main"
     count="7"
     uniqueCount="7">
<si>
    <t>Name</t>
</si>
<si>
    <t>Age</t>
</si>
<si>
    <t>Gender</t>
</si>
<si>
    <t>Eric</t>
</si>
<si>
    <t>M</t>
</si>
<si>
    <t>Cheryl</t>
</si>
<si>
    <t>F</t>
</si>
</sst>

Writing and Hosting a Web Service in the SharePoint 2010 Demo Virtual Machine

Eric White - Independant Consultant — Fri, 16 Jul 2010 20:48:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

Coding and hosting a WCF web service is a great way to experiment with Business Connectivity Services (BCS) and External Content Types (ECT). In the near future, I’m going to be writing about some of the BCS scenarios where you need to manually code the BCS model. When putting together many-to-many associations or one-to-many where you don’t have a foreign key, you need to manually edit the BCS metadata model.

My favorite way to develop/experiment/write about SharePoint 2010 is to use the 2010 Information Worker Demonstration and Evaluation Virtual Machine. It is a great virtual machine – finely tuned, and has all of the necessary software installed, including SharePoint Server 2010, Visual Studio 2010, Office 2010, etc. I do on occasion build my own virtual machines when absolutely necessary, but I consider it a necessary evil, not something that I enjoy.

Recently, I wrote a blog post / MSDN article on building and hosting a WCF web service. However, to host the web service in the demo / evaluation virtual machine, you need to add a new web site, which is different from the procedure I presented in that post. This blog post presents the correct procedures for building/hosting the web service in that VM. Once you’ve hosted the web service, you can create an ECT from it, create a list for the ECT, set permissions for the ECT in SharePoint 2010 Central Administration, and then view the list. To do this, you can follow the procedure in Consuming a Claims-Enabled WCF Web Service as a SharePoint 2010 External Content Type. If you follow the steps in this blog post, you can ignore steps 26, 27, 28, and 38, which are associated with consuming the claims-aware web service. At some point in the near future, time permitting, I’m going to write procedures for building a claims-aware web service in that virtual machine.

This web service contains only two methods: a ‘finder’ to retrieve a collection of items, and a ‘specific finder’ to retrieve a single item. The ‘database’ behind the collection is just an initialized list. The ‘schema’ of this little ‘database’ is very simple. It is a single flat table consisting of two fields – an integer CustomerID, and a string CustomerName. CustomerID is a unique ID.

return new List<Customer>()
{
    new Customer
    {
        CustomerID = 1,
        CustomerName = "Bob",
    },
    new Customer
    {
        CustomerID = 2,
        CustomerName = "Bill",
    },
    new Customer
    {
        CustomerID = 3,
        CustomerName = "Cheryl",
    },
};

After building and configuring this web service, you can use SharePoint Designer 2010 to create an ECT from it, and then view the data in a SharePoint list.

As you probably know, BCS in SharePoint server 2010 is read/write. If you supply additional methods to create, update, and delete items, then you can fully maintain the data in a list. I’m interested in keeping this web service absolutely as simple as possible, so this is only a read-only implementation.

As I mentioned, the procedure that I present here shows how to host the web service under IIS. This is the way that most implementers of such a web service will want to host it. Hosting it as a service of IIS gives lots of benefits such as process recycling, process health monitoring, and message based activation.

Creating a Web Service

Create a directory, C:\MyWebService, which will contain the web service. If you use a different directory than this, you will need to alter these procedures as appropriate.
Start Visual Studio 2010.
Click File => New => Project.
1. For the installed template category, select WCF.
2. For the template, select WCF Service Application.
3. Target the .NET Framework 3.5.
4. For the location of the project, browse to the directory that you created in step 1.
5. Name the project CustomersService.
6. Do not create a directory for the solution.
  
  Note: Don’t forget to target the .NET Framework 3.5. By default, Visual Studio will target the .NET Framework 4.0, and you must change to 3.5 for the procedure presented here to work.
In the project, rename IService1.cs to ICustomers.cs. Visual Studio 2010 may ask whether you would also like to perform a rename in this project of all references to the code element ‘IService1’? Click Yes. Actually, we’re going to replace all code in all modules, so it doesn’t matter whether you click Yes or No.
In the project, rename Service1.svc to Customers.svc. After renaming these items, the Solution Explorer will look like this.
Replace Customers.svc with the following single line of markup. Right click on Customers.svc, and select View Markup. Copy, paste, and save.

<%@ ServiceHost Language="C#" Debug="true" Service="CustomersService.Customers" CodeBehind="Customers.svc.cs" %>
Replace Customers.svc.cs with the following code.

using System;
using System.Collections.Generic;
using System.Linq;

namespace CustomersService
{
    public class Customers : ICustomers
    {
        // Finder
        public List<Customer> GetAllCustomers()
        {
            return new List<Customer>()
            {
                new Customer
                {
                    CustomerID = 1,
                    CustomerName = "Bob",
                },
                new Customer
                {
                    CustomerID = 2,
                    CustomerName = "Bill",
                },
                new Customer
                {
                    CustomerID = 3,
                    CustomerName = "Cheryl",
                },
            };
        }

        // Specific finder
        public Customer GetCustomerByID(int CustomerID)
        {
            return GetAllCustomers().FirstOrDefault(c => c.CustomerID == CustomerID);
        }
    }
}

Replace ICustomers.cs with the following code.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;

namespace CustomersService
{
    [ServiceContract]
    public interface ICustomers
    {
        [OperationContract] //finder
        List<Customer> GetAllCustomers();

        [OperationContract] //specificFinder
        Customer GetCustomerByID(int CustomerID);
    }

    [DataContract]
    public class Customer
    {
        [DataMember]
        public int CustomerID { get; set; }

        [DataMember]
        public string CustomerName { get; set; }
    }
}

Replace Web.config with the following markup. In Solution Explorer, right-click on Web.config and select Edit. Copy, paste, and save. Important note: this is the Web.config that configures this specific web service. It resides at C:\MyWebService\CustomersService.

<?xml version="1.0"?>
<configuration>
<system.serviceModel>
    <services>
      <service behaviorConfiguration="CustomersService.Service1Behavior"
        name="CustomersService.Customers">
        <endpoint address="" binding="wsHttpBinding" contract="CustomersService.ICustomers">
          <identity>
            <dns value="localhost" />
          </identity>
        </endpoint>
        <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
      </service>
    </services>
    <behaviors>
      <serviceBehaviors>
        <behavior name="CustomersService.Service1Behavior">
          
          <serviceMetadata httpGetEnabled="true"/>
          
          <serviceDebug includeExceptionDetailInFaults="false"/>
        </behavior>
      </serviceBehaviors>
    </behaviors>
</system.serviceModel>
</configuration>

Build the application.

Add a new web site.

Start Internet Information Services (IIS) Manager. Click Start => All Programs => Administrative Tools => Internet Information Services (IIS) Manager.
In Internet Information Services (IIS) Manager, right-click on Sites, and select Add Web Site.
In the Add Web Site dialog box, in the Site name field, enter Customers. In the Application Pool field, select DefaultAppPool. In the Physical path field, enter C:\MyWebService\CustomersService. In the Port field, enter 8181. In the Host name field, enter www.contoso.com. Click OK.
Validate that the web service is running. Start Internet Explorer, and browse to http://www.contoso.com:8181/Customers.svc. If the web service is running, you will see the following:

Another interesting approach to testing the Web service is to use the WCF test client. Start a Visual Studio command prompt. Enter wcftestclient to run the WCF test client. Click File => Add Service. Enter http://www.contoso.com:8181/Customers.svc as the endpoint address, and click OK.

If the service was added successfully, you will see the methods that the service exposes.

Double-click on GetAllCustomers. This opens a window that allows you to configure the request and invoke the request. Click on Invoke to see the response from the Web service.

Creating a Collection from Singletons and Collections using LINQ

Eric White - Independant Consultant — Thu, 15 Jul 2010 15:16:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

A key operation when doing pure functional transformations is the process of creating complex hierarchies of objects. We see this when transforming some data source (such as an Open XML WordprocessingML document) to a LINQ to XML tree, and we see this when writing a recursive descent parser. The recursive descent parser for Excel formulas is nothing more or less than a pure functional transform from the source string to a parse tree, which is a hierarchical object graph. In both of these cases, objects contain ‘child’ or ‘constituent’ objects. An XElement object can contain child XElement objects, and a symbol node in a parse tree is made up of ‘constituent’ or ‘child’ symbols. When constructing an object there are times that you may have one or more singletons, and one or more collections that will comprise the child objects of the object you are constructing.

This post is one in a series on using LINQ to write a recursive-descent parser for SpreadsheetML formulas. You can find the complete list of posts here.

The question is: what is the best way to create a collection of some type T when you have one or more singletons of T, and one or more collections of T. You want the resulting collection to contain all of the singletons and all of the collections in some particular order.

The simplest way to create a collection that contains one element from a singleton is to create an array with a single element:

int z = 5;
var z1 = new[] { z };

So if you have a couple of singletons and a collection, you can create a new collection by creating arrays for the singletons, and then using the concat extension method to assemble the lot into a new collection:

int x1 = 5;
int x2 = 7;
int[] x3 = new[] { 1, 2, 3 };
var c = new[] { x1 }.Concat(new[] { x2 }).Concat(x3);

This isn’t bad, but there is a better way. We can write a method that takes a params array of objects as an argument, and constructs a collection from those arguments.

It is necessary for the method to take a params array of object because in C#, there is no way to declare that a particular parameter can either be a singleton of T or a collection of T. The type system doesn’t allow for this. We have to rely on runtime checking of types passed into the method.

using System;
using System.Collections.Generic;
using System.Linq;

public class MyException : Exception
{
    public MyException(string message) : base(message) { }
}

class Program
{
    static IEnumerable<int> AssembleCollection(params object[] args)
    {
        foreach (var arg in args)
        {
            int? i = arg as int?;
            if (i != null)
            {
                yield return (int)i;
                continue;
            }
            var collection = arg as IEnumerable<int>;
            if (collection != null)
            {
                foreach (var c in collection)
                    yield return c;
                continue;
            }
            throw new MyException("Internal error - argument is not int or collection of int");
        }
    }

    static void Main(string[] args)
    {
        int x1 = 5;
        int x2 = 7;
        int[] x3 = new[] { 1, 2, 3 };
        var c = AssembleCollection(x1, x2, x3);
        foreach (var i in c)
            Console.WriteLine(i);
    }
}

When assembling a collection, it is easier to write code to call AssembleCollection than to use the concat extension method. Compare the two approaches:

var c = AssembleCollection(x1, x2, x3);
var c = new[] { x1 }.Concat(new[] { x2 }).Concat(x3);

In my opinion, the first approach is easier to read.

This is, of course, the approach that LINQ to XML takes. In the following statement, x1, and x2 are singletons, and x3 is a collection.

XElement c = new XElement("z", x1, x2, x3.Where(x => x.Value == "5"));

And if you look at the signature of the XElement constructor, you see the idiom that I’m discussing in this post.

public XElement(XName name, params Object[] content);

The LINQ to XML object model is very, very good. I have learned a huge amount from it.

Recursive Descent Parser: A Simple Grammar

Eric White - Independant Consultant — Tue, 13 Jul 2010 14:08:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

To learn how recursive descent parsers work, it is helpful to implement a very simple grammar, so for pedagogical purposes, I’ve defined a grammar for simple arithmetic expressions. The parser will construct a syntax tree from expressions that we can then examine as necessary. Just for fun, after implementing the parser, we will write a small method that will evaluate the formulas.

This post is one in a series on using LINQ to write a recursive-descent parser for SpreadsheetML formulas. You can find the complete list of posts here.

In these expressions, operands are floating point numbers, but for simplicity, I’ve eliminated the ability to have exponents. Floating point numbers are the only variety of operand; to demonstrate how to write the parser, it’s not necessary to include the idea of variables or other types of operands. When writing the parser for Excel formulas, we’ll need to deal with both exponents and other types of operands, as well as many more varieties of issues.

There are five binary (sometimes called infix) operators. Operator precedence needs to be honored when parsing formulas:

Operator	Precedence
+	1
-	1
*	2
/	2
^	3

There is one prefix operator, ‘-‘, which has higher precedence than the infix operators.

Operands can consist of a significand (sometimes called mantissa) part, followed by a fractional part.

The grammar allows for white space at appropriate places in the expression. The allowance of white space in this simple grammar parallels the allowance of white space in the Excel Extensions to the Office Open XML SpreadsheetML File Format (.xlsx) Specification.

The grammar allows for use of parentheses.

Here are some examples of formulas that should parse properly:

"(1+3)/3"
" (1+3) "
"-123"
"1+2*(-3)"
"1+2*( - 3)"
"12.34"
".34"
"-123+456"
"-(123+456)"
" ( 123 + 456 ) "
"1+2-3*4/5^6"
"-.34"
"-12.34"

One of the important characteristics of a parser is to report when an expression is invalid per the grammar. Here are some expressions that should throw exceptions:

"-(123+)"
"-(*123)"
"*123"
"123a"
"1."
"--1"

Here is the grammar, as I’ve defined it. There are only eleven rules (other than the rules that are comprised only of terminals):

formula = expression
expression = *whitespace nospace-expression *whitespace
nospace-expression = open-parenthesis expression close-parenthesis /
expression infix-operator expression / numerical-constant / prefix-operator expression
numerical-constant = [neg-sign] significand-part
significand-part = whole-number-part [fractional-part] / fractional-part
whole-number-part = digit-sequence
fractional-part = full-stop digit-sequence
neg-sign = minus
digit-sequence = 1*decimal-digit
prefix-operator = plus / minus
infix-operator = caret / asterisk / forward-slash / plus / minus

// The following symbols are comprised only of terminals.
decimal-digit = %x30-39
whitespace = %x20
plus = "+"
minus = "-"
asterisk = "*"
forward-slash = "/"
caret = "^"
full-stop = "."
open-parenthesis = "("
close-parenthesis = ")"

In the next post in this series, I’ll start discussing some of the C# / LINQ techniques that we can use to make coding this parser super-easy.

Exploring the BCS Metadata Schema

Eric White - Independant Consultant — Fri, 09 Jul 2010 18:18:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

When developers are first getting started using BCS, they typically use SharePoint Designer (or perhaps Visual Studio 2010) to put together their BCS model. Using either of those tools, it is pretty easy to discover an external data source, and follow the appropriate steps to connect BCS to the data source and create an external content type (ECT). This is great for the most common scenarios. But sooner or later, you will encounter scenarios where you can’t use SharePoint Designer or Visual Studio to configure your model for you. In these cases, you need to edit the BCS metadata schema directly as an XML file. In addition, even if you never leave the confines of SharePoint Designer or Visual Studio for creating ECTs, it is important to understand the BCS metadata schema. I love using great tooling, but I always want to understand what's going on under the hood.

I’m currently in the process of writing a couple of MSDN articles on creating more advanced associations using BCS, such as creating a one-to-many association where you don’t have a foreign key, and creating many-to-many associations. In both of these scenarios, you need to edit the BCS model directly. As I was wrapping my head around the BCS metadata schema, I thought it would be helpful to me to use an outlining tool to explore the schema. OneNote 2010 has the best outlining tool that I've ever used. It has the necessary capability of collapsing and expanding nodes, promoting and demoting nodes, and moving nodes around by dragging/dropping. In addition, you can place tables at any node, and then format those tables as you like. I put together a OneNote page that contains the elements and attributes of the BCS metadata schema. Being able to expand and contract the nodes enabled me to see the big picture. I made links back to the MSDN documentation topics, so it is easy to navigate to the documentation for each element, attribute, and property.

I thought this outline would be useful to others, so I posted the OneNote page to my Live SkyDrive. You can download it here. After you download it, open it using OneNote 2010. If you are doing SharePoint development, you probably are aware of the 2010 Information Worker Demonstration and Evaluation Virtual Machine. If you don’t have OneNote 2010 on your local machine, you can open this OneNote page using OneNote in that virtual machine. Note that unless you've configured the VM to allow it (which isn't recommended), you can't navigate back to MSDN from the OneNote page.

Note that this is not official documentation in any shape or form. It is just another way to navigate to the official docs.

Here is what the BCS Schema looks like in the OneNote file:

The BCS Metadata schema allows you to supply additional properties for certain elements. When you supply these additional properties, you use the Properties element, which children Property elements. Each Property element has a Name and Type attribute. The value of the property is text content of the Property element.

<LobSystemInstances>
<LobSystemInstance Name="AdventureWorksWS">
    <Properties>
      <Property Name="WcfAuthenticationMode"
                Type="System.String">PassThrough</Property>
      <Property Name="WcfEndpointAddress"
                Type="System.String">http://ericwhit211:8080/WebService.asmx</Property>
      <Property Name="ShowInSearchUI"
                Type="System.String"></Property>
    </Properties>
</LobSystemInstance>
</LobSystemInstances>

I’ve included those properties in my study notes:

I took a few shortcuts when putting together these notes.

Some elements (such as AccessControlList, and LocalizedDisplayNames) occur at a variety of places in the document. I didn’t repeat the attributes and child elements for those elements over and over again in the OneNote page. They are coded in the same way in every location, and it didn’t add any clarity to the outline to repeat that information.

In a few places, a number of attributes are self-explanatory. For instance, in the example below, it is pretty clear that the Name attribute is the name of the Method, so I didn’t bother writing a description for the attribute. As I said, these are just study notes that I assembled for myself that provide a convenient way to navigate through the online documentation.

There is one issue around the TypeDescriptor element that bears mentioning. A TypeDescriptor element can contain a child TypeDescriptors element, which then contains one or more TypeDescriptor element. You will find this when a TypeDescriptor is describing a collection of some type. I’ll examine this issue further when I write about TypeDescriptor elements in more detail. In the outline, I didn't repeat the attributes and properties of the TypeDescriptor that is a descendant of another TypeDescriptor element.

I’ve grappled with the issue of documenting XML vocabularies. The approach taken by the BCS documentation is essentially the same approach as taken by the Open XML specification. For each element, you list its attributes, its child elements, and its parent element. This is pretty good for a reference, but it doesn’t help to put the big picture together. Instead of using OneNote for this purpose, I can imagine putting together a web page with links to documentation of elements and attributes organized in a hierarchical control. This might be a great way to navigate through XML documentation.

Great Video on JavaScript by Douglas Crockford

Eric White - Independant Consultant — Wed, 07 Jul 2010 14:00:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

As a software engineer, it is necessary to stay abreast of relevant technologies. Working with SharePoint exacerbates the problem – if you are going to be a competent SharePoint developer, you must be competent in a wide variety of technologies – C#, VB.NET, ASP.NET, JavaScript, Silverlight, XSLT, LINQ, etc.

One key tool in my toolbox is that I download just about every session for Mix, TechEd, etc. I then put them on my Zune, and listen to them while doing dishes, yard work, walking to the bus, or while on the Stairmaster. I listen to some of them more than once if they are particularly good, and on occasion, some session is so good that I watch it on my computer and make notes.

The presentation at Mix 2010 by Douglas Crockford on JavaScript/EcmaScript is fantastic. I had heard folks talk about the functional programming underpinnings of JavaScript, but I never really understood until I listened to his session.

Download: TechEd 2010 Sessions

Download: Mix 2010 Sessions

Recursive Descent Parser using LINQ: The Augmented Backus-Naur Form Grammar

Eric White - Independant Consultant — Tue, 06 Jul 2010 14:12:00 GMT

[Blog Map] This blog is inactive. New blog: EricWhite.com/blog

A grammar is a device to define syntax for a language. A grammar is made up of rules, sometimes called productions. Each rule defines a symbol, when can then be further used in other rules. Grammars are not hard to understand; most developers instinctively understand grammars when they see them. When you learn a new programming language, almost without thinking about it, you assemble some version of the grammar in your head. One of the benefits of reading the grammar of a language is to make sure that the conceptual grammar you’ve mentally assembled matches the actual grammar of the language.

This post is one in a series on using LINQ to write a recursive-descent parser for SpreadsheetML formulas. You can find the complete list of posts here.

Microsoft devotes a great deal of effort towards writing interoperability documents. If you are a document format geek like me (or even if you only peripherally use document formats), you can find a treasure trove of information on MSDN under Microsoft Office File Format Documents.

The grammar that we want to use to parse SpreadsheetML formulas is in the interoperability document: Excel Extensions to the Office Open XML SpreadsheetML File Format (.xlsx) Specification. This grammar is expressed in Augmented Backus–Naur Form (ABNF).

In this post, I’m going to distill ABNF to just the set of rules and grammar syntax that we need to understand to write a parser for the grammar in the Excel Extensions Specification (linked above). I’ll take all examples of ABFN grammar from that spec.

Terminals

Terminals express the actual text of the programming language. A grammar expresses a terminal either as a quoted string, or sometimes as a list of values, such as the hex values for the digits from “0” – “9”.

decimal-digit= %x30-39

Following is another symbol that uses a literal string terminal.

full-stop = "."

Just to be clear, this is the terminal:

decimal-digit= %x30-39
^^^^^^^

Where this is the grammar rule:

decimal-digit= %x30-39
^^^^^^^^^^^^^^^^^^^^^^

The terminal in the full-stop rule is just the string literal:

full-stop = "."
^^^

Or

A symbol can be comprised of one OR another symbol. In ABNF, “OR” is expressed as a forward slash. The following symbol definition defines constant to be one of the list of varieties of constants.

constant = error-constant / logical-constant / numerical-constant / string-constant / array-constant

The logical-constant symbol is one of two terminals, expressed as quoted strings:

logical-constant = "FALSE" / "TRUE"

Adjacent Symbols

Two symbols separated by a space indicate that you must first have the one symbol, followed by the second symbol. The following rule specifies that the fractional-part symbol requires a full-stop followed by a digit-sequence.

fractional-part = full-stop digit-sequence

Optional

A symbol or terminal that is optional is surrounded by square brackets. The following definition of the exponent-part symbol indicates that the sign before the digit-sequence is optional.

exponent-part = exponent-character [ sign ] digit-sequence

The definition of exponent-character is of course:

exponent-character = "E"

The following examples could produce an exponent-part symbol:

E10
E+10
E-10

Zero or More

If a symbol is preceded by an asterisk (*), zero or more of those symbols can occur at that point in the production of the symbol being defined. The following rule says that an expression can be made up of a ref-expression, or zero or more instances of the whitespace symbol, followed by a nospace-expression, followed by zero or more instances of the whitespace symbol.

expression= ref-expression / *whitespace nospace-expression *whitespace

The symbol bring-to-front-params is defined to be an open parenthesis followed by zero or more space symbols, followed by a close parenthesis.

bring-to-front-params = "(" *space ")"

N or More

If a symbol is preceded by a number followed by an asterisk, it indicates that you must have at least n instances of that symbol. The following defines a digit-sequence to be one or more decimal digits:

digit-sequence = 1*decimal-digit

Exactly N Symbols

If a symbol is preceded by a number, it indicates that you must have exactly n instances of that symbol. The following defines that an escaped-double-quote is comprised of exactly two adjacent double-quote symbols.

escaped-double-quote = 2double-quote

N to M Symbols

The following defines that the and-params symbol consists of an open parenthesis, followed by either a single argument-expression or an argument, followed by 1-254 comma/argument pairs, followed by a close parenthesis.

and-params = "(" (argument-expression / (argument 1*254("," argument))) ")"

Grouped Symbols

Symbols in a production can be grouped by parentheses, and then preceded by a symbol quantifier. The following defines that the constant-list-rows symbol consists of one constant-list-row, followed by zero or more pairs of symbols, where the pair is a semicolon, followed by a constant-list-row.

constant-list-rows = constant-list-row *(semicolon constant-list-row)

Exceptions and Special Rules

In some places, the grammar defines some special rules in text. In our case, the following special rule is defined for an array-constant:

An array-constant MUST NOT contain
An array-constant.
Columns or rows of unequal length.

In addition, following the grammar, there is additional text that describes further restrictions or exceptions. As necessary, we’ll need to incorporate those restrictions.

You can see that grammar rules are not very complex.

My approach for coding the recursive descent parser will be to paste the grammar rule directly into the class that implements the rule as a C# comment. This makes it very easy to correlate the grammar to the code that implements the rule.

In the next post, I’m going to define a super-small grammar that is a subset of the Excel formulas grammar. Then in subsequent posts, we’ll implement and test a parser for that small grammar.

Eric White's Blog

Leaving Microsoft, Blog Moving to EricWhite.com

Using the OData Rest API for CRUD Operations on a SharePoint List

Query a List

Create an Item

Update an Item

Delete an Item

Comparing Excel Automation to Excel Services

Excel Automation

Excel Services

To Summarize

Microsoft Word 2010 Developer Building Blocks

User Interface Extensions

Canvas Extensions

Programmability Extensions

Supported for Previous Versions

Action Pane

ActiveX Controls on Word Document

ActiveX Controls on User Form

AltChunk

Application Level Add-Ins

Automation

Backstage

Business Data Connectivity Object Model

COM Extensions

Command Bars

Content Controls

Context Menu

Custom Task Pane

Custom XML Parts

Document Level Customization

Document Information Panel

Local Database

Open XML SDK

Ribbon

Smart Documents

User Controls

VisualBasicforApplications

How to Install and Activate the IW Demo/Evaluation Hyper-V Machine

Getting Started using the OData REST API to Query a SharePoint List

Procedure: Query a SharePoint list using OData

Understanding the Difference between Custom Task Panes and Action Panes

Understanding the Architecture of Office 2010 Managed Add-Ins and Customizations

What is the Difference between ‘Word Automation’ and ‘Word Automation Services’?

Word Automation

Word Automation Services

Excel Automation vs Excel Services

Understanding the Three Approaches to Office Development using VSTO

Office Application Scenarios

Overview of Office 2010 Application Development

A Short History of Office Development

Visual Basic for Applications (VBA)

Extending Office by using COM

Managed Add-Ins using Visual Studio Tools for Office (VSTO)

What is an Office Application

Office/SharePoint Building Blocks and Developer Stories

Easy SharePoint 2010 Setup for Developers on Windows 7

SharePoint Developer Building Blocks: Technologies for Creating SharePoint Applications

Table of Contents

Introduction

What is a SharePoint Application?

Comparing and Contrasting SharePoint Development with other Forms of Development

Building Scalable .NET Web Applications

SharePoint Development vs. Database Application Development

Client-Side Data Access Technologies

Server-Side Data Access Technologies

SharePoint Development compared to Traditional Rich Client Development

Languages and Platforms

SharePoint Application Development Terms

Varieties of SharePoint Applications

SharePoint Capabilities

SharePoint Developer Scenarios

SharePoint Developer Building Blocks

Important Building Blocks

Access Services

Administration

Alerts

ASP.NET Application Pages

Business Connectivity Services (BCS)

Business Connectivity Services Client-Side API