Microsoft Business Connectivity Services Team Blog

Office 2010 and SharePoint 2010 Public Beta

The BCS team is excited to announce the Office 2010 and SharePoint 2010 public betas are available for download from www.microsoft.com/2010. Now for the first time anyone can download and test drive the new capabilities Business Connectivity Services has to offer. So go ahead and download it now, try it out and tell us what you think about it.

Along with the software you can also download the SharePoint 2010 (Beta) SDK. Randall and the SDK team did a great job announcing its availability on their blog post: SharePoint Content and Resources. Being in a beta stage, the SDK documentation is continuously updated; so for the most up to date documentation check out the BCS documentation on MSDN.

- Lionel A. Robinson, Program Manager

The Notion of Associations and the External Item Picker

In real world problems there is a frequent need to find related or complimentary information for a specific business entity. For example, while looking at a customer profile, I would love to see additional information about it such as what are the latest orders placed by that customer, or the top 5 products it usually buys or any other relevant information for it, you name it. Business Connectivity Services supports the ability to associate two External Content Types (ECTs) to support such scenarios. This blog post introduces the concept of associations in BCS, how the concept is materialized in the product UI and a quick example on how you can define a simple association between two ECTs using SharePoint Designer 2010 (SPD).

The most common type of association is the one-to-many or master-detail one. Following the classic Customer-Orders example, by defining the association BCS will enable the following scenarios:

• While looking at a customer profile I am able to see to the orders submitted by that customer. (master-detail report)
• While browsing an order I can navigate to the details of the customer that placed it.
• While creating an order you need to assign a customer for it, but you want to pick from a list of available and existing customers rather than guessing the customer’s primary key (identifier) on the order creation form. Once a customer is picked, you want to see the name of the customer rather than its primary key.
• While browsing an order, you want to see more details about the customer that placed it.(reverse lookup)

It’s really easy to create an association between two ECTs. As a prerequisite we need to start with two existing ECTs, in this case Customer and Orders. Then you need to do define a navigation method on the ECT that contains the foreign key, in this case the Order:

1. Right click Orders table and select ‘New Association’

2. A wizard will pop-up in SPD prompting to select to which ECT to create the association, we need to select Customers by clicking on ‘Browse’, then we click ‘Ok’ and ‘Next’

3. Configure input parameters for the navigation method, by specifying the identifier of the associated ECT, in this case the CustomerID. Make sure to select the identifier in the data source elements panel and then hit on the ‘Map to Identifier’ check box on the right.

4. Skip the filter configuration

5. Configure output parameters for the navigation method. The navigation method is meant to return the Orders for the specified Customer so we need to map the return identifier, in this case the OrderID.

6. Click on ‘Finish’.

7. Create BCS artifact, such as external lists, profile pages etc.

Once this is complete we’ll notice the associations are visible in the UI in several parts of BCS components:

1. On the external list forms: The most visible presence of associations will be that on the order creation form, an external item picker will be placed to select a customer for the order. Users now are enabled to pick from an existing list of customers for this form. If the external list is synchronized to Outlook or to SharePoint Workspaces, a rich client version of the picker will be shown on the forms to create orders as well.

On both cases it’s possible to show a friendly display name on the external item picker control upon customer selection to show something more meaningful that the number that identifies the customer in the external system, for instance the customer name, or whatever field that is tagged as the title field in SPD.

2. On the profile pages: Notice how once an association is defined, the orders for the customer will appear on the customer’s profile page. By the same token, in the profile page for Orders, if there is a reverse association defined user will be able to navigate and see more customer details.

3. Business Data Parts: The Business Data List and Business Data related list web parts take advantage of associations and can enable interesting scenarios such as the one shown in the figure below in which the Business Data List part on top lists a set of customers and upon customer selection it shows the related orders on the Business Data related list part below.

So in this post we reviewed how to define association between ECTs on BCS and also saw them in action through external lists, profile pages and business data parts. This topic is just the tip of the iceberg of the association’s concept; on subsequent posts we’ll review more complex scenarios around associations such as self-referential associations, multiple associations, associations between ECTs of different external systems, composite identifier associations and more.

-Juan Balmori Labra, Program Manager

Making Web Services BCS Friendly – Part 1

As mentioned in the overview post, Business Connectivity Services in SharePoint 2010 allows end users to interact with external data using SharePoint and Office UX. For this, BCS provides an abstract data access layer that SharePoint and Office applications can consume. The abstraction is achieved through some stereotype methods supported in BCS. These stereotypes are based on some of the common patterns in popular external systems.

Consider an External list in SharePoint 2010, one of the primary presentation features for external data in SharePoint 2010. A very basic form of an external list depends on two methods:

1. A method for reading the details of an item given its identifier
2. A method to query a list of items given a filtering criterion

For bringing external data into SharePoint via a backend web service, the web service will have to support at least the two methods listed above.

In BCS, these two methods are represented through two stereotypes called “Specific Finder” and “Finder” respectively. Each stereotype imposes some semantic requirements that the backend web service methods have to satisfy for them to be mapped correctly.

In this post, we will review the semantic requirements for a “Specific Finder” and a “Finder” and the design recommendations for the corresponding web services methods to make them BCS friendly. These recommendations are applicable to .Net assemblies and WCF services as well.You can see the full list of stereotypes on MSDN.

Specific Finder

This stereotype is used to read an item given its identifier. For example, given a “customer” business object, this method can be used to obtain detailed customer information for a given customer Id.

Recommendations for the corresponding web service method

To qualify as a Specific Finder, the corresponding web service method should:

• Take an identifier (or combination of identifiers) as one of the input parameters
• Return a set of fields (called a “view” in BCS. See definition below)
• Always return the item identifier as one of the fields returned

View

A “View” is a collection of fields, each of which has a name, a type, and, optionally, localized display names. It is a schematic definition – unlike views in SQL databases which include both the schematic definition and the data returned as part of the query.

Example: If “Employee” is the business object, then a “view” of employee may contain the following fields:

• Id
• Name
• Address
• Designation

It is possible to have multiple views of the same business object in BCS. For example another view of “Employee” may contain the following fields:

• Id
• Name
• Address
• Designation
• AnnualIncome
• StartDate
• SeniorityLevel
• EmergencyContact

Note: If the view returned by the read operation includes fields that automatically change just by reading the item (e.g. TimeStamp for tracking when the item was “Last Read”), they need to be marked with a special attribute called “significant” in the BCS application model.  This is because as part of an update operation, BCS performs a read before updating the item to detect data conflicts. This is done by a state comparison between the cached or in-memory state of the item and the current state in the external system. If this attribute is set to false, these fields will be excluded from the hash calculation to determining data conflicts. If not, data conflicts will be produced for every update operation.

Finder

This stereotype is used for reading a list of items given a filtering criterion. For example, given a business object called “Customer”, this method can be used to obtain a list of customers whose order amount is in a certain range.

Recommendations for the corresponding web service method

To qualify as a Finder, the corresponding web service method:

• Should take the filtering criterion as the input parameter(s) to limit the number of items returned (especially when returning a large number of items)
• Should return collections of items for a given business object
• Should return the identifier of the item as part of the view returned for each item
• Should support rich filtering (See filtering support in BCS for details on what filters to support)
• The “view” for each item returned by this method must be equal to or a subset of the view for the specific finder method. This is because the create and update operations are dependent on the specific finder view. If the finder returns more fields, the extra fields will not be update-able (more details on create/update operations will be covered in the future posts). Also, if finder only returns a subset of the data, a specific finder call will be executed to bring the rest to ensure that the cached items are ‘complete’. Hence both specific finder and finder having identical views is the best case scenario. BCS will verify the views for both during run time and avoid making the extra call if they are identical.

Multiple Specific Finders:

In SharePoint 2010, a business object can have multiple specific finders with different views. Following are some scenarios where multiple specific finders are useful:

• Role based views: Given an EmployeeId, one specific finder method can be used to present a limited view to peers and the global address book while another specific finder method can be used to present the other detailed view to the employee or the employee’s manager.
• Cache optimization: BCS has rich support for taking business data offline to office clients like Outlook and SharePoint Workspace. Administrators can disable offlining for detailed views to optimize cache usage on the client side. This allows users to have a limited view offline and access the detailed view while online.

Note: If there are multiple specific finders with different views of a business object, it is good to have at least one finder method for each specific finder method.

Bulk Specific Finder

This stereotype allows querying for multiple items given a collection of identifiers.

Recommendations for the corresponding web service method

To support this stereotype, the corresponding web service method should take a collection of identifiers as input and return a detailed view for each of the items. All other requirements are exactly same as the specific finder.

In this post we discussed the two basic stereotypes required for presenting external data in a “read only” external list. In the next post we will discuss the create, update and delete stereotypes required for enabling CUD capability followed by future posts that will cover the other advanced stereotypes such as Association Navigators, Id Enumerator etc.

This post was updated 11/20/2009

- Sanjay Rama, Program Manager

BCS Team Channel is now Live!

The first of a series of episodes on the BCS Team Channel is live! Watch our very own Sanjay Rama introduce BCS and show how external data can be viewed and manipulated (Create, Read, Update and Delete data) through the browser.

Upcoming videos will include synchronizing external data into Outlook, SharePoint Workspace, and other increasingly advanced scenarios.

- Lionel Robinson, Program Manager

Types of Solutions

Solutions that leverage Business Connectivity Services (BCS) typically fall into one of three high level categories:

• Simple solution leveraging out-of-box capabilities
• Intermediate declarative solutions
• Advanced code-based solutions

The first two do not require the use of code, although they can re-use code-based components that have been published by developers. This is important for two reasons:

1. They dramatically increase the set of users who can build these types of solutions, freeing up developers and IT organizations to focus their resources on solving other problems, and
2. Central IT retains control over the types of things that these solutions can do, which creates fewer management headaches and has the side benefit of making it easier to “convert” them to a Central IT-supported app rolled out to a larger audience in the future. (Because they are created and customized using a standard set of capabilities and tools)

The Advanced category is entirely code-based, leveraging Visual Studio as the environment to compose and publish re-usable components or entire end-to-end solutions.

Figure 1: Three different types of BCS solutions, types of users who create them, the tools used to create them, and common scenarios that can be accomplished with each.

Let’s review the characteristics and benefits of each type of solution:

Simple Solution Leveraging Out-of-box Capabilities

In SharePoint Foundation 2010, you can surface external data in an external list or on a web part page via the Business Data web parts or the Chart web part. You can also add an External Data column to a standard SharePoint list or document library. That column can then be exposed as a Quick Part (aka content control) in Microsoft Word.

In Outlook 2010, you can take an external list offline from SharePoint Server 2010 with Enterprise Client Access License and allow users to interact with the external data from within Outlook. Users see the same formatting (contact, appointment, task or post) as regular Outlook items and can use the same gestures to interact with them. This type of solution runs under the native BCS Outlook Add-in, which is installed with Office 2010 and loaded at Outlook startup.

In Microsoft SharePoint Workspace 2010, you can take an external list offline from SharePoint Server 2010 with Enterprise Client Access License and allow users to interact with the external data from within SharePoint Workspace. Business Connectivity Services does not provide an extensible programming model to extend this type of a simple solution, but you can associate an InfoPath Form with the External List, which then opens up the ability to customize the form and present that customized form on both the server and on the client in SharePoint Workspace.

Users in Outlook and SharePoint Workspace can synchronize data directly with the external system(s) on an automatic basis (default is 3 hours) or by explicitly clicking an action. They can also check for updates to the external list – i.e. The structure of the external list, its forms or views.

Tool Support

You create external content types by using SharePoint Designer, Visual Studio, or an XML editor. Then, you create an external list using the browser or SharePoint Designer and click a button in the SharePoint ribbon to connect it to Outlook or SharePoint Workspace. Web part pages and external data columns can be created using the browser or SharePoint Designer.

Intermediate Declarative Solution

The most common types of Intermediate solutions are Simple solutions that have been further customized to add capabilities. These additional capabilities include InfoPath forms, SharePoint workflow, and SharePoint web part pages. You can customize InfoPath forms that present External Data by changing the look and feel, adding declarative rules/business logic, or adding code-behind. The latter requires the form to be published as an admin-deployed form to the server – consult the InfoPath 2010 documentation on MSDN for more information about these capabilities, walkthroughs and samples. You can also create or add capabilities to SharePoint workflows through SharePoint Designer by either a) configuring the out-of-box SharePoint List activity to read data from, or write data to an external list, or b) re-using a custom workflow activity built in Visual Studio (and published to SharePoint) that interacts with external lists or the BDC runtime object model. Finally, you can create web part pages that leverage out-of-box web parts (Business Data Item, List, Related List, Actions, Item Builder, and Filter as well as the Chart web part) and optionally configure part-to-part connections to send data between them. The look and feel of parts can be customized by editing the XSLT of each part in the web part toolpane.

Another type of Intermediate customization involves external data in Outlook 2010. An example of a common scenario that can be enabled in this space would be showing related Order information in a custom taskpane when a user has a Business Contact inspector open.

Starting with basic Outlook elements, you can customize the view that is shown for a folder of external data. This is done by opening the folder in Outlook and using the standard commands to build and save a new view. BCS provides a command to save the customized view to SharePoint and place it a sub-folder of the External List, making it available to future users who connect the list to Outlook.

Additional Outlook customizations are slightly more complex than other Intermediate customizations and require the creation of XML files (for example, BDC Model, Solution Manifest (OIR.Config), Subscription, Ribbon, and Layouts) and then a ClickOnce package by using BCS SDK tools. Users can then deploy the solution in Outlook by installing the ClickOnce package. In such a solution, you can define custom taskpanes and present external data to users via External Data Parts (either out-of-box or code-based custom parts) hosted in a taskpane. You can also define ribbon files and custom actions (exposed in the ribbon or in an external data part) that either trigger code or launch a browser pointing to a URL. Finally, you can customize Outlook forms by starting with the auto-generated forms that BCS provides, tweaking them and saving/exporting the file(s). Customizations here leverage the Business Connectivity Services rich client runtime (including the BCS Outlook add-in) which presents the elements defined in the XML files at runtime.

Tool Support

A variety of tools including InfoPath Designer (for forms), SharePoint Designer (for workflows), a browser (for SharePoint web part pages) and Outlook (for customized forms and views) can be used to build intermediate solutions.

To create the XML files needed for your Outlook declarative solution, you can use any XML editor. Visual Studio can provide IntelliSense, which can be helpful when creating these XML files. Samples to get you started will be available as part of the BCS Developer documentation and SDK resources on MSDN. Business Connectivity Services provides an SDK tool that can be used to create a ClickOnce package for Outlook declarative solutions.

Advanced Code-Based Solutions

These solutions can involve the creation of reusable components (.net assembly connector to aggregate or transform data from external systems, custom web parts, custom workflow activities, code-behind for InfoPath forms, and code-based actions or external data parts for use in Outlook declarative solutions) or an entire end to end solution that leverages the public Business Connectivity Services object model.

A code-based Microsoft .NET Framework solution created in a tool such as Visual Studio can use any element of the public Business Connectivity Services object model and can enable users to interact with external data. It can register with the Business Data Connectivity (BDC) service by using the BDC object model to present data in SharePoint, an Office 2010 application such as Microsoft Excel, or a custom application. This object model is installed with SharePoint Foundation 2010 and Office 2010. External data can be retrieved directly from the external system while connected, or it can be retrieved locally from the BCS rich client cache provided that it’s already available (typically achieved by taking external lists offline to SharePoint Workspace or Outlook). This type of solution can be used to extend BCS to Office applications that are not supported out of the box, such as Excel or PowerPoint.

In a code-based end to end solution, the developer controls all of the user interface, packaging, and deployment. This type of solution cannot make use of the Business Connectivity Services rich client runtime, which is used by Simple and Intermediate solutions to integrate data into Office applications.

Tool Support

Code-based solutions are developed in a tool such as Visual Studio 2010. A BDC project type is available to facilitate the creation of .net assembly connectors.

To summarize, there’s a broad spectrum of solutions that can be built using BCS. These range from simple solutions that rely on out-of-box capabilities with little or no customization, to intermediate solutions that involve customizing a wide range of features in SharePoint and Office 2010. Advanced solutions involve the creation of code via Visual Studio, and can either be complete end to end solutions, or provide re-usable code-based components that can be included in an intermediate solution.

This post was updated 11/20/2009

- Brad Stevenson, Sr. Lead Program Manager

BCS Overview Demo Video

In this three-part video Brad Stevenson goes through a high-level overview out-of-the-box functionality provided by Business Connectivity Services. You’ll be able to observe:

• Full read/write capability of BCS
• Create an external content type which connects to a SQL backend
• Bring external data into SharePoint through an external list
• See how the external list provides the same functionality of a normal SharePoint list
• Take the external data into Outlook and SharePoint Workspace
• Auto-generated forms and customized InfoPath forms for external lists
• External Data Columns and using Word templates to fill in information using external data

We hope you enjoy the video and stay tuned for more demos from BCS like Rolando Jimenez’s tooling demo and a series of short 5 minute tours of BCS put together by the team.

Video Part 1 of 3

Video Part 2 of 3

Video Part 3 of 3

- Lionel A. Robinson, Program Manager

Announcing the Business Connectivity Services Resource Center

The SharePoint IT Pro content team is proud to announce the launching of our new Business Connectivity Resource Center. We will keep this resource center up to date with timely information to help you design, build, and deploy SharePoint sites and solutions that integrate external data and services.

Because the Business Connectivity Services Resource Center is going live as Microsoft prepares to release SharePoint Server 2010 Beta, we have made sure that the site links to all the available Business Connectivity Services Beta content, including content for IT professionals, solution designers, and developers. To keep you up to date, we will continue to add links during Beta and beyond. Bookmark this page to keep learning about the latest Business Connectivity Services content.

Along with linking to our own content about Business Connectivity Services, we will use this resource center to highlight great community content. So please use the ratings and feedback control at the top of the page, the Site Feedback control at the bottom, or reply directly to this blog entry to suggest content that you’d like us to link to. You can also use these mechanisms to request additional content that you want us to write or to give us feedback on the resource center.

- Emily Schroeder and Rob Silver, SharePoint IT Pro content team

BCS documentation is now live on MSDN along with the SharePoint 2010 Developer Center

The new SharePoint 2010 Getting Started site is now live and is a great resource to ramp up for SharePoint 2010. You can also find the BCS documentation on MSDN for even greater detail into how things work.

Keep this blog bookmarked or subscribe to the RSS to get the latest updates on what the BCS team is writing about. In the coming week you’ll find posts that talk about how to get your current Web services ready for BCS, why you should choose BCS over other integration services, and we are even working on a sequence of short videos that demonstrate our features in action! We love the feedback that we are getting from you through the contact section of the blog site so keep them coming, also don’t hesitate to ask for content. Our main focus is to answer the burning questions that you have.

- Lionel A. Robinson, Program Manager

BCS at SharePoint Conference 2009

SharePoint Conference was a success and we were incredibly excited about the customer feedback during the sessions and from the twitter feeds. If you were lucky enough attend the conference in Las Vegas hopefully you attended the BCS sessions. You can revisit the sessions you attended or look into those you missed out on using your MySPC access on the conference website. The following is a table of the core and partner BCS sessions; do note that for the links to work best log in to the conference site first.

Core BCS Sessions

Partner BCS Sessions

- Lionel A. Robinson, Program Manager

Overview of Business Connectivity Services

This post kicks off a series of conversations about the new Business Connectivity Services (BCS) capabilities of SharePoint and Office 2010. At a high level, BCS is all about connecting end users with enterprise data that they need to do their job – without having to leave the applications that they use today: Office and SharePoint.

BCS is an evolution of the Business Data Catalog (BDC) capabilities of SharePoint 2007 that enhances the capability of SharePoint as a platform for developing composite applications. It provides out-of-box features, services and tools that streamline development to deeply integrate external data and services. BCS provides the capability to connect SharePoint 2010 and Office 2010 applications to any external system, whether it be a Line-of-Business (LOB) system, (such as Microsoft Dynamics, Oracle, or Siebel) a web 2.0 service, or a custom home-grown application.

How is BCS different from BDC in SharePoint 2007?

BDC entities, web parts and Business Data list columns were designed to provide a read-only window into external systems. BCS provides much deeper integration of external data directly into SharePoint and Office UI in a fully read-write fashion. External Content Types (ECTs) provide a superset of entity capabilities by allowing solution designers to describe both the structure of the external system and how that data should behave within SharePoint and Office. We believe that end users should not have to understand where data comes from; they should be able to view and interact with it in a simple and consistent way across client and server applications.

Key BCS features

Here is a peek at some of our headline capabilities:

• Read-write. You can create, read, update and delete external data from SharePoint and Office applications.
• More connectivity options. In addition to database and web services, connect to WCF services or plug your own code into a .Net assembly connector to handle transformation or aggregation of data. Full support for Claims-enabled services as well as Secure Store Service (formerly SSO) to map user credentials.
• Design and customization tools. Huge investments in SharePoint Designer 2010 and Visual Studio 2010 to allow creation and customization of code-less and code-based solutions. Composite solutions can be collaboratively built by a team with each member using the tool(s) of their choice. Build applications faster and easier.
• Rich Client integration. Expose data as a native SharePoint list and then connect it to SharePoint Workspace or Outlook. Customize InfoPath forms to add business logic surfaced consistently in SharePoint and SharePoint Workspace. Customize Outlook to provide views, forms, ribbon buttons or show contextual data in a taskpane – all without writing code!
• Work online or offline. External data is cached in a SQL CE database installed with Office 2010. When connectivity is lost, the cache automatically goes into offline mode. When connectivity is restored, BCS can synchronize data changes directly to the external system.
• Application Lifecycle. Deploy composite SharePoint solutions to Office 2010 machines, and enable users to check for updates to those solutions.

BCS Architecture

Here’s a diagram showing the BCS tooling and runtime components across SharePoint and Office 2010.

This isn’t a complete tour of BCS capabilities, but rather the 10,000 foot view. Over the next few months we will dive into details. The team looks forward to your feedback through the beta newsgroups and this blog.

- Brad Stevenson, Sr. Lead Program Manager

Welcome and let’s get started!

This blog is the place where the Business Connectivity Services (BCS) product team will help you (Microsoft SharePoint Server 2010 and Microsoft Office 2010 users, developers and IT professionals) learn about the new BCS features in SharePoint Server 2010 and Office 2010. On October 19^th, 2009 Steve Ballmer will take the stage at the SharePoint Conference to unveil the exciting new capabilities of SharePoint Server 2010. From that point forward, the BCS Team will use this blog to bring you informative “How To” guides, insight into features, and demos of what we’ve been building.

In the meantime, for a sneak peek of what’s new in SharePoint Server, check out the SharePoint Server website.

We’re incredibly excited to tell you all about BCS and look forward to your feedback – how you plan to use BCS, which topics you’d like to hear about, and any questions related to blog content. Send us your feedback by posting comments on our blog posts or using the Email Page on this blog.