XML Connector for SharePoint 2013

Create an item details page when using the custom XML indexing connector for SharePoint 2013

2012-10-05T11:57:00Z

Introduction

In this example we are using a custom XML indexing connector to feed XML data directly in to the index. The blog post Create a custom XML indexing connector for SharePoint 2013 describes how to implementing a custom XML indexing connector to feed XML content into the index.

A publishing site is used to display the content. A managed navigation structure is set up on the publishing site, and the navigation nodes, or terms, are configured to use the same rollup page for every node. The blog post Display XML content in a publishing site when using the custom XML indexing connector for SharePoint 2013 describes how to display XML content in a Content Search Web Part(CSWP) on a publishing site. The publishing site uses managed navigation. A single category page, with one CSWP, is assigned to all terms within the managed navigation term set. The query in the CSWP is set to use the terms from the managed navigation term set.

Short recap of previous steps:

At this point we have fed XML data to the search engine, and crawled the content. The publishing site is up and running, using managed navigation. For some terms or nodes in the navigation structure there are listed items. This depends on how many items you have included in your XML file. If you have used our very simple example with only 2 items, you will find these items in the “Cameras” node.

The title of each item consists of a link, but currently this doesn’t display a details page for any of the items.

This blog post

In this blog post I will show you how to create an item details page in your publishing site. When clicking on the title of one of the items displayed on a category page, an item details page will be displayed, rendering data for only that particular item. The steps I will cover are:

How to create an item details page
How to make the item details page show XML content from the right item by:

Modifying the XML file
Creating managed properties
Modifying the category page
Modifying the item details page

..ok, let’s start.

Note: Going forward you will see links like:

http://sphvm-20942/sites/pub1/Pages/DetailsPage.aspx?id=C234

Be sure to rename the parts of these links so they fit your setup.

Sphvm-20942 : Name of my test server. Replace it with the name of your server.

Pub1 : Name of my publishing site collection. Replace it with the name of your publishing site.

DetailsPage.aspx :Name of my item details page. Replace it with the name of your item details page.

C234: ID-number of an individual item. Will change from item to item.

How to create an item details page

On your publishing site, click the Setting menu, then Site contents, and then the Pages icon.
Click on the Files tab, and in the ribbon choose New Document and Catalog-Item reuse.
Give the page a name (e.g. DetailsPage), and select Blank Catalog Item as page layout.
Click Create.

Verify that you have created a page by clicking on the name.

The URL should now be something like: http://sphvm-20942/sites/pub1/Pages/DetailsPage.aspx

We will come back to this DetailsPage later.

How to make the Item Details Page show XML content from the right item.

Modifying the XML file

The XML file contains a field called “Url”. This has to be modified so it points to the page you created in the previous step. In addition, you will have to add the ID as a parameter, so the Content Search Web Part can show the right data.

Let’s go back to the XML file that we have indexed.

<?xml version="1.0" encoding="utf-8" ?>

<ID>C234</ID>

<Title>Contoso camera series2</Title>

<ItemCategoryText>Cameras</ItemCategoryText>

<Url>http://sphvm-20942/sites/pub1/Pages/DetailsPage.aspx?id=C234</Url>

<Category>Cameras/SLR</Category>

<Description>This camera is making great pictures. Easy to use to an affordable price. </Description>

</Product>

<ID>C235</ID>

<Url>http://sphvm-20942/sites/pub1/Pages/DetailsPage.aspx?id=C235</Url>

<Title>Contoso compact camera</Title>

<ItemCategoryText>Cameras</ItemCategoryText>

<Category>Cameras/compact</Category>

<Description>This camera is making great pictures. Easy to use to an affordable price. </Description>

</Product>

</Products>

The field “ID” contains a unique number for each item.

For the field “Url”, insert the URL of the item details page you created in the previous step, and add “?id=<ID if the item>” to the end of the URL. For example, for the item C234, the URL= http://sphvm-20942/sites/pub1/Pages/DetailsPage.aspx?id=C234 has been added.

You will have to do this for every item in the XML file.

Initiate a full crawl.

Creating managed properties

We will now create two managed properties, and map them to the crawled properties “Url” and “ID”. This will enable you to edit the Content Search Web Part so that the correct URL to the item details page is used. I will show you how this is done later in this blog.

The manage property DetailsUrl shall map to the crawled property Url.

The managed property ItemId shall map to the crawled property ID.

In Central Admin => Manage service applications => Search service application => Search Schema, click on the tab Managed properties.
Click on “New managed property”
- Set Name=DetailsUrl
- Set Type=Text
- Select:
  - Searchable
  - Queryable
  - Retrievable
  - Allow multiple values
  - Safe for anonymous
  - Under Mapping to crawled properties, select Add a mapping.
    - Filter on “Custom XML connector” and choose “Url”
    - Click OK
    - Click OK to save your new managed property.

Then, do the same steps to create the next managed property.
- Set Name=ItemId
- Map it to the crawled property “ID”.

In Central Admin => Manage service applications => search service application => Search Schema, click on the tab named Crawled properties.
Under “Category” choose “Custom XML connector” to display all crawled properties for our XML file fed by this connector.
- Verify that the crawled properties “ID” and “Url” are mapped to the managed properties “ItemId” and “DetailsUrl” respectively.
- Initiate a full crawl to index these managed properties.

Modifying the category page

If you have set up your publishing site as described in the blog post Display XML content in a publishing site when using the custom XML indexing connector for SharePoint, your category page contains a Content Search Web part, which uses the display template “Picture on left, 3 lines”. This display template contains placeholders for managed properties, and we will now change the "Link URL" placeholder from Path to DetailsUrl, so that the managed property that contains the correct link will be shown in the Web Part.

On the publishing site, go to Site contents => Pages, and select the Rollup page 1 that you created in earlier (Creating a publishing site when using the custom XML indexing connector for SharePoint 2013).

Click on Edit page, and edit page template.
On the Content Search Web Part, activate the menu on the upper right corner and select Edit web part.
Under Property mapping, check the box for “Change the mapping for managed properties…”, and under LinkUrl, select the managed property DetailsUrl.

By doing this the “Link URL”= http://sphvm-20942/sites/pub1/Pages/DetailsPage.aspx?id=C234 is used when you click on the title for the product.

This happens because we have mapped the placeholder “Linked URL” in the display template to the managed property “DetailsUrl, which contains the right value from the Url-field in the XML file.

Click OK, and then Save the page.

On your publishing site, in the navigation menu, click on Cameras. Click on the item with the title “Contoso camera series2”. An empty details page is displayed. However the URL to the page is correct, and should be: http://sphvm-20942/sites/pub1/Pages/DetailsPage.aspx?id=C234

Modifying the item details page

We now have an item details page that is shown when you click on the title of the item. We will now insert a Content Search Web Part into the details page, and modify the query string so that the correct XML data is displayed through search

Navigate to the items details page by navigating to Site Contents => Pages => DetailsPage
Click the Settings menu, and Edit Page.
In the Web Part Zone, click Add a Web Part.
In the Categories section, click Content Rollup. In the Parts section, click Content Search, and then click Add.

Click Edit Web Part.
In the Web Part tool pane, in the Properties section, click Change query.
In the Web Part query builder, click Switch to advanced mode.
In the Query text section, type ItemId={QueryString.id}

Click OK for the “Build your query” GUI and web part editor, and SAVE your page.

On your main page, navigate to Cameras, and click on link for the item "Contoso camera series2".

The category items page will now look like this:

This is how it works:

The query you specified in the CSWP uses a query variable. A query variable is a placeholder for a value. When a query is run, this placeholder is replaced with a value of the current page. In our example, we are using the query variable {QueryString,<parameter name>},that allows you to query for a value in the URL of the current page. <parameter name> is in our case id. When the URL of a page is http://sphvm-20942/sites/pub1/Pages/DetailsPage.aspx?id=C234, the query variable will be replaced by C234.

The query also uses the managed property ItemId, which we created in an earlier step.

So, when the query is run, the query in the CSWP will resolve to ItemId=C234, and will display the item where the value of the managed property ItemId is C234.

Spicing up the item details page:

If you think that our item details page is boring at this moment, you may be right. It is only showing the title. There is not much data to play with for each item in our XML file, but let’s see what we can do. We also have values for the fields “Price” and “Description”. Let’s make managed properties for these, so we can display this data in the web part.

In Central Admin => Manage service applications => Search service application => Search Schema, click on the tab Managed properties.
Click on “New managed property”

Set Name=ItemPrice
Set Type=Text
Select:
- Searchable
- Queryable
- Retrievable
- Allow multiple values
- Safe for anonymous
Under Mapping to crawled properties, select Add a mapping.
- Filter on “Custom XML connector” and choose “Price”
- Click OK
Click OK to save your new managed property

Then, do the same steps to create the next managed property.
- Set Name=ItemDescription
- Map it to the crawled property “Description”.

Then, let us map these managed properties in the Content Search web part.

Navigate to the items details page by navigating to Site Contents => Pages => DetailsPage
Click the Settings menu, and Edit Page.
On the Content Search Web Part, activate the menu on the upper right corner and select Edit web part.
Under Property mappings, check the box for “Change the mapping for managed properties…”.
For “Link URL”, leave this blank, as we do not want to link to any pages from the item details page.
For “Line2” select the managed property “ItemDescription”.
For “Line3” select the managed property “ItemPrice”.

Click OK, and SAVE your page.

Navigate back to your main page and the term Cameras, select the item Contoso camera series2.

The item details page will now look like this:

This shows how you can play around with managed properties to display content from the XML file. We are still missing an image, but I will talk about that in an upcoming blog.

Display XML content in a publishing site when using the custom XML indexing connector for SharePoint 2013

2012-10-02T12:13:00Z

In our blogpost “Create a custom XML indexing Connector for SharePoint 2013” we showed you how to set up the custom XML connector, make some simple content, and feed & index this. In this blog we will show you how to surface these items in a publishing site including a navigation structure that will look much like if the data where originating from a product catalog.

So at this point you have set up the XML connector, created and mapped the manage property ItemCategoryText , and crawled your XML data from your local directory, so everything is indexed and ready to be fetched by search.

The publishing site, on which you want to display this content, uses managed navigation By specifying a query in a Content Search Web Part that contains terms from your managed navigation term set, you can display indexed items that are associated with each term. To do this, a field in the .xml file has to have the same value as a term in the term set that drives your managed navigation. In our sample .xml file, the field ItemCategoryText contains this value. This value specifies to which term in the term set the entry belongs. For example, for the item with the title “Contoso series2”, the value for ItemCategoryText is “Cameras”. So, when users navigate to the "Cameras" page on the publishing site, the Content Search Web Part can display all entries that have the value "Cameras" for ItemCategoryText (I will show you how to specify the query in the Content Search Web Part later in this blog).

To create a publishing site

The next step is to create a publishing site where the .xml data should be displayed.

On the home page of the SharePoint Central Administration website, in the Application Management section, click Create site collections.
On the Create Site Collection page, enter a title and URL for the site collection.
In the Template Selection section, click the Publishing tab, and select the template Publishing Portal template.

To import a term set to the publishing site

By default, a publishing portal site collection uses managed navigation. You will use the ImportTaxonomy PowerShell script to import the term set as defined in the Taxonomy file. This term set will then be set to drive managed navigation on the publishing portal.

Right click on the file ImportTaxonomy, and select Run with PowerShell.
At the prompt, type the URL of the newly created publishing site.
To verify that the term set has been added to the navigation, open a browser and navigate to your publishing site.

To create a new page and associate it with the terms driving managed navigation

In your publishing site collection, navigate to Site content, and Pages.
Click on the Files tab, and select New Document and Article page.

Give it a Title, e.g. Rollup page 1. The page URL will automatically be filled in.
For the Page layout, choose (Catalog-item-reuse) Blank catalog Item.
Click Create.

You now need to associate this page with the term set you have created.

In Site settings, Term store management, expand the site collection node.
Click on Site navigation and click the Term-driven pages tab.
Select the Target Page Settings checkbox, and use Browse to find your recently created page (Rollup-page-1).
Click Save.

You now have an empty page associated to all your terms in your navigation tree.

To set query in a Content Search Web Part to display the .xml content

Navigate to the newly created page by clicking on “Cameras”.
Click the Settings menu, and Edit Page.
In the Web Part Zone, click Add a Web Part.
In the Categories section, click Content Rollup. In the Parts section, click Content Search, and then click Add.

Click Edit Web Part. In the Web Part tool pane, in the Properties section, click Change query.

In the Web Part query builder, click Switch to advanced mode.
In the Query text section, type ItemCategoryText={URLToken.1}

When this query is run (i.e. when a user clicks on this page), the variable {URLToken.1} is replaced with the last property in the URL.

To see how the query will resolve, click Test Query.

Save your work, and return to your main page and click on Cameras.

Tip: If you are using the 800 item XML file, there are more than 2 items in a term. By default the web part is set up to show only 3 items. Change this by going into the web part editor again.

Set Number of items to show =10

Set Display template => Control = List with paging.

Save your work and return to the main page and navigate to Computers/Laptops.

Your page should now look like:

How to make the publishing site available for anonymous users.

In order to make this publishing web site accessible to anonymous users, you will have to configure “Anonymous access” in:

The web application

Central Admin => manage web applications=> <your web application> => Authentication providers =>Default =>Anonymous access

The Site collection

Site settings => Site permissions => Anonymous access

This ends this blog

Create a custom XML indexing connector for SharePoint 2013

2012-10-01T11:07:00Z

Having a way to feed XML content from a network share or local file directory into the search index is highly useful. This blog post describes how to install and deploy a custom XML connector to the Search service application. The custom XML connector will enable you to crawl content from a network share or local directory, and feed it to the search index. When content is in the search index, it can be displayed in a public facing web site through search.

The follow-up blog post “Display XML content in a publishing site when using the custom XML indexing connector for SharePoint 2013” will describe how you can configure a publishing site to display your XML content through search.

The diagram below shows how a custom XML connector feeds XML content from a network share and local file directory to the search index. This content is then shown in Search Web Parts on a publishing site.

Required components

Before you begin, make sure that you have installed the following:

SharePoint 2013 Server
Visual studio 2012

Install and deploy a custom XML indexing connector for SharePoint Server 2013

In the procedure below, you will extract the files needed to create the custom XML extractor, and install this on your local SharePoint Server 2013 installation on the default Search service application.
Note: The user that performs this operation has to be an administrator for the Search service application.

To install the custom XML connector for SharePoint Server 2013

Download the .zip file at the bottom of this blog. The file includes the following:

-A Microsoft Visual Studio Project to install and deploy the custom XML connector.

-A PowerShell script to create a term set on a publishing site.

-A text file that contains the taxonomy of the term set.

-An .xml file that contains sample .xml product data.

Extract the contents to a folder on your computer, for example c:\CustomXMLConnector .
On the Start menu, click All Programs, click Microsoft Visual Studio 2010, and then click Visual Studio Tools and open a Visual Studio command prompt.
To install the XmlFileConnector.dll, at the command prompt, type the following command and run it:

gacutil /i <ExtractedFolderPath>\bin\Debug\XmlFileConnector.dll

where:
<ExtractedFolderPath> is the folder where your extracted files are located, for example c:\CustomXMLConnector.

Expected result after running this command is "Assembly successfully added to the cache".

To add the correct key and values to the registry, double click the file named xmldoc.reg
On the Start menu, click All Programs, then click Microsoft SharePoint 2013 Products and open a SharePoint 2013 Management Shell as an administrator.
To configure the custom XML connector, at the command prompt, type the following command and run it:

$searchapp = Get-SPEnterpriseSearchServiceApplication -Identity "Search Service Application"
New-SPEnterpriseSearchCrawlCustomConnector -SearchApplication $searchapp –Protocol xmldoc -Name xmldoc –ModelFilePath "<ExtractedFolderPath>\Model.xml"

where:

<ExtractedFolderPath> is the folder where your extracted files are located, for example c:\CustomXMLConnector. The Model.xml file is included in the Visual Studio project.

To confirm the configuration, at the command prompt, type the following command and run it:

Get-SPEnterpriseSearchCrawlCustomConnector -SearchApplication $searchapp

Expected result:

To restart the search service, at the command prompt, type the following commands and run them:

net stop osearch15

net start osearch15

Create new crawled property category for the custom XML connector

When the custom XML connector crawls content, the crawled properties discovered during crawl will have to be added to a crawled property category. You will have to create this category.
Note: The user that performs this operation has to be an administrator for the Search service application.

To create a new crawled property category for the custom XML connector

On the Start menu, click All Programs, then click Microsoft SharePoint 2013 Products and open a SharePoint 2013 Management Shell as an administrator.
To create a new crawled property category, at the command prompt type the following commands and run them, where: <ConnectorName> is the name you want to give the custom XML connector, for example Custom XML Connector:

$searchapp = Get-SPEnterpriseSearchServiceApplication -Identity "Search Service Application"
New-SPEnterpriseSearchMetadataCategory -Name "<ConnectorName>" -Propset "BCC9619B-BFBD-4BD6-8E51-466F9241A27A" -searchApplication $searchapp

The Propset GUID, BCC9619B-BFBD-4BD6-8E51-466F9241A27A, is hardcoded in the file XmlDocumentNamingContainer.cs and should not be changed.

To specify that if there are unknown properties in the newly created crawled property category, these should be discovered during crawl, at the command prompt, type and run the following:

$c = Get-SPEnterpriseSearchMetadataCategory -SearchApplication $searchapp -Identity "<ConnectorName>"

$c.DiscoverNewProperties = $true

$c.Update()

where:

<ConnectorName> is the name you want to give the custom XML connector, for example Custom XML Connector.

To confirm the creation of the crawled property category, on the home page of the SharePoint Central Administration website, in the Application Management section, click Manage service applications, and then Search service application. On the Search Service Application page, in the Queries and Results section, click Search Schema and then Categories.

Create new content source for the XML content

To specify what, when and how the XML content should be crawled, you have to create a new content source for your XML content.
Note: The user that performs this operation has to be an administrator for the Search service application.

To create a new content source for the XML content

On the home page of the SharePoint Central Administration website, in the Application Management section, click Manage service applications
On the Manage Service Applications page, click Search service application.
On the Search Service Administration Page, in the Crawling section, click Content Sources.
On the Manage Content Sources page, click New Content Source.

On the Add Content Source page, in the Name section, in the Name box, type a name for the new content source, for example XML Connector.
In the Content Source Type section, select Custom repository.
In the Type of Repository section, select xmldoc.
In the Start Address section, in the Type start addresses below (one per line) box, type the address from where the crawler should being crawling the XML content. The start address syntax is different depending on where the XML content is located:

If the XML content is located on a local drive, use the following syntax:
xmldoc://localhost/<XMLcontentfolder>/#x=doc:id;;urielm=url;;titleelm=title#

where:
<XMLcontentfolder> is the location of your XML content, for example C$/XMLCATALOG.
You can also change the variables doc:id,url and title. To use our example file, you should type: xmldoc://localhost/C$/XMLCATALOG/#x=Product:ID;;titleelm=Title;;urlelm=Url#

If the XML content is located on a network drive, use the following syntax:
xmldoc://<SharedNetworkPath>/#x=doc:id;;urielm=url;;titleelm=title#

where:
<SharedNetworkPath> is the location of your XML content, for example C$/XMLCATALOG.
You can also change the variables doc:id,url and title. To use our example file, you should type: xmldoc://localhost/C$/XMLCATALOG/#x=Product:ID;;titleelm=Title;;urlelm=Url#

Important: Make sure the crawler account has access to the local drive or network share.

Note: In the specified address, xmldoc is a protocol that was added to the registry. The part enclosed in #, contains properties that will be extracted from the document and will be associated with document ID, title and navigation url. #x=Product:ID matches the content in the .xml sample file that you will use later.

Verify that your newly created content source is shown on the Search Service Application page.

Create sample XML content

Now that you have successfully installed and deployed the custom XML connector, you can use it to extract XML content and display this through search on a publishing site.

To create XML sample content

Create a folder on your computer, for example C:/xmlcatalog.
Create an .xml sample file with following content, and store it in your newly created folder.

<?xml version="1.0" encoding="utf-8" ?>

<Products>

<Product>

<ID>C234</ID>

<Title>Contoso series2</Title>

<ItemCategoryText>Cameras</ItemCategoryText>

<Url>http://<server_name>/sites/pub1/Pages/DetailsPage.aspx?id=C234</Url>

<Category>Cameras/SLR</Category>

<Price>799</Price>

<Description>This camera is making great pictures. Easy to use to an affordable price. </Description>

</Product>

<Product>

<ID>C235</ID>

<Title>Contoso Camera</Title>

<ItemCategoryText>Cameras</ItemCategoryText>

<Url>http://<server_name>/sites/pub1/Pages/DetailsPage.aspx?id=C235</Url>

<Category>Cameras/compact</Category>

<Price>350</Price>

<Description>This camera is making great pictures. Easy to use to an affordable price. </Description>

</Product>

</Products>

To enable crawled items from XML document to appear in CBS with managed navigation make sure to include a field ( e.g. ItemCategoryText.) which will be mapped to the taxonomy term set.

Each record should include a field which will point to the associated category. An item called “Contoso series2” should point to “Cameras”. Then it will be possible to associate menu selection and results in CBS.

The publishing site, on which you want to display this content, uses managed navigation (I will show you how to import a term set to drive managed navigation later in this blog). By specifying a query in a Content Search Web Part that contains terms from your managed navigation term set, you can display indexed items that are associated with each term. To do this, a field in the .xml file has to have the same value as a term in the term set that drives your managed navigation. In our sample .xml file, the field ItemCategoryText contains this value. This value specifies to which term in the term set the entry belongs. For example, for the item with the title “Contoso series2”, the value for ItemCategoryText is “Cameras”. So, when users navigate to the "Cameras" page on the publishing site, the Content Search Web Part can display all entries that have the value "Cameras" for ItemCategoryText (I will show you how to specify the query in the Content Search Web Part later in this blog).

To crawl newly created content source

Now that you have some sample .xml content, you will need to crawl this content.

On the home page of the SharePoint Central Administration website, in the Application Management section, click Manage service applications
On the Manage Service Applications page, click Search Service Application.
On the Search Service Administration Page, in the Crawling section, click Content Sources.
Start a full crawl of the content source you created earlier.

When crawling is complete, ensure that crawled properties where created. On the Search Service Application page, in the section Queries and Results, click Search Schema, and then Categories. Your newly created category should contain 17 crawled properties.

To see the crawled properties, click on the category name.

Verify that a crawled property was created for the ItemCategoryText field by typing ItemCategoryText in the Crawled properties box, and then click the search arrow.

To create a new managed property for ItemCategoryText

Now that you have crawled properties for the .xml content, you have to create a managed property for the field ItemCategoryText. You will use this managed property when you define the query in a Content Search Web Part.

On the Search Service Application page, click Managed Properties.
On the Managed Properties page, click New Managed Property.
In the Property name field, type a name for the managed property, for example ItemCategoryText.
Check the following check boxes:

Searchable
Queryable
Retrievable
Allow multiple values

In the Mappings to crawled properties section, click the button Add a Mapping.
In the Search for a crawled property name section, type ItemCategoryText, and click Find.
In the Select a crawled property section, click ItemCategoryText, and OK.

Verify the mapping by searching for the managed property ItemCategoryText, and see that it is mapped to ItemCategoryText property.

In order to add the new managed property to the search index, you have to do an incremental of the content source.

To start an incremental crawl

On the home page of the SharePoint Central Administration website, in the Application Management section, click Manage service applications
On the Manage Service Applications page, click Search Service Application.
On the Search Service Administration Page, in the Crawling section, click Content Sources.
Start an incremental crawl of the content source you created earlier.(see figure below)
When the incremental crawl has complete, verify that the crawl completed without any errors. In the Diagnostics section, click Crawl Log, and verify that no errors were found.

Using a bigger product set.

The .zip file also includes a bigger product set, containing 800 items.

As you now have a custom XML connector that works:

Copy the Product.xml file to your content source location (C:\XMLCATALOG\)
Initiate a new incremental crawl as stated in the previous section.
When the incremental crawl has complete, verify that the crawl completed without any errors. In the Diagnostics section, click Crawl Log, and verify that no errors were found.

You have now crawled the xml document from your local drive, created and mapped the proper managed property (ItemCategorytext) to the field Itemcategirytext in the xml file, and indexed it.

This blog post ends here.

If you are interested in seeing how you can set up a publishing site to display your XML content through search, please visit our blog post: “Display XML content in a publishing site when using the custom XML indexing connector for SharePoint 2013”.

Acknowledgements

Authors: Anders Fagerhaug (andersfa) and Armen Kirakosyan (armenk)

Please also see the blog post from Nadeem Ishqair on the same subject for SharePoint 2010.

http://blogs.msdn.com/b/nadeemis/archive/2012/06/04/creating-a-bcs-based-search-connector-framework.aspx