Imagine Think Create Share : HowTo

How to: Programmatically Manage the Crawl of a Content Source in PowerShell

carloshm — Tue, 31 Mar 2009 19:53:00 GMT

Hi,

If you have read my previous post you may think, why did you stop there? Well, that is what I thought too :), and started with this article from MSDN.

We wanted to start managing the crawling of our content source more programmatically, as we have seen that running several at the same time affects the overall process.

So running these powershell scripts as scheduled task and monitoring the status can improve in crawling.

You may improve them reusing the context and content sources and doing some pipeline

to be included in the overall file:

## SharePoint Reference
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Administration")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.Office.Server.Search.Administration")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.Office.Server.Search")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.Office.Server")

To start an incremental crawl of the content source

function global:StartIncremental-Crawl($url, $csname)
{
trap [Exception] {
  write-error $("ERROR: " + $_.Exception.GetType().FullName);
  write-error $("ERROR: " + $_.Exception.Message);

  continue;
}

$s = new-Object Microsoft.SharePoint.SPSite($url);
$c = [Microsoft.Office.Server.Search.Administration.SearchContext]::GetContext($s);
$sc = new-Object Microsoft.Office.Server.Search.Administration.Content($c);

$cs = $sc.ContentSources[$csname];

$cs.StartIncrementalCrawl();

$s.Dispose();
}

StartIncremental-Crawl -url http://your_site_url -csname "your content source name"

To start a full crawl of the content source

function global:StartFull-Crawl($url, $csname)
{
trap [Exception] {
  write-error $("ERROR: " + $_.Exception.GetType().FullName);
  write-error $("ERROR: " + $_.Exception.Message);

  continue;
}

$s = new-Object Microsoft.SharePoint.SPSite($url);
$c = [Microsoft.Office.Server.Search.Administration.SearchContext]::GetContext($s);
$sc = new-Object Microsoft.Office.Server.Search.Administration.Content($c);

$cs = $sc.ContentSources[$csname];

$cs.StartFullCrawl();

$s.Dispose();
}

StartFull-Crawl -url http://your_site_url -csname "your content source name"

To pause a crawl in process

function global:Pause-Crawl($url, $csname)
{
trap [Exception] {
  write-error $("ERROR: " + $_.Exception.GetType().FullName);
  write-error $("ERROR: " + $_.Exception.Message);

  continue;
}

$s = new-Object Microsoft.SharePoint.SPSite($url);
$c = [Microsoft.Office.Server.Search.Administration.SearchContext]::GetContext($s);
$sc = new-Object Microsoft.Office.Server.Search.Administration.Content($c);

$cs = $sc.ContentSources[$csname];

$cs.PauseCrawl();

$s.Dispose();
}

Pause-Crawl -url http://your_site_url -csname "your content source name"

To resume a paused crawl

function global:Resume-Crawl($url, $csname)
{
trap [Exception] {
  write-error $("ERROR: " + $_.Exception.GetType().FullName);
  write-error $("ERROR: " + $_.Exception.Message);

  continue;
}

$s = new-Object Microsoft.SharePoint.SPSite($url);
$c = [Microsoft.Office.Server.Search.Administration.SearchContext]::GetContext($s);
$sc = new-Object Microsoft.Office.Server.Search.Administration.Content($c);

$cs = $sc.ContentSources[$csname];

$cs.ResumeCrawl();

$s.Dispose();
}

Resume-Crawl -url http://your_site_url -csname "your content source name"

To stop a crawl of the content source

function global:Stop-Crawl($url, $csname)
{
trap [Exception] {
  write-error $("ERROR: " + $_.Exception.GetType().FullName);
  write-error $("ERROR: " + $_.Exception.Message);

  continue;
}

$s = new-Object Microsoft.SharePoint.SPSite($url);
$c = [Microsoft.Office.Server.Search.Administration.SearchContext]::GetContext($s);
$sc = new-Object Microsoft.Office.Server.Search.Administration.Content($c);

$cs = $sc.ContentSources[$csname];

$cs.StopCrawl();

$s.Dispose();
}

Stop-Crawl -url http://your_site_url -csname "your content source name"

To check the crawl status values for a content source

function global:Get-CrawlStatus($url)
{
trap [Exception] {
  write-error $("ERROR: " + $_.Exception.GetType().FullName);
  write-error $("ERROR: " + $_.Exception.Message);

  continue;
}

$s = new-Object Microsoft.SharePoint.SPSite($url);
$c = [Microsoft.Office.Server.Search.Administration.SearchContext]::GetContext($s);
$sc = new-Object Microsoft.Office.Server.Search.Administration.Content($c);

Write-Output $sc.ContentSources;

$s.Dispose();
}

Get-CrawlStatus -url http://your_site_url | Format-Table -property CrawlStatus, CrawlStarted, CrawlCompleted

Enjoy!

How to: Programmatically Export the Crawl History to a CSV File in PowerShell

carloshm — Tue, 31 Mar 2009 18:51:00 GMT

Hi,

When I came across the article at MSDN How to: Programmatically Export the Crawl History to a CSV File I thought I would never create such a tool just for that specific feature, as you end up with additional requirements in order to create an admin tool.

But today I needed to get data from crawl history, and I didn't want to get them from SQL (remember it is not supported ;)), so I started to write down a simple powershell script to do it. And then I realized that for this atomic actions, indeed it is a great options!: you give admin people multiple commands that they can use/combine to monitor/get information about the environment (and yes many, many things more)

## SharePoint Reference
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Administration")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.Office.Server.Search.Administration")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.Office.Server.Search")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.Office.Server")

function global:Get-CrawlHistory($url)
{
trap [Exception] {
  write-error $("ERROR: " + $_.Exception.GetType().FullName);
  write-error $("ERROR: " + $_.Exception.Message);

  continue;
}

$s = new-Object Microsoft.SharePoint.SPSite($url);
$c = [Microsoft.Office.Server.Search.Administration.SearchContext]::GetContext($s);
$h = new-Object Microsoft.Office.Server.Search.Administration.CrawlHistory($c);

Write-OutPut $h.GetCrawlHistory();

$s.Dispose();
}

Then you can just execute: Get-CrawlHistory -url http://your_site_url/ | Export-Csv your_path_and_file_name

Then you can import to excel and make some charts.

In order to filter the information some useful columns should be denormalized: CrawlType, ContentSourceID, Status.

Cheers!

Inside SharePoint - Customize the Mobile Home Page

carloshm — Tue, 30 Dec 2008 02:22:00 GMT

Hi,

These days I've been reviewing different ideas I wrote down in the last months. One of those was about how to extend mobile sharepoint features. So I started re-reading the SDK, landing at How to: Customize the Mobile Home Page through Redirection article. It certainly describes in depth the mechanism of the redirection and indeed it is a great extensibility option, however I found that there are some scenarios where it would need to implement a different approach. Some of those could be:

Use a different name for the redirection
Differentiate the redirection process from the site definition
Customize different home pages for sites with the same site definition
Customize OOB sites, avoiding duplicates names

OOB Feature Redirection

From the article we can see how this (bi-level) redirection mechanism works:

The mobile home page has a short URL with an "m" folder appended to the end of the regular URL (for example, http://Server/sites/Site/m/) that redirects the request to the mobile default.aspx page.
The default.aspx page then redirects the user to the actual home page, according to the current site definition type

The first step uses the Mobility Shortcut URL feature (FeatureID: f41cc668-37e5-4743-b4a8-74d1db3fd8a4 - Name:MobilityRedirect Scope:Web) creating a Module that specifies the virtual path of the folder for the redirection.

<Module Name="mobile" Url="m" Path=""> <File Url="default.aspx" /> </Module>

This will create a folder at the root level of the Site (from a virtual point of view).

In order to test the OOB funcionality, you will need to:

Activate MobilityRedirect feature (it is not activated by default) through the command line as it is hidden.

stsadm -o activatefeature -name MobilityRedirect -url http://Server/sites/Site
To test if it is activated browse to http://Server/sites/Site/m/

If it is working it will redirect yout to a page that looks like figure 1, otherwise it will give you a 404 error

figure 1

Custom Shortcut Feature Redirection

So to solve the scenarios mentioned, the new approach will use a feature to redirect to the final home page, instead of an intermediate redirection page. The feature itself would use a custom control to read the actual final page .

As the OOB feature uses /m/ I would not recommend the same name, as there will be collisions in activating and deactivating processes. Indeed you may find that after deactivating the feature it is still working. This would need to be resolved as part of a complete deactivation in the deactivation event.

So in order to accomplish our solution, we would need to create a feature with a custom shortcut url and a custom file redirect.This scenario would be possible following these steps:

Create a folder at %PROGRAMFILES%\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\FEATURES called SharePoint.Search.Extensions.MobileRedirect or any name you choose

Create a file inside this folder called feature.xml with the following content:

<?xml version="1.0" encoding="utf-8"?> <Feature Id="[GUID]" Title="Mobile Redirector" Description="Mobile Redirector" Version="1.0.0.0" Scope="Web" Hidden="FALSE" DefaultResourceFile="core" xmlns="http://schemas.microsoft.com/sharepoint/"> <ElementManifests> <ElementManifest Location="elements.xml" /> </ElementManifests> </Feature>
Change [GUID] with a valid GUID. Something similar to 00000000-0000-0000-0000-000000000000. I would recommend you the guidgen tool from Visual Studio

Create a file inside this folder called elements.xml with the folllowing content:

<?xml version="1.0" encoding="utf-8"?> <Elements xmlns="http://schemas.microsoft.com/sharepoint/"> <Module Name="mobile" Url="mob" Path=""> <File Url="default.aspx" /> </Module> </Elements>

Create a file inside this folder called default.aspx with the following content:

<%@ Page Language="C#" EnableViewState="false" inherits="Microsoft.SharePoint.MobileControls.SPMobilePage, Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"%> <%@ Assembly Name="Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %> <%@ Register TagPrefix="mobile" Namespace="System.Web.UI.MobileControls" Assembly="System.Web.Mobile, Version=1.0.3300.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" %> <%@ Register TagPrefix="SharePoint" Namespace="Microsoft.SharePoint.WebControls" Assembly="Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %> <%@ Register TagPrefix="SPMobile" Namespace="Microsoft.SharePoint.MobileControls" Assembly="Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %> <SPMobile:SPMobileForm RunAt="Server"> <SPMobile:SPMobileUrlRedirection Runat="Server" PageFileName="[DestinationPage]" /> </SPMobile:SPMobileForm>
Change [DestinationPage] with your desired destination page

Create a file at %PROGRAMFILES%\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\LAYOUTS\MOBILE with the [DestinationPage] name.Something similar to MyNewHomePage.aspx
Install the feature running: stsadm -o installfeature -name SharePoint.Search.Extensions.MobileRedirect
Activate through the command line or through the UI

stsadm -o activatefeature -name SharePoint.Search.Extensions.MobileRedirect -url http://Server/sites/Site
Browse to http://Server/sites/Site/_layouts/ManageFeatures.aspx and click Activate for our feature

Browse to http://Server/sites/Site/mob/ and check the new HomePage

This can be extended with some functionality/Improvements:

Include a custom control and admin page to change the [DestinationPage]
Add localization and custom images to the feature
Change the redirection mechanism to choose between 301 or 302 (permanent or temporary)
Improve feature deactivation
Of course, create a solution to deploy it

Finally, you can have different shortcuts changing the Url attribute of the Module node in the elements.xml file:

<Module Name="mobile" Url="i" Path=""> <File Url="default.aspx" /> </Module>

For example: Create a new shortcut called /i/ to redirect to a IPhone version, for a customized collaborative scenario.

However, in the case of a publishing site you should implement variations to target different devices.

Bye!