Welcome to MSDN Blogs Sign in | Join | Help

PowerShell Intro for SharePoint Administrators - Part 1

PowerShell has been nicely integrated into SharePoint 2010 and is the eventual replacement of stsadm. Stsadm still exists with SharePoint 2010 for backward compatibility reasons but PowerShell is here to stay and will be the most widely adopted and used out of the two. PowerShell is included as a prerequisite which is automatically downloaded and installed during the prerequisite install of SharePoint 2010. Part 1 of this nugget will focus on basics of PowerShell including features and where to start. Part 2 will contain some more advance topics such as scripting and manipulating objects by setting properties or running methods against them.

 

Basics

Windows PowerShell™ is a task-based command-line shell and scripting language designed especially for system administration. Built on the .NET Framework, Windows PowerShell™ helps IT professionals and power users control and automate the administration of the Windows operating system and applications that run on Windows.

Built-in Windows PowerShell commands, called cmdlets, let you manage the computers in your enterprise from the command line. Windows PowerShell™ providers let you access data stores, such as the registry and certificate store, as easily as you access the file system. In addition, Windows PowerShell™ has a rich expression parser and a fully developed scripting language.

Windows PowerShell™ includes the following features:

  • Cmdlets for performing common system administration tasks, such as managing the registry, services, processes, and event logs, and using Windows Management Instrumentation.
  • A task-based scripting language and support for existing scripts and command-line tools.
  • Consistent design. Because cmdlets and system data stores use common syntax and naming conventions, data can be shared easily and the output from one cmdlet can be used as the input to another cmdlet without reformatting or manipulation.
  • Simplified, command-based navigation of the operating system, which lets users navigate the registry and other data stores by using the same techniques that they use to navigate the file system.
  • Powerful object manipulation capabilities. Objects can be directly manipulated or sent to other tools or databases.
  • Extensible interface. Independent software vendors and enterprise developers can build custom tools and utilities to administer their software.

Note: The basics section was taken directly from technet because I couldn’t have written this any better. See the resources section at the bottom of this blog for direct access to the site.

 

PowerShell Features

TAB – Expansion

It’s possible if you do not know an entire cmdlet to tab through what you have typed in order to find a match. Cmdlets always start with a verb-cmdlet name combination. Tab expansion is great for quickly tabbing through all matching cmdlets. For Example, the entire first part of the name (the verb) and the hyphen that follows it must be inputted. For example, if you type get-co and then press the Tab key, Windows PowerShell will automatically expand this to the Get-Command cmdlet. If you press Tab key again, Windows PowerShell replaces this with the only other matching cmdlet name, Get-Content.

 

Pipelining

Pipelining is the process where the output of one command is piped to a second command using the Pipline operator |.

For Example:

get-spwebapplication –identity http://contosoweb/ | new-spcontentdatabase –name contosodb

In this example, the output of get-webapplication is piped over to new-spcontentdatabase cmdlet. The new-spcontentdatabase cmdlet has –webapplication parameter and it’s required. You don’t need to specify this parameter because it’s piped over.

 

Format List

Format-List cmdlet formats the output of a command as a list of properties in which each property is displayed on a separate line. Typically, the alias of FL is used instead via piping.

For example: get-spwebapplication –identity http://contosoweb | FL

 

Aliases

An alias is another name you assign to a cmdlet, function, script, etc…. There are some built-in aliases within PowerShell. You can look at those by running the get-alias cmdlet. It’s possible to create aliases by using the Set-Alias cmdlet.

For Example, create an alias named DanCan which runs get-spwebapplication:

Set-Alias DanCan get-spwebapplication

Now running DanCan will produce the same result as get-spwebapplication.

 

Variables

PowerShell contains basic programming principles such as using Variables. Variables are simply an object that holds something. For Example, you can create variables to hold a specific content database. What you do with that variable is endless in terms of options available which will be discussed in the next nugget. For now, it’s important to understand how to declare variables.

To declare a variable named Var to hold content database named contosoDB type the following:

$var = get-spcontentdatabase –identity contosoDB

Type $var and it will output what’s stored in the variable.

 

 

Getting Help with get-help

PowerShell is accessible via the start menu:

clip_image002

Note: PowerShell is referred to as SharePoint 2010 Management Shell in SharePoint 2010

 

Scenario:

Great, it’s opened now what do I do! In this scenario, an administrator, Dan, must use PowerShell to create a new site collection using team site template and is unsure of which cmdlet or syntax to run.

The Get-Help cmdlet will output every alias, cmdlet, and function using a prefix of *. It’s great for finding out which cmdlet to run and get help on how exactly to run the cmdlet.

For Example: get-help *

Try it out and you’ll see it dump out lots of stuff like aliases and cmdlets. If you want to dump out just cmdlets you could type the following:

Get-help * | where {$_.category –eq “cmdlet”}

All SharePoint cmdlets start with SP so to output only SharePoint cmdlets you type the following:

Get-help *-SP* | where {$_.category –eq “cmdlet”}

This is great but I’d rather pipe it out to txt file. To pipe any output to a text file append > c:\filepath\filename.txt

Get-help *-SP* | where {$_.category –eq “cmdlet”} > c:\odst\output.txt

Note: Don’t worry about understanding the entire syntax, it will be more familiar after reviewing both PowerShell nuggets.

 

Verbs are appended to cmdlets which describe the action taken. For Example: creating, removing, or setting a new/modified value on an object. Since I’m looking to create I could be even more granular by typing the following:

Get-help new-SP* | where {$_.category –eq “cmdlet”}

clip_image004

After some mining, the administrator Dan needs to use the New-SPSite cmdlet. In order to properly run this command Dan needs to know the required parameters and the correct syntax. By default, running the following provides a brief summary and outputs syntax:

Get-help new-spsite

What it doesn’t tell you is which parameters are required and a brief description of each parameter. The following gives you that information:

Get-help new-spsite -full

A partial of the output:

SYNOPSIS
Creates a new site collection at the specified URL.
 

SYNTAX
New-SPSite -Url <String> -OwnerAlias <String> [-AssignmentCollection <SPAssignmentCollection>] [-Confirm <SwitchParameter>]] [-ContentDatabase <SPCon tentDatabasePipeBind>] [-Description <String>] [-HostHeaderWebApplication <SPWebApplicationPipeBind>] [-Language <UInt32>] [-Name <String>] [-OwnerEmail <String>] [-QuotaTemplate <SPQuotaTemplatePipeBind>] [-SecondaryEmail <String>] [-SecondaryOwnerAlias <String>] [-SiteSubscription <SPSiteSubscriptionPipeBind>] [-Template <SPWebTemplatePipeBind>] [-WhatIf [<SwitchParameter>]] [<CommonParameters>]
DETAILED DESCRIPTION

The New-SPSite cmdlet creates a new site collection with the URL and owner specified by the Url and OwnerAlias parameters.

PARAMETERS
-Url <String>
Specifies the URL that the new site collection uses. If this is not a host header site, the URL must start with the containing the Web application URL
        Required?                    true
        Position?                    1
        Default value               
        Accept pipeline input?       True
        Accept wildcard characters?  false

The required parameters are URL and Owner. Dan also wants to specify a team site template so it will look like the following:

New-spsite –url http://contosoweb/sites/snackattack -OwnerAlias contoso\farmadmin -template STS#0

Note – To get a full list of installed templates run the following: get-spwebtemplate

TIP: Get-Command cmdlet –syntax can also if an Administrator just needs to review the syntax.

 

Resources:

http://technet.microsoft.com/en-us/library/bb978526.aspx

http://technet.microsoft.com/en-us/library/dd347700.aspx

http://technet.microsoft.com/en-us/library/dd347728.aspx

http://technet.microsoft.com/en-us/library/dd347730.aspx

Posted by Russmax | 0 Comments

SharePoint 2010 Granular Backup-Restore Part 2

This is part 2 of granular backup/restore blog in which I’ll cover the Recover data from an unattached content database option in Central Administration. This is located in backup/restore section under granular backup/restore. SharePoint 2010 utilizes SQL snapshots both to create and to restore from. This blog will also cover the overall uses of snapshots since they can be used in multiple ways. You can be has granular as possible in choosing what you want to restore from snapshots whether it’s the entire site collection or a specific document library\list.

clip_image002

 

Snapshot Basics

Before going though the steps it’s important to know the basics of SQL snapshots. SQL snapshots were introduced in SQL 2005 timeframe and only available to Enterprise and Developer editions of SQL. SQL 2008 maintains this rule in that snapshots are only available to Enterprise and Developer editions of SQL. SQL snapshots are a read-only copy of a data base as it existed at snapshot creation. Snapshots operate at the DB page level so when a page is about to be modified for the first time in the source DB, the page is first copied to the snapshot thus preserving the data record. If the page already exists and has been modified post snapshot, it’s not updated or copied to the snapshot. At snapshot creation, all pages from source db are considered new and are copied to the snapshot. Snapshots are linked to the source database where they originated. If the source database goes offline for any reason, then the snapshot is unavailable. This is why snapshots are great for granular backup/restore operations but this shouldn’t be your main backup/restore method. Keep in mind that snapshots are limited to content databases only. To learn more about snapshots check out the following:

http://msdn.microsoft.com/en-us/library/ms175158.aspx

 

What can SharePoint 2010 Administrators do with snapshots?

SharePoint 2010 Administrator can now do common administrative tasks when it comes to snapshots. The following lists some common things which can be performed on a SharePoint 2010 server using PowerShell.

· Create snapshots

· Delete snapshots

· Restore snapshot

· Use snapshots for granular recovery operations <- Can also use Central Administration

Note: You cannot create, delete, or restore snapshots on SharePoint 2010 standalone server due to the fact it runs on SQL 2008 express edition. Snapshot operations are not supported against SQL server express edition.

 

Creating Snapshots

Two common methods exist for creating snapshots of SharePoint 2010 data.

 

Method 1: Using SQL Server

Snapshots are created via TSQL commands within SQL Management Studio query window. SQL Management Studio doesn’t expose any UI for creating snapshots. If my content database is named "WSS_Content" then I would run the following query from SQL Management Studio:

CREATE DATABASE ContentSnapshot on (NAME = "WSS_Content", Filename = 'c:\snapshot\contentsnap.ss') as SNAPSHOT OF "WSS_Content";

Once this is completed, the snapshot resides in the snapshots folder.

clip_image004

 

Method 2: Using PowerShell aka “SharePoint Management Console”

SharePoint Management Console can also be used to create snapshots. Yes, this is a fancy way of saying powershell but get used to hearing it since it’s listed that way on the start menu. You can now create SQL snapshots from a SharePoint 2010 server. Using SharePoint Management Console you can input something like the following:

$ContentDB = get-spcontentdatabase WSS_Content

$ContentDB.Snapshots.CreateSnapshot()

Below, a new snapshot is created within SQL Management Studio:

clip_image006

 

 

Recover Data from an unattached content database

The recover data from an unattached content database option is used for pulling data out of a snapshot or detached content database and exporting to a file. The exported file can be imported back into production using the sp-import cmdlet via powershell. Both Central Administrator and Powershell can be used to pull data out of a snapshot. Three specific operations exist depending on what specifically you want to pull out of the snapshot. Selecting the recover data from an unattached content data base option exposes the following operations.

clip_image008

Browse Content – Provides ability to browse for a specific site collection, Site, or list. Once a selection is made, the same operations are available which is backup site collection or Export site or list.

Backup Site collection – Provides the ability to backup a site collection from a snapshot which can then be restored into production

Export Site or List – Provides the ability to export a specific site, list, or document library from a snapshot

 

 

Walkthrough

Using Granular backup/restore to pull a document library from a snapshot and import into production

 

Using Central Administrator

This example demonstrates exporting a document library named hr out of a snapshot named “contentsnapshot2”:

1.) Within Central Administration, Select backup/restore, Recover Data from an unattached content database

2.) The following screen is where the SQL server name is populated and the corresponding snapshot name:

clip_image010

After filling out required fields and selecting Export site or list click next.

3.) After clicking next, drill down to specified list as well as includes options for exporting security, version, and specify export path.

clip_image012

Select the start export button to pull the data from the snapshot into the export.cmp.

4.) Finally, run the import-spweb cmdlet via powershell to import the data back into production. For these steps, see the previous nugget.

 

Using Powershell

This example demonstrates exporting a document library named hr out of a snapshot named “contentsnapshot”:

1.) Get the snapshot database and assign it to variable named $snappy:

$snappy = get-spcontentdatabase -ConnectAsUnattachedDatabase -DatabaseServer contososql -DatabaseName contentsnapshot

2.) Pipe $snappy with export-spweb cmdlet to pull hr list out of the snapshot:

$snappy | export-spweb http://contosoweb –usesqlsnapshot –itemurl /hr –path \\contososql\bu\mybackup.cmp

3.) Finally, run the import-spweb cmdlet via PowerShell to import the data back into production. For these steps, see the part 1 of granular backup/restore blog.

Posted by Russmax | 0 Comments

SharePoint 2010 Granular Backup-Restore Part 1

Hello! Russ Maxwell here and I’d like to provide a glimpse into SharePoint 2010 granular backup/restore. Several things have changed and have been improved in this area. This article is specifically themed around granular backup\restore and what you need to know.

Key Concepts:

Granular Backup: Granular backup has been placed into its own section within Central Administrator. This includes multiple options including the following options:

· Perform a site collection backup

· Export a site or list

· Recover data from an unattached content database

clip_image002

Basics

In SharePoint 2007, all granular backup and restore operations were only available using stsadm. SharePoint 2010 has integrated granular backup restore operations into both Central Administrator and PowerShell. Some backward compatibility does exist with stsadm but PowerShell is the new and improved replacement. Stsadm will be left out of this blog. Part 1 of this blog will include all granular backup/restore options except for “Recover data from an unattached content database”. Part 2 of this series will be dedicated to that topic. Performing backup operations deemed as granular can be performed from Central Administrator or PowerShell. Granular restore operations are only available using PowerShell. SharePoint 2010 is more flexible in terms of what can be backed up and restored. It’s possible to backup and restore site collection, sites, lists, document libraries, and items. The options for performing granular backups using Central Administrator are:

· Perform a site collection backup

· Export a site or list

· Recover data from an unattached content database (Covered in Part 2 of this series)

 

Granular backup operations

Backing up a Site Collection

You can backup a specific site collection using either Central Administrator or Powershell.

Using Central Administrator

  1. Select Backup and Restore
  2. From the UI select Perform a site collection backup
  3. Select the appropriate site collection from the pull down menu
  4. Include proper UNC path including backup file named like: filename.bak
  5. Selecting Start backup redirects to _admin/sitebackuporexportstatus.aspx page.

Using Powershell

backup-spsite -identity http://contosoweb -path \\server\backupshare\bufile.bak

 

Backing up a Site, Library, or List

Exporting a Site/sub-site

Export a specific site using Central Administrator:

1. Select Backup and Restore
2. From the UI select Export a site or list
3. Select Site Collection\Site "leave list drop down clear"
4. Optional categories is to export full security or export versioning history
5. Selecting Start export redirects to _admin/sitebackuporexportstatus.aspx page.

Using Powershell

export-spweb -identity http://contosoweb -path \\servershare\bu\site.cmp

Export a specific list or library:

1. Select Backup and Restore
2. From the UI select Export a site or list
3. Select Site Collection\Site
4. Select a list
5. Optional categories “export full security or export versioning history”
6. Selecting Start export redirects to _admin/sitebackuporexportstatus.aspx page.

Using Powershell

export-spweb -identity http://contosoweb -path \\servershare\bu\hrpowershell.cmp -itemurl /cake

Note: This example demonstrates exporting a document library named cake from Contosoweb site.

 

Granular restore operations

Granular restore operations require the use of Powershell as no UI in Central Administrator exists. Two cmdlets are available for granular restore operations.

restore-spsite

This cmdlet is used to restore site collections that have been backed up using “Perform a Site Collection Backup” in central administrator or backed up using backup-spsite cmdlet.

For Example: restore-spsite -identity http://contosoweb -path \\server\backupshare\backupfile.bak

You have plenty of parameters available but the example demonstrates the required parameters. To get the full scope of parameters you can run:

get-command restore-spsite -syntax

What you should know about using restore-spsite

The identity/URL parameter shouldn't point to a site collection that is present. If the intention is to overwrite an existing site collection, you must use the -force parameter.

import-spweb

This cmdlet is used to restore sites, list, and libraries exported with “export site or list” option in central administrator or using export-spweb cmdlet.

For Example: import-spweb -identity http://contosoweb/ -path \\appserver\bu\list.cmp

Note: You can’t decipher if a list or library is being imported into the contosoweb site using import-spweb. This has been predefined in the backup prior to running import-spweb.

 

What you should know about using import-spweb

Sites and subsites: Importing a site/subsite works as long as you specify a URL with a site that contains a matching template. For Example, importing a team site to http://contosoweb requires that I first provision a site using the team site template using URL http://contosoweb. Attempting to import to a site using a blank template or different template will result in an error.

List and Libraries: It’s important to determine exactly what items you want to import into a list or document library. If no items exist, then running the command in the example above will suffice and all items will be imported. What if a user accidently deletes 5 items out of 1,000 and you simply want to restore those 5 items? By default, the import-spweb cmdlet overwrites items if they exist with the restored version. In this case, the item version prior to restore can be salvaged by restoring the previous version as long as versioning is enabled. To prevent a scenario where you want items which currently exists to be ignored by import operation, you must include the -updateversions parameter with ignore

So in this example, to restore 5 items without affecting the 995 items which currently reside in a list named odst I can run the following:

Import-spweb –identity http://contosoweb/ -path \\appserver\bu\odst.cmp -updateversion ignore

Note: New Items created within the ODST library after initial export will not be touched by an import operation.

 

Check Granular backup job status

A sleek new page has been added to check on granular job back status. Two ways to get to this page is through Central Administrator and after initiating a granular backup automatically redirects you to the _admin/sitebackuporexportstatus.aspx page.

clip_image004

Stay tuned for Part 2 of this series which I’ll discuss how SharePoint 2010 leverages snapshots using the Recover data from an unattached database option.

Posted by Russmax | 2 Comments

SharePoint 2010 Installing prerequisites without an internet connection.

How to automate prerequisite install without an internet connection

I want to cover a few things about the prerequisite installer that everyone should know about.  First, all but one prerequisite component is installed locally.  The rest of them are downloaded from the web during setup.  This is great for servers with an internet connection but what if you want to install prereq’s without an internet connection.  This blog demonstrates how to run automate the install of prerequisite components on a box without requiring an internet connection. 

First, prerequisites need to be manually downloaded and moved over to server:

SQLClient http://go.microsoft.com/fwlink/?LinkId=123718

SyncFramework http://go.microsoft.com/fwlink/?LinkID=141237

MSChart http://go.microsoft.com/fwlink/?LinkID=122517

PowerShell   http://download.microsoft.com/download/D/0/E/D0E6D2C1-2593-4017-B26D-7375BC9263D5/PowerShell_Setup_amd64.msi

Geneva Framework http://download.microsoft.com/download/F/3/D/F3D66A7E-C974-4A60-B7A5-382A61EB7BC6/MicrosoftGenevaFramework.amd64.msi

Microsoft ADOMD.NET http://download.microsoft.com/download/A/D/0/AD021EF1-9CBC-4D11-AB51-6A65019D4706/SQLSERVER2008_ASADOMD10.msi

Automate Installing Prerequisites without Internet Connection

It's possible to install each prerequisite manually on a server with no internet connection but a more automated option exists.  The steps are the following:

1.) Place the downloaded files into a directory.  For my example: C:\sp

2.) Run the following command via command prompt within the root directory of the install media:

PreRequisiteInstaller.exe /SQLnCli:C:\sp\sqlncli.msi /ChartControl:C:\sp\MSChart.exe
/PowerShell:C:\sp\PowerShell_Setup_amd64.msi /Sync:C:\sp\Synchronization.msi
/IDFX:C:\sp\MicrosoftGenevaFramework.amd64.msi /adomd:C:\sp\SQLSERVER2008_ASADOMD10.msi

Posted by Russmax | 0 Comments

SharePoint 2010 Configuring Search Service Application using PowerShell

Create Search Service Application using PowerShell

It might be necessary at some point to use PowerShell to provision search service applications.  For Example, setting up a search service application for hosted sites requires you to use PowerShell.  The following steps manually take you through this process and I highly recommend going through the steps to become more familiar with the command-lets.  ​

Creating Search Service Application using PowerShell

1. Create Application Pool

Creating a an application pool for your search service application and throwing the object into a variable called $ app:

      $app = new-spiiswebserviceapplicationpool –name contososearch-apppool –account domain\user

2. Create search service application

      $searchapp = new-spenterprisesearchserviceapplication -name ContosoSearchServiceApplication -applicationpool $app

Note: Add the -partitioned switch after -name if the search service application will be consumed in a hosted environment.

     3. Create search service application proxy

new-spetnerprisesearchserviceapplicationproxy -name Contososearchserviceapplicationproxy -searchserviceapplicationuri https://contoso:32844/topology/topology.svc?topologyid=c60d41c9-2966-47df-931a-f3ff6f9fbb46"&"appid=bde9d415-7f65-44c8-8fcf-928aa710430c

            Note: Add the -partitioned switch after -searchserviceapplicationuri if the search service application will be consumed in a hosted environment.

Important:  It's recommended to use double quotes around URL but that will fail because it processes the & as an operator.  You must put quotes around the & to pass it as a string.  It doesn't work when using both sets of quotes.  For Example, One set of double "" for wrapping URI and single " for wrapping the & symbol. 

Problem finding the searchserviceapplicationURI?

You can run the following commands to determine this:

A. $contoso = get-spenterprisesearchserviceapplication  ContosoSearchServiceApplication

    B. $contoso | get-member  

URI should be listed as a property

C. $contoso.URI

        Will display the absoluteURi which is what you will use in the –searchserviceapplicationURI parameter

Verify the search service application proxy is online.

If it's not online, you can change the status by punching in the following:

$changestatus = get-spenterprisesearchserviceapplicationproxy hostedsearchproxy

$changestatus | get-member

This will display all the methods and properties available.  One of those properties is status.  To change this property you could type something like this:

$changestatus.status = “online”

Finally, you must update the change by calling the update method.

$changestatus.update()

4. Provision Search Administration Component

Configure the administration component of the associated Searchserviceapplication.  You can do this with the following steps:

$searchapp = get-spenterprisesearchserviceapplication “name of service application”

$si = get-spenterprisesearchserviceinstance –local

set-enterprisesearchadministrationcomponent –searchapplication $searchapp  –    searchserviceinstance $si

 

5. Provision Crawl and Query Components

By default, a search application created in PowerShell has a crawl topology but is missing the following:

· crawl component           

· query component

You cannot add a crawl\query component to the default crawl\query topology because it's set as active and the property is read only.  The easiest way around this is creating a new crawl topology and new query topology.  After creating both, they will be set as inactive by default.  This allows for both crawl components to be added to crawl topology and query component to be added to newly created query topology. Finally, you can set this new crawl topology to active. 

 

Steps are the following:

a. Create object for corresponding search application

$searchapp = get-spenterprisesearchserviceapplication "name of service application"

b. Create Crawl Topology

Create object and input output of newly created crawl topology.

$ct = $searchapp | new-spenterprisesearchcrawltopology

What if you already created a new crawl topology but never dropped it in a variable.  That's okay, you can still drop it in a variable.  First, you must know the ID for the crawl topology which should still be marked as inactive.

$searchapp | get-spenterprisesearchcrawltopology

Let's say the ID in this case is 2e25dcbf-ee71-41e4-b843-3f17d1064386.

$ct = $crawltopology  = $searchapp | where-object {$_.id -eq "2e25dcbf-ee71-41e4-b843-3f17d1064386"}

c. Provision Crawl Component

Create a crawl component for new crawl topology.  $ct can be passed as the identified crawl topology.  We still need to know the crawl store ID because it's     required to create the new-spenterprisesearchcrawlcomponent. 

To find the Guid we can run the following:

$searchapp.crawlstores

                   Copy the GUID for ID.  For Example:

74278909-cb14-46ee-96e2-69ad77ffdfc0

Create a new crawl component.

                   new-spenterprisesearchcrawlcomponent -searchapplication $searchapp -crawltopology $ct -crawldatabase "74278909-cb14-46ee-96e2-69ad77ffdfc0"

Finally, set the new crawl topology as active.

$ct | set-spenterprisesearchcrawltopology -active

d. Create Query Topology

                        Create new Query topology into object $qt

                        $qt = $searchapp | new-spenterprisesearchquerytopology -partitions 1

                        $p1 = ($qt | get-spenterprisesearchindexpartition)

e. Provision query component

    Create a query component for new query topology

                        new-spenterprisesearchquerycomponent -indexpartition $p1 -querytopology $qt -searchserviceinstance $si

                        $p1 | set-spenterprisesearchindexpartition

                        Assign the property store to the index partition

                        $p1 | set-spenterprisesearchindexpartition -propertydatabase "GUID HERE"

                        Where is the propertydatabase GUID?

$searchapp.propertystores to copy the ID "GUID"

Finally, set the query topology as active

$qt | set-spenterprisesearchquerytopology –active

Posted by Russmax | 0 Comments

SharePoint 2010 Multi-Tenant Hosting Part 2 “Configuring”

Part 2 - Configuring Hosting

Configuring hosting requires powershell so the steps are all based off of using it.

1. Create a subscription and assign sites to it:

$sub = new-spsitesubscription

$sub  

2. Pulling the site collection or set of site collections you wish to join to the site group:

get-spsite

$site = get-spsite | where {$_.url -eq "http://contoso"}

$site

​Now you have two variables.  Variable 1 $sub object contains a new spsitesubscription.  Variable 2 $site contains a site collection.

3. Add  the site collection $site, to the newly created site subscription $sub.

set-spsite -identity $site -sitesubscription $sub

Check whether it has been added correctly by doing the following:

get-spsitesubscription

If a database ID exists, then you can type the following

get-spdatabase | where-object {$_.id -match "full or partial guid"}

Will output the results of the associated site collection.

 

4. Create a secondary subscription and associate a different site collection within same web application for demonstration purposes using the steps above.

5. Create a SubscriptionSettings Service Application and Proxy

A.  Start the WSS Subscription Settings Service

B. Create Service Application and Proxy via PowerShell

$appPool  =  New-SPIISWebServiceApplicationPool -Name SettingsServiceApppool -Account domain\use

$sa = new-spsubscriptionsettingsserviceapplication –Name SubscriptionSettingsServiceApplication –Databasename SubscriptionSettingsServiceApplicationDB –applicationpool $appPool

$sap = new-SPSubscriptionSettingsSericeApplicationProxy –ServiceApplication $sa

6. Creating the Tenant Admin Site for each site group

$sub = get-spsitesubscription –identity “http://server”

$tasite = new-spsite –url “http://Contoso/sites/tasite1” –template “tenantadmin#0” –owneralias domain\username –sitesubscription $sub

7. Provision a search service application in hosting mode.  Please see the Configure Search Service application using Powershell blog.

8. Feature's and hosting

Once a feature has been installed into the farm, it's available to all sites and can be activated through manage features pages.  The way to control this in a hosting scenario is you only provide the features available to a given site group through various PowerShell commands.  Any features not listed are excluded and not available for all site collections that belong to the corresponding site group.  Steps are below:

Create a feature set: 

$fs =New-SPFeatureSet

Adding features to a feature set:  

$farm = Get-SPFarm

$feature1 =$farm.FeatureDefinitions | where{$_.ID -eq "02464c6a-9d07-4f30-ba04-e9035cf54392"}

Add-SPFeatureSetMember -Identity $fs -FeatureDefinition $feature1

Adding feature set to subscription:

Set-SPSiteSubscriptionConfig -Identity $sub -FeatureSet $fs

Posted by Russmax | 0 Comments

SharePoint 2010 Multi-Tenant Hosting Part 1

Hosting 101 Part 1

​Before providing the steps of setting up hosting, it's important to understand the main concepts behind O14 hosting.

O14 has a rich set of hosting features which triumph over the previous version in many ways.  It's now simple to setup hosting on the site collection level. 

For Example:

  • Tenant "customer 1" on site collection 1 is hosted on web application A
  • Tenant "customer 2" on site collection 2 is hosted on web application A

Each tenant would only have full administrator rights on his/her assigned site collection.  Customer 2 and users accessing site collection 2 wouldn't be able to access Customer 1's site.

Also, service applications that are in hosting mode would keep each tenants data separate from another tenants.  For example, one shared search service application could service customer 1\site collection 1's data and customer 2/site collection 2's data while keeping them separate from each other.

For Example: Users searching in site collection 1 will not be able to search and find content that resides in site collection 2.  Users searching in site collection 2 will not be able to search and find content that resides in site collection 1.  They will be able to search and locate data within the site collection they are searching from.

Site Groups

The segmentation is possible through the use of site groups also known as site subscriptions.  In the example above, customer 1's site collection belongs to site group 1 and customer 2's site collection belongs to site group 2.

Things to know about site groups:

1. Sites can belong to only one site group at a time.

2. Sites cannot join a site group that contains sites that exists on a different web application.

3. A site group can span across more than one content database

4. No GUI interface for managing site groups.  PowerShell is required to create/manage/remove site groups.

Tenant Admin Site

A hosted customer is referred to as a tenant.  You can provision a tenant admin site which gives the tenant full administrator rights over the site collection.  The tenant admin site can be used to create additional sites for example after self service site creation is enabled.

Service Applications and Hosting

Be default, a service application is consumed at the web application level.  So all sites, under a web application would consume from the same service application and data would be shared.  In hosting mode, a shared service application partitions data where every site group has its own partition.  The partition isn't shared meaning other site groups wouldn't be able to see this data even though all sitegroups are using the same service application.  Configuring hosted service applications may differ based on the type of service application that is being deployed. 

For Example:  Deploying a Shared search service application requires you to use PowerShell with addition of -partitioned switch

Features

It's also possible to deploy features in an "a la carte" manner to site groups.  So one site group can have more features available than another.  This is configurable from PowerShell.

Hosting Part 2 provides a step by step walk though using PowerShell

Posted by Russmax | 0 Comments

Configuring Kerberos Authentication in SharePoint 2010 Part 1

Configure Kerberos Authentication in O14 Part 1

When configuring Kerberos with O14 you will be using IIS 7.0.

Integrated windows authentication is now handled in kernel mode and enabled by default.  This technically was primarily for ease of use and performance boosts since auth is no longer happening in user mode.

 

What are the benefits?

No longer need to register SPN’s for application pool accounts because the AD computer account will be used to decrypt the service ticket.  This is true even if the application pool identify is domain\username.

Note: This works as long as you’re using the netbios or FQDN of the server to access a site.

Unfortunately, in a Web Farm scenario like SharePoint this doesn’t work because you won’t be passing the Netbios/FQDN of the server to access the site unless it’s a standalone install for testing purposes.

The work around for enabling Kerberos in a web “SharePoint” farm you need to specify the application pool identity for the associated web application.  Then you need to create an SPN using setspn tool.

Farm scenario task:

Enable Kerberos for the following:

·         SharePoint Web Application “Sharepoint – 80”

·         Site is named contoso.com

·         Web Application is using domain account, ”contoso\farmadmin”, as application pool identity.

Step 1: Setting useAppPoolCredentials to true in application.config file.

The goal is to continue to use kernel mode authentication in IIS 7.0.  I don’t recommend disabling it for the pure purpose of using Kerberos in an O14 environment.  The first step is setting the useAppPool Credentials to true in application.config file for the associated web site. 

In this example, I want to set this attribute on my “SharePoint – 80” web application:

Locate the application.config file in the following dir:

c:\windows\system32\inetsrv\config\

When you open application.config host file with notepad you will see something like this for web application “WebApplicationName”:

<system.webServer> <security> <authentication> <windowsAuthentication enabled="true" </authentication> </security> </system.webServer>

I don’t like the fact that useKernelMode was missing from the application.config file by default.  The following steps walk you through enabling both attributes.

A.) Disable Kernel-mode authentication for the associated web site by going into IIS Manager.  Select the website\authentication\Advanced Settings and uncheck Enable Kernel-mode authentication.

B.) Perform IIS reset and then go back into Advanced settings and re-enable Kernel-mode authentication by checking the check box.  Perform an IIS reset.

C.) Launch the applicationhost.config file within c:\Windows\System32\inetsrv\config to verify you see the following:

<system.webServer> <security> <authentication> <windowsAuthentication enabled="true" useKernelMode="true"/> </authentication> </security> </system.webServer>

D.) Run appcmd and set useAppPoolCredentials attribute to true for the associated web application.

Appcmd set config “SharePoint - 80” /section:windowsauthentication /useAppPoolCredentials:true /commit:MACHINE/WEBROOT/APPHOST

Now checking application.host config file you should see the following for the associated web application:

<system.webServer> <security> <authentication> <windowsAuthentication enabled="true" useKernelMode="true" useAppPoolCredentials="true" /> </authentication> </security> </system.webServer>

Step 2: Trust the service account for delegation

A.) Launch Active Directory Users and Computers

B.) Locate account running as the application pool identity

C.) Go to properties on the account, select delegation tab

D.) Select “Trust this user for delegation to any service (Kerberos only)

Step 3: Set SPN

SPN is required to map the service/host name to the Application Pool identity. 

A.) Install SPN from the following location:

http://www.microsoft.com/downloads/details.aspx?familyid=5fd831fd-ab77-46a3-9cfe-ff01d29e5c46&displaylang=en

B.) From cmd prompt, run the following cmd:

Setspn.exe –a http/contoso.com contoso\farmadmin

Step 4: Enable Kerberos on the Web Application

A.) Launch Central Admin and select Application Management

B.) Select Manage Web Application and choose the appropriate web application

C.) From the ribbon, select Auth Providers

D.) Select the associated zone and enable Negotiate (Kerberos) and save

Step 5: Verify that Kerberos authentication is working

A.) Go to the security log on the WFE

B.) Filter on all Event ID’s 4624’s

clip_image001

In the above event, you can see the logon process is using Kerberos.  If you scroll up on the event further, you can also get the source computer as well as user account used to log in.  It’s easier to filter on this event with the logon account to confirm on a high traffic server.

With part two, I’ll discuss using Kerberos with service applications.

Posted by Russmax | 1 Comments

Where have I been?

I’m checking in as it’s been awhile since I’ve posted anything.  I’m currently a beta rotational engineer supporting SharePoint 2010.  I’ve been so wrapped up in supporting the new product in it’s early stages that I haven’t had a lot of time to post on 2007.  I’ll be providing an ample amount of blog post on SharePoint 2010 so feel free to check back at a later date.

Next week, I’ll be attending the SharePoint conference Oct 19-22nd in Vegas and will be available via the CSS booth in the exhibition hall.  Stop by and say hello and feel free to ask questions. 

Posted by Russmax | 0 Comments

Configuring Performance Monitor Log for SharePoint Performance Issues.

This is part 2 of troubleshooting SharePoint performance problems.  In order to properly diagnose and troubleshoot SharePoint performance problems, Microsoft CSS support often depends on a performance monitor (perfmon) log during problem period.  In addition, we might request a baseline perfmon of healthy behavior. 

Part 2 of this blog is dedicated to walking you through setting up performance monitor.  Why is that necessary?  We have a very specific method in collecting this data.  A shout out is in store for Jason at MSFT for originally documenting most of these steps.

Steps are below:

 1. Open Performance Monitor, click the + button. Under Performance object, select .NET CLR Memory. Under "Select counters from list", verify that you see Process ID   

 

 

 

 

 

 

 

 

 

 

 

 

In this case, it's clear that the PID information is missing so you must fix the counters as this is what enables a mapping of w3wp to a process ID. 

a)      Click Start, click Run, type cmd, and then click OK.

b)      At the command prompt, type unlodctr .NetFramework, and then press ENTER.

c)       Use the cd command to change to the Windows\Microsoft.Net\Framework\v2.0.50727 folder.

d)      At the command prompt, type lodctr corperfmonsymbols.ini, and then press ENTER

Reference the following article for more information: http://support.microsoft.com/kb/922775

2. Add this registry key so that Performance Monitor will list the PID of the W3WP as part of the counter detail.

 

The following KB article explains how to do this: http://support.microsoft.com/kb/281884

 

3. Open Performance Monitor

 

4. Expand Performance Logs and Alerts

 

5. Right click on Counter Logs, click New Log Settings and type a name for the Perfmon log and click OK

 

6. Click on the Add Objects button. Then add the following objects and click Close       

  • .NET CLR Exceptions 

  • .NET CLR Loading

  • .NET CLR Memory

  • .NET Networking

  • ASP.NET

  • ASP.NET Applications

  • Memory

  • Network Interface

  • Physical Disk

  • Process

  • Processor

  • System

  • Web Service

  • Web Service Cache

Note:  Add the Threads object when troubleshooting CPU Spikes/CPU high utilization.

7. Make sure to use "Run As" so we can accurately capture the .NET counters. This should be run as the same account as the application pool.

 

8. Setup performance logging at 5 second intervals.

 

9. Select the Log Files tab and Configure button and set the preferred directory path and file name.  Limit the log file size to 250 MB and hit OK.  

 

Example screenshot below:

 

 

 

 

 

10. Select the Schedule tab and the start up option is your decision on whether or not you want to automatically start the log or manually. 

 

The Stop Log section settings need to match the following screenshot:

 

 

 

The goal is that no information is going to be overwritten.  So when the buffer limit of 250 MB is reached, a new perfmon log will be created and the existing one is preserved.

 

Hit okay and if you set it to start automatically based on a specified time, then you can check Performance Monitor and verify its running.  If you set the Start Log option to manual, then you need to go into Performance Monitor and manually start the log. 

 

 

  

 

Great, now perfmon log is up and running but we still need one more piece of information.  The PID information is helpful but we need to be able to map which PID or PID's are mapped to a specific application pool.  This requires the use of IISAPP.   Steps for running IISAPP are the following:

 

11.  Go to the same server you configured perfmon and launch cmd prompt.  Type the following:

iisapp

After hitting enter, you might get prompted to register cscript.  Hit yes and run iisapp again.

You will see an output like the following:

 

You can see the value in the screen shot.  Now when I launch a perfmon to review and look at process information, I can map the PID to the associated application pool. 

For data collection purposes, I recommend piping this out to a txt file like running iisapp > c:\iisappout.txt.

I would run this at least every 30 minutes and you can easily configure a script or batch file to automate this process.

Note: IISAPP doesn't work in Windows 2008.  In order get PID  <-->  Application Pool mapping you must run the following:

Launch cmd prompt and run the following command from the c:\Windows\System32\inetsrv directory:

appcmd.exe list wp

Question/Answer Section

Q: Why do I suggest 5 second interval for sampling period?

A: It really depends on how often the performance problem reproduces and how long you decide to leave the log running.  If your capture period is 8 hours or less, then I recommend 5 seconds.  If your capture period is more than 8 hours, I assume that it's unknown when the problem will occur which requires a long running perfmon.  In this case, I recommend a sample period of 120 second interval.  The key is managing how rapidly the perfmon log (BLG file)  grows in size. 

Q: Shouldn't I set the log to overwrite instead of creating new BLG's during long running perfmon sample?

A:  This is personal preference but several conditions exists where you probably want to set the log to overwrite

Conditions:

1.  You are limited on available free space on directory holding the perfmon output (BLG files)

2.  Your not sure when the performance problem will reproduce and it could take 2/3 days or even a few weeks. 

 

I will cover a nice in depth look of SPDiag in part 3 of Troubleshooting SharePoint Performance series.

Troubleshooting Security Only Crawl

What is Security Only Crawl?


Before troubleshooting Security Only Crawl’s, let’s start out with what is a security only crawl?  Security Only crawl's take place when users are added/removed from Sharepoint groups and/or explicitly added/removed from a list.  When incremental crawl starts, these security changes, “Updated ACL’s”, must be pushed down to all affected items within the index.  We security trim these results.

For Example, User A is able to search on all items in a specific document library named HR Docs.  You don’t want User A to be able to search and discover documents which were crawled previously from Document Library named HR Docs.  You remove User A from the only Sharepoint group that has permissions to view these documents and iniate an incremental crawl.  What will happen is that the security change will be discovered and the updated ACL will be processed.  The gatherer will commit the security changes to all items within HR Docs library.  When the security only crawl is completed, User A will not return results for querying items within HR Docs library.

Note:   The items within the document library are not fetched but the security of the item is. 

Troubleshooting Security only crawls?

This is an interesting topic and one that is rarely documented which is the primary reason for this blog.  What would necessarily lead you down the path to troubleshooting security only crawls?  Well there are a few scenarios such as:

1.)  User A is still able to search and return items which he shouldn’t be due to being removed from Sharepoint groups.

2.)  User A is not able to search items even though he was recently added to a Sharepoint group with rights to the items in question.

3.)  Incremental crawls are intermittently taking a long time to crawl very few items.

Scenario three is more frequent and tends to cause the most confusion for Sharepoint Administrators so I’ll discuss this in more detail.  What you will learn from scenario three should help you in troubleshooting scenarios one and two.   
 

Scenario 3:  My incremental crawls are taking longer than expected to crawl a very small # of items.

You perform an incremental crawl which takes 30 minutes.  You perform an incremental crawl the following day and crawl the same # of items and it takes 3 hours.  This starts to become a trend and typically raises a red flag.   The other symptom is that the crawl status page within Central Administrator will appear as crawling with no items updated for a long period of time.  Eventually, the crawl status will update with crawl complete.  The assumption is that the crawler appears to be getting hung or stuck during these long incremental crawls.  The problem is that you really don’t know if the crawler is still running or not.  I typically see the above symptoms when security only crawls are taking place.  Please keep in mind their are several moving parts to the incremental crawl process and I'm just identifying a common one.   

Note:  There is no easy way to tell if a security only crawl is running.  The reason is because the search status page doesn’t log any security related events.  The only thing you will see is that it’s still crawling.

How can you verify that security only crawls are running? 

I found two methods to assists in determining if security only crawls are occurring. 

 

Method 1:

Whenever a document is added/removed or User is added/removed from a group, this change gets logged in the change log “eventcache table” within that associated content database.  Please see my original blog in determining which content database you should be checking the change log post incremental crawl period.  The key is comparing the change log between short/long incremental crawls to see what’s different between the two. 

The eventtypes column within Eventcache table contains the type of changes that the crawler will enumerate through during incremental crawl time. 
This is one way to determine if security only crawls are contributing to your long crawl times.  You will need to query the eventtype column for all events that were part of your long running incremental crawl. 

You would be looking for event types 4194304 and 2097152.
 
2097152 - “A user is added to a Sharepoint Group”
4194304 - “A user is deleted from a Sharepoint Group”

Steps are the following:

A.) Dump the MSSCrawlHistory from Search database table and look for the latest Crawl's with Status 11 (One long running and one that ran quickly)

B.)  The screen shot shows me Crawl ID 8162 and 8165 "I'll use these crawl ID's in my example"

C.) Next, open the MSSChangeLogCookies table within the Search DB and you'll see something like the following for the corresponding crawl ID's gathered from reviewing the MSSCrawlHistory table:

ChangeLogCookie_old will contain:

1;0;888eef75-d584-4edf-b242-f5161d4c3c44;633686171531600000;2564

ChangeLogCookie_new will contain:

1;0;888eef75-d584-4edf-b242-f5161d4c3c44;633686187284500000;2577

It’s important to understand the bold values above.  The long Guid “888eef75-d584-4edf-b242-f5161d4c3c44” is the actual database were crawling against. This is important because this is the content database where you’ll need to query the eventcache table for the batch of changes processed during this crawl.  To find the actual database value you can run something like the following against Config DB:

select * from objects with (NOLOCK) where ID = '888eef75-d584-4edf-b242-f5161d4c3c44'

The last 4 digits 2564 and 2577 are the changes from the eventcache table that the crawler processed.   

D.)    Next, You want to run a query against the eventcache table and output to a file:

Select * from eventcache with (NOLOCK) where ID BETWEEN 2564 AND 2577

E.) Perform the above steps for both short/long running incremental crawls and compare the differences from the eventcache output via Step D.  

Conclusion from Method 1:  If you see eventtypes 4194304 and 2097152 only during your long running incremental crawls, it’s a fairly safe bet to say that the security only crawls are the issue.  I would combine both methods to confirm that 99 %. 


 

Method 2:

When the crawl log stops reporting events and appears hung again you should open the ULS logs on the indexer and see a ton of events like the following:

04:21.0  mssearch.exe (0x092C)     0x09A8 Search Server Common    GathererSql   0    Medium      CGatherer::CommitTransaction succeeded, URL sts3://site/siteurl=/siteid={ee7f629c-a36c-4327-b5f1-0652878fa4a3}/weburl=docs/webid={ba2b56ba-27d1-4cfb-90b4-e638d3f3b0ed}/listid={4d587236-3886-440f-a0d7-0d33622724cd}/folderurl=/itemid=27, CrawlID 51, SourceDocID 229 - File:d:\office\source\search\search\gather\server\gatherobj.cxx Line:9449

This event tells us that the gatherer is busy committing transactions to the index.  You’ll notice some subtle differences between the events.  For Example: Itemid and SourceDocID will be different for every event.

The goal is finding out the following about the event: 

1.) What exactly is being committed to the index?
2.) Was this an item that was actually added/changed?    

To figure out which item is being hit you would run the following SQL queries based on the SourceDocID, List ID, and ItemID.  I use the above event as an example:

Query 1 against SearchDB:

select * from MSSCrawlURL where docID = '229'

"Look at the DisplayURL and see what URL it's hitting as this usually includes a file name"

Query 2 against ContentDB:

select * from AllDocs where listid = '4d587236-3886-440f-a0d7-0d33622724cd' and doclibrowid = '27'

“Look at the LeafName and confirm this matches the same item as Query 1”

Both of these queries should result in the same file.  I would run the above two queries on multiple events. Maybe grab like 10/20 items and then hit the Crawl logs after the crawl completed and see if these items were crawled. If not, then it’s very safe to say that security only crawls were taking place in addition to the method I provided earlier.


Question:

Why do security only crawls take so long?

Answer:  

The time difference in crawl can be attributed to expansion of the SharePoint Group and also that the group is at the site collection level and affects items beyond the list.  If a Sharepoint group has several thousand users at site collection level, you can see how this can be very expensive.  Also, a large number of items within that site collection can add to the delay because new ACL changes will be pushed down to every item affected by the security change.   

Question: 

How can I work around this and prevent security only crawls from affecting incremental crawl times?

Answer:  

Instead of users explicitly added to Sharepoint groups, add AD groups instead.   Managing adding/removing users from Active Directory security groups will not cause ACL changes within Sharepoint.  Because of this, no security only crawls will occur.

 

Engaging Microsoft with Sharepoint performance issues

Sharepoint Performance Blog Series Part 1

Where do I start with this performance issue?  HELP!

I’m going to write up several blogs in a Sharepoint Performance series to provide our customers with some insight on how to approach and troubleshoot these challenging issues.  This first one will pertain to what data Microsoft is looking for when you call in with a Sharepoint performance problem.
Performance issues can be extremely difficult, time consuming, frustrating, and expensive to troubleshoot and resolve.  Many times it involves an entire IT support team as well as several Microsoft engineers to determine two things. 


1.)    Attempt to identify source of performance pain point

2.)    Provide relief as quickly as possible 


Section 1:

What is the specific issue?

This is the first question that a Microsoft Support Engineer should ask.  The following are bad examples of customers reporting a performance problem:

A.)    My Sharepoint servers are extremely slow

B.)     My clients can’t connect to Sharepoint

You can see that these responses open up a lot of questions which can be time consuming to narrow down a proper scope of the specific issue.  

 

Here are two examples of customers calling into properly report a problem with a performance issue:

A.)   Pages take in excess of two minutes to render for all users accessing Sharepoint site collections\sites under Web Application named “Sharepoint Content-WebApp”.   

 B.)    Clients are unable to connect to Sharepoint sites during the hours of 2:00 PM – 3:00 PM.  Clients can connect to all Sharepoint sites outside of this time frame.  All web front ends appear to be effected. 

 

Section 2:

What data can I collect prior to calling into Microsoft for a Sharepoint performance problem?

I’ve had the above question asked by several customers.   The reason is because several hours are taken understanding issue, collecting data, and uploading data before Microsoft can even start looking into the problem.  By gathering the proper data prior to calling into Microsoft, several hours might be saved.  It’s important to understand that the type of performance issues encountered may result in Microsoft requesting additional data. 

 

Data to collect in no specific order:

1.)    SPS Report run on every WFE

 Download: http://www.codeplex.com/spsreport/Release/ProjectReleases.aspx?ReleaseId=5706

 

2.)     Collect a performance log during the time of the problem on a minimum of one Web Front-end.  A performance capture before, during, and after the problem is preferred.  I mention a minimum of one Web Front-end because it’s easier to grab data off of on server over many to start.  The question is which Web Front-end to I collect data from?  For example, Sharepoint pages are loading slowly on the client so a host file has been configured to go directly to a specific Web Front-End named WFE1 bypassing an NLB.  Pages are still loading slowly so you know that WFE1 is a good starting place to start collecting Performance logs.

Important!  Collect a baseline performance log now during healthy server performance for at least a 48 hour span of your highest user load on each Web Front-end.

Note:   Please see part 2 in this series (Coming Soon) for instructions on how to configure Performance Monitor log on Sharepoint servers.

 

3.)    Collect ULS logs off of every Web-Front end during the time of the problem.  The assumption is that the ULS logs are in a non-default directory.  If your ULS logs are in the default directory, you can ignore Step 3 because SPS Reports will grab them. 

You can find the directory path by performing the following steps:

A.)   Launch Central Administrator

B.)    Select Operations Tab

C.)    Select Diagnostic Logging

D.)   Trace log location will be near the bottom of the screen

Note:   We recommend the default logging as is.  It’s recommended not to enable verbose logging unless directed by Microsoft.

 

4.)    Synchronous unfiltered network traces of the problem. 

I prefer that customers use Netmon 3.2 to capture although it's not required.  So in the example before, pages are rendering extremely slow for a client.  To gather synchronous traces, we’ll need a network trace on a client and a specific Web Front-end at the same time during a client attempting to launch and render the page.  Again, one way to determine which Web Front-end a client is hitting is by configuring a host file on the client.  This assumes that an NLB exists which is usually the case.

 

5.)    Collect IISAPP before, during, and after performance.

IISAPP is a command line utility which will dump out the application pools and corresponding process ID’s (PID’s).   It’s extremely important to gather this because it’s one way were able to map the Application Pool to the PID within the Performance Log.   Also, by gathering more than one will tell us if the PID’s are changing which is an indication of w3wp process being recycled due to manual intervention or automated via application pool recycling.

 

Steps to run IISAPP:

a.)    Click Start, Run and type CMD
b.)    Type IISAPP

If prompted  -  proceed to step c. or  No prompt  -  proceed to step f

c.)    You might receive a popup which says

“This script does not work with Wscript” Click OK


d.)    You’ll should get a second prompt to register cscript which says:

“Would you like to register cscript as your default host for VBScript?”

e.)    You’ll get a successfully registered Cscript prompt  (Click OK)
f.)     Run IISAPP again and you should see the output to verify it works
g.)    To collect output for Microsoft, run the command and output to file.

(For example:  C:\>iisapp > c:\sharepointdata\iisappoutput.txt)

 

Additional Request:

Report the exact times you had or have the problem and provide some details around what led you to capture the above data set. 

I like the data organized so before calling in.  For example:  The performance logs compressed in one zip file and ULS logs zipped in another for example.  

How to determine the number of changes an incremental crawl will process prior to initiating the crawl

I've had customers ask this question several times because they would like a good understanding of how long the incremental crawl is going to take.  The # of changes an incremental crawl must process has a lot to do with why one incremental crawl takes 10 minutes and another crawl takes several hours.  This isn't the only reason why an incremental crawl may take longer than expected but gives you some insight beforehand how many changes you are dealing with. 

Understanding the Basics:

The incremental process is dependent on the protocol handler being used.   This blog is solely focused on detecting changes against Sharepoint sites.  When detecting changes against Sharepoint sites, we use the "Sharepoint 3.0 (sts3) protocol handler".  

We will first attempt to get changes from the last crawl.  We do this through MSDMN.exe process and hit a webservice called sitedatawebservice.  The URL is:

http://servername/_vti_bin/sitedata.asmx

In my case the url is http://russmaxwfe/_vti_bin/sitedata.asmx

For incremental calls we will use the GetChanges method.  It uses a soap protocol and will gather last change ID and which we received during last crawl which will enable the webservice to return us a list of all new changes. 

 

How to detect changes before starting incremental crawl:

This can all be accomplished by a series of SQL queries.  I want to remind readers that performing updates/edits to the database directly are 100% not supported.  The first table you need to check is the MSSChangeLogCookies table within the Search database.  This table keeps track of the last change that the crawler processed for each content database.  You'll want to look at the ChangeLogCookie_new column and you'll see several rows but the output of each will look something like this:

 

1;0;888eef75-d584-4edf-b242-f5161d4c3c44;633579660402500000;2386

 

The GUID, 888eef75-d584-4edf-b242-f5161d4c3c44, is the actual database were crawling against.  The last value, 2386, is the latest change ID.   So first, we need to find which content database this row is referencing.  To do this, we take the Guid, 888eef75-d584-4edf-b242-f5161d4c3c44, and perform the following query against the objects table of the configuration database:

 

select * from objects with (NOLOCK) where ID = '888eef75-d584-4edf-b242-f5161d4c3c44'

 

This will output the name of the content DB.  So for my case, it's MOSS_ContentDB.  So at this point, we know that last change that was processed against the MOSS_ContentDB is 2386. 

Now we need to determine all of the changes from 2386 to latest from the MOSS_ContentDB.  The eventcache table within the content database contains all of the changes up to the most recent.   So in our example above, we need to know all of the changes greater than 2386 so we perform the following query:

 

 select * from eventcache with (NOLOCK) where ID > '2386'

 

The ID column will show you all changes after 2386.  The last row will be the latest change so in my case it's 2396.  So before starting the incremental crawl, I know that the crawler will process 10 changes against this content database.   After running an incremental crawl if I check the MSSChangeLogCookies table in the Search DB, I'll see the following:

ChangeLogCookie_old column will contain:

1;0;888eef75-d584-4edf-b242-f5161d4c3c44;633579660402500000;2386 

ChangeLogCookie_New column will now contain:

1;0;888eef75-d584-4edf-b242-f5161d4c3c44;633579795185130000;2396

And the process repeats itself...

Intro

Greetings!

My name is Russ Maxwell and I'm a Support Escalation Engineer at Microsoft supporting SharePoint.  I officially started in SharePoint sometime in November 2007.  I've been with Microsoft going on four years now and will continue immersing myself in this product and blogging on random SharePoint topics that I feel will benefit anyone outside MSFT. 

My posts will be varied in technical depth and totally random.  My reasoning for doing this is to help any reader whether they are new to the product or have extensive experience...

-Russ

 

Posted by Russmax | 0 Comments
 
Page view tracker