RCormier's SharePoint Blog

How To: Perform Bulk Uploads of Files to SharePoint

2013-05-01T15:16:00Z

This post is going to discuss how to perform a bulk file copy from a folder on the file system to a SharePoint document library. Along with this blog post, I will also provide a downloadable script, which can be modified to work in your environment. As we are copying files from the file system to SharePoint, required metadata may need to be populated via default values or via modification to the script. Additional work could be performed in order to import the metadata from an xml file. This functionality is currently not available in the version of the script provided. This version of the script simply takes exactly what is on the file system and duplicates it in a SharePoint document library.

Background

I was recently working with a customer who needed to copy files from one SharePoint environment to another SharePoint environment. Although SharePoint does have this functionality built-in via Export-SPWeb and Import-SPWeb, this was not successful in the referenced environment. This was later determined to be related to corruption in the environment.

Approach

In this case, the approach was simple enough. Read the folder and file structure on the file system and replicate that structure in SharePoint.

Solution

The solution in this case is a little simpler than what was needed to originally export files from SharePoint to the file system, but the same basic principles apply. What's important in this case is to ensure that we create the folder structure before we start putting files in those folders. In order to do this, we need to be able to separate files from folders. We can use Get-ChildItem in order to accomplish this. Since we are only looking for folders, we can apply a filter.

Once we have a recursive list of all folders, we can loop through this list, creating each folder, and uploading the files in each folder as we go along. Retrieving a list of all files is also achieved by using Get-ChildItem using a slightly different set of filters.

Creating folders is done by using the $SPList.Folders.Add method. When we call this method, we'll have to pass the URL of the folder, the type of object to create, and the name of the folder. We also need to ensure that the folder is published, which can be performed by setting the "_ModerationStatus" property to "0". This is illustrated in the screenshot below.

Uploading Files to SharePoint is also relatively simple using PowerShell. We can use the $SPFolder.folder.files.add method for this. When we call this method, we need to pass a filename, the file stream, and a Boolean of whether or not to overwrite. We then need to perform certain tasks such as checking in, publishing and approving the file. This is illustrated in the screenshot below.

Once completed, the source directory and the destination library should look identical.

Download the Script

The script can be downloaded from the following location:
BulkFileImport.ps1 (compressed)

Usage

This script does require some edits. All edits are explained at the top of the script. If you are using multiple source directories and/or multiple destination directories these can be chained together in a single script by inserting multiple iterations of the sample script block. Each source folder and destination library will need to be added to the script using the three lines of PowerShell which are included. Here is a brief explanation.

This line indicates which subfolder within the directory should be used for source files. Do note that this will be appended to the $Directory parameter set higher up in the script:
$SourceFolder = "PowerShell Scripts"

This line indicates which library within the SharePoint site collection the files should be imported to. This parameter should be the title of an existing list in the SharePoint environment:
$DestinationLibrary = $Site.RootWeb.Lists | ? {$_.title -eq "Shared Documents"}

This line calls the ImportFiles function and uses the source folder and the destination library as parameters:
ImportFiles ($Directory + "\" + $SourceFolder) $DestinationLibrary

Feedback

As always, feedback and suggestions are always welcome. If you do have any ideas on how to improve the script, I'd love to hear them.

You can also follow me on Twitter:

RCormier_MSFT

How to: Perform Bulk Downloads of Files in SharePoint

2013-03-31T01:26:00Z

This post is going to discuss how to perform a bulk file copy from a SharePoint library to a folder on the file system. Along with this blog post, I will also provide a downloadable script, which can be modified to work in your environment. As we are copying files from SharePoint to the file system, all metadata will be stripped. Additional work could be performed in order to export the metadata to an xml file. This functionality is currently not available in the version of the script provided.

Background

I was recently working with a customer who needed to export files from one SharePoint environment and import them into another SharePoint environment. Though SharePoint does have this functionality built-in using Export-SPWeb and Import-SPWeb, in this case an export resulted in no files being exported from the desired directory. This was later determined to be corruption of the files after the export was unsuccessful on the desired directories.

Approach

In this case, the approach was simple enough. Read the folder and file structure from the specified list and replicate that structure on the file system.

Solution

This solution is tricky enough because of the way files are accessed in SharePoint lists. Enumerating all files in the list can be performed, however folder structure would not be maintained by calling $List.Items. In this case, what we have to do is perform two separate actions. One action to retrieve the root folder of the list and enumerate the items, and one action to enumerate each folder and each file in each folder. In order to do this, we call $List.rootfolder.files

We also need to enumerate all folder, and enumerate all of the files in each folder. This can be done by using $List.folders. This will list all folders (including subfolders) for the list.

$Folder.folder.files allows you to enumerate all files in a specific folder

Now that we have a collection of all of the files and their directories, we can export the files to the file system. This can be performed by accessing the binary steam of the SharePoint file, and assigning it to the binary stream of a file system object. This is performed by using the following snippet of code.

Once the script has been exported, the structure of the files and folders will be maintained on the file system as they were in the SharePoint site.

Download the Script

The script can be downloaded from the following location:
BulkFileExport.ps1 (compressed)

Usage

This script does require some edits. All edits are explained at the top of the script. Each child directory must be specified in the script. Each library which is to be exported will need to be added to the script using the three lines of PowerShell which are included. Here is a brief explanation.

This next line will create a directory at the specified location. In this example, a directory will be made beneath the root directory and will be named StyleLibrary
New-Item -Path ($Directory + "\StyleLibrary") -ItemType Directory

This next line will retrieve a list with the specified title, and assign it to a variable. The variable name is not important, as long as the correct variable is passed to the ExportFiles Function in the next line
$StyleLibrary = $Site.RootWeb.Lists | ? {$_.title -eq "Style Library"}

This next line will export files from the list specified via the first variable and to the subdirectory specified in the second variable.
ExportFiles $StyleLibrary "StyleLibrary"

Feedback

As always, feedback and suggestions are always welcome. If you do have any ideas on how to improve the script, I'd love to hear them.

You can also follow me on Twitter:

RCormier_MSFT

How to Copy SharePoint Documents Between Site Collections Using PowerShell

2012-11-16T14:29:00Z

This post is going to discuss copying SharePoint list items between site collections using PowerShell. Similar to other "How to with PowerShell" blog posts that I have written, I'm also going to provide you with a downloadable script. The sample script that I provide will effectively copy all documents from a specified source library to another specified destination library. The net result will be that all files from the source location will be copied to the destination location, with metadata preserved and intact. The exceptions in this case for metadata being created by, created date and time, modified by, and modified date and time. Version history is also not preserved. It may be possible to preserve some of this metadata as well, so please feel free to leave your comments and suggestions below or to any of the sample scripts I provide and create your own.

Background

I was working with a customer recently who needed to move a subset of their data from one site collection to another in order to provide representative data to a group of developers. The data provided had to be portable, but also had to be dynamically collected using values provided by certain business units. What we had decided on in the end was a site collection which contained the necessary data. This site collection would be stored in its own content database, and that content database could be packaged with the images used to create developer environments.

Approach

The approach that we had decided on was simple in theory. Determine which items we wished to include in the new site collection, and duplicate the data as needed. If we hadn't had complicated filters, we could use content deployment to get the job done. If we weren't moving the data into a remote site collection, we could use the MoveTo method that each file has, etc. But for our specific requirements, things would have been a lot easier.

Solution

As I mentioned, the first thing we had to do was determine which files we wanted to include in our representative data. This is pretty straightforward. First get the web that contains the list, then get the list, then get the items you want. In this example, we use Get-SPWeb to get the web, then we use the SPWeb.Lists property to return the lists we want, and then we can use the SPList.Items property to retrieve the items we want. Of course, each of these can accept filters via a piped statement, such as:
$MyList = $MyWeb.Lists | ? {$_.title –eq "Shared Documents"}

In this example, I'm pulling all items from a library with only two items

Here is a screenshot of the original list in the browser

Now that I have a collection of files, I need to loop through my files and send them to my desired destination. This process is a little more complicated than I had envisioned it to be originally. I'm going to summarize in this blog post, however you can refer to the comments in my script for more detail as to what steps and decisions are being made along the way.

One of the first things that I do in my loop is pull out the binary stream from the file, using the SPFile.OpenBinary method, and assign that to a variable. I'll be using this later to populate the contents of the destination file. Next I'll be creating a new file by calling the SPFileCollection.Add method in the destination library. After the file is created, I then retrieve a list of all SPListItem.Fields objects that are not read-only and compare those to SPFile.Properties of the source file and the destination file. For each property, if the property does not exist for the destination file, I create the field using SPFile.AddProperty. Finally I set the value of the property using SPFile.Properties on the destination file, passing the same value that exists on the source document.

Again, there are a lot of little things happening here, and instead of posting a screenshot for each step, it's probably easier to refer to the comments in the example script I provide.

The result should be that your destination library will have exactly the same files in exactly the same folder structure as source library, as shown in the following screenshot:

Download The Script

This script can be downloaded from the following location:
Download CopyFilesAndFolders.ps1 (zipped)

Usage

The script requires four parameters to be set. These correspond to the source web, the source library title, the destination web, and the destination library title. Edit these four parameters to suit your environment and then execute the script.

Feedback

As always, feedback and suggestions are always welcome. If you do have any ideas on how to improve the script, I'd love to hear them.

You can also follow me on Twitter:

RCormier_MSFT

Search Returns An Error: Object reference not set to an instance of an object

2012-11-14T01:09:00Z

This post is going to talk about how to resolve the 'Object Reference not set to an instance of an object' message you may run into when you perform a search from a SharePoint site. This is something you may run into most often when setting up a small lab environment, and I'll explain why below, as I go through resolving the issue.

Background

One of the greatest things about SharePoint, is that even after years of using the product, it will continue to find new ways to break that you've never seen before. And this is what happened to me. I had just built a new SharePoint 2010 farm in order to test some scripts that I had been working on. One of my customers was setting up PDF search in their SharePoint 2010 environment and was having an issue getting PDF documents to return in their search results. My first instinct was to send over this document:

How to install and configure Adobe PDF iFilter 9 for SharePoint 2010

In response they indicated that they had been following this documentation and that it had in fact not worked. This documentation has always been solid for me, and has always done the trick – so I had to do some testing to make sure nothing changed, and that my guidance was still valid.

Issue

I had just created my new Search Service Application a few days prior, and I hadn't even performed a full crawl of the index yet. I followed the documentation, made my PDF icon appear and everything, uploaded some sample PDF documents and then kicked off a full crawl of my SharePoint sites content source. Everything looked great, and I had over 300 items in my index. Everything looked great until I did a search. Once I performed a search, I was greeted with an "Object reference not set to an instance of an object" message, and a correlation ID. Using Merge-SPLogFile, I passed in the correlation ID and got the following output (cleaned it up a little so that it's somewhat readable):

Videntityinfo::isFreshToken reported failure.

CoreResultsWebPart::OnInit: Exception initializing: System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.Office.Server.Search.WebControls.CoreResultsWebPart.SetPropertiesOnQueryReader() at Microsoft.Office.Server.Search.WebControls.CoreResultsWebPart.OnInit(EventArgs e)

Internal server error exception: System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.Office.Server.Search.WebControls.CoreResultsWebPart.SetPropertiesOnQueryReader() at Microsoft.Office.Server.Search.WebControls.CoreResultsWebPart.OnInit(EventArgs e) System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.Office.Server.Search.WebControls.CoreResultsWebPart.SetPropertiesOnQueryReader() at Microsoft.Office.Server.Search.WebControls.CoreResultsWebPart.OnInit(EventArgs e)

Watson bucket parameters: SharePoint Server 2010, ULSException14, 06175311 "sharepoint server search", 0e00178d "14.0.6029.0", 17853a8f "microsoft.office.server.search", 0e0017ee "14.0.6126.0", 5021bda3 "tue aug 07 18:15:15 2012", 000032a1 "000032a1", 0000002d "0000002d", 4a6d3421 "nullreferenceexception", 67316a39 "g1j9" ff503ece-c286-454b-a5f2-998ca970808f

What does this mean?

So here's what I understood from this: My Core Results Web Part needed access to something, and it wasn't available or did not exist. I just had no idea what could have been missing. I've set SharePoint search up hundreds of times. I walked through my configuration, made sure that all of the services were started, and made sure that everything looked like it was properly configured.

That Eureka moment

Looking at my configuration, and then mentally walking through the process, it had sort of donned on me. I had two servers in my farm. One for all web functions, and one for all application functions. My Search query and site setting service, and my SharePoint Enterprise Search Service Application Proxy both existed on the application server AND I was using my app server as my browser. I had forgotten to follow this document on my application server:

You receive error 401.1 when you browse a Web site that uses Integrated Authentication and is hosted on IIS 5.1 or a later version

For anyone who is not familiar with this document – this is the famous 'DisableLoopbackCheck' document. Whenever my query web part tried to access the proxy (hosted on the same server the request was coming from), it was being denied access and couldn't determine where the query component was even hosted.

The solution

After figuring this much out, all I had to do was run through the documentation create the DisableLoopbackCheck registry key on my application server. Once I created this key and rebooted the server, the problem went away. This allowed me to validate that the documentation for configuring PDF indexing in SharePoint 2010 was still valid and worked perfectly.

Feedback

As always, any feedback is always welcome.

You can also follow me on Twitter:

RCormier_MSFT

Creating Claims-Based Web Applications in SharePoint 2013

2012-10-30T17:29:00Z

This post is going to talk about how to create claims-based web applications in SharePoint 2013. This is actually something else that I ran into when I was configuring my own lab environment, and everybody will run into this at one point or another in the lifecycle of their SharePoint 2013 environment. The reason for that is that Windows classic mode authentication is deprecated in SharePoint 2013. And this is well documented in this article:

What's new in authentication for SharePoint 2013

Background

As I said, I ran into this issue when I was configuring my own lab environment. I was using some existing scripts I had for SharePoint 2010, since most of them work, in order to create several web applications. If you haven't seen the documentation on TechNet, don't worry. SharePoint is pretty good about telling you that classic mode authentication has been deprecated in SharePoint 2013. This is a screenshot of what you will see if you configure a Windows classic mode authentication web application:

This is the text contained in the informational message, in case you're encounter the issue:

WARNING: The Windows Classic authentication method is deprecated in this release and the default behavior of this cmdlet, which creates Windows Classic based web application, is obsolete. It is recommended to use Claims authentication methods. You can create a web application that uses Claims authentication method by specifying the AuthenticationProvider parameter set in this cmdlet. Refer to the http://go.microsoft.com/fwlink/?LinkId=234549 site for more information. Please note that the default behavior of this cmdlet is expected to change in the future release to create a Claims authentication based web application instead of a Windows Classic based web application.

The good news is, we won't have to think about this forever – and it will end up being automatic eventually.

Issue

I ran into a few issues with this. The first issue I ran into with this was that using my SharePoint 2010 scripts and simply passing the AuthenticationProvider parameter my web applications were not consistently being deployed using windows claims authentication. The first web application would get created with windows claims authentication, every subsequent web application would ignore the AuthenticationProvider parameter and the web application would be created using windows classic authentication.

When this occurred, this would set off a chain of failures in my scripts, particularly when creating web application policies and assigning object cache accounts. The error message that you when this happens is also pretty helpful, and shouldn't set you back too far. Here's a screenshot:

This is the text contained in the error message, in case you encounter the issue:

The specified principal does not correspond to the authentication mode of this collection

Resolution

I never did find out why I was having inconsistent AuthenticationProvider behavior. It was however resolved by rewriting my PowerShell script, and breaking everything out into functions created using claims authentication. I've made the script that I'm using available in the TechNet Gallery. The script itself does have a significant amount of conflict and error detection, and should make deploying claims authentication web applications in SharePoint Server 2013 pretty simple.

Download

The script can be downloaded from the following location:
Download CreateSP2013ClaimsWebApplication.ps1 (zipped)

Feedback

As always, if you have any questions or feedback, let me know. If you have any ideas to optimize the script, I'd like to hear that too. Thanks for reading!

You can also follow me on Twitter:

RCormier_MSFT

You Shall Configure your MAXDOP When Using SharePoint 2013

2012-10-25T06:40:00Z

In this post, I'm going to talk about why and how to configure your Maximum Degree of Parallelism on your SQL database instance. The reason I'm explaining this now is because of something I ran into while setting up my first SharePoint Server 2013 RTM farm tonight.

Background

Microsoft has been telling customers for years to set their Maximum Degree of Parallelism (MAXDOP) to 1 for SQL instances which host SharePoint database. For the most part, this guidance has been widely ignored with very little consequence, with the exception perhaps of sub-optimal performance.

For those of you who are unfamiliar with what this setting actually does, you can refer to the following documents:
max degree of parallelism Option
Recommendations and Guidelines for 'max degree of parallelism' configuration option

The second link actually does a pretty good job of explaining in detail what this option does – and here's the short summary: The SQL Server configuration option 'max degree of parallelism' controls the number of processors used for the execution of a query with a parallel plan.

A higher MAXDOP value would typically be used when you have a very small number of concurrently executing queries relative to the number of processors. A lower MAXDOP value would typically be used when you have a very large number of concurrently executing queries relative to the number of processors. The default value of '0' will effectively limit the MAXDOP value to the number of processors.

SharePoint is a system in which there will be a very high number of concurrently executing queries; therefore this system favors a lower MAXDOP value. As indicated in the following documents, the optimal MAXDOP value for SharePoint environments is 1.
Storage and SQL Server capacity planning and configuration (SharePoint Server 2010)
Enterprise intranet collaboration environment technical case study (SharePoint Server 2010)

Historically, Microsoft has been making this recommendation, and adding query hints in order to limit MAXDOP for each query to 1 in order to limit the impact of setting MAXDOP to any value other than 1, or leaving it at 0. This does not appear to be necessary going forward, and I'll do my best to explain why.

What Happens If I Haven't Set MAXDOP to 1?

The answer here is pretty simple. If your MAXDOP value has not been set to 1 on the SQL server instance on which you're attempting to host your databases, you will not be able to create databases on the instance. The error message you will encounter is the following:

New-SPConfigurationDatabase : This SQL Server instance does not have the required "max degree of parallelism" setting of 1. Database provisioning operations will continue to fail if "max degree of parallelism" is not set 1 or the current account does not have permissions to change the setting. See documentation for details on manually changing the setting

Also seen in this screenshot:

How Do I Work Around This Problem?

There are basically two ways to remediate this issue. The first one (not recommended) is to grant the account running the PowerShell cmdlets, or executing the Configuration Wizard, sysadmin access to your SQL instance. The second, more desirable approach, is to set the MAXDOP value to 1 in the SQL instance.

The MAXDOP option is an advanced option. The documentation above does provide the SQL statements to change this option. I'll briefly explain how to do this using SQL Management Studio. My screenshot is from SQL Server 2012, the process will be the same for SQL Server 2008 R2, and SharePoint 2013 does not support older versions of SQL:

Open SQL Management Studio and connect to the instance which will host SharePoint databases
Right-click the Instance you wish to manage
Select 'Properties'
Select the 'Advanced' Node
The Second option from the bottom will be 'Maximum Degree of Parallelism'. Change this value to 1
Click 'OK'

Your value should appear as in the screenshot below:

After having set the MAXDOP value to 1, you should be able to proceed and create your farm and/or additional databases, all the while following at least one best practice.

What Does This Mean In The Enterprise?

This should mean that enterprises will start to be a little more mindful of what kind of applications, if any, are going to share database instances with SharePoint. People in the enterprise need to consider what type of query patterns are going to be used in their applications, and whether or not the performance of these applications will be negatively impacted by having a MAXDOP value of 1. What I anticipate will happen is that we're going to see more SQL instances that are dedicated to hosting SharePoint databases, possibly for multiple farms, and overall I see this as a good move.

Feedback

As always, if you have any questions or feedback, let me know. I'd be interested in finding out what your thoughts are regarding this change in SharePoint 2013

You can also follow me on Twitter:

RCormier_MSFT

How to Determine Where a User Has Been Granted Access in SharePoint 2010

2012-10-23T14:46:00Z

This post is going to discuss how to determine where a user has been granted access to your SharePoint environment. The script I provide is scoped to an individual web application and searches for all users based on their domain. I'll explain why I did that below. Both of these elements could easily be modified to suit your purposes – such as searching for a single user account across all web applications in the farm.

Background

I was recently working with a customer who had discovered a configuration issue in their farm. The environment has two domains. In their particular case, a two-way trust was required and had been enabled. Only users from one specific domain were intended to have access to the SharePoint environment. Two elements of oversight resulted in users of the other domain having access to the site. The first oversight is that the people picker had not been scoped to only one domain. This can be configured using the following article: Peoplepicker-searchadforests: Stsadm property (Office SharePoint Server). I do realize that the documentation specifies SharePoint 2007, however it is also applicable for SharePoint 2010, as indicated in this article: People Picker overview (SharePoint Server 2010). The next issue is that people had also been granting access to 'NT Authority\Authenticated User', which includes users authenticated to both domains.

Approach

The approach I'm taking is a series of enumerations. Permissions are applied at the web, list, and object level, so we need to find a way to get all the way down there. First we need to enumerate all site collections, and all webs in those site collections, then all lists in those webs, finally we could enumerate all list items in those lists as well.

For the purposes of this sample script and this blog post, I've chosen to go only as far as the individual lists. The approach you would take to report on individual list items would be effectively the same, including one additional enumeration for all list items in each list.

Once we have a list of all these webs and lists, we need to find a way to find out who has been granted access to the item, and how they have been granted access to that item.

Solution

Now that we know the approach, the solution is pretty simple. The first thing that you have to do is enumerate through site collections. Getting a list of all site collections can be accomplished using Get-SPSite.

As I said, we're going to be enumerating through them and performing some additional actions. For that, I've chosen to create some ForEach-Object loops.

User security is applied not at the site level, but at the web level. Which means we actually need to get a list of all webs in all sites. This can be placed inside the first ForEach loop. What we'll be doing is using the SPSite object's AllWebs property.

If a web is not using unique security, then it will be inheriting from the parent. This means we don't need to check roles on that object. But first we need to find out how to do that for each web. PowerShell again makes this very simple, because every web has a HasUniqueRoleAssignments property. This property will tell us if the site uses unique permissions or not.

Once we've determined if the web has unique role assignments, we can start looking for what those role assignments are. There are two places to look for. Users who have been assigned permissions directly, and users who have been added to SharePoint groups. PowerShell again makes gives us a break. To discover each user who has been assigned permission directly, use the RoleAssignments property of the Web object.

In order to discover users who have been granted permission through groups, this is a little more difficult – but not much. From the RoleAssignments from the web, we have a list of users and a list of SharePoint groups. We can enumerate the members of all group objects by calling the users property of the members property of the RoleAssignments property for the web. This may be getting a little complicated, but I've included a screenshot below. In this screenshot, we can see that two accounts have been granted full control because they are members of the SharePoint Owners group.

This has gotten us as far as enumerating all webs where unique permissions have been applied. So how do we do this for lists? It's effectively exactly the same. Each Web object has a Lists property.

Another great thing about PowerShell in SharePoint, every single list object has a RoleAssignments property, just like webs.

In order to return users who have been explicitly granted access, or who have been granted access via a SharePoint group, you would use the same script as you did for the web object.

As I mentioned, the sample script I'm providing currently looks for users from a particular domain. It would take very little effort to transform this script and search for specific user accounts. If I get any requests for it, I may just add it as a secondary download from this blog post.

Download This Script

Download SecurityReport.ps1 (zipped)

Usage

There are three variables which need to be altered before running this script. Failing to alter them may result in the script not running, or producing useless data. One is the web application URL, one is the unwanted domain prefix, and one is the log file directory where the output will be generated.

Feedback

As always, if you have any questions or feedback, let me know. If you have any ideas to optimize the script, I'd like to hear that too. Thanks for reading!

You can also follow me on Twitter:

RCormier_MSFT

Domino URL commands break SharePoint Mobile View

2012-10-04T02:41:00Z

This post is going to discuss an issue I ran into where certain links in a links list can cause Mobile views to fail to render. This was actually encountered when using Domino URL Commands, specifically the ?OpenDocument URL command in my case. When you run into this situation, the error message you will receive is the following, which is not really helpful and is not really friendly. The message you will get is Value cannot be null. Parameter name: str as displayed in the screenshot below.

Background

I was recently working with a customer who had a number of lists to reference documents that were pretty critical to operations in their environment. Whenever users used their IPhones or Android devices, they were unable to access lists, or the content in the lists, the error message above was returned to the user. Upon further investigation, it was discovered that even using any of the supported browsers (desktop or mobile) to access the mobile view the issue would occur. For anybody who has never accessed a mobile view, all you have to do is append ?Mobile=1 to the end of a page URL. It has to be a page URL, one example of such would be http://contoso.com/pages/default.aspx?mobile=1.

Investigation

During our investigation, we had determined that any supported desktop browser could access the desktop version of the list view without any issue. Any supported browser would fail when attempting to view the mobile view of the list. Typical troubleshooting techniques (netmon, fiddler, ULS logs, etc.) did not turn up anything too interesting. We started by eliminating things like special characters from the site name, list name, folder names, etc. We were also able to rule out the possibility that it was due to the length of the URL.

Through further investigation, we found out that this issue only occurred in lists of links to operational documents. Upon further investigation, it was determined that each of these links included Domino URL commands. For anyone who is not familiar with, or who has not caught on to what Domino URL commands are, please check the following link:

Domino URL cheat sheet

One example of a Domino URL command, as noted above is the ?OpenDocument command. This instructs Domino to open the document.

Other investigations reveal that although Android devices do allow you to see the "desktop" version of the web site, this was not the case using our IPhone devices, which were running IOS 5. There are however a number of third party browsers available for download or purchase which allow you to view the desktop version of the page to work around the issue.

Reproducing the Issue

Reproducing the issue is actually pretty easy once you understand what's happening.

Step 1.
Take any links list in your environment, and add a URL that includes a URL command, such as the following:
http://MyServer.MyCompany.com/somepage.aspx?OpenDocument

Step 2.
Access the Mobile view of your list by appending ?Mobile=1 to your list view

Step 3.
Observe that your list view is now broken

Assumptions

Since it reproduces with supported desktop browsers, it's not a problem with the mobile client. It's a problem with the view itself. My assumption, has not been confirmed, is that the mobile view is trying to rewrite or re-factor the URL. When it gets to a URL with a URL command in it, it doesn't handle the URL well, and throws up an ugly error.

Resolution

Remove URL Commands from the links in the links list. In the case of ?OpenDocument, it really doesn't matter. You can remove that part of the URL and the link will continue to function just fine.

Feedback

As always, if you have any questions or feedback, let me know. If you have any other workarounds, I'd like to hear that too. Thanks for reading!

You can also follow me on Twitter:

RCormier_MSFT

How to Fix Navigational Links After Relocating a Site Collection

2012-09-27T06:01:00Z

This post is going to discuss how to fix navigational links after relocating a site collection within the same web application. Relocating a site collection to a new URL is a relatively common task. One reason you may relocate a site collection is if a certain area within your organization is suddenly renamed as part of an organizational change. Hopefully this blog post will help you overcome this type of situation with relative ease. Though it is possible to fix up the quick launch navigation as well, the sample script I'm providing only fixes top link navigation links.

Background

I was recently working with a customer that was going through an organizational change. Part of that organizational change required that the URL of a few site collections be changed. In order to accommodate the required URL change, the site collection was backed up and restored to a new URL. Scattered throughout the environment was hundreds of links that pointed to the previous URL for the site collection. Rather than manually browse the entire organizational structure to make sure that all navigational structures were updated to the new URL of the site collection, we decided to leverage PowerShell. This proved to be a significant reduction in total effort required.

Approach

The overall approach to this solution is pretty straightforward. What we want to do is discover any links that are used in the site navigation which contain a reference to the old URL, and update them with a reference to the new URL. Based on my particulars needs in my situation, I started at the web application level. I could have easily started at the farm level, or even at the site collection level in a relatively simple manner. Of course, the navigational elements are actually scoped at the web level, so we need to find a way to get all the way down to enumerating all of the webs, having provided only a web application. That doesn't sound too hard, right? So first we'll start with Get-SPSite –WebApplication $WebapplicationURL. This will return us with a list of all SPSites. From here we can basically loop through all of these site collections, enumerating all webs. This can be accomplished by using $SPSite.allwebs. Now we have a collection of webs. For each of these webs we'll have to evaluate the navigation nodes, which we will look at in the solution part of this post.

Solution

Now that we know what our approach is, let's take a look at how to put all of that together. The first step I've taken is to get a list of all site collections within the scope of the supplied web application. I gave away the answer up top, but here's what that looks like:

We all do this every day, so there's nothing new here. The next thing we have to do, is create several nested foreach loops. Since everything is nested, it's hard to show how all that works in a way that's easy to follow. So first I'm going to explain how foreach works, and then I'll explain the individual parts that are in each nested foreach loop. A foreach loop basically takes a collection of objects, and executes your PowerShell script block against each of the elements in the collection. The syntax you use is as follows:

Foreach($Object in $Collection)
{
PowerShell script block in here
}

The following screenshot illustrates that very concept:

Hopefully I haven't lost anybody yet. Getting back to where we were, we have a list of site collections, now we need a list of webs. How do we do that? We'll want to make a foreach loop for each site collection. We want to return all webs in all site collections. This is simple enough. Inside our foreach block, we'll do the following:

$TargetWebs = $TargetSite.allwebs

Now we have a collection of webs, however each web has its own set of navigational controls. This includes the global navigation (top links) and the current navigation (quick launch). This means we have one more foreach loop which will enumerate through all webs in $TargetWebs. In that loop, we'll want to get the navigation control. We can do this by getting the navigation property of the web. Returning the navigation object isn't that exciting.

And now we're getting to the moment we've all been waiting for, accessing the actual links! This we can do getting the TopNavigationBar or QuickLaunch properties of the TargetNav object. That should look like this:

The next thing we need to do is do a foreach loop to enumerate through all of the elements in the TopNavigationBar or QuickLaunch. In this loop, we're going to be converting the URL returned to a string object, and using split against it to split each string at each "/". We can then use another foreach loop. When we encounter a chunk of the string that contains the old URL, we replace it with the new URL, and then rebuild the string. Once we've completed this, we update the url property of the object with the new string. Since relative links start with a leading slash, this will cause our new link to start with a double forward slash. For that reason, we use substring(1), to start at the second position in the array – or simply remove the first leading slash.

The next step is just a simple update to the property using PowerShell.

This can also be verified in the UI

Download This Script

Download UpdateTopLinks.ps1 (zipped)

Usage

This script accepts four parameters. Once is the web application url, one is the old URL chunk, one is the new URL chunk, and the last is the logging directory. Simply update these four parameters and run the script.

Feedback

As always, if you have any questions or feedback, let me know. If you have any ideas to optimize the script, I'd like to hear that too. Thanks for reading!

You can also follow me on Twitter:

RCormier_MSFT

How to Update Inactive User Profile Information in SharePoint

2012-09-09T02:22:00Z

This post is going to discuss user profile synchronization, and how to update user profile information for users determined not to be active in SharePoint Server 2010. First we'll discuss what happens during the various phases of user profile synchronization, at a high level, and how SharePoint determines whether or not a user is active.

Background

I was recently working with a customer on one of their internal helpdesk issues. A user had recently had a life event, which resulted in their last name being changed. When logging into a particular site collection, their user information (welcome user name) continued to display their previous last name. All other site collections showed their current last name. The customer had taken typical troubleshooting steps prior to my engagement. This included performing a user profile import, ensuring that the user profile information in the user profile service application was up to date, running the stsadm –o sync operation. All to no avail.

Approach

To understand why this was happening, we had to take a look at what was "supposed" to happen. Synchronizing user profile information in SharePoint Server 2010 is really a multi-step process. I'm going to break it down into two large processes – each of those processes could be broken down further, and of course many existing blogs do just that. Much of the information can be found in Spencer Harbar's blog. He actually has an entire category dedicated to user profiles

The first major step involved with having fully up-to-date user profile data across all site collections is to ensure that your user profile service application contains the most up-to-date information. In order for this to happen, your user profile service application must have the necessary synchronization connections. Once synchronization connections have been properly configured, a profile import must occur. Assuming all is configured properly and is functioning properly, performing a full import will ensure that the information in your user profile service application is the latest and greatest. If you're not really sure where to start, this is a great starting point:
Configure profile synchronization (SharePoint Server 2010)

The second major step involved is synchronizing information from the user profile service application to SharePoint. There are two timer jobs responsible for this

User Profile to SharePoint Quick Synchronization – this job is responsible for synchronizing user information from the user profile service application to SharePoint users recently added to a site and occurs by default every 5 minutes.
User Profile to SharePoint Full Synchronization – this job is responsible for both synchronizing user information from the user profile service application to SharePoint users and synchronizing site memberships from SharePoint to the user profile service application. This runs every hour by default.

When a user first accesses a site, a copy of some user information is copied to the site collection. To be more specific, it's copied into the UserInfo table in the content database in which the site collection exists. This helps speed up certain aspects of working with SharePoint sites, such as accessing list views. Since these timer jobs are responsible for keeping that information up to date, we need to make sure these timer jobs continue to run.

The thing that's a little obscure to many people, myself included at the time, is that these timer jobs are only responsible for updating user information for "active users". So – what exactly is an active user? Basically, an active user is any user who has modified any data within the scope of a site collection. The information is a little dated – but to the best of my knowledge it remains unchanged up to and including SharePoint Server 2010. I haven't yet investigated how this all works in SharePoint Server 2013 Preview, so I can't comment on that yet. The documentation that I'm referring to is the following, and you won't have to read too far to identify what an active user is: [MS-WSSFO3]: Windows SharePoint Services (WSS): File Operations Database Communications Version 3 Protocol Specification. As I said, the protocol documentation appears to reference WSS 3.0, however this does not appear to have changed in SharePoint Server 2010.

Solution

So – Now that we know what's happening, what do we do to resolve this issue? I'll discuss what I did first, and then I'll discuss a method that would have saved me a couple hours of work (if I knew then what I know now). If you've been following my blog this year, you may have guessed – I decided to write a PowerShell script. The meat of the script is really not complicated.

First we're going to use Get-SPUser to return the user object we're targeting. That's pretty simple.

As you can see, the display name isn't what I want it to be. Luckily DisplayName is a property which can be set using PowerShell. I can manually set this property by calling the object and setting the DisplayName property, as shown below.

Now when that user, or any user logs into the targeted site, the display name will reflect the value I had set manually.

Of course, scripting that didn't take me "hours", creating functions, performing error handling, etc. is what took me hours.

Now let's get back to what could have saved me hours of work. Apparently the SharePoint product group was aware that people would still want to update user information for accounts which are not "active" in the site. It's included with stsadm –o sync, it's just not documented very thoroughly (yes, that is a SharePoint 2007 lync, stsadm is deprecated in SharePoint Server 2010. There is no PowerShell equivalent to the sync operation, according to Stsadm to Windows PowerShell mapping (SharePoint Server 2010)). You will however find documentation about this hidden gem in this article: Troubleshooting User Profile Sync issues in Microsoft Office SharePoint Server 2007.

To save you even more reading, and to make a long story short – this all could have been avoided by simply running the following command:
stsadm –o sync –ignoreisactive 1

Download the Script

Download UpdateUserDisplayNames.ps1 (zipped)

Usage

In the script, there is a constant. This would require you to put in your NetBIOS domain name. Asides from this constant, no further configuration is required. Simply make this edit, and run the script from a SharePoint server.

Feedback

As always, if you have any questions or feedback, let me know. If you have any ideas to optimize the script, I'd like to hear that too. Thanks for reading!

You can also follow me on Twitter:

RCormier_MSFT