March, 2010

  • Jie Li's GeekWorld

    FAQ: SharePoint 2010 Remote BLOB Storage (RBS)

    • 51 Comments

    Recently on SharePointPro Connections I got a lot of questions for SharePoint RBS. To help people to get clear on this topic here’s a FAQ… For installation and other documentations please refer to my pervious blog post on this topic: SharePoint 2010 Beta with FILESTREAM RBS Provider. Todd Klindt also has a very nice step by step guide: Installing Remote Blob Store (RBS) on SharePoint 2010.

    Q: What is RBS?

    From SQL RBS Team Blog:

    RBS(Remote Blob Store) is a set of standardized APIs that allow storage/retrieval of BLOBs outside of your main SQL database where a dedicated BLOB store is desirable for various reasons. This uses a provider model for plugging in any dedicated BLOB store that implements these RBS APIs.

    Q: Which version of SQL Server can I use for SharePoint RBS?

    SQL Server 2008 and SQL Server 2008 R2 both support RBS. A RBS Library needs to be downloaded and installed on SQL Server to enable the feature. All SQL editions (Express, Standard, Enterprise…) support RBS. Licensing requirement may involve depending on the scenarios.

    Q: What is FILESTREAM?

    FILESTREAM is a SQL Server 2008 feature to store BLOB content on to file system.

    From FILESTREAM Overview:

    FILESTREAM integrates the SQL Server Database Engine with an NTFS file system by storing varbinary(max) binary large object (BLOB) data as files on the file system. Transact-SQL statements can insert, update, query, search, and back up FILESTREAM data. Win32 file system interfaces provide streaming access to the data.

    FILESTREAM uses the NT system cache for caching file data. This helps reduce any effect that FILESTREAM data might have on Database Engine performance. The SQL Server buffer pool is not used; therefore, this memory is available for query processing.

    SQL FILESTREAM feature does not allow to store content on anything other than local storages. (SMB shares cannot be used for store BLOB content)

    Q: Is there a comparison of FILESTREAM and RBS?

    Please check SQL RBS team blog: SQL Server Remote BLOB Store and FILESTREAM feature comparison.

    Q: What is RBS FILESTREAM Provider?

    RBS FILESTREAM Provider is a free out-of-the-box provider shipped by Microsoft SQL RBS Team that allows a deployment to use a SQL Database (local or remote) as a dedicated BLOB store. This provider utilizes the FILESTREAM as the BLOB storage mechanism and ties the two technologies together.

    Q: Any benefit for using RBS with SharePoint?

    By using RBS for SharePoint, customer maybe able to leverage cheaper storage, improve performance, and enable better integration stories with 3rd party technology for their SharePoint databases. But be careful, the benefit is different case by case. You need to investigate your scenarios to see if RBS really fits you.

    Q: Any benefit to use RBS FILESTREAM Provider?

    Benefit for Windows SharePoint Services 3.0 with Windows Internal Database upgrade:

    We no longer support Windows Internal Database (WID) with SharePoint Foundation 2010. However, SQL Server Express has a 4GB DB size limit. The direct upgrade of a content DB > 4gb from WID to SQL Server Express is not supported. To get around of this limit, customer can use RBS FILESTREAM Provider to store all the BLOB content on to file system so the DB size will still remain small.

    Benefit for other scenarios:

    We are currently recommending to use this technology with document archiving and media streaming scenarios. Tests show that with RBS FILESTREAM Provider large BLOB objects can have better I/O compared with putting everything in the database.

    Q: Can RBS FILESTREAM Provider support SMB shares to store the content, for example a NAS device?

    No. SQL FILESTREAM feature does not allow to store content on anything other than local storages. Therefore RBS FILESTREAM Provider has the same limitation. 3rd party RBS Providers do not have this limitation if they are not leveraging SQL FILESTREAM feature.

    Q: How is the BLOB content stored on the file system with RBS FILESTREAM Provider? Is it encrypted? Does TDE helps with the encryption?

    The BLOB objects are stored with GUID names to make sure they can be linked from the RBS tables. The content is not encrypted. TDE does not apply to the content in file stream.

    Q: I heard there’s a remote feature in RBS FILESTREAM Provider, is that supported by SharePoint 2010?

    This feature allows SQL RBS FILESTREAM Provider to store BLOBs on to a remote SQL Server DB/Instance. SharePoint 2010 will support this remote feature of RBS FILESTREAM Provider.  However the test work will not be finished at the time of SharePoint 2010 RTM. We will announce the supportability of this remote feature later when the test is done.

    Q: Is there a licensing requirement for SQL Server when I want to use RBS for SharePoint?

    Yes, there is. If the BLOB objects are stored on local SQL Server file system then you can use any SQL Server SKU. For any other scenarios, local SQL Server needs to be Enterprise. The following table shows the license requirement for different scenarios.

    Scenario Local SQL SKU Remote SQL SKU
    FILESTREAM Provider,
    BLOB stored on local SQL Server file system
    Any SQL SKU N/A
    FILESTREAM Provider,
    BLOB stored on Remote SQL Server file system
    SQL Enterprise SQL Standard or above
    3rd Party RBS Provider SQL Enterprise N/A

     

    Q: Any 3rd party RBS provider I can use?

    Currently StoragePoint (acquired by Metalogix recently) and EMC are providing RBS Providers for different storage options. NetApps, Opentext, Commvault are also working on it.

    AvePoint also made their DocAve Extender free – a EBS provider for 2007, and a RBS provider for 2010. (thanks to Chris Musico for pointing out)

    STEALTH Software also offers RBS providers. (Thanks Mark)

    Q: Is EBS still supported?

    EBS(External BLOB Storage API) is introduced by SharePoint 2007. It is still supported in this release (2010). Moving forward, in order to have a more unified storage solution, we will only support RBS in the next release.

    Q: How should I backup the content if I’m using RBS?

    By using SharePoint Backup and Restore, it doesn’t really matter where the BLOB objects are stored – this is taken care by SQL RBS. So when you use SharePoint Backup on a RBS enabled content database, it will still backup everything. You can restore the backup to a non-RBS enabled database too, the BLOB objects will be inside the database – SQL RBS will decide where to put them, not SharePoint.

    If you are trying to backup the database by using SQL Backup, keep in mind this will not backup the BLOB objects. You need to manually backup the database first, then manually backup (XCOPY or other file backup tools) these BLOBs. When restore, using a reverse order to restore BLOBs first, then the databases. UPDATE: SQL Backup now can take care of FILESTREAM BLOB objects.

    Q: Can RBS help to remove the 2GB file size limit in SharePoint?

    No, using RBS with SharePoint will have the same file size limit.

    Q: Can I enable RBS on a content database with content existing? How to migrate those content into the BLOB store? (Thanks to Chris Givens)

    Yes, you can enable RBS on a content database and then migrate content into the external storage. There's a "Migrate()" method to handle this process. For example using Windows PowerShell, $cdb is the content db with RBS enabled, the following command will do the job:

    $cdb.Migrate()

    Q: Should I consider using RBS all the time?

    No. RBS may be used when :

    • You have huge content dbs for document archiving so you want to reduce storage cost (terabytes of data)
    • You have large media files to stream to the audience
    • You need to use RBS to integrate 3rd party storage/archiving solutions to SharePoint. (For example EMC Documentum)

    For example, if you only have 100GB of data, seperated in several content databases, and most of the content are documents, go for RBS will not benefit your server farm.

    Q: Are SQL Mirroring, Clustering and Log Shipping supported when using RBS FILESTREAM Provider?

    Log Shipping and Clustering are supported. SQL FILESTREAM cannot be used together with SQL Mirroring.

     

    If you have additional questions please leave them in the comment, I can add them into this list later.

    Jie.

  • Jie Li's GeekWorld

    SharePoint 2010 with Windows PowerShell Remoting Step by Step

    • 15 Comments

    With all the improvements in SharePoint 2010 for IT Professionals, I always put Windows PowerShell support as the number one. Maybe this has something to do with my past Linux/Unix background, but the main reason is, I’m a really really lazy person. If something can be put into automation, then why bother to click through it manually every time? Schedule it to run at certain time everyday can save me a lot of time. In the past SharePoint versions, STSADM is okay, but it’s limited and hard to play with. Although you can use Windows PowerShell to call object models directly, but that is too complex and indeed a developer stuff. Now, with SharePoint 2010 Windows PowerShell cmdlets, scripting can be really fun!

    But someone asked me this question:

    You are telling me scripting is great – but isn’t that just a server thing? I still need to open remote desktop on my laptop to connect to the server box and then do the shell stuff, can’t I have something like SSH?  Just run my script remotely without opening my browser, remote desktop, only Windows PowerShell…

    Definitely you can do it! Windows PowerShell v2 RTM on Server 2008/R2, which is also a requisite of SharePoint 2010, supports “remoting”. So you can manage SharePoint 2010 remotely with Windows PowerShell prompt on your local machine.

    Let’s try it!

    Enable Remoting support on SharePoint Server box

    A few steps are necessary to setup Windows PowerShell Remoting for SharePoint.

    Enable Windows PowerShell Remoting

    Windows PowerShell Remoting needs to be enabled first by calling the following cmdlet in Windows PowerShell:

    Enable-PSRemoting

    snap0089

    This command will do a quick configuration of Windows Remote Management (WinRM). A HTTP listener will be created by WinRM and firewall exceptions will be created automatically. If you get a Kerberos error, it could be possible that SPN for HTTP/yourservername is not there and you need to use setspn to add it. Most of the time you won’t have the issue.

    You can test if the remoting is working by type Enter-PSSession –ComputerName localhost on the same server box.

    However, there’re two extra requirements for SharePoint remoting. I just list them here, if you want further details, Zach Rosenfield, the Program Manager who owns SharePoint Windows PowerShell support, explained in his blog SharePoint PowerShell “Remoting” Requirements.

    Increase memory limit for remote shell

    Some of the SharePoint cmdlets could run for quite a long time and require a lot of memory. By default, a remote shell will be allocated 150 MB of memory, this may cause some of the command to fail, for example site collection creation. Use the following command to increase this limitation to 1000MB. This is only necessary if you need to run those commands on that server. 

    Set-Item WSMan:\localhost\Shell\MaxMemoryPerShellMB 1000
     
    If this value is too low, then you may have error messages like:System.Management.Automation.RemoteException: Process is terminated due to StackOverflowException.
     

    Setup CredSSP support

    Credential Security Service Provider(CredSSP) authentication should be used if you need to do “double hop” with your credentials. It does not mean using other authentication methods you can’t run the cmdlets at all, depending on different security permission scenarios, they may or may not work. CredSSP is the best way to deal with the situation.

    In some of the situation, even without CredSSP the cmdlets still work. For example, my current account is in Microsoft domain. The target server is in contoso.com domain. I used Negotiate authentication with a username and password to logon this server remotely, then created a new content database without any problem. You can test your environment to choose the best way – certain domain policy may prevent client machine from delegating credentials, which is required by CredSSP. But still, please use CredSSP in any case if possible.

    snap0117[4]

    To enable CredSSP on the server, use the following command:

    Enable-WSManCredSSP –Role Server

    snap0103[3]

    You can use Get-WSManCredSSP to check if it is enabled.

    Setup client machine for Remoting

    Enable CredSSP support

    To use CredSSP, you need to run the following command in Windows PowerShell, where * can be replaced with the server name you want to connect:

    Enable-WSManCredSSP -Role client -DelegateComputer *

    snap0111[3]

    Use Get-WSManCredSSP to check if it is enabled correctly.

    Create and enter a remote session of Windows PowerShell

    If your current user on client machine has permission to the SharePoint farm and Windows PowerShell on the remote box, you can use Enter-PSSession to create and enter the remote session.

    For example, connecting to sharepoint.contoso.com…

    Enter-PSSession -ComputerName sharepoint.contoso.com

    If it works, the command prompt will be changed to [sharepoint.contoso.com]: PS C:\Users\Administrator\>.

    The session will be closed when you type exit or Exit-PSSession. You can also use New-PSSession to create the session to use with Invoke-Command.

    To connect to a machine with CredSSP and a different credential, you can use

    Enter-PSSession -ComputerName sharepoint.contoso.com -Authentication CredSSP –Credential domain\username

    This will pop up a dialogue for you to type in password. If you want this process to be fully automated, you can store the credential first into a file.

    Store and use credentials for scripting

    A credential in Windows PowerShell is a object which contains username (as plain text) and password (as secure string).

    First, use the following command to covert password from keyboard input to a secure string in a text file.

    Read-Host -AsSecureString | ConvertFrom-SecureString | out-file C:\crd-sharepoint.txt

    snap0099[5]

    When you need to create a credential object, read this password (the secure string) from the file and create the credential with the following command:

    $pwd = Get-Content C:\crd-sharepoint.txt | ConvertTo-SecureString

    then create the credential (replace myusername with your domain\username):

    $crd = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList "myusername",$pwd


    snap0100

    Then you will be able to use this credential in the command line without any dialogue.

    Enter-PSSession -ComputerName sharepoint.contoso.com -Authentication CredSSP -Credential $crd

    Load SharePoint Windows PowerShell Snap-in

    Unlike SharePoint Management Shell, You need to load this snap-in manually to use the cmdlets for SharePoint.

    Add-PSSnapin Microsoft.SharePoint.Powershell 

    Then everything will work.

     

    Further readings

    Zach Rosenfield’s Blog

    http://sharepoint.microsoft.com/blogs/zach

    Zach’s blog is my favorite. The following articles are highly recommended to read…

    SharePoint 2010 PowerShell Permissions Explained

    http://sharepoint.microsoft.com/blogs/zach/Lists/Posts/Post.aspx?ID=56

    SPModule.HelloWorld()

    http://sharepoint.microsoft.com/blogs/zach/Lists/Posts/Post.aspx?ID=54

    Remote Install of SharePoint (with SPModule)

    http://sharepoint.microsoft.com/blogs/zach/Lists/Posts/Post.aspx?ID=55

    Webcast - Getting Started: Windows PowerShell for SharePoint 2010 Administrators, by Todd Kindt on TechNet

    http://technet.microsoft.com/en-us/sharepoint/ee518673.aspx

    Technical Reference: Windows PowerShell for SharePoint Server 2010

    CHM references for download. Please note there’ll be some changes in the cmdlets between beta and RTM.

    http://technet.microsoft.com/en-us/library/ee662539(office.14).aspx



    Got Questions?

    Ask them on TechNet Forum! If we got enough questions we may even open a separate section for Windows PowerShell!

    SharePoint 2010 - Setup, Upgrade, Administration and Operation

     

    Jie.

Page 1 of 1 (2 items)