The SharePoint Redneck

Ridiculously Busy

2007-11-29T19:41:00Z

Wow...did anyone forsee this SharePoint thing getting this huge this fast? I knew if was awesome, but I didn't foresee an adoption rate like this. Due to everyone loving out portal (take that Google), I have been swamped. Running around the country helping people become the best SharePoint administrators they can be is tough work. But the holidays are here and I hope to get a little down time. I have been seeing some things recently that are definately blog worthy and have also had a chance to do a little programming. Look for me on CodePlex hopefully in the near future when I get some of my code into a Product Group project.

Please don't forget about me. If you need your daily fix of SharePoint blog, check out the SharePoint Premier Field Engineering blog called From the Field (http://sharepoint.microsoft.com/blogs/fromthefield). This is a blog that consists of posts from many of my collegues around the world who do the same job I do.

Until next time, keep the duct tape handy!

A Change of Direction

2007-09-21T07:58:00Z

I wise old redneck once told me that every really good redneck story (or military story for that matter) had to start of with "No shit...there I was..." I laughed at the time, but after many years of considering myself in the redneck ranks, I have found this to be true. This post will be no exception.

No shit...there I was...sitting at dinner with a buddy who happens to be an Exchange engineer (don't hold it against him) and we were talking about email addresses. He had explained to me that the nice young lady who performed my technical screen when I was interviewing for my current position had a vanity email address that he had given her and he might be able to get me something too. So I sat and thunk about it for awhile and originally came up with the address of SharePoint.Samurai@microsoft.com because we are often called upon to go into the fray and be victorious warriors. But the more I thought about it, the more I started to think I had it wrong.

We aren't samurai doing battle in the name of the mighty SharePoint. We are more like rednecks. Now I know there are some of you who will read this and not get it. Coming from the south and having married into a family where redneck is an art form, I have seen my share of guys say "Here...hold my beer and watch this!" But that isn't all the redneck society is all about. Those are the folks who rightly deserve Darwin awards. I am talking about all the other rednecks out there that know the deal and have no allusions to what this life really is. I am talking about the guys who can (and do) fix their own trucks, trap their own food, build their own houses and can find a way to make just about anything work in some fashion.

SharePoint is this huge tool that can be just about anything to anyone who want to mold it into the shape they see fit. But who better to make those ideas come to life than the Cooters and Duke boys of the world. A little duct tape, some sheet metal and a little bit of time (not to mention a fast orange car) and we can get the visions of management shoe-horned into the architecture better than anyone else. We can hack it and push it and change it in a way that goes from undoable to done.

So...today I have decided to change this blog from "The SharePoint Traveler" to "The SharePoint Redneck" and will try and post more information about making this tool I have risked my family's well-being on into the thing of beauty I am sure it can be. As I find those little trick questions that my customers are always asking me, I will try and get them down and distill them into helpful posts we can all use. BDC, Excel Services, WorkFlow, just plain admin. All of these topics will cause us pain at one time or another. Maybe we can use that redneck spirit and ingenuity work for us. Who's with me? Post your comments here or feel free to send them to me in email at SharePoint.Redneck@microsoft.com.

SharePoint Rednecks Unite!

It's a Migration Thing...Part 2

2007-09-21T07:17:00Z

I get a lot of emails and comments asking for a followup to my original post on migrating SharePoint from one SQL server to another. Folks want to know how to move from one SQL server to another in WSSv3/MOSS since it is a bit different than it was in WSSv2/SPS 2003. Well...my buddy Ron beat me to it. We have been discussing this on and off and he finally got all the steps fleshed out in rough form and posted on his bloag (http://blogs.msdn.com/ronalg/archive/2007/09/20/change-the-sql-server-in-a-moss-2007-wss-3-0-farm.aspx) for us. As we get time, one of us will try and build out a nice document like you can find in my original post. But we tend to be pretty busy out there working with you fine folks face-to-face. I hope this will tide you over until then.

Import/Export and New Content Databases

2007-09-19T08:03:00Z

Earlier this evening, there was a discussion on an internal discussion list about the best way to cut a subsite out of a site collection and make it into a separate site collection in a separate database. We went back and forth and I finally suggested an option using STSADM from the command line. Now you need to understand that while this seems to work like a charm in WSSv3 (and for the purposes of this blog I will not be talking about WSSv2 or SPS 2003), there are some pitfalls to doing this in MOSS. The main one being that this has been known to break some webs that utilize the collaboration portal or publishing portal templates (or templates created from them). But for your regular run-of-the-mill team sites and such, this should work like a charm.

Your first step is to have a subsite. We will say that we have a subsite located at http://server/mysubsite that we want to make into a site collection located at http://server/mynewcollection in a separate database. We would start by exporting out the site collection to a file as such:

stsadm.exe –o export –url http://server/mysubsite -filename mysubsite.bak –includeusersecurity

Now why would we want to create a site collection in a new database? Well…it seems that Microsoft recommends that you try to keep your content databases under 25GB if possible. We recommend this for various reasons to include ease of backup and being able to maintain separate SLAs (service level agreements) for differing site collections (say HR or marketing). Do we want hundreds of separate content databases? No…of course not. But if possible, group like collections into separate databases (say IT collections or HR collections) for ease of maintenance later.

Since we don’t want to use the normal convention of http://server/sites/sitecollection, we are going to need to add a managed path. We can do that through the GUI or via command line as such:

stsadm.exe –o addpath –url http://servername/mynewcollection -type explicitinclusion

We make this an explicit inclusion because we want to put the new collection directly at this point. If we created it as a wildcard inclusion, then it would be like the /sites/ that we normally use. So you could optionally create a wildcard inclusion for IT and the URLs would stem off as in http://server/IT/sitename.

Now we have to create the new site collection in the new database. We do this to make a placeholder. If we didn’t, then there would be nothing to import to in the final step. At this point, we really don’t need to add a lot of fluff. Just do the basics.

stsadm.exe –o createsiteinnewdb –url http://server/mynewcollection -owneremail portal@server.com –ownerlogin DOMAIN\username

This gets us most of the way there. If you wanted to get fancy, you can always look at the syntax of the createsiteinnewdb option and notice that you could specify the database server, username and even the database name. This could come in handy if you are a bit anal about your naming conventions like I am. I like to see all of my content databases named as Content_xxx so they are grouped together for easy maintenance.

Our container is ready and all we have to do in import our backed up subsite into the new site collection.

stsadm.exe –o import –url http://server/mynewcollection -filename mysubsite.bak –includeusersecurity

Once this has been complete, you should be able to browse out to the new site collection. This could also be converted into a batch file or a VBScript in a pinch. For some of the syntax and an example of how to write a parameterized script, please see my last blog post. There is a zip file at the end that has a lot of batch files and a script that will get you on your way. And if that seems too difficult, you can always send me a note and I will try and help out as best as I can.

If you are trying to get the collaboration portal or publishing portal templates to export/import correctly, you should look at a post located at http://stsadm.blogspot.com/2007/09/convert-sub-site-to-site-collection.html to see how he did it. One word of caution though. The way he does it actually touches the database. This particular method is not recommended by Microsoft and may put you in an unsupportable state. I refer you to this post to see possible ways of doing something similar for education purposes only.

Recursive Backup and Recovery in SharePoint

2007-03-17T09:01:00Z

OK boys and girls. It is time for a post that is more on topic than I have been lately. Today’s story actually stems from an internal discussion between several of us Premier Field Engineers who specialize in SharePoint. We were having a discussion on the best way to perform site collection backups in a more automated fashion. One of my buddies (shout out to JC) had a VBScript (don’t know where he got it) that automated the process, but the script was broke and needed a little TLC.

It was actually a very simple concept. Enumerate the sites out to XML (you can do that via STSADM) and then loop through the XML to grab a backup of each site collection. Since there was a limitation in STSADM that it didn’t automatically lock the site prior to backup, we were originally using a Keith Richie tool called SPLSBackup. Because the STSADM in MOSS does perform locks, I switched over to the native tool for the newer version. At the end of this post I will have a zip file that contains all of the files for both 2003 and 2007.

The final part of this script would only launch a certain number of processes at a time. This could be set in the script to your whim. Then the script would loop through and launch that many processes. It would then wait until a process completed and launch another. In this fashion, it would move through all the site collections until finished. Throw in a little logging and you have a pretty usable script that you can run as needed to perform site collection level backups. Now on to the good stuff.

To best show how all of this works, I have built a couple of demo batch files. A quick not about setup though. I usually have a clean install of MOSS when I go onsite to demo this to customers. These batch files that I include are just there to get the demo running and clean it up afterwards. So use in the wild may be different than demonstrated. Don’t say I didn’t warn you!

Part of the first batch file I ~~stole~~ repurposed from a demo that Mr. Geoffrey Edge gave at TechReady 4 earlier this year. It is a quick batch file that uses STSADM to create a site collection and a bunch of nested webs underneath. I added in a couple more site collections as well. A separate batch file in the zip will also create the same collections each in a separate content database.

stsadm.exe -o createsite -url http://portal/sites/css -title "Customer Service & Support" -sitetemplate STS -ownerlogin domain\user -owneremail user@domain.com

stsadm.exe -o createweb -url http:// portal /sites/css/account -title "Accounts" -sitetemplate STS
stsadm.exe -o createweb -url http:// portal /sites/css/account/setup -title "Account Setup" -sitetemplate STS
stsadm.exe -o createweb -url http:// portal /sites/css/account/changes -title "Account Changes" -sitetemplate STS
stsadm.exe -o createweb -url http:// portal /sites/css/account/addresschanges -title "Address & Billing Changes" -sitetemplate STS
stsadm.exe -o createweb -url http:// portal /sites/css/account/servicechanges -title "Service Changes" -sitetemplate STS
stsadm.exe -o createweb -url http:// portal /sites/css/account/namechanges -title "Name Changes" -sitetemplate STS
stsadm.exe -o createweb -url http:// portal /sites/css/account/statements -title "Statement Changes" -sitetemplate STS
stsadm.exe -o createweb -url http:// portal /sites/css/account/taxes -title "Taxes" -sitetemplate STS
stsadm.exe -o createweb -url http:// portal /sites/css/account/billingdetails -title "Billing Statement Details" -sitetemplate STS

Once this is done, you can alter the VBScript as needed for your environment. Listed below are the main items that will need to be altered.

Const STSADM_PATH = "C:\path_to_stsadm\stsadm"
Const STSADM_LOGPATH = "C:\path_to_where_you_want_your_logs_to_go\"
Const STSADM_BAKPATH = "C:\ path_to_where_you_want_your_backup_files_to_go \"
Const STSADM_LOGFILE = "LogFileName.txt"
Const STSADM_XMLFILE = "XMLFileName.txt"
Const STSADM_PORTAL_URL = http://portalURL/
Const NUM_BACKUP_PROCESSES = 4

Now you can run the VBScript and sit back to see what happens. If all goes as planned, you should have a directory full of backup files for later use as well as an XML file (the output of the site enumeration we used in our loop) and a log file. The log file should look something like this:

3/14/2007 12:37:18 PM: Begin backup
3/14/2007 12:37:20 PM: Enum Sites
3/14/2007 12:37:20 PM: Wrote XML
3/14/2007 12:37:20 PM: Read XML into DOM
3/14/2007 12:37:20 PM: Executing Command 'stsadm -o backup -url "http://portal" -filename "C:\backups\portal.bak"'
3/14/2007 12:37:20 PM: Executing Command 'stsadm -o backup -url "http://portal/sites/css" -filename "C:\backups\portal_sites_css.bak"'
3/14/2007 12:37:20 PM: Executing Command 'stsadm -o backup -url "http://portal/sites/hr" -filename "C:\backups\portal_sites_hr.bak"'
3/14/2007 12:37:20 PM: Executing Command 'stsadm -o backup -url "http://portal/sites/it" -filename "C:\backups\portal_sites_it.bak"'
3/14/2007 12:37:22 PM: Waiting for command to execute and complete
3/14/2007 12:37:23 PM: Waiting for command to execute and complete
3/14/2007 12:37:25 PM: Waiting for command to execute and complete
3/14/2007 12:37:26 PM: Waiting for command to execute and complete
3/14/2007 12:37:27 PM: Waiting for command to execute and complete
3/14/2007 12:37:28 PM: Waiting for command to execute and complete
3/14/2007 12:37:32 PM: Waiting for command to execute and complete
3/14/2007 12:37:33 PM: Waiting for command to execute and complete
3/14/2007 12:37:33 PM: Found Open Slot: g_intOpenProcessSlot is 1
3/14/2007 12:37:33 PM: Executing Command 'stsadm -o backup -url "http://portal/sites/sales" -filename "C:\backups\portal_sites_sales.bak"'
3/14/2007 12:37:33 PM: g_intTotalProcessesLaunched is 5
3/14/2007 12:37:34 PM: Waiting for command to execute and complete
3/14/2007 12:37:34 PM: Found Open Slot: g_intOpenProcessSlot is 3
3/14/2007 12:37:34 PM: Found Open Slot: g_intOpenProcessSlot is 2
3/14/2007 12:37:34 PM: Executing Command 'stsadm -o backup -url "http://portal/sites/marketing" -filename "C:\backups\portal_sites_marketing.bak"'
3/14/2007 12:37:34 PM: g_intTotalProcessesLaunched is 6
3/14/2007 12:37:34 PM: Backup of portal site collections complete
Total Backup Processes: 6

Now that you have some backup files, what can you do with them? The main possibilities come in 2 flavors. This first is recovery of a whole site collection. Let’s say that one of your junior SharePoint admins accidentally typed in the following command:

stsadm –o deletesite –url http://portal/sites/css

At this point, the recycle-bin is useless to you as you can’t recover site collections from it. What do you do? You start by deciding on a mean enough punishment for the bad admin. It should include some form of shame and ridicule enough to get the point across. Maybe a rolled up newspaper comes into play along with the words “Bad junior admin!!” Once that is done, you can reach into your backup directory and pull out this command:

stsadm –o restore –url http://portal/sites/css -filename portal_sites_css.bak

Another flavor is if someone needs to recover something from a site, sub-site or list that is no longer in the recycle-bin or for some reason can’t be properly recovered. I will often build a separate web application within MOSS (or on another server if you wish) that I use for recovery. You can then run the same script as above except on the other application/server. My recovery web application is usually hidden and uses a non-standard port (I like to use 1130 since it is the anniversary of the day my wife really took over my life (love ya honey!) So the command would be:

stsadm –o restore –url http://portal:1130/sites/css -filename portal_sites_css.bak

There are probably other tricks that you can do from here depending on your abilities and how far you want to automate this. My colleagues who use PowerShell may have additional comments on this, but that is another topic way off my current beaten path (“Bad PowerShell junkies!!”)

I have delivered this little demo to 3 customers so far and as far as I know, 2 of the 3 have added this bit of flair to their kit bag. This is a great thing to be able to automate. My only word of caution here deals with the limitations of STSADM backups in SharePoint 2003. You should read this link (http://support.microsoft.com/default.aspx/kb/889236/en-us) on the subject. This is the reason that in 2003 we used SPLSBackup as mentioned earlier. This file is included with the 2003 zip file.

I hope this post is helpful to you. If you have questions or comments, you can add them to the end of this post and I will do what I can to get you answers. And to my friends in California who asked for this post to be written sooner rather than later (you know who you are), here it is. Until our paths cross again! Cheers.

"What we have here is failure to communicate!"

2007-03-09T22:45:00Z

As I stated in my last post (wow...over a month ago already?), I would like to talk about communication between departments for a moment. I know this gets away from SharePoint a bit, but is very important and does affect many SharePoint shops. I see a lot of companies who have split out their departments. They have seperate systems, database, and development departments or groups who all work on the same systems in different capacities. When this happens, politics and bureaucracy conspire to break down communications between these groups.

A case and point of this is where I was recently on site with a company who was migrating their SharePoint 2003 farm over to MOSS 2007. The group who owned SharePoint was responsible for admin and development of the SPS environment, but had to go through the DBA group to create new databases and manage backups. They had to go through the systems folks to work access and patches. The systems group also had a corporate policy for doing their patch management.

Because the company is so large and has so many servers, they patched by groupings of racks. They listed on a bulletin board what servers were being patched and when. Not only is this inefficient, it puts the responsibility of looking for patch schedules on the SharePoint admins. While I was on site with them, the systems folks patched their servers, which triggered some weirdness with the database, which impacted the SharePoint migration testing on the day of migration. Not good.

My point here is that because nobody was talking, bad things happened. This led to additional time and frustration trying to get everything sorted. And since the systems folks were so isolated from other teams in this case, added stress was caused as the migration rolled on. So it is very important that if you do have disjointed and uncommunicative departments of this nature, you should appoint someone (or a committee) that can act as the central communications hub and will facilitate getting the necessary coordination. If this isn't done, many long nights and some marathon internet sessions searchin Monster.com may insue. Cheers.

A Lesson in General Security

2007-02-01T09:12:00Z

In case anyone has noticed, it has been over a month since my last post. I didn’t do this on purpose, but have been on the road pretty extensively lately. As it turns out, I have witnessed some very cool things (MOSS 2007 is really growing on me) and some not so cool things. As I see the latter, I try to temper my gut reaction with the understanding of what it can be like to be an administrator in a world where corporate logic and funding are not always moving in positive directions. My mama (probably like most everyone else’s) told me once not to jump to conclusions about a person until I have walked awhile in their shoes. Well mom (I don’t think she reads my blog), I have been there and walked that walk, so now I am gonna talk that talk.

A couple of weeks ago I visited a customer site and spoke with an admin who was probably a bit overwhelmed to be doing the job that she should have been better trained for. A little downsizing had placed a SPS 2003 server farm in her hands with very little documentation. So the powers that be in her organization called us in and we took a drive out to see them. Once onsite we found that while there were several issues. One of the issues was the separation of job responsibilities between departments who don’t communicate. I will save this particular issue for my next post.

The main issue I wanted to talk about is security. Once we started diving into the issues at hand, we determined that we needed administration rights to run some of the tools that would provide us with a prognosis of the health of the farm. We found several things upon asking for this level of access.

First, everything was running under a single service account. While this is not always a bad thing, someone (preferably the server admin) should at least know the password to the account. Within SharePoint, this account is what has access to the database, runs the application pools, manages the content, runs search and indexing and performs all of the scheduled jobs within the farm (if you are only using one account). Our administrator’s personal account did not have the permissions we needed either. Even though she was a server admin, she didn’t have the appropriate permissions the tools needed to access the database.

Second, their only Active Directory administrator was out of the office for another week. I am sure that I shouldn’t have to lecture anyone on the need for redundancy. There should never be just one domain admin who can create AD accounts. Risk mitigation (aka Disaster Recovery) always tells us to have a backup.

Third, we were unable to get a hold of the DBA(s) in order to get the necessary permissions. They seemed to be unavailable or uncommunicative. This is one of the communications issues I alluded to earlier.

To make a long story shorter, we were able to get around the whole security issue by adding our commands to a scheduled script file they were using to perform backups of their sites. It took us exploiting their security to make our tools work. This should not happen this way. I strongly encourage anyone reading this blog to take a good look at how you manage the knowledge/storage of service account passwords. At my last job, we had them printed and placed in a sealed envelope in a safe that only a select few had access to. If someone left the company who had access, we went through the process of changing the password and updating the stored version.

If you don’t already know what all it would take to update WSS/SPS with a new service account password, please take a look at the link below. Together we can not only ensure our SharePoint environments stay secure, but also that we can get to the data we need when we need it. Cheers.

http://support.microsoft.com/kb/837813/en-us

Planning? We don't need no stinkin' planning...do we?

2006-12-19T09:07:00Z

In the past 3 weeks, I have visited several customer sites. All four of these visits involved discussions on upgrading from WSSv2/SPS to WSSv3/MOSS. That being said, I saw 4 very different environments and 2 major methods of thought about planning their upgrades. The first method is to take a step back and plan out the upgrade. The second is to just do the upgrade with limited or no planning and pray it works.

One of these customers used method #1. They are currently using WSSv2 with no SPS implementation. Still, they are working on a solid migration strategy that includes 1-2 months worth of planning and testing before actually jumping the fence and making the move. This is not only the smartest way of preparing for the move, but the Microsoft recommended method.

Another customer wants to use method #2. Limited testing and almost no plan. Since their environment is larger and includes much more customization/branding, this is a really easy way to spike blood pressure and cause massive amounts of overtime. The one thing they have going for them is that they are also trying to migrate to new hardware at the same time. At least there will be a rollback strategy if things go south in the process.

Do I ever condone method #2? Very rarely (but yes). Take my first customer above. They have a very limited architecture, small content databases and almost no deviations from "out of the box". They could easily restore their environment into a virtual machine and run a test to see if anything breaks. Most likely it won't and they are fine. But it still wouldn't address any of the reasons they are moving to MOSS.

Moving from SPS to MOSS allows an organization to take a step back and look at what they have. They can determine what is good and bad in their current implementation and then channel what they want to keep across to the new system. They can rethink their topology (or taxonomy) and determine if there is a better way to do it. This allows for change that can benefit the organization and allow MOSS to increase productivity and usher in cool new features instead of carrying over some of the things that may be holding you back right now.

So...what is the moral of my blog here today? Plan, test, plan, test and test again. And as you test, make sure you take detailed notes of what breaks and how you fix it. Keep this handy and build a checklist from your lessons learned in testing. This will allow you to perform as smooth an upgrade as possible and make you a hero...or at least not stroke out in the data center at 1AM due to crashing SharePoint servers.

Happy travels...

It's a Migration Thing...

2006-12-10T05:11:00Z

I was recently working with a customer in North Carolina. They were in the process of upgrading the hardware that was running their SharePoint 2003 farm. Part of what they wanted to accomplish was to move from a SQL 2000 cluster for their backend to a SQL 2005 cluster. This included not only the backup and restore of the databases, but the reconfiguring of their portal farm and all the servers in it.

The problem arose when we tried to get a good set of instructions on how to do this. I searched the internet and spoke with many colleagues before I finally got the steps right. But the information I got was very rough and not as detailed as I wanted. So I built out a document that walks you through the process. In this document, you will get a basic plan on backing up and restoring your SPS/WSS databases. Since the process may be different based on the backup/restore process of each company, I left that up to you. But I did provide a step-by-step walk-through of how to tell SharePoint that the databases are somewhere else. This includes screen shots and should provide a good guide.

Something to note before you start this. You should be on Service Pack 2 of WSS and SPS before migrating over to SQL 2005. I also recommend that you have all the latest hot fixes available for SPS/WSS before starting this process. That being said, open the attached document and see what happens.

Until the next time, I wish you happy travels.

A New Beginning

2006-12-09T12:30:00Z

Howdy there y'all. My name is Steven Roe and I am a Field Engineer for Microsoft. My specialty (as you may be able to tell from the blogspace's title) is SharePoint. That being said, in my job I travel around a lot working with customers in both proactive and reactive capacities. While I am onsite, I get various questions and see many different ways of using this technology. What I would like to do with this space is take these experiences and provide them back to the community.

As I find new and interesting nuggets of information, I think it best serves the community at large if we can get it back out to others who may be facing the same obstacles. I have learned so much by reading the blogs of folks like Keith Richie and Joel Oleson. By giving back these experiences, we can all prosper and move forward. So I will endeavor to add at least one post a week (we field folks get very busy) and hope you will comment and provide additional feedback to me. Feel free to drop me a note and let me know what you think. I may even throw in some of my feelings about the XBox 360 and my new Zune while I am at it to give a change of pace.

See you on the road!