Microsoft SharePoint Team Blog

October 2009 Cumulative Update Packages for SharePoint Server 2007 and Windows SharePoint Services 3.0

The server-packages of October 2009 Cumulative Update for Microsoft Office SharePoint Server 2007 and Windows SharePoint Services 3.0 are ready for download. October 2009 Cumulative Updates introduce more rules on Pre-Upgrade Checker, which can help customers to prepare the upgrade of their SharePoint farm to SharePoint 2010.

Download Information

Windows SharePoint Services 3.0 October 2009 cumulative update package
http://support.microsoft.com/hotfix/KBHotfix.aspx?kbnum=974989

Office SharePoint Server 2007 October 2009 cumulative update package
http://support.microsoft.com/hotfix/KBHotfix.aspx?kbnum=974988

Detail Description

Description of the Windows SharePoint Services 3.0 October 2009 cumulative update package
http://support.microsoft.com/kb/974989 (link may not be live yet)

Description of the Office SharePoint Server 2007 October 2009 cumulative update package
http://support.microsoft.com/kb/974988 (link may not be live yet)

Installation Recommendation for a fresh SharePoint Server

To keep all files in a SharePoint installation up-to-date, the following sequence is recommended.

1. Service Pack 2 for Windows SharePoint Services 3.0 and language packs
2. Service Pack 2 for Office SharePoint Server 2007 and language packs
3. October 2009 Cumulative Update package for Windows SharePoint Services 3.0
4. October 2009 Cumulative Update package for Office SharePoint Server 2007

Please note: Start from April 2009 Cumulative Update, the packages will no longer install on a farm without a service pack installed. You must have installed either Service Pack 1 (SP1) or SP2 prior to the installation of the cumulative updates.

After applying the preceding updates, run the SharePoint Products and Technologies Configuration Wizard or “psconfig -cmd upgrade -inplace b2b -wait” in command line. This needs to be done on every server in the farm with SharePoint installed.

The version of content databases should be 12.0.6520.5000 after successfully applying these updates.

You can also refer to April Cumulative Update post for deployment guides, slipstream how-to links and FAQs.

Jie Li

Technical Product Manager, SharePoint

Short Overview of SharePoint Features in Visual Studio 2010

As you probably know, Visual Studio 2010 was announced last week and it contains a lot of great features and project templates for SharePoint developers. Below is a short overview for some of the SharePoint development related features and project templates.

Configurable deployment

With all new SharePoint project templates you can leverage new configurable deployment feature which lets you configure the way you want to deploy or retract your project. Besides using provided, out of the box deployment steps (Run Pre-Deployment Command, Run Post-Deployment Command, Recycle IIS Application Pool, Retract Solution, Add Solution, and Activate Features) you can use SharePoint extensibility to create your own, custom deployment steps and deployment configurations.

Sandboxed and farm solutions

Some SharePoint projects can be deployed either as sandboxed or farm solutions. Sandboxed solutions run in a secure and monitored process that has limited resource access and with farm solutions user must have SharePoint administrator privileges to run or deploy the solution. You can read more about SharePoint sandboxed solution here.

Extending SharePoint Tools

Even though Visual Studio 2010 contains a set of project templates you can also extend them. You can create extensions for projects, project items, define your own project item types and create deployment extensions. You can read more about extending SharePoint tools on MSDN.

Feature and Package Designer

Feature and package designers give you the ability to customize features in your solution and with packaging designer you can customize which features are getting deployed and how. More about feature and package designer is here.

SharePoint Explorer

SharePoint Explorer is a new tool window that gives you a view into your SharePoint server. You can get a hierarchical view of lists, sites and workflows on your SharePoint server.

SharePoint Project and Project Item Templates

The following SharePoint specific project templates and project item templates are available in Visual Studio 2010:

Project Templates

· Empty SharePoint project

· Visual Web Part project

· Sequential and State Machine Workflow

· Business Data Connectivity Model

· Event Receiver

· List Definition

· Content Type

· Module Project

· Site Definition

Project Item Templates

· Empty Element

· Web Part

· User Control

· Application Page

· Association Form

· Initiation Form

· Business Data Connectivity Resource Item

· List Instance

· List Definition From Content Type

· Global Resources File

Besides above mentioned project templates, there are two import project templates for importing .WSP file contents and importing reusable workflows:

· Import Reusable Workflow

· Import SharePoint Solution Package

How to download, install and get started

If you are a MSDN subscriber, you can download Visual Studio 2010 from here. Download will be available to everyone on October 21st.

If you want to know how to download and install Visual Studio 2010 watch Channel9 video.

To get you started, head over to MSDN and read some of the walkthroughs on SharePoint Development in Visual Studio 2010.

Peter Jausovec
(http://blogs.msdn.com/vssharepointtoolsblog/)

SharePoint 2010 Resources

As Steve Ballmer announced this morning at the SharePoint Conference, and Jeff Teper notes below in his post, the public beta of SharePoint 2010 and Office 2010 will be available in November.

For more resources, take a look at:

- SharePoint 2010 Website - to view SharePoint 2010 in action

- SharePoint 2010 forum- for SharePoint 2010 questions

- SharePoint 2010 PressPass- for the SPC 2009 keynote video, a Q&A with Jeff Teper, and more

- SharePoint 2010 Developer Center - for developer info

- http://www.mssharepointitpro.com - for IT Pro info

- http://www.microsoft.com/sharepoint - for more SharePoint information

SharePoint 2010 Developer Center

The SharePoint 2010 Developer Center is now live on MSDN. This new sub-site includes Getting Started modules, as well as a Beta version of the SharePoint 2010 SDK.

To read more, take a look at the SharePoint developer documentation team blog, or head straight to the SharePoint 2010 Developer Center to see detailed, public technical information and instruction around both SharePoint Foundation and SharePoint Server 2010.

SharePoint 2010

This is my third and final post as part of our disclosure of SharePoint 2010 today. The previous posts covered the SharePoint History and how we Engineer SharePoint. This morning, Steve Ballmer and I are kicking off the SharePoint Conference so it is time to talk about SharePoint 2010! It is incredibly fun and rewarding to unveil this release driven input from you and innovative ideas from the team. As Steve announced, we will release the Beta of Office 2010 and SharePoint 2010 in November and look forward to your feedback as we finalization the product and documentation for release in the first half of next year.

Describing SharePoint 2010 in 1 Sentence, 8 Categories and 40 Feature Areas

SharePoint is a broad solution so we often get asked how we would describe it in a sentence. For SharePoint 2010, we settled on “The Business Collaboration Platform for the Enterprise and the Web”. A few words are worth explaining. We decided “Collaboration” was broad enough to cover the spectrum of publishing, sharing, finding, analyzing and managing information that SharePoint enables. We chose “Platform” not only because custom solutions are a major focus of the 2010 release but also "platform" conveyed a solid base for all the out-of-box usage that never requires a developer. Finally, we want to call out “the Web” to highlight both internet scenarios reaching customers and partners and the cloud-based delivery of SharePoint Online. For SharePoint 2007, we used a pie diagram chart to describe the major SharePoint categories such as “Enterprise Content Management”. This release, we picked words we thought were both simpler and gave us more freedom to innovate beyond traditional category boundaries. We settled on Sites, Communities, Content, Search, Insights and Composites as the new category names. Within each of these plus Administration and Development, I will highlight 5 major feature areas for a total of 40. At the next level down are hundreds of exciting new features which will be covered on www.microsoft.com\sharepoint and subsequent posts from the team. Needless to say, this is the biggest release of SharePoint yet and we hope you find it as exciting as we do!

Sites

In 2007, we expanded SharePoint to a single platform for intranet, extranet and internet sites. For SharePoint 2010, we improved the experience for this range of sites spanning browsers, Microsoft Office and mobile devices. The top five investment areas here are:

1. SharePoint Web Experience – We updated the SharePoint UI to make it simpler to access a growing range of tools. Highlights include incorporating the Office ribbon, in place web editing, AJAX responsiveness and richer navigation. We also expanded the reach of SharePoint sites through multi-lingual support, improved accessibility including WCAG 2.0 support and cross-browser support built on XHTML compliance.

2. Office Client – We continue to support previous versions of Microsoft Office working against SharePoint 2010. Office 2010 enhances this with features like offline editing with asynchronous saves as well as exposing SharePoint features through the new Office Backstage UI. Via the Backstage, you can access the context around the document including tags, related tagging and people.

3. SharePoint Workspace – In this release, we evolved and renamed Groove as SharePoint Workspace which provides great local and offline read-write access to SharePoint lists and libraries. SharePoint Workspace has a consistent experience with Office 2010 and SharePoint 2010 including the Office ribbon. It supports advanced features like bringing external business data offline and is smart about synching changes and not entire files.

4. Office Web Apps – We made SharePoint 2010 a great place to host the new Office Web Apps so you can view and update content from within a browser and include Office content as part of your web site (e.g. an Excel spreadsheet as part of “Sales Metrics Portal"). The Office Web Apps provide a familiar user experience, high fidelity viewing and essential editing without loss of data or formatting. They include Word, Excel, PowerPoint and OneNote. The OneNote client and Web App support is one of the coolest features of the release to enable multiple people to collaborate on a rich canvas online or offline. In addition to the Office Web Apps, we updated InfoPath Forms Services and Excel Services and added, new for 2010, Visio and Access Services.

5. SharePoint Mobile Access – We both improved the experience for mobile web browsers and are introducing a new SharePoint Workspace Mobile client so you can take Office content from SharePoint offline on a Windows Mobile device. These clients let you navigate lists and libraries, search content and people and even view and edit Office content within the Office Web App experience running on a mobile browser.

Communities

In the first post, I talked about breaking down organization and technology silos as key driver of our vision. Since then we have tried to make SharePoint the ultimate Swiss army knife for collaboration with smart connections across people and teams. You told us you want to embrace new Web 2.0 approaches within a unified experience which we included in SharePoint 2007. For SharePoint 2010, we expanded and enhanced the set of collaboration and social networking tools for both organic and managed communities across your organization. The top five investment areas:

1. Collaborative Content – Building on the new SharePoint user experience, we made it much easier to create and find content in SharePoint sites. This includes not only improved blogs and wikis (both simple and enterprise) but also calendars, discussions, tasks, contacts, pictures, video, presence and much more. With Office 2010, multiple people can simultaneously author content on a SharePoint site.

2. Social Feedback and Organization – With SharePoint 2010, we are introducing a consistent experience for organizing, finding and staying connected to information and people including bookmarks, tagging and ratings. We have taken a holistic approach across search, navigation, profiles, feeds and more. We are bringing together informal social tagging with formal taxonomy described below so you can choose the right approach for a given set of content. We have been using these features internally for a while and I think you will find the not only useful but fun.

3. User Profiles – We enhanced user profiles to reflect colleagues, interests, expertise – either via explicit tagging or recommendations based on Outlook and Office Communicator. The model is opt-in so users can manage what information is shared publically. They decide when an interest is something they want to share or be asked about by others in the organization.

4. MySites – We significantly enhanced MySites in SharePoint 2010 building on the updated SharePoint UI and user profile. We streamlined MySites to give you quick access to your content, profile and social network while continuing to let you customize, target and personalize pages to the needs of different roles and users in your organization. The enhanced newsfeed helps track interests and colleagues.

5. People Connections – In SharePoint 2003, we introduced a universal person hyperlink and presence icon so you can always navigate to a user’s MySite, send them mail, start an IM, call, etc. In this release, we enhanced this UI in conjunction with Outlook and Office Communicator as well as greatly improved the colleague tracking and people search features with new algorithms and user experience leveraging expertise, social data and more. MySites also include a new interactive organization browser built using Silverlight to give you another way to navigate the organization. In larger companies, org. chart browsing via the address book is one of the most popular features in Outlook and we think this takes that experience to the next level.

Content

SharePoint 2007 brought together document management, records management and web content management with a consistent user experience, architecture and platform. We built a common platform for metadata, security, workflow, etc. SharePoint 2010 adds scale and depth in these areas as well advancing the user experience. The top five investment areas:

1. Large Lists and Libraries – We made architecture and user experience investments so you have much larger document libraries with metadata driven navigation to help users go quickly to the content that is most important to them. Libraries will scale to tens of millions and archives to hundreds of millions of documents. This is a key investment for high-end document and records management but also helps organizations with lots of smaller sites. We enhanced the workflow capabilities and tools in SharePoint Designer.

2. Enterprise Metadata – We are addressing your feedback to support content types and taxonomies across not only across sites but also server farms. We have made applying this metadata easy (and valuable to users) in both the SharePoint and Office client user experience. The top-down taxonomy and bottoms-up social tagging (sometimes called folksonomy) combine to help improve search, navigation and people connections.

3. Document Sets – We are introducing a way to manage a collection of documents as a single object for workflow, metadata, etc. within SharePoint and Office so experience more closely models your work product (e.g. a proposal that may contain a presentation, budget, contract, etc.).

4. Web Publishing including Digital Asset Management – We made a number of key improvements to make it easier to publish rich sites on the intranet or internet. We used the new browser ribbon and editor experience to speed site customization, content authoring and publishing tasks. We added digital asset management features like thumbnails, metadata and ratings for images as well as video streaming from SharePoint. Finally, we improved content deployment robustness from authoring to production for larger scale sites.

5. Governance and Records Management – Compliance is an increasingly important requirement for organizations. We enhanced the Records Managements features in 2010 building on the scalable storage and enterprise metadata support described above. We improved the sophistication and flexibility of our governance tools. Just a few new features include location-based file plans, multi-stage dispositions, in-place records and e-discovery.

Search

As discussed in my first post, enterprise search is a big investment area for Microsoft from Search Server Express to SharePoint’s standard search to the new FAST Search for SharePoint. We added depth at all levels in 2010. While many customers will be fine with the base SharePoint search capabilities, FAST Search for SharePoint will meet the most sophisticated needs. FAST Search for SharePoint supersets the base SharePoint user experience, APIs and connectors. This is the first step, but a big one, and we will add more consistency and enhancements across our tiers of search in the future. We will continue to sell and enhance FAST ESP standalone as well. The top five investment areas here:

1. Interactive Search Experience – We built a richer search experience providing flexible navigation, refinement and related searches. Both Standard and FAST Search for SharePoint get query completion, spell checking, wild cards and more. FAST enhances this experience enabling feature content for common queries and providing more flexible navigation and document thumbnails and previews including in slide navigation of PowerPoint presentations which is a common end user scenario.

2. Relevance – We improved the out-of-box ranking and expanded the relevance factors including social data such as tagging and usage (clicks). FAST Search adds more configurable set of relevance inputs for custom applications and specialized corpuses.

3. People Search – We greatly improved people finding based on social networking and expertise algorithms and tailored user experience for people including getting views of authored content. As users frequently do not know or recall the spelling of people’s names, we built a new phonetic search algorithm that works much better than previous approaches to spell checking for names. In testing, we had a lot of fun coming up with crazy ways to misspell each others' names to see if we could stump it.

4. Connectivity – We know lots of data lives outside SharePoint so expanded and improved our connectors to index web sites, file servers, SharePoint, Exchange, Lotus Notes, Documentum and FileNet. The updated Business Connectivity Services (previously the BDC) described below makes it much easier to index an arbitrary source such as a custom database. You can create this search connection without code using the new SharePoint Designer.

5. Scale and Platform Flexibility – We made significant performance and scalability improvements through our search technology. Optimizing for 64-bit helped but we also introduce partitioned indices and scale-out query servers in SharePoint search this release. FAST scales-out even further and has significantly more pipeline extensibility to handle the largest collections and most complex value-added processing and search applications. We think both end users and IT will be immediately excited about the new capabilities supporting hundreds of millions of documents with great index freshness and query latency.

Insights

Historically, business intelligence has been a specialized toolset used by a small set of users with little ad-hoc interactivity. Our approach is to unlock data and enable collaboration on the analysis to help everyone in the organization get richer insights. Excel Services is one of the popular features of SharePoint 2007 as people like the ease of creating models in Excel and publishing them to server for broad access while maintaining central control and one version of the truth. We are expanding on this SharePoint 2010 with new visualization, navigation and BI features. The top five investment areas:

1. Excel Services – Excel rendering and interactivity in SharePoint gets better with richer pivoting, slicing and visualizations like heatmaps and sparklines. New REST support makes it easier to add server-based calculations and charts to web pages and mash-ups.

2. Performance Point Services – We enhanced scorecards, dashboard, key performance indicator and navigation features such as decomposition trees in SharePoint Server 2010 for the most sophisticated BI portals.

3. SQL Server – The SharePoint and SQL Server teams have worked together so SQL Server capabilities like Analysis Services and Reporting Services are easier to access from within SharePoint and Excel. We are exposing these interfaces and working with other BI vendors so they can plug in their solutions as well.

4. “Gemini” – “Gemini” is the name for a powerful new in memory database technology that lets Excel and Excel Services users navigate massive amounts of information without having to create or edit an OLAP cube. Imagine an Excel spreadsheet rendered (in the client or browser) with 100 million rows and you get the idea. Today at the SharePoint Conference, we announced the official name for “Gemini” is SQL Server PowerPivot for Excel and SharePoint.

5. Visio Services – As with Excel, users love the flexibility of creating rich diagrams in Visio. In 2010, we have added web rendering with interactivity and data binding including mashups from SharePoint with support for rendering Visio diagrams in a browser. We also added SharePoint workflow design support in Visio.

Composites

The single biggest area we increased our investment from SharePoint 2007 was making it easier for everyone – users, IT, partners, etc. – to build custom solutions on SharePoint that automate processes and connect disparate information. Some of the scenarios are more IT driven. Analysts often call them “Composite Applications”. Others are more end user centric or “Mash-Ups”. We thought “Composites” was a good short word to describe the breadth of solutions built with SharePoint. The top five investment areas:

1. SharePoint Designer – We revamped the SharePoint Designer experience to focus on the building blocks of a SharePoint solution vs. HTML source code. The user experience gets easier including the Office Ribbon and new tools for building workflows and connecting to external data. We have made SharePoint Designer customizations safe out-of-box in 2010 so IT can let users customize sites without risk. SharePoint Designer is also a great tool for mashing-up SharePoint (which now exposes REST) and external data.

2. InfoPath Forms Service – InfoPath is the best way to have a common form definition render in the browser as well as in a rich and offline client. For 2010, we improved the design environment to make it easier to build rich forms declaratively with little to no code and more client-side validation. We have also made it straightforward to use InfoPath forms as native SharePoint forms both on the web and when offline from within the SharePoint Workspace client.

3. Access Services - Users have long loved the ability to create database applications quickly with forms and views. Access Services lets you publish new Access solutions to a SharePoint site where they can be managed centrally and accessed (necessary pun) from a web browser.

4. Sandbox Solutions – In SharePoint 2007, custom code requires the farm administrator to trust the code running on the server. In SharePoint 2010 we are introducing a new SharePoint custom code sandbox with isolation and resource limitations (memory, SQL, CPU) that allows administrators to let others safely add and consume custom solutions without impacting overall farm performance and stability. While it does not cover the full SharePoint object model it addresses key scenarios like custom web parts and event receivers. We will use this and the client side object model described later to support custom SharePoint solutions in SharePoint Online as well.

5. Business Connectivity Services – We expanded the read-only Business Data Catalog from SharePoint 2007 to support create, read, update, delete, search and offline access to line-of-business (LOB) data. This data, such as a customer record from a database, web services, etc. is called an External List in SharePoint 2010 and it is mapped to an External Content Type so this data looks and behaves like native SharePoint lists. You can not only update this data from within SharePoint but can take it offline from SharePoint Workspace and, where it makes sense like contacts, in Outlook with offline editing. There is great support for BCS in SharePoint Designer and Visual Studio 2010. This perhaps our biggest “Wow, how did you do that?” demo. We will be building on the BCS for future LOB connectivity solutions.

Administration

We understand the most critical capability for you to introduce these features to your users is making it easier to manage. We have gotten a lot of great feedback in this area and made big investments for SharePoint 2010. As I have mentioned in previous posts, our experience with our CAT team, running SharePoint Online with SharePoint 2007 and targets for higher scale informed the design of SharePoint 2010. You have the choice of Server, Online or a mix. Even if you run SharePoint yourself, you benefit from our design and experience with Online. Beyond the fundamentals of scalability and reliability which got a lot of focus, the top five investment areas:

1. Improved Upgrade – We changed the model for SharePoint 2010 vs. the previous release. We will get your existing sites up and running with the existing SharePoint 2007 user interface including your customizations. You can preview and switch to the new UI at your convenience. With SP2 of SharePoint 2007, we released an upgrade checker tool that reports any potential issues. There are two key things to think about in planning the migration. First, as we announced a while ago, SharePoint 2010 is 64-bit only. Second, thin about places where you may have invested in custom code than can be replaced with out-of-box features such as the new enterprise metadata features described above.

2. Throttling, Health Monitoring, Analytics – Performance and reliability was a big focus for this release to address the scale of the largest organizations, web sites and the SharePoint Online service. We invested in optimizing the SharePoint, Windows Server and SQL Server architecture for scale and availability. We introduced more resource governors throughout SharePoint to prevent bulk operations or asynchronous jobs from slowing down the server. We built in proactive and extensible health reporting, monitoring and resolution in SharePoint 2010 based on analyzing a wide range of deployments. We will enhance this based on your feedback and our experience during the Beta and beyond. We are introducing a new usage analytics logging and reporting and are publishing the SQL schema for this analytics database so you can create your own reports.

3. Web and PowerShell Admin – We improved the web based administration interface for SharePoint 2010 and put it through the level of usability testing we had previously focused on for end user features. However, the biggest thing we heard from you was an improved scripting interface for SharePoint for simplified administration of farms. With the release of PowerShell, we were able to switch over all our administration to that framework and will ship with hundreds of commandlets you can use, edit and enhance. The administration framework is built on a new multi-tenant model we are be using on SharePoint Online but know is also of interest to 3^rd-party hosters and large organizations looking to do server consolidation.

4. Scalability and Availability – We made the shared services and federation model much more flexible to support richer scale out as you add services, sites and applications to SharePoint. We reduced the downtime for SharePoint 2010 with improved patching and SQL Server configuration, backup-restore, log shipping support and more.

5. Identity Management and Security – We have introduced more flexibility identity lifecycle management including updates between SharePoint with identity sources like Active Directory, LDAP servers and HR applications.

Development

I covered the higher-level solutions features under “Composites” above. Many of these enable building solutions with much less code than possible before. We also invested in a number of lower level development features as well for hard core developers. The top give investment areas:

1. New SharePoint APIs – This bullet is a blog post itself! The new UI framework has more extensibility in the ribbon and natively uses XSLT DataViews in lists vs. previous CAML views. There are new APIs for AJAX and Silverlight applications that make it make it much easier to access SharePoint data with less code and better performance. We significantly improved list access and programmability with REST, ATOM, JSON and LINQ including richer data relationship, validation, joins and projections over SharePoint lists which as described above can now reach far higher scale points.

2. Application Lifecycle – We have converged and improved on WSP as the packaging and deployment format for SharePoint solutions. You can save as WSP in SPD and bring that into Visual Studio 2010.

3. Visual Studio 2010 Support – SharePoint 2010 is a first class target for Visual Studio 2010. This includes F5 deployment and debugging (applause welcome …) as well as designers for various SharePoint project types, web parts, workflow, business connectivity services and integration with the VS Server Explorer. The early feedback on this has been so great, we decided to highlight it in Steve Ballmer's keynote at the SharePoint Conference.

4. Developer Dashboard View – If you have the rights, you can turn on a mode for a SharePoint page which will render at the bottom to show full trace and latency through the SharePoint, .NET and SQL layers. You can use our reporting tools described earlier to identify any slow pages in your site and then turn on this view to see a custom web part has bogged down the page by making repeated expensive SharePoint object model calls.

5. Development on Windows 7 – We now support development on Windows 7 and Vista client machines. Although it isn’t a supported configuration for production, we heard you that you want to use it as a development environment.

Summing Up

I hope this was a helpful overview. There will be many more details shared on www.microsoft.com\sharepoint and this blog in the coming days. We will follow the Conference, Blog and Forum traffic to update the answers to common questions and feedback and get the documentation ready for RTM. I thought I’d address two big questions quickly:

· Timeline – As Steve Ballmer announced today, Office and SharePoint Server 2010 will be available for broad Beta in November. The release is on track for the first half of 2010 as we have said for a while. We are upgrading all of Microsoft after having used earlier builds in production for thousands of users for the last year. While we have some tuning to do on the way to release, expect the Beta to be feature complete. We encourage and welcome all your feedback for this release, the documentation and beyond. We are committed to making this a solid release so your data will help tell us exactly when we are ready to ship in the first half of next calendar year.

· Packaging – We incorporated your feedback as we simplified packaging and naming this round. Here is the quick tour. The Server family includes SharePoint Foundation (simpler new name for “WSS”) and SharePoint Server (simpler name for “MOSS”). SharePoint Server continues to have Standard and Enterprise CAL tiers of features. We are enhancing our free basic search offering in Search Server Express 2010. The FAST Search capabilities will be available via the FAST Search for SharePoint add-on server for customers with the Enterprise CAL. For internet access by customers, we are enhancing the existing per server licensed SharePoint for Internet Sites and introducing a lower priced “Standard Version” for small to medium sites. As we announced a while back, SharePoint Designer is free. We think this combined with enhanced usability, designers and safety in SharePoint Designer 2010 client will expand the number and robustness of custom solutions. SharePoint Workspace (formerly Groove) is included in Office Professional Plus. We will announce updates to the SharePoint Online offerings closer to release. We will post additional details on this blog as we get closer to release.

Thanks again for reading the posts. I hope to see you at the SharePoint Conference. We look forward to your feedback and seeing your great sites and solutions on SharePoint 2010!

Jeff

Jeff Teper – Corporate Vice President, SharePoint Server, Microsoft

Engineering SharePoint

This is the second post in the run up to the SharePoint Conference on October 19th. Last time, I covered the history of SharePoint. This time, I will talk about how the broader Microsoft Office team designs and builds SharePoint highlighting improvements for the 2010 cycle. This is our fourth release of SharePoint plus several releases of Office before that. There has been continuous learning and improvement in our product design and development approach, particularly as we have expanded to servers and services during the last decade. For additional perspective, check out the Office 2010 Engineering Blog.

Process as a Reflection of the Product

You told us you wanted easy, comprehensive, flexible and reliable productivity tools and infrastructure and as a result we have continually evolved our team and process to reflect that mission. There are three key areas we focus where we differentiate our development approach - shared vision, innovative ecosystem and holistic quality.

First, a shared vision is the foundation of everything we do. SharePoint is a large and integrated product serving a broad set of customers. We have learned from discussions, research and instrumented builds, that organizations and users do a broad range of tasks to get work done. There’s ad-hoc brainstorming, statistical research, scheduling and project management, metrics calculation and analysis, internal and external communications and more. However, these activities should not be silos. Just as you told us you want your word processor and spreadsheet to be consistent and work together, you asked for simplicity spanning team collaboration sites to reporting portals. Hence, we always start with a shared vision, design, development and release process bringing together the teams building Office and SharePoint.

Second, we know you need SharePoint to be adaptable to specific industries, organizations and individuals. Building a healthy and innovative ecosystem on our platforms has always been a priority at Microsoft and we increased our investment in the SharePoint team for the 2010 release. We made big bets on customization, programmability, interoperability and upgradability that I will cover in the next post. While we have been working with a range of customers and partners for quite a while on 2010, we will ramp the broader ecosystem education and feedback cycle at the SharePoint Conference next week.

Finally, the most important driver of our engineering process is to meet your mission critical needs for the holistic quality of Office and SharePoint. We know this extends beyond the software to documentation, support, training and skills across the industry. We have the largest quality assurance investment (people, schedule, industry testing) in the industry and made a number of key improvements for 2010 to deliver SharePoint as a server and service. I will highlight some of these below. As I covered in the previous post, we do a few things out-of-band including value-added innovation spanning Microsoft, 3^rd-parties partners and the open source community but we make tradeoffs in our core engineering approach towards a well-designed, integrated and reliable wave of productivity solutions.

The Team

SharePoint is built by the Office group which includes approximately 40 teams and thousands of people. The core teams are in Redmond, Washington, but we have teams around the world with some of the larger sites in Silicon Valley, Boise, Boston, Ireland, Norway, India, China and Japan. These teams provide diverse experiences and perspectives and are a great test bed for our global collaboration tools. We often get asked about our development culture. I think the best description is a balance of top-down and bottoms-up vs. one extreme or the other. We try to balance consistency with innovation and empowerment with governance. If that sounds a lot like our vision for the SharePoint product, that is the idea. The team should reflect the product. While we consider a shared vision, schedule and process core to our approach, we have lot of brainstorming and collaboration as we iterate on this shared plan. After the vision is set, we push down almost all the decisions to the teams about what to build and how features should work as long as they are consistent with fundamental tenets like security and performance. That bottoms-up innovation is how features like My Sites and the Business Data Catalog got created and we have many examples like that coming in 2010. Of the 40 teams, almost everyone is contributing to SharePoint or building light-up scenarios that work with SharePoint. There is a core SharePoint team (actually two – one for Windows SharePoint Services and one for SharePoint Server building on top). However, the code comes from across the organization. Some teams (Word, Excel, PowerPoint, Visio, Access, InfoPath, OneNote, etc.) have been client-centric but increasingly are building client-server scenarios including the Office Web Apps (e.g. the 2010 PowerPoint Web App) or SharePoint features (e.g. InfoPath Forms Services). Other teams are more server-centric such as our Content Management, Search and Line-of-Business Integration groups but even they are focused on end-to-end scenarios. There is a mantra inside our group to optimize customer scenarios top of mind and not “ship the org chart”. Interoperability is a key part of this process from day one including publishing all the interfaces between our clients and servers. We have published SharePoint APIs and protocols for a long-time. We formalized this approach as part of Microsoft’s Interoperability Principles and significantly expanded the depth of our documentation in the last couple years including sharing preliminary information earlier than ever before for the 2010 wave.

Team investments and innovations come from a lot of different places and we used all of these approaches as part of the 2010 development:

· Existing Teams – Most of the 40 teams (e.g. Excel, SharePoint) evolve from previous releases coming up with ideas based on customer research, technology investigation, brainstorming, etc. Some of the work the Excel team has done for business intelligence in 2010 is a good example of a team evolving its mission in a big way.

· New Teams Created for the Release – OneNote is a great example of an effort where we created a new team within the Office organization. The core Office Web Applications team is the best example of this in 2010. Prior web work had been done in Outlook + Exchange, Excel and InfoPath, but we wanted to create a shared team to provide shared capabilities for the 2010 wave just as we have done in the rich Office client for several releases.

· Incubations – As I mentioned last post, SharePoint itself evolved from a couple different incubations. The Line-of-Business Integration team is the best example of this in 2010. We had a sub-team inside SharePoint Server building the Business Data Catalog (BDC) in SharePoint 2007, but we beefed that up considerably with the LOBi team who had been working outside SharePoint on some early stage work that we decided to bet on for this wave. We have a number of partnerships with Microsoft Research and other teams looking at things that will come in future releases.

· Acquisitions – While we don’t do a lot of acquisitions, we have been fortunate to make some great companies join us dating back to PowerPoint, SharePoint Designer (previously FrontPage), Web Content Management (from NCompass Labs) and SharePoint Workspace (previously Groove). The FAST Search team is a great example of an acquisition enhancing our team in 2010.

Hopefully, this gives you the sense that to build a broad product we have invested in a large team with a variety of sources of experience and innovation, but all striving towards a shared vision for building easy, integrated and interoperable SharePoint and Office clients, servers and services.

The People

The roles within Microsoft R&D have been covered a few books and blogs before so I will not go into great detail. At the highest level:

· Software Development Engineers (SDE) own the architecture and write the code

· Software Development Engineers in Test (SDE\T) own the test strategy, plans and automation investment and test the code

· Program Managers own the vision, specifications, project management and coordination outside the team with marketing and others

· Product Planners own the customer, market research and feedback processes

· User Assistance owns the documentation from in-product help to IT guidance to SDKs

· Designers owns the look and feel of the product from the overall approach to refining specific features

· Usability owns the prototype and product usability testing , analysis and recommendations

In addition, we have lots of key roles outside R&D helping us design, build and support SharePoint. Product Managers own marketing programs, customer and partner engagements, and readiness activities (like the SharePoint Conference) and provide some of the most important feedback to Program Managers.

While this sounds like a lot of roles, it is not as compartmentalized as you might think. Most of the work in Office and SharePoint is done in what is called “Feature Crews” consisting of a PM, SDE and SDE\T. They work extremely closely together on the whole lifecycle of an area of the product from brainstorming to scheduling to prioritization to development to testing to dogfooding to early adopter customer projects. Some Feature Crews use approaches like Scrum, some don’t. At this level we really leave it up to the team how they want to operate day-to-day and we try to keep the organization chart very flat above them. Most new hires have just 2-3 layers between them and an area VP/GM like me. We have a common engineering system (specs, build, test framework, etc.) but we encourage a lot of creativity and best practice sharing across Feature Crews. One we used for 2010 is promoting great early work the SDE\T team had done using real world data sets to validate upgrade much earlier in this cycle than 2007. Each release, there are dozens of new things we try in our approach to building better software. Some work. Some don’t payoff. For SharePoint 2010, our big new investment in people and roles was our Customer Assistance Team (CAT) that I mentioned in the last post. We were initially worried about creating another role. We already had the SharePoint Center of Excellence within our Microsoft Consulting Services (MCS) organization who looks at key customer and partner projects and brings the feedback into the team which was a key part of us designing 2007. However, we decided with the growth of the business and range of deployments, we really wanted dedicated people sitting within the engineering group living and sharing complex manageability, governance and application development customer scenarios. Our CAT team shares customer overviews, architectures and feedback on the http://office SharePoint site and in atrium sessions and works with developers and testers 1x1. In fact, I expect some of our top ranked sessions at the upcoming SharePoint Conference to be delivered or co-authored by members of our CAT team. In sum, while a broad product needs some expertise and specialization, the culture of the group is very much “make the right decision for the customer, make others great and get stuff done” so it really is a team effort which we think is an ideal culture for building a collaboration product.

The Feedback Loop

Customer and partner feedback has long been the key to the Office and SharePoint engineering process. We are fortunate to serve a very diverse set of organizations and users around the world. As you would imagine, one of our most valuable sources of input is discussions with IT leaders in large organizations. However, we are careful not to get skewed too much away from direct end user feedback. We talk to lots of end users and partners. We synthesize what is similar and unique when we to school teachers about collaborating with their students vs. VPs of manufacturing trying to share best practices in plant operations across their factories. A few of the mechanisms we emphasized while building SharePoint 2010:

· Statistical Research – We have a few different instrumentation and telemetry mechanisms we use in Office and SharePoint including SQM (actions and profile data), Watson (crashes) and Feedback from the App (e.g. Send a Smile/Frown). At scale, this is the best data we can get to help us go from gut decisions to data-driven decisions so we appreciate you using them. We have rigorous privacy guidelines around this data. This helps us understand everything from typical network latencies to query term patterns to document library sizes. Over the last few years, we have also gotten lots of operational data from SharePoint Online. While we always looked at activity profiles from Microsoft’s own 100,000+ person intranet / internet running on SharePoint and a set of early adopter customers, it is valuable to have detailed data from our much larger online farms supporting a broader range of customers. This also helps tuning the user experience, performance, deployment documentation, etc.

· Product Planning – Planning does a broad range of primary and secondary research we share across the team. One of our favorite things they do is “Day in the Life” research where we follow real people around doing their jobs irrespective of whether they use software to get tasks done to keep us grounded in addressing unmet needs.

· Internet Blogs and Forums – We read these intensely looking for the key trends and feedback so please keep it coming.

· Conference Chalk Talks –Some of our highest bandwidth discussions come after sessions at the SharePoint Conference, TechEd, the PDC and regional events where we have someone from the development team brainstorming with people on a particular topic – be it designs for high availability or how to evangelize SharePoint to new users. There is something about getting an unplanned set of people together without too much structure that sparks a lot of creativity about where we should take things next.

· MVPs, Customer and Partner Advisory Councils – These are more formal activities where we try to balance being inclusive but small enough to be manageable early in the cycle when our plans are still forming. Our MVPs have always given us great feedback during every stage of the development process. Our Partner Advisory Council meeting is small enough that everyone fits around U shaped table so people feel comfortable chiming in if they like or hate an idea. We have extranet SharePoint sites and Live Meetings for these relationships so we can collaborate virtually as well. We work hard to make sure these cover a cross section of organizations and geographies as we are not able to include as many people as we would like in these forums.

· Field Advisory Councils – We have a group of Microsoft employees in technical sales, consulting and support we call “SharePoint Insiders”. They are mix of new and old employees - some have been part of our feedback loop since pre-release SharePoint V1 @ 2000. They are a very direct group and do not mince words when reflecting the feedback they hear from you or opportunities they see.

· Usability Labs – During the development cycle we do 1000s of tests (paper prototypes, terminology, screenshots, working code, etc.) with a sample of people following a pretty rigorous methodology. The labs are here on campus but anyone can watch a test live from their office. This is one of my favorite parts of the development cycle. If we show a design for a new feature to 8 people and only 2 of them can complete the task, it is back to the drawing board.

· Early Adopter and Beta Program – We have a staged disclosure where we work with a cross section of early adopters to get early product feedback followed by a broad beta to help us understand what we need to fix across a broad cross section of scenarios. The upcoming broad beta is a very critical part of the process, so again, thanks for your participation when we roll that out.

I could keep going but this is covers the biggest investments.

The Vision

My second favorite day in a SharePoint release (after the day we ship) is the day the team rolls out the vision. This includes a vision statement, pillars (e.g. embracing cloud delivery, supporting business processes), tenets (e.g. upgradability, security) and, of course, the schedule which grounds us in the realities of tradeoffs needed in any software business and project. The schedule is based on all the data we analyzed from the last release about how long it took us to complete various parts of the engineering cycle vs. hopes. It gets rolled out to the team in three forms – a one page Word document we can all keep on our desktops or walls to guide us on tradeoffs through the release, a longer Word document that goes into detail and a big PowerPoint demo to the team. The latter is my favorite part. We take everyone in team into a big conference center for the vision roll-out meeting. There are a couple text slides to describe the pillars and tenets but 90% of the meeting is the experience we want users, developers and IT to see. One big giant three hour demo. Everyone leaves very fired up. With the vision in place, we unleash specing, development and testing. Here is where lots of details get written down, debated, reviewed and finalized. Along the way, the teams come up with new ideas, get feedback from customers, partners and the market and make tradeoffs to adapt our plans but that is much easier to do from a grounded vision. All of the collaboration and tracking is via our intranet http://office SharePoint site I will describe below. However, we have a few key face-to-face checkpoints along the way. The first is “demo day” after the first coding milestone. We get everyone back together to demo the real code to see how it holds together and where we can share ideas or need to adjust the plans. The second checkpoint comes later when the code is close to polished. The design team takes the original vision PowerPoint and replaces the mockup bitmaps with real screenshots of the working builds. Again, it is a great chance to holistically see where the designs have evolved from the vision day (which they always do a lot!) and where we still need to tweak. There is always a mix of some cool new things we added and some things that we had to defer but each release we have been about 80-90% consistent with the original vision. I think a clear collaborative vision is a key to building a successful team and broad product like Office or SharePoint. We obviously hope you use SharePoint to foster the same kind of planning and collaboration in your organization.

The Process

I will not going into the detail of how we do software engineering. As with our roles, there’s been a number of good books and blog posts over the years that have covered at least parts of what we do. One of the best has been the Engineering 7 blog done for Windows 7. I did want to highlight a few things we evolved for SharePoint 2010.

· Dogfood – A lot has been written about the importance of dogfood (running your own software internally for real work) at Microsoft to build usability and quality in and give us a data on the project status. When I first joined Microsoft in 1992, I was put on a very early dogfood version of what became Exchange for all my e-mail and used a pre-alpha version of what became Access for a customer tracking app. It was rocky at first but a great chance to get data, give feedback and help ship these products. The Windows NT team compiled Windows NT using early builds of Windows NT which was called self-hosting and developers got a visit from Dave Cutler, the NT development manager, if you they broke this. Dogfooding is the most critical to driving a sufficient level of real world stabilization to facilitate external feedback. This release, we embraced dogfooding earlier and more frequently than ever in SharePoint. We have been using Office and SharePoint 2010 builds to do our work in the team for over 18 months. For me it has been no net ever since. Mails via Outlook 2010. Documents to SharePoint 2010. For the client software, we have a http://voteoff site where people in the development team vote and comment if a particular build is good or not. While we have lots of manual and automated testing where we find the vast majority of our issues, but the data we get from dogfooding is an indispensable part of the process. If we find an issue during dogfooding, it not only improves our product but also flags areas for increasing our test automation. For SharePoint, over the course of the release we have dozens of upgrades to our dogfood and puppyfood (the corny name for our pre-dogfood servers for the really brave– another 2010 cycle invention). We just upgraded http://office again this week and the build is looking very good. We will start the process to upgrade the SharePoint servers for all of Microsoft which we see as a critical part of validating the beta release before making it available to all of you.

· Early Adopter Interaction – In the past, we gave the software to early adopters who described their scenarios and feedback. This release we had Feature Crews actually build sites with these early adopters. This turned out to be extremely educational. Another new thing we did year was hosting trade fairs in our Building 16 atrium where each team had a booth to show off their customers’ projects and developers would walk around and learn about how features would be used and feedback. We gave fun awards for the best sites including most creative scenarios and most interesting bug found in the process. It was a great chance for developers in the team to see customer scenarios face-to-face, ask questions, etc.

· Performance Analysis – SharePoint performance is very complex and multi-dimensional depending on the hardware, software and network configuration, data set, usage profile, etc. Our CAT team has done a lot of great work to publish much better performance guidance for SharePoint 2007 than we had at 2007 RTM. They also helped the entire team dramatically increase its performance analysis and investments for the SharePoint 2010 cycle. We probably collective 2 orders of magnitude more performance data vs. during the 2007 cycle. For example, we don’t just want to understand the median latency of a page but the 95^th percentile latency across a browser and bandwidth matrix as users don’t want the product to be slow even 5% of the time when an asynchronous job might impact performance. From looking at hundreds of our largest customers in detail, the top two opportunities for performance improvement are the hardware configuration and custom code written to SharePoint. So a lot of what we did to help our development team we put in the product for 2010. Some of this is tuning, isolation and throttling but one of our favorites is a report that shows you your slowest pages and lets you turn on the “Developer Dashboard” on a page so you can see exactly where the time is spent. A couple releases ago when we upgraded Microsoft overall corporate portal to SharePoint 2003, it took us several hours to track down a slow page to a custom web part that did too many round trips to a non-SharePoint database. In SharePoint 2010, you will know this proactively and can diagnose in minutes. We also have a new way to prevent this Web Part from slowing down the site which I will describe in the next post.

· Grid Team – In the last couple of posts I mentioned that SharePoint Online is big part of our development cycle. It is a way to reach many new customers. It also gives us a much larger scale environment than our own SharePoint intranet, extranet and internet apps to analyze and tune. This release, we created a dedicated team called “The Grid” team to analyze and optimize in great detail the total costs, time, reliability and other factors of a SharePoint deployment we hope to reach 10s and eventually 100s of millions of users. Almost all of what we learn there helps make the product more reliable for customers and partners hosting their own servers from a small company to a 250,000 employee enterprise. There is a bunch of great learning from things like virtualization with Hyper-V to best practice analysis to operations monitoring we will share in the coming months. The Grid team will be at the SharePoint Conference.

SharePoint on SharePoint

The key to successful dogfooding is picking a real scenario and not just a playground. So we run the Office and SharePoint projects on early builds of SharePoint 2010. Except for the first couple builds, there is no fall-back site. We have to make it work. I often get asked about how to balance empowerment with oversight by SharePoint customers, so I will describe in a little detail what we do on http://office as we have a mix of ad-hoc stuff with some pretty formal engineering processes. Part of http://office is collaboration sub-sites for each team, where they can do whatever they find useful except, and this is a big one, open up security to anyone they want in Microsoft. We use SharePoint’s security permission and inheritance features to restrict who in Microsoft has access to information about future versions of Office. The core of http://office is more structured sites including Vision, Schedule, Specs, Test Plans, Status, Metrics, etc. A few people running the overall project customize these but almost everyone contributes and consumes. Right now, I spend a lot of time looking at the Metrics site. I have two monitors on my desktop and one usually has an IE window with this maximized. It uses our BI features including Excel Services and SQL Server integration to bring together data from across the teams on a variety of metrics. It uses Excel features like heat maps (e.g. teams with more than 5 bugs per developer are red) and sparklines (new mini charts that fit in a cell in Excel 2010 showing) to show areas for focus and key trends. Finally, we have a bunch of social networking tools on SharePoint. I keep my internal team blog there. We have Engineering Wikis for developers, testers and program managers that are living training and best practice sites that new hires in particular find very useful. Everything on http://office is indexed so rather than everyone sending e-mail to everyone about everything, we use alerts, RSS feeds and some new 2010 social networking features I will describe in the next post to keep track of things that are of interest to us. I mentioned our design and usability specialists earlier. I subscribe to a weekly alert for all the test and research posts they make to the SharePoint site. Others may want that more or less frequently. http://office is used by thousands of people around the world daily so we also get great operational data that helps us tune the work-in-progress code as well. So dogfooding helps throughout the cycle - from validating new collaboration features to ensuring good code quality to final stabilization and documentation work. Of course this is not sufficient; it complements manual and automated testing, early adopter testing and pilots, broad betas and other parts of our process.

Three Common Questions

I thought I would wrap-up this post with the top three questions I hear from customers and partners about how we think about building SharePoint.

· Do you guys think of SharePoint as an application or a platform? SharePoint is both but we start by aspiring to be an application. We want to solve as many problems as we can out-of-box without requiring a developer. Of course, many sites and solutions need developers and in the core SharePoint teams we have lots of people working on developer features and 2010 will be a major leap in programmability and developer productivity as I will cover in the next post. But I think we should always be looking at custom solutions and asking how we could have enabled them using less code. While this is this the right philosophy, there will always a huge variety of needs for building solutions on top and we’re very focused on supporting that as well. As I covered in the last post, while we are focused on solutions, we made some very clear architecture bets in SharePoint on integrated storage, rendering, security, etc. so the platform is a big internal focus as well.

· How do you guys make tradeoffs vs. product X, Y or Z? SharePoint is a broad product and we position it as such so we understand we invite this comparison. We have been asked to contrast SharePoint to everything from SAP to Facebook! We make tradeoffs focused on broad deployment of a general purpose server and service spanning collaboration, portals, content management, search, business intelligence and business process integration. All else being equal, we will prioritize capabilities used by the broadest number of organizations and users. Over the last couple of releases we have substantially increased SharePoint R&D to invest in depth as well. To use the Office analogy, not every Excel user creates PivotTables but most organizations have lots of people creating them and many more people interacting with them. You saw a leap in depth investments in content management in 2007 and will see a similar leap in Search and BI in 2010.

· What do you guys really think about the cloud? Is it an opportunity or a threat? What if I want to or need to keep running my own servers? We love the cloud! We have been running SharePoint Online for customers based on SharePoint 2007 for a few years. Last year, we announced some of the largest enterprises adopting SharePoint Online as well as a multi-tenant service for small-medium sized organizations. You can choose to use SharePoint Server, Online, partner hosting or a combination. We see all models going forward. While SharePoint Online helps organizations who may not have access to IT resource, we believe it helps in larger organizations with deep IT groups well. We think we’ll do the very best job running it which translates to happier users and IT and lets IT professionals spend more time higher business value activities for technology adoption than just low level operations tasks. Even for organizations who want to run SharePoint, you will benefit because we build our experience running high availability, low cost services into the product and guidance. Again, your choice: server, service or mix. We apply and share what we learn from our SharePoint Online experience.

Thanks again for reading and your feedback. Next post from me will be an overview of the 2010 capabilities. I promise!

Jeff

Jeff Teper – Corporate Vice President, SharePoint Server, Microsoft

SharePoint Conference 2009 has officially SOLD OUT!

SharePoint Conference 2009: SOLD OUT! We’re officially SOLD OUT! With 7000+ SharePoint experts, developers, partners, users, MVP’s and enthusiasts this is officially the BIGGEST SHAREPOINT CONFERENCE EVER!!!  We’ve had a phenomenal response to the conference this year with a 94% increase in attendance and we can’t wait to see you all in just over 10 days!

Discounted Hotel Room Rates: SOLD OUT! Due to high demand in registration our discounted hotel room rates at The Hotel and Mandalay Bay Hotel, are SOLD OUT!  Luckily Las Vegas is a big place with plenty of hotel rooms, so if you’re looking for accommodation check out some of the deals at the neighboring properties like the Luxor, but act quickly!

Post Conference Workshops: SOLD OUT! The SharePoint Developer Deep Dive and SharePoint Server 2010 Installation and Upgrade post-conference workshops are both SOLD OUT!   If you’re interested in attending one of these post-conference workshops open to all registered SharePoint Conference 2009 attendees, please contact spc@microsoft.com to join the wait-list.

Registered? Why Not Tell Everyone!  We're excited you've decided to register for SharePoint Conference 2009 and you should be too! Here are some images you can use on your website and in your email signature to let others know you'll be attending.

For more information visit www.mssharepointconference.com.  The Microsoft SharePoint Conference 2009 Team

SharePoint History

Hello!

As we are just two weeks away from disclosing SharePoint 2010 at the SharePoint Conference starting October 19^th, I wanted to write three posts to provide context on the upcoming release. This first post will cover the history of SharePoint. I hope it will provide some useful perspective behind our vision and what we have learned as well as a few fun anecdotes. The second post will cover the engineering process for SharePoint 2010 - how we design and develop SharePoint in the Office team, what new approaches we have taken during the 2010 development cycle and my take on a few frequent questions I hear from customers and partners. The third post will coincide with the opening of SharePoint Conference and cover the major feature investments. After that, our team will start blogging in depth about the new SharePoint capabilities. For folks who cannot wait, we have highlighted a few of the new features on the SharePoint 2010 Preview Site as well as commented on a few places where Office 2010 works with SharePoint on the Office Engineering Blog.

Before we jump in to this post, I want to thank everyone who has been with us on the journey so far. Since the first beta of the first release, we have been extremely fortunate to have your support and get your feedback. It has been extremely rewarding to see all things you are doing with SharePoint. We have posters of many of your sites on the walls of Building 16 on the Microsoft Redmond Campus that are a source of pride and inspiration for the team to take SharePoint to the next level. We have loved hearing your ideas about how we could support you better and where the product should go. We think SharePoint 2010 will be another big step forward but we know there is more to be done and are thankful you care enough to keep pushing us. Finally, we want to thank the Microsoft teams around the world who has worked so hard to build and support the product with special thanks to a couple dozen people who were with us from the very beginning in 1998. We are very fortunate to have a great team, customers and partners and that is what keeps us fired up to come to work every day.

Ok, on to the post.

Pre-History

As long-time Microsoft followers know, in the 1990s we had several efforts targeting information access and sharing. The rise of web on the internet and intranet was a great catalyst to focus these plans. I will highlight three threads that I consider part of the SharePoint pre-history in the late 1990s. The first thread is focused on end users via the FrontPage and then Office Server Extensions. These extensions installed on web servers to let users create and edit web sites, post Office documents, participate in discussions and get subscriptions. Many ISPs hosted the Server Extensions helping millions of consumers and business users collaborate on the web. There was strong integration with Office 97 and Office 2000 so you might say these were the first cloud-based Office web collaboration releases! The second thread focused on more sophisticated web site developers and was called Site Server which was a grab bag of tools and services to help create and manage sites. Finally, we had a set of internal incubations. One was called “TeamPages” which was developed inside the Access team to enable users to create and edit simple web-based lists. Under the covers, this was one of the first uses of XML - the “CAML” markup inside SharePoint that pre-dated XSLT. The other was called the “Digital Dashboard” which was initially an effort between our marketing, Excel and SQL teams to make it easy to create dashboards in Outlook but quickly moved to a web-centric design. We learned a lot about the needs of end users, developers and IT from these efforts that helped us design the first release of SharePoint and make a strategic bet for Microsoft in the second release. This process continues at Microsoft as we have both coordinated planned efforts as well as organic prototyping. For example, our Office Labs Group created a social networking project called “Townsquare” that we used inside Microsoft before refining the ideas in SharePoint 2010’s new social networking features.

Vision

As part of planning of what became SharePoint(s) V1 and Office XP, we did lots of research around the world. We visited different size and type organizations and met with a wide range of users, developers and administrators. We talked to lots of people and looked at lots of intranets, document and knowledge management deployments. It was immediately clear there was an opportunity to help both end users and IT be more efficient and effective. Many customers told us about intranets or ECM systems they had rolled out with lots of fanfare and then failed to get used. We heard lots of stories of stale web site. Users shared documents via e-mail vs. the official repository. Many customers told us about their challenges with different silos that had grown up to meet the needs of different business units. One Fortune 50 company told us they had 600 different content management systems! Imagine the difficulty for users trying to collaborate, developers trying to link business processes and administrators trying to keep costs down and reliability up. Users told us they wanted to go around IT. IT told us when they did this they create flakey web sites. But as with many latent needs, there was no single definition of the ideal solution in the industry. So we worked on a plan to address both usability and manageability challenges. As I was writing the blog, I went through some of our early presentations and I was struck by a one slide PowerPoint deck we gave to Bill Gates early in the process. We knew reviews with Bill were always intense so we decided to come in with a single slide explaining our vision with lot of screenshots of what we thought the product might look like. We have lots of fancy slides now to explain SharePoint but I thought it would be fun to show what we shared with Bill exactly as is below. The neat thing is ten  years later, it still applies to SharePoint 2010. And yes it was an intense meeting!

SharePoint V1 and Office XP

As some of you recall, there were two SharePoints at the beginning. These evolved from projects codenamed “Office Server” and “Tahoe” around the Office XP development cycle. “Office Server” evolved out of the FrontPage and Office Server Extensions and “Team Pages” and targeted simple, bottoms-up collaboration. “Tahoe” was built on shared technology with Exchange and the “Digital Dashboard” and was targeted at top-down portals, search and document management. They shared Office client integration and “Tahoe” had web part and search links so top-down portals could contain integrate with bottoms-up sites. Coming up with the name “SharePoint” was a fun part of the process. We were in a giant conference room with dozens of proposed names on the wall. Everything from words we would imbue with meaning in the market to clear names like “Office Document Server” that was descriptive but boring and limiting. We agreed on “SharePoint” in a single meeting but had many discussions then and in the years ahead about what were the right words around “SharePoint”. We eventually did elect to use the same name for both projects to convey the integration and future strategy. We decided to ground “Tahoe” in the “Portal” category because it was an area of a lot of buzz (“Portal Power” had was a big cover story for InformationWeek, there were portal conferences, reviews, budgets, etc.) and we had seen Lotus Notes and Exchange had struggled to define a new groupware category. So “SharePoint Team Services” and “SharePoint Portal Server” were the first names. The most memorable part of this release cycle was the end game. I remember hours being in what we then called the “war room” (a hyperbolic term we avoid now). Projected on the wall was an Excel spreadsheet with the status of 20 customers who we wanted to know were live with their deployment before we shipped the software. Program managers would come into room with updates from daily calls with these early adopters about issues, how much content was stored in the system and number of active users. When everyone was green, we released and had a fairly memorable ship party. As we still do today, we had screenshots of each of these customers’ portals hung on the walls so everyone in the team would know we weren’t writing code for ourselves but to help these customers solve business problems. Another gratifying aspects of this first release was the reviews. EWeek’s headline was “SharePoint’s a Hit” and Computer Reseller News called SharePoint a “Whopper of a Product”. But we knew there was a lot more to do to build a truly integrated product and experience between the two SharePoint and we could do a better job of holistic planning across scenarios and technologies. Since not many people see a server software box anymore vs. just getting a download or CD, I thought it would be fun to show the SharePoint 2001 package:

SharePoint 2003 and Office 2003

While it was great to see the rapid adoption of STS and SPS, it was clear customers wanted a more integrated and comprehensive solution from us. They told us like they liked the WYSWIG HTML editing of SharePoint Team Services and the Web Part declarative and reusable editing of SharePoint Portal but wanted to use both models on the same site? Inside Microsoft in 2001, we were deciding how much to bet on SharePoint. Would it be a good product in a broad family of servers or an even more fundamental bet for the company? It was exciting period because we were at an inflection point on both our platform and application strategies. On the platform side, we were making a very fundamental bet on Web Services including collaborating with vendors across the industry on the evolution of XML, SOAP and much more. We saw this and the increased maturity and scalability of SQL Server would give SharePoint a foundation for highly scalable web-based solutions. With growing computing power, we could host even the largest companies on a few sever farms (Microsoft has 3 for its worldwide SharePoint deployment). On the application side, we were hearing customers wanted Office to go beyond personal productivity to organizational productivity and we had to decide whether Microsoft would invest in content management, portals, unified communications, business intelligence and many other new scenarios. As you can guess now, in classic Microsoft fashion we decided to bet big. However, we were pretty quiet about this until we were close to beta of the 2003 Wave. The entire Office team had SharePoint as a vision pillar and we designed the user experience, architecture, extensibility and in a more holistic way than we had done in the previous release. We evolved STS into a more scalable and flexible platform and built SPS on top of it. One of the many heated discussions during this time was between the WYSWIG and data-driven camps on web user interface framework and we ultimately reconciled these in the SharePoint page model where pages have zones where the web site owner can decide who can add web parts. At this point, we also knew that customers, partners and Microsoft would be hosting SharePoint on the internet and we made a lot of core investments in role-based delegation, partitioning, stateless front-ends, etc. to enable this. These architecture bets enabled us to ultimately offering SharePoint Online to both dedicated and multi-tenant customers in the 2007 release. While this was a long release, we could not have gotten the integrated experience and platform as lots of different skunk works efforts. It really needed teams to research, plan, develop and test together. At the same time, it was a very aggressive project and we had to make a lot of tradeoffs about what to leave for the following release. One of the most controversial was item level security. We knew this was a very critical feature for some projects. However, it was going to be big investment for us on top of everything else we were doing and was not realistically going to make our 2003 schedule. So we decided to defer it until the next release when we could do it right. We still use this as an example in the team of the tough tradeoffs needed for balancing features vs. dates software projects. One of the best bets we made in this release was MySites. This was a classic bottoms-up feature from the development team. In this period, most portal products allowed users to have personalized pages ala MSN, Yahoo, etc. The team proposed we give every user their own personalized site and is still very proud they shipped it in 2003 before there was a MySpace or Facebook. It is a great example about not just doing what customers ask for explicitly but what we think they might eventually find most useful. When we first released MySites in 2003, many IT people first asked us how to turn it off. By 2007, they were asking us how to turn it back on as their employees and even their CEOs were asking them for enterprise social networking. Below is a picture we used many of times inside and outside the team as we focused on what I call “The Scalable SharePoint Release”. We launched it as SharePoint Portal Server 2003 built on top of Windows SharePoint Services V2. There were many important lessons from this project about the power of shared vision and a collaboration culture. However, we learned while we provided IT a central infrastructure users where users could go create their own sites, we had not yet provided all the management tools IT needed. That and depth category investments were the focus on SharePoint 2007.

SharePoint and Office 2007

With the integrated experience and framework in place, we added depth in the various SharePoint modules. Hence, I think of 2007 as “The Pie Release” because of the picture we used below to describe it. Content Management was our biggest area of feedback and, therefore, investment. As customers put more content in SharePoint and built more sites with it, we got a lot of requests to take this to the next level – more formal processes, more oversight and more sophisticated web publishing. Building on the new Microsoft platform technologies such as Windows Workflow and Windows Right Management, we also invested document and records management across the server and client. We were very aware customers had a wide variety of ECM applications in place. So while we invested to be a leader, particularly breaking down many of the user experience and programmability barriers, we made sure SharePoint was an open platform and worked with vendors across the industry on a variety of integration approaches. One of the biggest sources of feedback in 2003 was the relationship between SPS and our Content Management Server (CMS) 2002 web publishing product which was used by a lot of internet sites. The teams came together and proposed a shared experience and architecture that was one of the highlights for 2007. You can see many of the internet sites that used these features by navigating from a growing number of 3^rd-party blogs and sites likes www.topsharepoint.com. This was another good example of the power of a shared long-term vision. We had a lot of pressure to do something more tactical but we again decided to focus on the right solution. Complementing content management, we increased our search depth with a focus on new relevance algorithms and innovative people and business data search. Finally, to enable customers to build business process integration and business intelligence portals, we added Excel Services and InfoPath Forms Services. Besides being exciting features, we gained invaluable learning for the team how to have an architecture that worked in both the rich Office client and on a web server with consistency, fidelity, round tripping, etc. We have expanded on this and our experience with Outlook + Outlook Web Access for the new set of read-write Office Web Apps as part of the 2010 wave. While customers appreciated the depth and management knobs, after this release we learned we needed to invest more in content and readiness (training our employees, customers and partners) and it took us a while to catch-up with the demand of 2007. This blog was a big part of that investment that I will discuss below. We are very focused on addressing technical readiness for the 2010 release and the depth you will see around the SharePoint Conference this month with over 240 sessions and 40 hands-on labs will be a big step towards this. We are working on deep content and training and will have thousands of people in Microsoft and partners ready by launch.

Supporting SharePoint 2007

If you have been a frequent reader of this blog, you know notice we have spent very few of the hundreds of posts talking about SharePoint 2010. This is for two reasons. First, we did not want to start talking about the next release until it was solid and we were near something actionable (e.g. getting educated on the product, try out the beta). Second, we want to keep the focus of Microsoft and industry on great SharePoint 2007 deployments. Since the release we have made a number of investments our team has talked about in this blog. I will highlight just a few things because many have been posted about already:

- Enhancing Technical Content and Guidance. Since the release of 2007, we doubled the size of the SharePoint documentation team and increased the content on TechNet, MSDN and this blog. We created a virtual team, called the SharePoint Content Partnership Council, spanning R&D, documentation, consulting, support and product management that reviews your feedback and prioritizes the content to be written or improved. We have focused on the key areas you have told us about such as interoperability and governance. We also evangelized a lot of consultants and partners to post “real world” experience to the blog since know that is very important to you.

- Increasing Microsoft and Partner Staffing and Training. We were bullish on the adoption of SharePoint 2007 but frankly we underestimated the demand and didn’t have enough trained Microsoft staff and partners at launch. Since then we expanded our internal support and consultant staff significantly and introduced training programs reaching thousands of people outside of Microsoft. We introduced a new multi-level certification including a “Master” designation and a packaged offering of 5, 10 and 15 day planning sessions for customers called SharePoint Deployment Services that over a thousand organizations have used. This has been a great source of feedback. We have already started the process of training Microsoft employees on SharePoint 2010 and the SharePoint Conference starts the process for the broader industry. Much more to come that you will hear about in this blog in the coming weeks.

- Creating a Customer Assistance Team (CAT) – In the past, Program Managers in our development team worked with our consultants, customers and partners to get feedback and publish best practices. In 2003, we created an elite team of experienced consultants called the SharePoint Center of Excellence (COE) to drive this within our consulting organization and partners and publish this externally. After the 2007 release, we went a step further creating the CAT team within R&D that lived and breathed the most demanding deployments to make sure our content on topics like capacity planning or disaster recovery was deep. They were a keep part of the planning and design for 2010.

Enhancing SharePoint 2007

One topic some people ask us about is responding to a wide range of requests for new features and solutions. The approach we take for the core SharePoint product is to optimize on bringing the most functionality we can together in a consistent and reliable product we can deliver and predictable release cadence our customers and partners can absorb. However, we know there are places people need additional solutions and I expect some of you might be surprised by the investments we have made so I thought I would walk through them below.

- Cumulative Updates – We heard your feedback that you want something more convenient than hot fixes and more frequent than Service Packs so we introduced Cumulative Updates for SharePoint and Office. This was a big change for our development, testing and release process. It wasn’t all smooth at the beginning and we heard your feedback on this blog about simplifying these and my sense is the feedback is now very positive but we continue to tweak this.

- SharePoint Administration Toolkit – We created a team to address feedback on IT operations requests and release supported tools that we have posted on this blog. Unlike core features, these are more straightforward to release because they don’t invalidate the underlying code nor create a wide mix of configurations for us to test. We are continuing to invest here.

- Toolkits and Codeplex – As SharePoint is mission critical, you asked us to make sure the base product is robust, performant, secure, localized, upgradable, documented, supported, etc. Not surprisingly, this is expensive and causes us not to do as many features as if we had a lower release bar. Some of you are fine with that. Others have told us you would like some starter solutions to save you from writing value-added code yourselves. Hence, we have invested in a broad range of kits for SharePoint (Silverlight, Community, Site Templates, Patterns and Practices, etc.) and fostered a vibrant open source community on Codeplex while setting expectations these are not packaged products. I just checked and there are now 876 projects for SharePoint. We know these are in various stages of quality and adoption and unfortunately we can’t bless them but do think it is important to foster the community and listen to which ones you find valuable.

- SharePoint Partner Solutions and Services – While we aspire to provide the most comprehensive foundation in the industry, customers have asked us for a specific range of solutions and services so we are eager to support our 4000+ partners around the world building on top of SharePoint. They have the expertise, focus and reach to meet a large and growing range of needs. We also strive to make SharePoint a great business opportunity to foster the investment and innovation in these firms. We want them to see SharePoint 2010 and well beyond as their best business opportunity.

- Search Investments – We heard that great search is a critical part of a successful SharePoint deployment. We also see highly scalable and flexible search as an enabling technology for innovation in smarter portals, content management, social networking, business intelligence and more in the future. Therefore we did two things following the “good, better, best” approach that people have liked with SharePoint. First, we released a free version of SharePoint Search named Search Server Express. Second, we acquired the leading enterprise search firm, FAST. FAST’s ESP is the most sophisticated and flexible technology serving the world’s largest publishing, media, commerce and telecommunications sites on the internet as well as many of the most demanding internal search applications. We continue to invest in both standalone products and products optimized for SharePoint (ESP for SharePoint). Over the last year and a half the combined Microsoft Enterprise Search Group has come together and will deliver ltos of innovation in user experience, relevance, connectivity in the 2010 release. We will continue to invest in standalone search technology and have some exciting work coming in the next year there as well.

- SharePoint Online – Building on SharePoint 2007, we released both Dedicated (single tenant) and Standard (multi-tenant) releases for SharePoint Online. This is all part of our Business Productivity Online Suite including SharePoint Online, Exchange Online, Office Communications Online and Live Meeting. The response to both the dedicated and multi-tenant offering has been outstanding. Customers have told us they like getting access to the most comprehensive and flexible set of collaboration and communication tools with the reliability, security and manageability they need. Large organizations such as Coca Cola Enterprise have adopted SharePoint Online to give them more agility and free up their IT staff to partner with their business users on more critical activities than configuring server farms. It also lets us reach new business who previously were not able to deploy collaboration technologies because of lack of internal IT expertise. SharePoint Online also lets our partners work with us on a recurring revenue stream and focus their business on higher value and profitable services. Finally, because Microsoft does the operations and incurs the cost of running SharePoint Online it is an incredible feedback loop at high scale that helps drive product and documentation improvements that we will share with customers running their own servers. As I will cover in the next blog post, the development of SharePoint 2010 was very much informed by our experience during the last few years on SharePoint Online.

So as you can see the team has been very busy enhancing SharePoint 2007 and not just working on 2010.

Lessons Learned

To sum up our lessons learned from the first decade of SharePoint that we will apply to our next ten years:

- Customers come first. There is a lot of exciting technology in SharePoint but the objective is not a building a computer science project. What matters is solving business problems. We love to see your sites so keep the case studies coming!

- Long-term vision and commitment. Business is faster paced than ever and we and you need to continue to adapt. At the same time, success usually only comes from a clear long-term vision, commitment and feedback loop. I think that has been a key to your response to SharePoint as we discussed above.

- Balance of both innovation and execution. This is something we stress in the Office R&D group. Sometimes it means we incubate new technologies like we have done in the history of SharePoint. Sometimes in means we make bets where we believe we can improve the customer experience like MySites in SharePoint 2003 and the Ribbon in Office 2007. At the same time, shipping a product and service as comprehensive as SharePoint Server and SharePoint Online is a complex engineering project and you are depending on us to do it with rigor and quality.

- Viral and top down adoption. As hopefully you have seen our approach for SharePoint dating back to the first release is to strike the balance between empowerment and governance so people can be more productive while the organization can manage its knowledge, security and costs.We will continue to focus on this balance and let you dial the knob where is appropriate for your organization.

- Feedback loop. We need to continue to listen to your feedback about what we’ve done well and where we can improve. Even if we can’t address the feedback immediately, we must always listen, analyze and decide how we can best address it. We have a number of approaches for this in Office and SharePoint that I will cover in the next post but we welcome your feedback in this blog and at the upcoming events.

Thanks for your support and for reading this long. I hope it was interesting. I will post in a week about the development process we used for SharePoint 2010.

Jeff

Jeff Teper – Corporate Vice President, SharePoint Server, Microsoft

Install SharePoint Server 2007 on Windows Server 2008 R2

Starting from Service Pack 2, Windows SharePoint Services 3.0 and SharePoint Server 2007 support Windows Server 2008 R2 and Windows Server 2008 SP2. When you try to install SharePoint bits on Windows Server 2008 R2 directly, you may see the following dialogue:

This is because Windows SharePoint Services 3.0 and SharePoint Server 2007 bits without SP2 slipstreamed are not supported on Windows Server 2008 R2. The KB article 962935 is not live on the web site yet.

To install on Windows Server 2008 R2, for Windows SharePoint Services 3.0 you can download the slipstream builds here:

Windows SharePoint Services 3.0 with SP2 (x86)
http://www.microsoft.com/downloads/details.aspx?FamilyId=EF93E453-75F1-45DF-8C6F-4565E8549C2A&displaylang=en

Windows SharePoint Services 3.0 with SP2 (x64)
http://www.microsoft.com/downloads/details.aspx?familyid=9FB41E51-CB03-4B47-B89A-396786492CBA&displaylang=en

For SharePoint Server 2007, you can follow Create an installation source that includes software updates (Office SharePoint Server 2007) to create one. Or you can also read on, we will go through the complete steps to create a new slipstream build for SharePoint Server 2007.

Installation Steps

1. Copy the content of SharePoint Server 2007 setup files from the installation media to a folder on your hard drive.

2. Delete everything inside Updates folder.

3. Download Windows SharePoint Services 3.0 SP2 and Office SharePoint Server 2007 SP2 to a folder.

Make sure your Office SharePoint Server 2007 SP2 is downloaded after July 29^th.

4. Open a command prompt,  change directory to the folder you put the downloaded patches, and run the following two commands:

wssv3sp2-kb953338-x64-fullfile-en-us.exe /extract:[Path to installation bits]\Updates /quiet

officeserver2007sp2-kb953334-x64-fullfile-en-us.exe /extract: [Path to installation bits]\Updates /quiet

Change [Path to installation bits] to where you put the bits. These will extract all the content from the two packages to Updates folder. SharePoint installation program will automatically read this folder to apply the patches.

5. Delete wsssetup.dll. This is a very important step so please don’t miss it.

6. If you also need the Cumulative Updates to be applied when install SharePoint, download the latest Windows SharePoint Services 3.0 and SharePoint Server 2007 Cumulative Update packages and extract them into Updates folder like step 4.

7. Your slipstream build of SharePoint Server 2007 is done!

8. Go and install it on your Windows Server 2008 R2 box, after the installation, the site version will show 12.0.0.6421 or possibly a higher version if you added additional cumulative update files.

Jie Li
Technical Product Manager, SharePoint

How we did it: Speeding up SharePoint.Microsoft.com

SharePoint.Microsoft.com is a critical communications channel for SharePoint. We wanted to provide visitors with a rich and helpful experience and at the same time educate them on what SharePoint can do for their businesses. The site leverages our industry-leading web content management platform, and provides useful content and resources to the visitors. In addition, we also recognized that there were fundamental Internet challenges that needed to be overcome in order to improve page loading time, especially for our global visitors. We teamed up with Aptimize Software to optimize SharePoint.Microsoft.com for high performance.

Today’s guest post is by Ed Robinson, Chief Executive Officer of Aptimize Software. This post will review how we did it and the underlying performance techniques that improved the page loading speed of the SharePoint marketing website. We hope you can leverage the same process to optimize performance for other websites.

Tony Tai

SharePoint Senior Product Manager

Introduction

The Performance challenge

As designers and UI specialists all over the world continue to push the boundaries of the web experience, page payloads continue to rise. 15 years ago, web designers challenged themselves to ensure pages were under 15KB. Fast forward to 2009 and it’s not uncommon to find landing pages that run 500KB and up. Thankfully bandwidth availability has grown rapidly alongside this evolution… or has it? For many Internet users around the world, bandwidth is still a challenge and even if you address bandwidth limitations, there are physical limitations as data travels around the world or bounces off satellites. As a result, sites targeting a global audience, regardless of platform or technology, need to consider the impact of low bandwidth, high latency internet connections.

The Solution

There are many strategies to address website performance, most of which have focused on server side processing, client side caching, and edge caching. SharePoint developers can take advantage of each of these with SharePoint’s scalable server processing, utilization of web browsers built-in object caching, and integration with caching offered by content delivery networks. Windows 2008 R2 and Windows 7 together offer even more capabilities – with caching of head office files locally to branch offices for faster access by Windows 7 PCs within the branch offices.

While all these practices are encouraged, they don’t work to reduce the number of round trips made between the browser and the web server and they don’t take steps to dynamically optimize the HTML, JavaScript, CSS or image output from the server.

Of course, with a lot of manual processing and continuous updates, it is possible to take the work done by developers and designers to minimize and consolidate this content and thus reduce the overall number of round trips and the size of the page payload. Ideally, there should be a tool to dynamically optimize page payload, leaving developers and designers should be left to carry out high value work, producing easy to read, well commented HTML, CSS and JavaScript along with easy to manipulate images.

So why are we telling you this? – SharePoint is the fastest growing server product in Microsoft’s history and as a result, there is a huge volume of traffic coming to our website everyday with requests from all over the world. We built our site to showcase the best of what SharePoint and Silverlight can achieve on the web and so like other rich, dynamic sites, we’ve pushed the boundaries of design, resulting in a higher page payload. We’ve certainly done a lot of work to optimize server processing, but rather than accept that some visitors with lower bandwidth and higher latency would have a sub optimal experience, we went in search of technology to improve the experience for these users. We found a great partner in Aptimize that have a solution specifically designed to address the bandwidth and latency issue and they have worked on their product to ensure that it integrates smoothly with SharePoint.

Now let’s take a look at how Aptimize works, how easy it was to setup up and configure on SharePoint, and the results we experienced. Keep in mind though, Aptimize isn’t restricted to SharePoint so whether you’ve joined the ranks of Ferrari, Kraft, AMD and others, or you have a custom .net or PHP site, you can see the same benefits

This article walks through the three-step-process for analyzing and improving performance using frontend optimization techniques with Aptimize’s Website Accelerator.

Step 1: Understand the current performance

A realistic client performance goal for every website is: pages that load in under five seconds. Five seconds for the server to process the page and deliver every piece of content to a browser anywhere in the world over a normal broadband connection.

So how did the original SharePoint.Microsoft.com stack up?

The first thing the team did was analyze and understand the current performance. Aptimize used the free WebPageTest tool to record the current load times from around the world: USA, United Kingdom and New Zealand. WebPageTest records the load time, and also produces a waterfall diagram showing how each element of the page loads.

When measuring page load times, three metrics are important:

First view is the time to load a page for the first time

Repeat view is the time to load a page once it has already been visited and the browser has it cached

Start Render is the length of time a user sees a blank page before it starts to render. This is important since it gives

The first view page load times ranged from about 10.6 to 15.3 seconds from around the world, and 6.1 – 9.4 seconds for repeat view.To see where the time is spent, WebPageTest also generates a waterfall diagram for the First View and Repeat View

This waterfall shows three stages

a) Server processing. When a browser requests a page, the webserver processes the request, and produces an HTML page. This is delivered in the first HTTP request is completed. After this, the server’s work is done. We look for a server processing time of less than 1.5 seconds – any longer than this indicates a server processing issue. For SharePoint.Microsot.com, the HTML page was returned in 473 milliseconds. This is fast, and indicates no server tuning is required.

b) Browser processing. After the browser loads the HTML file, it begins to render the HTML by starting at the top of the HTML file and processing each line of the HTML, drawing the page and loading the external JavaScript, StyleSheet and image files. The browser processing is 10.1 seconds, representing about 95% of the total page load time.

c) Post-load actions. After the page has loaded, the Silverlight controls begin to render the navigation and content for the main page, and the LivePerson service is activated. Although Silverlight can be optimized, this is outside the scope of this article.

Once the page has been loaded by the browser for the first time, some elements are cached on the local machine and a repeat view is typically faster since the browser doesn’t have to load the resources it has cached. Notice how the waterfall is shorter for the repeat view.

After recording the current performance, ruling out server problems, the website is ready for browser processing optimization.

Step 2: Optimize the website for speed

There is an established method for optimizing a website for speed using performance best practices.

Sever optimization

The load profile for SharePoint.Microsoft.com showed that the server, database and application itself were running well within expectations – SharePoint was able to produce the HTML page in under half a second which is better than most websites. No optimization was needed on the server.

Reduce HTTP requests

Browser processing, the time it takes to load every JavaScript, StyleSheet and image resource on the page contributed 95% to the load time. With 96 resources, this was a page that could benefit from optimization. Reducing the number of resources would result in server output that requires less round trips for a browser to load the page.

For a first view, the browser loads 96 files and resources

There are some simple, well established techniques for reducing the number of JavaScript, CSS and image resources:

· Merge JavaScript files into fewer files

· Merge the StyleSheet files into fewer files

· Reduce the number of images using CSS Sprites and CSS inlining

CSS sprites are a useful technique for reducing the number of images. Multiple individual images can be copied side by side into an image tile, and then referred to individually within the page, using an offset into the image tile.

Even though all the PNG images appear as individual images in the webpage, they are stored and loaded as one tile, which actually looks like this:

CSS inlining is another technique to reduce image counts – with CSS inlining small background image files are converted into a Base64 encoded stream and copied into the CSS StyleSheet itself.

After merging JavaScript files, Stylesheets and image files the number of resource files in the page was reduced by 64%

Increase caching

Configuring resources with far-future-expires further reduces the load time for repeat views. With far-future-expires, static resources (images, JavaScript, StyleSheets) are set with an “expires” cache header that instructs the browser to cache the resource for a year, without checking for updates. This greatly reduces the HTTP requests, since the browser doesn’t need to confirm it has the latest version of each resource.

Out-of-the-box, SharePoint sets the best caching for the resources shipped with the product, but can’t anticipate what people add to their own sites. Configuring every resource with far-future-expires creates problems if the resources change – the browser doesn’t check for a newer version, so the changed resource will never be downloaded.

With Aptimize Website Accelerator, we were able use far-future-expires to their fullest potential, setting far-future-expires for the static resources, and excluding dynamic resources (such as search results) that change with each search. If static resources change, Aptimize Website Accelerator automatically detects the change and alters the URL of the resource to force the browser to download an update.

This process reduces the HTTP requests, load time and page size for repeat views of the same page.

Reduce size

The final technique is to reduce file sizes. The JavaScript files and CSS StyleSheets contained whitespace and comments. While this represents sound coding practice and is critical to ensure that developers can understand and interact with these files, whitespace and comments aren’t necessary for browser render. As a result, whitespace and comments were removed.Gzip compression was already turned on in the original site which helps to reduce file sizes more. Images should not be Gzip compressed since JPEG, GIF and PNG images are already compressed as part of their file format.

Remove duplicates

Any duplicate JavaScript and Stylesheet files that were downloaded more than once were changed to a single reference during the merge process.

Manual or Automatic

While all the steps described above can be applied by manually altering the resource files, these techniques were applied automatically using the Aptimize Website Accelerator, a software product that can dynamically optimizes SharePoint pages for speed.

Step 3: Results

After making these changes, the team measured the load times again using WebPageTest from locations around the world.

The waterfall graph for first load shows a reduction in HTTP requests from 96 to 35

The repeat view waterfall also shows a reduction in HTTP requests from 50 to 9.

New page load times

First view page load times were reduced 46% to 64% with great improvements for people on high latency connections. Repeat view load times were reduced 15% - 53%, and start render time was reduced more than 50%. These benefits are primarily due to the reduction in the number of HTTP requests.

By reducing HTTP requests, increasing caching, and reducing file sizes, the SharePoint pages are now optimized for speed using best practices for website performance. Future optimizations are possible for the post-load actions by optimizing the Silverlight controls but the SharePoint pages themselves are now high performance. The installation and configuration process took only a few hours effort, and the results were immediate.

Ed Robinson – Chief Executive Officer, Aptimize Ltd

Conclusion

A faster website improves the visitor’s experience, and with some smart use of technology, we can continue to design and build sites in line with industry best practices, while dynamically addressing issues of low bandwidth and high latency. As a result, we can ensure that site visitors all over the world experience the richness of our site, without compromise!

Tony Tai – SharePoint Senior Product Manager, Microsoft Corporation

Patching a mirrored farm with minimal downtime

For administrators of large SharePoint server farms, a newly-published article describes how to update server farms that use database mirroring, while incurring minimal perceived downtime for users. This solution is for enterprise customers who have high-availability needs and infrastructure. The article describes a step-by-step process for Office SharePoint Server farms that incorporate SQL Server mirroring. You can achieve similar results on Office SharePoint Server farms that are set up in a clustered server environment.

In brief, the solution requires that you:

· Disable mirroring and set the mirror databases to read-only

· Route network and database traffic to the read-only databases

· Apply software updates to the offline servers

· Route network and database traffic to the updated servers

· Apply software updates to the secondary farm

· Re-enable mirroring and set the mirrored databases to read/write

To read the full solution, see Configure a server farm for minimal downtime during software updates

SharePoint Conference 2009 Update: We’re SELLING OUT FAST!

SharePoint Conference 2009 is selling out fast!  Register today to secure your spot before we sell out! If you’re in need of a little encouragement to hit the registration button then read on, we’ve got a new set of sneak peek session titles to share and an update on conference hotel availability and Post- Conference workshops.

Session Sneak Peek: 15 New Sessions! Check out the new sneak peek sessions below, and don’t forget, they’re just a small sampler of almost 250 sessions we have scheduled.  You can look forward to being the first to see, hear about and get hands-on with SharePoint 2010 during the conference!  There’s so much more to come! Understanding Office 2010 and the Office Web apps Office Web apps: Deployment and Manageability Customizing Office 2010 Backstage view and Ribbon What's New in Office 2010 for Developers  Deep-Dive into SharePoint 2010 My Sites and Social Networking Architecture SharePoint 2010 Governance: Planning and Implementation SQL Server Best Practices for SharePoint Deployments Overview of Access Services in SharePoint 2010 Introduction to SharePoint Applications Using InfoPath and Forms Services 2010 Launching and Supporting Large Global Sites:  Lessons Learned from AMD.com (Customer Session presented by AMD) How SharePoint Helped Employee Communications Do More with Less (Customer Session presented by Dow Jones) Planning, Deploying and Administrating Excel Services and Project "Gemini" in SharePoint Server 2010  Advanced Web Part Development in Visual Studio 2010 Enterprise Content Management for the Masses: How SharePoint 2010 Delivers on the Promise The 2010 Lineup: SKUs and Licensing

Discounted Hotel Room Rates: SOLD OUT!   Due to high demand in registration our discounted hotel room rates at The Hotel and Mandalay Bay Hotel, are SOLD OUT!  Luckily Las Vegas is a big place with plenty of hotel rooms, so if you’re looking for accommodation check out some of the deals at the neighboring properties like the Luxor, but act quickly!

Post Conference Workshops: SOLD OUT! The SharePoint Developer Deep Dive and SharePoint Server 2010 Installation and Upgrade post-conference workshops are both SOLD OUT!   If you’re interested in attending one of these post-conference workshops open to all registered SharePoint Conference 2009 attendees, please contact spc@microsoft.com to join the wait-list.

Already Registered? Why Not Tell Everyone!  We're excited you've decided to register for SharePoint Conference 2009 and you should be too! Here are some images you can use on your website and in your email signature to let others know you'll be attending.

For more information visit www.mssharepointconference.com.  We look forward to seeing you next month! The Microsoft SharePoint Conference 2009 Team

Announcing the release of patterns and practices Developing SharePoint Applications guidance

Patterns and practices has released a new version of guidance for building collaborative applications that extend your LOB systems.  The guidance helps customers accelerate constructing advanced applications through examples and documentation. The guidance delivers a guide, a reusable library, and a reference implementation for a partner portal extranet.  It compliments product documentation and helps architects and developers in the following areas:

· Use SharePoint capabilities to make more powerful applications

o Integrate information from Line of Business Systems

o Take advantage of publishing and content oriented capabilities

o Create collaborative interactions around business processes

o Design multi-site topologies with complex security and isolation needs, such as  a partner extranet

· Build applications that are easier to scale, maintain, and grow

o Improve maintainability, testability, and layering through patterns

o Use techniques to improve flexibility, diagnostics, operations and performance

o Show how to use SharePoint’s feature and solution framework

· Improve application quality through testing

o Demonstrates unit testing and integration testing SharePoint applications

o Describe experiences with acceptance testing SharePoint applications including stress and scale testing

· Improve and accelerate team productivity

o Accelerate adoption of recommended practices with library components

o Show how to build an effective team development environment

o Understand fundamental design and implementation decisions

Developing SharePoint Applications guidance integrates new guidance with the original release, SharePoint Guidance – November 2008, into a single download. The guidance contains the following components:

Visit http://www.microsoft.com/spg to view the release.

Microsoft Virtualization: Best Choice for SharePoint

Virtualization continues to be a hot topic with many customers recognizing the benefits of virtualizing SharePoint including reduced server hardware costs, power and space savings, improved server utilization and rapid server provisioning.  Additionally, by choosing MS Virtualization (hyper-v + System Center) customers  benefit from a lower cost solution (both up front and ongoing) that is already part of Windows Server and an integrated end to end management solution for both physical and virtual environments.  While the SharePoint team recommends MS Virtualization as the best choice for their customers, regardless of the hypervisor being used customers should consider specific deployment scenarios to determine whether they should virtualize or not (we have found that most customers find that a mixed physical/virtual environment is optimal).  For example, both the Web and Application roles are ideal candidates for virtualizing SharePoint allowing customer to easily provision additional servers for load balancing and fault tolerance (web role) and individual application use to adjust to resource requirements.  Other scenarios such as Production SharePoint farm with large Database and Index roles or Dedicated Index and Query role within a Sharepoint farm for high utilization should remain physical.  Detailed MS recommendations can be found here

For more information on virtualizing SharePoint  and other Microsoft server applications please visit http://www.microsoft.com/virtualization/solutions/business-critical-applications/default.mspx .  Also check out the latest MS virtualization blog here.

SharePoint 2010 Partner Training for SharePoint 2007 Partners!

SharePoint 2007 Partners: You’ve heard the buzz about SharePoint 2010 and seen the sneak peeks. Now learn how you can get ready to start building with and deploying SharePoint 2010!

SharePoint 2010 Ignite is a 5 day instructor-led training program for SharePoint 2007 partner developers and implementers coming soon to a city or virtually to a PC near you!

This is an invitation-only training event for partners and we are accepting nominations today.

Want to know more about Ignite and explore how you can participate? Visit the Ignite Page, review the pre-requisites and learn more at http://sharepoint.microsoft.com/topics/Pages/IgniteProgram4.aspx.