Janne Mattila's blog : Microsoft Office SharePoint Server 2007

Story of wrong content type and InfoPath publishing location

jannemattila — Wed, 29 Apr 2009 13:54:35 GMT

I found myself troubleshooting on issue that I thought shouldn’t be issue anymore. I have created InfoPath form and published it to the SharePoint. However when I created new form using that template it ended up using wrong content type at my library. And this happens even with the SP2 installed (both at the client and at the server) so I had to dig a little deeper.

Here are the phases that I took:

1. Created new InfoPath “Example form”.
2. Published it to the server called “myserver” as an site content type.
3. Added new content type to “mylist”.
4. Clicked “New example form” on the library and filled the form and saved it to the library.
5. Noticed that form is not the content type that I was expecting. It was default content type at the list... (which was Office document in my case since I had multiple content types at my list)

Did some testing and noticed that it actually works fine IF I use the “myserver” in the address bar of IE and not the fully qualified domain name “myserver.example.loc”. And now you’re thinking that I’ve just missed the Alternate Access Mappings (AAM) settings but no... I have those set for both addresses so that wasn’t the case.

Next I tried to fiddle around with InfoPath and tried different publishing addresses at the “Enter the location of your SharePoint or InfoPath Forms Services site:” –publishing wizard dialog:

And this is the place when it got interesting... Since I noticed that if I use “myserver” address at the publishing wizard dialog then content type goes wrong when using SharePoint through “myserver.example.loc” address. And also if I use the longer address at the publish wizard it doesn’t work at the SharePoint using the short address form.

So next I thought that Security Level could be the problem (which was set to Domain):

...but if that would be the case then I couldn’t even use the form. Now the form works normally expect it will store the saved item into wrong content type.

For me the case was closed. I just published the form in “long address format” and I was happy :-) But I still wonder what went wrong in the background...

Anyways... Happy hacking!

J

SQL Server Profiler and SPQuery

jannemattila — Sat, 21 Jun 2008 22:58:08 GMT

Question: I’m using SPQuery to retrieve items from my SharePoint list and now I want to know little bit more about SPQuery properties. Actually I’m interested about performance related things since I’m not sure about my query currently. I currently have following code to retrieve my data:

using (SPSite site = new SPSite("http://mysharepoint"))
{
  using (SPWeb web = site.RootWeb)
  {
    SPList docs = web.Lists["MyDocs"];
    SPQuery query = new SPQuery();
    query.Query = @"<Query>
        <Where>
         <And>
           <Eq>
            <FieldRef Name='MyField' />
            <Value Type='Text'>12345</Value>
           </Eq>
           <Eq>
            <FieldRef Name='ContentType' />
            <Value Type='Text'>MyContentType</Value>
           </Eq>
         </And>
        </Where>
      </Query>";

    SPListItemCollection items = docs.GetItems(query);
    foreach (SPListItem item in items)
    {
      Console.WriteLine("Item: " + item.Title);
    }
  }
}

Answer: To answer this question I need to step into dark side for a minute... I mean that I’m going to ask you to play with SQL Server Profiler to analyze your query more closely. This is of course to give some ideas how to tune your query using query properties and nothing else. I don’t want you to start use any custom SQL queries or other weird hacks that you might come up. Just tune the query and that’s it.
NOTE: Do this stuff only in you own dev box. Don’t even think anything else.

Before we start using the SQL Server Profiler you might want to stop SharePoint related services so that it’s easier to read the SQL trace. I’ve stopped IIS Admin (+related), Office SharePoint Server Search and Windows SharePoint Services*. Now you can start SQL Server Profiler and you’re ready to start tracing. Just create new trace (File->New Trace...) and select Tuning for your template:

And if I now run your query I’ll get something like 200+ rows in my trace window. But you’re probably just interested for the rows that tell what is going on. If you pause the trace after the query then it’s most likely the last line in the grid:

It gives you fairly long SQL (mine was ~4500 characters long) and it’s not trivial to understand what it is really doing.

But first you probably notice the interesting SELECT TOP 2147483648 ... And that’s the first thing you might want to change (of course you know best what you’re trying to achieve). But for that you have property called RowLimit. So if you add “query.RowLimit = 10;” to your code and re-run the trace you get following output:

So now you have SELECT TOP 11 ... which is better (note that the actual limit is your RowLimit + 1). If you only need to display n rows from database then don’t retrieve all and display just n => Use RowLimit to narrow down the resultset.

Next thing we need to take into consideration is ViewFields. If I know that I’m only going to show Title, then why should I need to retrieve more fields. Let’s narrow down the fields using ViewFields:

1 2	query.ViewFields = @"<FieldRef Name='ID'/> <FieldRef Name='Title'/>";

And again let’s test the trace and compare the query for the first trace. You may notice that the SQL is only ~3000 chars long anymore and amount of columns at the resultset is a lot less (screenshot from management studio when both queries have been executed):

Notice the size of scrollbar…. larger is better :-)

Now I have just added few lines of code and the result is ~1500 shorter SQL and we’re only getting the amount of rows from DB that we need to. I’m not going to play with another properties but if you’re interested you might want to start with Include* properties.

I showed small example that how can use SQL Server toolset to verify (at least for some extend) that your queries are going for the right direction.

Anyways... Happy hacking!

J

Link: SharePoint slow spin-up times…

jannemattila — Thu, 29 May 2008 18:14:50 GMT

I accidently found myself on this page: SharePoint 2007 Quirks - Solving painfully slow spin-up times. After reading that article I was forced to test that. And my stsadm experience was really fast after that! So this link deserves a link post even if I don’t do those normally :-)

Anyways... Happy hacking!

J

CRM 4.0 (or SharePoint or custom application) and DebugView

jannemattila — Wed, 07 May 2008 10:04:03 GMT

Every now and then I’m find myself trying to solve same issues over and over again :-) That’s why I found myself (again) using DebugView as my debugging assistant at remote box. If you don’t know what DebugView is then you should definitely try it out. I’m going to give you few ideas how you could use it at your applications. You can download the DebugView from Technet.

Why do I use DebugView? Well I want to get debug messages from running system BUT... I don’t want to write to EventLog or to File since it’s totally unnecessary to write all the messages all the time. I just want messages when I’m ready to observe the system. And DebugView is handy tool for that. Here is list of steps how I normally implement that kind of approach:
1) Create log class that implements logging/tracing/debugging (this is pretty much just wrapper to few simple method calls) (I recommend making this as singleton)
2) Use you log class in your application/solution
3) Use DebugView to follow the trace messages written by your log class

In 1) I mention that I recommend using singleton approach at the log class. This is important especially at the CRM 4.0 where you need to add default trace listener so that you’ll get the trace messages. Here is the code example for that:

1
2
3

using System.Diagnostics;
// ...
Trace.Listeners.Add(new DefaultTraceListener());

After you have added the default listener you can just use this oneliner:

1	Trace.WriteLine("My Trace: " + message);

Now your trace can be seen at the DebugView:

(here is CRM 4.0 where I have menu item that points to custom ASPX page and I’m using “My Trace” to see what’s happening in there)

DebugView has many built-in functionalities like “Save As” etc. that can help you on your debug/trace efforts. So I recommend that you learn to play around with it.

I also had the magic word “SharePoint” in my title. That is for the simple reason that this same story applies to SharePoint too. And don’t forget... Since System.Diagnostics.Trace is implemented at the .NET Framework this story also applies to all other applications which are built on top of that.

You might also want to read my previous post about tracing: CRM 4.0, SharePoint and ASP.NET Trace.

Anyways... Happy hacking!

J

Custom SharePoint application and System.IO.FileNotFoundException

jannemattila — Sat, 23 Feb 2008 23:55:32 GMT

Question: I have normal user access rights at SharePoint but I'm administrator of the physical SharePoint box. I don't have access rights to SharePoint databases but I still need to run few lines of code against SharePoint. I have created small console application but when I run it I get nasty System.IO.FileNotFoundException:

What can I do?

My answer: So you have used Microsoft.SharePoint.dll and probably you have something like this in your code: "using (SPSite site = new SPSite("..."))" and that causes the exception. Well that exception is actually pretty understandable since you're using API that connects directly to the SharePoint databases... and you mentioned that you don't have access rights to databases so it makes sense. And on top of that you have only standard user rights to SharePoint so we're at the dead end anyway.

That means that we need to change our approach a little bit. Of course the solution differs little bit what you're trying to do but I think that you have few options. First one is that you could use SharePoint web services to implement your functionality. I know that it's not the easiest way to go but it should to the trick. Another approach could be custom application page that does the trick. It is a bit "hacky" approach but if you need to "do something against SharePoint" that is something you could try. I'll give you example so that you'll get the idea what I mean.

First create your custom aspx page into layouts folder (C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\LAYOUTS\). I created MySharePointAdminTask.aspx to the root of layouts but you should add subfolder with correct naming (e.g. Company.Project.Functionality). I added following code to my page:

<%@ Page language="C#" %>
<%@ Import Namespace="Microsoft.SharePoint" %>
<%@ Import Namespace="Microsoft.SharePoint.WebControls" %>

<script runat="server">
 void Page_load(object sender, EventArgs e)
 {
  if (Request.IsLocal == true && Request.IsSecureConnection == true)
  {
    SPSecurity.RunWithElevatedPrivileges(delegate()
    {
      using (SPSite site = new SPSite(SPControl.GetContextSite(HttpContext.Current).ID))
      {
        using (SPWeb web = site.RootWeb)
        {
          // TODO: add your code here
          // Example:
          Response.Write(web.Title + " at url " + web.Url);
        }
      }
    });
  }
  else
  {
    Response.StatusCode = 404;
    Response.SuppressContent = true;
    Response.End();
  }
 }
</script>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 
 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
 
<html xmlns="http://www.w3.org/1999/xhtml" >
<head id="Head1" runat="server">
  <title>MySharePointAdminTask</title>
</head>
<body>
  <form id="form1" runat="server">
  <div>
    <!-- TODO: add your code here -->
  </div>
  </form>
</body>
</html>

I have some basic checks at line 8 to see that is this request local and is the connection secure but in order to make this even a bit safe you need to add more checks to it. If that line fails then everybody who knows the url can execute code with elevated privileges. So you need to be really careful if you want to use this kind of approach and using code behind class would be much better than this inline example. On lines 12 to 20 I'll just do my fancy SharePoint stuff (in this example I'll just write out title and url of the site).

On lines 25 to 27 I'll just make the page look like 404 page just in case some of our checks didn't pass. We only want to run this code by the local user. So let see what is the end result:

For the local user it looks like this (and the user has only normal SharePoint access rights):

For the remote user (even with System admin rights):

So this kind of approach is okay for small SharePoint operations but if you need to do more complex and long running operations then you need to reconsider your options. It's a bit hacky approach but it works :-)

Anyway... Happy hacking!

J

CRM 4.0, SharePoint and ASP.NET Trace

jannemattila — Sat, 23 Feb 2008 21:12:25 GMT

Sometimes I tend to forget how much stuff is built into .NET Framework. Framework gives you nice set of features that you can use without writing a single line of code. ASP.NET Trace is one of them. I know that it's nothing new but I think that it's still used mainly in custom ASP.NET applications. But if you work with products like CRM or SharePoint... you kind of forget that those applications are built on top of ASP.NET and you can still benefit from the features that are part of the framework. So let's refresh our memory so that we can use this feature to build better solutions.

See also good post regarding tracing SharePoint in here "Enabling Page Level Tracing For SharePoint ASPX Forms".

Okay let's first enable trace at the web.config (same changes apply for CRM and SharePoint):

<?xml version="1.0"?>
<configuration>
 <!-- ... -->
 <system.web>
  <!-- ... -->
  <trace requestLimit="100" enabled="true"/>
  <compilation debug="true" />
  <!-- ... -->
 </system.web>
 <!-- ... -->
</configuration>

So in order to enable the trace you just have to make sure that you have trace (line 6) and debug enabled (line 7). After that you're good to go and trace your application.

I just retrieved front page from my SharePoint (http://demo1) and then clicked url to custom application page. Then I typed in the trace url http://demo1/Trace.axd and checkout the results:

And in more detailed (I just clicked the Pages/Default.aspx):

And same thing works with CRM 4.0 too... but remember that it's unsupported to modify web.config in CRM so create back up copy so that you can restore the original file when you need to clean up your modifications.

In my CRM I opened url http://crmserver/Contoso to goto tenant named Contoso. After that I used my custom aspx page from ISV-folder. And then I checked the trace:

And if you look at more detailed view of the trace:

And you can easily see the amount of stuff that is put into the application state and especially for CRMWindowInfo_CacheKey. Using this technique you can find e.g performance issues in your code quite easily.

I actually used this method to debug some AJAX and web services issues and it worked really well because I could check out from the trace what has happened and when.

To summarize.... you should refresh your memory about things that you get straight from the platform. It can save lot of your time. At least it happened to me :-)

Anyways... Happy hacking!

J

Catching unhandled exceptions in SharePoint

jannemattila — Mon, 04 Feb 2008 19:13:14 GMT

If you have done some dev stuff with MOSS you have most likely seen this:

"An unexpected error has occurred. " is something that you probably don't want to see at your browser.... you want to have customized error page. In ASP.NET application you normally put Application_Error into you global.asax file. However in SharePoint that place has been taken by the product itself :-) So if you want to do customized approach then you can take HttpModule approach which I'm going to go through in this post.

So let's create our custom exception handler http module. Here's the code for that:

using System;
using System.Web;

public class MyExceptionHandler : IHttpModule
{
  public void Dispose()
  {
  }

  public void Init(HttpApplication context)
  {
    context.Error += new EventHandler(context_Error);
  }

  void context_Error(object sender, EventArgs e)
  {
    Exception[] unhandledExceptions = HttpContext.Current.AllErrors;

    foreach (Exception ex in unhandledExceptions)
    {
      // TODO: log your errors
    }

    HttpContext.Current.Server.ClearError();
    HttpContext.Current.Response.Clear();
    HttpContext.Current.Server.Transfer("/_layouts/MyCustomErrorPage.aspx");
  }
}

You can probably see from the code that I'll attach my code to the Error event and in my event I'll do some basic stuff and then transfer to my MyCustomErrorPage.aspx. I used Server.Transfer just because I want user to stay at the same url where exception happened. If I would use Response.Redirect it would "change" the url at the users browser. Same "change" would happen if your custom error page would be normal SharePoint publishing page (i.e. /Pages/MyCustomErrorPage.aspx). If the url stays the same the user can actually press F5 and retry the operation right away. Of course it can be bad thing too and you may want to redirect to another page to avoid creating the same exception all over again. I'll let you decide what you want :-) So do some testing and then decide what's good for you.

But one important thing to notice. You need to put your IHttpModule before SharePoint specific modules in your web.config or otherwise your error routines may not work as you would expect. Here's example from that:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration>
 <!-- ... -->
  <httpModules>
   <clear />
   <add name="MyExceptionHandler" type="MyExceptionHandler,Example, 
      Version=1.0.0.0, Culture=neutral,PublicKeyToken=34a2bd01f6f6eb10" />
   <add name="SPRequest" type="Microsoft.SharePoint.ApplicationRuntime.SPRequestModule, 
      Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
   <add name="OutputCache" type="System.Web.Caching.OutputCacheModule" />
   <!-- ... -->

  </configuration

See line 6 where I put my exception handler definition.

Anyways... Happy hacking!

J

Create Silverlight 1.1 applications for SharePoint with VS2008

jannemattila — Thu, 20 Dec 2007 00:26:56 GMT

Title says it all :-) My plan is to create small walkthrough how you can create your own Silverlight 1.1 application with Visual Studio 2008 and then run it in SharePoint. And I don't want to install a lot of stuff to my SharePoint environment so I'll just create simple web part to host the Silverlight application. All development work will be done outside SharePoint environment.

If you're new to Silverlight then I suggest you start by looking at least silverlight.net and cool Coding4Fun Silverlight 8-Ball example. The first link is starting point for Silverlight development and the second one is really cool sample that shows how can you create 8-Ball game with Silverlight.

Now you know something about Silverlight :-) So we can get started with my example.

I have VS2008 running on my Vista and I have downloaded and installed Microsoft Silverlight 1.1 Tools Alpha for Visual Studio 2008 to help me developing Silverlight applications. It has new project template that has all the necessary stuff already set up to build Silverlight 1.1 application. I also have SharePoint running in my Virtual PC and it's configured so that I can connect to it directly from my host machine (i.e. http://demo1:1000 in my host IE connects to the virtualized SharePoint).

At first I'll modify SharePoint so that I can host my new applications in there. Let's add few folders under _LAYOUTS:

ClientBin is folder that will contain all Silverlight client .dll files (=if you build your Silverlight project you automatically get ClientBin under your project folder which contains your projects client assemblies).

Silverlight folder is used to store .xaml files (=Silverlight pages). I also need to change settings for both of them in order make things work. First I'll change ClientBin settings:

Set only Read access and also set Execute permissions to None:

Then I'll add .dll to allowed mime types (and .pdb for future debugging purposes :-):

Then I'll change the Silverlight folders settings to support .xaml extension:

Now SharePoint is set up... but let's create web part that will actually host the Silverlight application in our page. Here's the code for that:

using System;
using System.Runtime.InteropServices;
using System.Web.UI;
using System.Web.UI.WebControls.WebParts;

namespace Silverlight_Web_Part
{
  [Guid("fc852641-9955-433c-945e-d773f46351eb")]
  public class Silverlight_Web_Part : WebPart
  {
    string xamlFile;
    [WebBrowsable(true), Personalizable(true)]
    public string XamlFile
    {
      get { return xamlFile; }
      set { xamlFile = value; }
    }

    public Silverlight_Web_Part()
    {
      this.ExportMode = WebPartExportMode.All;
    }

    protected override void Render(HtmlTextWriter writer)
    {
      if (string.IsNullOrEmpty(this.XamlFile) == false)
      {
        writer.Write("<div id='SilverlightControlHost' " + 
          "style='width: 640px; height: 480px;'></div>");
        writer.Write("<script type='text/javascript' " +
            "language='JavaScript' src='/_layouts/Silverlight.js'></script>");
        writer.Write("<script type='text/javascript' " +
          "language='JavaScript' src='/_layouts/SilverlightStarter.js'></script>");
        writer.Write("<script type='text/javascript' " +
          "language='JavaScript'>createSilverlight('" + xamlFile + "');</script>");
      }
      else
      {
        writer.Write("Xaml file is not selected.");
      }
    }
  }
}

My web part uses Silverlight.js and SilverlightStarter.js files. SilverLight.js is the same file that is included at the project when you create Silverlight 1.1 project in VS. SilverlightStarter.js is almost copy paste from that project template too but I have just modified it a little bit to support sending XAML file as parameter:

function createSilverlight(filename)
{
  Silverlight.createObjectEx({
    source: filename,
    parentElement: document.getElementById("SilverlightControlHost"),
    id: "SilverlightControl",
    properties: {
      width: "100%",
      height: "100%",
      version: "1.1",
      enableHtmlAccess: "true"
    },
    events: {}
  });
   
  document.body.onload = function() {
   var silverlightControl = document.getElementById('SilverlightControl');
   if (silverlightControl)
   silverlightControl.focus();
  }
}

And if you checkout my web part code on line 35 you actually notice that I'll just pass on the XAML file that has been set in web part properties:

Now we just need to add some Silverlight stuff and we're done. So here is the content of the Page.xaml (=THE application :-):

<Canvas x:Name="parentCanvas"
    xmlns="http://schemas.microsoft.com/client/2007" 
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" 
    Loaded="Page_Loaded" 
    x:Class="MySilverLight.Page;assembly=/_LAYOUTS/ClientBin/MySilverLight.dll"
    Width="640"
    Height="480"
    Background="White"    
    >

 <Rectangle x:Name="MyRectangle" 
       Canvas.Top="25" Canvas.Left="25" 
       Width="50" Height="50" 
       Fill="Gray" MouseLeftButtonDown="MyMouseLeftButtonDown" />
 <TextBlock x:Name="MyText" 
       Text="Click either rectangle or me" 
       MouseLeftButtonDown="MyMouseLeftButtonDown" />
</Canvas>

And my code behind Page.xaml.cs contains following code:

using System;
using System.Windows.Controls;

namespace MySilverLight
{
  public partial class Page : Canvas
  {
    DateTime started = DateTime.Now;

    public void Page_Loaded(object o, EventArgs e)
    {
      // Required to initialize variables
      InitializeComponent();
    }

    public void MyMouseLeftButtonDown(object o, EventArgs e)
    {
      string runningTime = (DateTime.Now - started).TotalSeconds.ToString("F2");
      System.Windows.Shapes.Rectangle rectangle = o as System.Windows.Shapes.Rectangle;
      if (rectangle != null)
      {
        MyRectangle.Opacity = 1;
        MyText.Text = "You clicked rectangle! Application has been running for " + 
          runningTime + " seconds.";
      }

      System.Windows.Controls.TextBlock textBlock = o as System.Windows.Controls.TextBlock;
      if (textBlock != null)
      {
        MyRectangle.Opacity = MyRectangle.Opacity * 0.8;
        MyText.Text = "You clicked text block! Rectangle is fading away...";
      }
    }
  }
}

Now you're probably eager to know what our application actually does :-) Here is our application just after request:

And if user clicks the rectangle...

If user clicks the text...

Now we have working development environment so that we could build ant test Silverlight 1.1 applications and host them in SharePoint. Here are the steps so that you can (try :-) to do it yourself.

1) Create new Silverlight project in VS2008
2) Code :-)
3) Build your solution
4) Copy contents of ClientBin\*.* to the SharePoint _LAYOUTS\ClientBin folder
5) Copy *.xaml files from your project to the SharePoint Silverlight folder (Note: you need to verify correct path in x:Class! See my Page.xaml and line 5 in source)
6) Create page that uses Silverlight web part and select your newly created XAML file

Here was my small intro to Silverlight 1.1 with SharePoint. In this example I just used static .xaml files sitting in file system in order to make everything as simple as possible... I just wanted to demo how you can start experimenting with this combination. This can be extended and improved in many different ways and I'll probably come back to those in my future posts.

Anyways... Happy hacking!

J

External datasource (SharePoint list) and Forms Server

jannemattila — Tue, 11 Dec 2007 13:32:44 GMT

If you have following setup...

You need to create InfoPath Form that you're going to use in Forms Server (=Browser compatible)
Your form contains external data source to SharePoint list
Data from the list needs to be in certain order in your form (in drop-down list box)

...you'll have issue because your items will be sorted in the order which they have been created. Underlying reason for that is the fact that Forms Server uses SharePoint object model to retrieve the items. So if you need other sort criteria then I suggest you to create your own custom Web Service that then returns the list in correct order. This is actually good in architectural sense since this gives you extra layer for business logic. And I really recommend you do that. See my previous posts for more information: From InfoPath to Database via Web Service and Adding own custom web service to SharePoint.

Anyways... Happy hacking!

J

Use dynamically created InfoPath form instance to pre-fill form

jannemattila — Thu, 08 Nov 2007 01:05:41 GMT

If you want to pre-fill InfoPath form before user gets his/her hands to it... you have two ways to do that:

Call Web Services from InfoPath

It normally takes quite long

Dynamically create InfoPath form instance

Fast and there isn't any limitations where to get data

Both methods have been mentioned many times in different sites but I still decided to create a demo about the "Dynamically create InfoPath form instance" to make this crystal clear... and give some code to get you started on this if you're interested.

So I created really simple InfoPath Request Form:

I added submit rule to the button (I just used fixed url for the form library).

Then I created following UI for my solution:

Here is the code behind the astonishing UI (a.k.a. button inside web part):

using System;
using System.Runtime.InteropServices;
using System.Web.UI;
using System.Web.UI.WebControls.WebParts;
using Microsoft.SharePoint;
using System.Web.UI.WebControls;

namespace InfoPath_Creator
{
  [Guid("16ba4e60-d230-45d3-9966-da959d20a90b")]
  public class InfoPath_Creator : System.Web.UI.WebControls.WebParts.WebPart
  {
    public InfoPath_Creator()
    {
      this.ExportMode = WebPartExportMode.All;
    }

    protected override void CreateChildControls()
    {
      Controls.Clear();
      base.CreateChildControls();
      this.ChildControlsCreated = true;

      Button createRequestForm = new Button();
      createRequestForm.Text = "Create request";
      string url = this.Page.Request.Url.ToString();
      if (url.IndexOf('?') == -1)
      {
        url += "?";
      }
      else
      {
        url += "&";
      }
      url += "CreateRequest=true";

      createRequestForm.OnClientClick = "document.location.href='" + url + "'; return false;";
      this.Controls.Add(createRequestForm);
    }

    protected override void Render(HtmlTextWriter writer)
    {
      this.RenderChildren(writer);

      if (string.IsNullOrEmpty(this.Page.Request["CreateRequest"]) == false)
      {
        string requestType = "Request for dynamic request forms";
        string requestTitle = "My dynamically created request";
        string estimatedCost = "123";
        string user = SPContext.Current.Web.CurrentUser.Name;
        string costCenter = "999888777"; // TODO: get cost center from UserProfile
        int classification = 54321;

        string xml = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
          "<?mso-infoPathSolution solutionVersion=\"1.0.0.8\" productVersion=\"12.0.0\" " + 
          "PIVersion=\"1.0.0.0\" href=\"" + 
          "http://demo1:1000/PressReleases/Request%20forms/Forms/template.xsn\" " +
          "name=\"urn:schemas-microsoft-com:office:infopath:" +
          "Request-forms:-myXSD-2007-11-07T19-32-12\" ?>" +
          "<?mso-application progid=\"InfoPath.Document\" " +
          "versionProgid=\"InfoPath.Document.2\"?>" + 
          "<my:myFields xmlns:my=\"" +
          "http://schemas.microsoft.com/office/infopath/2003/myXSD/2007-11-07T19:32:12\" " +
          "xml:lang=\"en-us\">" +
          // Here I fill the fields:
          "<my:RequestType>" + requestType + "</my:RequestType>" +
          "<my:RequestTitle>" + requestTitle + "</my:RequestTitle>" +
          "<my:EstimatedCost>" + estimatedCost + "</my:EstimatedCost>" + 
          "<my:User>" + user+ "</my:User>" + 
          "<my:CostCenter>" + costCenter + "</my:CostCenter>" + 
          "<my:Classification xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">" + 
          classification + "</my:Classification>" +
          "</my:myFields>";
        this.Page.Response.Clear();
        this.Page.Response.AppendHeader("Content-Disposition", "attachment;filename=Request-" + 
          DateTime.Now.ToString("yyyy-MM-dd-HH-mm-ss") );
        this.Page.Response.ContentType = "application/ms-infopath.xml";
        this.Page.Response.Write(xml);
        this.Page.Response.Flush();
        this.Page.Response.End();
      }
    }
  }
}

In lines 54 to 73 I created the XML for the form (it looks like real mess and I'm sorry about it!). Most important stuff happens between lines 66 and 72 where the actual field values are stored. Of course you could use some magic code to get the correct values to the variable at lines 47 - 52. And you can also probably see the URL of the XSN file (line 57) and as well as the namespace (line 63). It looks like this when user clicks the Create request button:

And when user presses the button in InfoPath the form data is submitted to the form library:

So hopefully after this post you'll remember that this method is powerful (and not the messy XML :-) if you want fill InfoPath with data. You have much more possibilities than just simple web service request in InfoPath.

By the way... Why do I always end up with titles with word "dynamic*" in it :-) (see my previous post for example)

Anyways... Happy hacking!

J

Dynamically create static Excel files for Excel Service

jannemattila — Tue, 06 Nov 2007 20:07:50 GMT

Excel Services is pretty nice for displaying Excel files over the network. But if you're developing Excel Services solution for the first time you might first think of the limitations or something that forces you to build your overall solution in certain way. So I thought I'll write little bit about my idea about creating dynamically static Excel files. This might be something that you could be interested in... or not :-) It of course depends on the solution that you need to build. I just want to give you few ideas that you can use in your own projects. Okay here we go!

We want...

...to use 2 different languages (Finnish and English)
...have static texts in Excel that needs to be translatable
...use pivottables
...export data into .xlsx and to .csv (and possibly to some other formats as well)
- And the end user cannot see any formulas in their exported files
...filter the data from UI
- Translation for that: We need to pass parameters to the database query
...have easily maintainable system (=minimize the amount of Excels)
...flexible solution. So if we later want more complex scenarios this needs to be supported in your solution.
...performance performance performance (but no extra costs!)
- But we only update the incoming data quite rarely

Okay... list is quite long and I need to discuss a little bit about those demands.

1 to 3) Supporting more than one language can be issue if your data contains texts that needs to be translated. Also pivottable column names, captions and total captions needs to be translated. So translating static texts is easy compared to that :-)

4) Exporting to .xlsx is easy. Just use Excel Services API and retrieve "snapshot" and you're good to go. But if you want to convert it to other formats then you need to do some extra work.

5) You can use filtering but if you think of 8) you just can't do filtering of 2 million rows if you just want to view 50 rows... that would kill the performance. So since you need to think of performance you probably want to pass parameters to your database queries. If plan to get your relational data into your Excel using Office Database Connection (ODC) you cannot unfortunately use parameters since querytext will be "hardcoded" into the ODC. Of course you can achieve that with multiple ODC files but it could create mess. You can store those in Data Connection Library (DCL) to ease up the maintenance pain but still it's quite challenging to do that (=my personal opinion). And if you then think of the 6) you don't want to create maintenance hell. And of course if it gets too complicated you would most likely have issues with ODC approach. But if you use OLAP you could manage with only one ODC.

6) You don't want to have 1,5 million different files and then update them manually? Okay... I'll get the picture.

7) I know that predicting future is hard so let's create solution that is flexible enough so that you can extend it in many ways. I don't want to give limitations to your future needs.

8) So you don't want to buy 5 new servers with lot of processors? What about memory then... it's cheap? So that's okay... let's try to create as static files as we only can. This way we can use "memory over CPU" approach as much as possible. And creating stuff in cache before users are going to use the system would be nice.

I'm ready to go to next phase and show you my example that I have created.

Implementation of my solution

Phases:

Create database for the example
Create template Excel file
- This is then used to create the "static copy instance"
Create SharePoint structure for the demo
Create custom web part that hosts EWA and fills the other requirements as well
- Exports to different file formats
- Filter the query from UI
- Translate texts that are needed in order to get the file in users native language

1. Create database for the example

In my example I'll use legendary AdventureWorks database :-) (I know that for some developers this itself will cause some hatred towards me ;-)

I did minor changes to the AdventureWorks data since I want my demo to support English and Finnish. I modified French culture to be Finnish culture and then modified the texts in description to be example same as the name (so that I'll can demonstrate the translations in database layer) (Note: I didnt' change the ProductModel translation since I want to "translate" it in my code).

My plans is to demonstrate product data. Here's an example of the data:

From that data you can easily see that I'm going to use culture info to get the correct data to my Excel. Of course you could create your own SQL/stored procs to handle translations as you wish.

2. Create template Excel

Now I'm going to create Excel that uses previously modified data. Since my plan is to dynamically fill the Excel with data I'm just going to create "almost empty template" that will be placeholder for the real data. Here are screenshots from my MyEWA.xlsx Excel file:

Display-sheet has static text in A2 and then it has Pivottable that retrieves data from the Data. If you're really sharp you probably noticed that row 5 is missing from the Display-sheet... I'll come to that later.

Data-sheet contains Header row and then it contains 1 data row. This data row can been seen from the pivottable in the Display-sheet.

Third sheet is Parameters and in this example I'll only use it to pass Culture to the Excel:

So if we now take closer look at the Display-sheet and see the formula in cell A2:

A2 cell contains following formula:

=IF(Parameters!B1="en-US";"Here is report about products.";"Tässä on raportti tuotteista")

It's pretty easy to understand that if the B1 cell in Parameters-sheet is set to en-US then the text will be "Here is report about products". and if it isn't then "Tässä on raportti tuotteista" text will appear in the cell (latter text is same as the English one but in Finnish :-). By now probably everybody knows already that we're going to change the value in Parameters!B1 dynamically... and use it to translate the static texts inside Excel into correct language.

This same can be achieved if you dynamically copy text over specific cells. I have example of that in my code but it's commented because I didn't use it in my solution. But if you need to check this approach you can create new sheet i.e. Translations and have three columns Location, Text in fi-FI and Text in en-US. And then dynamically go through those translations and copy text to correct location i.e. Data!A5.

3. Create SharePoint structure for the demo

SharePoint site structure is following in my demo (in your case you can have whatever names... I just used culturenames to make this as simple as possible):

Example portal
- en-US:
  - default.aspx
  - MyEWA.aspx
- fi-FI
  - default.aspx
  - MunEWA.aspx

Obviously en-US is site that has regional settings (Site settings->Site administration: Regional settings) set to English locale:

And fi-FI site is set to Finnish locale.

4. Create custom web part that hosts EWA and fills the other requirements as well

Now we're ready to show the user interface for our solution. My solution is MyEWA web part that contains all the necessary controls and functionality this solution needs. As always my I'll cut short in the UI implementation. I'll just add few buttons and dropdown but no fancy look & feel.

UI in MyEWA.aspx:

UI in MunEWA.aspx:

(Note: Anyone who understands Finnish may laugh at my translations since I translated them smile in my face :-)

UI is pretty easy and straight forward: 1 dropdown to select product (=this is used as filter criteria in DB request) and 2 export buttons (Excel and CSV). If user would press Export Excel output would be something like this:

And if user would press Vie Exceliin output would be something like this:

Both of those export files are Snapshots (=they don't contain formulas just data).

If user would use Vie CSV -button (=Export to CSV) it would look something like this:

(Note: You can see [again!] empty row 5... but I'll explain reason for that soon)

Now I think we're ready to start reading some code. Just to remind you that I have put all the functionality into this one .cs file but in real life you don't do that! You refactor this kind of approach into several classes so that it is more maintainable. My "one file approach" is for demonstration purposes only. Also improving error handling is out of scope of this demo. Access rights is also something that needs to be solved since that process needs to write files to file system. Of course that can be solved with impersonation but anyway that needs to be taken into account. But finally here's the code:

using System;
using System.Runtime.InteropServices;
using System.Web.UI;
using System.Web.UI.WebControls.WebParts;
using System.Xml.Serialization;
using Microsoft.SharePoint;
using Microsoft.Office.Excel.WebUI;
using System.IO;
using System.Threading;
using Microsoft.Office.Excel.Server.WebServices;
using System.Web.UI.WebControls;
using System.Web;
using System.Resources;
using System.Data;
using System.Data.SqlClient;
using System.Text;

namespace MyEWA
{
  [Guid("97db47dc-7f23-4975-b246-26c237e74243")]
  public class MyEWA : System.Web.UI.WebControls.WebParts.WebPart
  {
    ExcelWebRenderer ewa;
    DropDownList productsList;
    Button exportSnapshot;
    Button exportSnapshotCSV;
    string xlFileWithoutPath;
    string xlFile;
    string xlPath;
    string xlStorageDirectory;
    string xlTemplateFile;
    string culture;

    public MyEWA()
    {
      this.ExportMode = WebPartExportMode.All;
    }

    private string ConnectionString
    {
      get
      {
        // ERROR: don't ever store connection string in your class!
        return "...";
      }
    }

    // TODO: refactor this into separate class!
    private DataTable GetExcelData(string product, string culture)
    {
      using (SqlConnection conn = new SqlConnection(this.ConnectionString))
      {
        // If you have translations inside your database you can pass on the culture to the db:
        SqlCommand cmd = new SqlCommand(
          "SELECT Production.vProductAndDescription.ProductModel, " +
          "Production.vProductAndDescription.Description AS Product, " +
          "Production.Product.ListPrice AS Price " +
          "FROM Production.vProductAndDescription INNER JOIN " +
          "Production.Product ON " +
          "Production.vProductAndDescription.ProductID = Production.Product.ProductID AND " +
          "Production.vProductAndDescription.ProductModel LIKE @Product+'%' AND " +
          "Production.vProductAndDescription.CultureID LIKE @Culture+'%'", conn);

        cmd.Parameters.AddWithValue("@Product", product);
        cmd.Parameters.AddWithValue("@Culture", culture.Substring(0,2));

        SqlDataAdapter adapter = new SqlDataAdapter(cmd);
        DataSet ds = new DataSet();
        adapter.Fill(ds);

        return ds.Tables[0];
      }
    }

    protected override void CreateChildControls()
    {
      // Get default locale from current site:
      // (Note: this can be changed from site settings)
      culture = SPContext.Current.Web.Locale.ToString();
      if (string.IsNullOrEmpty(this.Page.Request["Language"]) == false)
      {
        // I'll give way to change the language on the fly:
        culture = this.Page.Request["Language"];

        Thread.CurrentThread.CurrentCulture = new System.Globalization.CultureInfo(culture);
        Thread.CurrentThread.CurrentUICulture = new System.Globalization.CultureInfo(culture); 
      }

      Controls.Clear();
      base.CreateChildControls();
      this.ChildControlsCreated = true;

      Label productLabel = new Label();
      productLabel.Text = MyEWAResources.ProductList_Select;
      this.Controls.Add(productLabel);

      productsList = new DropDownList();
      productsList.ID = "productsList";
      // TODO: get your product filter list using SQL statements
      // and remember to store that in cache => performance.
      // Something like this:
      // this.Page.Cache.Add("ProductList", ...);
      // And of course you can use SqlCacheDependency if you want.

      // I'll just fill in static filters (I'm lazy!):
      productsList.Items.Add(new ListItem(MyEWAResources.ProductList_Item1, "Mountain"));
      productsList.Items.Add(new ListItem(MyEWAResources.ProductList_Item2, "Road"));
      productsList.Items.Add(new ListItem(MyEWAResources.ProductList_Item3, "HL"));
      productsList.AutoPostBack = true;
      this.Controls.Add(productsList);

      // Create some buttons:
      exportSnapshot = new Button();
      exportSnapshot.Text = MyEWAResources.ExportSnapshot;
      this.Controls.Add(exportSnapshot);

      exportSnapshotCSV = new Button();
      exportSnapshotCSV.Text = MyEWAResources.ExportSnapshotCSV;
      this.Controls.Add(exportSnapshotCSV);

      ewa = new ExcelWebRenderer();
      this.Controls.Add(ewa);
    }

    protected override void OnLoad(EventArgs e)
    {
      this.EnsureChildControls();
      base.OnLoad(e);

      // Get values from UI:
      string productFilter = productsList.SelectedValue;

      // TODO: get these values from web part properties!!!
      // This is the UNC path of the Excel files:
      xlPath = "//demo1/ExcelServices/xlsx";
      // This is folder where static instances will be created (under previous folder)
      xlStorageDirectory = "Storage/";
      // This name of the static Excel file:
      xlFileWithoutPath = string.Format("MyEWA-{0}-{1}.xlsx", culture, productFilter);
      // This name of the template Excel file
      // NOTE: You should store this inside SharePoint!!!
      xlTemplateFile = xlPath + "MyEWA.xlsx";
      xlFile = xlPath + xlStorageDirectory + xlFileWithoutPath;

      // Let's add Export calls to the client side of the buttons:
      exportSnapshot.OnClientClick = this.BuildExportScript("xlsx", xlFileWithoutPath);
      exportSnapshotCSV.OnClientClick = this.BuildExportScript("csv", xlFileWithoutPath);

      // Do we need to export file? (new .aspx page for this would be good idea!)
      if (string.IsNullOrEmpty(this.Page.Request["ExportFormat"]) == false &&
          string.IsNullOrEmpty(this.Page.Request["File"]) == false)
      {
        // Yes we need to export file

        // TODO: add sanity checks for the filename
        xlFile = xlPath + xlStorageDirectory + this.Page.Request["File"].ToString();
        string format = this.Page.Request["ExportFormat"].ToString();
        switch (format)
        {
          case "csv":
            ExportSnapshotAsCSV();
            break;
          case "xlsx":
            ExportSnapshotAsExcel();
            break;
          default:
            throw new Exception(
              string.Format("Export format '{0}' not supported!", format));
        }
      }

      // Does the cache file already exists?
      bool useCachedXlFile = File.Exists(xlFile);
      if (useCachedXlFile == true)
      {
        // Cache file exists but let's compare timestamps:
        FileInfo src = new FileInfo(xlTemplateFile);
        FileInfo dst = new FileInfo(xlFile);
        if (src.LastWriteTimeUtc > dst.LastWriteTimeUtc)
        {
          // Template file is newer than cache file!
          useCachedXlFile = false;
        }
      }

      if (useCachedXlFile == true)
      {
        // TODO: add your own custom logic to check that cache is still valid
        // I.e. check site property bag about 'UpdateExcels' timestamp
      }

      if (useCachedXlFile == false)
      {
        // We need to create the cache file!

        // Let's get data to the Excel:
        DataTable excelData = this.GetExcelData(productFilter, culture);
        // In my example 'Description' field is actually already translated in the db
        // so we need to translated only the 'ProductModel' in our data
        // (this is here just to demonstrate that text can be translated in many 
        // different layers: Excel, Code, DB etc.)
        if (culture.StartsWith("en", StringComparison.OrdinalIgnoreCase) == false)
        {
          // This culture isn't the "original" database culture 
          // so we need to translate "ProductModel" field

          // TODO: implement real translation 
          foreach (DataRow row in excelData.Rows)
          {
            row["ProductModel"] = "Suomeksi - " + row["ProductModel"];
            row.AcceptChanges();
          }
          excelData.AcceptChanges();
        }

        using (ExcelService es = new ExcelService())
        {
          Status[] status;
          string sessionId = es.OpenWorkbook(xlTemplateFile, culture, culture, out status);
          // Set culture to the parameters:
          es.SetCellA1(sessionId, "Parameters", "Culture", culture, out status);

          // TODO: get your fancy data with current information:
          object[] excelDataArray = new object[excelData.Rows.Count + 1];
          object[] excelRowArray = new object[excelData.Columns.Count];

          // You might think that you could change the columns of your data with following code:
          // for (int i = 0; i < excelData.Columns.Count; i++)
          // {
          //   // Get correct labels for the texts:
          //   excelRowArray[i] = MyEWAResources.ResourceManager.GetString(
          //     "Column_" + excelData.Columns[i].ColumnName);
          // }
          // BUT BUT BUT... You cannot change columns since Pivottable won't work after that :-(

          excelDataArray[0] = excelRowArray;

          // Fill the data with retrieved values:
          for (int i = 0; i < excelData.Rows.Count; i++)
          {
            excelRowArray = new object[excelData.Columns.Count];
            for (int j = 0; j < excelData.Columns.Count; j++)
            {
              excelRowArray[j] = excelData.Rows[i][excelData.Columns[j]];
            }
            excelDataArray[i + 1] = excelRowArray;
          }
          
          RangeCoordinates range = new RangeCoordinates();
          range.Column = 0;
          range.Row = 1;
          range.Width = excelData.Columns.Count;
          range.Height = excelData.Rows.Count + 1;

          // Store data into Excel:
          es.SetRange(sessionId, "Data", range, excelDataArray, out status);

          // You might think that you could change the Pivottable captions on the fly:
          // es.SetCellA1(sessionId, "Display", "A5", "My new pivot caption", out status);
          // BUT BUT BUT.. It doesn't work (=nothing happens) :-(

          // Let's refresh our datasources (=update pivottable):
          es.Refresh(sessionId, null, out status);

          // Following 'block' of code can be used if you want to have
          // separate sheet in your Excel file for translations:
          //
          //bool doneTranslations = false;
          //range = new RangeCoordinates();
          //range.Column = 0;
          //range.Row = 1;
          //range.Width = 2;
          //range.Height = 10;
          //while (doneTranslations == false)
          //{
          //  // So we get all the data from Translations-sheet and then we
          //  // copy the text over to the locations defined in Excel:
          //  object[] rowData = es.GetRange(sessionId, 
          //    "Translations", range, false, out status);
          //  for (int i = 0; i < range.Height; i++)
          //  {
          //    if (rowData is object[])
          //    {
          //      object[] columnData = rowData[i] as object[];
          //      if (columnData[0] == null)
          //      {
          //        // No more translations
          //        doneTranslations = true;
          //        break;
          //      }
          //      string[] translationRange = Convert.ToString(columnData[0]).Split('!');
          //      es.SetCellA1(sessionId, translationRange[0], translationRange[1],
          //        Convert.ToString(columnData[1]), out status);
          //    }
          //  }
          //  range.Row += range.Height;
          //}
          
          // Do we calculations that we need to refresh?
          es.CalculateWorkbook(sessionId, CalculateType.CalculateFull, out status);
          byte[] wb = es.GetWorkbook(sessionId, WorkbookType.FullWorkbook, out status);
          es.CloseWorkbook(sessionId, out status);

          // Let's write our cache file to the disk:
          // (you need to have write rights for that location!)
          File.WriteAllBytes(xlFile, wb);
        }
      }

      // TODO: set all necessary parameters to EWA:
      ewa.ShowWorkbookParameters = false;
      ewa.WorkbookUri = Uri.UriSchemeFile + Uri.SchemeDelimiter + xlFile;
      ewa.ToolbarStyle = ToolbarVisibilityStyle.None;
      ewa.ChromeType = PartChromeType.None;

      // TODO: you can also get settings from various places:
      if (string.IsNullOrEmpty(this.Page.Request["NamedItem"]) == false)
      {
        ewa.VisibleItem = this.Page.Request["NamedItem"].ToString();
      }
      if (string.IsNullOrEmpty(this.Page.Request["Rows"]) == false)
      {
        ewa.RowsToDisplay = Convert.ToInt32(this.Page.Request["Rows"]);
      }
      if (string.IsNullOrEmpty(this.Page.Request["Columns"]) == false)
      {
        ewa.ColumnsToDisplay = Convert.ToInt32(this.Page.Request["Columns"]);
      }
    }

    private string GetSnapshotFile()
    {
      byte[] wb;
      string snapshotCacheFile = xlFile + ".snapshot.xlsx";
      if (File.Exists(snapshotCacheFile) == false)
      {
        // No cache file yet.. so let's create it:
        using (ExcelService es = new ExcelService())
        {
          Status[] status;
          string sessionId = es.OpenWorkbook(xlFile, culture, culture, out status);
          // Note! If we take the 'normal' excel file then we return formulas as well
          // => we really want to use the snapshot file!
          wb = es.GetWorkbook(sessionId, WorkbookType.PublishedItemsSnapshot, out status);
          es.CloseWorkbook(sessionId, out status);
        }

        File.WriteAllBytes(snapshotCacheFile, wb);
      }

      return snapshotCacheFile;
    }

    private string BuildExportScript(string exportFormat, string file)
    {
      string url = this.Page.Request.Url.ToString();
      if (url.IndexOf('?') == -1)
      {
        url += "?";
      }
      else
      {
        url += "&";
      }
      url += "ExportFormat=" + exportFormat + "&File=" + file;
      return "window.frames['MyEWA'].location='" + url + "'; return false;";
    }

    private void ExportSnapshotAsExcel()
    {
      byte[] fileData = File.ReadAllBytes(this.GetSnapshotFile());
      // TODO: set meaningful name for the file (end user sees it File Download dialog)
      string filename = "ExcelData.xlsx";
      this.Page.Response.ClearHeaders();
      this.Page.Response.ClearContent();
      this.Page.Response.AppendHeader("Content-Disposition", 
        "attachment;filename=" + filename);
      this.Page.Response.AddHeader("LAST-MODIFIED", DateTime.Now.ToString("r"));
      this.Page.Response.ContentType = 
        "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
      this.Page.Response.OutputStream.Write(fileData, 0, fileData.Length);
      this.Page.Response.OutputStream.Flush();
      this.Page.Response.End();
    }

    private string GetSnapshotFileCSV()
    {
      StringBuilder sb = new StringBuilder(2048);
      string snapshotCacheFile = xlFile + ".snapshot.csv";
      if (File.Exists(snapshotCacheFile) == false)
      {
        // No cache file yet.. so let's create it:
        using (ExcelService es = new ExcelService())
        {
          Status[] status;
          string sessionId = es.OpenWorkbook
            (this.GetSnapshotFile(), culture, culture, out status);
          RangeCoordinates range = new RangeCoordinates();
          range.Column = 0;
          range.Row = 0;
          // TODO: get real range according to the data (not fixed :-)
          // You could store the value when you fill the Excel with data...
          range.Width = 5;
          range.Height = 300;

          object[] data = es.GetRange(sessionId, "Display", range, false, out status);
          foreach (object row in data)
          {
            if (row is object[])
            {
              object[] columns = row as object[];
              foreach (object columnData in columns)
              {
                if (columnData != null)
                {
                  sb.Append(columnData);
                }
                sb.Append(";");
              }
              sb.Append(Environment.NewLine);
            }
          }

          es.CloseWorkbook(sessionId, out status);
        }

        File.WriteAllBytes(snapshotCacheFile, 
          Encoding.Convert(Encoding.UTF7, 
          Encoding.GetEncoding("windows-1250"), 
          Encoding.UTF7.GetBytes(sb.ToString())));
      }

      return snapshotCacheFile;
    }

    private void ExportSnapshotAsCSV()
    {
      byte[] fileData = File.ReadAllBytes(this.GetSnapshotFileCSV());

      // TODO: set meaningful name for the file (end user sees it)
      string filename = "CSVData.csv";
      this.Page.Response.Clear();
      this.Page.Response.AppendHeader("Content-Disposition", "attachment;filename=" + filename);
      this.Page.Response.AddHeader("LAST-MODIFIED", DateTime.Now.ToString("r"));
      this.Page.Response.ContentType = "text/plain";
      this.Page.Response.OutputStream.Write(fileData, 0, fileData.Length);
      this.Page.Response.Flush();
      this.Page.Response.End();
    }

    protected override void Render(HtmlTextWriter writer)
    {
      this.RenderChildren(writer);

      // Let's add our export iframe:
      writer.Write("<iframe name=\"MyEWA\" src=\"about:blank\" width=\"1\" height=\"1\" />");
    }
  }
}

I'm not going to explain code since most of that is quite easily understandable... but if you have questions then post comment to this entry and I'll get back to you.

You can probably see that I have added a lot of "// TODO:"-markers to identify places you most likely will be doing some modifications if you plan to use my code. I also left some code blocks that I have tried and noticed that it doesn't work as you would expect. Good example is that you cannot change the caption / labels of the pivottable. You CAN do that in Excel but you cannot do that using Excel Services API. So in other words... this doesn't work:

es.SetCellA1(sessionId, "Display", "A5", "My Pivottable caption", out status);

That is the reason why I have actually hidden that row. User sees still caption but it's just normal text that is formatted so that it looks like caption :-) But if I unhide A5 it would look like this:

But again... that caption isn't visible if you export Excel (row 5 is empty but user needs to Unhide that row in order to see it):

Another Excel tip that you need is Pivottable filters. Because if you don't set any filter you would get "(blank)" rows into table. That can be filtered away with label filters:

Set "Does Not Equal..." -filter to be empty:

And after that you don't get those "(blank)" rows in your pivottable.

If you're interestested that how did my solution look like inside VS:

So I had resource files for default language and fi-FI. Since I used SharePoint Web Part template (I like some much the F5 integration :-) I needed manually copy the fi-FI folder under my projects Debug folder under the web applications bin directory: C:\Inetpub\wwwroot\wss\VirtualDirectories\1000\bin. I just wanted to mention this if you have issues with your resource files :-)

Performance?

I ended up doing a lot of static Excel files... so I think that it should be fast since now there isn't any database queries happening (not verified). BUT at least I managed to demonstrate that you could create static files so that if everything goes fine you should benefit from memory of your app servers. And if your data changes rarely you can easily create script that forces all the cache files to be created (just loops all the necessary HTTP requests). This would be trivial to implement. Here is one example call:

http://demo1:1000/fi-FI/Sivut/MunEWA.aspx?ExportFormat=csv&File=MyEWA-fi-FI-Mountain.xlsx&Language=en-US

You can modify ExportFormat and File and Language in order to get file that you want.

And if you think this solution more deeply... actually you don't need even database at the production environment since you create static copies of Excels in another environment (i.e. publishing environment) :-) That would be another way to look at this solution... just distribute static Excel files between environments.

What's other possibilies I have?

You probably at least two more options:

OLAP + CUBE* -functions inside Excel
- You need to create ODC for the OLAP connection
  - Store this in Data Connection Library in SharePoint
  - You need to configure Single Single-On Service (SSO) to pass the credentials
- You would have a better way to handle data
- Downside:
  - If you change filter => new database request!
  - You will still have same issues with pivottable that with "old relational" pivottable
UDF (User Defined Function)
- Create still all the data retrieval logic with .NET
  - In Excel you would have something like this "=MyRetrieveData(Parameters!B2:Parameters:!B6)"
- Downside:
  - No preview :-(
  - Database requests
- NOTE: You can use this in your Excels even if you dynamically create the static instances!
  - It could be probably good idea to use this for complex logic

I haven't validated those as detailed as I have this "static Excel instances" approach so I might be missing something important about the other approaches :-)

More information about Excel Services and Excel

As always... internet is full of stuff around Excel Services but you might want to check out at least these:

http://technet2.microsoft.com/Office/en-us/library/eea3ace8-0863-429a-b1e8-041254ed2fc41033.mspx?mfr=true -- White papers: Excel Services step-by-step guides

http://msdn2.microsoft.com/en-us/library/bb267252.aspx#Office2007ExcelServicesUnlimited_SharePointLists -- Extending the Excel Services Programmability Framework

http://www.microsoft.com/downloads/details.aspx?FamilyId=2D779CD5-EEB2-43E9-BDFA-641ED89EDB6C&displaylang=en -- Excel 2007 Document: Designing SQL Server 2005 Analysis Services Cubes for Excel 2007 PivotTables

http://blogs.msdn.com/cumgranosalis/ -- Cum Grano Salis

http://blogs.msdn.com/luisbeonservices/ -- LuisBE on Services

http://blogs.msdn.com/excel/ -- The team blog for Microsoft Excel and Excel Services

http://msdn2.microsoft.com/en-us/library/bb758869.aspx -- Chapter 1: An Introduction to Excel Services

http://msdn2.microsoft.com/en-us/library/bb758868.aspx -- Chapter 3: Excel Web Access

Final words

Well that ended up being looong post :-) I'll probably still work on this subject so I might get back with follow-ups. I'll probably do some performance tests so that I'll get some indications that how well this does work.

This subject is actually quite complex. I don't consider this case as solved... I just consider this as good kickstart :-) There are still many open questions like: "What's the best place to do translations?" (and many more). Well I don't have answer to that since I think it's not that black and white :-) You may see that some are easy to translate in DB but same are too hard (or database model doesn't support it) and then you need to do that somewhere else.

If you plan to use this code you first need to generalize it since my solution is quite fixed to one certain Excel file. But I think that is actually quite easy task and I'll let you work on that.

Anyways... Happy hacking!

J

Adding own custom web service to SharePoint

jannemattila — Wed, 26 Sep 2007 22:10:34 GMT

I posted awhile back about InfoPath and Web Service data connection and I got question about using own web services in SharePoint (or at least I interpreted it that way :-). I tried to describe idea to have own custom "proxy" web service that would do all the necessary stuff inside SharePoint so that you could use local API and not the web service API. So if you know about application pages then you know what I'm talking about. Same things apply to web services too. But as always... I'm not good at telling stuff so let's jump to the code :-)

I just quickly created Microsoft.MCS.WebService.ExampleWebService.asmx and added it to the layouts folder (=C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\LAYOUTS). This time I skipped features/solutions and did my example the easy way (as always :-).

Here is the content of the .asmx file:

1 2	<%@ WebService Class="Microsoft.MCS.WebService.ExampleWebService, Microsoft.MCS.WebService,Version=1.0.0.0,Culture=neutral,PublicKeyToken=a030a6768fa75cfc" %>

And code behind is in here (I signed it and put it into GAC):

using System;
using System.Web.Services;
using Microsoft.SharePoint;
using System.Security.Principal;

namespace Microsoft.MCS.WebService
{
  public class ExampleWebService : System.Web.Services.WebService
  {
    [WebMethod]
    public string WhoAmI()
    {
      return WindowsIdentity.GetCurrent().Name +
          " <->" + SPContext.Current.Web.CurrentUser.LoginName;
    }
  }
}

Code is pretty simple since it just returns string that contains current users WindowsIdentity name and LoginName using SharePoint API.

If you run this stuff directly from SharePoint using browser you'll get something like this:

And now you can use this stuff from InfoPath as well:

So if you want to have weird SharePoint functionality (like enumerating lists) in your InfoPath forms you could use your own "proxy" web service to make it happen. This example was just skeleton that shows that this kind of stuff can be done but the exact implementation is left to you. I hope that I answered one of the questions that have been asked in my blog :-)

And if you try to update stuff in your code you'll most likely end up with SPException saying that "The security validation for this page is invalid. Click Back in your Web browser, refresh the page, and try your operation again."... but don't worry because that's the way it works :-) You can turn off the security validation check temporary with AllowUnsafeUpdates = true. I'll leave rest of that stuff for you...

Anyways... Happy hacking!

J

MOSS: Backup and restore and missing Timer job definitions

jannemattila — Fri, 31 Aug 2007 13:44:47 GMT

If you have your SharePoint application running on Server1 and you decide to move it to Server2 with backup and restore... you'll (may) end up having application at the Server2 that doesn't have all the necessary timer job definitions. There is however way to get those timer job definitions back but it's not the most elegant way :-)

So you have restored your system (for example "stsadm -o restore -url http://demo1:1000/ -filename Backup.dat") to Server2 and you'll go to Central Administration to see all the timer job definitions of your application:

It looks like everything is fine... but if you compare this list to another application:

So quite easily you can spot that this list is a lot longer for the another application (14 vs. 19)! If you compare them by each entry then you'll notice that your application is missing:

...
Scheduled Approval SharePoint - 80 Minutes
Scheduled Page Review SharePoint - 80 Daily
Scheduled Unpublish SharePoint - 80 Minutes
...
Variations Propagate Page Job Definition SharePoint - 80 Minutes
Variations Propagate Site Job Definition SharePoint - 80 Minutes
...

That means that if you want to use Scheduled publishing you won't be able to do so :-( And you would have problems with Variations too.

But let's try and see what actually happens Scheduled publishing. I created page that should be published at 1:00 PM:

After a while error message comes to the toolbar (it comes few minutes after it should be published):

So "An error occurred trying to publish this page on the set schedule. Republish the page with a new date or view the log if the problem continues." will be you destiny... and page won't be published. But this was expected since we're missing timer job for that.

In order to fix this you need to actually create site collection to your application that will create those timer job definitions (ex. Publishing Portal or Collaboration Portal). NOTE: You really need to use one of those templates that will create those timer jobs. If you use ex. Blank Site you won't have those timer job definitions that you're looking for. After you have created the new site collection you can go and verify that it has created those 5 timer jobs. After this you can go and delete the newly created site collection and restore your own system back to the application.

Reason for this seems to be that restore just creates those 14 timer job definitions and doens't restore the ones that are build on top of the base WSS. So in order to get those back you need to do this small trick that I have just described in this post.

To summarize... if you want to transfer your system you need to follow couple of extra steps:

Create new web application
Create new site collection to your application

Use Publishing Portal or Collaboration Portal templates!

Verify that you have all the necessary timer jobs
Delete your newly created site collection
Restore your own system

I hope I made someones life a bit easier with this post. I don't think that this is the first thing that comes to your mind when you're WCM page gives you error message when it should be published :-)

Anyways... Happy hacking!

J

Failed to determine definition for Feature...

jannemattila — Mon, 27 Aug 2007 16:04:41 GMT

I was working on SharePoint case and found myself staring at the logs... again :-)

I found 11 lines with "Failed to determine definition for Feature with ID..." and it was repeating quite often. And that error is something new for me... And if you have stared SharePoint logs for an year that's really amazing! Exact error messages were like this:

...
Failed to determine definition for Feature with ID '065c78be-5231-477e-a972-14177cc5b3c7'. Skipping this feature for element querying consideration.
Failed to determine definition for Feature with ID '0806d127-06e6-447a-980e-2e90b03101b8'. Skipping this feature for element querying consideration.
Failed to determine definition for Feature with ID '2510d73f-7109-4ccc-8a1c-314894deeb3a'. Skipping this feature for element querying consideration.
Failed to determine definition for Feature with ID '00bfea71-dbd7-4f72-b8cb-da7ac0440130'. Skipping this feature for element querying consideration.
...

So I searched (with Vesa Juvonen) those guids from 12\TEMPLATE\Features-folder and found out that those features were actually Enterprise Edition features (like Excel Services). MOSS "couldn't find" those features even if they were OOB features. The reason for this was actually quite understandable... since there was a site collection that has been moved from MOSS Enterprise to Standard. And in Standard you don't have those Enterpise Edition features available even you have those activated in you site collection. The lesson of this exercise was... don't mix and match editions :-)

Anyways... Happy hacking!

J

How to install Windows SharePoint Services 3.0 Tools to XP or Vista

jannemattila — Thu, 16 Aug 2007 06:42:24 GMT

I got email from Philip Edney with a hint that how you can install Windows SharePoint Services 3.0 Tools: Visual Studio 2005 Extensions (VSeWSS) for none-Windows 2003 server OS. Of course since that's not supported you'll have to do small registry hack (but that's what developers do :-):

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\12.0]
"Sharepoint"="Installed"

I just tested it with Windows Vista and it works! Before the hack it gives me "This product can be installed if Windows SharePoint Services 3.0 has been installed first." and after the hack it went fine. I think there're a lot of people who would like to try/use this option even if they fall for unsupported mode.

But thanks to Phil to let me know about that possibility! Update: Phil mentioned that he actually got the information from Raju Sakthivel so the credits chain just goes on and on :-)

Anyways... Happy hacking!

J