UK Application Development Consulting

Silverlight out-of-browser and initParams

2009-11-12T10:00:00Z

Silverlight 3 supports a very cool 'deployment model' where you can run the Silverlight Application 'out-of-browser' (OOB) and kick it off from your start menu. It even works offline. This is the future, get to like it.

However, because the plug-in is no longer hosted in your page you lose all those juicy settings and, perhaps worst of all, you initialization parameters aren't available when the app runs OOB. This is frustrating because a lot of people pass in settings from their host web-application's web.config file using this very mechanism.

However, it only takes a little bit of code to work around this scenario by caching the initParams in isolated storage whenever the applicaiton is run in-browser. Here's some code that does just that:
private void Application_Startup(object sender, StartupEventArgs e)
{
    var mp = new MainPage();
this.RootVisual = mp;
    IDictionary<string, string> initParams = LoadInitParams(e);
// this just adds the init params to the MainPage for demo purposes only
foreach (var kv in initParams)
    {
        mp.LayoutRoot.Children.Add(new TextBlock { Text = kv.Key, FontStyle = FontStyles.Italic });
        mp.LayoutRoot.Children.Add(new TextBlock { Text = kv.Value, Margin = new Thickness(5,0,0,0) });
    }
}
private static IDictionary<string, string> LoadInitParams(StartupEventArgs e)
{
    IDictionary<string, string> initParams;
// if running out of browser retrieve 'initParams' from cache
if (Application.Current.IsRunningOutOfBrowser)
    {
using (IsolatedStorageFile isf = IsolatedStorageFile.GetUserStoreForApplication())
using (IsolatedStorageFileStream stream = new IsolatedStorageFileStream("initParams.txt", System.IO.FileMode.Open, isf))
        {
            DataContractSerializer ser = new DataContractSerializer(typeof(Dictionary<string, string>));
            initParams = (Dictionary<string, string>)ser.ReadObject(stream);
        }
    }
// otherwise write initParams to cache
else
    {
        initParams = e.InitParams;
using (IsolatedStorageFile isf = IsolatedStorageFile.GetUserStoreForApplication())
using (IsolatedStorageFileStream stream = new IsolatedStorageFileStream("initParams.txt", System.IO.FileMode.Create, isf))
        {
            DataContractSerializer ser = new DataContractSerializer(typeof(Dictionary<string, string>));
            ser.WriteObject(stream, initParams);
        }
    }
return initParams;
}
Easy peasy.

One recommendation here though is that you probably want a layer of indirection between Initalization Parameters and your 'silverlight configuration' in this case. The reason for this is the initParams will only get updated when you run the app in-browser. It's possible that your user will never run the app in-browser again. And even if the XAP gets updated your new parameters may never be seen, unless the app runs in-browser. So I'd recommend storing a static and constant URL in your initParameters that specifies an XML file (or web service) on the web server that contains your silverlight configuration.

You can then download this at will, even out-of-browser, whenever the network is available.

Originally posted by Josh Twist on 21 October 2009 here.

Getting started with WinDbg and Sos.dll

2009-11-10T10:00:00Z

I wrote this largely to get the commands right in my own head – as a result, this is more a list of commands and some semblance of a sequence you can run them in. If you’ve never used these tools before this will get you out of the gate on WinDbg and the debugger extension SOS.dll.

If you’re looking for scenario's and walkthroughs I can’t stress enough that Tess Ferrandez is the go-to-guy for this and does a much better job; including a superb hands-on lab and crash walkthroughs on her Blog. All the commands I've listed are also documented here

At the end I’ve put three quick sequences together – looking at an web site, reviewing an exception and getting the IL for a particular method.

Symbols

Before beginning confirm you’ve setup your symbols and know where sos.dll is:
1) setup debug symbol and path – the easiest way is to configure a local store (for example c:\symbols) and use the public symbol server – create an environment variable _NT_SYMBOL_PATH and set it to SRV*c:\symbols*http://msdl.microsoft.com/download/symbols. If you have symbols in multiple places, add these locations separated by ‘;’. Note, if you’re using windows 7, windows symbols for your version are already installed to c:\windows\symbols.

If you set your symbol path to use the public server, or a symbols server, anything that requires you to resolve function names will cause the symbols to be downloaded – so some functions will take a few seconds to run, until you’ve built up your cache. WinDBG will set its status to busy while this happens.

2) If you’re debugging a 32 bit process or memory dump, copy the sos.dll from C:\Windows\Microsoft.NET\Framework\v2.0.50727 to the local directory for your windbg install (typically c:\program files[(x64)]\Debugging Tools for Windows). For 64 bit – use the framework64 version. It isn’t mandatory to copy the file here but makes .load a little quicker to run.

Attach to a running process

Pretty straight forwards – open windbg, hit F6, choose the process to connect to. You’ll be asked if you want to save the workspace – select no. Assuming we attached successfully, type ‘g’ – the debugger command for go. To break into the process again, hit ctrl-break. If all you have is a dump, open the dump from the file menu (open Crash Dump) – all of the following will work if a full user dump was taken (rather than a mini dump).

While the process is running, if any exceptions are being caught in the application, you’ll get ‘CLR exception - code e0434f4d (first chance)’ appearing in the console. See ‘Breaking When an Exception is thrown’ if you want to break on these.

If you get an error, typically ‘request not supported’. check you’re using the right ‘bitness’ of debugger for the process you’re connecting to or the right chip architecture for WinDbg (eg, IA64, x64, x86). You may also have to run windbg elevated.

Load SOS.Dll

Ctrl-Break into the process and load the SOS.dll by typing:

.Load sos.

If you didn’t copy sos into the same directory as windbg you’ll have to specify the full path. Note the command is prefixed with a ‘.’. You’ll get a message if sos can’t be found. If you get a x08004005 error when calling any of the following commands, make sure you’ve copied the right version (either x64 or x86) into your WinDbg directory.

Two other commands that will be of use here:

.sympath <path to symbols> // use to set or get the path to symbols.
.reload // force a reload of any symbols

If the symbols are wrong often you’ll get a warning telling you to check them, and you’ll also get hex instead of the missing function names when resolving call stacks.

Help!

SOS – it’s a debugger extension, so it’s commands are prefixed by ‘!’. Get started by calling:

!Help.

If you get the message ‘no export help found’ then sos hasn’t been loaded – just try and reload it using .reload sos.

What’s currently running?

When you break into the process you’ll end up sitting on a specific thread, which is identified by the number to the left of the command prompt. You can get the native call stack for the thread you’re currently on using the WinDbg command:

Any hex in the output where you can see names and check you’ve got the right symbols. You can get the managed call stack for the current thread using an SOS extension:

!CLRStack

if there’s no managed call stack, well, you’ll get an error telling you this. Otherwise you’ll get the managed stack for the current thread. Use –p and –l to return parameter and local variable information. Use –a to return both.

Get a list of the managed threads in the process using:

!Threads

From this you can get a list of ThreadIds (the column on the far left is the id to use with the next commands) and whether or not they’re gc enabled (true means managed code, false means unsafe code), how many locks are open for that thread, it’s apartment model and finally any exceptions. This last column will also indicate which one of the threads is the Finalizer. If a thread holds an exception, you’ll find the address at the far right. In all fairness, it’s a good place to start any exploration.

To jump onto a different thread use the ‘~’ to identify you’re issuing a thread command, then the command. For example:

~<<thread id>> s // switch context to the thread id, eg ~27 s // switch to thread id 27
~* e kb // for all threads in the process, execute kb (dump the stack)
~* e !CLRStack // you probably get the idea, but for completeness: for all threads, evaluate !clrstack

WinDbg allows to to shell out – you might be interested in parsing some of the output using the windows Batch command - FIND. The following evaluates all threads to see what has the given function on their stack:

.shell -i - -ci "~* e !clrstack" FIND “<<FunctionName>>” for example .shell -i - -ci "~* e !clrstack" FIND “DoStuff”

Note the syntax above is sensitive, the dash after ‘–i’ is not a typo.

Finally – you can get all the objects associated with a set of calls:

!DumpStackObjects

this will, er, Dump all the objects within the bounds of the current stack. If, for example you’ve got a function calling out to Sql – this will help you get the Command object associated with that call stack if you’ve switched to that thread.

Allocations

Get the allocations for this program using:

!dumpheap –stat

This will dump the types of objects and their sizes on the heap. The stat switch says give me the summary. The 4 columns in the output are Method Table, number of objects, total size and type. Typically you’ll see System.String and System.Object high on this list. This will also tell you which generation each object is currently in.

To get a look at any of these objects, for example, System.String we can ask for the address of all objects of a given type over a fixed size. For example:

!dumpheap –type System.String –min 200

This will return a table of results detailing the address, method table and size of the object. Following this is a second table with Method Table Count of Objects, Total Size and the class name. Getting the address of an object from the previous command means you can start to examine it and it’s fields. If that object is a string, frequently you’ll get xml or other goodies buried within the process.

!dumpobj <<address>>

Often a command might just return a pointer to that object, for example the following command which looks for handles with no parents (an increasing number of these may indicate a leak).

!gchandleleaks

You can resolve the address of the object using the WinDbg macro ‘poi’ (pointer of int):

!dumpobject poi(<<handle>>)

Once you’ve got the address of an object you can resolve what else might be referencing that object using:

!gcroot <<address of object>>

Here you’re looking either for explicit object references, or if objects are rooted in the domain (statics). It might also be useful to understand what the allocation graph is – call:

!traverseheap <<outputfile.log>>

This will produce a log file output that can be presented in the clr profiler available here and is very useful for evaluating object graphs, and viewing how objects are laid out in each heap. be warned though – even for process of a couple of 100Mb this can often take over an hour.

Some Commands You’re Gonna love

!aspxpages is one of a great place to start if you’re looking at a web process. It checks the heap for any existing HttpContext objects and attempts to resolve which thread either executed or is executing it, whether it’s still currently running and if the request completed or not, and how long it took and what the return code was – it’s a very quick indicator of health. Occasionally it will report that a thread is no longer running (it’s Id will be XXX) – this may not be the case – so have a look at !threads to see if there is a suspect that may be running that request.

!dumpdatatables - returns a table of addresses of objects that are data tables, nextRowID (rowcount) and number of columns. No of Cells = nextRowId * RowCount.

!dumpobject {address of columns} will return the column names allowing you to establish how much data is in datatables.

Is there a large amount of concurrency in our process?

!syncblk

A high number under ‘monitor held’ may indicate a contention problem.

Note that the number of monitors held = 1 for the owner + 2*waiting number of waiting threads.

Check the help file, but .Net 2.0 uses thin locks, so you might also want to run i syncblk returns nothing.

!DumpHeap -thinlock

Breaking when an Exception is Thrown

There are two ways to do this – the easiest is to use the debugger break into the process, select Debug –> Event Filters –> Enable CLR Exception. When an exception is thrown the debugger will break into the process and sit you on the thread that has the Exception. Call !Threads to which will return the type of exception and it’s address, you can then call:

!PrintException <<address>>

The second way is to use !StopOnException – see the help for this. I might come back an update this.

Possible Sequence of Events for troubleshooting a hanging web app

!aspxpages – look for requests that didn’t complete – find their thread id.
~<<threadid>> s – switch to that thread.
!clrstack get the command at the top of the stack
!DumpStackObjects – get the objects associated with this call stack.

Possible sequence of commands for getting the exception text:

Set filters to enable CLR Exception
g (go)
!threads to get the exception address and type
!PrintException <<address>> to get the exception text and any inner exception.

Get the IL for a method on the stack

!dumpstack – lists the return address for each call. Take the return address the line above the function you’re interested in – for example, I’d like to see DoStuff() so take the return address 000007ff00170223:

Child-SP RetAddr Call Site
000000000016e4f8 000007fefdc71203 ntdll!NtDelayExecution+0xa
000000000016e500 000007fef8de176d KERNELBASE!SleepEx+0xb3
000000000016e5a0 000007fef89a77b5 mscorwks!DllCanUnloadNowInternal+0xf37d
000000000016e620 000007fef8f8f2f9 mscorwks!CreateApplicationContext+0x391
000000000016e680 000007ff00170223 mscorwks!ReOpenMetaDataWithMemory+0x1ff59
000000000016e830 000007ff00170195 ExceptionalApplication!ExceptionalApplication.Program.DoStuff()+0x53
000000000016e880 000007fef8b5d502 ExceptionalApplication!ExceptionalApplication.Program.Main(System.String[])+0x25

We want the method descriptor, so call

!IP2MD 000007ff00170223

which returns the Method Descriptor:

0:000> !ip2md 000007ff00170223
MethodDesc: 000007ff000239b8
Method Name: ExceptionalApplication.Program.DoStuff()
Class: 000007ff00162230
MethodTable: 000007ff00023a48
mdToken: 06000002
Module: 000007ff000233d0
IsJitted: yes
CodeAddr: 000007ff001701d0

Now you can call !dumpil <<MethodDescriptor>> which get’s you the IL:

!dumpil 000007ff000239b8

That’s it for this post – I’d definitely encourage you to look at Tess’s blog and DumpAnalysis.Org for more.

Enjoy!

Originally posted by Ryan Simpson on 8 October 2009 here.

.NET Naming Conventions

2009-11-05T13:00:00Z

I often get asked about the Naming Conventions I adhere to when writing code (C#, naturally).

It made sense to share these in a blog post so I can refer to it in future.

Some of these guidelines (well, one, the underscore on private fields) are negotiable as a matter of style. However, the public stuff is non-negotiable. For this is how .NET APIs should be and failure to adhere to this reflects badly on your code. No, no, no.

I thought a good way to present this would be an example class demonstrating the rules and some comments to help, so here goes:

using System;

// Namespaces are PascalCased
namespace TheJoyOfCode.NamingConventions
{
    // Class names are PascalCased
    public class ExampleClass
    {
        // All public fields, including constants are PascalCased
        public static const string PiAsAString = "3.14";

        // All private fields are camelCased with an underscore [1]
        private readonly string _privateMember;

        // All protected members are PascalCased
        protected int ProtectedField = 12;

        // All internal members are PascalCased
        internal int InternalField = 13;

        // All private methods are PascalCased
        // *** NOTE - All parameters are camelCased
        private double Multiply(double valueA, double valueB)
        {
            // local variables (scoped within a method) are camelCased (no underscore)
            double result = valueA * valueB;
            return result;
        }

        // All private Properties are PascalCased
        // *** NOTE - Acronyms of 2 characters are UPPERCASED (e.g. UI, IO)
        private string UIElementName { get; }

        // All (public and private) properties are PascalCased
        // *** NOTE - Acronyms longer than 2 characters are PascalCased (e.g. Html, Xml)
        public int HtmlLength { get; set; }

        // All public methods are PascalCased
        // *** NOTE - All parameters are camelCased
        // *** NOTE - Abbreviations are not treated as Acronyms (so _Id_entification is Id, not ID).
        private void AlignObjectById(string id, Alignment alignment)
        {
            throw new NotImplementedException();
        }

        // Nested classes are PascalCased, even Private ones
        private class NestedClass : IDisposable
        {
            public void Dispose()
            {
                throw new NotImplementedException();
            }
        }
    }

    // Enums are PascalCased and not plural (unless marked [Flags] in which case the name should be plural)
    public enum Alignment
    {
        // Enum members are PascalCased
        Top,
        Bottom,
        Left,
        Right,
    }
}

// [1] - Note the underscore isn't as recommended by StyleCop but since it applies only to private members, can be considered a matter of style and one that I personally use.

... and as for #region blocks I do not use regions and I don't negotiate with terrorists either.

Originally posted by Josh Twist on 8 September 2009 here.

How to define and host workflow services with multiple receives

2009-11-03T13:00:00Z

Note – this article is based on Workflow 4 Beta 1, some things will change for Beta 2]

First off let me apologise for the length of this post. If you just want the code, skip to the bottom.

I don’t know about you but some words just don’t seem to make their way into my skull very easily, and correlation is one of them. It doesn’t matter how many times I say it in my head, even really slowly “c-o-r-r-e-l-a-t-i-o-n”, it still makes little sense.

So, in the spirit of sharing I thought I’d post what I know about correlation (in as far as it’s used in Workflow 4), and add in some code to cement the subject. First off though I thought I’d pop upstairs and look at the dictionary definition of correlation so here goes…

Correlate – have a relationship or connection in which one thing affects or depends on another
Correlation – connection, association, link, tie-in, tie-up, relationship, interrelationship, interdependence, interconnection, interaction

Thanks to the “Concise Oxford English Dictionary” for the former definition, and the “Oxford Paperback Thesaurus” for the latter. I hope they won’t mind me using their definitions here but if they do then someone else wrote this article, it wasn’t me, honest. As an aside, the “Concise OED” is a bit of an oxymoron if you see the physical size of it, but then again if you saw the full OED then you’d realise why this was called the concise version. Anyhow, enough English language for now!

So, how does correlation apply to a workflow then?

In the old days (well, pre Workflow 4), the common way to deal with a workflow instance was to know it’s instance ID which was a GUID. You had a workflow definition (i.e. some XML or a coded workflow), and when you created an instance of that workflow definition you had a workflow instance, and it was uniquely identifiable by its workflow instance ID. Simple.

If I wanted to do anything with a WF 3.x workflow, all I needed to know was it’s GUID and I could load it up, send messages to it, find it in the persistence database and so on. When I created a workflow instance I could optionally choose to assign my own GUID to it rather than having the system generate one for me. Life was good.

Now, lets say you were exposing a workflow as a service. You might have a few operations defined on that service and want to call these in whatever order suited you. You’ll define the first operation (which effectively kicks off the workflow), then subsequent operations will be called on the same instance of the workflow. In traditional programming you would maybe implement an interface something like the following…

[ServiceContract]
public interface IStudent
{
    [OperationContract]
    Guid EnrollStudent(string name, DateTime dob);

    [OperationContract]
    void AddExamResults(Guid studentId, string examName, int mark);

    [OperationContract]
    void Graduate(Guid studentId);
}

So the initial operation is EnrollStudent, and this returns the GUID of that student. Then some time later (days/weeks/months/years) you call AddExamResults and pass through the GUID in order to hook up the results with the right student. Finally at some point in the future the Graduate method is called (OK, maybe they flunked, but I’ll keep this simple). That’s all fairly easy to understand.

Now, if you were doing this in the workflow world things would be a little different. As of now (Workflow 4 Beta 1) you can’t do contract first workflow services, so you need to define the service contract within the workflow itself. Hopefully this limitation will be fixed by the time Beta 2 drops as I quite like contract first development these days.

In the service contract I defined above, the thing that identifies one student from another is the GUID returned from the initial EnrollStudent method. You can conceptually see that this key uniquely defines the student instance, and as long as you use that every time you want to ‘talk’ to that student instance then everything will be just fine.

Moving this service into the world of Workflow 4, we need some identifier that does the same job – it ties my requests to an instance of a workflow. Suppose I have two clients. The first calls EnrollStudent and this constructs an instance of my workflow and then returns the unique ID back to the caller. The second client can call EnrollStudent and will get a different ID and so now there will be two distinct workflow instances ‘running’ (they may be persisted to the database). Now when I get calls to the AddExamResults method I need to have these go to the appropriate workflow instance, and it’s correlation that does that. You don’t have to use GUIDs by the way – correlation in WF 4 allows you to define anything in the message as the correlation ‘handle’ – indeed you can use multiple items in the message as this handle which is very cool indeed. However for this example I’m using just one thing – a GUID.

So, finally, we have my definition of correlation as it applies to WF 4…

Correlation – hooks messages to the workflows they are intended to reach

OK, so that’s a fairly poor definition and I don’t think the OED will be knocking on the door any time soon asking me to write for them, but hopefully you get the drift. Enough waffle though, lets get to some code. I’ll try to emulate the above service contract with WF 4, and use code to call that service.

Example – Defining the Workflow

In this example I’ll use code throughout – no XAML here folks! First off I need to define some data contracts for the arguments I’m passing in to the service methods. These mimic the parameters I’m passing to the service contract I outlined above…

[DataContract(Namespace = "http://www.morganskinner.com")]
public class EnrollStudentArgs
{
    [DataMember]
    public string Name { get; set; }
    [DataMember]
    public DateTime DateOfBirth { get; set; }
}

[DataContract(Namespace="http://www.morganskinner.com")]
public class AddExamResultsArgs
{
    [DataMember]
    public Guid StudentId { get; set; }
    [DataMember]
    public string ExamName { get; set; }
    [DataMember]
    public int Mark { get; set; }
}

So here I have two data contracts, one for the EnrollStudent operation and another for the AddExamResults operation. I need to do it this way as there’s no way to define multiple arguments to the Receive activity at present (this will be coming in Beta 2). You would currently need to do this for every method that takes more than a single argument, and whilst it’s a bit of a pain to have to do it that’s just the way it is for now. Well actually that’s a bit of a lie, there is an activity that will do this in Beta 1 but some changes are afoot for Beta 2 so I decided to show you this method which works for now.

Now we need a workflow. For this example I’ve chosen to build the workflow up as follows…

An initial call to the EnrollStudent operation is made. This kicks off a new workflow and passes in the EnrollStudentArgs, consisting of the Name and DateOfBirth. This data is stored away within the workflow and a new GUID returned to the caller.
The workflow then sits in a loop, waiting for a call to AddExamResults or a call to Graduate.
- If AddExamResults is called the data is output to the console
- If Graduate is called then the loop will exit and the workflow will complete

So, the overall workflow will be defined roughly as follows …

I’ve simplified it a bit as in the workflow there are actually many more activities, but hopefully this image conveys the main structure. I’ll build the code up step by step and explain what the bits are as I go along.

The first thing you’ll need to define are some variables and the initial Receive activity…

// Correlation handle to link the initial receive to the subsequent send
Variable<CorrelationHandle> enrollHandle = new Variable<CorrelationHandle>();

// Arguments received from the initial call to EnrollStudent
Variable<EnrollStudentArgs> enrollArgs = new Variable<EnrollStudentArgs>();

// The generated student Id
Variable<Guid> studentId = new Variable<Guid>();

// The namespace I'm using for all of the contracts etc
string ns = "http://www.morganskinner.com";

// This Receive activity kicks off the whole workflow
Receive receive = new Receive
{
    CanCreateInstance = true,
    CorrelatesWith = enrollHandle,
    OperationName = "EnrollStudent",
    ServiceContractName = XName.Get("IStudent", ns),
    Value = new OutArgument<EnrollStudentArgs>(enrollArgs)
};

The CorrelationHandle is used to link the initial Receive and the subsequent SendReply together. Here I’ve also defined that the output value from the Receive should be stored away in a variable called enrollArgs. This variable will be globally scoped in the workflow (not that it needs to be – I could have defined it locally). I’ve also defined the studentId variable which will be used to store the unique Id created by the code. In a real solution you might just pass this Guid up from the client in the initial call – however that would have been less for me to demonstrate so I chose this method instead.

What we have defined is now nearly enough to so the first stage of the workflow – the initial Receive, the computation of the Guid and the SendReply to pass this Guid back to the client. I say nearly enough – there is one more thing we need to define up front but I’m going to gloss over that just for a moment and show the workflow up to this point…

return new Sequence
{
    Variables = { enrollHandle, enrollArgs, studentId },
    Activities =
    {
        receive,
        new WriteLine { Text = "Assigning a unique GUID" },
        new Assign<Guid>
        {
            To = new OutArgument<Guid>(studentId),
            Value=new InArgument<Guid> (Guid.NewGuid())
        },
        new WriteLine { Text = new InArgument<string>(env=>
            string.Format("GUID assigned was '{0}'", studentId.Get(env)))} ,
        new SendReply
        {
            CorrelatesWith = enrollHandle,
            Value = new InArgument<Guid>(studentId),
            Request=receive,
            AdditionalCorrelations = {{ "StudentIdQuery", operationHandle}},
            CorrelationQuery = new CorrelationQuery
            {
                SelectAdditional = { studentIdOutputQuery } 
            }
        }
    }
}

So here I’ve defined a Sequence, and inside it are the receive activity we’ve defined earlier (this cannot be defined inline as the SendReply activity needs a reference to it). I’ve added some WriteLine activities so that we can see what the workflow is up to, and then defined the SendReply which is where it all gets a bit hectic.

The first three properties are fairly simple – CorrelatesWith defines a correlation handle link between the SendReply and the earlier Receive. The Value is simply what we want to return from the SendReply, and Request=receive links this SendReply with the earlier Request activity.

Now, if you remember the API I wanted to expose, I want to return a Guid from the initial call and use this same Guid on subsequent calls to identify the workflow instance that should process that call. That’s what the AdditionalCorrelations and CorrelationQuery properties are defined for. What I’m defining here is a query that will strip out the return value (i.e. the Guid) and associate this with another correlation handle – in this case the operationHandle.

So, in order to get this to compile I need to define the following before defining the Sequence shown above…

// The correlation handle defined after the initial receive/send pair
Variable<CorrelationHandle> operationHandle = 
    new Variable<CorrelationHandle>();

// Define a message context
XPathMessageContext messageContext = new XPathMessageContext();
messageContext.AddNamespace("local", ns);

// This query retrieves the output argument sent from the SendReply
MessageQuerySet studentIdOutputQuery = new MessageQuerySet
{
    { "StudentId", new XPathMessageQuery("sm:body()/ser:guid", messageContext) }
};
studentIdOutputQuery.Name = "StudentIdQuery";

Here I’ve defined the second correlation handle which will be used by the subsequent calls from the client (AddExamResults and Graduate). Then I’ve defined the message context which is used when defining the XPathMessageQuery objects that are used within the MessageQuerySet. I’ve defined a namespace prefix of “local” to refer to the namespace of my classes – we’ll need this again later in the example when retrieving values passed up from the client.

The MessageQuerySet object is used to define which bits are stripped from the message. In this case I have just one named value - “StudentId”, and this comes from the rather cryptic “sm:body()/ser:guid” XPath statement.

Now I’m not here to give you a lesson in XPath (and indeed you wouldn’t want one from me either!). If like me you think that XPath is akin to Voodoo Magic then I wouldn’t blame you.

The sm:body()/ser:guid implies that I’m looking throughout the ‘body’ of the message message for the ‘guid’ element from the ‘ser’ namespace and returning that. If you look at the actual data returned from the SendReply activity on the wire you would see something like the following (I captured this using the most excellent Fiddler2 tool)…

HTTP/1.1 200 OK
Content-Length: 203
Content-Type: text/xml; charset=utf-8
Server: Microsoft-HTTPAPI/2.0
Date: Fri, 04 Sep 2009 09:15:52 GMT

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
    <guid xmlns="http://schemas.microsoft.com/2003/10/Serialization/">
      ddafc236-bec4-4745-bc6d-882f5ab3b051
    </guid>
</s:Body>
</s:Envelope>

Here you can probably infer that we’re using ‘ser’ as a prefix for the Serialization namespace. It makes sense now, but I don’t much like magic prefixes – there are a bunch that are currently defined and used within the framework but are not currently documented. The documentaton should be sorted before WF 4 is released. If you’re interested these constants currently live on the XPathMessageContext class.

With all that defined we’re at the point where we should have a working Receive and SendReply (and of course a few other activities). Now it gets even more interesting.

Now I want to be able to call either the AddExamResults or the Graduate methods. These both pass the Guid along on the wire, but the former also passes other information too. In order to get to the right workflow instance I therefore need to extract the student Id from these calls and match it up to my earlier extract.

For this to work I’ll need another couple of message query sets…

// Retrieve the AddExamResultsArgs.StudentId passed to AddExamResults
MessageQuerySet studentIdQuery = new MessageQuerySet
{
    { "StudentId", 
        new XPathMessageQuery
            ("sm:body()/local:AddExamResultsArgs/local:StudentId", 
            messageContext) }
};

// And this one retrieves the value passed in to the Graduate method
MessageQuerySet graduateId = new MessageQuerySet
{
    { "StudentId", new XPathMessageQuery("sm:body()/ser:guid", 
                                         messageContext) }
};

The first message query set defines the XPath that will strip out the value of the StudentId property from the AddExamResultsArgs class. You’ll see that I’ve included the ‘local’ namespace prefix for both of these elements – it’s necessary, you won’t get anywhere without it.

The second message query set is used for the call to the Graduate method – here I’m just passing up a simple Guid and so need to strip just that Guid out of the message.

With a couple more variables defined (a boolean flag ‘finished’ and a variable to record the exam results) and two more receive activities we’re pretty much sorted…

// A variable used to hold the exam results just added
Variable<AddExamResultsArgs> examResultArgs = 
    new Variable<AddExamResultsArgs> ( ) ;

// Flag indicating that we're done
Variable<bool> finished = new Variable<bool> { Default = false };
// This recieve activity waits for the AddExamResults operation to be called
Receive receiveAdd = new Receive
{
    OperationName = "AddExamResults",
    ServiceContractName = XName.Get("IStudent", ns),
    Value = new OutArgument<AddExamResultsArgs>(examResultArgs),
    CorrelatesWith = operationHandle,
    CorrelationQuery = new CorrelationQuery
    {
        Select = studentIdQuery
    }
};

// And this activity waits for the Graduate operation to be called
Receive receiveStop = new Receive
{
    OperationName = "Graduate",
    ServiceContractName = XName.Get("IStudent", ns),
    CorrelatesWith = operationHandle,
    CorrelationQuery = new CorrelationQuery
    {
        Select = graduateId
    },
    Value = new OutArgument<Guid>(studentId)
};

In the above I’ve defined the other receive operations that my workflow will process. These again need to define their operation names and the name of the service contract to which they belong, and the only other tricky part is to define the correlation information. The AddExamResults operation is passed an argument of the AddExamResultsArgs type, so I need to strip out the AddExamResultsArgs.StudentId in order to correlate this message with the workflow. That’s just what the CorrelatesWith and CorrelationQuery are doing.

The second operation is just passed a Guid, so again I’m using a query that will retrieve the Guid passed to the Graduate operation.

Finally we get to the rest of the workflow…

return new Sequence
{
    Variables = { enrollHandle, enrollArgs, studentId, 
                  finished, operationHandle },
        …
    new While
    { 
        Condition = ValueExpression.Create<bool>
            ( env => !finished.Get(env)),
        Body = new Pick
        { 
            Branches = 
            { 
                new PickBranch 
                {
                    Variables = {examResultArgs},
                    Trigger = receiveAdd,
                    Action = new WriteLine
                    { 
                        Text = new InArgument<string>
                          (env=>string.Format
                             ("Student '{0}' received '{2}' in '{1}'", 
                             examResultArgs.Get(env).StudentId, 
                             examResultArgs.Get(env).ExamName, 
                             examResultArgs.Get(env).Mark)
                          )
                    }
                },
                new PickBranch
                {
                    Trigger = receiveStop,
                    Action = new Sequence
                    {
                        Activities = 
                        {
                            new Assign<bool>
                            {
                                To = new OutArgument<bool>(finished),
                                Value = new InArgument<bool>(true)
                            },
                            new WriteLine { Text = new InArgument<string>
                                (env=>string.Format("Student {0} Graduated", 
                                    studentId.Get(env)))}
                        }
                    }
                }
            }
        }
    }
}
 
Here I’ve defined the While loop, and within it a Pick activity which is similar to the Listen from WF 3.x. It has any number of child branches and will wait until one of the branches completes. In this case my branches are awaiting calls to WCF methods that I’ve defined by using Receive activities.

The first branch schedules the Receive activity that awaits a call on the AddExamResults method (and writes this data out to the console when it arrives), and the second branch waits for the Graduate call to be made and then sets the boolean ‘finished’ flag so that the workflow completes.

Example – Hosting the Workflow

Now we have a workflow we need to host it. Again I’m just using code rather than XAML to show you all the stuff you need.

// Define the base address for the service
// Note - use the full name of the PC rather than localhost here
string serviceBaseAddress = "http://dev10-vpc:8080/StudentService";

using (WorkflowServiceHost host = 
          new WorkflowServiceHost(GetStudentWorkflow(), 
          new Uri(serviceBaseAddress)))
{
    // Define the service namespace
    string ns = "http://www.morganskinner.com";

    // And add the endpoint
    host.AddServiceEndpoint(XName.Get("IStudent", ns), 
                            new BasicHttpBinding(), 
                            serviceBaseAddress);

    ServiceMetadataBehavior smb = 
        host.Description.Behaviors.Find<ServiceMetadataBehavior>();
    // If none, add one
    if (smb == null)
        smb = new ServiceMetadataBehavior();

    // Setup properties of the service metadata behavior
    smb.HttpGetEnabled = true;
    smb.MetadataExporter.PolicyVersion = PolicyVersion.Policy15;

    // And set it
    host.Description.Behaviors.Add(smb);

    // Now open for business...
    host.Open();
    Console.WriteLine("StudentService is ready.");
    Console.ForegroundColor = ConsoleColor.Red;
    Console.WriteLine("Press <enter> to exit.");
    Console.ResetColor();
    Console.ReadLine();
    host.Close();
}
 
Here I’ve defined the base address for the service to include the physical name of the machine I’m running on (dev10-vpc). If you use ‘localhost’ it will still work fine, however Fiddler2 won’t be able to pickup the traffic and display it for you.

I then construct a WorkflowServiceHost and add an appropriate endpoint to it. After that I’ve defined a metadata behaviour to permit clients to get the service metadata over Http (I could have added a metadata endpoint/binding instead). After that it’s just a case of opening the service and waiting for calls. You don’t need a .config file as all of the configuration has been done in code.

The last (and you’ll be glad to know, easiest!) part is creating the client.

Example – Creating a client

Construction of the client is simple. Add a service reference to the service (in my instance http://dev10-vpc/StudentService) and then call it. I’ve deliberately used different client instances here to prove that

static void Main(string[] args)
{
    Console.WriteLine("Press a key to connect to the student service");
    Console.ReadLine();

    Guid studentId;

    using (StudentService.StudentClient client = new StudentService.StudentClient())
    {
        studentId = client.EnrollStudent(
            new EnrollStudentArgs { Name = "Fred Bloggs", 
                                    DateOfBirth = new DateTime(1980, 09, 04) }).Value;

        Console.WriteLine("Student Id returned was '{0}'", g);
    }

    Console.WriteLine("Now press a key to call the service again");
    Console.ReadLine();

    ReportExamResults(studentId, "Maths", 90);
    ReportExamResults(studentId, "English", 80);
    ReportExamResults(studentId, "Woodwork", 100);

    Console.WriteLine("Now press a key to call the Graduate operation");
    Console.ReadLine();

    // And finally graduate...
    using (StudentService.StudentClient client = new StudentService.StudentClient())
    {
        client.Graduate(studentId);
    }
}

private static void ReportExamResults(Guid studentId, string examName, int mark)
{
    using (StudentService.StudentClient client = new StudentService.StudentClient())
    {
        client.AddExamResults(new AddExamResultsArgs 
                              { 
                                  StudentId = studentId, 
                                  ExamName = examName, 
                                  Mark = mark
                              });
    }
}
 
So there you have it. A worked example of correlation in Workflow 4 Beta 1. If you want to download the code for this sample please click here.

Originally posted by Morgan Skinner on 09 September 2009 here.

Working with memory mapped files in .NET 4

2009-10-29T12:00:00Z

oI have been exploring different new features that come with the .NET 4, beyond the most popular ones like dynamic types and covariance; I was interested in performance enhancements. For this reason I am going to publish a couple of blog entries were I explore these different features.

Memory mapped files may sounds alien to the managed code developer but it has been around for years, what is more, it is so intrinsic in the OS that practically any communication model that wants to share data uses it behind the scenes.

So what is it? A memory mapped file allows you to reserve a region of address space and commit physical storage to a region (hmmm, sounds like virtual memory, isn’t it?) but the main difference is that the physical storage comes from a file that is already on the disk instead of the memory manager. I will say that it has two main purposes:

· It is ideal to access a data file on disk without performing file I/O operations and from buffering the file’s content. This works great when you deal with large data files.

· You can use memory mapped files to allow multiple processes running on the same machine to share data with each other.

The memory mapped file is the most efficient way for multiple processes on a single machine to communicate with each other. What is more, if we check other IPC methods we can see the following architecture:

Impressive, isn’t it? Now you have the power of this technology available on the System.IO namespace (instead of using the Pinvoke approach).

Now let’s quickly explore how it works. We have two types of memory mapped files models, one model is using a custom file, this can be any file that the application accesses it and the other one using the page file that we are going to share it with the memory manager (this is the model that most of the technologies above use).

Let’s explore the custom file model. The first thing that we need to do is to create a FileStream to the file that we are going to use, this can be an existing file or a new file (keep in mind that you should open this file as shared, otherwise no other process will be able to access it!). With the stream in place, we can now create the memory mapped file. Let’s see an example:

MemoryMappedFile MemoryMapped = MemoryMappedFile.CreateFromFile(

new FileStream(@"C:\temp\Map.mp", FileMode.Create), // Any stream will do it

"MyMemMapFile", // Name

1024 * 1024, // Size in bytes

MemoryMappedFileAccess.ReadWrite); // Access type

I have use one of the simplest constructor, we define the stream to use and we provide a name. The object needs to know the size of it in bytes and the type of access that we need. This will create the memory mapped file but to start using it we will need a map view. We can create one using the following syntax:

MemoryMappedViewAccessor FileMapView = MemoryMapped.CreateViewAccessor();

This map covers the file from the first byte until the end. If we need now to write or read information from it we just call the map view methods with the correct offset.

int Number = 1234;

FileMapView.Write(0, Number);

FileMapView.Write<Container>(4, ref MyContainer);

We can see that we can write built in types or custom types with the generic version. The good thing about the memory mapped files is persistence, as soon as you close it the contents will be dumped on the disk, this is a great scenario for sharing cached information between applications.

Now if we want to read from it, the other process needs also to create a memory mapped file, we can use the other static initialize that opens an existing one or creates one if it does not exist.

MemoryMappedFile MemoryMapped = MemoryMappedFile.CreateOrOpen(

new FileStream(@"C:\temp\Map.mp", FileMode.Create), // Any stream will do it
"MyMemMapFile", // Name

1024 * 1024, // Size in bytes

MemoryMappedFileAccess.ReadWrite); // Access type

Create the map view and read it:

using (MemoryMappedViewAccessor FileMap = MemoryMapped.CreateViewAccessor())

{

Container NewContainer = new Container();

FileMap.Read<Container>(4, out NewContainer);

}

That’s it, really simple isn’t it? Now, there is a small drawback with this approach and is related to the size of the memory mapped file. If you don’t know in advance you will need to create a large file
“just in case”, this can be a very large file with a lot of wasted space, it would be nice to have the ability to reserve the space instead of committing all of it, isn’t it?

In order to solve this issue you can use the page file, this has the advantage of allowing you to commit data on the fly but introduces another issue: you don’t own the file and the map will last until the last handle is destroyed. But think about it, for certain scenarios this is absolutely valid.

Now if we revisit the sample we will need to change some initialization parameters, for this particular one I will use the full constructor so I can introduce some other features that are also applicable to the custom files one.

MemoryMappedFileSecurity CustomSecurity = new MemoryMappedFileSecurity();

MemoryMappedFile PagedMemoryMapped = MemoryMappedFile.CreateNew(

@"Salvador", // Name

1024 * 1024, // Size

MemoryMappedFileAccess.ReadWrite, // Access type

MemoryMappedFileOptions.DelayAllocatePages, // Pseudo reserve/commit

CustomSecurity, // You can customize the security

HandleInheritability.Inheritable); // Inherit to child process

The memory mapped file security allows you to customize who or which process can have access to the resource, this can be quite important when you want to protect sensitive information and you don’t want other processes changing the file map. You can explore that object and see all the different settings that you can change. The reference is here. If we explore the construction of the memory mapped file we can see that there is no stream, we just name the resource. This will create an association between a section of the file based on the size and the map name, this is how both processes will access the file. Note as well that I am setting the property “DelayAllocatePages”, this implements a pseudo reserve/commit model where we will use the space once is needed. Finally, the other interesting parameter is the handle inheritance model, this will allow to share this resource with a child process if is required.

The access to the file uses the same syntax as the previous example, remember that if you close the memory mapped file this will be non accessible, this issue catches many developer.

Finally, another interesting area is the creation of multiple map views, these can work on the same memory mapped file accessing different areas of the files. This will allow you to properly protect the content and allowing you to synchronize the access.

You can do this defining different offsets and lengths when you create your map view.

MemoryMappedViewAccessor WriteMap = MemoryMapped.CreateViewAccessor(0, 1024,
MemoryMappedFileAccess.ReadWrite);

MemoryMappedViewAccessor ReadMap = MemoryMapped.CreateViewAccessor(1025, 1024,
MemoryMappedFileAccess.Read);

Now you can enjoy the power of this high performance data sharing model on your applications when you compile them with .NET 4.0! Note that this blog is based on beta 1 information, I will check the post once is released.

Originally posted by Salvador Alvarez Patuel on 10 June 2009 here.

Detecting Server.Transfer

2009-10-27T12:00:00Z

How do you know when a page is being rendered as the result of a Server.Transfer, rather than a Response.Redirect or the user browsing directly to a page?

Actually it’s quite easy, assuming you’re using the default ASP.NET pipeline. In reality the “thing” that is responsible for handling an HTTP request is aptly called an HttpHandler – that is, they implement the IHttpHandler interface. Of course, you can create your own handlers if you just want to return a document, or your own manually rendered content, or similar.

But... the Page class also implements IHttpHandler, and ASP.NET therefore uses some cleverness to work out which page should be rendered, and then uses an instance of that Page-derived class as the HttpHandler.

So the bottom line is that if you do a Server.Transfer, the HttpHandler for the request will be the page that was originally being rendered... Therefore, the following code when executed from within a page will display “true” if a Server.Transfer has occurred;

TransferredLabel.Text = (Context.Handler != this).ToString();

Easy huh? Try the attached if you want to see this in action.

Originally posted by Simon Ince on 13 July 2009 here.

New .NET Attributes

2009-10-22T09:00:00Z

The very first talk I gave on .NET was on custom attributes – some time way back in late 2000 or early 2001. At the time I was espousing the benefits of using custom attributes to add on details of unit tests and bug fixes.

There have been many additions to .NET since then and I spied a couple of *really* cool new attributes in .NET 4 the other day so had to share them.

Enter (drum roll please) :-

[Optional]
[DefaultParameterValue]

Yay!

These are (I believe) a real productivity boost, and will make your code much easier to read. I bet you have some code like the following hanging around, don’t you…

public class Thingy
{
    public Thingy()
        : this("Default", -1)
    {
    }

    public Thingy(string name)
        : this(name, -1)
    {
    }

    public Thingy(string name, int whatever)
    {
        _name = name;
        _whatever = whatever;
    }

    // Other code...

    private string _name;
    private int _whatever;
}

Now there’s nothing desperately wrong with this, although I feel

that 3 constructors is a bit ugly, I’d rather just have one. And

what if I want to change the default value for the ‘whatever’ value

– there are two places it’s used in the code above and I’d need to

update both which is a bit on the fragile side.

With these swanky new attributes I can have just the one constructor…

public class Thingy
{
    public Thingy([Optional, DefaultParameterValue("Default")] string name,
                  [Optional, DefaultParameterValue(-1)]int whatever)
    {
    }
}

Now that’s a whole lot better if you ask me! My defaults are in one place, I have only one constructor rather than three (obviously this works on methods too), and Intellisense and the C# environment understand the [Optional] attribute and present the appropriate stuff to me.

Now as if that wasn’t enough we’ve updated C# to support this without even having to write attributes (thanks to Wes Haggard for pointing this out to me)…

public class Thingy
{
    public Thingy(string name = "Default",
                  int whatever = -1)
    {
    }
}

Ooh, how cool is that? All in all I’m very happy with this little addition to .NET and C#.

Originally posted by Morgan Skinner on 05 August 2009 here.

Polymorphism in WCF

2009-10-20T12:00:00Z

Polymorphism in any OO language is taken as a given – and people would be shouting about it a lot if a language didn’t support it. However, when you’re writing a service interface with WCF you might also want to permit some form of polymorphic behaviour at runtime.

I was working with a customer yesterday and they wanted to be able to extend their system by adding in custom commands. A command might be something like “Print customer record” or “Add customer to ringback queue”. Some of these commands could be executed locally on the client PC, however others might need to be sent up to the server.

On the day I came up with some suggestions but had a nagging doubt that what I’d created wasn’t as great as it could be. Now after a sleep and some more thinking I’ve realised the error of my ways.

The typical Command pattern espoused by the GOF is to have a command object that knows how to execute itself. So in .NET you might dodge up an ICommand interface as follows :-

public interface ICommand
{
    void Execute();
}

Then you would create subclasses of this command object and be off on your merry way. Note that you could also use the ICommand interface that already exists in the framework – I’m a great one for reusing what’s already available and not re-inventing the wheel. That’s not really appropriate in this instance as I my commands will be remotely executed.

Now, what I want to do is have some commands executed on the server – for the sake of argument I’ll call these ‘Remote commands’. So when the Execute() method is called, what I actually want to call is a WCF service (and pass up something representing the command to the server). To do that I need to be able to pass up an arbitrary collection of arguments. I could use a dictionary but a discrete class feels better :-

/// <summary>
/// The command arguments class allows arbitrary commands to be executed
/// </summary>
[DataContract]
public class CommandArguments
{
}

And then I might need a derived class for adding a customer to a ringback queue…

[DataContract]
public class CustomerRingbackArguments : CommandArguments
{
    /// <summary>
    /// The ID of the customer we'll ring back
    /// </summary>
    [DataMember]
    public int CustomerId { get; set; }

    /// <summary>
    /// The duration until we call them back
    /// </summary>
    [DataMember]
    public TimeSpan RingbackWhen { get; set; }

    /// <summary>
    /// The reason for the call
    /// </summary>
    [DataMember]
    public string Reason { get; set; }
}
 
So far so good. Now I need a service that I can call and pass these command arguments to…

/// <summary>
/// This service executes commands
/// </summary>
[ServiceContract]
public interface ICommandService
{
    /// <summary>
    /// Execute the command associated with the passed arguments
    /// </summary>
    /// <param name="arguments">Arguments to be used by the command when executed</param>
    [OperationContract]
    void Execute(CommandArguments arguments);
} Execute(CommandArguments arguments);
}

If I were then to code up the service and try to use it I would not be able to pass up an instance of my CustomerRingbackArguments class as WCF doesn’t know about the type. Typically in this case you would attribute up the service as shown below…

[ServiceContract]
[ServiceKnownType(typeof(CommandArguments))]
public interface ICommandService

The problem here though is that for every new CommandArguments derived class I would need to update this service contract. With a couple of command argument derived classes then that wouldn’t be too much of an issue, but what if I were to have 20 or 200 different command argument classes? That would start to make my interface look pretty awful. There is a better way – and one I only noticed today.

The ServiceKnownType attribute can be used to indirectly specify which types are used by the service contract. So, rather than attributing up your code you can define a static method (even on another class) which returns all types that the service interface should use in (de)serialisation.

To use it just choose another overload of the ServiceKnownType attribute :-

[ServiceContract]
[ServiceKnownType("GetKnownTypes", typeof(CommandServiceHelper))]
public interface ICommandService

Now when WCF is looking for types it will call the static method GetKnownTypes on my CommandServiceHelper class. This method is defined as follows :-

public static IEnumerable<Type> GetKnownTypes(ICustomAttributeProvider provider)

All you need to do here is round up all of your CommandArguments derived classes and you’re away! Now you will also need some mechanism to determine just what happens on the server when you get a given CommandArguments derived class sent up to the server, but that’s typically going to be a simple type based lookup mechanism (aka dictionary). I’ll leave that up to you to sort out.

It just goes to prove yet again that old adage – every problem in computing can be solved by another level of indirection.

Originally posted by Morgan Skinner on 5 August 2009 here.

Executing JavaScript after a Partial Render

2009-10-15T12:00:00Z

I had a fun day debugging some ASP.NET plus jQuery this week, and came across something I’ve known is possible for some time, but that I’ve never actually needed to do... and that was to ensure a bit of JavaScript ran once an UpdatePanel had refreshed as part of a partial render.

It turns out that this is easy; all you need to do is register your JavaScript code block with some methods on ScriptManager, rather than using the standard ClientScript methods.

For example, consider the HTML below (I’ve omitted the rest of the page including the script manager, but it is in the download);

<asp:Button

ID="ClickMeButton"

runat="server"

Text="Click Me to Refresh Below"

onclick="ClickMeButton_Click" />

<asp:UpdatePanel

ID="MyUpdatePanel"

runat="server"

UpdateMode="Conditional">

<asp:AsyncPostBackTrigger

ControlID="ClickMeButton"

EventName="Click" />

</Triggers>

Last Updated:

<asp:Label

ID="UpdateTimeLabel"

runat="server" />

</ContentTemplate>

</asp:UpdatePanel>

</div>

</div>

Here we have a simple UpdatePanel, with a button that triggers the async postback. There is also an empty DIV with ID “log”.

In the server-side code of the button click event I have some code that looks a bit like this;

string time = DateTime.Now.ToLongTimeString();

ScriptManager.RegisterStartupScript(

UpdateTimeLabel,

typeof(Label),

"UpdateTimeLabelStartupScript",

CreateScript(time),

true);

The CreateScript function simply returns some JavaScript in a string that reads as follows (the {0} is replace with a time);

$('#log').append('Last Server Run: {0}<br />');

This uses jQuery to append a message to our “log” DIV.

There is also an alternative though; using the Sys.Application.add_load method to attach a page load script function, or creating a pageLoad JavaScript function that is a synonym for the same thing. I’ve done the latter;

function pageLoad() {

$('#log').append('pageLoad executed<br />');

}

This function is executed when the page initially renders and when every partial render completes. This means the content of my “log” DIV rapidly starts to look like this;

pageLoad executed

Last Run: 21:56:12

pageLoad executed

Last Run: 21:56:14

pageLoad executed

Last Run: 21:56:19

pageLoad executed

Easy huh?

I hope that adds some clarity to your options for executing script as the result of a partial postback. Check out the attached if you want to try it yourself.

Originally posted by Simon Ince on 29 July 2009 here.

Background and Foreground GC in .NET 4

2009-10-13T12:00:00Z

Another interesting new feature of the CLR 4 comes from the Garbage collection team. On this version, they are adding some performance enhancements on the memory allocation process. The feature is commonly called “Background GC”. But what does it actually mean?

As applications are starting to consume more memory and some of them moving to wider memory spaces under 64bits processes we have started to see some latency issues while allocating memory when the full GC is running. As you may remember, for workstation version of the CLR we use a concurrent GC. This means that the GC thread will run in parallel without blocking the application execution (well, we try to minimize the blocking time). This thread will scan Gen 2 in order to mark dead objects. This operation can take some time if the memory allocation is quite large and this prevents ephemeral collections while is running. Ok, now you may be asking what does it mean?, let me explain it with some graphics.

Let’s analyze how the current concurrency GC works and the scenario that we are improving:

Now, our application needs to perform a full GC, for this it will scan Generation 2 and try to mark the dead objects as free objects. This is executed by the GC thread, the simplified steps that it will take are the following ones:

1) It will start marking the objects, checking the stacks and the GC roots. This operation will allow further allocations, this means that your application may create a new object and this will be allocated in generation 0.

2) Now there are further allocations that the GC needs to suspend the EE (Execution engine) and this will stop all threads on your application. At this stage no allocation is allowed and your application may suffer some latency.

3) The EE is resumed in order to continue working on the heap and other bits and pieces that the GC needs to handle; at this stage the allocation is allowed. But what happen if our ephemeral segment is full while this collection happens?

4) At this stage the ephemeral collection cannot swap segments and the allocation will be delayed, adding latency to your application.

As you can see, the problem is that a single GC thread cannot cope with those two operations at the same time. The current ephemeral segment is 16mb (note that this may change in the future so don’t relay on this value!). This means that you can only allocate up to 16mb or whatever is available at the time of allocation, and as you have seen on the example this space may run out before the GC collection finishes! I hope now you understand why we don’t recommend you to call GC.Collect() without a good reason J.

Ok, now let me introduce you to the background GC. This model has been optimized to reduce the latency introduced by the scenario described above. The solution came from the idea of creating a background GC that works as it has described above and a foreground GC that will be only triggered when the ephemeral segment needs to be collected while performing a generation 2 collection.

Now, if we repeat the scenario above and we try to allocate memory on the ephemeral segment while the background GC is marking the foreground GC will execute an ephemeral collection:

The ephemeral foreground thread will mark the dead objects and will swap segments (as this is more efficient rather than copying the objects to generation 2. The ephemeral segment with the free and allocated objects becomes a generation 2 segment.

As you can see now the allocation is allowed and your application will not need to wait for the full GC to finish before allowing you the allocation.

Note that this enhancement is only available on workstation, as the server version is a blocking GC per core and we didn’t have enough time to port these enhancements into it, but is definitely in our plans in the near future. We have tested this solution in 64 cores but our future objective is to hit the 128 core mark as the SQL team has provenJ.

I hope this blog post makes a bit clearer how the GC works and what kind of enhancements we are including in .NET 4.

Originally posted by Salvador Alvarez Patuel on 10 June 2009 here.

Integration Services Design Principals

2009-10-08T12:00:00Z

Whilst doing some design work today for a customer project I realised there are a set of principals I try and adhere to when creating SQL Server Integration Services packages. The list is no doubt incomplete but this is what I have so far.

Minimise IO

This is a general data processing principal. Usually disk and, to a lesser extent, network performance determine the overall processing speed. Reducing the amount of IO in a solution will therefore increase performance.

Solutions that consist of multiple read-process-write steps should be redesigned into a single read-process-process-process-write step.

Prefer Sequential IO to Random IO

Disks perform at their best when sequentially reading or writing large chunks of data. Random IO (and poor performance) manifests when procedural style programming occurs - signs to look out for are SQL statements modifying/returning only few rows but being executed repeatedly.

Watch out for hidden random IO - for example, if you are reading from one table and writing to another in a sequential manor then disk access will still be random if both tables are stored on the same spindles.

Avoid data flow components that pool data

Data flow components work on batches of data called buffers. In most instances buffers are modified in place and passed down stream. Some components, such as "Sort" cannot process data like this and effectively hang on to buffers until the entire data stream is in memory (or spooled to disk in low memory situations). This increased memory pressure will affect performance.

Sometimes SQL is the better solution

Whilst the SSIS data flow has lots of useful and flexible components, it is sometimes more efficient to perform the equivalent processing in a SQL batch. SQL Server is extremely good at sorting, grouping and data manipulation (insert, update, delete) so it is unlikely you will match it for raw performance on a single read-process-write step.

SSIS does not handle hierarchical data well

Integration Services is a tabular data processing system. Buffers are tabular and the components and associated APIs are tabular. Consequently it is difficult to process hierarchical data such as the contents of an XML document. There is an XML source component but it's output is a collection of tabular data streams that need to joined to make sense.

Execute SSIS close to where you wish to write your data

Reading data is relatively easy and possible from a wide variety of locations. Writing data, on the other hand, can involve complex locking and other issues which are difficult to optimise on a network protocol. In particular when writing data to a local SQL Server instance, SSIS automatically used the Shared Memory transport for direct inter-process transfer.

Don't mess with the data flow metadata at runtime

It's very difficult to do this anyway but worth mentioning that SSIS gets it's stellar performance from being able to setup a data flow at runtime safe in the knowledge that buffers are of a fixed format and component dependencies will not change.

The only time this is acceptable is when you need to build a custom data flow programmatically. You should use the SSIS API's and not attempt to write the package XML directly.

Originally posted by James Snape on 30 June 2009 here.

Essential Tips On Kerberos for SharePoint Deployers

2009-10-06T12:00:00Z

This definitely isn't the first blog post on this topic, and it certainly won't be the last - but hopefully it will bring some peace and understanding to those struggling to get Kerberos working in a live SharePoint deployment.

First off, I must credit Martin Kearn's excellent posts here that were invaluable in getting me started - and will be invaluable to you too.

Why Use Kerberos?

There are a number of good resources that aid understanding Kerberos - the best (imho) being from Keith Brown's .NET security guide, published on the web. If you want to know what Kerberos is, what SPNs are, what delegation is etc. go read items 59-63 of his book here:

http://www.pluralsight.com/wiki/default.aspx/Keith.GuideBook.HomePage

But to summarise the key points from a SharePoint perspective, if you want better authentication performance than NTLM, use Kerberos. If you want to pass your user's credentials through to back end resources (like for example accessing the database exposed through a Business Data Catalog) - then you'll need Kerberos delegation.

Crucial SPN Knowledge

Lets just define the parts of an SPN first - an SPN looks something like this ( a typical SPN for a SQL Server in this case ):

MSSqlSvc/bob.domain.local:1433

Where "MSSqlSvc" is the service class, "bob.domain.local" is the host and 1433 is the port. The port part (including the colon) is optional.

SPNs are basically arbitrary strings that applications "know" how to assemble. They are stored in AD in a multi-valued attribute called servicePrincipalName on user principals. (Multi-valued attributes are what they sound like, attributes that can have none or more values and are typically displayed as a comma-delimited list.)

SetSPN is the tool you should use to create SPNs. When you use SetSPN -A you are just setting this attribute. It's perfectly valid to do this with adsiedit.msc or any other AD editing tool - I don't recommend you do this because SetSPN does some validation to ensure the SPN follows syntax conventions that a direct edit would bypass - I'm just making the point that there is no magic going on and that SPNs are just strings. To set an SPN (with SetSPN or otherwise) you must have permission to write to ServicePrincipalName attribute on the target principal (and that is your least privilege requirement) - although it's a good idea to be able to read them as well.

When a client application wishes to authenticate to a server application with Kerberos, the client must acquire a ticket from the KDC ( = Key Distribution Center which is a role played by Domain Controllers in Windows). To do this, it must tell the KDC which SPN it wants a ticket for. The KDC will take this SPN and look for a principal that has a matching string in its servicePrincipalName attribute.

At the point a couple of things can go wrong: (1) The KDC is unable to find a match in the AD. (2) The KDC finds more than one match in the AD. In either case, you're stuffed and Kerberos authentication will fail. At this point most applications will silently fall back to NTLM authentication and everything will appear to work - unless you need delegation or you have demanded Kerberos authentication. Not many apps demand Kerberos, they usually "Negotiate" an authentication protocol. If you are trying to delegate, then what often happens is that the machine at the end of the second hop gets a NULL identity turning up (see Keith Brown's explanations linked to earlier).

So remember: you can not have two identical SPNs registed to different accounts in your forest. If you do this, neither will work.

So how does the client know which SPN to ask for? Well that's very interesting. It turns out the answer is largely "because it does". The service class and host + port parts of an SPN are completely arbitrary. The client must therefore "know" these things to correctly form an SPN. The one thing it doesn't know (or need to know) is the identity of the principal running the service - the KDC figures this out and passes back a ticket that only the principal against which the SPN is registered will be able to decrypt.

In the case of Internet Explorer, the SPN requested is formed by using a service class of HTTP and a host as specified in the URL, converted to an FQDN (fully qualified domain name) if possible.

Here's the rub: IE never appends the port number to the host name when contacting the KDC for a ticket - even if your server is on a port other than port 80! So for all those SharePoint sites you have on non-80 ports, if you've been specifying a port number in your setspn command, then you haven't been using Kerberos!

Because of the way IE works, its typical to register two SPNs for every web-site. One with the netbios Host name and one with the FQDN host name - this keeps all the bases covered. Just don't put in the port!

So if I ask for http://moss:10000/sites/test for example, then IE will pass an SPN of "HTTP/moss.sharepoint.local" to the KDC (assuming moss is resolved to a box in my sharepoint.local domain) or just "HTTP/moss" if it can't be resolved.

UPDATE: Thanks to Brad Turner for his research on this point - it turns out that there was a fix released for IE that enabled a special registry key setting that changes IE's behaviour so the port number IS included. However, I don't recommend you do this because it is a client side fix, and it's probably hard to guarantee all your clients will have this setting. Even if you use Group Policy to enforce a registry setting, I still don't like it. If you really want to know the details, drop me a line; but be aware I will raise my eyebrow at you!

If you are using host-headers on your website, then use those in your SPN. Remember it's whatever is in the address bar that IE will use to request the SPN. A host-header is set on the dialog you get from the "Advanced..." button on the "Web Site" tab of a web site in IIS 6.0 Manager. It allows IIS to have multiple web-sites on the same port and IIS looks at the address (host header) requested by the client to work out which website to direct the request to. For example, in my MOSS installation I have something like:

portal.sharepoint.local, mysite.sharepoint.local, ssp.sharepoint.local etc. all pointing to the same IP address. The host header can be specified in Central Admin when creating web applications.

You Probably Need Multiple Host Headers

This throws up an interesting point that in most cases this means you will need multiple host-headers in order to be able to use Kerberos in a least privilege environment. Every application pool in a least privilege environment has its own identity. This means that your web applications, which are sitting in different application pools, can't be differentiated by simply using different ports. If you did this, since IE ignores the port when forming its SPN request, the SPNs for each application pool identity would have to be the same - and you can't register the same SPN against multiple accounts! (Oh how I wish this would throw an error!) Thus, a different host header is required to get a Kerberos ticket for each application pool identity in which your SharePoint web applications reside.

How Do I Know Its Working?

Good question. The easiest way is to download Kerbtray.exe (don't worry this says Windows 2000 - it works on all versions). This tool lets you examine the Kerberos tickets for the logon session of the current interactive user only. This means that you can't do something like "Run As" to see the tickets of a service - because you'll be in a different logon session even though its the same principal. However, you can run this tool in a couple of useful places:

Run it during initial set up of the farm. At this point the setup account connects to SQL server so you should see a ticket for your SQL box - MSSqlSvc/sql.sharepoint.local:1433 for example.
Run it on any client machine connecting to a SharePoint web site. You should see a ticket for the web site in the list - HTTP/portal.sharepoint.local for example.

Also, on SQL Server 2005 you can run a query to check how clients are authenticating. You should see "KERBEROS" in the auth_scheme column:

SELECT login_name, program_name, host_name, auth_scheme
FROM sys.dm_exec_connections C INNER JOIN sys.dm_exec_sessions S
ON C.session_id = S.session_id

Delegation

If you want to delegate then you need to do more than just create SPNs. You must allow the services that will delegate right to do so. This is described in Keith Brown's guide again. For SharePoint, the services that are most likely to need rights to delegate are the application pool accounts and the SSP service account. You almost certainly don't need to give the computer accounts delegation rights!

A common scenario is BDC delegation where you want your end-users to authenticate against a database directly when querying a BDC web-part for auditing purposes. In this case you just need the application pool hosting the site hosting the web part to have delegation rights - and that's it!

Hope that helps!

James

ADDENDUM:

Since writing this article the infrastructure update has added some new functionality that includes custom SPNs for SSPs. It's covered in a new technet article and you can read all about it here: http://technet.microsoft.com/en-us/library/cc263449.aspx#section14

Originally posted by James World on 20 April 2007 here.

Unshelving a Shelveset to a Different Branch in TFS

2009-10-01T12:00:00Z

[Note: In this post, I am assuming that you are familiar with the TFS Branching Guidance. If you haven't read the guide, I strongly recommend that you do so! I will be referring to the branch names suggested for the basic branch plan.]

Scenario: After an application is released to the production environment, a defect has been raised by the users. The production support team decides to fix this defect before the next version is released so they start by modifying the appropriate files in the REL branch to fix the defect. When the developer working on the defect finishes his job, he shelves those changes and asks the support team lead to review those changes before they are checked in. The support team lead analyses the changes and because of the risks involved, she talks to the users and they agree to wait until the release of the next version to get the fix.

Challenge: When you create a shelveset, the server path of each shelved file is stored with the shelveset so when you try to unshelve that shelveset, the files in the same branch are modified. You cannot unshelve the shelveset to a different branch such as DEV so you will need to do this manually by unshelving to the original branch and then manually copying the modified files across, which is time consuming and error prone.

Solution: TFS 2008 Power Tools includes a command line tool (TFPT.exe), which allows you to perform advanced operations in TFS. One of these operations is the unshelve command, which allows you to change the server path of the files in a shelveset during the unshelve operation. This means you can use this command to unshelve any shelveset to a different branch. In our scenario, the shelveset was originally created in the REL branch so the development team can use the TFPT tool to unshelve the shelveset to the DEV branch.

So let's assume that the support team have changed the source files in the REL branch and we want to unshelve this shelveset to the DEV branch. In order to do this, we need to:

- Make sure the TFPT command line tool is installed!
- Go to the command prompt and change the folder to a location that is mapped in the desired workspace.
- Run the following command:
TFPT.exe unshelve ReleaseHotFix1 /migrate /source:$/MyProject/REL /target:$/MyProject/DEV
Where ReleaseHotFix1 is the original shelveset name and $/MyProject/REL and $/MyProject/DEV are the server paths to the REL and DEV branches respectively.

When you run the unshelve command with the migrate option, the tool will prompt you to choose which version of the file (in origin or target) you want to keep. It will also allow you to perform a file merge if needed, which will be done in the merge tool.

The following diagram shows the sequence of events that need to happen across multiple branches in our scenario:

Events:

S1: Production support team shelve their changes in the REL branch.
U1: Development team unshelve the shelveset to the DEV branch by running the TFPT tool and using the shelveset migrate option.
C1: After resolving any potential conflicts in the DEV branch, the modified files are checked in.
RI1: At some point, the checked-in changeset will be reverse integrated (merged) back into the MAIN branch.

Originally posted by Mehran Nikoo on 07 July 2009 here.

Improving Performance of images used with 3D

2009-09-29T12:00:00Z

I've been working on some 3D using WPF and was making heavy use of some largish (around 1 megapixel) images with ImageBrushes applied to DiffuseMaterials. Naturally, being WPF there was some animation thrown in for good measure to make sure I'm working that processor like a bad boy.

All was going well, except for one annoying niggle that I couldn't fix. Whenever a new image was displayed on screen there was a notable stutter in the animation whilst *something* happened. For starters, I assumed this was the I/O time to read the image from disk but I tried pre-loading the images in all kinds of ways without any succcess.

Failing to get a result, I turned to some graphics heroes in the WPF team who immediately suspected it was the Fant scaler resizing the image on first load (I too thought he'd misspelt Font for a while). The quick fix, it transpires is to swap your ImageBrush for a VisualBrush.

Whoa! But VisualBrushes are slower than ImageBrushes. I know, but it works. The key to the trick is removing the Fant scaler and going for the faster, Linear scaler. So we went from:

<DiffuseMaterial.Brush>
    <ImageBrush ImageSource="{Binding Source}" />
</DiffuseMaterial.Brush>

... to ...

<DiffuseMaterial.Brush>
    <VisualBrush RenderOptions.CachingHint="Cache">
        <VisualBrush.Visual>
            <Image Source="{Binding Source}" RenderOptions.BitmapScalingMode="Linear" />
        </VisualBrush.Visual>
    </VisualBrush>
</DiffuseMaterial.Brush>

And now it's lightning fast and silky smooth. There's a little loss in quality of the scaled bitmap but you'd have to be looking closely to spot it. And you'll be too busy admiring the smooth animation to do so.

Originally posted by Josh Twist on 21 April 2009 here.

And now for something completely crazy: Binding without WPF

2009-09-25T12:00:00Z

I was struggling with a rather complex ViewModel (from Model-View-ViewModel or MVVM) that was actually composed of a number of ViewModels.

Most of the problems was internal notification of change - i.e one ViewModel wanted to know when another ViewModel changed. Sounds crazy but I'm comfortable it was a valid scenario. Because manually wiring up to INotifyPropertyChanged is about as much fun as nail-varnishing your eyeballs, I often work around the problem with a series of back references. However, it dawned on me that what I really want is WPF's awesome binding functionality. You know, it can bind deep on the properties of properties or properties and you don't have to worry about releasing event handlers to avoiud memory leaks.

If only we could use it without WPF - right in our ViewModels. And so my crazy experiment began...

From my brief foray into the internals of Binding (using Reflector and some contacts in the WPF product group) it seems that Binding is pretty coupled to DependencyProperties and there's no obvious way to use it without them. However, I managed to get my scenario working with the (semi-proud unveiling) of the BindingObject, which is used like this:

BindingObject bo = new BindingObject(sourceObject, "Property.That.IWant.ToBindTo", e => Changed(e));

And when the 'deep' property "Property.That.IWant.ToBindTo" on the sourceObject changes, the 'Changed' method is fired. The BindingObject is disposable (IDisposable) and this releases the references and unsubscribes the delegate so that everything can get Garbage Collected. Here's my slightly pokey looking implementation.

public class BindingObject : DependencyObject, IDisposable
{
    private bool _suppress;
    private Action<DependencyPropertyChangedEventArgs> _onChanged;

    public BindingObject(object source, string bindingPath, Action<DependencyPropertyChangedEventArgs> onChanged)
    {
        using (SuppressNotifications())
        {
            _onChanged = onChanged;

            Binding binding = new Binding(bindingPath);
            binding.Source = source;
            BindingOperations.SetBinding(this, ValueProperty, binding);
        }
    }

    public object Value
    {
        get { return (object)GetValue(ValueProperty); }
        set { SetValue(ValueProperty, value); }
    }

    public static readonly DependencyProperty ValueProperty =
        DependencyProperty.Register("Value", typeof(object), typeof(BindingObject), new UIPropertyMetadata(null, ValueChangedCallback));

    private static void ValueChangedCallback(DependencyObject d, DependencyPropertyChangedEventArgs e)
    {
        var instance = (BindingObject)d;
        Action<DependencyPropertyChangedEventArgs> onChanged = instance._onChanged;
        if (onChanged != null && !instance._suppress)
        {
            onChanged(e);
        }
    }

    public void Dispose()
    {
        BindingOperations.ClearBinding(this, ValueProperty);
        _onChanged = null;
    }

    public IDisposable SuppressNotifications()
    {
        return new Supresser(this);
    }

    private class Supresser : IDisposable
    {
        private BindingObject _bindingObject;

        public Supresser(BindingObject bindingObject)
        {
            _bindingObject = bindingObject;
            _bindingObject._suppress = true;
        }

        public void Dispose()
        {
            _bindingObject._suppress = false;
        }
    }
}

However pokey it looks though, it dramatically simplified the scenario I was tackling. It works in unit tests (with the total absence of WPF) too.

Thoughts please?

Originally posted by Josh Twist on 2nd March 2009 here.