Hani Tech

Using a 32bit Native DLL in Windows Azure

hania — Fri, 20 Mar 2009 10:24:00 GMT

With the Mix2009 release of Windows Azure tools, we now support native code execution. This also includes the ability to debug native code. As you would imagine, you can do PInvoke’s to system DLL’s or to your packaged native DLL with your role.

The image in Windows Azure runs on a 64bit OS. Role hosts (web and worker hosts) are also 64 bit executables which result in your web and worker roles runing as 64bit binaries. That means if you want to PInvoke directly into your native DLL, your native DLL must be compiled for 64bit platform, otherwise you would get the error:

An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)

The simplest solution is to rebuild the assembly we are trying to load to 64bit, but there are some cases that we cannot do that like if we don’t own the source code. One way is to host the 32bit dll in a 32bit program that runs on WoW. Then we can use WCF to marshal all calls between the role process (which is 64bit) and our out-of-proc dll (which is 32bit).

I will show you an example on how to do that with a WebRole. My demo involves a 32bit NativeLibrary.dll that contains an Add function; Also, a managed DllHost86.exe that acts as a WCF server and as a host for the Native Library. The managed DllHost86 is configured to be built for x86 specific:

Both the NativeLibrary and the Dll-Host are configured to output their binaries to the webrole Redist folder. Also, we added these files as contents to the web role project to make sure they are packaged when launching under the devfabric or deployed to the Cloud:

The interactions between the Dll-Host and the web role goes like the following:

On any request to the web role (using Application_BeginRequest event), we check if the DllHost86 process is running.
1. If not running, launch the process.
Also, on any request to the web role, try to connect the WCF client to the WCF server hosted inside the DllHost86 process. Once connected, cache the client object in the server current session (Application session). Next time we actually get the client from the session and just check if the connection is valid.
The binding used on WCF is NetNamedPipeBinding for optimal performance on the same machine communication.
The time out on the WCF link is set to infinity.

The code (from Global.asax) is shown next:

private const int ClientInitTimeOut = 20; // in seconds

protected void Application_BeginRequest(object sender, EventArgs e)

    // Make sure that our dll host is running

    EnsureDllHostRunning();

    // Make sure the client is connected

    EnsureCalcServiceClientConnected();

private void EnsureDllHostRunning()

    Process[] p = Process.GetProcessesByName(Path.GetFileNameWithoutExtension(dllHostPath));

    if (p.Length == 0)

        Application["CalcServiceClient"] = null;

        ProcessStartInfo psi = new ProcessStartInfo(dllHostPath);

        Process dllHost = Process.Start(psi);

private void EnsureCalcServiceClientConnected()

    CalcServiceClient client;

    client = (CalcServiceClient)Application["CalcServiceClient"];

    if (client == null || client.State != System.ServiceModel.CommunicationState.Opened)

        client = GetCalcServiceClient();

        Application["CalcServiceClient"] = client;

private CalcServiceClient GetCalcServiceClient()

    CalcServiceClient serv = null;

    int retryCount = 0;

    bool connected = false;

    while (retryCount < ClientInitTimeOut * 10)

try

            EndpointAddress address = new EndpointAddress("net.pipe://localhost/CalculatorService");

            NetNamedPipeBinding binding = new NetNamedPipeBinding();

            binding.ReceiveTimeout = TimeSpan.MaxValue;

            serv = new CalcServiceClient(binding, address);

            serv.Open();

            Debug.WriteLine("state = " + serv.State);

            if (serv.State == System.ServiceModel.CommunicationState.Opened)

                connected = true;

                break;

        catch (Exception e)

            Debug.WriteLine(e.Message);

        retryCount++;

        System.Threading.Thread.Sleep(100);

    if (!connected)

        throw new TimeoutException("Couldn't connect to the calculator service.");

    return serv;

So far, we setup the web role to startup the dll host and setup the client connection to the WCF server inside the DllHostx86. The code there is just a standard code that uses ServiceHost and just waits forever. Here is the code from the Program class:

        static void Main(string[] args)

            Uri address = new Uri("net.pipe://localhost/CalculatorService");

            NetNamedPipeBinding binding = new NetNamedPipeBinding();

            binding.ReceiveTimeout = TimeSpan.MaxValue;

            using (ServiceHost host = new ServiceHost(typeof(Calculator)))

                host.AddServiceEndpoint(typeof(ICalcService), binding, address);

                ServiceMetadataBehavior metadata = new ServiceMetadataBehavior();

                host.Description.Behaviors.Add(metadata);

                Binding mexBinding = MetadataExchangeBindings.CreateMexNamedPipeBinding();

                Uri mexAddress = new Uri("net.pipe://localhost/CalculatorService/Mex");

                host.AddServiceEndpoint(typeof(IMetadataExchange), mexBinding, mexAddress);

                host.Open();

                Console.WriteLine("The receiver is ready");

                Console.ReadLine();

So, the service we expose contains an Add methods that is implemented internally to call the native method using PInvoke. Here is a sample of the implementation:

    class Calculator : ICalcService

        [DllImport("NativeLibrary.dll", EntryPoint="Add")]

        static extern UInt32 NativeAdd(UInt32 a, UInt32 b);

        #region ICalcService Members

        public int Add(int num1, int num2)

            Console.WriteLine("Received numbers: {0}, {1}", num1, num2);

            return (int)NativeAdd((uint)num1, (uint)num2);

        #endregion

I tested the sample above in the cloud and it works fine. I also attached it for your reference. Here is how it looks like:

So, in this blog, I demonstrated how to use a 32bit dll in the Cloud by means of hosting it under a 32bit process and then use WCF communication to do the marshaling.

Hope it helps.

Getting rid of Warning VSP2013 in MSBuild

hania — Wed, 04 Feb 2009 22:58:09 GMT

Running instrumentation and tests as part of the build machine could produce the VSP2013 warning. Our build machine is 64 bit and the VSInstr.exe tool seems to only work on platform dependent assemblies instead of just platform diagnostic as usually your assemblies are built using the platform (Any CPU).

The default vsinstr.exe used is the 32bit version even if you are instrumenting on a 64bit machine. This is because MSTest.exe and Visual Studio are both 32bit applications. So, if MSTest is going to run your tests it should operate on a 32bit or platform independent assemblies. Yet, you can change the default to use the 64bit version of vsinstr.exe by changing this registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\9.0\InstalledProducts\Instrumentation\Location

to point to the x64 folder. In my machine it’s: C:\Program Files (x86)\Microsoft Visual Studio 9.0\Team Tools\Performance Tools\x64

This should remove the warning. But as I said, if you are running your tests using MStest.exe as part of the build machine, this won’t allow your tests to run.

Off course, the easiest way to fix the warning without any hacks is to build your assemblies using the platform x86. If you don’t want to do that because you still want to produce platform independent assemblies, you can do this simple hack.

Write a command line file in the same folder of the VSInstr.exe tool that calls VSInstr.exe with –nowarn:2013.

Here are the steps:

Create the file “VsInstrNoWarn2013.cmd” under “C:\Program Files (x86)\Microsoft Visual Studio 9.0\Team Tools\Performance Tools”
Write the following in the command file:
"%~dp0\VsInstr.exe" -NOWARN:2013 %1 %2 %3 %4 %5 %6 %7 %8 %9
Change the key value of “HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\9.0\InstalledProducts\Instrumentation\VSInstr” from VsInstr.exe to VsInstrNoWarn2013.cmd

In this way, MSTest will execute VSInstrNoWarn2013.cmd instead and all arguments will be forwarded.

I tested this and seems to work fine as part of a build machine. I haven’t tested this as part of a regular Visual Studio run.

Statement Translation using Functional Programming in C#

hania — Sun, 01 Feb 2009 10:16:00 GMT

I will demonstrate here how we can use the functional programming feature in C# to program something like a statement/expression translation. Basically, we will translate a statement tree into a function that we can call to run the program. I am not using Linq Expression as demonstrated in my previous blog. Instead, I will show how to use functional programming.

The expression list we support is a very small subset of a real programming language, the definition in detail is:

int	an integer constant
variable	variable reference
plus	expression + expression
multiply	expression * expression

The classes are:

    // base class for the expression node

    abstract class Expression

    // defines a constant integer

    class Int : Expression

        public Int(int i)

            Value = i;

        public int Value { get; set; }

    // defines a variable reference

    class Variable : Expression

        public Variable(string name)

            Name = name;

        public string Name { get; set; }

    // defines a plus expression

    class Plus : Expression

        public Plus(Expression left, Expression right)

            Left = left;

            Right = right;

        public Expression Left { get; set; }

        public Expression Right { get; set; }

    // defines a multiplication expression

    class Multiply : Expression

        public Multiply(Expression left, Expression right)

            Left = left;

            Right = right;

        public Expression Left { get; set; }

        public Expression Right { get; set; }

Our supported statements are just a very small subset of the whole programming language. It contains, an assignment, if, while, skip, and a sequence of statements. The definition in detail is:

Assignment	variable := expression
if	if (expression) then statement else statement
while	while (expression) then statement
seq	statement, statement
skip	does nothing, e.g. could be used with the else statement.

The classes that represent our statements:

    abstract class Statement

    class Skip : Statement

    class Seq : Statement

        public Seq(Statement s1, Statement s2)

            S1 = s1;

            S2 = s2;

        public Statement S1 { get; set; }

        public Statement S2 { get; set; }

    class Assign : Statement

        public Assign(string variable, Expression expression)

            Variable = variable;

            Expression = expression;

        public string Variable { get; set; }

        public Expression Expression { get; set; }

    class If : Statement

        public If(Expression condition, Statement @then)

            : this(condition, @then, new Skip())

        public If(Expression condition, Statement @then, Statement @else)

            Condition = condition;

            Then = @then;

            Else = @else;

        public Expression Condition { get; set; }

        public Statement Then { get; set; }

        public Statement Else { get; set; }

    class While : Statement

        public While(Expression condition, Statement body)

            Condition = condition;

            Body = body;

        public Expression Condition { get; set; }

        public Statement Body { get; set; }

After we defined our language, we can just use it to construct our program, or use a help of a scanner/parser. What we care here is just to translate both expressions and statements nodes.

The way we are going to access our variables is through a heap. Our heap is a function that takes a variable name as a string and returns an integer (I assume we deal only with integer types)

heap := var => int

We will translate an expression into a function that takes a heap (heap function) and returns an integer value. For example, the addition expression will use the heap to return the result of the sum.

expression := heap => int

Statement will get translated into a function that takes a heap and returns another heap. For example, the assignment statements, will take the current heap and return a new heap that contains the new variable.

statement := heap => heap

The HeapFunc is declared as a delegate:

        delegate int HeapFunc(string variable);

The code for expression translation is simple and straight forward:

        // returns a function that takes a heap and returns an integer

        Func<HeapFunc, int> TranslateExpression(Expression e)

            if (e == null)

                throw new ArgumentNullException("Expression cannot be null");

            if (e is Int)

                // whatever the heap is, return the value

                return (h => ((Int)e).Value);

            else if (e is Variable)

                // use the heap to lookup the variable name

                return (h => h(((Variable)e).Name));

            else if (e is Plus)

                // translate the left expression into a function

                var f1 = TranslateExpression(((Plus)e).Left);

                // translate the right exression into a function

                var f2 = TranslateExpression(((Plus)e).Right);

                // run the functions on the given heap to get integers and sum them

                return (h => f1(h) + f2(h));

            else if (e is Multiply)

                // similar to the plus expression

                var f1 = TranslateExpression(((Multiply)e).Left);

                var f2 = TranslateExpression(((Multiply)e).Right);

                return (h => f1(h) * f2(h));

            else

                throw new NotSupportedException("Not supported expression.");

The code for statement translation is shown next:

        // returns a function that takes a heap and returns a heap.

        Func<HeapFunc, HeapFunc> TranslateStatement(Statement s)

            if (s == null)

                throw new ArgumentNullException("Expression cannot be null");

            if (s is Skip)

                // just return the same heap

                return (h => h);

            else if (s is Assign)

                Assign assign = (Assign)s;

                // translate the expression

                var f = TranslateExpression(assign.Expression);

                // the resulted heap will check if the variable is the one responsible for

                // if so return the result of the expression, else use the previous heap to look up.

                return (h) => (var => var == assign.Variable ? f(h) : h(var));

            else if (s is Seq)

                Seq seq = (Seq)s;

                // translate the first statement

                var f1 = TranslateStatement(seq.S1);

                // translate the second statement

                var f2 = TranslateStatement(seq.S2);

                // apply the first translation on the heap, which will return a new heap

                // that will be applied to the second translation.

                return (h) => f2(f1(h));

            else if (s is If)

                If @if = (If)s;

                // translate the condition expression

                var fcond = TranslateExpression(@if.Condition);

                // translate the then statement

                var fthen = TranslateStatement(@if.Then);

                // translate the else statement

                var felse = TranslateStatement(@if.Else);

                // apply the heap to the condition function. If the value is not zero, use

                // the heap returned by applying fthen, otherwise use the heap returned by

                // applying felse.

                return (h) => (fcond(h) != 0 ? fthen(h) : felse(h));

            else if (s is While)

                While @while = (While)s;

                // translate the condition expression

                var fcond = TranslateExpression(@while.Condition);

                // translate the body statement

                var fbody = TranslateStatement(@while.Body);

                // apply the body multiple times then return the final heap.

                return (h) => {

                    while (fcond(h) != 0)

                        h = fbody(h);

                    return h;

};

            else

                throw new NotSupportedException("Statement not supported.");

Sample:

I wrote a simple example to show how we can translate a simple program into a functional code. My example returns the sum of numbers between 0 and 10.

        // this is our initial heap. our heap is empty, so calling this function

        // sould just fail because we have no variable

        int InitialHeapFun(string variable)

            throw new Exception("cannot find variable '" + variable + "'");

        void Run()

            // sum := 0

            // i := 10

            // while (i)

            //  sum := sum + i

            //  i := i - 1

            // end

            Statement myProgram = new Seq(

                new Assign("sum", new Int(0)),

                new Seq(

                    new Assign("i", new Int(10)),

                    new While(new Variable("i"),

                        new Seq(

                            new Assign("sum", new Plus(new Variable("sum"), new Variable("i"))),

                            new Assign("i", new Plus(new Variable("i"), new Int(-1)))

);

            // translate the program into (heap=>heap)

            var f = TranslateStatement(myProgram);

            // run the program using our initial heap

            var heap = f(InitialHeapFun);

            // print the sum value. this outputs 55

            Console.WriteLine(heap("sum"));

There are a lot of other cool examples that would benefit from this method of translation. For example, we could translate an XML file that contains different actions. The result function will run much faster using just function calls. You can think of it as an XML compiler.

The code is attached for your convenience.

Silverlight debugging in Windows Azure

hania — Fri, 16 Jan 2009 20:27:10 GMT

To debug silverlight applications and controls in Windows Azure, you need to have the following components installed:

1- SilverLight 2 and SilverLight tools for Visual Studio (Most likely you already have these since you are developing Silverlight in Visual Studio.

2- January 2008 CTP of both the Azure SDK and Visual Studio Tools.

To enable Silverlight debugging, go to the web role’s properties and then to the Web page. Make sure you enable Silverlight in the debuggers section at the bottom of the page.

If Silverlight debugging is enabled you cannot debug any script in the page. To debug your script, you need to disable Silverlight debugging as you cannot debug both at the same time.

Generating the Lambda Expression dynamically

hania — Wed, 29 Oct 2008 01:53:00 GMT

The last post, I showed a simple way to build a lexicon scanner in .NET. We can now extend on the scanner and build a parser that generates a Lambda expression that can be compiled and called for further use. The parser I used is just a top-down recursive parser that builds up the Expression tree. The main method is BodyExpression that returns the Expression object.

The class LambdaCalc, holds all the logic. You can pass a string that will get parsed into a lambda expression. For example:

LambdaCalc lc = new LambdaCalc("(a, b) => -a + 2 * b");

Console.WriteLine(lc.Function(1, 2));

Function is a property of a delegate type declared as:

public delegate double Func(params double[] args);

So, the way we pass arguments to our lambda calculator is through an array of doubles. Internally, in the generated lambda expression, all parameter references are converted into array references. The previous example would generate the following lambda expression:

args => (-args[0] + (2 * args[1]))

To generate a Lambda Expression, you need to use one of the Lambda static methods inside the Expression class. Using the generic version, allows us to strongly define the delegate that the lambda expression will map to. For our case, this is how I generated the Lambda expression:

Expression<Func> lambdaExpr = Expression.Lambda<Func>(body, this.funcParamsArg);

body: is an expression that got parsed using our BodyExpression method.
funcParamsArg: is of type ParameterExpression and it’s the argument that will get used by the lambda expression.

The BodyExpression method eventually calls also other methods inside the Linq Expression class, for example, we use Expression.Add(leftExpr, rightExpr), to generate the additive expression, and the same goes for multiplication and so on.

The thing that worth mentioning is that in order to use the argument from inside the body, you need to use the same ParameterExpression object, otherwise you would get the following exception when compiling your Lambda Expression:

System.InvalidOperationException: Lambda Parameter not in scope

I attached the code bellow for the parser. You need the code from LambdaCalcScanner for this to work.

Writing a simple lexicon scanner in .NET

hania — Fri, 24 Oct 2008 08:36:00 GMT

I came across Linq and the new Lambda Expression classes in .NET 3.0. I thought, it would be a good idea to dynamically build a lambda expression from a math string, something like: (a + b) * 1.5. The string is parsed into a lambda expression and then is compiled into a delegate. Then, if you well, you can call the delegate to get the result of the formula.

So, in other words, we are building a scanner and a parser. The first part that I will talk about here is building a scanner using the Regex class in .NET.

The tokens that we need to scan for are defined next:

// Tokens that represent the input

internal enum Token

    OpenParan, CloseParan,

    Arrow,

    Comma,

    Plus, Minus, Multiply, Divide,

    Constant,

    Variable,

    Other       // Represents unrecognized charachters

The regular expression pattern then is defined next:

// The pattern used with the regular expression class to scan the input

const string Pattern = @"

        (?'OpenParan' \( ) | (?'CloseParan' \) ) |

        (?'Arrow' => ) |

        (?'Comma' ,  ) |

        (?'Plus' \+ ) | (?'Minus' - ) | (?'Multiply' \* ) | (?'Divide' / ) |

        (?'Constant' (\.\d+|\d+(\.\d+)?) ) |

        (?'Variable' [a-zA-Z]\w* ) |

        (?'Other' [^ \r\n\t])";

// Regular expression used to scan the input

private static Regex MathRegex = new Regex(Pattern, RegexOptions.IgnoreCase | RegexOptions.IgnorePatternWhitespace | 
RegexOptions.Singleline | RegexOptions.Compiled);

The pattern basically tries to find any of the groups and name each group by the given name. We also assign the group ‘Other’ to anything else that we couldn’t match.

The scanner then can be implemented as an enumerable method that returns the found tokens.

// Enumurable to get tokens from the given expression (scanner)

public static IEnumerable<TokenEntity> GetLambdaCalcTokens(this string exp)

    Token[] tokens = Enum.GetValues(typeof(Token)).OfType<Token>().ToArray();

    foreach (Match m in MathRegex.Matches(exp))

        // Check which token is matched by this match object

        foreach (Token token in tokens)

            if (m.Groups[token.ToString()].Success)

                yield return new TokenEntity(

                    token,

                    m.Index,

                    m.Value);

    // return the end string token, to indecate we are done

    yield return new TokenEntity(Token.Other, exp.Length, "\0");

The only problem with using enumerable is that it doesn’t allow us to peek forward without moving next. This could be a problem with complex parsers, but the parser that I am writing here is simple enough that can be written without this feature.

The previous method is written as an extension on the string, so it can be easily called on any string like “(a) => a + 1.5”.GetLambdaCalcTokens()

The class TokenEntity is defined as:

// Holds token info

internal class TokenEntity

    public TokenEntity(Token token, int startPos, string value)

        this.Token = token;

        this.StartPos = startPos;

        this.Value = value;

    // Token type

    public Token Token { get; private set; }

    // Start position in the original string

    public int StartPos { get; private set; }

    // Value

    public string Value { get; private set; }

    public override string ToString()

        return string.Format("{0} at {1}: {2}", Token, StartPos, Value);

Running the previous code on: "(a, b) => ; a + b * 0.1" , results with the following tokens:

OpenParan at 0: (
Variable at 1: a
Comma at 2: ,
Variable at 4: b
CloseParan at 5: )
Arrow at 7: =>
Other at 10: ;
Variable at 12: a
Plus at 14: +
Variable at 16: b
Multiply at 18: *
Constant at 20: 0.1
Other at 21:

I am attaching the source code for the simple scanner. I will follow next with the parser and how we can build lambda expression dynamically.