Haytham speaks from Microsoft

Writing proper async code

Kasparov — Sun, 12 Jun 2011 23:03:13 GMT

Alsalam alikom wa ra7mat Allah wa barakatoh (aka peace upon you)

Asynchronous calls have always been there for more efficient use of CPU time. You run something asynchronously when you know it’ll take some time to execute and you don’t want to block your current thread while waiting for that code (In some cases you actually do want to block the thread but then spanning another thread is, in this case, more or less a waste of resources In My Humble Opinion).

For the purpose of this post, let’s take as an example downloading a remote file, and printing the contents of the file to screen.

I would imagine this as one way to implement it:

private void TestSync()
{
    Console.WriteLine("Starting Test() " + GetCurrentContext());
    WriteLinePageTitle(DownloadStringSlowNetworkSync(new Uri("http://sample.com")));
    Console.WriteLine("Test() is done with big delay " + GetCurrentContext());
}

private string DownloadStringSlowNetworkSync(Uri uri)
{
    Thread.Sleep(500);
    return new WebClient().DownloadString(uri);
}
This is an example for a synchronous operation. TestSync() will not return till the string is downloaded. Here is a run:

As you can see, it all ran on the same thread.

If a UI component is calling TestSync() this means execution will block UI. And that’s the typical use of of Asynchronous method.

Here is the typical way to implement async:

private void TestAsync1()
{
    Console.WriteLine("Starting Test() " + GetCurrentContext());
    BeginDownloadStringSlowNetworkAsync(
        new Uri("http://sample.com"),
        (sender, e) =>
        {
            WriteLinePageTitle(e.Result);
        },
        (ex) =>
        {
            WriteLinePageTitle("Exception is thrown: " + ex.Message);
        });
    Console.WriteLine("Test() is done with no delay " + GetCurrentContext());
}

private void BeginDownloadStringSlowNetworkAsync(
    Uri uri, 
    DownloadStringCompletedEventHandler completedCallback, 
    Action<Exception> errorCallback)
{
    Thread.Sleep(500);
    try
    {
        var client = new WebClient();
        client.DownloadStringCompleted += completedCallback;
        client.DownloadStringAsync(uri);
    }
    catch (Exception ex)
    {
        errorCallback(ex);
    }
}

This is a little bit better, now UI is not blocked (notice UI running in thread 1 continued working fine)… 
There are 2 down sides of this implementation,
1- Code isn’t intuitive, it’s way different than the counter sync code.
2- Exception handling is not propagating well enough, stack trace isn’t preserved for instance.

Now with the better version, using Async Library.

public async Task TestAsync2()
{
    Console.WriteLine("Starting Test() " + GetCurrentContext());
    RunAsync();
    Console.WriteLine("Test() is done with no delay " + GetCurrentContext());
}

private async void RunAsync()
{
    WriteLinePageTitle(await DownloadStringTaskSlowNetworkAsync(new Uri("http://sample.com")));
}

public async Task<string> DownloadStringTaskSlowNetworkAsync(Uri address)
{
    await TaskEx.Delay(500);    // Simulate 500ms of network delay
    return await new WebClient().DownloadStringTaskAsync(address);
}

You will notice a couple of things here:
1- The result is identical to the async code we wrote above
2- Code looks intuitive, very similar to sync version
3- There are two new keywords async and await. And that’s what I actually want to talk about..

async: you decorate your methods with async to be able to call await inside them.
await: you prefix method calls with await when you want the library to create callbacks for you to handle async operation.

In DownloadStringTaskSlowNetworkAsync() method, we call DownloadStringTaskAsync, which is an extension method included in Async library. What await does is that it creates a callback method that includes the rest of your code (return statement in this case), then calls the method DownloadStringTaskAsync() and registers that callback, when the callback is invoked (ie when the string is actually downloaded), it invokes the rest of your code (return… etc).

Going one level up, RunAsync(), it awaits on DownloadStringTaskSlowNetworkAsync, which means WriteLinePageTitle will not be called till DownloadStringTaskSlowNetworkAsync returns… till then, RunAsync will just return a void task.

One more level up, here we called RunAsync without prefixing the call with await, this means, the function will behave exactly like our previous Begin method, it’ll start, and control flow will directly continue to print “Test() is done…”.

A few things to note:
1- The leaves of the calls (DownloadStringTaskAsync… etc) are implemented by Async Library, if you want to implement your own leaf method, it’s a little bit more interesting, I’ll save that for a coming post.
2- await doesn’t block current thread, it rather wraps up the rest of code in your method and creates a callback to execute the code after async operation is done.
3- If you surrounded an await statement with try,catch, catch statements will be included in that callback, which means you will be able to catch exceptions for async methods the same way you catch exceptions for sync methods

Last to note, Async Library is available for WP7 as well as WPF/Silverlight… I’ve to tell you, it makes async code WAY easier to write!

Stay Tuned

WebParts fail to import for readers? here is a possible fix

Kasparov — Wed, 09 Jun 2010 23:42:06 GMT

One of our partners faced a problem that I think we will eventually see occurring frequently with external customers, so I thought to document the problem’s symptoms and a possible fix..

Problem Context:

You have a v2 WebPart (didn’t try it with v3 but it’s likely to repro as well), that you deployed to a server.

You (or some developer who uses your WebPart), creates a feature, with an elements file that tries to deploy that WebPart into a page, something like this:

<Elements xmlns="<a href="http://schemas.microsoft.com/sharepoint/"">http://schemas.microsoft.com/sharepoint/"</a>>
 <Module Name="MyModule" Url="Shared Documents" >
  <File Url="Test.aspx" Type="GhostableInLibrary" >
   <AllUsersWebPart WebPartZoneID="Header" WebPartOrder="1">
    <![CDATA[
     <?xml version="1.0" encoding="utf-8"?>
      <WebPart xmlns:xsd="<a href="http://www.w3.org/2001/XMLSchema"">http://www.w3.org/2001/XMLSchema"</a> xmlns:xsi="<a href="http://www.w3.org/2001/XMLSchema-instance"">http://www.w3.org/2001/XMLSchema-instance"</a> xmlns="<a href="http://schemas.microsoft.com/WebPart/v2"">http://schemas.microsoft.com/WebPart/v2"</a>>
      <Title>A Test</Title>
      <FrameType>Default</FrameType>
      <Description />
      <IsIncluded>true</IsIncluded>
      <ZoneID>Header</ZoneID>
      <PartOrder>0</PartOrder>
      <FrameState>Normal</FrameState>
      <Height>222px</Height>
      <Width></Width>
      <AllowRemove>false</AllowRemove>
      <AllowZoneChange>false</AllowZoneChange>
      <AllowMinimize>false</AllowMinimize>
      <IsVisible>true</IsVisible>
      <DetailLink />
      <HelpLink />
      <Dir>Default</Dir>
      <PartImageSmall />
      <MissingAssembly />
      <PartImageLarge />
      <IsIncludedFilter />
      <ID>g_A4D135E1_0834_4b88_AA94_E7EC5BA80E93</ID>
      <Assembly>MyWebPartAssembly, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35</Assembly>
      <TypeName>MyWebParts.WebPart1</TypeName>
      </WebPart>            ]]>
   </AllUsersWebPart>
  </File>
 </Module>
</Elements>

Login as admin, enable the feature.. do NOT browse to Shared Documents\test.aspx.
Login as reader/visitor and try to browse to that page, you will be greeted with an ErrorWebPart saying:
Error
Web Part Error: A Web Part or Web Form Control on this Page cannot be displayed or imported. You don't have Add and Customize Pages permissions required to perform this action.
If, at this point, you logged in as Admin, the WebPart will render just fine and any reader logs in from now on will render just fine as well

The problem:

Well, it’s hard to call it a problem, it’s a design decision we took. Sharepoint at that first page visit, will do the WebPart import into the page, and as it’s a reader who tries to do the import, it fails.

The solution:

The failure (as the text tells) is because the logged in user doesn’t have AddAndCustomizePages permission, this error pops up in different places but the reason for that specific one is because MyWebPart1 is not marked as SafeAgainstScript=True (I’ve a post about it here). You don’t want to mark just any WebPart as SafeAgainstScript=True because that can expose you, the site and your users to danger. At this point you have 3 options:

If you know for sure that non of your WebPart properties accepts strings from users (or that you do proper encoding/stripping to any script you might get), then go ahead and mark the WebPart as SafeAgainstScript=True in the <SafeControls> entry.
If you are unsure, then your best bet is to mark it as SafeAgainstScript=True and put [RequiresDesignerPermissions] attribute on all properties of your WebPart. This will allow readers to import the WebPart but not to change any.
Just let your users be aware of this scenario .

That’s it for now… stay tuned!

What should I talk about?

Kasparov — Thu, 17 Dec 2009 18:25:51 GMT

Alsalam alikom wa ra7mat Allah wa barakatoh (aka Peace Upon You)

One of the problems every blogger faces is how to answer the above question… We often ask our fellow friends for suggestions, or even post a whole post and let people suggest topics in the comments… well, no more! skribit (http://skribit.com/) tries to find a better solution to that, by subscribing, it gives you a widget code that you can put as a side bar in your blog to let people suggest topics to talk about…

And I’m starting this right away, I’ve included the suggestions box in this post, please use it to suggest topics for me to talk about… after a while I’ll move it into the sidebar and keep it there for the foreseeable future :)

Note: if you are using RSS, you might not be able to get the box, please visit the original post link.

SkribitView suggestions for this blog here.

Thanks,

Stay tuned ;)

Contributors, Contributors, Contributors [Sharepoint]

Kasparov — Thu, 03 Dec 2009 19:14:03 GMT

Alsalam alikom wa ra7mat Allah wa barakatoh (aka Peace Upon You)

I’m planning to start a series of posts, targeting SharePoint developers, introducing some parts of features I got exposed to. I’ll start this by talking about new features we are releasing as part of our security solution in Windows Sharepoint Foundation 2010.

One of the famous threats in the internet is XSS (Cross Site Scripting [Wikipedia link]). Let me describe it in a short scenario,
BadUser creates a page with a form and some javascript code that posts the form data to BadSite.com, then BadUser uploads that page into GoodSite.com/userPages, now he also got access to whatever information that site provides about its users.
GoodUser browses to the page in GoodSite.com, fills the form and violla, BadUser gets the information + whatever his script can gather about GoodUser..
The problem here is that GoodUser didn’t really know that this page will go post data somewhere else… and has no way to know except by using some advanced techniques…

This can be applied on WebParts, some WebParts allow users to directly inject nonencoded text that gets rendered into WebPart Pages (WikiPages).. eg. ContentEditorWebPart. For this, we have introduced new web.config flag called SafeAgainstScript. which basically if set to false, will prevent non-privileged users from changing properties that might not be encoded.

And into the details…

We, generally, prevent contributors from modifying unsafe properties in WebParts. For that to be flexible, we introduce 2 new flags

SafeAgainstScript SafeControls [MSDN Link] flag, setting this flag to false (it’s false by default) on any SafeControl entry, means contributors are not allowed to insert these WebParts into WebPart Pages / Wiki Pages, and if a Designer (a user with Add and Customize Pages permission [MSND Link]) has already added that, then contributors won’t be allowed to modify any of its properties except for the Title.

<SafeControls> 
         <SafeControl Assembly="Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=XXXXXXXXXXXXXX" 
Namespace="Microsoft.SharePoint.WebPartPages" TypeName="TitleBarWebPart" Safe="True" AllowRemoteDesigner="True" SafeAgainstScript="True" /> 
</SafeControls>

RequiresDesignerPermission property-level attribute. It comes in effect only when a WebPart is marked as SafeAgainstScript=True in Web.Config SafeControls entries, WebPart developers can use it to say “even though this WebPart is safe, these specific properties are not”

[RequiresDesignerPermission] 
public string Content 
{ 
    get { return _content; } 
    set { _content = value; } 
}

Here is a matrix that explains which flag will have the upper hand in different situations:

SafeAgainstScript=False

	Contributors	Designers & higher
RequiresDesignerPermission=True	No Access	Can Access
RequiresDesignerPermission=False	No Access	Can Access

SafeAgainstScript=True

	Contributors	Designers & higher
RequiresDesignerPermision=True	Can Access	Can Access
RequiresDesignerPermission=False	No Access	Can Access

For the built-in WebParts, we have taken the effort to hide the properties from the toolpane in situations where users have No Access to modify them. For custom WebParts, developers have two choices
1. Use our built in ToolPane provider, and we will take care of hiding and showing these properties.
2. If you need to provide a custom ToolPane, you will need to take care of that. (I understand we need to improve at this point to provide you with the public APIs to make your life easier)

I’ll put together a sample custom WebPart that demonstrate these new flags, and will update this post once I get it uploaded.

Stay Tuned!

Invent your own language… using Oslo Part II

Kasparov — Sun, 24 May 2009 21:54:26 GMT

Alsalam alikom wa ra7mat Allah wa barakatoh (Peace Upon You)

Part II: Consume the Abstract Syntax Tree
… Do some action! …

If you have not read Part I, we have created our first grammar to recognize this language:

Send "D:\Reports\Templates\Regular.tmpl" to sm1@hotmail.com
Send "D:\Reports\Templates\Special.tmpl" to sm2@hotmail.com,sm3@hotmail.com

Visit here [Part I link] to find the reporter.mg file listing at the end.

In this post, we will explore how to compile/parse the generated tree programmatically. And we will dig a little bit one cool way to do .NET code generation works.

Create a new project (You can pick a console one for simplicity), Add Reference to:
C:\Program Files\Microsoft Oslo SDK 1.0\Bin\Microsoft.M.Grammar.dll and
C:\Program Files\Microsoft Oslo SDK 1.0\Bin\System.Dataflow.dll

Microsoft.M.Grammar.dll contains MGrammarCompiler which we will use to compile our .mg file into .mgx (just as we used mg.exe tool)

System.Dataflow.dll contains DynamicParser class which will be our starting point to parse the generated AST.

This is the compile function:

private static void Compile()
{
    MGrammarCompiler compiler = new MGrammarCompiler();
    compiler.FileNames = new string[] { 
        @"C:\Users\HaythamAlaa\Documents\reporter.mg" };
    compiler.OutFile = "reporter.mgx";
    compiler.Execute(ErrorReporter.Standard);
}

Basically, we pass the fileName(s) and output file then just call Execute and you are done.

Then, we add a Parse method:

private static void Parse()
{
    FileStream mgxFile = File.OpenRead("reporter.mgx");
    DynamicParser parser = DynamicParser.LoadFromMgx(mgxFile, 
                                "Basic.Languages.Reporter");
    object root = parser.Parse<object>(
            @"C:\Users\HaythamAlaa\Documents\reportsData.bundle", 
            null, ErrorReporter.Standard);
    IGraphBuilder graphBuilder = parser.GraphBuilder;

    Traverse(root, graphBuilder);

DynamicParser loads the mgx file we compiled earlier, and then we ask it to parse the bundle file… note that parser.Parse<object> object is obligatory (I believe this will be refactored in the release)

As all nodes returned of type object, parser provides an IGraphBuilder object that helps us traverse and query the AST.

Traverse method should just loop over all nodes (and sub nodes) and output their values.

Traverse is very straight forward. However, there is a couple of tricks we need to be aware of while traversing the tree…
First, here is the basic Traverse Method:
```
private static void Traverse(object root, IGraphBuilder graphBuilder)
{
    if (root is string)
    {
        Console.WriteLine(root.ToString());
        return;
    }

    Console.WriteLine("->" + graphBuilder.GetLabel(root));

    foreach (object childNode in graphBuilder.GetSuccessors(root))
    {
        Traverse(childNode, graphBuilder);
    }
}
```
The output should be like this:
```
->Commands
->Send
D:\Reports\Templates\Regular.tmpl
->Emails
sm1@hotmail.com
->Send
D:\Reports\Templates\Special.tmpl
->Emails
sm2@hotmail.com
sm3@hotmail.com
```
First thing to notice, this is a simple Depth First Search (or Visitor Pattern as architects like to call it).

Second, The leaf nodes of the tree are all strings, the non-leaf ones can be of type SimpleNode or SequenceNode (or something else?) depending on how they were projected in MGrammar.
e.g:
```
=> Send{. . .};
```
Would produce:
```
Send{
...


}
```
That will be parsed as SimpleNode

While,
```
=> Send[. . .];
```
Would produce:
```
Send[
.....
]
```
And will be parsed as SequenceNode

I didn’t read any recommendation on when to use what… but I personally prefer to use {} when you are thinking of a Class and use [] when you are thinking of an Array… more of this will come later when we start generating code.

Third, GetSuccessor: This method can visit any node and iterate over its children. There is also GetSequenceElements and GetEntityMembers… probably what we want to do is something like this:
if (graphBuilder.IsEntity(root))
foreach (… graphBuilder.GetEntityMembers(root))
else if (graphBuilder.IsSequence(root))
foreach (… graphBuilder.GetSequenceElements(root))
… same thing for IsNode

Note the order of checking is important, (e.g IsSequence before IsNode) because a sequence node is also a “node”…
Let the fun begin, first, let’s declare a method that accepts a path and a list of emails as parameters,
```
public static void Send(string reportPath, params string[] contacts)
{
    Console.WriteLine(reportPath);
    foreach (string str in contacts)
        Console.WriteLine("->" + str);
}
```
Now, we want to map every Send node into a method call with the right parameters…
We will use System.Linq.Expressions namespace [MSDN Link].

This namespace contains virtually all types of operations you can perform in .NET, Assignment, MethodCall, Lambda Expression, ArrayIndexing…. etc

2 things we are interested in for now, are MethodCallExpression (to be able to represent our Send method) and Lambda Expression to be able to compile into MSIL and actually run these instructions

Here is an example for this in action
```
MethodCallExpression expression = 
    Expression.Call(typeof(Program).GetMethod("Send"),
    new Expression[] { Expression.Constant(path, typeof(string)),
                       Expression.Constant(emails, typeof(string[])) });
LambdaExpression lambda = Expression.Lambda(expression);
lambda.DynamicInvoke();
```
Expression is an abstract base class that contains static methods to instantiate all types of expressions.. here we call Expression.Call to create a MethodCallExpression.
It accepts a MethodInfo for the method you want to call…
and a list of parameters.
You can then call lambda.DynamicInvoke() to compile into MSIL and execute whatever expressions are inside…

Here is the full listing for creating the lambda expressions out of a passed node:

   1: private static LambdaExpression[] TraverseAndBuildTree(object root,

   2:                                         IGraphBuilder graphBuilder)

   3: {

   4:   if (graphBuilder.GetLabel(root).ToString().ToLower() == "send")

   5:   {

   6:     IEnumerable<object> childs = graphBuilder.GetSuccessors(root);

7:

   8:     string path =

   9:         graphBuilder.GetSuccessors(childs.First()).First().ToString(

  10:     List<string> emails = new List<string>();

  11:     foreach (object objContact in

  12:         graphBuilder.GetSequenceElements(childs.Last()))

  13:     {

  14:       emails.Add(objContact.ToString());

  15:     }

16:

  17:     MethodCallExpression expression =

  18:       Expression.Call(typeof(Program).GetMethod("Send"),

  19:       new Expression[] { Expression.Constant(path, typeof(string)),

  20:           Expression.Constant(emails.ToArray(), typeof(string[])) })

  21:     LambdaExpression lambda = Expression.Lambda(expression);

  22:     return new LambdaExpression[] { lambda };

  23:   }

24:

  25:   List<LambdaExpression> expressions = new List<LambdaExpression>();

  26:   if (graphBuilder.IsSequence(root))

  27:     foreach (object childNode in

  28:         graphBuilder.GetSequenceElements(root))

  29:     {

  30:       expressions.AddRange(

  31:           TraverseAndBuildTree(childNode, graphBuilder));

  32:     }

  33:   else if (graphBuilder.IsNode(root))

  34:     foreach (object childNode in

  35:         graphBuilder.GetSuccessors(root))

  36:     {

  37:       expressions.AddRange(

  38:           TraverseAndBuildTree(childNode, graphBuilder));

  39:     }

  40:   return expressions.ToArray();

  41: }

25 – 40: We just traverse all nodes available and concatenate the lambda expressions into one big list.
6 – 15: We get the values of path and email parameters from AST.
17 – 22: We create the Call Expression.

You can then use this method like this:

LambdaExpression[] lambdas = TraverseAndBuildTree(root, graphBuilder);
foreach (LambdaExpression lambda in lambdas)
    lambda.Compile().DynamicInvoke();

That’s it ! Spend sometime and play around with expressions ;)

Congrats, you’ve reached your 2nd Checkpoint!

The technique described here works well with simple languages, but once your language gets more complicated, you will need a more structured design (and hopefully an automated process) which we will explore in the next part.

Part I: Create the Grammar (What your users write).
Part II: Consume the abstract syntax tree (Do some action). [This post]
Path III: Compile your language into MSIL.

Static Constructor… saved my day

Kasparov — Thu, 21 May 2009 18:47:47 GMT

Alsalam alikom wa ra7mat Allah wa barakatoh (Peace upon you)

I was designing some classes today and I got stuck with an old problem I used to fix in an ugly way…

class A
{
    public static void InitializeMe()
    {
        SomeClassManager.Listeners.Add(typeof(A));
    }
}

It would have been easy if:

You know when exactly to call InitializeMe() so that you get hooked up at the right time (before Listeners start iterating over its elements)
You know what classes contain an InitializeMe function.. you can use reflection to get a list of all classes and then check whether they have the method or not.. –you can’t use interfaces as they don’t allow static members-

What if you are designing a plugin model for an application, and want to call some static initialization code… again you will have to do reflection (which is slow).

And here comes static constructors!

class A
{
    static A()
    {
        InitializeMe();
    }

    public static void InitializeMe()
    {
        SomeClassManager.Listeners.Add(typeof(A));
    }
}

That simple.. Static Constructor [MSDN Link] will be called once this type is “loaded” which happens in .NET when your assembly first gets loaded into memory.. which is exactly the time I wanted… and it saved my day!

P.S, Check MEF (Managed Extensibility Framework) if you are interested in supporting plugins in ur apps, http://www.codeplex.com/MEF
And here is a getting started guide, http://code-inside.de/blog-in/2008/11/19/howto-first-steps-with-mef-hello-mef/

Invent your own language… using Oslo. Part I

Kasparov — Fri, 15 May 2009 04:47:00 GMT

Alsalam alikom wa ra7mat Allah wa barakatoh (Peace Upon You) Part I: Create the Grammar …What your users will write… If you are writing enterprise software, you probably came across this problem many times when you want to give IT Admins the ability to customize your application through some sort of scripts. The typical solution used to be VBS on windows and now PowerShell.. they both offer great ability to expose your public APIs and let others consume them. However, there is no way to customize the syntax to your application needs. If you are writing a software that does some reporting and want to let users decide what template should be used when sending to every customer… wouldn’t it be nice if you let them write something like this: Send "D:\Reports\Templates\Regular.tmpl" to sm1@hotmail.com, sm2@hotmail.com Send "D:\Reports\Templates\Special.tmpl" to sm3@hotmail.com Or how about a language that describes mathematical equations such as this: [[x, y] ; [1, 2] ; [2, x + 5]] * Func(2, 3, 5) – x2 + y * j ^ 3 + d - mul(3) This may look similar to Matlab syntax… in fact it’s. The first part “[[x, y] ; [1, 2] ; [2, x + 5]] ” declares a 2x2 matrix. You can think of all sorts of symbols and operations that can be put into an equation… Latex and MathML [Wikipedia links] come into mind. Latex is not only for representing equations but it’s one feature of it. My point is, even if you want to reuse one of the existing languages out there in your application, you don’t have to rewrite a parser, buy a commercial package or get an OSS that might limit how you will be able to distribute your software. MGrammer (included in Oslo) is one elegant way to achieve this… Or maybe you just don’t like the brackets () in C# and want to define a bracketless C# (VB maybe 8-) )… There might be a debate around this and whether you will want each app to have it’s own Domain Specific Language (DSL) or is it better to have a common known scripting language (like VBS or PowerShell).. I won’t go into this debate actually here… but I would just say, “it depends”. Let’s get to work, First thing you need to do is to get Oslo installed [Microsoft Downloads Link]. It needs SQL server ( Free Express Edition works fine) –I installed it without SQL Server, it gave me an error but it’s ok as long as you don’t do any stuff that requires this-… Oslo comes with a toolchain. We will actually make use of only 2 tools out of these; IntelliPad.exe (GUI tool) and Mg.exe (CMD tool) We will be using IntelliPad to write MGrammar (.mg file), test it on our examplr data and make sure the syntax tree is in the right format. We will then use Mg tool to compile the MGrammar file into .mgx file which represents you compiled language parser.. you can then use it from any .NET app. We will make a grammar that recognizes that first language above, let’s call it Reporter . We will get to know MGrammer as we move on… Open notepad (or notepad++ you geeky guys), type in this example and save as reportsData.bundle. Send "D:\Reports\Templates\Regular.tmpl" to sm1@hotmail.com Send "D:\Reports\Templates\Special.tmpl" to sm2@hotmail.com, sm3@hotmail.com This will work as our test data to make sure our grammar works fine. Open IntelliPad: and save that empty file into Reporter.mg (you have to put the extension yourself or else IntelliPad won’t recognize the file as an MGrammar one) Now close IntelliPad, and double click on your file to open IntelliPad again in MGrammar Mode (it’s a bug if you ask me) The container in MG is the module element (much like namespaces or packages) inside a module you can have different elements (language, type, … etc) in MG we are interested in language element. Now go ahead and type your first lines module Basic.Languages { language Reporter { } } From MGrammar Mode menu, choose Tree Preview, this allows you to see how the data match as you start writing more code. It will ask you for an input file, go ahead and choose reportsData.bundle we created in step 1. Here is how it should look like now: Add this statement (syntax Main = any*; ) into language section, you will notice that the right tree got populated with a long list of characters. This statement is the entry point (Main), here it tells Mg that the syntax you are expecting is “any*” which basically matches with any number of any type of characters. We know that every statement we have starts with “Send” and has a “to” in the middle. So we can refine that any* matching with something more useful, like this: syntax Main = "Send" any* "to" any*; This instructs MG that you are expecting a word “Send” then some text then “to” then some text. You will notice that the preview tree now has a little structure… “Send” node and some characters in side then a “to” node with the rest of the statement.. But there is an error saying that the second statement didn’t match anything. This is because our main statement matches only one instance. We can define it like this: syntax Main = ("Send" any* "to" any*)+; or we can be more descent and define another syntax, say SendCommand.. like this syntax SendCommand = "Send" any* "to" any*; syntax Main = SendCommand+; As you might have guessed, + means 1 or more, * means 0 or more (same as in Regular Expressions) and ? means 0 or 1 occurrence. To make things a little bit more descent, we can define tokens for “Send” and “to”… Tokens can pretty much looked at as the basic building element for a statement or a matching pattern.. Here is our code after defining the tokens: token SendToken = "Send" ; token ToToken = "to" ; syntax SendCommand = SendToken any* ToToken any*; syntax Main = SendCommand+; We need to match the quoted string (file path) but we don’t really bother for now to validate whether it’s a valid file path or not (Left as an exercise to the reader) So we will go ahead and define a more complex token for that, token AlphaNumeric = 'a'..'z' | 'A'..'Z' | '0'..'9' | '_' | '-'; token Path = (AlphaNumeric | ':' | '\\' | '.')+; token Email = (AlphaNumeric | '@' | '.')+; token QuotedPath = '"' Path '"'; syntax SendCommand = SendToken QuotedPath ToToken Email; ‘a’..’z’ defines range (as you might have guessed) other things are straight forward. This is how the right preview tree should look like by now. Things are getting to look better, right? maybe you started to get the feeling it’s just like regular expressions, in fact, it’s pretty much the same concept (String matching after all if you want the truth) but writing in MGrammar gives you a lot of other options when writing your rules than you have when matching with regEx Also you will almost write 0 Lines of code to get your abstract syntax tree (AST) built ;) You will notice IntelliPad is complaining about the ‘,’ in the second example line. He’s right, we didn’t define how a list of emails may look like. syntax ListOfEmails = Email | ListOfEmails "," Email; syntax SendCommand = SendToken QuotedPath ToToken ListOfEmails; This is a recursive rule. It recognizes “a@b.com ” and “a@b.com,c@d.com ” but it doesn’t recognize “a@b.com ,” in other words, the “,” must be between 2 emails… which is exactly what we want. We are almost done, we now need to modify the resultant tree to make it look, well, better. We need to fix the Send nodes so that we don’t actually put “Send” and “to” into the resultant tree. We will use projection, that’s we will define how do we want our rules to be projected (output) into the syntax tree… syntax SendCommand = SendToken p:QuotedPath ToToken list:ListOfEmails => Send[p, list]; We named the match result of QuotedPath as p and ListOfEmails as list.. then the projection is Send[p, list]; would create a node called Send, with 2 children only. Here is the output: Main[ [ Send[ "\" D:\\Reports\\Templates\\Regular.tmpl\ "" , ListOfEmails[ "sm1@hotmail.com" ] ], Send[ "\" D:\\Reports\\Templates\\Special.tmpl\ "" , ListOfEmails[ ListOfEmails[ "sm2@hotmail.com" ], "," , "sm3@hotmail.com" ] ] ] ] Now we need to remove that “\” before and after the path… they don’t look neat. token QuotedPath = '"' p:Path '"' => Path[p]; syntax SendCommand = SendToken p:QuotedPath ToToken list:ListOfEmails => Send[valuesof(p), list]; First modification would make a node for every path and would remove the double quotes like this Path[ “D:\…..” ] valuesof(p) will extract the node contents and project it to the tree, in this case the contents is the path with singles double quotations (if you know what I mean). You can go ahead and try to modify the Email rule so that it doesn’t project the “,” and projects all emails in one list rather than nested lists. Here is how the tree looks like after all my modifications (with those left to you as exercise :)) Commands[ Send[ "D:\\Reports\\Templates\\Regular.tmpl" , Emails[ "sm1@hotmail.com" ] ], Send[ "D:\\Reports\\Templates\\Special.tmpl" , Emails[ "sm2@hotmail.com" , "sm3@hotmail.com" ] ] ] Now, to compile use .mg file, open cmd, browse to the folder where reporter.mg file is and type this: "C:\Program Files\Microsoft Oslo SDK 1.0\Bin\mg.exe" reporter.mg This will generate a reporter.mgx file in the current directory. Now run: "C:\Program Files\Microsoft Oslo SDK 1.0\Bin\mgx.exe” reportsData.bundle -r:reporter.mgx This will generate the M structure (similar to AST with some key differences that we will explore in the next post) into a separate file. Viola ! Conclusion: We used IntelliPad to write and test our MGrammer. MGrammar is a descriptive language to define the grammar of any language. We need to define module and language sections in .mg file. syntax Main is the starting point, syntax es can be recursive. tokens are the basic elements. We use projection rules on syntax and token to reshape the resultant tree. MGrammar is awesome! Congrats , you reached your first Checkpoint ! Here is the full listing of Reporter.mg file. module Basic.Languages { language Reporter { token SendToken = "Send" ; token ToToken = "to" ; token AlphaNumeric = 'a'..'z' | 'A'..'Z' | '0'..'9'; token Path = (AlphaNumeric | ':' | '\\' | '.')+; token Email = (AlphaNumeric | '@' | '.')+; token QuotedPath = '"' p:Path '"' => Path{p}; syntax ListOfEmails = e:Email => Emails[e] | list:ListOfEmails "," e:Email => Emails[valuesof(list), e]; syntax SendCommand = SendToken p:QuotedPath ToToken list:ListOfEmails => Send{valuesof(p), list}; syntax Main = s:SendCommand+ => Main{Commands[valuesof(s)]}; interleave whitespace = ( " " | "\r" | "\n" | "\t" )+; } } Path I: Create the Grammar (What your users write). [This post] Part II: Consume the abstract syntax tree (Do some action!). Path III: Compile your language into MSIL. References: A good tutorial for MGrammar: http://msdn.microsoft.com/en-us/library/dd441702.aspx You will also find some good documents in C:\Program Files\Microsoft Oslo SDK 1.0\Documents 2 in particular are interesting: MGrammar Language Specification.docx MGrammar in a Nutshell.docx Have a nice time! Creating a .NET language by Haytham Abuel-Futuh is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License . Based on a work at blogs.msdn.com ....(read more)

Friend Assemblies in .NET

Kasparov — Fri, 01 May 2009 02:03:00 GMT

Alsalam alikom wa r7amat Allah wa barakatoh (Peace upon you..) Long story short, you have AssemblyA (with source code) and AssemblyB… you want AssemblyB to be able to access internal members (types, data, methods.. etc) of AssemblyA… You can of course do that through Reflection but come on, it’s not a descent way to do it.. right?? .Net framework offers an attribute called InternalsVisibleToAttribute [ MSDN Link ]. It works on .NET 2.0 and above. An example grabbed from MSDN: 1: // AssemblyA.dll 2: using System.Runtime.CompilerServices; 3: using System; 4: 5: [assembly:InternalsVisibleTo( "AssemblyB" )] 6: 7: // internal by default 8: class Class1 9: { 10: public void Test() 11: { 12: Console.WriteLine( "Class1.Test" ); 13: } 14: } 15: 16: // public type with internal member 17: public class Class2 18: { 19: internal void Test() 20: { 21: Console.WriteLine( "Class2.Test" ); 22: } 23: } This way, you can easily use Class1 and Class2.Test from AssemblyB code (obviously after adding reference to AssemblyA). I had 2 questions in mind after reading this, This can be some sort of a hack, I simply create another asesmbly that replaces your AssemblyB and violla, I’ve access to your internal members… A: You can include the public token key for AssemblyB in InternalsVisibleTo constructor Can I selectively determine what classes (types) are visible to AssemblyB? .NET offers a class called StrongNameIdentityPermission [ MSDN Link ] Which can solve both of these problems actually. I’ll not go into details for how to do it, but let me state what can you do using this class.. You can require that calling assemblies (e.g AssemblyB) must be strongly signed using a private key you provide.. (Imagine you are building a multi-tier application and you don’t want anybody else to just call your public classes…) StrongNameIdentityPermission works only on public classes, while InternalsVisibleToAttribute works on assembly level. One last thing, it’s actually kind of funny.. InternalsVisibleTo constructor has 2 overloads (only in .NET3.5), the second accepted named parameter named AllIntenalsVisible (bool) which gave me hope that I can somehow selectively choose what to make visible… but… here is what I found in MSDN Documentation: This API supports the .NET Framework infrastructure and is not intended to be used directly from your code. Catch you later…...(read more)

Javascript Execution Context

Kasparov — Mon, 21 Jul 2008 15:24:00 GMT

Alsalam alikom wa ra7mat Allah wa barakatoh I've faced a strange javascript exception when running a piece of code on Firefox/Safari, while it works fine on IE. Here is what Firefox 3.0 says: uncaught exception : [ Exception ... "Illegal operation on WrappedNative prototype object " nsresult: "0x8057000c (NS_ERROR_XPC_BAD_OP_ON_WN_PROTO)" location: "JS frame : After some searching, I found out that the problem is related to how Firefox/Safari handles the execution context for any called function. Have a look on this code: 1: < html > 2: < head > 3: < script language ="javascript" > 4: function ExecuteRef(str, arg) { 5: str(arg); 6: } 7: function TestRun(objID){ 8: var obj = document.getElementById(objID); 9: var funcRef = obj.parentNode.removeChild; 10: ExecuteRef(funcRef, obj); 11: } 12: </ script > 13: </ head > 14: 15: < body onload ="javascript:TestRun('delete1');" > 16: 17: < div id ='delete1' > This should not appear </ div > 18: < div > This should alwasy appear </ div > 19: 20: </ body > 21: </ html > What this code is supposed to do is to remove the first div -'delete1'- just when the page load. What actually happens on FF/Safari, is that nothing is deleted, while on IE the first div no longer appears. We have 2 functions here, TestRun which given an ID for an element, it creates a reference for that object -ln 8- and a reference for the removeChild method -ln 9- , then passes them both to ExecuteRef, which will simply execute that -ln 5- . Why is that happening? Is it a problem with FF Javascript Engine AND Safari's one? or a problem with IE's ? Actually it's not a mistake at all. It's all about how each engine determines the context of function execution. Let's speak in code language: var funcRef = obj.parentNode.removeChild; funcRef is now a function pointer for removeChild. When you execute funcRef(args), it's translated as funcRef(this, args) in all javascript engine. this way it's easy for your function body to reference member variables of the object it's defined in. The difference between IE & FF/Safari engines, is that IE preserves the original caller object, in this case the "obj.parentNode".. so that when it translates funcRef(args) to funcRef( this , args) it makes sure that " this " is "obj.parentNode" and your code works fine. On the other hand, FF/Safari don't preserve the original caller, instead they just try to execute the function in the new context. Which in turn means that in funcRef( this , args), this refers to the ExecuteRef method itself, and as there is no function called removeChild defined inside ExecuteRef, the calling just fails giving that weird exception above. I've explored different solutions for that problem. I'll try to summarize them here: 1. Explicitly declare the parameters of the function pointer: var funcRef = function (x) { obj.parentNode.removeChild (x); } This way, jscript engine will have to preserve the calling object -obj.parentNode- 2. If you already know the full parameters that will passed, you can get use of the closure feature JS supports. var funcRef = function (){obj.parentNode.removeChild(obj);};ExecuteRef(funcRef); 3. Use Function.call / Function.apply: this requires that you pass the context to ExecuteRef.. function ExecuteRef(context, str, arg) { str.call(context, arg);} function TestRun(objID){ .... ExecuteRef(obj.parentNode, funcRef, obj);} That's it so far. Thanks for reading!...(read more)

Hello World!

Kasparov — Thu, 19 Jun 2008 22:03:52 GMT

Alsalam alikom wa ra7mat Allah wa barakatoh (aka Peace upon you)

This's my first post on MSDN blogs, I've been blogging since 2005 on blogspot though... a mixture of personal and technical posts you can find on http://haythamalaa.blogspot.com

I'll keep this blog technical except for this first post .

A brief about me:

Haytham Alaa Abuel-Futuh, Muslim, Egyptian. Graduated from faculty of Computer and Information Sciences, Ain-Shams University, Cairo, Egypt.
he was actively joining ACM during school days, and was a part of a team which participated in our regional contest for 2 consecutive years, getting the 9th then 5th positions respectively.

He got a research scholarship in University of Paderborn, Germany in 2007 during which he -along with his teammates- developed RDES (Remote Debugging for Embedded Systems).

He then joined Critical Sites -the development division of NTP Software-the world leader in managing massive unorganized data- in late 2007 till early May-2008 when he moved to Microsoft.

Keep in touch.

Haytham Abuel-Futuh,
SDE, WSS, MCDC,
http://haythamalaa.blogspot.com.