Sajay : .NET Framework

WF 4.0 First Look!

Sajay — Thu, 30 Oct 2008 00:05:00 GMT

More coming up

Here is the First Look at WF 4.0 talk by Kenny

http://channel9.msdn.com/pdc2008/TL17/

WCF Scalability and IIS

Sajay — Wed, 20 Aug 2008 00:03:00 GMT

Couldn't get better than this. Wenlong has pretty much detailed out to the gory details of understanding WCF scalability in these posts.

The fact you realize is that one bottle neck is enough to kill your scalability.

Finally Zermatt's here

Sajay — Sat, 12 Jul 2008 19:57:00 GMT

IIts been a long wait. After working with customers for getting out their token service and doing all the claim based black magic, finally we have a framework. Its not a bubble gum and match stick situation more and its a full framework.

If you want to build applications and solve your federation problems which i have had a lot of previous posts on then you really want to take a look at this. Vittorio is put up quite a cool set of links and a very neat overview which I don't want to cut paste here.

The one thing i have seen is the ease of integrating with Zermatt. Those of you who want to just go to the gory details head out to Keith's post- or white paper directly on connect-

I still remember working with Vittorio and Martin in getting up the STS during the indigo beta days .. nice to see this out which really makes it easy to get your security around your solution. I would like to emphasize on the fact that claim's based approach is a security solution and not just an approach. The spec is so rich if you have the time to digest it cause it breaks down complex problems of multi-hop scenario and delegation really simple. Transformations are another area which I am interested in and would see how Zermat solves this problems.

Happy spelunking.

Throttling using managed semaphores and named monitors

Sajay — Thu, 04 Oct 2007 08:06:57 GMT

Firstly I would like to explain what is a managed semaphore. This was introduced to me during a session by Andy Clymer (developmetor). Basically the requirement of a semaphore is pretty clear but then again we don't have to resort to kernel objects and mutexes if not required to step out from our little managed world. Hence the concept of managed semaphore. Basically the idea is to allow only a certain number of threads to any critical section. This enables the number of parallel threads executing a block of code to the max count of the semaphore.

Let look at a simple piece of code doing this.

We would like to do something like this.

semaphore.Acquire()

try{

... // throttled code

...

}

finally{

semaphore.Release();

}

Now lets see a class that would enable use to do something like this.

public class Semaphore
{
      private int current;
      private int max;
      private object semLock = new object();

      public Semaphore(int max)
      {
          this.current = 0;
          this.max = max;
      }

      public void Acquire()
      {
          bool acquired = false;

          lock (semLock)
          {
              Debug.Assert(current <= max, "Current has gone out of range..");

              while (current >= max)
              {
                  Monitor.Wait(semLock);
              }

              current++;
              acquired = true;
              Debug.Assert(current <= max, "Current has gone out of range..");
          }
      }

      public void Release()
      {
          lock (semLock)
          {
              Debug.Assert(current <= max, "Current has gone out of range..");

current--;

              // Mark a single thread that has called Monitor.Wait(semLock)
              // as ready to run...
              // only run it when it can be granted the semLock Monitor.
              Monitor.Pulse(semLock);

Debug.Assert(current >= 0, "Current has gone out of range..");

}
}
}

Now that we have a class that can enable throttling we need to figure out how to enable resource based throttling.

This implementation would suffice if the resource itself is capable of parallelism. Assume writing to a file or a port. Now we sometimes have scenarios where only one thread can write to a file. But we might need multiple threads to write to different files/ports. Think of a simple processor where we are sending out messages to different ports but we can have only on thread sending out messages to a particular port.

Hence we have the resource itself usage mutually exclusive but different instances of the resources can be accessed in parallel.Now to write code like this would could end up writing double locks which is quite messy code in itself and can end up in deadlocks if you don't acquire locks in the correct sequence. Would like to call out the fork and knifes problem. (lets leave this for another post).

Thinking about this lead me to write a named semaphore. Basically the idea was to keep a dictionary of the resources currently accessed.

If a resource makes a request we give it a lock. This is under the condition that there is no other thread owning the current resource. This also checks if the current number of locks obtained are <= the max count hence checking our throttling count. Basically to put it in a nutshell. A thread can acquire a lock if there are enough number of available semaphores and there is no parallel thread who has acquired the lock for the current request.

To put this down at one shot we can just rewrite the semaphore class to have a dictionary and a count. and this is what I ended up with.

public class KeyBasedSemaphore<T> where T : struct
{
    Dictionary<T, object> locks = new Dictionary<T, object>();
    object thisLock = new object();
    int max = 0;
    int current = 0;

    public KeyBasedSemaphore(int maxCount)
    {
        this.max = maxCount;
    }

    public void Acquire(T key)
    {
        lock (thisLock)
        {
            Debug.Assert(current <= max, "Current has gone out of range..");

            bool acquired = false;
            while (locks.ContainsKey(key))
            {
                Monitor.Wait(thisLock);
            }

locks.Add(key, new object());

            //Wait if there are already max
            //number of instances running.
            while (current >= max)
            {
                Monitor.Wait(thisLock);
            }

            acquired = true;
            current++;
            Debug.Assert(current <= max, "Current has gone out of range..");
        }
    }

    public void Release(T key)
    {
        lock (thisLock)
        {
            Debug.Assert(current <= max, "Current has gone out of range..");

            //Remove the lock on the instance and
            //Decrement the counter.
            locks.Remove(key);
            current--;

//Pulse any waiting processes.
Monitor.Pulse(thisLock);

            Debug.Assert(current >= 0, "Current has gone out of range..");
        }
    }
}

OK I admit its dirty in its own sweet way. I send this and pretty much got a request to refractor it. Basically Andy gave a very clear suggestion and asked me as the code would become clearer it I split out the intent. Primarily what we needed to do was the separate the semaphore with the named set of locks.

This is primarily what we need to achieve.

Obtained a named monitor ( name = file or port etc )

Obtain the Semaphore

Do Activity

Release Semaphore

Exit named monitor

Well this is from Andy

"I would therefore be inclined not to modify the Semaphore class to produce the required behaviour but to create two new classes..Produce a three class solution using the original semaphore class, and then a class to encapsulate the new synchronization pattern and a second to implement named monitor functionality using a monitor.

As for performance, in the implementation you supplied If a thread is currently in a wait state waiting for access to the “port/file” it is being woken up every time a release is made irrespective if its for the resource it is interested in, by refactoring as above you could make it so they only woken up when the actual resource they are after becomes available. Having said that my way means using more objects for synchronization and that cost may out way any benefit..It will almost certainly depend on the behaviour of the application logic, if there is high contention for a given resource I think my way would be more optimal if not then using a single synchronization object may very well be quicker..

So unless performance is absolutely crucial I still like the idea of keeping semaphore behaviour and the exclusive access behaviour in two separate classes, wrapped by a third to implement your required synchronization pattern, because it makes the intent clearer and allows the re-use of common components."

And finally the the named monitor class that Andy refactored it out to.

class NamedMonitorCollection<K>
   {
       private class NamedMonitor
       {
           public int usageCount = 0;
       }

private Dictionary<K, NamedMonitor> namedMonitors = new Dictionary<K, NamedMonitor>();

       public void Enter(K name)
       {
           NamedMonitor monitorToAquire = null;

           lock (namedMonitors)
           {
               // Is named monitor currently in use, if not create an entry
               if (namedMonitors.ContainsKey(name) == false)
               {
                   namedMonitors[name] = new NamedMonitor();
               }

monitorToAquire = namedMonitors[name];

               // Register threads interest in this monitor
               monitorToAquire.usageCount++;
           }

           // Attempt to aquire the appropriate monitor
           Monitor.Enter(monitorToAquire);
       }

       public void Exit(K name)
       {
           lock (namedMonitors)
           {
               Debug.Assert(namedMonitors.ContainsKey(name), "This monitor is unknown");

Monitor.Exit(namedMonitors[name]);

               // Un register interest in the named monitor
               // if no interested parties then remove it from the list
               // of known named monitors
               namedMonitors[name].usageCount--;
               if (namedMonitors[name].usageCount == 0)
               {
                   namedMonitors.Remove(name);
               }
           }
       }

       /// <summary>
       /// IEnumerable method
       /// </summary>
       /// <returns></returns>
       internal IEnumerable<K> GetMonitorsInUse()
       {
           lock (namedMonitors)
           {
               foreach (K monitor in namedMonitors.Keys)
               {
                   yield return monitor;
               }
           }
       }

       public int inUseCount
       {
           get
           {
               lock (namedMonitors)
               {
                   return namedMonitors.Count;
               }
           }
       }

}

Now if we look at the code above we are surely making more locks but the code is much more clearer. The overheads can come into play only if we have a large number of resources waiting for threads and this would end up in a large number of locks and contentions. On the other hand the first implementation still works with only one lock and hence might have minor improvement. Then again we can write everything in assembly.

Hope this might help in some ideas in doing interesting parallel processing in a purely managed world :)

Disabling JIT Optimizations

Sajay — Thu, 15 Mar 2007 14:04:00 GMT

INI files are not dead yet :)

Now if you wanted to disable JIT optimization then place this into your application directory as a <appName>.ini file

[.NET Framework Debugging Control]
GenerateTrackingInfo=1
AllowOptimize=0

Why would you want to do this?
Mostly when debugging in release mode you might not the get the correct stack dump. Eg: JIT might inline method definitions and you could actualy not see the method in which the exception is being raised. So when your debugging in a production environment. try to disable JIT optimizations before geting the stack dump.

Custom Databinding using Property Descriptors

Sajay — Fri, 29 Sep 2006 15:51:00 GMT

Creating large flat classes for data binding is one way of doing things. But you should check out this article before anyone decides to tightly couple your object model with your data representation. Mike has put this down beatifully.

http://blogs.msdn.com/mikechr/archive/2006/05/17/600697.aspx

Iterating over a nested Member

Sajay — Fri, 25 Aug 2006 11:32:00 GMT

You have a Collection of Products. Every product effectively maps to an Item. Now I needed to bind this Product collection to a grid and display the item name. So basically it would be something like prod[i].Item.Name or any other property can be accessed for item. Basically the idea is to Iterate over an inner member inside every element of the parent collection.

If you wanted to bind this with a GridView you can have very complex Databinding syntax for this by casting the items etc. Another option that I wanted to bring up here is creating an IEnumerable view using the parent collection.

public class ProdItemList:IEnumerable

{

private List<Product> Products;

public ProdItemList(List<Product> products)

{ this. Products = products; }

public IEnumerator GetEnumerator()

{

foreach (Product product in Products)

yield return product.Item;

}

You can then bind the DataSource and it would effectively bind to the properties in Item. With something like ItemGrid.DataSource = new ProdItemList(products);

I really liked this article regarding IEnuerable and 2.0. I am sure there are many approaches for this but just thought i'd put it down here for further discussion.

Can you call private members from a different class?

Sajay — Thu, 06 Jul 2006 20:26:00 GMT

Standing by OO principles we would have to say "Private memebers are only visible to to the class in which it is defined"

This is the MSDN Defintion for the C# private keyword
"The private keyword is a member access modifier. Private access is the least permissive access level. Private members are accessible only within the body of the class or the struct in which they are declared.

Nested types in the same body can also access those private members.

It is a compile-time error to reference a private member outside the class or the struct in which it is declared."

The whole idea of Private access modifier being only a language construct and not an runtime principle is quite interesting.It brings up one of the core concepts of accessmodifiers only down to the degree of language compilation and enforcing them during runtime still takes a back seat.

The whole fundamental of .NET is the CLR and if we can me an excution of a private method at runtime, would this defeat the the purpose of making something private? To lay it out, this is what I am trying to ask.

using System;

using System.Reflection;

using System.Collections.Generic;

using System.Text;

public class A

{

private static void YouCantCallMe()

{

Console.WriteLine("I got called.");

}

class Program

{

static void Main(string[] args)

{

Type t = typeof(A);

t.InvokeMember("YouCantCallMe", BindingFlags.InvokeMethod |

BindingFlags.Static | BindingFlags.NonPublic, null, null, null);

}

The runtime with the help of reflection helps you get around one of the principles of OO which prohibits visibility of private memebers outside the current body. Personally i find it pretty cool that i can do stuff like this with reflection.
Any thoughts on this ?

Encode Hex String

Sajay — Thu, 29 Jun 2006 19:54:00 GMT

I needed to get the hash as a string from a byte[].
I was snooping around X509Certificate class and saw a GetHashString method.
After a bit more digging i ended up at System.Security.Util.Hex this had the method i was looking for. Check out the EncodeHexString method.

public static String EncodeHexString(byte[] sArray)

{

String result = null;

if(sArray != null) {

char[] hexOrder = new char[sArray.Length * 2];

int digit;

for(int i = 0, j = 0; i < sArray.Length; i++) {

digit = (int)((sArray[i] & 0xf0) >> 4);

hexOrder[j++] = HexDigit(digit);

digit = (int)(sArray[i] & 0x0f);

hexOrder[j++] = HexDigit(digit);

}

result = new String(hexOrder);

}

return result;

}

// converts number to hex digit. Does not do any range checks.

static char HexDigit(int num) {

return (char)((num < 10) ? (num + '0') : (num + ('A' - 10)));

}

Validating the Certificate Chain

Sajay — Tue, 23 May 2006 13:55:00 GMT

Ususally when tesitng with different certificates we usually need to check if the certificate chain is valid and this snippet came in quite handy in many places.

X509Certificate2 cert = FedUtil.LookupCertificate(StoreName.My, StoreLocation.LocalMachine, "CN=MyCertificate");
X509Chain chain = new X509Chain();
bool pass = chain.Build(cert);

Helper method to look up the certificate from the Store. I refactored this out of the WCF samples in the SDK.

public static X509Certificate2 LookupCertificate(StoreName storeName,

StoreLocation storeLocation,

string subjectDistinguishedName)

{

X509Store store = null;

try

{

store = new X509Store(storeName, storeLocation);

store.Open(OpenFlags.ReadOnly);

X509Certificate2Collection certs = store.Certificates.Find(X509FindType.FindBySubjectDistinguishedName,

subjectDistinguishedName, false);

if (certs.Count != 1)

{

throw new Exception("Certificate not found or more than one certificate found");

}

return (X509Certificate2)certs[0];

}

finally

{

if (store != null) store.Close();

}