I know the answer (it's 42)

A blog on coding, .NET, .NET Compact Framework and life in general....

Posts
  • I know the answer (it's 42)

    Bring your child to work

    • 1 Comments

    Today a gang of kids raided Microsoft India as we celebrated "Bring your child to work" day. The kids were given realistic looking blue badges, they played around with Xboxes, tablet PCs and had a blast.

    Kids at work DSCF1696 DSCF1726 DSCF1706
  • I know the answer (it's 42)

    What is work

    • 1 Comments

    "What is work? Work is of two kinds: first, altering the position of matter at or near the earth's surface relatively to other such matter; second, telling other people to do so." ~ Bertrand Russell's 1932.

    Found this via BoingBoing. My version is as follows

    "What is work? Work is of two kinds: first, write lots of code to create some bugs; second, fix those bugs".

  • I know the answer (it's 42)

    Which font do you use

    • 1 Comments

    I was reading Sriram's posts about his fling with fonts which reminded me that I have a peculiar attachment with courier new. I use it liberally. My emails are in Courier new teal and my website is also in the same font but uses a different color scheme (white over black). Since most of the time I'm coding and editors typically use fixed width font like courier new, I kind of got used to it.

    Each time I boot up a new box, I set the console font to courier and some insanely small size. Again I have an obsession with seeing as much as I can in a single screen.

    However, working in Adobe had got me in touch with some people for whom fonts are the very basis of existance. I saw one of them cringe when one mentioned "font format". He went into a lengthy lecture on how modern fonts are actually languages that specify curves and not a format like the older Bitmapped fonts.

  • I know the answer (it's 42)

    Filling income tax

    • 1 Comments

    The Indian income tax law allows certain expenditure and investments to be tax-exempt. The law also demands various proofs if you want to claim exemptions for this. This include original medical bills, rent-receipts, investments proofs. The list goes on. As you can guess this results in standing in a long queue and a lot of other hassles.

    However, I don't care much about these things. I have s imple solution for this. I use the following Ruby script to get around all of this.

    outFile = File.open "c:\IncomeTax", "w"
    outFile.print "Lots of tax"
    outFile.close

    Ruby-Dooby-dooooo

     

  • I know the answer (it's 42)

    .NET Compact Framework MemMaker

    • 1 Comments
    2007_01_28 120
    Glen recently discovered that by keeping his application’s EXE empty and putting all the forms, code, resources, and data in managed DLLs, he reduced the amount of virtual memory his app uses inside its slot while at the same time taking advantage of memory outside the slot in the 1 GB shared memory area

    Read more about this very interesting finding at Rob Tiffany’s blog at http://blogs.msdn.com/robtiffany/archive/2009/04/09/memmaker-for-the-net-compact-framework.aspx

  • I know the answer (it's 42)

    Silverlight on Nokia S60 devices

    • 1 Comments
    Frustrated?

    In many of my blog posts (e.g.here and here) I refer to .NET Compact Framework and Symbian OS (S60) and obviously folks keep asking me via comments (or assume) that we are porting .NETCF on S60 devices. So I thought it's time to clarify :)

    The short answer is that we are not porting .NETCF to S60 devices, but we are porting the Silverlight on S60 devices. This was jointly announced by Microsoft and Nokia, read Nokia's press release here.

    We are in very early stage of development and it is very hard to tell how much will be in it (e.g. will it be SL v1.0 or v2.0). However, we are working hard and as and when more details emerge I will share it out from this blog. So keep reading.

  • I know the answer (it's 42)

    Back to Basics: Memory leaks in managed systems

    • 1 Comments
    Kolkata Trip 2009

    Someone contacted me over my blog about his managed application where the working set goes on increasing and ultimately leads to out of memory. In the email at one point he states that he is using .NET and hence there should surely be no leaks. I have also talked with other folks in the past where they think likewise.

    However, this is not really true. To get to the bottom of this first we need to understand what the the GC does. Do read up http://blogs.msdn.com/abhinaba/archive/2009/01/20/back-to-basics-why-use-garbage-collection.aspx.

    In summary GC keeps track of object usage and collects/removes those that are no longer referenced/used by any other objects. It ensures that it doesn’t leave dangling pointers. You can find how it does this at http://blogs.msdn.com/abhinaba/archive/2009/01/25/back-to-basic-series-on-dynamic-memory-management.aspx

    However, there is some catch to the statement above. The GC can only remove objects that are not in use. Unfortunately it’s easy to get into a situation where your code can result in objects never being completely de-referenced.

    Example 1: Event Handling (discussed in more detail here).

    Consider the following code

    EventSink sink = new EventSink();
    EventSource src = new EventSource();
    
    src.SomeEvent += sink.EventHandler;
    src.DoWork();
    
    sink = null;
    // Force collection
    GC.Collect();
    GC.WaitForPendingFinalizers();

    In this example at the point where we are forcing a GC there is no reference to sink (explicitly via sink = null ), however, even then sink will not be collected. The reason is that sink is being used as an event handler and hence src is holding an reference to sink (so that it can callback into sink.EventHandler once the src.SomeEvent is fired) and stopping it from getting collected

    Example 2: Mutating objects in collection (discussed here)

    There can be even more involved cases. Around 2 years back I saw an issue where objects were being placed inside a Dictionary and later retrieved, used and discarded. Now retrieval was done using the object key. The flow was something like

    1. Create Object and put it in a Dictionary
    2. Later get object using object key
    3. Call some functions on the object
    4. Again get the object by key and remove it

    Now the object was not immutable and in using the object in step 3 some fields of that object got modified and the same field was used for calculating the objects hash code (used in overloaded GetHashCode). This meant the Remove call in step 4 didn’t find the object and it remained inside the dictionary. Can you guess why changing a field of an object that is used in GetHashCode fails it from being retrieved from the dictionary? Check out http://blogs.msdn.com/abhinaba/archive/2007/10/18/mutable-objects-and-hashcode.aspx to know why this happens.

    There are many more examples where this can happen.

    So we can conclude that memory leaks is common in managed memory as well but it typically happens a bit differently where some references are not cleared as they should’ve been and the GC finds these objects referenced by others and does not collect them.

  • I know the answer (it's 42)

    NETCF: GC and thread blocking

    • 1 Comments
    Hyderabad Zoological Park

    One of our customers asked us a bunch of questions on the NETCF garbage collector that qualifies as FAQ and hence I thought I’d put them up here.

    Question: What is the priority of the GC thread
    Answer: Unlike some variants of the desktop GC and other virtual machines (e.g. JScript) we do not have any background or timer based GC. The CLR fires GC on the same thread that tried allocation and either the allocation failed or during pre-allocation checks the CLR feels that time to run GC has come. So the priority of the GC thread is same as that of the thread that resulted in GC.

    Question: Can the GC freeze managed threads that are of higher priority than the GC thread?
    Answer: Yes GC can freeze managed threads which has higher priority than itself. Before actually running GC the CLR tries to go into a “safe point”. Each thread has a suspend event associated with it and this event is checked by each thread regularly. Before starting GC the CLR enumerates all managed threads and in each of them sets this event. In the next point when the thread checks and finds this event set, it blocks waiting for the event to get reset (which happens when GC is complete). This mechanism is agnostic of the relative priority of the various threads.

    Question: Does it freeze all threads or only Managed threads?
    Answer: The NETCF GC belongs to the category “stop-the-world GC”. It needs to freeze threads so that when it is iterating and compacting managed heap nothing on it gets modified. This is unlike some background/incremental GC supported by other virtual machines.

    GC freezes only managed threads and not other native threads (e.g. COM or host-threads). However, there are two exclusions even on the managed threads

    1. It doesn’t freeze the managed thread that started GC because the GC will be run on that thread (see answer to first question)
    2. Managed threads that are currently executing in native p/invoke code is not frozen either. However, the moment they try to return to managed execution they get frozen. (I actually missed this subtle point in my response and Brian caught it :))

    Hope this answers some of your questions. Feel free to send me any .NET Compact Framework CLR questions. I will either answer them myself or get someone else to do it for you :)

  • I know the answer (it's 42)

    Silverlight on Nokia S60 platform

    • 1 Comments
    Bidar - Fort

    Today at MIX 2010 we announced Silverlight beta for Nokia S60 phones. Go download the beta from http://silverlight.net/getstarted/devices/symbian/

    Currently the supported platform is S60 5th Edition phones and in particular the 5800 XpressMusic, N97 and N97 mini. It works in-browser and only in the Nokia default web-kit based browser.

    Silverlight on S60 uses .NET Compact Framework (NETCF) underneath it. I work in the dev team for the Execution Engine of NETCF and I was involved at the very beginning with getting NETCF to work on the S60 platform.

    NETCF is extremely portable (more about that in later posts) and hence we didn’t have a lot of trouble getting it onto S60. We abstract away the platform by coding against a generic platform abstraction layer (PAL), but even then we had some challenges. Getting .NET platform on a non-Windows OS is challenging because so much of our code relies on Windows like functionality (semantics) which might not be available directly on other platforms.

    As I was digging through emails I found the following screenshot that I had sent out to the team when I just got NETCF to work on the S60 Emulator…

    image

    As you can see the first code to run was a console application and it was not Hello World :). We have come a long way since then as evident from the touch based casual game running on the N97 shown below

    image

  • I know the answer (it's 42)

    WP7: When does GC Consider a Local Variable as Garbage

    • 1 Comments
     

    Consider the following code that I received:

    static void Foo()
    {
        TestClass t = new TestClass();
        List<object>l = new List<object>();
        l.Add(t); // Last use of l and t
    
        WeakReference w = new WeakReference(t);
    
        GC.Collect();
        GC.WaitForPendingFinalizers();
    
        Console.WriteLine("Is Alive {0}", w.IsAlive);
    }

    If this code is is run on the desktop it prints Is Alive False. Very similar to the observation made by a developer in http://stackoverflow.com/questions/3161119/does-the-net-garbage-collector-perform-predictive-analysis-of-code. This happens because beyond the last usage of l and t, the desktop GC considers l and t to be garbage and collects it.

    Even though this seems trivial, it isn’t. During JITing of this method the JITer performs code analysis and ensures that runtime tables are updated to reflect whether at a particular time a given local variable is alive or not (it is not used beyond that point). This needs a bunch of data structures to be maintained (lifetime tables) and also dynamic analysis of code.

    In case of Windows Phone 7 (or other platforms using NETCF) for the same code you’d get Is Alive True. Which means the NETCF GC considers the object to be alive even beyond it’s last usage in the function.

    Due to the limited resources available on devices, NETCF does not do these dynamic analysis. So we trade off some runtime performance to ensure faster JITing (and hence application faster startup). All local variables of all functions in the call stacks of the all managed threads in execution during garbage collection is considered live (hence also object roots) and not collect. In NETCF the moment the function Foo returns and the stack is unwound the local objects on it including t and l becomes garbage.

    Do note that both of these implementations follow the ECMA specification (ECMA 334 Section 10.9) which states

    For instance, if a local variable that is in scope is the only existing reference to an object, but that local variable is never referred to in any possible continuation of execution from the current execution point in the procedure, an implementation might (but is not required to) treat the object as no longer in use.”

    Hence even though on the desktop CLR it is not important to set variables to null, it is sometimes required to do so on WP7. In case t and l was set to null GC would’ve collected them. So the guidance for WP7 is

    1. In most cases setting local variables to null is not required. Especially if the function returns fairly soon
    2. If you have locals that hold onto very large data structures and that function will remain executing for a long time then set those variables to null when you are done using them (e.g. in between the last usage of a variable in a large function and calls to a bunch of web-services in the same function which will take a long time to return).
    3. Use dispose patterns where required. It’s important on devices to free up limited resources as soon as possible.
  • I know the answer (it's 42)

    C# code for Creating Shortcuts with Admin Privilege

    • 1 Comments
    Seattle skyline

    Context

    If you just care about the code, then jump to the end Smile

    In the CLR team and across other engineering teams in Microsoft we use build environments which are essentially command shells with a custom environment setup using bat and cmd scripts. These scripts setup various paths and environment variables to pick up the right set of build tools and output paths matching the architecture and build flavor. E.g. one example of launching such a shell could be…

    cmd.exe /k %BRANCH_PATH%\buildenv.bat <architecture> <build-flavor> <build-types> <etc...>

    The architectures can vary between things like x86, amd64, ARM, build flavors vary between debug, check, retail, release, etc… The build-types indicate the target like desktop, CoreCLR, metro. Even though all combination is not allowed, the allowed combination approaches around 30. In case of .NET Compact Framework which supports way more architectures (e.g. MIPS, PPC) and targets (e.g. Xbox360, S60) the combination is even larger.

    For day to day development I either need to enter this command each time to move to a different shell, or I have to create desktop shortcuts for all the combination. This becomes repetitive each time I move to a different code branch. I had created a small app that I ran each time I moved to a new branch and it would generate all the combination of shortcut given the branch details. However, our build requires elevation (admin-privilege). So even though I created the shortcuts, I’d have to either right click and use “Run as administrator” OR set that in the shortcuts property.

    image

    image

    This was a nagging pain for me. I couldn’t find any easy programmatic way to create a shortcut with Administrator privilege (I’m sure there is some shell API to do that). So finally I binary compared two shortcuts, one with the “Run as administrator” and one without. I saw that only one byte was different. So I hacked up a code to generate the shortcut and then modify the byte. I am sure there is better/safer way to do this, but for now this “Works for me”.

    The Code

    Since I didn’t find any online source for this code, I thought I’d share. Do note that this is a major hack and uses un-documented stuff. I’d never do this for shipping code or for that matter anything someone other than me would rely on. So use at your own risk… Also if you have a better solution let me know and I will use that…

       1:  // file-path of the shortcut (*.lnk file)
       2:  string shortcutPath = Path.Combine(shortCutFolder, string.Format("{0} {1}{2}.lnk", arch, flavor, extra));
       3:  Console.WriteLine("Creating {0}", shortcutPath);
       4:  // the contents of the shortcut
       5:  string arguments = string.Format("{0} {1} {2} {3}{4} {5}", "/k", clrEnvPath, arch, flavor, extra, precmd);
       6:   
       7:  // shell API to create the shortcut
       8:  IWshShortcut shortcut = (IWshShortcut)shell.CreateShortcut(shortcutPath);
       9:  shortcut.TargetPath = cmdPath;
      10:  shortcut.Arguments = arguments;
      11:  shortcut.IconLocation = "cmd.exe, 0";
      12:  shortcut.Description = string.Format("Launches clrenv for {0} {1} {2}", arch, flavor, extra);
      13:  shortcut.Save();
      14:   
      15:  // HACKHACK: update the link's byte to indicate that this is a admin shortcut
      16:  using (FileStream fs = new FileStream(shortcutPath, FileMode.Open, FileAccess.ReadWrite))
      17:  {
      18:      fs.Seek(21, SeekOrigin.Begin);
      19:      fs.WriteByte(0x22);
      20:  }
  • I know the answer (it's 42)

    My first Windows Phone App

    • 1 Comments
    lighthouse

    imageI have been involved with Windows Phone 7 from the very beginning and worked on delivering the CLR for the GA version and all the way through Nodo and Mango.

    All this while I was busy working on one of the lower layers and seeing bits and bytes flying by. For some time I wanted to try out writing an app for the phone to experience how it feels to be our developer customer. Unfortunately my skills fall more on system programming than designing good looking UI. I had a gazillion ideas floating in my head but most were beyond my graphic and UI design skills. I was also waiting for some sort of inspiration.

    Over the last school break our local King Country Library System which earned the best library system of the year award run an interesting effort. Students received a sheet which they filled up as they read books. On hitting 500 minutes and 1000 minutes of book reading they got prizes. One of the prize was a Kaleidoscope. Not the one with colored glass pieces like I had as a child, but one through which you could see everything around. This was the inspiration I was waiting for and I wrote an WP7 Mango app that does the same. Interestingly the app got a lot of good reviews and is rated at 5 stars (something I didn’t expect). It’s free, go check it out.

    Get the app here, documentation is at http://bonggeek.com/Kaleidoscope/

  • I know the answer (it's 42)

    Fastest way to switch mouse to left handed

    • 0 Comments
    Milky way

     

    I think I was born left handed, unfortunately I was brought up to be right handed. This was not uncommon in India 30 years back. Left hand usage was looked down upon.

    However, the good thing is I am still ambidextrous (equal handed) in some things like using the mouse. For ergonomic reason I keep switching between left and right hand usage to ensure I do not wear out both my wrists with 12 hour daily computer usage.

    The main problem I face when I switch and even otherwise when I am in left-hand mode is that most PC’s are configured for right hand use. In a course of the day I use many machines (upwards of 10) as I remote into data-center machines and even 3-4 local machines. The fastest way I have found to switch between left and right hand mouse is just run the following command either from the console or from WindowsKey+R

    rundll32.exe user32.dll,SwapMouseButton

    Basically this command calls the SwapMouseButton win32 function in user32.dll.

    If you know of anything more brief either using command shell or powershell do let me know.

  • I know the answer (it's 42)

    Custom Resolution of Assembly References in .NET

    • 0 Comments

    Right now I am helping out a team with an assembly resolution conflict bug. I thought I’d share the details out, because I am sure others have landed in this situation.

    In a complex managed applications, especially in cases where the application uses dynamically loaded plugins/addons, it’s common that all the assemblies required by these addons are not present in the assembly probing path. In the perfect world the closure set of all assembly references of an application is strong-named and the CLR binder uses standard probing order to find all assemblies and everything works perfectly. However, the world is not ideal.

    Requirement of having to resolve assemblies from different locations do arise and .NET has support for that. E.g. this stackoverflow question http://stackoverflow.com/questions/1373100/how-to-add-folder-to-assembly-search-path-at-runtime-in-net has been rightly been answered by pointing to AssemblyResolve event. When .NET fails to find an assembly after probing (looking through) the various folders .NET uses for assemblies, it raises an AssemblyResolve event. User code can subscribe to this event and supply assemblies from whatever path it wants to.

    This simple mechanism can be abused and results in major system issues. The main problem arises from over-eager resolution code. Consider an application A that uses two modules (say plugins) P1 and P2. P1 and P2 is somehow registered to A, and A uses Assembly.Load to load P1 and P2. However, P1 and P2 ships with various dependencies which it places in various sub-directories which A is unaware of and the CLR obviously doesn’t look into those folders to resolve the assemblies. To handle this situation both P1 and P2 has independently decided to subscribe to the AssemblyResolve event.

    The problem is that for all cases CLR fails to locate an assembly it will call these resolve-event handlers sequentially. So based on the order of registering these handlers, it is possible that for a missing dependency of P2 the resolution handler of P1 gets called. Coincidentally it is possible that the assembly CLR is failing to resolve is known to both P1 and P2. Could be because the name is generic or because maybe it’s a widely used 3rd party assembly which a ton of plugins use. So P1 loads this missing assembly P2 is referring to and returns it. CLR goes ahead and binds P2 to the assembly P1 has returned. This is when bad things start to happen because maybe P2 needs a different version of it. Crash follows.

    The MSDN documentation has already called out how to handle these issues in http://msdn.microsoft.com/en-us/library/ff527268.aspx. Essentially follow these simple rules

    1. Follow the best practices for assembly loading http://msdn.microsoft.com/en-us/library/dd153782.aspx
    2. Return null if you do not recognize the referring assembly
    3. Do not try to resolve assemblies you do not recognize
    4. Do not use Assembly.Load or AppDomain.Load to resolve the assemblies because that can result in recursive calls to the same ResolveEvent finally leading to stack-overflow.

    The skeleton code for the resolve event handler can be something like

    static Assembly currentDomain_AssemblyResolve(object sender, ResolveEventArgs args)
    {
        // Do not try to resolve assemblies which your module doesn't explicitly handle
        // There will be other resolve handlers that are called in-sequence, let them
        // do their job
        if (args.RequestingAssembly == null || !IsKnownAssembly(args.RequestingAssembly))
            return null;
    
        // parse and create name of the assembly being requested and then use your own logic
        // to locate the assembly
        AssemblyName aname = new AssemblyName (args.Name);
        string path = FindAssemblyByCustomLogic(aname);
    
        if (!File.Exists(path))
            return null;
                
        Assembly assembly = Assembly.LoadFile(path);
        return assembly;
    }

    Here you need to fill in the implementation of IsKnownAssembly which only returns true for assemblies that belong to your module.

  • I know the answer (it's 42)

    Quick/Dirty Function Coverage using Windbg

    • 0 Comments

    To find code coverage at line and block granularity you need a full-fledged code coverage tool. However, sometimes you can use a quick and dirty trick in WinDbg to see which functions are being called. This works well when you need to do this for a small set of functions, which is what I recently needed to do. Lets say it was for all the functions of a class called Graph.

    First I got the application under the debugger, wherein it automatically broke into the debugger at the application start. Then I added breakpoints to all these functions using the following

    0:000> bm test!Graph*::* 10000
      2: 003584a0          @!"test!Graph::Vertex::`scalar deleting destructor'"
      3: 00356f80          @!"test!Graph::Vertex::~Vertex"
      4: 00358910          @!"test!Graph::AddVertex"
      5: 00356b70          @!"test!Graph::~Graph"
      6: 003589d0          @!"test!Graph::RangeCheck"
      7: 003589b0          @!"test!Graph::Count"
      8: 00357ce0          @!"test!Graph::operator[]"
      9: 003561a0          @!"test!Graph::Vertex::Vertex"
     10: 00356170          @!"test!Graph::Vertex::Vertex"
     11: 00358130          @!"test!Graph::`scalar deleting destructor'"
     12: 003588a0          @!"test!Graph::AddEdge"
     13: 003551e0          @!"test!Graph::Graph"

    Here I am telling windbg to add breakpoints on all methods in the Graph class with a very large hit count of 0x10000. Then I just let the application proceed and play with the various controls. Finally I closed the application, when it again broke under the debugger. At this point I just list the breakpoints.

    0:000> bl
     1 e 00352c70     0001 (0001)  0:**** test!main
     2 e 003584a0     fd8f (10000)  0:**** test!Graph::Vertex::`scalar deleting destructor'
     3 e 00356f80     fcc7 (10000)  0:**** test!Graph::Vertex::~Vertex
     4 e 00358910     ff38 (10000)  0:**** test!Graph::AddVertex
     5 e 00356b70     ffff (10000)  0:**** test!Graph::~Graph
     6 e 003589d0     d653 (10000)  0:**** test!Graph::RangeCheck
     7 e 003589b0     c1de (10000)  0:**** test!Graph::Count
     8 e 00357ce0     fda7 (10000)  0:**** test!Graph::operator[]
     9 e 003561a0     fd8f (10000)  0:**** test!Graph::Vertex::Vertex
    10 e 00356170     ff38 (10000)  0:**** test!Graph::Vertex::Vertex
    11 e 00358130     ffff (10000)  0:**** test!Graph::`scalar deleting destructor'
    12 e 003588a0     ec56 (10000)  0:**** test!Graph::AddEdge
    13 e 003551e0     ffff (10000)  0:**** test!Graph::Graph

    The key fields are marked below

    13 e 003551e0 ffff (10000) 0:**** test!Graph::Graph

    10000 indicates that after 0x10000 times the breakpoint is hit should the debugger actually break on this. FFFF indicates how many times it is left for this break to happen. So a simple subtraction (0x10000 – 0xFFFF) tells us that this function has been called once. It’s easy to see that one Graph object was created and destroyed (1 call to ctor and 1 to dtor) and that the Graph<T>::Count has been called 15906 times (0x10000 – 0xC1DE). So I didn’t really miss any of the functions in that test. If I did it would say 10000 (10000) for the function that I missed.

  • I know the answer (it's 42)

    .NET: Loading Native (NGEN) images and its interaction with the GAC

    • 0 Comments

    It’s common for people to think that NGEN works with strong named assemblies only and it places output files or uses GAC closely. This is mostly not true.

    If you are new to this there’s a quick primer on NGEN that I wrote http://blogs.msdn.com/b/abhinaba/archive/2013/12/10/net-ngen-gac-and-their-interactions.aspx.

    GAC

    The Global Assembly Cache or GAC is a central repository where managed assemblies can be placed either using the command like gacutil tool or programmatically using Fusion APIs. The main benefits of GAC is

    1. Avoid dll hell
    2. Provide for a central place to discover dependencies (place your binary in the central place and other applications will find it)
    3. Allow side-by-side publication of multiple versions of the same assembly
    4. Way to apply critical patches, especially security patches that will automatically flow all app using that assembly
    5. Sharing of assemblies across processes. Particularly helpful for system assemblies that are used in most managed assemblies
    6. Provide custom versioning mechanisms (e.g. assembly re-directs / publisher policies)

    While GAC has it’s uses it has its problems as well. One of the primary problem being that an assembly has to be strongly named to be placed in GAC and it’s not always possible to do that. E.g. read here and here.

    NIC

    The NIC or Native Image Cache is the location where NGEN places native images. When NGEN is run to create a native image as in

    c:\Projects>ngen.exe install MyMathLibrary.dll

    The corresponding MyMathLibrary.ni.dll is placed in the NIC. The NIC has a similar purpose as GAC but is not the same location. NIC is placed at <Windows Dir>\assembly\NativeImages_<CLRversion>_<arch>. E.g. a sample path is

    c:\Windows\assembly\NativeImages_v4.0.30319_64\MyMathLibrary\7ed4d51aae956cce52d0809914a3afb3\MyMathLibrary.ni.dll

    NGEN places the files it generates in NIC along with other metadata to ensure that it can reliably find the right native image corresponding to an IL image.

    How does the .NET Binder find valid native images

    The CLR module that finds assemblies for execution is called the Binder. There are various kinds of binders that CLR uses. The one used to find native images for a given assembly is called the NativeBinder.

    Finding the right native image involves two steps. First the IL image and the corresponding potential native image is located on the file system and then verification is made to ensure that the native image is indeed a valid image for that IL. E.g. the runtime gets a request to bind against an assembly MyMathLibrary.dll as another assembly program.exe has dependency on it. This is what will happen

    1. First the standard fusion binder will kick in to find that assembly. It can find it either in
      1. GAC, which means it is strongly named. The way files are placed in GAC ensures that the binder can extract all the required information about the assembly without physically opening the file
      2. Find it the APPBASE (E.g. the local folder of program.exe). It will proceed to open the IL file and read the assemblies metadata
    2. Native binding will proceed only in the default context (more about this in a later post)
    3. The NativeBinder finds the NI file from the NIC. It reads in the NI file details and metadata
    4. Verifies the NI is indeed for that very same IL assembly. For that it goes through a rigorous matching process which includes (but not limited to) full assembly name match (same name, version,  public key tokens, culture), time stamp matching (NI has to be newer than IL), MVID (see below)
    5. Also verifies that the NI has been generated for the same CLR under which it is going to be run (exact .NET version, processor type, etc…) .
    6. Also ensures that the NI’s dependencies are also valid. E.g. when the NI was generated it bound against a particular version of mscorlib. If that mscorlib native image is not valid then this NI image is also rejected

    The question is what happens if the assembly is not strongly named? The answer is in that case MVID is used to match instead of relying on say the signing key tokens. MVID is a guid that is embedded in an IL file when a compiler compiles it. If you compile an assembly multiple times, each time the IL file is generated it has an unique MVID. If you open any managed assembly using ildasm and double lick on it’s manifest you can see the MVID

    .module MyMathLibrary.dll 
    // MVID: {EEEBEA21-D58F-44C6-9FD2-22B57F4D0193}

    If you re-compile and re-open you should see a new id. This fact is used by the NativeBinder as well. NGEN stores the mvid of the IL file for which a NI is generated. Later the native binder ensures that the MVID of the IL file matches with the MVID of the IL file for which the NI file was generated. This step ensures that if you have multiple common.dll in your PC and all of which has version 0.0.0.0 and is not signed, even then NI for one of the common.dll will not get used for another common.dll.

    The Double Loading Problem

    In early version of .NET when a NI file was opened the corresponding IL file was also opened. I found a 2003 post from Jason Zander on this. However, currently this is partially fixed. In the above steps look at step 1. To match NI with its IL a bunch of information is required from the IL file. So if that IL file comes from the GAC then the IL file need not be opened to get those information. Hence no double loading happens. However, if the IL file comes from outside the GAC then it is indeed opened and kept open. This causes significant memory overhead in large applications. This is something which the CLR team needs to fix in the future.

    Summary

    1. Unsigned (non strong-named) assemblies can also be NGEN’d
    2. Assemblies need not be placed in GAC to ngen them or to consume the ngen images
    3. However, GAC’d files provide better startup performance and memory utilization while using NI images because it avoids double loading
    4. NGEN captures enough metadata on an IL image to ensure that if its native image has become stale (no longer valid) it will reject the NI and just use the IL
  • I know the answer (it's 42)

    Windows Phone 7, my story

    • 0 Comments
    Olympic Peninsula - Hurricane Ridge

    Now that Windows Phone 7 is released all the gadget/tech blogs like TechCrunch, Engadget , Gixmodo are humming with reviews. Given that it’s a Version 1 product (ignore that 7 in the phone name) the reviews are great.

    Seeing all the buzz around gets me thinking about how I got involved in the project and my experiences. As the Windows Phone 7 head said “These are the good old days”

    Not so long ago, sometime after I joined the .NET Compact Framework team (and drew this cartoon) we started hearing about Windows Mobile reset and the awesome new phone being built and how our .NET Compact framework will be it’s runtime. Then we started getting access to CEPC images (these are phone OS that works on desktop virtual machines) to start developing and debugging our code. It felt strange at the beginning to see the ugly blue tiles and the truncated or cut-off text. These CEPC didn’t have hardware acceleration and hence there was no animation. The standard PC displays clipped off most of the phone display area and made the UI look really weird. I thought it was getting the wrong font as well as the text on top seemed to be cut off. All in all I wasn’t that impressed and just didn’t get the Metro UI.

    Working on the runtime is not all that glamorous and my view of Windows Phone 7 remained pretty much confined into the debugger. I’d interact with it over command shell and see bits and bytes flying by. So this is the view at 10 feel elevation (you can ignore the source file open in the IDE Smile, I wasn’t going to give a faux pas and reveal our source code, would I?).

    image

    As time went by and I had a chance to see the first prototype device, I suddenly “got” the Metro UI. It felt wonderful, a no pretense UI which doesn’t try to fake 3D buttons and kind of gets out of they way. The at a glance UI totally made sense (just look on the locked screen or the start window and know your email count, schedule and other bunch of things).  I started playing with the initial apps (Jelly Splat Smile) and then the realization dawned, each time someone picks up the device and launches any app, our code would start executing. Exactly the reason I joined Microsoft.

    As more and more applications started getting developed by 3rd parties the power of our developer tools really shows up. IMO with the Windows Phone Developer tools (Visual Studio 2010 based), Expression, XNA Game Studio, Emulator, Silverlight and host of other 1st and 3rd party tools, WP7 easily has the best development story out there and it’s just V1.

    Now when I see my daughter play Flowerz or I watch a Netflix movie and get a 10,000 feet view, it feels strangely wonderful and at the same time disconcerting (did you see http://abstrusegoose.com/307 ?). I strongly believe that Windows Phone 7 is one of the best Phones I have ever used and it passes the significant other test with flying colors. 

     

    image

  • I know the answer (it's 42)

    Delay Sending an Email

    • 0 Comments
    Cascade Mountains, WA

    Like many engineer I am partially fueled by the engineers angst. I am not always able to vent it in code and sometimes resort to sending really dumb emails :).

    I figured out that I can tolerate delaying sending my emails over having to smack my face every other day. Basically all emails I send is delayed by about 5 minutes and that makes a huge difference. Somehow you always know you’ve sent something you shouldn’t have in the first 2 minutes of sending that email.

    This is how you set this up on Outlook 2010… Bring up the new rules UI from the ribbon (Home > Rules > Managed Rules and Alerts)

    image

    Next you’d get another UI just hit next and yes on the following warning.

    image

    After that choose the delay duration. I use 5 minutes

    image

    Bonus!!!

    In case you want to pretend that you are hard at work when you aren’t then you can also use the timed email feature in Outlook. In the new email window use the following.

    image

    image

    Unfortunately if you are an software engineer your work is measured by the amount and quality of code you check in and not emails so sadly this doesn’t work Smile

  • I know the answer (it's 42)

    Emulator updates for the new Windows Phone Mango tools

    • 0 Comments

     

    GPS

    One of the most requested feature for the emulator was support for sensor. Developers were apparently asking their managers for travel budget to go to Hawaii/Europe so that they could test out their location based apps :). Today at MIX11 we announced that GPS sensor support for the emulator will be shipped in the next version of the tools. Some of the features that will be present are

    1. Developers would be able to launch a new tool window from the emulator and in that click on locations on maps to feed that location to apps running in the emulator. This covers building apps like Yelp, Weather app on the device that needs static location information
    2. Have pre-recorded location and able to play that out so that new locations are sent to the phone at a preset rate. This covers building apps like sports tracker, navigation apps.
    3. The app running in the emulator will not need to change in anyway or even know that the data is simulated. It will come over the usual sensor channels and even apps like the built in bing maps app will pick up the fed in data.

    Here’s a short video which I quickly took from the laptop of our PM. Being away from family in Las Vegas was getting me down so he took me to Seattle in a second (or rather simulated it).

    Direct link http://www.youtube.com/watch?v=wt0DdiYqdkk 

     

    Accelerometer

    Another new sensor support is the accelerometer. So no more picking up the PC monitor running the emulator and rotating it to see if the emulator gets the movements. There’s going to be a new tool window using which the developer can feed in motion to the apps running in the emulator. Also some pre-canned motions like shake will be present. Here’s another video of that

    Direct link http://www.youtube.com/watch?v=Gc1kuXj7eCE 

  • I know the answer (it's 42)

    Managing your Bugs

    • 0 Comments
    Rainier

    I have worked on bunch of large scale software product development that spanned multiple teams, thousands of engineers, many iterations. Some of those products plug into even larger products. So obviously we deal with a lot of bugs. Bugs filed on us, bugs we file on other partner teams, integration bugs, critical bugs, good to have fixes and plan ol crazy bugs. However, I have noticed one approach to handling those bugs which kind of bugs me.

    Some people just cannot make hard calls on the gray area bugs and keeps hanging them around. They bump the priority down and take those bugs from iteration to iteration. These bugs always hang around as Priority 2 or 3 (in a scale of Priority 0 through 3).

    I personally believe that bugs should be dealt with the same way emails should be dealt with. I always follow the 4 Ds for real bugs.

    Do it

    If the bug meets the fix bar and is critical enough, just go fix it. This is the high priority critical bug which rarely gives rise to any confusion. Something like the “GC crashes on GC_Sweep for application X”

    Delete it (or rather resolve it)

    If you feel a bug shouldn’t be fixed then simply resolve the bug. Give it back to the person who created the bug with clear reasons why you feel this shouldn’t be fixed. I know it’s hard to say “won’t fix”, but the problem is if one was to fix every bug in the system then the product will never ship. So just don’t defer the thought of making this hard call. There is no benefit in just keeping the bug hanging around for 6 months only to make the same call 2 weeks prior to shipping, citing time constraint as a reason. Make the hard call upfront. You can always tag the bug appropriately so that for later releases you can relook.

    Defer it

    Once you have made a call to fix the bug it should be correctly prioritized. With all bugs you know you won’t be fixing out of the way, its easy to make the right call of fixing it in the current development cycle or move it to the next iteration.

    Delegate it

    If you feel someone else needs to take a look at the bug because it’s not your area of expertise, QA needs to give better repro steps, someone else needs to make a call on it (see #1 and #2 above) then just assign it to them. No shame in accepting that you aren’t the best person to make a call.

  • I know the answer (it's 42)

    MIX 2010 Announcements

    • 0 Comments
    Bidar Trip - Mohamad Gawan Madrasa

    Today is a very big day for me and my team. The stuff we have been working on is finally went live on stage MIX 2010.

    Windows Phone 7 is easily the most dramatic re-entry/reset Microsoft has undertaken in any field it’s in. We announced the phone in MWC and the reception it got was phenomenal. However, in the new connected devices world, the application platform+market story is as or more important than the device itself. Today at MIX we just disclosed details of that.

    1. The application model supported on Windows Phone 7 series will be managed only and will leverage Silverlight, XNA and the .NET Compact Framework. So basically you can reuse your C#/.NET and Silverlight skills to build amazing experience on the phone. I will get into details of this in the coming posts
    2. Microsoft just announced free Windows Phone Developer Tools (preview bits available for download head over to http://developer.windowsphone.com/ )
      1. This builds on the amazing development experience brought in by the Visual Studio 2010 shell.
      2. You can either get a standalone Visual Studio 2010 Express for Windows Phone Or an addin for your previously installed VS 2010
      3. Windows Phone 7 series emulator (yes you do not even need a device to develop for Windows Phone)
    3. A version of Expression Blend is also demonstrated for the Windows Phone (a free version will be available for download in the coming months)
    4. A new Windows Phone Marketplace is being put into place
      1. Developers can earn 70% revenue from the applications they sell of the marketplace
      2. The registration to the market place is not free but it’ll be free for the DreamSpark program for students

    Our team was involved with building the IDE, the emulator and the core .NET runtime that powers the applications. So watch out and in the coming posts I’ll deep dive into these areas.

  • I know the answer (it's 42)

    Inside the Windows Phone Emulator

    • 0 Comments
    US

    Learn from the dev lead and PM of Windows Phone 7 Emulator on how it works and delivers the awesome performance.

    Some key points

    1. The emulator is essentially a x86 based Virtual Machine running full image of Windows Phone 7 OS
    2. It emulates various peripherals (e.g. audio, networking), the list hopefully will grow in the future
    3. Supports multi-touch if your host PC has a touch screen
    4. It uses the GPU on the host PC to accelerate the graphics
    5. Shameless plug: One of the reason it’s possible to emulate x86 and still have managed apps running as-is, is because the runtime (.NET Compact Framework) supports both x86 and ARM seamlessly (provides JITer for both architectures and runs the same managed code on both seamlessly)

    Get Microsoft Silverlight

  • I know the answer (it's 42)

    We Believe in Sharing

    • 0 Comments
    Good Morning

    In Building NETCF for Windows Phone 7 series we put in couple of features to enhance startup performance and ensure that the working set remains small. One of these features added is code/data sharing.

    Native applications have inherent sharing where multiple processes can share the same executable code. However, in case of managed code running on NETCF 3.5 even when multiple applications use the same assembly, the managed code in them are JITed in context of each process separately. This results in suboptimal resource utilization because of the following

    1. Repeated JITing of the same code
    2. Memory overhead of having identical copies of the same JITed code for every process running them. The execution engine also maintains records of the various types loaded and even though two (or more) processes may be loading the same types from the same assemblies; per process copies are maintained for them.

    Together the overhead can be significant.

    In the latest version of the runtime shipping with Windows Phone 7 (.NET Compact Framework 3.7) we added a feature to share both the JITed code and these type information across multiple managed processes.

    image

    The runtime essentially uses a client server architecture. We added a new kernel mode server which is responsible of maintaining a system wide shared heap. The server on receiving requests from the various client processes (each managed app is a client) JITs code and type information into this shared heap. The shared heap is accessed Read-only from all the client apps and Read/write from the server.

    When a process requests for the same type to be loaded or code to be JITed it just re-uses the already loaded type info or JITed code from the shared heap.
    What is shared

    1. Various Execution engine data-structures like loader type-information
    2. JITed code from a predefined list of platform assemblies (user code is not shared).

    Do note that user code is not shared and neither is all platform code. Sharing arbitrarily would actually increase cost if they ultimately didn’t get reused in a lot of applications. The list of assemblies/data shared was carefully chosen and tuned to ensure only those are shared that provided the maximum performance benefit.

    Sharing provides the following advantages

    1. Warm startup time is significantly reduced. When a new application is coming up a large
      percentage of the initial code and type-info is already found on the shared heap and hence
      there is significant perf boost. The exact number depends on the applications but it’s common
      to get around 30% gains
    2. Significant reduction in net working set as a lot of commonly used platform assembly JITed code is re-used
    3. Obviously there are other goodness like less JITing means less processing on the device

    Like user code the system is also capable of pitching the shared code when there is a low memory situation on the device.

  • I know the answer (it's 42)

    Silverlight: Where are my colors

    • 0 Comments
    Barsha Mangal 

    The Silverlight System.Windows.Media.Colors class is a trimmer counterpart of the WPF Colors class. E.g. Colors.AliceBlue is available in WPF but not present in Silverlight. However, these standard colors are indeed available in Silverlight XAML.

    In effect in Silverlight XAML you can use

    <Border Background="AliceBlue" BorderBrush="Coral" BorderThickness="2" CornerRadius="10" >

    However, if you try that in code it fails to compile

    foo.Background = new SolidColorBrush(Colors.Coral);

    This means if you ever need to use standard colors in Silverlight code you have to use RGB values, which for Coral would be something like

    foo.Background = new SolidColorBrush(Color.FromArgb(255,255,127,80));

    This is kind of painful. Moreover, you need to frequently be able to convert between HTML color, RGB and XAML standard color names when you are developing an Silverlight application. To make the job easier I hacked up a small Silverlight 2 app at http://www.bonggeek.com/Ag/Colors/. You can do the following with this

    1. Change colors with sliders, RGB, HTML, XAML standard color names to see what the color really looks like
    2. All the format is kept in sync, this means if you change the sliders to get to the color Salmon then the RGB value and HTML will be updated and the drop down will switch to show the selected colors name
    3. You can easily create variants of standard color. E.g. select Salmon using the drop down and then push the sliders to get to a slightly lighter/darker shade of the color

    image

    Enjoy and extend as you please. Sources are available here.

  • I know the answer (it's 42)

    NETCF: Count your bytes correctly

    • 0 Comments
    Pirate

    I got a stress related issue reported in which the code tried allocating a 5MB array and do some processing on it but that failed with OOM (Out of Memory). It also stated that there was way more than 5 MB available on the device and surely it’s some bug in the Execution Engine.

    Here’s the code.

    try{
        byte[] result;
        long GlobalFileSize = 5242660;  //5MB
        result = new byte[GlobalFileSize];
        string payload = Encoding.UTF8.GetString(result, 0, result.Length);
        System.Console.WriteLine("len " + payload.Length);
    }
    catch (Exception e)
    {
        System.Console.WriteLine("Exception " + e); 
    } 

    The problem is that the user didn’t count his bytes well. The array is 5MB and it actually gets allocated correctly. The problem is with the memory complexity of the UTF8.GetString which allocates further memory based on it’s input. In this particular case the allocation pattern goes something like

      5MB  -- Byte Array allocation (as expected and works)
    
      5MB  -- Used by GetString call
     10MB  -- Used by GetString call
      5MB  -- Used by GetString call
     10MB  -- Used by GetString call

    So GetString needed a whooping 30MB and the total allocation is around 35MB which was really not available.

    Morale of the story: Count your bytes well, preferably through a tool like Remote Performance Monitor

Page 13 of 15 (374 items) «1112131415