October, 2009

  • Kirill Osenkov

    Irony

    • 7 Comments

    (Ironically, this post is not about irony in it's traditional sense)

    Irony (http://irony.codeplex.com) is an open-source .NET compiler construction framework written by Roman Ivantsov. It is a ".NET Language Implementation Toolkit". The language grammar is described in C# (or any other .NET language).

    So instead of generating the scanner and the parser from a grammar description written in an external DSL, Irony uses a .NET object graph to host the grammar description (internal DSL), and it uses this in-memory grammar description to drive the scanner and the parser at runtime.

    One huge advantage of this approach is that the language description becomes orthogonal to the scanner and parser implementation. Writing and maintaining a language becomes easier. See an example of grammar definition further down in this post.

    Arithmetic expression evaluation

    The reason I'm talking about all this is that I was looking around for an expression parser for Live Geometry. I was able to compile expressions earlier using DLR, but there were two issues with that.

    1. First, DLR has quite a footprint – adding a dependency on DLR to a Silverlight binary can grow the size of the .xap file from 400 KB to 1400 KB – and that's not good.
    2. Second, DLR didn't seem to expose the parse trees easily – and I do need the access to the parse trees for inspection and manipulation, such as calculating dependencies between formulas.

    So I started looking around. There are really a lot of related projects out there, I'll just mention the ones on CodePlex:

    1. http://antlrcsharp.codeplex.com
    2. http://csparser.codeplex.com
    3. http://expressioneval.codeplex.com
    4. http://flee.codeplex.com
    5. http://ilcalc.codeplex.com
    6. http://lazyparser.codeplex.com
    7. http://linqovercsharp.codeplex.com
    8. http://ncalc.codeplex.com
    9. http://simpleexpressioneval.codeplex.com
    10. http://simplemathparser.codeplex.com
    11. http://expressionscompiler.codeplex.com

    There might be more. My requirements were for the tool to output a parse tree and give me full access to it. Another requirement is that the grammar description has to be as simple as possible and as flexible and extensible as possible (because I planned to extend the language with custom elements such as method calls and property access). However full blown C# and LINQ parsers were an overkill. Other good projects were really fast, but didn't give me the parse tree.

    So I settled on Irony, and so far I'm pretty happy that I did. It didn't provide the Silverlight version out of the box, but I'm talking with Roman about Silverlight support. For now, it took me an hour to make the Irony sources build for Silverlight and I built a nice little Irony.Silverlight.dll for my own project.

    Expression parser in Irony

    Here's how I declared the grammar for the language that I need:

    // This grammar is based on the ExpressionEvaluatorGrammar from Irony.Samples
    // Copyright (c) Roman Ivantsov
    // Details at http://irony.codeplex.com
    [Language("Expression", "1.0", "Dynamic geometry expression evaluator")]
    public class ExpressionGrammar : Irony.Parsing.Grammar
    {
        public ExpressionGrammar()
        {
            this.GrammarComments = @"Arithmetical expressions for dynamic geometry.";
    
            // 1. Terminals
            var number = new NumberLiteral("number");
            var identifier = new IdentifierTerminal("identifier");
    
            // 2. Non-terminals
            var Expr = new NonTerminal("Expr");
            var Term = new NonTerminal("Term");
            var BinExpr = new NonTerminal("BinExpr");
            var ParExpr = new NonTerminal("ParExpr");
            var UnExpr = new NonTerminal("UnExpr");
            var UnOp = new NonTerminal("UnOp");
            var BinOp = new NonTerminal("BinOp", "operator");
            var PostFixExpr = new NonTerminal("PostFixExpr");
            var PostFixOp = new NonTerminal("PostFixOp");
            var AssignmentStmt = new NonTerminal("AssignmentStmt");
            var AssignmentOp = new NonTerminal("AssignmentOp");
            var PropertyAccess = new NonTerminal("PropertyAccess");
            var FunctionCall = new NonTerminal("FunctionCall");
    
            // 3. BNF rules
            Expr.Rule = Term | UnExpr | FunctionCall | PropertyAccess | BinExpr;
            Term.Rule = number | ParExpr | identifier;
            ParExpr.Rule = "(" + Expr + ")";
            UnExpr.Rule = UnOp + Term;
            UnOp.Rule = ToTerm("+") | "-" | "++" | "--";
            BinExpr.Rule = Expr + BinOp + Expr;
            BinOp.Rule = ToTerm("+") | "-" | "*" | "/" | "^";
            PropertyAccess.Rule = identifier + "." + identifier;
            FunctionCall.Rule = identifier + ParExpr;
            this.Root = Expr;
    
            // 4. Operators precedence
            RegisterOperators(1, "+", "-");
            RegisterOperators(2, "*", "/");
            RegisterOperators(3, Associativity.Right, "^");
    
            RegisterPunctuation("(", ")", ".");
            MarkTransient(Term, Expr, BinOp, UnOp, AssignmentOp, ParExpr);
        }
    }

    That's it! Note how Irony uses operator overloading for | and + to build rules. We need ToTerm("+") call on any of the operands to give the C# compiler a hint about the types so the operator overloading can succeed. This is a good example of an internal DSL hosted in C#.

    Consuming the parse tree

    Now, here's how to use the ExpressionGrammar class to create and use a parser:

    Grammar grammar = new ExpressionGrammar();
    Parser parser = new Parser(grammar);
    ParseTree parseTree = parser.Parse("sin(2 * x) + 1");

    Producing a LINQ Expression Tree from a parse tree

    Now, given the parse tree from the previous step, I just quickly wrote an ExpressionTreeBuilder that produces a bound expression tree out of it, and a Binder that helps resolve names. The Compiler class is a façade for the whole thing:


    public class Compiler
    {
        public static Func<double, double> CompileFunction(string functionText)
        {
            ParseTree ast = ParserInstance.Parse(functionText);
            ExpressionTreeBuilder builder = new ExpressionTreeBuilder();
            Expression<Func<double, double>> expression = builder.CreateFunction(ast.Root);
            Func<double, double> function = expression.Compile();
            return function;
        }
    
        static Parser ParserInstance = new Parser(ExpressionGrammar.Instance);
    }
    
    
    public class Binder { public void RegisterParameter(ParameterExpression parameter) { parameters.Add(parameter.Name, parameter); } ParameterExpression ResolveParameter(string parameterName) { ParameterExpression parameter; if (parameters.TryGetValue(parameterName, out parameter)) { return parameter; } return null; } Dictionary<string, ParameterExpression> parameters = new Dictionary<string, ParameterExpression>(); public Expression Resolve(string identifier) { return ResolveParameter(identifier); } public MethodInfo ResolveMethod(string functionName) { foreach (var methodInfo in typeof(System.Math).GetMethods()) { if (methodInfo.Name.Equals(functionName, StringComparison.InvariantCultureIgnoreCase)) { return methodInfo; } } return null; } }

    public class ExpressionTreeBuilder
    {
        public ExpressionTreeBuilder()
        {
            Binder = new Binder();
        }
    
        public Binder Binder { get; set; }
    
        public Expression<Func<double, double>> CreateFunction(ParseTreeNode root)
        {
            ParameterExpression parameter = Expression.Parameter(typeof(double), "x");
            Binder.RegisterParameter(parameter);
            Expression body = CreateExpression(root);
            var result = Expression.Lambda<Func<double, double>>(body, parameter);
            return result;
        }
    
        Expression CreateExpression(ParseTreeNode root)
        {
            if (root.Term.Name == "BinExpr")
            {
                return CreateBinaryExpression(root);
            }
    
            if (root.Term.Name == "identifier")
            {
                return Binder.Resolve(root.Token.Text);
            }
    
            if (root.Term.Name == "number")
            {
                return CreateLiteralExpression(Convert.ToDouble(root.Token.Value));
            }
    
            if (root.Term.Name == "FunctionCall")
            {
                return CreateCallExpression(root);
            }
    
            return null;
        }
    
        Expression CreateCallExpression(ParseTreeNode root)
        {
            string functionName = root.ChildNodes[0].Token.Text;
            Expression argument = CreateExpression(root.ChildNodes[1]);
            MethodInfo method = Binder.ResolveMethod(functionName);
            return Expression.Call(method, argument);
        }
    
        Expression CreateLiteralExpression(double arg)
        {
            return Expression.Constant(arg);
        }
    
        Expression CreateBinaryExpression(ParseTreeNode node)
        {
            Expression left = CreateExpression(node.ChildNodes[0]);
            Expression right = CreateExpression(node.ChildNodes[2]);
    
            switch (node.ChildNodes[1].Term.Name)
            {
                case "+":
                    return Expression.Add(left, right);
                case "-":
                    return Expression.Subtract(left, right);
                case "*":
                    return Expression.Multiply(left, right);
                case "/":
                    return Expression.Divide(left, right);
                case "^":
                    return Expression.Power(left, right);
            }
            return null;
        }
    }

    This just demonstrates the principle. One could easily extend this to write a full blown expression compiler, but this is good enough for my purposes for now. Live Geometry now uses this to evaluate math expressions and plot function graphs. As always, you can get the source from here.

  • Kirill Osenkov

    Indexed Properties in C# 4.0

    • 12 Comments

    Executive summary:

    1. In C# 4.0 you can call parameterized properties declared in COM using the indexer syntax, for instance instead of excel.get_Range("a") you can now write excel.Range["a"].
    2. You can’t declare your own indexed properties from C#. We have no plans of adding the ability to declare your own properties with parameters. Instead, the recommended way is to use a type with an indexer.
    3. The feature is immediately available in Visual Studio 2010 Beta 2 for your indexing pleasure.

    So I guess this hasn’t had a lot of press coverage so far (although Sam hinted about it). Even the published C# 4.0 language specification and the C# Future page still don’t mention it as of now. This is because we’ve implemented this feature very late in the cycle, as a DCR (Design Change Request). The language design team felt that we should complete the COM interop story in 4.0 and this one was the last missing piece of the puzzle. When Paul was in Redmond this summer, he was tasked with testing the IDE support for this feature, and I was helping out with the test infrastructure.

    The new syntax

    The pattern is very simple. Wherever you use COM interop and have to call get_X() and set_X(), now you can just call X[], which we feel is a more natural syntax:

    // before
    excel.get_Range("A1").set_Value(Type.Missing, "ID");
    
    // after
    excel.Range["A1"].Value = "ID";

    Let’s also take Scott Hanselman’s example from Beta 1:

    var excel = new Excel.Application();
    excel.Visible = true;
    excel.Workbooks.Add();
    excel.get_Range("A1").Value2 = "Process Name";
    excel.get_Range("B1").Value2 = "Memory Usage";

    Now you can simplify this even further:

    var excel = new Excel.Application();
    excel.Visible = true;
    excel.Workbooks.Add();
    excel.Range["A1"].Value = "Process Name";
    excel.Range["B1"].Value = "Memory Usage";

     

    This is just syntactic sugar – the compiler emits calls to the get_ and set_ accessors behind the stage.

    Omitting []

    In case that all the parameters are optional and none of the arguments are specified, you should omit the empty []. Having Value[] is illegal.

    This is the reason you can replace Value2 with Value in the example above: Value is an indexed property and you’re calling it without specifying any arguments – in this case we omit the brackets [] altogether. Earlier, without indexed properties support, we had to introduce the ugly Value2, because you otherwise had to call get_Value().

    IDE support

    My team, on the IDE side, provided IntelliSense support for this new language feature:

    • completion list:

    image

    • parameter help:

    image

    • however Quick Info still shows you that in reality the call simply binds to the get_Range method:

    image

    As we were designing the feature, it turned out that adding compiler support for it is not the only tricky part. There were a couple of interesting problems in the IDE space as well. For example, what do you show in Quick Info for the following intexed property call?

    A.B[C]++;

    Do we now show get_B or set_B?

    When the IDE can’t guess which accessor we’re talking about (for example, in incomplete code), we by default bind to the get_accessor.

    Backwards compatibility

    For backwards compatibility reasons, using the accessors get_ and set_ directly is still valid and available in IntelliSense, because we didn’t want to break all existing COM interop code out there.

    Why not allow declaring indexed properties in C#?

    A common question that we expect we’ll be getting is “why just consume? why not allow to declare such properties in C#?”. Well, the answer is not even that we first have to cost, design, spec, prototype, implement and test this feature, but rather that we think that declaring a type with an indexer is a preferred approach. It’s useful to separate the responsibilities:

    • The property is there to get an object. The property belongs to the parent object.
    • The indexer is there on the returned object to enumerate it. The indexer belongs to the returned object.

    We shouldn’t be mixing these together.

    Static or dynamic?

    Accessing indexed properties is supported from both static and dynamic code.

    kick it on DotNetKicks.com
  • Kirill Osenkov

    Visual Studio 2010 Beta 2 Known Issues

    • 14 Comments

    [This is Part 1. Read Part 2 here]

    Now that we’ve shipped Beta2 and the world is busy downloading the fresh new bits, I’m very excited to know what do you guys think? Will you like it? Will there be major issues that we missed? Time will show :)

    By definition, this Beta 2 release is not final, and there are still bugs lurking around out there. We are very busy fixing those bugs for RTM, but for now, there are some that we haven’t had time to fix before Beta2.

    This post lists some of the known issues that are in VS 2010 Beta 2. It’s not in my powers to maintain a comprehensive list here, I’ll just mention the ones which I was personally involved with in my day-to-day work. Apologies that I have found them too late, but I guess better late then never...

    Updates – new!

    Posted the Part 2 of the issues (updated 11/12/2009)

    http://blogs.msdn.com/kirillosenkov/archive/2009/11/12/visual-studio-2010-beta-2-known-issues-part-2.aspx

    All further updates will happen there or will wait until Part 3.

    The official Beta2 known issues list (updated 10/26/2009)

    http://go.microsoft.com/fwlink/?LinkID=166199

    Updating the .sln file from Beta1 to Beta2 (updated 10/26/2009)

    Need to upgrade .sln file from Beta2 in order to be able to double-click it. See Jon’s post here for more details:

    http://msmvps.com/blogs/jon_skeet/archive/2009/10/26/migrating-from-visual-studio-2010-beta-1-to-beta-2-solution-file-change-required.aspx

    Setup dialog UI layout wrong on High DPI

    image

    Embarrassing! Unfortunately, not everyone on the Visual Studio team is rigorous about non-default testing, such as High DPI, Accessibility, etc. We will hopefully fix this one before RTM. Also recent data shows that non-96 DPI is a very common setting, a lot of people actually use 120 DPI and others. I personally use 120 DPI, that’s how I find these bugs.

    SmartTag high DPI issues

    image

    With high DPI, the SmartTag menu is missing horizontal menu separator bars. Also the right vertical edge of the SmartTag button disappears. These bugs are a recent regression from WPF introducing the UseLayoutRounding API. The WPF team is looking into fixing these issues before the release.

    QuickInfo tooltip trimmed at high DPI

    image

    This is another regression from the new DWrite technology. The last word in the tooltip is missing! This one is fixed already.

    Thin line on top of the active document tab

    image

    Again, another high DPI issue. You can only notice this one if you look closely. We’ve fixed this already as well.

    Outlining Expand/Collapse glyphs are shifted several pixels up

    image

    Fixed already. A lot of people complained about this internally.

    Zoom combobox in the bottom-left corner of the editor isn’t aligned with the horizontal scrollbar

    image

    This is fixed.

    Don’t tear off the Call Hierarchy tool window!

    When you try to undock the C# Call Hierarchy toolwindow and drag it away (for example, to another monitor), VS will crash. This one is too embarrassing, because I was the one who is responsible for testing Call Hierarchy. To my defense, I was on vacation when this regressed and when I came back and found this, it was too late to fix. Shell UI team changed something about the WPF toolwindow implementation and we had some layout logic that didn’t expect double.Infinity as an argument, so we crashed. This one’s already fixed in recent builds.

    Vertical separator bar in the Call Hierarchy tool window doesn’t resize the details pane

    This also only reproes under 120 DPI and above. You can’t resize the two panes below by dragging the vertical bar:

    image

    And again, this is one that I should have caught and missed. When I caught this, it was too late to fix for Beta2. The WPF team is looking at this one right now.

    Error message connecting to a TFS server via HTTPS

    Well, guess what, the TFS client team has done it again! TFS HTTPS story was broken in Beta1, and it still has a bug in Beta2. However the good news is that there is a really easy workaround this time.

    When adding a new TFS HTTPS webserver (e.g. a Codeplex server at http://codeplex.com), after entering your credentials you will see this:

    image

    Don’t panic! Just enter your credentials again and things will work just fine. If you used the Tip (http://blogs.msdn.com/kirillosenkov/archive/2009/09/27/tip-don-t-enter-your-codeplex-credentials-every-time.aspx), then you will be unaffected by this and things will hopefully run smoothly.

    Ctrl+Alt+DOWN ARROW doesn’t bring up the active document list

    Ctrl+Alt+Down used to bring up the active document list, in Beta2 it doesn’t. This is fixed.

    Start Page: scroll wheel doesn’t work at the news feed

    Also the scrollbar thumb only moves in discrete steps. The Shell UI team decided not to fix this, because they don’t have time and resources for this. I actually was surprised to discover that the scrollbar didn’t work in 2008 either. Never noticed this until recently.

    VS Command prompt shortcut is not getting installed if you don’t install C++

    This one will be fixed as well.

    Hovering over ‘var’ during debugging doesn’t show the inferred type

    Jeffrey Richter told me about this minor annoyance. Although our language service correctly reports the inferred type when you hover the mouse cursor over ‘var’ in design time, it doesn’t work in debug mode. Since Jeff teaches a lot of courses in debugging and threading, this was bugging him ever since we shipped C# 3.0. Well, we finally fixed it for 2010 RTM. Thanks to Jeff for reporting this!

    Visual Studio doesn’t start maximized when first launched

    A while back I’ve logged a bug against the Shell team to “just start VS maximized for heaven’s sake”. They did the fix, but somehow it didn’t make it into the Beta2 branch. They’ll hopefully fix this for RTM.


    Well, these were the ones worth mentioning I guess. Apologies if you run into any of these or any other bugs for that matter. I will keep updating this post with more issues that I find or I think users should be aware of. Please keep in mind that this is an unofficial list.

    Please do let us know about any issues you find by submitting a connect bug. If the bug/suggestion/feedback is related to the C# language or IDE, also feel free to let me know directly or leave a comment on this blog. Since I work closely with the VS editor team, it’s worth watching their blog and giving them feedback: http://blogs.msdn.com/vseditor. Also, if you have any feedback about the new text rendering in WPF 4.0 and Visual Studio, the WPF Text team has a blog here: http://blogs.msdn.com/text.

  • Kirill Osenkov

    Saving images (.bmp, .png, etc) in WPF/Silverlight

    • 10 Comments

    I’ve recently added a new feature to Live Geometry that allows users to save the current drawing as a bitmap or a .png file. Just push the save button and pick the desired image format in the Save dialog:

    image

    Fortunately, both WPF and Silverlight support saving full visual contents of any visual into a file on disk. However the approach is somewhat different.

    Saving images in WPF

    WPF can save any Visual to an image and it supports several formats out of the box via a concept of Encoders. Here’s a sample for .bmp and .png:

    void SaveToBmp(FrameworkElement visual, string fileName)
    {
        var encoder = new BmpBitmapEncoder();
        SaveUsingEncoder(visual, fileName, encoder);
    }
    
    void SaveToPng(FrameworkElement visual, string fileName)
    {
        var encoder = new PngBitmapEncoder();
        SaveUsingEncoder(visual, fileName, encoder);
    }
    
    void SaveUsingEncoder(FrameworkElement visual, string fileName, BitmapEncoder encoder)
    {
        RenderTargetBitmap bitmap = new RenderTargetBitmap(
            (int)visual.ActualWidth,
            (int)visual.ActualHeight,
            96,
            96,
            PixelFormats.Pbgra32);
        bitmap.Render(visual);
        BitmapFrame frame = BitmapFrame.Create(bitmap);
        encoder.Frames.Add(frame);
    
        using (var stream = File.Create(fileName))
        {
            encoder.Save(stream);
        }
    }

    These types are all in System.Windows.Media.Imaging.

    Saving images in Silverlight 3

    In Silverlight, the encoders don’t come as part of the Silverlight runtime – but fortunately there is a project on CodePlex called ImageTools (http://imagetools.codeplex.com) that provides necessary support. You will need to download the following binaries and add them as references to your Silverlight project:

    • ICSharpCode.SharpZipLib.Silverlight
    • ImageTools
    • ImageTools.IO
    • ImageTools.IO.Png (only if you want .png support)
    • ImageTools.IO.Bmp (only if you want .bmp support)
    • ImageTools.Utils

    After that, you can call the ToImage() extension method on any Canvas:

    void SaveAsPng(Canvas canvas, SaveFileDialog dialog)
    {
        SaveToImage(canvas, dialog, new PngEncoder());
    }
    
    void SaveAsBmp(Canvas canvas, SaveFileDialog dialog)
    {
        SaveToImage(canvas, dialog, new BmpEncoder());
    }
    
    void SaveToImage(Canvas canvas, SaveFileDialog dialog, IImageEncoder encoder)
    {
        using (var stream = dialog.OpenFile())
        {
            var image = canvas.ToImage();
            encoder.Encode(image, stream);
        }
    }

    Since you can’t write to disk directly in Silverlight, you can pass a SaveFileDialog and use its stream, or you can obtain a stream elsewhere and pass that. The ToImage() extension method does the dirty work that we had to do ourselves in WPF.

    Big thanks to http://imagetools.codeplex.com for their awesome library and encoders!

  • Kirill Osenkov

    Links

    • 0 Comments

    There are several good folks out there who regularly accumulate interesting links about all things .NET and other stuff on their blogs (cast in alphabetical order):

    Alvin Ashcraft

    Alvin Ashcraft's Morning Dew

    .NET Development Resources from a Progressive.NET Perspective

    Arjan Zuidhof

    Arjan’s World

    Arjan Zuidhof’s opinionated linkblog, with a hang to Alt.NET

    Charlie Calvert

    Charlie Calvert's Community Blog

    Charlie is the C# Community Program Manager on our team and has a great series called “Community Convergence”. I hope he never runs out of roman numerals :)

    Chris Alcock

    Reflective Perspective

    The caffeine fueled thoughts of a UK Software Developer and home of ‘The Morning Brew’

    Jason Haley

    Interesting Finds

    Ramblings of a .Net developer
    .ver 3:0:0:0

    Scott Guthrie

    ScottGu’s Link Listing tag

    Yes, ScottGu is a linkblogger!

    Steve Pietrek

    A Continuous Learner’s weblog

    My continuous learning of .NET, C#, VB.NET, ASP.NET, Delphi, Business Intelligence, Software Design and Development, Project Management, Object Oriented Development, Unit Testing, Development Tools ramblings....

    And since I feel a little linkbloggy myself today, here it goes:

Page 1 of 1 (5 items)