jaredpar's WebLog

VsVim Update Released (Version 0.7.0)

Jared Parsons — Thu, 04 Feb 2010 18:22:00 GMT

I just released an update to VsVim for Visual Studio 2010 Beta2. This should be available shortly from the extension manager in Visual Studio or it can be downloaded directly at the following link

Link: http://visualstudiogallery.msdn.microsoft.com/en-us/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329

New Features

Visual Mode Support
- Character, Line and Block Mode
- All movement commands supported
- Basic insert-delete operations added
Command Mode Commands - :substitute, :redo,
Support for mark navigation to global marks
Implemented almost all delete-insert operations in relevant modes
Normal Mode Commands: CTRL-R,
Expanded range parsing

Notable Bug Fixes

Issues with disabling conflicting key bindings on startup
Issues with not properly intercepting standard VS commands and routing them to Vim
Timing issue on startup that would cause certain components to not get created

Source Code

Source for this release is available at the following location. It is released under the MS-PL.

http://cid-dc25b20f65f628f8.skydrive.live.com/self.aspx/Public/VsVim/VsVim-0.7.0.zip

Future Plans

The next planned major update is version 0.8.0. It will consist of the following updates

Expanded set of commands supported in all modes
The . operator in normal mode
Ability for users to change settings. Settings supported is baked into VsVim I just haven’t exposed it via command mode or any UI
Better UI in general. Right now I use MessageBox.Show for any UI I need to display other than status bar updates. I would like to expand that to include a more configurable UI

I also expect to release a 0.7.1 version once the Visual Studio RC goes public. As part of releasing RC, all Beta2 extensions will be unpublished in the gallery and I’ll need to release an RC compatible version. At that time I will provide a link to the Beta2 binaries for those not upgrading to the RC.

The usual caveats and expectations

This extension is being released by me, not by Microsoft. As such the support level for this extension is equivalent to the amount of free time I have to put into it.

Having fun with events in F#

Jared Parsons — Wed, 03 Feb 2010 13:00:00 GMT

Recently I ran into a situation where I needed to handle some events in F# in a special way. In this particular case I wanted to be able to disable and re-enable my handler based on changes in the program. Essentially the C# equivalent of continually adding and removing the handlers.

I started by using the F# Observable pattern. Disposing of the handler when I was through with it and recreating it on demand. This works great but after several uses I decided to write a full abstraction for it. For lack of a better name I call it ToggleHandler.

[<AbstractClass>]
type internal ToggleHandler() =
    abstract IsHandling : bool
    abstract Add : unit -> unit
    abstract Remove : unit -> unit
   
    static member Create<'T> (source:System.IObservable<'T>) (func: 'T -> unit) = ToggleHandler<'T>(source,func)
    static member Empty = 
        { new ToggleHandler() with 
            member x.Add() = ()
            member x.Remove() = () 
            member x.IsHandling = false }

and internal ToggleHandler<'T> 
    ( 
        _source : System.IObservable<'T>,
        _func : 'T -> unit) =  
    inherit ToggleHandler()
    let mutable _handler : System.IDisposable option = None
    override x.IsHandling = Option.isSome _handler
    override x.Add() = 
        match _handler with
        | Some(_) -> failwith "Already subcribed"
        | None -> _handler <- _source |> Observable.subscribe _func |> Option.Some
    override x.Remove() =
        match _handler with
        | Some(actual) -> 
            actual.Dispose()
            _handler <- None
        | None -> ()

The design goal was to support my standard pattern for consuming events. Typically I store all event handlers as let bindings within a type but the actual delegate handling the event is bound to a member. Member declarations are not available in let bindings so creating an event handler becomes a 2 step process: defining in the let binding and then actually creating inside of a do binding. ToggleHandler facilitates this by providing a very easy let binding story.

let mutable _clickHandler = ToggleHandler.Empty

The base class ToggleHandler is type independent so this will work for any event type. Creating the real binding inside of the initial do binding is likewise as easy (and lacking explicit types).

do
    _clickHandler <- ToggleHandler.Create _button.Click this.OnButtonClick
    _clickHandler.Add()

Now I can toggle my event handler at any point in the application by calling Add/Remove.

Full Sample:

type Form1() as this =
    inherit Form()

    let _button = new Button()
    let mutable _clickHandler = ToggleHandler.Empty

    do
        _clickHandler <- ToggleHandler.Create _button.Click this.OnButtonClick
        _clickHandler.Add()

    member private x.OnButtonClick (e:System.EventArgs) = 
        // Handle Click 
        ()

    member private x.ToggleHandler() =  
        if _clickHandler.IsHandling then _clickHandler.Remove()
        else _clickHandler.Add()

Easier script deployment in PowerShell 2.0

Jared Parsons — Tue, 02 Feb 2010 13:00:00 GMT

If you can’t tell from reading entries in my blog I’m a bit of a script junkie. I loathe typing out the same command sequence more than once. As such I go to great lengths to script as much as possible in life. I also enjoy sharing my scripts with other members of my team.

Unfortunately deployment of PowerShell scripts has long been a point of pain for me. In the early 1.0 days PowerShell wasn’t installed by default on any OS so the only people with PowerShell were other scripter's. Deploying to the masses simply wasn’t possible without installing PowerShell which is a significant adoption barrier. Even if PowerShell was installed, the default setup forbids the execution of any script file. Allowing scripts to run in 1.0 requires both Administrative privileges and a rather awkward command sequence: set-executionpolicy RemoteSigned.

These two combined to make it nearly impossible to distribute my scripts to the masses. Other devs, after explanations, understood the problems I encountered. But the masses simply did not. The wanted a quick, one or two click solution to their problem. Running a powershell script was simply too involved.

The best solution I found was to distribute both a .cmd and a .ps1 file. The .cmd file’s job was to set the special registry key to allow script execution and then run the script. This still hurt adoption because it required administrative privileges and an existing copy of powershell on the machine.

Thankfully the story is quite a bit better in 2.0 The primary improvement is simply adoption. PowerShell is installed by default in Windows7 and certain flavors of W2K8. Windows7 is very popular amongst my coworkers and hence I can depend on at least a PowerShell deployment when I write my scripts.

The next improvement surrounds the execution of scripts. The default is still to have script execution disabled but you no longer have to be an administrator to do so. Additionally you can specify on the PowerShell command line what the execution policy can be. This mean you can have a script execute without the user having to manually enable script execution.

The command line is still a bit awkward. But it’s easily solved by deploying both a .cmd and .ps1 file. The .cmd file simply calls the .ps1 file with the awkward command line. Users are comfortable clicking on a .cmd file and it removes all knowledge of powershell from the equation.

For example:

Fix-Setup.cmd

set SOURCE=%~dp0
set TARGET=%SOURCE%\Fix-Setup.ps1
powershell -ExecutionPolicy RemoteSigned %TARGET%

These small changes greatly increased the adoption rate of scripts within my group.

The many cases of ByRef

Jared Parsons — Thu, 21 Jan 2010 13:00:00 GMT

One of the overlooked or simply misunderstood features of the VB language is calling a function which has a ByRef parameter. Most languages support only a single method of passing parameters by reference [1], that is the scenarios directly supported by the CLR. The CLR has a lot of restrictions on the type of values it supports for ByRef parameters and these restrictions get in the way of VB’s goal to be a flexible language that strives to get out of the way of the user. Hence the compiler goes to great lengths to be flexible and support multiple avenues of ByRef passing, much beyond what the CLR natively allows.

This article will explore these different mechanisms. In order to reduce the code samples, I will be using the following 2 methods to explain the different mechanisms of ByRef Passing

    Sub FunctionWithInt(ByRef p1 As Integer)
        p1 = 42
    End Sub
    Sub FunctionWithObject(ByRef p1 As Object, ByVal p2 As Object)
        p1 = p2
    End Sub

CLR ByRef

The first is to simply use the CLR concept of passing by reference as defined by section 12.4.1.5.2 and 12.1.6.1 of the CLI specification. Any variable which meets any of the following criteria, does not require a type conversion, and is passed to a ByRef parameter will be passed directly in the CLR.

Argument of the current method
Local variable
Member Field of an object
Static Field
Array Element

No special code is needed or generated for this scenario.

Copy Back ByRef

While the CLR method of passing ByRef is very flexible, it disallows a number of useful scenarios. The most prominent of which is properties. Properties do not meet the CLR requirements for ByRef because under the hood they are simply a pair of function calls. The result of a function call cannot be directly passed by reference.

Without any language intervention this can be very confusing to users. Properties are very often simple get/set wrappers around fields and have almost the exact same usage scenarios. To the point that most users don’t see a functional difference between the two. Auto-implemented properties blur this line even further. Not being able to pass them ByRef creates an unacceptable inconsistency in their usage.

VB removes this inconsistency and allows properties to be passed by reference. This is implemented under the hood by means of a temporary variable. Temporaries are just local variables and hence can be passed by reference. The property value is assigned to a temporary which is then passed by reference and then after wards is copied back into the original property.

For example, take the following code sample

    Class C1
        Public Property P1 As Integer
        Public P2 As Integer
    End Class
    Sub CopyBackByRef()
        Dim v1 = New C1
        FunctionWithInt(v1.P1)
    End Sub

This will result in essentially the following code being generated

    Sub CopyBackByRef_Explained()
        Dim v1 = New C1
        Dim vbTemp = v1.P1
        FunctionWithInt(vbTemp)
        v1.P1 = vbTemp
    End Sub

This type of ByRef passing is used in the following 2 scenarios

The value is a Property containing both a getter and setter.
Passing the value to the function requires a conversion.

The first can be done with even the strictest Option settings. However #2 can only be used with Option Strict Off because it requires an implicit narrowing conversion.

Don’t Copy Back ByRef

So far we’ve only looked at scenarios where the user wants to actually see the value returned from the ByRef parameter. There are many scenarios where the language can infer the user does not care about the return value of the function. For example, what if I just want to pass a constant value?

    Sub DontCopyBackByRef()
        FunctionWithInt(42)
    End Sub

This code is legal in VB and represents another method of passing by ref. This is very similar to the copy back method of passing by reference. The only difference is that it never copies the value back. It essentially generates the following code

    Sub DontCopyBackByRef_Explained()
        Dim vbTemp = 42
        FunctionWithInt(vbTemp)
    End Sub

This type of ByRef is used in any scenario where the value being passed cannot be assigned to. For example

The result of function calls
Read Only Properties
Constant Values

Maybe Copy Back ByRef

Up until now we’ve examined cases where the compiler can examine both the value being passed and the parameter it is being passed to and make a determination about what direction the data needs to move in. What about late binding?

    Sub MaybeCopyBackByRef()
        Dim v1 As Object = Me
        Dim v2 = 13
        v1.FunctionWithInt(v2)
    End Sub

Here v1 is typed to object and hence FunctionWithInt is being accessed via late binding. In this case the compiler doesn’t know the actual method being invoked runtime. Hence it cannot know up front if the parameters are ByRef or ByVal and cannot make an up front decision on the variable passing mechanism.

In order to make late binding invokes flow as smoothly as normal method invokes, the compiler will generate code to conditionally update the original value based on the runtime information about the parameter. The late binder communicates this information via an array of Boolean values, one for each parameter passed to the function. The compiler will initialize this array with true for any values it knows are updatable and false for values that are not. The late binder will then examine every parameter to the function and set the corresponding index in the array to false if the method parameter is ByVal. If it is ByRef the returned value from the function will be copied back into the original parameter array.

The resulting code looks a bit like this. You can ignore all of the Nothing values as they are not important for this discussion.

    Sub MaybeCopyBackByRef_Explained()
        Dim v1 As Object = Me
        Dim v2 = 13
        Dim parameters = New Object() {v2}
        Dim isByRef = New Boolean() {True}
        NewLateBinding.LateCall(v1, Nothing, "FunctionWithInt", parameters, Nothing, Nothing, isByRef, True)
        If (isByRef(0)) Then
            v2 = CInt(parameters(0))
        End If
    End Sub

[1] Starting with version 4.0, C# now supports two versions of reference passing. In addition to the one available since 1.0 the ref modifier is now optional when making an interop call to a COM object: http://mutelight.org/articles/new-features-in-c-sharp-4.html

VsVim Update Released (Version 0.6.0)

Jared Parsons — Mon, 04 Jan 2010 06:44:41 GMT

I just released an update to VsVim for Visual Studio 2010 Beta2. This should be available shortly from the extension manager in Visual Studio or it can be downloaded directly at the following link

Link: http://visualstudiogallery.msdn.microsoft.com/en-us/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329

New Features

Normal Mode Commands: Y,Delete, Arrow Keys, gP, gp, z<CR>,zt, z., zz, z-, zb
Command Mode Commands: d[elete], <,>, j[oin], y[ank], p[ut],
Insert Mode: Fixed Intellisense window dismissal issue (see below)

Notable Bug Fixes

In previous versions intellisense didn’t play well with insert mode. In particular attempting to dismiss intellisense with the Escape key would exit insert mode but leave intellisense showing. This release fixes the behavior to instead dismiss intellisense and leave you in Insert Mode
Normal Mode O now enters insert mode upon completion
Count was not being applied to certain edit commands (x) in Normal Mode
Normal Mode “dw” was deleting the line break if the word was at the end of the line

Source Code

Source for this release is available at the following location. It is released under the MS-PL.

http://cid-dc25b20f65f628f8.skydrive.live.com/self.aspx/Public/VsVim/VsVim-0.6.0.zip

Future Plans

The next planned major update is version 0.7.0. The two main features of this release will be the addition of Visual Mode and the :substitute command. Additionally I plan to flesh out a lot of the areas in normal, command and visual mode.

The usual caveats and expectations

This extension is being released by me, not by Microsoft. As such the support level for this extension is equivalent to the amount of free time I have to put into it.

Making F# type inference friendly for C#

Jared Parsons — Tue, 15 Dec 2009 19:53:08 GMT

One of my current hobby projects, VsVim, requires me to make a lot of calls between F# and C# projects. The core Vim engine is a pure F# solution based on Visual Studio’s new editor. It additionally has a small hosting layer and a large test bed both written in C#.

When working with the exposed core Vim engine API, I’ve found a number of generated F# constructs which are not easily accessible from C#. The problem stems from the manner in which native F# types are exposed. Many of them are generic and lack type inference friendly helper methods that force awkward usage patterns in C#. Most painful is the FSharpOption<T> type because it’s a type I frequently expose in APIs.

FSharpOption<T> is the exposed type for the native F# option construct representing a value which may or may not be present. It’s similar to C#’s nullable type except that it applies to all types of values. The primary operations you want to do with an FSharpOption<T> are

Create an option with a value
Create an option without a value
Determine if it has a value
Determine if it does not have a value
Access the value

In F# using an option is an inherent part of the language and the hence the resulting code is very elegant.

let OptionExample = 
    let optionWithValue = Some(42)
    let optionWithoutValue = None
    let isSome = Option.isSome optionWithValue
    let isNone = Option.isNone optionWithoutValue
    Option.get optionWithValue

Unfortunately the equivalent C# code is not nearly so nice.

static int OptionExample() {
    var optionWithValue = new FSharpOption<int>(42);
    var optionWithoutValue = FSharpOption<int>.None;
    var isSome = FSharpOption<int>.get_IsSome(optionWithValue);
    var isNone = FSharpOption<int>.get_IsNone(optionWithValue);
    return optionWithValue.Value;
}

Too many explicit types!!! Using any explicit type with F# related code just feels wrong.

In C#, and most other .Net languages, 4 out of the 5 operations you want to do on FSharpOption require an explicit type parameter. This resulting code is a bit tedious with a simple type like int but once you get to more complex generics it can get extremely verbose. In the case of anonymous types, it’s simply not possible to use the FSharpOption<T> without a few wrappers.

Luckily most of these can be solved by using the familiar pattern of using a non-generic class with static generic methods. These allow C# users to take advantage of the languages type inference capabilities to reduce the verbosity of the code.

public static class FSharpOption {
    public static FSharpOption<T> Create<T>(T value) {
        return new FSharpOption<T>(value);
    }
    public static bool IsSome<T>(this FSharpOption<T> opt) {
        return FSharpOption<T>.get_IsSome(opt);
    }
    public static bool IsNone<T>(this FSharpOption<T> opt) {
        return FSharpOption<T>.get_IsNone(opt);
    }
}

Now we can rewrite the original sample a bit cleaner

static int OptionExample() {
    var optionWithValue = FSharpOption.Create(42);
    var optionWithoutValue = FSharpOption<int>.None;
    var isSome = optionWithValue.IsSome();
    var isNone = optionWithoutValue.IsNone();
    return optionWithValue.Value;
}

Notice we still haven’t fixed the None case. Fixing this is a beyond the scope of what I want to write here but it is possible in certain scenarios. You can take a look at how in one of my previous blog articles: Function C# Providing an Option.

This pattern is not just limited to the FSharpOption class but can be applied to many of the generic constructs F# exports to wrap their native types. In particular FSharpFunc<T,Result> and the various FSharpChoice<> types can be made a bit friendlier with a few wrappers.

VsVim Update Released (Version 0.5.4)

Jared Parsons — Mon, 14 Dec 2009 18:26:22 GMT

I just released an update to VsVim for Visual Studio 2010 Beta2. This should be available shortly from the extension manager in Visual Studio or it can be downloaded directly at the following link

Link: http://visualstudiogallery.msdn.microsoft.com/en-us/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329

Changes

This is mainly a minor bug fix release. Primarily there was a bug in the command mode jump implementation that made it flaky in big buffers. Additionally I implemented the normal mode commands r and O.

Source Code

Source for this release is available at the following location. It is released under the MS-PL.

http://cid-dc25b20f65f628f8.skydrive.live.com/self.aspx/Public/VsVim/VsVim-0.5.4.zip

The usual caveats and expectations

This extension is being released by me, not by Microsoft. As such the support level for this extension is equivalent to the amount of free time I have to put into it.

The File System is unpredictable

Jared Parsons — Thu, 10 Dec 2009 13:00:00 GMT

One of the more frequent questions I answer on StackOverflow is a variation of the following.

I’m doing XXX with a file, how can I know if the file exists?

The variations include verify no one else has the file open, if the file is in use, the file is not writable, etc …. The answer to all of these questions is unfortunately the same. Simply put you can’t. The reason why is the fundamental nature of the file system prevents such predictive operations.

The file system is a resource with multiple levels of control that is shared between all users and processes in the system. The levels of control include but are not limited to file system and sharing permissions. At any point in time any entity on the computer may change a file system object or it’s controls in any number of ways. For example

The file could be deleted
A file could be created at place one previously did not exist
Permissions could change on the file in such a way that the current process does not have access
Another process could open the file in such a way that is not conducive to sharing
The user remove the USB key containing the file
The network connection to the mapped drive could get disconnected

Or in short

The file system is best viewed as a multi-threaded object over which you have no reliable synchronization capabilities

Many developers, and APIs for that matter, though treat the file system as though it’s a static resource and assume what’s true at one point in time will be true later. Essentially using the result of one operation to predict the success or failure of another. This ignores the possibility of the above actions interweaving in between calls. It leads to code which reads well but executes badly in scenarios where more than one entity is changing the file system.

These problems are best demonstrated by a quick sample. Lets keep it simple and take a stab at a question I’ve seen a few times. The challenge is to write a function which returns all of the text from a file if it exists and an empty string if it does not. To simplify this problem lets assume permissions are not an issue, paths are properly formatted, paths point to local drives and people aren’t randomly ripping out USB keys. Using the System.IO.File APIs we may construct the following solution.

static string ReadTextOrEmpty(string path) {
    if (File.Exists(path)) {
        return File.ReadAllText(path); // Bug!!!
    } else {
        return String.Empty;
    }
}

This code reads great and at a glance looks correct but is actually fundamentally flawed. The reason why is the code changes depends on the call to File.Exist to be true for a large portion of the function. It’s being used to predict the success of the call to ReadAllText. However there is nothing stopping the file from being deleted in between these two calls. In that case the call to File.ReadAllText would throw a FileNotFoundException which is exactly what the API is trying to prevent!

This code is flawed because it’s attempting to use one piece of data to make a prediction about the future state of the file system. This is simply not possible with the way the file system is designed. It’s a shared resource with no reliable synchronization mechanism. File.Exists is much better named as File.ExistedInTheRecentPast (the name gets much worse if you consider the impact of permissions).

Knowing this, how could we write ReadTextOrEmpty in a reliable fashion? Even though you can not make predictions on the file system the failures of operations is a finite set. So instead of attempting to predict successful conditions for the method, why not just execute the operation and deal with the consequences of failure?

static string ReadTextOrEmpty(string path) {
    try {
        return File.ReadAllText(path);
    } catch (DirectoryNotFoundException) {
        return String.Empty;
    } catch (FileNotFoundException) {
        return String.Empty;
    }
}

This implementation provides the original requested behavior. In the case the file exists, for the duration of the operation, it returns the text of the file and if not returns an empty string.

In general I find the above pattern is the best way to approach the file system. Do the operations you want and deal with the consequences of failure in the form of exceptions. To do anything else involves an unreliable prediction in which you still must handle the resulting exceptions.

If this is the case then why have File.Exist at all if the results can’t be trusted? It depends on the level of reliability you want to achieve. In production programs I flag any File.Exist I find as a bug because reliability is a critical component. However you’ll see my personal powershell configuration scripts littered with calls to File.Exsit. Simply put because I’m a bit lazy in those scripts because critical reliability is not important when I’m updating my personal .vimrc file.

VsVim Update Released (Version 0.5.3)

Jared Parsons — Mon, 07 Dec 2009 07:44:27 GMT

I just released an update to VsVim for Visual Studio 2010 Beta2. This should be available shortly from the extension manager in Visual Studio or it can be downloaded directly at the following link

Link: http://visualstudiogallery.msdn.microsoft.com/en-us/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329

Changes

Normal Mode Block Cursor

This removes the red cursor in favor of a Vim block style cursor for normal mode. This is implemented as a simple adornment in the editor and in the end required much less work than I anticipated (also allowed the removal of a lot of hacky code). Thanks a lot to Noah for helping me out here.
Basic range support in command mode.

Currently limited to line number ranges, % and . (current line)
Command Mode Infrastructure Changes

This infrastructure change allowed my to correctly implement the few command mode commands I previously supported and laid the ground work for quick support of new commands
New Normal Mode Commands: gJ

This release is small on features and ordinarily wouldn’t qualify for an update. But the cursor change was so pleasant for myself that I decided to release a quick update just for that.

What’s Next?

Add command mode support for all normal mode commands that have a command mode equivalent
Resolve normal mode edits around a selection. Currently selection is ignored and it produces unexpected behavior.
More normal mode commands: In particular, Delete, gt, gT, global mark navigation
Lifetime issues around buffers and Vim’s internal object model
Navigation in a buffer which contains multiple font sizes

The usual caveats and expectations

This extension is being released by me, not by Microsoft. As such the support level for this extension is equivalent to the amount of free time I have to put into it.

Dev Connection Talk Slides and Code

Jared Parsons — Wed, 02 Dec 2009 21:21:45 GMT

Thanks to everyone who attended my sessions at Dev Connections. I’ve posted the material for both of my talks on my SkyDrive account. This includes the slides and projects.

http://cid-dc25b20f65f628f8.skydrive.live.com/browse.aspx/Public/DevConnections2009

Please let me know if you have any questions or problems with the materials or any additional feedback about the sessions.

VsVim Update Released (Version 0.5.2)

Jared Parsons — Tue, 01 Dec 2009 06:29:00 GMT

I just released an update to VsVim for Visual Studio 2010 Beta2. This should be available shortly from the extension manager in Visual Studio or it can be downloaded directly at the following link

Link: http://visualstudiogallery.msdn.microsoft.com/en-us/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329

Changes

Removal of conflicting key bindings

On startup VsVim will now look for any key bindings which conflict with implemented Vim commands. It will then provide a message box allowing you to remove the key bindings for this session of Visual Studio. Right now this is an all or nothing removal, future versions may make this a more granular process
Normal Mode no longer intercepts all keystrokes (such as Cut and Paste)

This bug in the previous release was a result of the command routing changes that occured between Beta1 and Beta2. I believe I know have them all worked out and VsVim should only be intercepting commands it intends to process
New Normal Mode Commands

Mark setting: m[a-zA-Z]
Mark Jump: `[a-z], ‘[a-z]

This is currently limited to local marks

Page Up, Page Down: CTRL-U,CTRL-D
Join: j

What’s Next?

The biggest area I want to focus on for the next release is Command mode and rounding, more normal mode behavior (especially mark support) and basic range support.

At a minimum I want to expand all of the normal mode commands I have implemented to have their corresponding command mode version implemented. Join, mark, etc … This shouldn’t be a big work item. Command mode is just something I’ve been neglecting up to this point in favor of infrastructure.

The other big item I want to work on is ranges. Many command mode commands operate on ranges and it’s something I’ve yet to implement at a core level. The basic mark support I added in this release starts the process but I have a ways to go.

Also, any suggestions users have are very welcome.

The usual caveats and expectations

This extension is being released by me, not by Microsoft. As such the support level for this extension is equivalent to the amount of free time I have to put into it.

Vim Emulator Editor For Beta2 Released

Jared Parsons — Tue, 17 Nov 2009 04:17:00 GMT

This is essentially the same release as the original but updated for some changes that occurred in the APIs between Beta1 and Beta2.

Link: http://visualstudiogallery.msdn.microsoft.com/en-us/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329

The biggest change came in the way in which Visual Studio routes commands. Vim, as you can imagine, needs to participate in command routing and these changes took awhile to sort out. I believe I’ve sorted out the issues but please send me a mail / comment if you find any bugs.

Caveats and Expectations

This extension is being released by me, not by Microsoft. As such the support level for this extension is equivalent to the amount of free time I have to put into it.

What’s next?

Right now the Visual Studio command system is the biggest inhibitor to implementing new features. Each command in Visual Studio is bound to a specific key stroke and often times this conflicts with key strokes my Vim emulator needs to process.

It’s easy to implement these commands in the core Vim engine. However the actual key strokes get intercepted by Visual Studio and are not processed.

My next feature will be essentially removing these commands so that they can make it to the Vim layer. It will require a bit of a UI since I’ll be essentially changing key bindings but hopefully I can get something basic out the door so that I can start focusing on real features again.

Type safety issue when assigning CComPtr instances

Jared Parsons — Wed, 04 Nov 2009 13:00:00 GMT

Recently while making a bug fix to our selection tracking code I discovered an unexpected behavior with CComPtr<T> instances. The crux of the fix included creating a new tracking mechanism exposed via COM in the type ISelectionTracking. The old interface, lets call it IOldTracking, was a completely unrelated interface in terms of inheritance hierarchies.

As part of the fix I changed the type of a field (m_spTracking) from CComPtr<IOldTracking> to CComPtr<ISelectionTracking>. I searched for assignments of m_spTracking and converted them to call the new API I added as part of the fix. I didn’t search terribly hard because I was depending on the compiler to catch any places I missed. ISelectionTracking and IOldTracking are incompatible types so any places I missed will show up as compilation errors.

Or so I thought …

I made my fix, ran our core check-in suites without error, checked in and moved onto the next bug. A couple hours later one of our other devs emailed me and informed me my check-in was breaking our larger, slower, suite bed run because m_spTracking was NULL. After some quick debugging I found myself looking at the following chunk of code which was apparently NULL’ing out m_spTracking in the suite.

  CComPtr<IOldTracking> spOldTracking;
  if ( SUCCEEDED(CreateOldSelectionTracking(&spOldTracking)) ) {
    m_spTracking = spOldTracking;
  }

Me and the other dev were quite shocked that this compiled at all. How is it possible to assign between CComPtr<ISelectionTracking> and CComPtr<IOldTracking>??? My first thought was I must have accidentally used a CComQIPtr somewhere (quickly verified that was not the case). After a bit of searching we found the cause was one of the operater= instances available on CComPtr<T>. Here is the definition

template <typename Q>
T* operator=(_In_ const CComPtr<Q>& lp) throw()
{
    if( !IsEqualObject(lp) )
    {
        return static_cast<T*>(AtlComQIPtrAssign((IUnknown**)&p, lp, __uuidof(T)));
    }
    return *this;
}

This templated operator allows for assignments between CComPtr instance no matter what the type is for the left and right side. The effect is that instead of doing compile type C++ type conversion rules, it will instead rely on runtime COM polymorphic assignment rules via IUnknown::QueryInterface. This moves assignment errors from compile time to runtime for unrelated interfaces.

This is further complicated because it only applies to assignment between CComPtr’s (and derived instances). If the right hand side of the assignment is a non-smart pointer, compile time C++ conversions will apply. To demonstrate …

        CComPtr<ISelectionTracking> spTracking;
        CComPtr<IOldTracking> spOld;
        ...
        spTracking = spOld;  // Fails at runtime
        spTracking = (IOldTracking*)spOld;  // Compilation Error

What surprised me though was talking to other developers about this issue. Most agreed with me that this is a bug in CComPtr<T>, or at least very unexpected behavior. A surprising number though did not expect this behavior but still considered it acceptable. The difference comes down whether you view CComPtr<T> as a simple smart pointer responsible for AddRef/Release semantics or as that plus an enabler of QueryInterface style conversions. I personally view CComPtr<T> as a simple smart pointer with know real understanding of QueryInterface style conversions and CComQIPtr<T> as a smart pointer which respects QueryInterface style conversions. As such this behavior is completely unexpected.

The fix in this case was pretty straight forward (use the new API) but I was still worried about how to prevent this type of problem in the future. In particular how to get the failure back to a compile time error. In the end we settled on using a stripped down version of CComPtr we already had in our code base going forward called CComPtrEx. I’ve previously blogged about about the need for this type here. It’s different from CComPtr in the following ways

Does not have the templated version of operator= and instead relies on compile time C++ conversions checks for assignment
Allows for interfaces which have multiple paths to IUnknown (can cause a compile time error in CComPtr).

Also for purposes of rigor, we temporarily commented out the CComPtr<T> operator, recompiled our code base and verified no new errors popped up.

Speaking at Dev Connections in Las Vegas Next Week

Jared Parsons — Tue, 03 Nov 2009 16:51:04 GMT

Next week I will be speaking at Dev Connections in Las Vegas. I will be running the following sessions

VMS02: Future Directions for Visual Basic
VMS04: Microsoft Visual Basic IDE Tips and Tricks

Both of these talks will spend a bit of time talking about all of the progress and exciting new features we’ve added in Dev10. Given I primarily work on the IDE these days, expect a bit of IDE content to work it’s way into the Future Directions talk as well.

Using F# Discriminated Unions in C# (Beta2)

Jared Parsons — Tue, 27 Oct 2009 12:00:00 GMT

While updating my VsVim editor extensions for Beta2 [1] I got hit by a change in the way F# exposed discriminated unions in metadata. My extension consists of a core F# component with a corresponding set of unit tests written in C#. It’s mostly API level testing and as such I use a lot of F# generated types in my C# test assembly.

In Beta1 all information which could be extracted from a discriminated type union was immediately available on the value. The underlying type presentation was less than desirable but these details were hidden by type inference and the very accessible API. The type wasn’t perfect because given a particular instance only the subset of the properties relevant to the union value type were valid. All others threw exceptions. But the code use of these methods and properties flowed very well.

For instance take the following F# definition

type ActionKind =
    | Mouse = 1
    | Keyboard = 2

type ActionResult =
    | Complete of (ActionKind * int)
    | Error of string
    | NeedMore of (char -> ActionResult)

The use case in C# was quite simple

[TestMethod]
public void TestActionBeta1(){
    var res = GetResult();
    Assert.IsTrue(res.IsComplete());
    Assert.AreEqual(ActionKind.Mouse, res.Complete1.Item1);
    Assert.AreEqual(42, res.Complete1.Item2);
}

Notice how no type information is necessary and the code flows quite naturally. C# type inference works great here and allows me to do what I need to do without fussing around with little stuff. The type in this case is a detail I don’t need to know about. It simply adds no value.

Discriminated Unions in Beta2 changed substantially in this area. Instead of generating the set of all values on the exposed type, there is now an inner type generated for every discriminated union value and the properties relevant to that union value are stored on the inner type. The outer type now contains only properties to determine which type of value it is (certainly an upgrade from methods!) [2]

For instance in the case of ActionResult there are 3 generated inner classes: Complete, Error and NeedMore. Each one contains a single property Item which contains the associated value(s). This means to get to the value portion a cast to the inner type must be inserted!

Lets take a a look at how the above test code has to change to deal with the Beta2 generation of ActionResult.

[TestMethod]
public void TestActionBeta2() {
    var res = GetResult();
    Assert.IsTrue(res.IsComplete);
    Assert.AreEqual(ActionKind.Mouse, ((ActionResult.Complete)res).Item.Item1);
    Assert.AreEqual(42, ((ActionResult.Complete)res).Item.Item2);
}

Notice the explicit casts which must be added to access the values. This makes it impossible to rely soley on C# type inference. I must now understand the underlying type structure of discriminated unions in order to use them. This extra cast adds no real value to my code.

My C# test assembly has literally hundreds of test cases which use this pattern on F# types. I didn’t know the return type of every method and found myself hitting “Goto Def” on a lot of “var” instances to discover the static type, going back to the original file and inserting the cast. It was a tedious and slow process.

Eventually I settled on a different solution. For every type I exposed in F# I added a set of extension methods in the form of AsXXX where XXX represented the name of the generated inner types. For example

public static ActionResult.Complete AsComplete(this ActionResult res) {
    return (ActionResult.Complete)res;
}

The advantage of this approach is 2 fold

Removes the need to explicitly name types in code and hence gets back the advantages of type inference
I can now use . on any of the values and let Intellisense help me find the appropriate method to use

This extension method allows me to get closer to the Beta1 style code

[TestMethod]
public void TestActionBeta2() {
    var res = GetResult();
    Assert.IsTrue(res.IsComplete);
    Assert.AreEqual(ActionKind.Mouse, res.AsComplete().Item.Item1);
    Assert.AreEqual(42, res.AsComplete().Item.Item2);
}

With these methods and a quick series of Find / Replace calls, I was back in business.

[1] It’s coming I promise!

[2] It also contains a handy set of factory methods for generating values but it’s not relevant to this discussion.