If you are reading this, I'm sure there is a decent chance that you've either already used Visual Studio 2010 Beta 2 or at least read about some of the new features. For those of you interested in writing extensions for VS, I wanted to share a couple of updates around editor extensibility, particularly for people who wrote extensions in Beta 1 and want to upgrade to Beta 2.

1 - Bye bye, IEnvironment

If you wrote any code that had classifiers or taggers or various other extensions that take an IEnvironment parameter, you'll be deleting those parameters in Beta 2. IEnvironment had a few historical reasons for being there (back when the editor was a part of a different project and not in Visual Studio yet), but it wasn't really serving a purpose anymore. In Beta 1, all it really did was confuse people and make you type an extra few characters and/or add an fxcop suppression since you weren't using it. Anyways, small change, but a simplifying one. (If you are curious about the one place that did use it for taggers, see #5 below).

2 - Mouse and key processors get more metadata

In Beta 2, mouse and key processor providers take the following metadata; note that some of these were required for mouse or key processors, but now the list applies to both:

1. Export, like all components
2. ContentType
3. Name and Order, so that you can position yourself before or after other known handlers. This is kinda important in cases where you want to preempt whatever normally happens in a certain event.
4. TextViewRole

Also, somewhat tangentially, the ContentType attribute now applies to any buffer in a view's graph - that's just a fancy way of saying that if you have something like an asp.net page that has embedded C#, your key or mouse processors will get loaded if they have [ContentType("CSharp")].

These fixes were necessary to write the triple click extension (github and vsgallery). Without the ordering, I couldn't put the mouse handler early enough to work correctly. Also, there were a few bugs in some of the default mouse handlers that have since been fixed.

Note that, unfortunately, the general guidance is to not use keyboard providers. The VS command system and how components like the editor plug in don't play very well with normal WPF keyboard input, so the short story is that you'll only get keyboard events for key combinations that aren't already bound to a command. Which leads me to the next new thing...

3 - Listening for the creation of IVsTextView adapters (IVsTextViewCreationListener)

We added this as an extension to the VS integration piece of the editor to allow people to listen for the creation of IVsTextView (which is what the underlying IWpfTextView/ITextView are wrapped in for the sake of people who still use the existing VS API). It's effectively identical to IWpfTextViewCreationListener, except that you get an IVsTextView instead of an IWpfTextView when called.

A quick note - if you want to use this extension, you'll need to add a reference to Microsoft.VisualStudio.Editor.dll. It's in the SDK, so that's not a problem, but it isn't a part of any of the standard editor project templates.

The biggest reason we introduced this was for extensions that want to add command handlers to the editor. You could do this in the past, somewhat painfully, by doing the following:

1. Write an IWpfTextViewCreationListener
2. The ugly step - listen for the GotAggregateFocus event on the text view and do the remaining steps in that handler. If you skipped this step, the eventually attempt at getting the IVsTextView would fail, since the two objects weren't fully hooked up yet.
3. With an [Import]ed IVsEditorAdaptersFactoryService, call GetViewAdapter to get the IVsTextView.
4. Call IVsTextView.AddCommandFilter
5. ...and don't forget to unsubscribe from GotAggregateFocus or use a flag so that you only listen to the first time the event is raised.

With the new event, you just need to:

1. Write an IVsTextViewCreationListener
2. Call IVsTextView.AddCommandFilter

If you do want to get the associated IWpfTextView for the IVsTextView adapter, you can do the opposite of #3 above (using GetWpfTextView); the upside, relative to the above list, is that you don't need to worry about initialization being finished. When you get the call in your creation listener, initialization has completed to the point that GetWpfTextView will always succeed.

Because command filters are still not entirely obvious to get correct, I'm working on a template for doing this for you. More in general, I'm working on a set of "New Item" templates for the editor, so that you can add new components (or at least a skeleton of a new component) with a couple clicks in an existing project. I have finished these yet, but you can see the work in progress on CommandFilter.cs.

4 - Custom text marker visual definitions

One of the other things missing in beta 1 was the ability to set the visualizations for text markers (the markers you can create with an ITagger<TextMarkerTag>). There were some built-in marker types, but they were kinda limited.

In Beta 2, that's now changed, and you can export a MarkerFormatDefinition that looks something like this:

And then create a TextMarkerTag with the name "mymarker", and you are all set. Note that you'll probably want some transparency on your fill brush, since the selection is drawn underneath markers.

5 - View tagger provider (IViewTaggerProvider)

In Beta 1, you exported your taggers with an ITaggerProvider. In Beta 2, there is an additional interface you can use called IViewTaggerProvider, which is passed both a buffer and a view in its CreateTagger method. There are a few cases where you may want to use this:

1. Your tagger needs information specific to a certain view – this may be the case if you are writing an extension that wants to, say, highlight all references in a view that match the reference under the caret. In this case, you need to know the caret position in the view to produce your results.
2. Your tagger wants to produce different results depending on the view – this is basically the same as #1, but it may help to also think of it in this way. With the highlight references example again, it's possible to have two different views over the same buffer (split window, for example, or the code definition window), where you want each view to show different results for the highlighted references, even though they are displaying the content of the same text buffer.
3. Your tagger wants to consume other tag information – this is the case I mentioned earlier about deserving its own blog article (which I will still get to, promise). Basically, if you have a tagger for type T that consumes, say, classification tags, you want to avoid accidentally including a classifier that is trying to consume tags of type T, or else you'll end up with an ugly recursive loop when trying to get tags. You can sidestep this by providing your tagger with an IViewTaggerProvider and consuming taggers (or classifiers) you get from an IBufferTagAggregatorFactoryService (or IClassifierAggregatorService.GetClassifier that takes a buffer). That way, you are guaranteed that any taggers/classifiers you may consume can't accidentally consume your extension as well.

The downside to #3 is that it is still hairy for other components to safely consume your tagger (that they are using via IViewTagAggregatorFactoryService) to produce tags; if two view-level taggers are each consuming other view-level taggers, you get the same problem as before. As such, any component you make that does this (such as Tomas Restrepo's Keyword Classifier), should be created with the understanding that it can't be safely consumed. For any extensions you write, you should probably stear clear of creating view-level taggers that consume a view-level ITagAggregator<ClassificationTag>, for example. In general, it's probably safest to follow this rule:

Never create a tag aggregator using an IViewTagAggregatorFactoryService inside a tagger provided with an IViewTaggerProvider.

...and more

These are just the biggest ones I can think of off the top of my head. I'm sure someone will point out things I've missed, which I'll likely cover in a new article.

Also, in case you missed them throughout the text, here are the various urls I've linked to:

• Triple click extension on github and on vsgallery
• Earlier article on updates to my extensions for Beta 2
• CommandFilter.cs skeleton, for providing a command filter in the editor.
• Tomas Restrepo's Keyword Classifier extension, on github
• Also, Tomas's blog, Winterdom, which includes posts about the extensions he wrote, as well as being a generally good read.

As always, comments welcome.

We've heard a few comments/complaints about how the gradient selection has been removed in Beta 2, both as an option and as the default selection color. Since it's come up so many times, I figured you could all use an explanation as to why we did it (effectively, scrolling performance and UX).

First off, for the people who do miss gradient selection, I wrote an extension that adds it back (when you have the rich-client experience enabled). You can find it here, on vsgallery. Also, the source is available on github.

A screenshot of the gradient selection plugin in action on Beta 2

Scrolling

One of the common complaints from Beta 1 and, to a much more significant degree, from the CTP, was that scrolling speed was a big problem, especially under remote desktop and if you were running a VM; essentially, any combination of software rendering and/or the limited bandwidth of a remote session was making scrolling painful.

Part of the reason why we had this issue was because of the way the gradient selection works - whenever you increase the height of the selection with a vertical gradient (as was the default), the editor needed to repaint the entire selection. Similarly, whenever you increase the width of a selection with a horizontal gradient, the selection would need repainted. In the worst case, then, where you have an arbitrarily complex brush, any change in selection size requires redrawing the entire selection. If you include the border (as a pen), then you have other interesting issues when the shape of the selection changes.

If you changed the VS settings to not automatically detect rich-client experience, so that the gradient was enabled over remote desktop or in software rendering mode, you'd get an idea for how painful this was. As the selection got larger, the editor would redraw more and more on each change. This was especially bad if the selection was partially off the screen, as drawing the gradient properly requires that the lines off the screen also went through the layout pass, so that we could know their height (usually, lines that aren't visible don't participate in layout).

In Beta 1, the gradient was off by default when not in rich-client mode, so you wouldn't see it by default over remote desktop. It was still painful if you turned it on by default or weren't automatically in simple graphics mode.

Smarter redraw

With a solid color selection, on the other hand, you can make certain assumptions about the selection. The most important of these is that if you add or remove areas to the selection, you only need to redraw those portions. As you increase the size of the selection, you only need to redraw the new areas that are being selected. Effectively, the cost of drawing the selection in this case is relative to the change in selection size, instead of relative to the absolute selection size.

And if part of the selection is off the screen, you can just ignore that completely — no need to even think about those lines until they are scrolled back into view.

So, if your selection brush (generally, a solid color brush) doesn't care about the size of the whole selection, and you don't have a border that also cares about the selection as a whole, then you can be much smarter about redraw.

You can still actually have a non-solid color brush when you make these assumptions, as long as the brush is drawn per line. If you have a vertical gradient in this case, the gradient will appear to start/stop on each line. So, for almost all purposes, this really only looks good with a solid color brush with no border.

UX

The other reason, which I know much less about, was UX. The basic explanation, as it filtered down to me, was the push to make things appear visibly simpler in the UI. Most people saw the gradient selection as pretty but without utility, which isn't a redeeming enough quality to warrant keeping it around.

Of course, the people who miss it have given good reasons for wanting it around. The best reason I've heard so far is that the gradient gives you visual clues about where you in a file. I can certainly see that, though the value of that seems to degrade when the selection is small (1-3 lines) and very large (larger than the screen). Then again, I could see an argument for while it would still be useful for a giant selection (you can tell if you are underneath, above, or right at the midline of the selection).

Extensibility

The real answer here is the answer I tend to give to everyone — extensibility. For many cases, the answer to "why doesn't the editor do foo" is isn't that such a thing is impossible, just that it isn't built-in.

So, while we did remove the gradient selection as an option in the product, we didn't remove the code that knows how to draw a selection with an arbitrary brush and pen (for the border). As such, getting the gradient selection back requires just a little bit of code. Here's the pertinent part:

There's a bit more to the extension than that; notably, if you supply a gradient and the editor is not in rich-client mode, it'll draw the gradient per-line instead of over the entire selection, so there's logic in there to clear out the gradient brush when rich-client mode is disabled.

The other thing that may not be immediately apparent from the code is that you could change this per language (per content type), or even (if you wanted) based upon some text in the file (like vim or emacs mode lines). I'm not sure how desirable that is, but it may be interesting for your python files to use a different selection color than the rest of your files. Who knows?

I've updated all my extensions for Beta 2. Here are descriptions of the changes, as well as links to the source for each on github and the extension on VS Gallery.

GoToDef

This was the only one that was somewhat substantial. There were essentially two changes:

First, the classifier that underlines the text has been rewritten (very slightly) as an ITagger<ClassificationTag>, so that the provider can be rewritten as an IViewTaggerProvider. This is going to get its own blog article, but the short story is that if you are writing a classifier/tagger that is consuming other classifiers/taggers, you should write it as a view-specific tagger to avoid accidentally using an aggregator that uses another classifier/tagger that uses your classifier/tagger that uses an aggregator that uses...ad infinitum.

Second, the mouse handler had some ugly logic for getting a service provider and getting the global shell command dispatcher. That has been simplified down (from ~20 lines to 1 line) to use the new [Import] System.IServiceProvider in Beta 2.

ItalicComments

This one was really minor. The only change of note is that IEnvironment is gone in Beta 2 from everywhere, so you can just delete all references to it and its namespace (it has ApplicationModel in it, but it should be obvious if you re-compile an extension you wrote against Beta 1).

TripleClick

This is the extension that I blogged about yesterday.

Oh, and I also wanted to add: I heart git/github very, very much. Internet props to all them git/github people.

Beta 2 was officially released today (to MSDN subscribers; the general public comes a little bit later in the week). Also of interest, Rico wrote a pretty long blog post about some of the new features, with nice props to the new editor. His whole history of visual studio series has been pretty interesting, so read it all if you have the time and inclination.

In honor of Beta 2's release, I wrote up a quick little extension tonight. A couple weeks (months? it's been busy around here) ago, we got a connect bug/suggestion that triple-clicking the mouse should select a whole line, instead of whatever it does now. Seemed like a good idea (I never really realized that triple-click wasn't there, but I tend to avoid the mouse as much as I can), so I thought I'd share that out with everyone here.

TripleClickMouseProcessor.cs on github (Ms-PL license and such).

It's pretty simple - all it does is detect triple clicks and selects the extent of the line under the mouse. It doesn't, for example, handle triple-click + drag, which should probably select full lines as you move your mouse around. Also, apologies to anyone already running beta 2, but I didn't put it on the extension manager yet. If you want to play around with this yourself, follow my handy dandy instructions in this post (towards the end), replacing "Beta 1" with "Beta 2". Apologies to Beta 1 users, but this won't work in Beta 1, because some of the attributes on the mouse processor weren't there in the earlier beta.

I'm planning on following this up, in decently short order, with a list of some of the updates for editor extenders. Certain API has changed (improved, I think) since the earlier release, and there are a few new extension points that may be of interest.

Thanks to Fiona and Tim for being all managerial and figuring out the licensing story for extensions that I've written, you can now find the code I've written for both the ItalicComments and GoToDef extensions.

I've placed them both under my new github account, so please fork them to your (collective) heart's content and release your own extensions. If you have any questions, feel free to ask them here, to my twitter account (noahsmark), or email me directly (noahric [at] ms).

If you don't have git installed on your Windows machine, github has a good walkthrough for working with Windows. I use msysgit, like it suggests.

Enjoy!

I had a professor in college, Sidney Marshall (also known as "the man, the myth, the legend: Sidney Marshall"), who was and still is a pretty big inspiration to me. He's a fairly big Lisp guy (part of what got me interested in Lisp in the first place), quotes various pieces of the ISO C++ spec conversationally, and is smarter than about every other person I've met. He one of those people who really does know a little bit about everything, too.  Whenever you hear him start a sentence with “You know what the real problem with blah is, right?” at 9 in the morning, don’t assume you’ll be on-time for your 10am class.

In one of our conversations, he made a rather interesting point that has stuck with me. I'm not sure what the original topic was, but he was explaining something along the lines of why it's so easy for him to write a lot of the cool stuff that he does. As he explained it, the biggest difference between him and a young kid like myself is that the time it takes for him to design something is near zero. When I try to solve a problem, I have to think about it or prototype around it for awhile, due to my inexperience with about everything. For him, however, he's been doing so much for so long that he can skip that step entirely and just start in on the solution.

I never really fully grasped the meaning of that until somewhat recently, when I started writing and helping others write extensions for the new editor. After working on the editor team for about 2 years, I know most of the editor pretty well (with the exception of IntelliSense, which I've never had the occasion to use, extend, or really take a look at). However, for almost everyone who didn't write the editor, it's an entirely new API, a different set of concepts from the previous Visual Studio editor, and even a few things that may be new or interesting in general for extending an editor.

I’ve found that I’m starting to understand what it is Sidney meant – now, when I sit down to write an editor extension, I don’t really spend any time in the design phase, which is an interesting way to write code :)

The GoToDef extension

To give you an idea of how I write editor extensions, I'll walk through the design of the GoToDef extension I recently wrote (partially modeled after an earlier version of it, written by Huizhong, a member of our excellent QA team).

A screenshot of the GoToDef extension, from the Visual Studio Gallery

Here's what I set out wanting the extension to do:

• Ctrl+clicking on an identifier executes GoToDefn in Visual Studio.
• Ctrl+mouse over on an identifier (hopefully only identifiers) turns the identifier into a html-esque link, so that you know what you can click on.

To start with, here are the major pieces of my component, the first 4 of which are exported to the editor via MEF:

1. A (view-specific) classifier for turning things into links. It's told which span of text should be underlined/blue.
2. A classification type for the underline, along with a format that sets the foreground color to blue and adds TextDecorations.Underline to the text.
3. A keyboard processor, for tracking when the ctrl key is held down and released. This isn't entirely sufficient, as you could press/release ctrl outside of the editor, but it's an important part of tracking that behavior.
4. A mouse handler that handles (a)mouse move, for updating the classifier on what should be underlined (if anything), and (b)mouse left-button up, for sending the command.
5. There's also a small, non-editor piece called CtrlKeyState, which is updated by the mouse and keyboard processor when they see the state has changed and events accordingly (so the mouse processor can do it's magic to determine if the word should be highlighted or not.

(Though it may sound like a lot, the whole thing is only a couple of hundred lines long). Then, I moved on to following services to use to make this job easier, from what the editor provides:

1. A classifier aggregator, to consume classifications that are provided by other extensions (like the C# and C++ language service). As it turns out, both those language services (and possibly others, though not VB, unfortunately) colorize/classify identifiers and things like User Types, which the mouse processor consumes to guess what things GoToDefn will succeed on. Ideally, we'd just be able to ask whichever language service if the identifier under the cursor can have GoToDefn executed on it, but it appears that QueryStatus for that command always succeeds, and whoever handles it on the language service side just pops up a little dialog box telling you it fails when you shouldn't have clicked it.
2. A text structure navigator, for determining the extent of the current word under the cursor.
3. An IVsTextEditorAdaptersFactoryService (whew, long name), which can be used to get, for example, an IVsTextBuffer from an ITextBuffer, which could be then used to get a service provider and find the shell service for executing global commands (since this gets into the ol' COM way of doing things, it's kinda ugly to write in managed code, and I mostly just used the code Huizhong wrote for that. Kudos to her for figuring it out :). Thankfully, post beta 1, you'll just be able to [Import] an IServiceProvider directly, and skip all those hoops you have to jump now.

It only took me a couple of hours to write the extension (as our PMs say, “it’s only code”; as I say, “I’m going to punch you”), with a little bit of that time spent getting over small hiccups along the way.

My point, though, is that once you understand the API of the editor, it doesn’t take long to create what it is you want.  Put alternatively, I don’t think there is much overhead imposed by the editor – the hardest part of writing an editor extension is the logic specific to the extension’s purpose and not the the logic that has to “deal” with the editor.

So the question becomes: how do you learn the editor, without having written it?

Editor extension, in brief

Oh, the irony.  Nothing I write is ever brief.

I'm not sure whether it was a conscious decision in all cases or not, but extension points in the editor tend to have a few characteristics:

Simplicity - Most extension points are very simple and have a very minimal API. Classifiers and taggers, for example, have only one method (for getting classification and tag spans) and event (for informing consumers that classifications or tags have changed) apiece. The contract is simple enough to comprehend pretty quickly.

Single purpose - Extension points tend to do only one specific thing. In the past, if you wanted to provide language specific features, you'd write a language service (which is still supported, by the way). Using the new extensibility mechanisms, however, you'd extend the features you want. You (or anyone else, for that matter) can write a classifier to provide colorization for any language, without regard to who else is providing other features around that language. Moreover, you can write a classifier for a language that already has a classifier written for it - the editor figures out how to combine your contributions.

Follow common patterns - Common API for common scenarios - tagging is probably the best example of this (which I say without any hint of bias, as I wrote most of it - as such, send your complaints my way :). If you want to provide, say, outlining regions, error squiggles, text marker highlights, classification (you can do it with either a tagger or classifier), or certain types of adornments in the text, you'll write a tagger to do so. All these things share a common trait - some component at the level of the buffer wants to provide information for others to consume, oftentimes at the visual level. It's pretty handy to separate these out, so that:

• The component that knows about errors in your language doesn't also have to know how to draw a red squiggly line in the editor
• If someone else wants to visualize errors differently (like by placing a little red line in the vertical scrollbar), they can do that without caring what errors are in the underlying language, or even what the underlining language is (an error is an error, in the most general sense).
• If a new language comes along and wants to provide errors (or someone wants to provide squiggles over something else in an existing language), they can do so without worrying about how to visualize it (a fun little extension to write: put squiggles underneath every instance of "TODO" or "UNDONE").
• The tags provided also go through an ITagAggregator, which takes care of things like combining tags from multiple taggers on the same buffer, and combining all the tags from different buffers in the same buffer graph. This makes it so you don't have to worry about other extensions and whether or not projection is involved - the real beauty of projection buffers is that you (almost) never need to know about them.

So we have the tagging subsystem, which unifies the API for all of these. You can also use it for simpler stuff (say you just have two components that want to communicate from the buffer level to the UI level), and you can define your own ITag types, and thus create new extension opportunities.

In general, then, editor extensibility can be thought of as being wide and not deep. That's not to say that the extensibility doesn't permeate fairly completely, but that you don't need to dig very much to hit water.

The hard part, then, is getting people who want to extend the editor up to speed on the breadth of extensions they can provide and the services they can consume.

How to learn the editor

Between my blog and the official editor team blog, there will be more articles upcoming on tackling this from a few different angles.

To start, I’m planning to write some articles explaining the extensions I wrote (and any more that I end up writing) and providing code samples.  This seems to be a common way of learning a new API, so I assume it will be helpful for many people.

At some point, Jack will be writing some articles on the team blog about the pieces of the editor and how they all fit together.  I’ll probably write a few posts on my blog about some things (Tagging, for sure; the differencing stuff, since I wrote it; writing classifiers/taggers and best practices around that; probably a description of how outlining works in the new editor; etc.).  If there is a specific topic you want to see covered, add it in the comments below.

There is some documentation on editor extensions and the like, with a decent overview of the common extension points for people to use (I’m not sure if there is one for various services that are provided).

Comments always welcome.

On this morning's little public editor conference call, someone remarked (and others agreed) that they wanted a way to do "collapse everything in this file but where the cursor is". Seems like an interesting enough scenario, and it's rather simple to write as an extension, so I went ahead and wrote one during the conference call:

CollapseAllButThis.cs on gist.github.com

One of the things it doesn't really do is handle commands (partially due to a desire for brevity, and partially because the MEF extensibility isn't there for that). It uses a keyprocessor to handle Ctrl+1 as the command for "collapse all but this", which has a few issues inside VS, which is why you should avoid using key processors inside visual studio:

• If there is a command bound to ctrl+1, the key processor will never see the key presses.
• There isn't an actual "command" that people could listen to or handle differently.
• You can't rebind the command to something else (except by changing and re-compiling the code.

I have a work item to make a MEF extension for handling existing commands in the editor, but it doesn't include the ability to register new commands. That's something you can do through DTE, but only if there is an add-in loaded. I'm mulling over creating a prototype extension for letting you create commands via MEF, but it's unfortunately not straightforward. Technically, it's also not really an editor thing, but it still seems like a useful thing to be able to do through MEF.

If you want to try this extension out, you can do what I do:

1. Download VS 2010 Beta 1 and the Beta 1 SDK
2. In VS, create a new project of type Editor Classifier (you can really use any of editor extension templates, I just tend to go to this one first). You can search for it by typing "editor" in the New Project dialog.
3. Delete all the code files except for one (or just delete all the code files and add a new one).
4. Replace the contents of the code file with the pasted gist.

...and you should be off and running. Let me know if that doesn't work for you in the comments, and please feel free to take it and change it however you like. At some point in the future, I'll write an article about how outlining works in the new editor, which should shine more light on the types of things you can do like this (in 100 lines of code or less - this one clocks in around 50, I think).

Comments and questions welcome!

Rico just put up an interesting post on editor performance, so if you have a few minutes, go check it out.

I thought I'd take a minute and give a bit more detail on one of the topics Rico mentions, which he called regions (but are really called markers; po-TAY-to, po-TAH-to, and all that).

(Old editor) VS markers

There are quite a few clients to the editor that want to track a piece of text in a given buffer through edits made to the buffer. Things like bookmarks, breakpoints, error squiggles, and outlining regions (a.k.a. folds) are all examples that either work through markers or just are conventionally tracked through markers. Markers can have some type of associated UI, like the red highlight of a breakpoint, but they don't have to. Outlining regions, for example, don't require markers to be created on the client side, but it's common for language services to create invisible markers to keep track of their positions.

That's all well and good, and it's a service that is truly required, but there are two seemingly small characteristics of markers that make them a huge performance tax on the old editor.

First and foremost, markers are tracked on every edit. Since there is only one real version of the old editor's buffer contents (i.e. what you see on the screen at any time), markers are always accurate against that version. This means that every time the user types a character, the editor has to go through every single marker and figure out where its new position should be. The simple cases are the ones you may expect - if the change comes before the marker, it pushes the marker towards the end of the file; if the change comes after the marker, the marker doesn't move.  However, there are various behaviors for when changes intersect the marker in some way, and there’s a rather nasty little piece of code in the old editor to manage that.

Added to that, there is an optional callback mechanism on markers, so clients can ask to be notified when the marker "moves", where "moves" means that the absolute offset from the start of the buffer or length have changes. This means that if you type a "u" at the very first position in the buffer, the editor has to walk every single marker to figure out if any clients have requested a callback, and inform them that their marker has moved 1 character towards the end of the buffer. Fortunately, this mechanism isn't used all that much, but it's still something that has to be dealt with.

In the past, clients have done some rather heroic things to mitigate these issues. Things like combining what would be multiple edits into a single edit, so that it would be one pass through the markers instead of many, were common. It was an unfortunate but necessary tradeoff.

As Rico said, we've provided a rather comprehensive shim layer over the old editor, which includes markers. Since the real issue with markers wasn't so much the implementation of markers but the design of markers, there wasn't too much we could do to make it faster. This is one of those areas where the only way to get a performance improvement is to move off markers and move forward to the new editor interfaces. That leads us to...

(New editor) Tracking spans (and text markers)

First off, in the new editor, we've split markers into a few parts.

Most marker visualization, like breakpoints use for the red text background, is accomplished through Text Markers, surfaced as TextMarkerTags, provided via Tagging.  Sorry for the confusing naming :(

Squiggle visualization, used for errors of various types, is accomplished through  SquiggleTags, which really should be called something like ErrorTag or similar, as they don't really have anything to do with squiggles, though they happen to be visualized as such.  I still have my fingers crossed that the name will be different in the future.

And the really important part, which is tracking text as the buffer changes, is exposed through tracking spans (ITrackingSpan).

A tracking span is used for basically the same purpose as an (invisible) marker - you create a span over a piece of text, and you can ask for that span's position after the text has changed. But there is one huge difference in how tracking spans operate: tracking spans are evaluated lazily.

If you create a tracking span and then never ask for its position, it will never do anything. Tracking spans don't participate in buffer changes, so you could create a million tracking spans and not pay any cost if you never actually ask them where the span has moved to.

We can do this because of another point Rico mentioned - Snapshots and Versions.

The new editor's data model (for the text buffer) is very functional (as in functional programming, not a measure of "it works well"). A buffer, at its core, is just a Snapshot/Version plus some methods for creating and applying edits. When an edit is applied, the buffer creates a new Snapshot, which shares almost all its data with the previous Snapshot, and places the change deltas in the Version. In this way, there is a complete history of the contents of the buffer (in the past Snapshots), and a complete history of what edits were made in the buffer (in the past Versions).

So, let's say you create a tracking span on version 2, and the user types a couple of times. You want to know where the span is now, so you ask for the span on version 6 (or you ask for the span against the current Snapshot; either works). The tracking span then walks from version 2 to version 6, seeing which changes would cause the span to move, and gives you back the span's position in the new version.

The worst case behavior of tracking spans, then, is that someone treats them exactly as markers. If you create a large number of tracking spans, and request every single span's position on a new version on every single text change, it will be doing just as much work as the markers (minus the callbacks). This is basically what the marker shims do, in order to fulfill the contract of the old marker APIs.  So, the laziness is only helpful if clients use it to their advantage.

Fidelity modes

Next, there are few different kinds of tracking "fidelity" modes. The most common usage of these spans is to keep tracking them forward as the buffer changes (this is basically the only service the old editor markers provided), and so the most common fidelity mode is forward fidelity. What the means is that, when the tracking span gets tracked to version 6, it remembers the span in version 6 and forgets the span from version 2. This means that the next time you ask for it's position in version 6, it won't require any work (besides the check to see if the version requested is the same as the cached span's version).

The other commonly used fidelity mode is backward fidelity. In this mode, the span never forgets the original span it was created on, so you can always track to any version/snapshot that is after the original version/snapshot, without loss of fidelity (tracking before the version that the span was created on will behave just like a forward fidelity span - it'll work, but it may not be exactly what you want). Unless you really need them, however, these should be avoided; because they always remember the old snapshot and version, they'll keep them in memory as long as they're alive, which means the CLR won't be able to collect them. Snapshots and versions are designed to only point forwards in time, so as soon as the last person stops referencing an old snapshot/version, they can all be collected along the chain until the next snapshot/version that someone is holding on to. If you create a backward fidelity tracking span on the very first snapshot, then no snapshots will ever be collected (same goes for a forward fidelity tracking span that you never track), until that tracking span becomes collectible.

Tracking modes

I should also talk briefly about tracking modes; there aren't any performance considerations associated with which mode you pick, but it's important to understand what each means.

There are four different SpanTrackingModes that can be used to specify the tracking behavior of the span when text is changed at its endpoints. There is also an ITrackingPoint, which is basically the same as a 0-length tracking span, and it has PointTrackingModes, so I'll start there.

First off, tracking modes only apply when text is inserted at the point (in the case of a tracking point) or at either of the endpoints (in the case of a tracking span). If text is inserted before the span/point (and not at the point or immediate start of the span), the point/span always moves towards the end of the buffer. If text is inserted after the span/point, it doesn't move. If text is inserted in the middle of the span, the endpoint of the span moves towards the end of the buffer.

There are two point tracking modes: Positive and Negative. Positive is analogous to "move like the caret/cursor does when you type". If you think of typing as inserting a character at the position of the cursor (or, as it visibly appears, just after the cursor, if you think of the cursor as appearing between characters), then the cursor always moves to the right of the newly inserted text. Positive means "towards the end of the buffer". If the cursor was negative, then, it would appear to not move, but the text would be inserted to the right of the cursor.

The four span tracking modes are just the four possible combinations of point tracking modes for its endpoints.

EdgeExclusive is the most common, and EdgeInclusive the next most (Negative and Positive are fairly rare, as there aren’t too many use cases for them).  EdgeExclusive has the nice property that abutting spans (spans that touch on the endpoints but don’t overlap) will never overlap when tracked in this way (since neither one will ever grow towards the other).  Thus, if you are tracking lots of spans as EdgeExclusive, then their relative order will never change, where order is a combination of start point and length.

Things to remember

In order to ensure your extension performs well, there are a few simple things to remember:

Use forward fidelity spans where you can, unless you absolutely need backward fidelity tracking.  If you do need backward fidelity tracking, try and use the tracking span for as short a period as possible (don’t hold references to the tracking span after you are done with it, and it will get collected).

Track only what you need, when you need it.  If you have, say, a sorted list of EdgeExclusive tracking spans that don’t overlap, and you want to find which spans intersect a given span (this is applicable to a tagger or classifier you may write), you can use a binary search and just track the items along the path of the binary search.  Because you are using EdgeExclusive tracking spans, you are guaranteed that the sorting will be stable through text changes, so you don’t actually need to track the “gaps” in the binary search.

Use new APIs, when appropriate.  If you have existing code that uses markers, it’s probably fine to keep around; for example, we’ve never seen a noticeable performance impact from the markers that come from breakpoints, because there aren’t usually that many.  However, if you had something that was creating a very large number of markers, and have found in the past that that was a performance issue, it may be a good idea to move off the old marker API.

Comments appreciated.  Thanks!

Just got this link from Chris – looks like the beta 1 editor docs are up.  Feast and be merry.

(Questions or comments can be put here, tweeted to @noahsmark, or you can get help on the editor forums).

Slow start, eh? I haven't written a post in here since I started this blog, as I've been a bit, uh, distracted by other work-like things. One of those things was implementing box selection in the new editor.

Quick side note: coming up in the near future, I'll write a bit about two extensions I put up on the extension gallery for the beta: GoToDef (ctrl+click does go to definition) and ItalicComments.

First off, here's a video that Brittany put together on the new box selection stuff that we've (Brittany as a PM, Luke as QA extrodinaire, and Ameen on some of the box selection UI stuff) been working on and recently checked in:

You can jump on over to the article on the VS Editor blog to read the full announcement-y version. I wanted to share a bit about getting it written, though.

Though it's not necessarily a technically complicated feature, it's one of those things that ends up touching a good deal of the codebase (effectively everything that wants to know about what text is currently selected). Though I could have "hidden" box selection, in a sense, by forcing people who wanted to know about selection mode to go looking for it, I decided for the slightly more onerous path of making people who ask about the selection deal with it, at least to some degree.

It's a somewhat irksome decision to make; an early prototype of the box selection feature went the opposite route, leaving the API alone as much as possible. While it simplified some of the cases (like asking for the selected span while the selection was in stream mode), it meant that things would fail in odd ways in box selection mode, since it was a piece of the API that wasn't necessarily obvious. It would be kinda like having some type called Gizmo, with a property on it called GizmoText, and another property called JustKiddingcommaHereIsTheRealText. Not quite that bad, though :)

The scarier one is that you could have written code against the old API that didn't change at the semantic level of the programming language (there was still a property, called SelectionSpan, that returned a SnapshotSpan), but the but the behavior at the semantic of the editor had changed, especially relative to other consumers that were doing "the right thing" with the new API.

I'm all about trying to make as few changes for consumers as possible, but if the consumer really does need to know about the change, you're doing them a disservice by allowing the same code to compile and run before and after the change, but with different behavior.

So, the new model is a bit more complicated than the original, though it lends itself to decently simple usage (using Linq or just iterating over the collection with foreach).

A selection has a collection of SelectedSpans or VirtualSelectedSpans (yeah, I changed the tense; I don't really have a great reason for why, besides that it sounds slightly better to me and would hopefully be a bit more obvious of a change than just adding an "s" to the old property name and making the ensuing build breaks slightly harder to figure out). The Virtual version takes into account virtual space, which is frustrating enough to deal with that it probably deserves its own blog post :)

In most cases, you can treat box selection and stream (regular) selection the same by writing code like:

A box selection on the clipboard is just an endline-separated string; when we read it back in for a paste, we look out for a special clipboard data object (that VS has used for awhile), and take that to mean "split this string back into separate lines and insert them a line at a time.

Multi-line insertion

One of the new (at least to VS) things about the implementation is multi-line insertion mode. If you make an "empty" box selection (or make a non-empty box selection and then delete it), you can type, backspace, and delete as if you had a gigantic cursor - whatever you type gets inserted on each line, backspace deletes a character on each line, and delete behaves similarly. It may not be a feature people will use all the time, but it's one of those nice things that makes you slightly happier a couple of minutes a week. As a recently filed bug report said, it'll keep you from cursing me for 5 minutes each week.

So, what would you use it for? Here are a few things (most of them are covered in the video):

• Appending something to the front of field names (depending on the file I'm in, the language it's written in, and who wrote it: "_", "m_", things like that).
• Adding "private" or "internal" in front of a set of fields.
• Adding "//" at the beginning of a couple of lines.
• Given a set of variable names that you've box selected, copied, and pasted, you could add various pieces of code around it, like "Console.WriteLine(" and ");".
• Since a box selection can now be entirely in virtual space, you can add a comment block past the end of a line by creating an empty box in virtual space and typing "//"
• Whatever else you write in the comments :)

An interesting side-effect of being able to create box selections entirely in virtual space is that alt+click (even before you drag) sets the cursor in virtual space. Originally, I thought this felt kinda buggy, and spent a bit of time thinking about how to work around it, until I realized the use for it.

I hate lining up arguments to method calls. Seriously hate. At some point, maybe the languages in VS will make TAB be nice and emacs-y and line up arguments; until that day, I spend a couple of seconds like this:

TAB, TAB, TAB, oh, crap, too far, backspace, backspace, NO, missed it again, space, space, space, space, space, space...

Now, I'm not usually big on the mouse, being a vim guy at heart, but here's one case where I can bow to the mouse:

var foo = some.Property.MethodCall(firstLongParameterName,
                                   **alt+click here**

Yeah, sign me up.

Anyways, comments welcome.

For those of you, like myself, who weren't lucky enough to be vacationing in beautiful Barcelona and happened to drop in on TechEd EMEA, or if you haven't found it elsewhere on the interwebs, I'd like to introduce you to... [drum roll]....

Scott Guthrie showing the "comment adornment" at PDC

Another pic of the same sample

The new editor!

Here are a few interesting things about the new editor:

1. The editor is designed to be extensible from the start. The editor is built from composable parts and uses the Managed Extensibility Framework (MEF) to tie things together. This means that people who want to add features to the editor or modify existing behavior get to use the exact same tools that we use when building features in the editor. In effect, the editor is virtually all public surface area.
2. The editor is written using the Windows Presentation Foundation (WPF). This allows us (and anyone who wants to extend the editor) to be fairly creative in the richness of our visualizations (such as the pictures above, where comments have been replaced by rich edit controls). This lets you do everything from gradient disease (you know exactly what I'm talking about) to useful things, like embedding controls directly into your editing surface and finding better ways to visualize information normally just presented as text.
3. The editor is written in managed code, which has benefits like the following:
   • Without getting too much into the religiosity of managed vs. native code, it is much easier to write managed extensions against the new editor than native ones against the old. No COM, no "whoops I accidentally leaked Visual Studio, my bad", and certainly NO COM (I can't stress that last one enough). Oh, and you don't have to mess around with COM.
   • You'll write less code (which I'm going to show here in the coming weeks and months) and (hopefully) safer code, which is generally a good thing.
   • You get to take your pick of favorite .NET language — we write mostly in C# (on the editor team), but we've seen extensions in C#, VB.NET, F#, and others. I'll probably provide a few samples on this blog in languages other than C#, so you'll be able to see what I mean.
4. For those existing extenders that don't want to update anything, we've provided "shim" interfaces that masquerade as the old editor. We haven't shimmed everything, but we've implemented the majority of the existing native interfaces.

A little about myself, since this is supposed to be one of those meet-and-greet type of posts:

My name is Noah, and I've been on the editor team (and at Microsoft) for a little over a year (I did an internship here before, if you want to count that). I'm a recent college grad (CS undergrad at RIT), so I'm still one of them thur youngins on the team. My favorite color is blue, and my favorite editors are vim and emacs; yes, I use both (primarily vim; and Emacs for things like slime, muse, and org-mode), so I don't buy into the whole "best editor evar" flame war (my asbestos long johns are on, though, just in case). I'm what our architect jokingly refers to as one of "those open source socialists", which is a big reason why I'm pretty heavily invested in making our editor as extensible and open as possible — I believe that the most important value of any product or project is the community that exists around it.

In the coming days and weeks, I'll be writing various samples/extensions for the editor, as well as giving an overview of the many ways that people can extend the editor. I'll put up samples for various things: features I miss from other editors, things that you couldn't (easily) do in the old editor, samples that somehow involve ice cream (I really like ice cream), etc. If you are an actual person reading this post and not the google search indexer, feel free to write a comment if you want to see something specific. I suppose, if you are the google spider, you can post a comment, too; we're very accepting of all people and bots on this blog, unless you are the prince of some African country who has seventy gigabazillion dollars and just needs $1000 from me to get set up in America.

The images in this post come from Long Zheng's Flickr photostream, from the "PDC08: Keynote 2" set.