Craig Skibo's WebLog : VSIP

Visual IL

CraigSkibo — Mon, 12 Jul 2004 20:08:00 GMT

After a lot of time spent working on my IL editor and IL project, I finally have a workable version ready for you to try out, if you would like. The MSI file that is linked below, includes code for a project and an editor. The project has templates to create a class library, console application, and a windows form application, supports an automation model, and to make IL development easier, you can even import existing DLL or EXE assemblies into your project (it will be decompiled and the generated IL file will be imported into the project). You can also build your project and then debug the generated code. The language service will do colorization of the IL file, supports statement completion and parameter info for opcodes and keywords, will find TODO comments and add an entry to the task list, and referencing external assemblies is as easy as using the Add Reference dialog box - right click the document and choose Insert Assembly Reference.

System Requirements:

Visual Studio 2005 Beta 1
You must have the VSIP Beta 1 SDK installed, without this the editor and project packages will refuse to load.

Known bugs:

The automation model for the project is not complete.
Renaming a document that is open does not work correctly. You can properly rename them if the file is not open.
If you open a file inside an IL project, close the project, then reopen it, the open file will not reopen.
Parameter tips for all opcodes and keywords do not have descriptive text.
Project and Project Item templates have some bugs. Specifically, there are no replacements done on the templates. For example the Windows Application template has the module name set to WindowsApplication1.exe. The assembly name for a console application is set to Project17. This is because I have not written wizards for generating projects yet.
After running the installer to install, you need to run the command “devenv /setup” from the command prompt to make sure the project and editor are properly installed.

I also found a few bugs in Visual Studio that cause problems with the project:

If you open a C#, VB, J#, etc. application then try to add a project to project reference in the C#, VB, or J# app to the IL project, it will fail to build. I am working with the VS project group to make sure this is fixed before VS final ships.
Parts of the automation model for the IL project will throw a cast exception. This is because of a bug in the EnvDTE PIAs, and should be fixed at least before Visual Studio final ships.

Note that this version will time bomb Aug 1, causing any projects to not load and any IL files you open will appear as if you did not have the packages installed. The reason I did this is because I will be updating the code over time with bug fixes and I would like people to look at the latest code. The time bomb will gently nudge (OK, force) people to use the latest bits. Also, this version does not have source code included. I am not sure when this will be released, after all, this will be the headline sample included with the next version of our book, and we would like to present the code with the text of the book.

Remember that this will be just a sample. While I tried to make it as useable as I could, it is not as full-featured as C#, VB, C++, etc. and does not include features such as a WinForm designer, all the smart tag functionality, etc. This is a beta quality sample, and while I tried to find and fix as many bugs as I could find, there may be the occasional bug which could cause data loss; so use at your own risk. If you do find a bug, please post it here and I will try to fix it as soon as possible.

You can download the code from http://workspaces.gotdotnet.com/VisualIL. When running the installer, the installation key is all 1 values.

Visual Studio Developer Center

CraigSkibo — Tue, 23 Mar 2004 22:15:00 GMT

If you have not noticed, the Visual Studio Developer Center on MSDN is now live. You will be able to find articles, whitepapers, etc. about programming both Automation and VSIP for Visual Studio. Visit the Developer Center at http://msdn.microsoft.com/vstudio/extend.

News, an update, and a tip

CraigSkibo — Mon, 22 Mar 2004 19:45:00 GMT

If you have a MSN Watch, then basketball is now available as a channel. Now if I only liked basketball… Where is baseball and hockey?

An update to my last post: as I suspected, the .NET Framework was not the cause of my performance problems in my Visual Studio language service. When you would scroll through a text document with about 200 lines of code, it would take a few seconds to redraw with each click in the vertical scroll bar – not exactly a well-performing application. A few modifications to the source, and everything is running smoothly now. I am rolling my own tokenizer for a language service and I had one method, IsAtBeginning, which was causing all the problems. IsAtBeginning is given a text string pszText, start at position nIndex, and a second string pszLookFor. IsAtBeginning would check to see if the text given in the variable pszLookFor is at that position nIndex in pszText. Also, the function needs to make sure that the character at nIndex-1 and the character at nIndex+Length of pszLookFor+1 is not a character that keeps the string at that location from being a special token. For example, if pszLookFor is the IL keyword .subsystem, then the function should return true when:

pszText = “.subsystem” and nIndex = 0

pszText = “ .subsystem” and nIndex = 1

pszText = “.subsystem ” and nIndex = 0

pszText = “.subsystem(” and nIndex = 0

pszText = “ (.subsystem” and nIndex = 0

and return false when:

pszText = “X.subsystem” and nIndex = 0

pszText = “X.subsystem” and nIndex = 1

pszText = “.subsystemX” and nIndex = 0

After some experimentation (which included commenting out all of my tokenizer and one by one adding bits of code back), I tracked down the problem to the IsAtBeginning function. Examination of that function showed that I was doing some unnecessary string manipulation – I translated the C++ code “if(!wcsncmp(pszText+nIndex, pszLookFor, wcslen(pszLookFor)) {…}” Into a number of C# string.Substring calls, which is not as efficient. The moral of the story: I always like to blame my code as the culprit before blaming somebody else’s. I could have said it was a problem with the .NET Frameworks, but after some investigation it was my fault.

Here is the code I use now (I am sure there is more optimization I could perform, but for now this is what I am using):

static string specialIDChar = "#$@?_";

bool IsAtBeginning(string text, int index, string lookFor)

{

//If the char before or after is a space, tab, or non alphanumeric,

// (meaning that there is not a character infront of it disqualifying it as our token) then verify

// that the text is what we are looking for:

int lenLookFor = lookFor.Length;

int lenText = text.Length;

if((lenText - index) < lenLookFor)

return false;

char before = (index == 0) ? ' ' : text[index-1];

char after = ((lenText - index) > lenLookFor) ? text[index+lenLookFor] : ' ';

if((specialIDChar.IndexOf(before) != -1) || (specialIDChar.IndexOf(after) != -1) || char.IsLetterOrDigit(before) || char.IsLetterOrDigit(after))

return false;

if(text.Substring(index, lenLookFor) == lookFor)

{

return true;

}

return false;

}

Now the tip: I was recently working on a pet project, a blog reader. The architecture is that I have an engine that manages syncing to and holding blog data, while there is a user control that manages display of the data. The engine defines a couple delegates that the user interface connects to so that it can be informed when a blog is being updated. I have been using serialization to save data to disk, but suddenly I was running into problems where I was getting an exception when trying to save data to disk because the control used to display data was not marked with the [Serializable] attribute. After some investigation, I found that the problem was with the delegates. It turns out that when the control connected to the events fired by the blog engine, and I tried to serialize the engine, it also tried to serialize anything connected to the events thrown by the engine. The solution in this case was to disconnect all events before serializing. This may not work in all cases, but for what I was doing (serializing in the destructor of the class, when any events would not need to fire any longer), it worked just fine.

Adventures in writing managed VSIP packages

CraigSkibo — Tue, 09 Mar 2004 22:48:00 GMT

It has been a little too long since I last made a post. A lot has been going on since the last time I wrote, mostly I have been working on getting bugs fixed for the “Whidbey” build of VS, and work has been proceeding on a new wizards tool for VS (while my last post said I would have some information about this wizard, we decided to wait for any announcements about it). Another task that has consumed my time is my IL editor/project sample. As you may know, I have been working on an IL editor and project system sample. If we get a Whidbey version of our book, then any chapters about VSIP will discuss these samples, and I am looking into getting them into a future version of the VSIP SDK. All major work on these samples were completed over the weekend, and only some minor bugs remain. As I mentioned before, the samples that I have been working on were written using ATL, but after some careful thought I decided that they should be converted to C#. So last night I got the latest build of the Everett Extras (interop assemblies for programming the Everett VSIP model – for now I am doing my development against the Everett version of VS) and sat down to convert my existing code from C++ to C#.

Converting the text editor took me a total of about 4 hours. 4 hours! I was expecting it to be much more complicated than it turned out to be, and most of those 4 hours were spent trying to track down an exception that was being thrown within my colorizer because of a bounds checking bug in my C++ code. I did not have any prior experience using the VSIP Extras SDK before last night, so I cannot say that it was easy because I knew how to use the Extras IAs, it was easy because there were only minor differences between the C++ and C# code.

The process was not flawless, however. I was bitten a few times because of some of the odd data types that the IAs seems to use and the conversion of those types to managed types. For example, there are a number of System.IntPtr objects that are passed back and forth, usually when a WCHAR* type is used on the unmanaged side. In this case you want to call System.Runtime.InteropServices.Marshal.PtrToStringUni to convert to a managed string (not PtrToStringBSTR like I was using at first, which would cause an immediate crash). I also had some problems setting up a service provider, I had to try overriding a number of different methods to get it working correctly – and there may be code in the Extras helpers to handle doing that in a cleaner way (don’t forget the Marshal.QueryInterface method, this will help you a lot when trying to implement your service provider). There also has been a noticeable slowdown when switching to a managed language, but most of that is, I believe, a problem with my code and not so much related to managed code. I am doing a lot of string manipulations in the colorizer, and a quick look over the code reveals many places where I can use StringBuilders rather than strings to speed up my code. I was also using a lot of pointers to keep track of positions into strings for optimizations, and when I converted to C# I had to undo those changes.

The next few days will be spent doing some optimizations on the code, and then I will start working on converting the project code to C#. When this experiment is done the packages will be written in both C# and VC for side by side comparisons, and maybe I will try converting to the other languages also. I don’t know of any packages written using VB and J#, and maybe as a proof of concept of using the IL editor/project I can convert them to IL.

Just as I was getting ready to post this, mail was circulated describing how to proffer a service from managed code using code such as this

container.AddService(service.GetType(), service, false);

inside the Initialize method of a package. I noticed that method last night and tried to use it, but I could not get it to work correctly. I will try it again with my code and see where I went wrong.

Adventures in writing managed VSIP packages

CraigSkibo — Tue, 09 Mar 2004 22:27:00 GMT

Just as I was getting ready to post this, mail was circulated describing how to proffer a service from managed code using code such as this

container.AddService(service.GetType(), service, false);

inside the Initialize method of a package. I noticed that method last night and tried to use it, but I could not get it to work correctly. I will try it again with my code and see where I went wrong.

Use VSASSERT to help debug

CraigSkibo — Tue, 13 Jan 2004 22:03:00 GMT

Here is a second tip for debugging that requires a slight .h file change. If you have the VSIP SDK installed and you need advanced assert help, use VSASSERT. VSASSERT is a nifty utility that is installed with the VSIP SDK that will put up an assert message. The advantage of this assert technology over the CRT or ATL’s assert is that you can hide all the VS asserts for that instance of VS, selectively choose to hide a particular assert, copy the assert text to the clipboard, break into the debugger at the line of the offending assert, etc. To use this assert, you need to #include the file vsassert.h into your code, and then for the debug build of your code link to vsassert.lib. Then, rather than using VC’s assert mechanism, you use VS’s like so:

VSASSERT(false, "This is a failed assert.");

Even though the assert ships with the VSIP SDK, your code does not need to be VSIP related to use VSASSERT, the code can be a VSIP package, an Add-in, or even code that is not related to VS, you just need to be using C++.

Now to the problem: To use VSASSERT, you will need to change a header file to get everything to work. If you were to compile anything that #includes vsassert.h, you will get the following error message:

c:\Program Files\VSIP 7.1\EnvSDK\common\inc\vsassert.h(249): fatal error C1017: invalid integer constant expression

You can fix this by opening vsassert.h, and on line 249 change the text from:

#if DEBUG

#ifdef _DEBUG

This simple change will fix up the problem, and you can start using a much better assert tool. (And yes, there is a bug tracking this to have somebody fix it)

Fix debugging QIs in ATL code

CraigSkibo — Mon, 12 Jan 2004 23:28:00 GMT

I have been doing a lot of VSIP package writing lately. While I was writing a project for VS I wanted to find out which interfaces VS is looking for on my package so I could implement those interfaces. I decided to turn on ATL’s QI tracking feature to make tracking down these interfaces easier. You can turn this feature on with a simple bit of code such as the following in the stdafx.h file:

#ifdef _DEBUG

#define _ATL_DEBUG_QI

#endif

What will happen is, with this code, every time an object in your project has its QueryInterface method called, it will dump a line of text such as the following into the VS output window:

CProjectHier – IUnknown

This means that the method CProjectHier::QueryInterface was called and it was passed an IID of IID_IUnknown (meaning that the caller wants the IUnknown interface). When the QI is called and it cannot supply the requested interface, it dumps out a line of text such as:

CProjectHier - IVsPersistHierarchyItem – failed

Because I have not yet implemented IVsPersistHierarchyItem on the CProjectHier class, ATL kindly tells me that the QI failed. I can then browse the text of the output window, see which interfaces I need to implement, and then implement them. Great! It saves me the time of wading through MSDN!

However, there is a bug in this mechanism. Since the bug is in a debugging feature and not in something that will affect production code (you should not ship code with this feature on, hence the #ifdef _DEBUG in the code above), this is not a serious problem. When ATL looks up the interface name of the IID that is passed to QI, it looks in the registry under HKEY_CLASSES_ROOT\Interface\{IID}, and takes the default value under this key as the name of the interface. These values are placed there when you register a typelib with the system; the API call LoadRegTypeLib will look at all the interfaces, coclasses, etc. in a tlb and place the correct entries under HKCR\Interface.

Many of the interfaces for the VSIP SDK are not defined in a tlb, they are defined only in .h or a metadata assembly if you are using the Extras SDK, and since they are not in a tlb the IIDs for an interface are never registered. The ATL code has a small bug in it that if the IID for the interface is not registered then it will print something like this for a successful QI:

CProjectHier –

or something like this if the QI failed:

CProjectHier – - failed

As you can see here, the name of the interface is missing. You can easily change the code in ATL to fix this. First, browse to the ATL include directory (“C:\program Files\microsoft visual studio .net 2003\vc7\atlmfc\include” if you used the standard install path) and open the file atlbase.h. Next, search for AtlDumpIID and replace it with the following code:

#if defined(_ATL_DEBUG_INTERFACES) || defined(_ATL_DEBUG_QI)

__forceinline HRESULT WINAPI AtlDumpIID(REFIID iid, LPCTSTR pszClassName, HRESULT hr) throw()

{

USES_CONVERSION_EX;

CRegKey key;

TCHAR szName[100];

DWORD dwType;

DWORD dw = sizeof(szName);

LPOLESTR pszGUID = NULL;

if (FAILED(StringFromCLSID(iid, &pszGUID)))

return hr;

OutputDebugString(pszClassName);

OutputDebugString(_T(" - "));

LPTSTR lpszGUID = OLE2T_EX(pszGUID, _ATL_SAFE_ALLOCA_DEF_THRESHOLD);

#ifndef _UNICODE

if(lpszGUID == NULL)

{

CoTaskMemFree(pszGUID);

return hr;

}

#endif

// Attempt to find it in the interfaces section

if (key.Open(HKEY_CLASSES_ROOT, _T("Interface"), KEY_READ) == ERROR_SUCCESS)

{

if (key.Open(key, lpszGUID, KEY_READ) == ERROR_SUCCESS)

{

*szName = 0;

if (RegQueryValueEx(key.m_hKey, (LPTSTR)NULL, NULL, &dwType, (LPBYTE)szName, &dw) == ERROR_SUCCESS)

{

OutputDebugString(szName);

}

else

{

goto Error;

}

else

{

goto Error;

}

// Attempt to find it in the clsid section

else if (key.Open(HKEY_CLASSES_ROOT, _T("CLSID"), KEY_READ) == ERROR_SUCCESS)

{

if (key.Open(key, lpszGUID, KEY_READ) == ERROR_SUCCESS)

{

*szName = 0;

if (RegQueryValueEx(key.m_hKey, (LPTSTR)NULL, NULL, &dwType, (LPBYTE)szName, &dw) == ERROR_SUCCESS)

{

OutputDebugString(_T("(CLSID\?\?\?) "));

OutputDebugString(szName);

}

else

{

goto Error;

}

else

{

goto Error;

}

else

OutputDebugString(lpszGUID);

if (hr != S_OK)

OutputDebugString(_T(" - failed"));

OutputDebugString(_T("\n"));

CoTaskMemFree(pszGUID);

return hr;

Error:

OutputDebugString(lpszGUID);

OutputDebugString(_T("\n"));

return hr;

}

#endif // _ATL_DEBUG_INTERFACES || _ATL_DEBUG_QI

Now, when an interface name cannot be found, it will display the IID of the interface in the name’s place. It is still not as convenient as printing the name of the interface, but at least you will get something. You can then search over the header files for that IID to see what interface you need. This is much better than putting a breakpoint on your COM map and watching every single QI, then checking to see if it succeeds or fails.

And remember, this code is not just for VSIP (although that is how I found the problem), you can use this code for all projects.

Crossing the VSIP Bridge - Part 2(a)

CraigSkibo — Tue, 02 Dec 2003 01:49:00 GMT

Sorry this took so long to get out, but I have had some connectivity issues lately and could not post to this blog.

It seems that my code to Crossing the VSIP Bridge - Part 2 has a bug in it that Oleg has kindly pointed out. Here is the problem:

sp.QueryService(guid1, guid2, ip);

Now what happens with the ip object? Implementations of QueryService first find the correct object that implements the service, then it performs a QueryInterface, looking for the interface identified by the second GUID. That interface is returned through the ip argument. The problem is the IntPtr is GCed, but not the interface that it points to (since all IntPtr knows is it points to random data). To fix this, after performing the line of code:

vsuishell = System.Runtime.InteropServices.Marshal.GetObjectForIUnknown(ip);

You should also do this:

System.Runtime.InteropServices.Marshal.Release(ip);

to make sure the interface that ip wraps is properly released.

Crossing the VSIP Bridge - Part 2

CraigSkibo — Mon, 17 Nov 2003 23:27:00 GMT

I have been putting together a presentation on how to use VSIP from automation (and vice-versa), and one bit of code that I have is how to call a VSIP interface from a macro. To make the VSIP calls, you need to first install the VSIP Extras SDK available from the vsipdev.com web site. Next, within the directory you installed into, copy the interop assemblies (IAs) into the {VS Install Directory}\Common7\IDE\PublicAssemblies folder. This is necessary to make the assemblies available. To your macro projects. Next, open the macros IDE, choose a project, right click the References node, and add a reference to the Microsoft.VisualStudio.* assemblies. After doing this, you are ready to call VSIP interfaces. The following code demonstrates this by retrieving the application name and showing it in a message box:

Sub ShowAppName()

Dim sp As Microsoft.VisualStudio.OLE.Interop.IServiceProvider

Dim vsuishell As Microsoft.VisualStudio.Shell.Interop.IVsUIShell

Dim guid1 As System.Guid

Dim guid2 As System.Guid

Dim ip As System.IntPtr

Dim appName As String

sp = DTE

guid1 = GetType(Microsoft.VisualStudio.Shell.Interop.SVsUIShell).GUID

guid2 = GetType(Microsoft.VisualStudio.Shell.Interop.IVsUIShell).GUID

sp.QueryService(guid1, guid2, ip)

vsuishell = System.Runtime.InteropServices.Marshal.GetObjectForIUnknown(ip)

vsuishell.GetAppName(appName)

MsgBox(appName)

End Sub

This could be more readable with a function to retrieve a service in a more generic way, but I will leave that up to you to exercise your programming skills.

One word of caution: some methods and interfaces are marked local in the idl from which the IAs were generated. While this bit of information does not transfer to the IAs, when you try to call those methods or interfaces, they will fail. This is because the Macros IDE runs in a separate process from VS, and making these cross-process calls will fail because you cannot remote GDI objects (HICONs, HBITMAPs, etc.).

Crossing the VSIP bridge

CraigSkibo — Thu, 16 Oct 2003 03:36:00 GMT

I had another person tell me today that they decided to scrap the Add-in code they wrote and start writing VSIP packages. My response to them is always “Why?” Not too many people know that you can go from the automation object model into VSIP, and from VSIP to the automation object model, with just a little bit of code. And that little bit of code can save you a lot of work, letting you keep your existing code base and migrating it forward to use something that may not be supported in one programmability model or the other.

When an Add-in, wizard, or macro is loaded/run, an instance of the DTE object is made available to that code. It is not well known (we actually tried to hide this information in the pre-free VSIP days) that DTE is a service provider, meaning you can get to any service on the VSIP side simply by doing a QI on the DTE object for the interface IServiceProvider, and then calling IServiceProvider::QueryService(…) to get to the desired service.

Since DTE is a service provider, that means you can get to any automation object model interface from your package. With the IServiceProvider provided to your package when it is loaded, you can call QueryService(SID_SDTE, IID_DTE, …) to get to a DTE object, then you can spelunk the object model to get where you want.

This means you can choose whether you want a package or and an Add-in for your code, and can use the other model for the path of least resistance to get the job done. Suppose you have a package and need to put one or two items in the task list. Skip the thousands of lines of code to create a task provider and talk directly to the TaskList object. Or use IVsUIShell from an Add-in to post a command with PostExecCommand rather than using the SendMessage-like ExecuteCommand.

Of course, you cannot do everything with an Add-in. For example, you cannot create projects or text document windows, and there are a few other things you cannot do, but you can mix and match the two programming models to fit your needs.

One warning: there is a specific license agreement that you must agree to (I have never read it, so I cannot give any more details) if you use VSIP, while the automation object model has no license restrictions regarding creating and freely distributing an Add-in, wizard, or macro. So make sure you understand the VSIP license before starting to use the VSIP interfaces.