Windows and .NET Programming

Building a large text file editor – Part II

Costin Boldisor — Fri, 02 Mar 2012 15:56:00 GMT

In the previous post I created a class that was going to be used as an interface between a large file editor and the file on the disk.

Since then I've been implementing the user interface and, obviously, the RevisionStream also needed some readjustment. Here are some major changes that were done:

- detecting encoding of text file

- support for Unicode (2-bytes characters)

- some bugs related to addition and deletion

Until now I've managed to repair most of the visible bugs but I'm pretty sure there are some also remaining. Which is why, apart from the UserControl that holds the large file editor, I've also created a Form for testing the control. The source code is available so everyone interested and go ahead and fix the bugs they discover. I will also appreciate if you post comments with feedback.

The UserControl LargeTextBox contains a RichTextBox and a vertical scroll bar.

Apart from the RevisionStream, the LargeTextBox control also contains:

- long topPosition; // The position in the changed stream of first line in RTB

- char[][] displayedLines; // The list of currently displayed text lines

- int[] lineLengths; // The list of lengths of currently displayed text lines

The most delicate part of the whole process is to detect the changes done by the user in the RichTextBox and update them in the Revision Stream. This is implemented in the method SynchDifferences where the cached lines are compared with the lines from the RichTextBox and Insertions and Deletions are added accordingly.

Let's see how this works on an Excel file. I create a Workbook with as many rows as Excel allows, namely 1,048,576. I saved this to Book2.xlsx and extracted the sheet1.xml. I then indented the XML to make more human-readable and loded it into the editor.

I then scrolled down to row 511,386 and added a new cell on the B column with the value of "0".

I saved the XML as sheet2.xml and then renamed it back to sheet1.xml (minor bug remaining - can't save to same file name). After repackaging the file in Excel, here's the result:

Building a large text file editor – Part I

Costin Boldisor — Wed, 25 Jan 2012 10:36:00 GMT

The purpose of this series of two blog posts is to illustrate how an editor for large files can be implemented. The first part will address the model behind the editor whereas the second part will include an actual UserControl that allows users to view and edit large files.

The basic idea behind a large file viewer is that you can’t load a very large file into a TextBox or RichTextBox control because the memory usage as well as the time to load the file will be enormous. Moreover, the user can only see a very small portion of the file on the screen at once, so it doesn’t make sense to load all the data in the file at once.

As such, what is needed from a large file viewer is to only display some small parts of the file in order to fill at least a screen. This is pretty straightforward and will be addressed in the second post. What is a bit more complex is to enable the user to edit the file (i.e. make insertions and deletions). The problem with editing the file is that edits need to be taken into account when computing the position in the stream. If the user deletes say the first 10 characters and then wants to read two characters at offset 4, what actually needs to be read is the two characters at offset 14 from the original file:

Original string: "0123456789abcdefgh"
String after deleting 10 characters: "abcdefgh"
Character at offset 4 (zero-based): "e"
Offset in initial string: 14.

Obviously, things get more complicated when the user starts to randomly delete and insert text at various locations in the file.

The solution I’m proposing is the class RevisionStream (see attached code). This class is wrapper around the FileStream class and supports four public methods:

AddDeletion – the caller signals that a deletion has been made by the user
AddInsertion – the caller signals that the user has inserted some text
Read – reads some bytes from the revised stream
SaveTo – saves revised stream to file

The basic idea behind this approach is that normally the largest part of the text will remain identical to the original stream and only some minor changes will be done by the user. So it makes sense to leave the original text on disk and only load to memory the parts that we need together with the changes the user makes.

Under the hood, the RevisionStream uses a list of Segments. A segment is either a reference to a contiguous section of the original text or a newly added text. The really difficult part is to adjust the segments from the RevisionStream when an insertion or a deletion is made.

Getting back to the previous example, the RevisionStream initially contains all the text from the original file, which is "0123456789abcdefgh". The list of segments will contain only one segment, [0, 17] (where 17 = the offset of the last character).

Scenario #1: The user deletes the first 10 characters

Resulting list of segments: [10, 17].
Resulting string: "abcdefgh"

Scenario #2: The user deletes the character ‘b’, which is located at offset 11.

Resulting list of segments: [0, 10], [12, 17].
Resulting string: "0123456789acdefgh".

Scenario #3: The user deletes the character ‘b’, which is located at offset 11, and writes "ZZZ" at the same offset.

Resulting list of segments: [0, 10],["ZZZ"],[12, 17].
Resulting string: "0123456789aZZZcdefgh".

The real challenge here is to deal with all the possible scenarios of deleting and inserting text (delete from more consecutive segments, delete at end of segment, etc.). Hopefully I’ve covered all the possible scenarios in the implementations of the private methods AdjustForInsertion and AdjustForDeletion but if you find any case that hasn’t been covered, please post a comment here.

There’s still some work to do here: using only long indices to ensure really large files can be read, making the Insertions encoding-independent, etc. However, up to this point, we can go ahead and build a pretty decent Large File Editor based on this Revision Stream.

Monitoring when registry keys are modified

Costin Boldisor — Tue, 29 Nov 2011 16:06:00 GMT

Let’s say you want to see where a certain setting from Word Options is saved in Registry.

The easiest method is using Process Monitor.

Add a filter on Process Name and Operation, start Word and that’s that:

If we uncheck “Allow background saves” from Word Options – Advanced – Save, this is what appears in Process Monitor:

The problem with Process Monitor is that we can’t leave it running for a long time because the page file will become too large and it will not be able to continue the capture.

So the alternative is Registry Auditing.

What we need to do is:

1. Run the following command from Command Prompt:

auditpol /set /subcategory:"Registry" /success:enable

Note: if the OS has a different language pack, the name “Registry” might differ. For instance, on a German Windows, the name is “Registrierung”. To see what the name of the subcategory is you can run:

auditpol /list /subcategory:*

2.       Open Registry Editor and navigate to the key which we want to audit (HKEY_CURRENT_USER\Software\Microsoft\Office\14.0\Word)
3.       Right-click on the key and choose “Permissions…”
4.       Click “Advanced” and switch to the Auditing tab
5.       Add a user or group and select Access: Set Value
6.       Apply settings

Now the registry changes are visible in the Event Viewer under Windows Logs\Security:

More detailed information on this topic:

- http://support.microsoft.com/kb/921469 How to use Group Policy to configure detailed security auditing settings for Windows Vista-based and Windows Server 2008-based computers in a Windows Server 2008 domain, in a Windows Server 2003 domain, or in a Windows 2000 domain.

- http://technet.microsoft.com/en-us/library/cc755264(WS.10).aspx Auditpol set

- http://support.microsoft.com/kb/324739 How to use Group Policy to audit registry keys in Windows Server 2003

Back to school - Office art geometry

Costin Boldisor — Mon, 31 Oct 2011 13:05:00 GMT

Let's start off with the following observation- the following two shapes have the same Left and Top coordinates:

What's different is obviously the rotation. What Office saves for a shape is the Left and Top coordinates together with a Rotation. So the Top coordinate, for instance, is not necessarily the uppermost part of a shape.

So let's solve the following problem: to which shape is the following arrow pointing?

Input data:

Arrow: Left = 98.74763 Top = 67.08575 Width = 162 Height = 60 Rotation = 45

Upper shape: Left = 264 Top = 36 Width = 168 Height = 72 Rotation = 0

Lower shape: Left = 264 Top = 114 Width = 168 Height = 72 Rotation = 0

The code for computing the "From" and "To" points:

 public static void GetFromAndTo(RectangleF bounds, double rot, 
 ref PointF from, ref PointF to)
 {
 
 double La, Ta, Wa, Ha, alpha;
 La = bounds.Left;
 Ta = bounds.Top;
 Wa = bounds.Width;
 Ha = bounds.Height;
 
 alpha = (rot * Math.PI) / 180;
 
 from.X = (float)(La + Wa * (1 - Math.Cos(alpha)) / 2);
 from.Y = (float)(Ta + (Ha / 2) - (Wa * Math.Sin(alpha)) / 2);
 to.X = (float)(La + Wa * (1 + Math.Cos(alpha)) / 2);
 to.Y = (float)(Ta + (Ha / 2) + (Wa * Math.Sin(alpha)) / 2);
 }

The "From" point has coordinates: (122.512 ; 39.8101) and the "To" point has coordinates:(237.0633 ; 154.3614)

What needs to be determined now is the distance between the "To" point and each rectangle. For that, we'll need to determine where the segment between "To" and the center of the rectangle intersects the bounds of the rectangle.

The line between "To" and the center intersects the bounds in found points at most. So we'll need to determine which one of these points is inside the bounds and between "To" and the center.

We start off by computing the line equation for the line that contains "To" and the center (Y = m * X + b). Then we find out the intersection by substituting the points on the bounds (e.g. Y = Top or X = Left + Width).

//Line equation between point and center
 double m = (cen.Y - src.Y) / (cen.X - src.X);
 double b = cen.Y - (cen.X * m);
 
 PointF[] pis = new PointF[4];
 pis[0] = new PointF((float)((bounds.Top - b) / m), bounds.Top);
 pis[1] = new PointF((float)(bounds.Left + bounds.Width), (float)((bounds.Left + bounds.Width) * m + b));
 pis[2] = new PointF((float)((bounds.Top + bounds.Height - b) / m), (float)(bounds.Top + bounds.Height));
 pis[3] = new PointF(bounds.Left, (float)(bounds.Left * m + b));
 sqdist = double.NaN;
 
 //Select the intersection with the bounds which is inside the bounds and between the two points
 for (int i = 0; i < 4; ++i)
 {
 
 if (IsInsideAndBetwen(pis[i], src, cen, bounds))
 {
 sqdist = Math.Pow(src.X - pis[i].X, 2) + Math.Pow(src.Y - pis[i].Y, 2);
 
 return;
 }
 }

The method IsInsideAndBetween(pi, src,cen, bounds) returns true if pi is between cen and src and if pi is inside the bounds.

 private bool IsInsideAndBetwen(PointF pi, PointF src, PointF cen, RectangleF bounds)
 {
 
 if (pi.X < bounds.Left ||
 pi.X > bounds.Left + bounds.Width ||
 pi.Y < bounds.Top ||
 pi.Y > bounds.Top + bounds.Height)
 return false; //It's outside bounds
 
 if (cen.X == pi.X)
 {
 if ((src.Y - cen.Y) / (pi.Y - cen.Y) < 1)
 return false;
 }
 
 else
 if (cen.Y == pi.Y)
 {
 if ((src.X - cen.X) / (pi.X - cen.X) < 1)
 return false;
 }
 else
 if ((src.X - cen.X) / (pi.X - cen.X) < 1 ||
 (src.Y - cen.Y) / (pi.Y - cen.Y) < 1)
 return false; //It's not between the two points.
 
 return true;
 }

Disclaimer: Sample Code is provided for the purpose of illustration only and is not intended to be
used in a production environment. THIS SAMPLE CODE AND ANY RELATED INFORMATION
ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR PURPOSE. We grant You a
nonexclusive, royalty-free right to use and modify the Sample Code and to
reproduce and distribute the object code form of the Sample Code, provided
that. You agree: (i) to not use Our name, logo, or trademarks to market Your
software product in which the Sample Code is embedded; (ii) to include a valid
copyright notice on Your software product in which the Sample Code is embedded;
and (iii) to indemnify, hold harmless, and defend Us and Our suppliers from and
against any claims or lawsuits, including attorneys’ fees, that arise or result
from the use or distribution of the Sample Code

Using DDE in C# for Office automation

Costin Boldisor — Tue, 26 Jul 2011 12:11:00 GMT

Looking for some code samples how to pass DDE messages from a .NET program I found out that there‘s a great library already in place that facilitates working with DDE: the NDde project from Codeplex: http://ndde.codeplex.com/

However, I just needed to make one call and including a new DLL into my code would have been too much.

So I just created a sample C# program derived from the NDde code that imports only the necessary functions from the Win32 API to pass an Open DDE message to Excel for opening a workbook.

Workbook wb = DDEHelper.OpenWorkbook(_excel, args[0]);

Compilation:

D:\Projects\DDE>csc /r:C:\Windows\assembly\GAC_MSIL\Microsoft.Office.Interop.Excel\14.0.0.0__71e9bce111e9429c\Microsoft.Office.Interop.Excel.dll DDEOpen.cs

Using .NET library in VBA

Costin Boldisor — Wed, 01 Jun 2011 14:56:00 GMT

Three easy steps for exporting .NET functionality to VBA:

1. Write a source file:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Runtime.InteropServices;

using System.Windows.Forms;



namespace InteropTest

{

    [Guid("BEB2C62D-5646-4B08-81DF-670314BFEE4A")]

    public class InteropTestClass

    {

        public void Help()

        {

            MessageBox.Show("Hello From .NET!");

        }



    }

}

The GUID needs to be unique.

2. Generate a strong key to sign the assembly:

sn -k MyKey.snk

3. Compile the C# file using CSC:

csc /target:library /out:tstint.dll MyClass.cs /keyfile:MyKey.snk

We now have a DLL that can be exported to another machine with the .NET Framework installed. On the target machnie, the assembly needs to be registered:

regasm tstint.dll /codebase /tlb:tstint.tlb

Now the library can be used in Office by selecting it from the list of references in VBA:

Recovering the pictures from a PPT file

Costin Boldisor — Fri, 13 May 2011 14:13:00 GMT

The PPT file format is very well documented as per Microsoft’s Open Specification Promise. However, accessing any part of a PPT file isn’t really a snap.

I’ve created a small C# program which parses the \Pictures stream from a PPT file. For that, one needs to:

Extract the Pictures stream from the PPT OLE storage
Parse the stream according to the OfficeArtFBSE format
Extract all the binary pictures according to format: WMF, EMF, PNG, JPG

This can be used for damaged PPT files where the shapes that contain pictures can’t be viewed anymore.

Complex editing of XLSX files

Costin Boldisor — Fri, 29 Apr 2011 09:21:00 GMT

Editing Excel OpenXML documents using the OpenXML 2.0 SDK offers better performace and improved supportability compared to normal Office automation (e.g. VBA). There are also great examples on the Internet to get started: Office Open XML Formats: Inserting Values into Excel 2007 Cells.

What I haven’t found until now is some clarifications as to how one can edit multiple Excel cells without corrupting the workbook. This is what I’m presenting in this blog post.

The fact is that whenever rows and cells need to be added to an Excel worksheet, they need to be sorted upon insertion. Take for instance the XML for the following worksheet where A1 = 1, B1 = 2 and B2 = 3. Here is the only way the XML should be saved like:

If we inserted say cell B1 before cell A1, Excel would show a corruption error when opening the XLSX.

The program that I’m attaching here does exactly this: the input is a worksheet, a row and a column and the output is a modified worksheet that contains the row and the cell required for that cell to be edited.

Other useful features:

- Convert column index to colum name (e.g. 28 to AB)

- Get the column index from cell reference (e.g. AB23 to 28)

- Selecting cells based on argument value using LINQ

Sample edited worksheet:

Note: The sample program does not create a blank workbook, you will need to save one from Excel and pass the full path and the name of the worksheet to the program.

Indenting XML files

Costin Boldisor — Thu, 17 Mar 2011 12:38:00 GMT

The operation of indenting XML files is pretty trivial when using the classes from the System.XML namespace. However, I often find myself trying to read canonical XML code from files. As such, a created a very simple tool that converts XML files to indented XML.

All you need to do is to drag and drop an XML file into the WPF window and there you have it!

And the difference:

List styles in Word document with preview

Costin Boldisor — Mon, 14 Mar 2011 15:58:00 GMT

The sample code from my previous post does a pretty good job listing the styles in a Word document. However, having a preview of these styles makes it a lot easier to spot the subtle differences than by using the Description object.

For generating a preview of the styles I used some code posted by L Webber on MSDN for manipulating clipboard. I’ve also added the functionality of finding out the clipboard format in which Word copies PNG and saving the content of the clipboard to a local file.

The result:

Note: please make sure to recompile the VBA proeject before running (simply open the VBE window) in order to avoid crashes.

Listing styles in Word document

Costin Boldisor — Fri, 25 Feb 2011 13:17:00 GMT

When dealing with Styles in Word you can never have too much control. This is when a listing of the existing styles and their descriptions helps a lot.

Happily VBA provides a powerful way of getting details about a certain style through the Description member of the Style class. Based on this I’ve built a small VBA script that generates an HTML list of the existing styles in the active document.

The number formatting for list styles is also included.

Brushing up on the WPF and Windows 7 Jump Lists

Costin Boldisor — Fri, 11 Feb 2011 15:48:00 GMT

This is just a summary on the new Windows 7 features and how these can be implemented in .NET. There is plenty documentation on both Windows Presentation Foundation and the Windows 7 SDK but actually implementing a solution is a bit harder.

So here we have it: a WPF application that create Jump Lists with shortcuts to other programs.

Programming with the Windows 7 APIs is described in good detail on the following MSDN magazine site: http://msdn.microsoft.com/en-us/magazine/dd942846.aspx

There is also a great .NET SDK for accessing the Windows 7 API: http://code.msdn.microsoft.com/WindowsAPICodePack

XML Highlight in RichTextBox

Costin Boldisor — Mon, 31 Jan 2011 11:20:00 GMT

Recently I’ve been looking for some piece of code that would perform XML highlighting in a Rich Text Box. The Internet provides several free samples and several commercial samples but I wasn’t really satisfied with what I found, mainly because it involved using an XMLTextReader object, which does not help for WYSWYG editing of the XML file.

So I created a simple C# function that formats the text content from a RichTextBox:

public static void HighlightRTF(RichTextBox rtb)

The complexity is roughly O( n ), where n is the number of characters in the XML file.

Here is the result:

You’ll find the function in the text attachment.

Create metafile from icon and label

Costin Boldisor — Wed, 22 Dec 2010 09:37:00 GMT

Going further on the Windows Metafile problematic from a previous post, I’ve created a sample code for illustration a pretty useful functionality from OLE.

What the OLE function implemented here does is to generate a metafile with the icon and label for a specific file:

The way to do this is:

1. Call SHGetFileInfo to get the HICON associated to a specific extension

2. Call OleMetafilePictFromIconAndLabel for generating a metafile from a label and an icon

3. Call CopyMetaFile to save the memory metafile to a WMF file

Attached you can find the CPP and EXE for this sample.

MSDN documentation for this function:

http://msdn.microsoft.com/en-us/library/ms683934(VS.85).aspx

DLL Injection

Costin Boldisor — Fri, 22 Oct 2010 11:54:00 GMT

In a previous post, I was discussing the idea of adding more events to Excel by adding a window hook.

Under the hood, the idea is that we can load a DLL in the Excel process simply by transforming that DLL into a COM add-in, which is automatically loaded by Excel.

This post covers the situation of programs that don’t implement an add-in mechanism so loading a DLL into their process requires DLL injection.

The topic of DLL injection has been covered in various other articles [1][2], so this post is mainly a summary and a sample of the basic mechanism for DLL injection.

We need to create a DLL that attaches a Windows hook when loaded. We then need to load this DLL through DLL injection in the address space of the target program.

So the basic steps are:

Create a DLL
In the DllMain method of the DLL attach a windows hook which will log some CBT messages
Start the target process
Allocate some memory in the address space of the target process
Write the path to the DLL in the allocated memory
Call the LoadLibrary function from the kernel32 library in the target process passing the allocated memory as an argument. This will load the library specified by the written path.

Notes:

Step 4: Uses VirtualAllocEx
Step 5: Uses WriteProcessMemory
Step 6: Uses CreateRemoteThread with the address of the LoadLibrary function taken from GetProcAddress

As mentioned in the following post, this is the main idea behind monitoring software such as Spy++ or other profiles.

A very cool implementation was done in: http://easyhook.codeplex.com/

Extra Events for Excel

Costin Boldisor — Mon, 06 Sep 2010 08:37:00 GMT

As a developer of Automation solutions for Microsoft Office one might feel the need of some extra events. Quoting from the MSDN VBA Object Model reference, here is the list of events from the Excel.Application object:

What if these events are not enough?

The best solution would obviously be if Microsoft extended the Object Model so as to include the needed events. This is pretty hard due to the large number of the possible application events (think for instance about the event “Sheet After Follow Hyperlink Success” – imaginary event triggered when a hyperlink has been clicked and the target web site can be reached)

My solution is to use Windows hooks [link to SetWindowsHookEx]. The idea is that many actions are triggered by Windows messages sent to or by windows in Office applications. I couldn’t find any public reference of the way Office applications react to windows messages but we can see this using the tool Spy++ that gets installed together with Visual Studio. Side note: the following blog post from the designer of Spy++ offers some insights about how the tool was built.

The most obvious message that can be tapped is the WM_ENABLE message sent received when a window gets enabled or disabled when a modal dialog is shown. This is not particular to Office applications, other applications have the same behavior (e.g. Notepad).

My first attempt was to intercept this message in VBA by basically adding a hook that calls a VBA function when a message is received. The problems with this approach are:

1) A hook of type WH_CALLWNDPROC is added for all messages reaching the Window’s main procedure, no only for WM_ENABLE. We’ll then have to check in the function what message was received and only care about WM_ENABLE

2) Invoking the VBA function is really slow. The approach actually works but, because the number of messages received by the window is enormous, Excel will basically hang because of the amount of VBA code it will need to execute.

As such, I decided to include this in a COM add-in, which obviously works a lot faster. Event with the Visual Studio Wizard, building a COM add-in directly from the IDTExtensibility2 interface is not really a piece of cake. Luckily there are some cool samples that helped me easily implement the Ribbon custom UI and the button click event handler:

Building a C++ Add-in for Outlook 2010

Using RibbonX with C++ and ATL

Having these samples as start point, I can go ahead and add a Window hook when the button gets pressed.

myHook = SetWindowsHookEx(WH_CALLWNDPROC,GetWinProc , (HINSTANCE)NULL, GetCurrentThreadId());

In the callback, I can simply check is a WM_ENABLE message was sent to the main Excel Application window:

LRESULT CALLBACK GetWinProc(

__in int code,

__in WPARAM wParam,

__in LPARAM lParam

){

if (code < 0) // do not process message

return CallNextHookEx(myHook, code,

wParam, lParam);

CWPSTRUCT *cwp = (CWPSTRUCT*)lParam;

if(cwp->message == WM_ENABLE){

if((long)(XLApp->Hwnd) == (long)(cwp->hwnd)){

if(cwp->wParam)

MessageBoxW(NULL,

L"Window enabled",

L"Message from ExcelHook",

MB_OK | MB_ICONINFORMATION);

else

MessageBoxW(NULL,

L"Window disabled",

L"Message from ExcelHook",

MB_OK | MB_ICONINFORMATION);

}

return CallNextHookEx(myHook, code,

wParam, lParam);

}

A sample usage scenario: closing the Print dialog. When I press CTRL-P the print dialog is opened but before that, I see a message from my add-in, “Window disabled”.

After closing the Print dialog, I see a message from the add-in: “Window enabled”.

Cool huh? :)

As future development, we have many possibilities as there are many other messages that can be intercepted . MSDN has the list of other windows messages we can intercept using the hooks: http://msdn.microsoft.com/en-us/library/ff468922(VS.85).aspx

EMF+ Untangled

Costin Boldisor — Mon, 23 Aug 2010 11:43:00 GMT

A common belief regarding the EMF format, at least as it appears by browsing technical blog posts, is that this format was never documented by Microsoft and that there are many unknown facts about it. This opinion is wrong, as the format has complete publicly available specifications posted on MSDN.

That being said, I do admit that the documentation for how EMF can be used is pretty scarce and, from a developer’s point of view, the expectations would be to have an SDK or something similar that would help parsing EMF and would create a bijection between the binary format and a class structure.

This post will not go as far as propose an SDK but will hopefully represent a first step into this direction.

First takeaway – listing EMF records and objects

The attached program (EMF Lister) will parse an EMF+ file and will list all the records from a binary perspective.

Achieving this is not really rocket-science, as we have a method from the Graphics class that enumerates the EMF records of a Graphics object.

Second takeaway – generating an EMF file

A very easy way demonstrated with this sample code is to create an EMF using the drawing methods for a Graphics object.

Let’s take the following record from the sample EMF file:

FillRects 49152(16 bytes) EE 82 EE FF 01 00 00 00 00 00 00 00 80 02 E0 01

This was generated from the C# method:

g.FillRectangle(Brushes.Violet, 0, 0, width, height);

(where width = 640, height = 480).

Checking the specifications for the FillRects record, we can see how the two link together:

EE 82 EE FF and the Flags: 49152 = define the Brush being used for filling rectangles.

01 00 00 00 = number of rectangles (one rectangle).

00 00, 00 00, 80 02/ 0x280, E0 01/ 0x1E0 = 0, 0, 640, 480 – the left, top, width and height of the rectangle.

For more details regarding how brushes and pens are saved, please check the EMF+ object specifications.

========================

Using the EMF Lister:

Usage: EMFLister.exe <EMF input> [TXT output]

for parsing EMF input

Or:

EMFLister.exe

for generating the sample.EMF file in the current folder

Additional information:

There is a very useful Codeplex project that (among other features) converts XAML to EMF: http://xamltoys.codeplex.com/

OLElsTool - Working with OLE storages

Costin Boldisor — Fri, 04 Jun 2010 06:21:30 GMT

I’m posting here a tool that can be used for viewing and modifying OLE storage.

The syntax is as follows:

Usage: OLElsTool.exe <File_In> <Options>

-L: list

-X <part> <toFile>: extract part

-M <part> <fromFile>: modify part

Sample usage: say we copy an Excel range and paste it into Word using Paste Special, as Link, Microsoft Excel Worksheet Object.

If we now save as DOC, the Excel object will be embedded as an EMF picture. Using this tool, we can view the contents of the OLE container using:

OLElsTool.exe Sample.doc -L > lst.txt

In the output file, we can see the field:

♥EPRINT

To extract it, use:

OLElsTool.exe Sample.doc –X ♥EPRINT picture.emf

Note: ♥EPRINT contains a non-alphanumeric character (0x03) that can be input by copy-pasting from the listing file (lst.txt).

Provided is the whole Visual Studio 2008 solution. It is written in Visual C++ using the Win32 API for working with OLE Storage.

OpenDoc tool unveiled

Costin Boldisor — Wed, 02 Jun 2010 07:42:00 GMT

After the previous blog posts with the OpenDoc tool, I’ve learned that, while this tool can be very helpful, it also raises a lot of questions mainly because it lacks proper documentation.

As such, I’ve decided to post the source code giving everybody who is interested the liberty of modifying it to suit their needs.

It goes without saying that this source code is provided “as-is” with no support or warranty from Microsoft. However, I am personally interested in any feedback you might have regarding this tool.

OpenDoc tool reloaded

Costin Boldisor — Fri, 02 Apr 2010 12:29:00 GMT

In a previous post, I described a tool for editing OpenXML documents from a managed context.

I've added three more features for some common tasks:

1. Remove a part

Syntax:

OpenDoc.exe -r <Input file> <Part>

Sample usage scenarios: removing the calculation chain, removing unwanted resources, removing slides, worksheets.

2. Remove all attributes

Syntax:

OpenDoc.exe -t <Input file> <Part> <XPath> removeAttibutes

Sample usage scenarios: fixing corrupt elements.

The attached archive shows a sample usage: repairing an Excel workbook by:

removing the calcChain.xml part
removing the calcChain relationship
removing the shared formulas from cell entries

The batch looks like this:

-t Corruptible.xlsm /xl/worksheets/sheet1.xml //n:sheetData/n:row/n:c/n:f[@t='shared'] removeAttributes
-r Corruptible.xlsm /xl/_rels/workbook.xml.rels //n:Relationship[@Target='calcChain.xml']
-r Corruptible.xlsm /xl/calcChain.xml

@Update 04/22/2010: Fixed but with long file names in batch mode

Working with OpenXML documents

Costin Boldisor — Mon, 08 Mar 2010 16:51:00 GMT

The default format for saving Excel, Word and PowerPoint documents has luckily changed since Office 2007 to the Open XML formats.

There is good documentation available for these formats on:

MSDN: Introduction to the Office 2007 file formats:

http://msdn.microsoft.com/en-us/library/aa338205.aspx

MSDN – sample code: Editing Excel cell values from .NET code:

http://msdn.microsoft.com/en-us/library/bb332058.aspx

ECMA: ECMA 376 Standard:

http://www.ecma-international.org/publications/standards/Ecma-376.htm

The following tool makes it very easy to view and modify Open XML documents.

The basic syntax is:

OpenDoc.exe <mode> <Input file> <Part> <XPath> [Value] [Attribute]

What the tools enables one to do is to select specific nodes from a part of an Open XML document and display its value and attribute, modify the value or attributes and to delete a node.

The easiest way to work with it is the batch mode:

OpenDoc.exe –b <Path to batch file>

A batch file is any text file that contains processing commands one per line.

Here is a sample batch file:

-sn Book1.xlsx /xl/worksheets/sheet1.xml //n:sheetData/n:row/n:c[@r='D1']/n:v

-m Book1.xlsx /xl/worksheets/sheet1.xml //n:sheetData/n:row/n:c[@r='D1']/n:v 20

-sn Book1.xlsx /xl/worksheets/sheet1.xml //n:sheetData/n:row/n:c[@r='D1']/n:v

-r Book1.xlsx /xl/worksheets/sheet1.xml //n:sheetData/n:row/n:c[@r='C1']

-sa Book1.xlsx /xl/workbook.xml //n:sheets/n:sheet[@name='Sheet2']

-m Book1.xlsx /xl/workbook.xml //n:sheets/n:sheet[@name='Sheet2'] MySheet name

What the six sample commands do is as follows:

- Display the XML of the node corresponding to address D1 from the XML part of the first worksheet from Book1.xlsx

- Modifies the value of this node to 20 (the cell D1 will show 20 now in Excel)

- Displays the same node again with the updated value

- Remove the XML node for cell C1

- Shows the attributes of sheet entry with the attribute ‘name’ equals ‘Sheet2’

- Modifies the value of attribute ‘name’ of the worksheet entry for Sheet2 with the value ‘MySheet’. If we open the workbook in Excel now, the name of the second sheet will be the one we’ve just assigned.

Command reference:

Command	Meaning
-sn; -showNode	Displays the XML content of the selected node
-sa; -showAtt	Displays the attributes of the selected node
-m; -modify	Modifies the Inner XML or the attribute value of the selection
-r; -remove	Removes the selected XML node

The XPath argument must evaluate to a legal XPath expression.

Example: //n:sheets/n:sheet[@name='Sheet2']

The reason why “n:” is used for qualifying every node name is linked to the XML namespaces that all the internal XML files inside an Open XML document are declared in. The namespace will be automatically resolved by the tool.

The download from the attachment contains the EXE, the sample XLSX and the sample batch file.

In order to run the program, the .NET Framework 3.5 is required.

External links in Excel files

Costin Boldisor — Fri, 05 Mar 2010 16:32:00 GMT

Topic

This is a post that contains a tool for analyzing the external links in an Excel Workbook.

The tool is intended to provide information regarding the linked workbooks, the cells from a workbook that contain external links, as well as the target worksheets and cells.

It will generate an XML report like this:

<?xml version="1.0"?>

<oldLink>C:\Test\Sites\XXXX.xlsm</oldLink>

<oldLink>C:\Test\Sites\YYYY.xlsm</oldLink>

<oldLink>C:\Test\Sites\ZZZZ.xlsm</oldLink>

<oldLink>C:\Test\Sites\TTTT.xlsm</oldLink>

</oldLinks>

…

The input workbook can be both XLS and XLSX/XLSM format (Office Automation is used).

You also have the option to replace the external links in a workbook by specifying a path where the target workbooks are located and then pressing the "Fix" button. This is very useful for changing the location of a workbook with lots of external links. (see http://support.microsoft.com/kb/328440)

To access all functionalities, open the file in Excel 2007 and choose the second tab from the ribbon. There is a single button there.

You can freely modify the VBA code (and I'll be very thankful if you provide feedback and/or bug reports).

Enjoy!

Download

Collecting crash dumps

Costin Boldisor — Mon, 01 Mar 2010 13:21:00 GMT

Introduction

Getting near to the cause of an application crash once one has a crash dump is one of the easiest debugging tasks. The main problem in these cases is how to actually acquire a crash dump. This post discusses some basic scenarios for getting a crash dump.

To test the basic scenarios, we’ll use an application where crash occur no so randomly as in real-life – the CrashMe application from http://windbg.info/.

All one needs to do to make this application crash is to press one of the buttons from the “Crash Actions” group like “Stack overflow” or “Division by NULL”.

Using the Windows error reporting mechanism

Windows has its own error reporting mechanism which detects when an application is crashing and attaches a debugger automatically. The details about this are provided in:

http://support.microsoft.com/kb/103861

In short: there is the registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug

The default value of this is: DRWTSN32 -p %ld -e %ld –g . This will change if you install other software, like, for instance, Visual Studio.

Using ADPlus from Debugging tools for Windows (CDB)

The Adplus script is very useful when one can easily reproduce a crash. This comes with the Debugging Tools for Windows (which include WinDBG, CDB and other debugging tools).

The newest version of Debugging Tools for Windows is part of the Windows Driver Kit 7.1.0, release 26^th February 2010: http://www.microsoft.com/whdc/DevTools/WDK/WDKpkg.mspx

For the old Adplus script, we just need the following steps:

a) Get the PID of the application that will crash using “tasklist” from Command Prompt or the PID column from Task Manager (Processes -> View -> Select columns…)

b) From Command Prompt, run:

cscript adplus.vbs –crash –p [PID]

This will attach the CDB debugger to the crashing process and will collect 1^st chance Mini Dumps and 2^nd chance full dumps for the attached process.

The dumps will normally be stored in the same folder where Adplus is installed into a special folder. The DMP files will have meaningful names like:

PID-3792__CRASHME.EXE__2nd_chance_IntegerDivide__full_0db0_2010-03-01_13-43-42-677_0ed0.dmp

Obviously, there are a lot of options available for Adplus. Just run it without any argument and you’ll see the whole range of options.

Note:The newest version of Debugging Tools for Windows contains an executable for Adplus as well as the old version Adplus_old.vbs

Using WinDBG

WinDBG provides more or less the same functionality as CDB with regards to collecting crash dumps. You can configure the exception handling in the debugger to break only for on 2^nd chance exceptions (check the WinDBG help).

To get a dump from WinDBG you will need to run the command .dump. Sample usage:

.dump /mfh /u C:\crash.dmp

The /u switch is very useful because it appends the date, time and PID to the name of the generated dump file.

What to do when the crashing process will be started in the future

An application crash is not always as easily reproducible as the ones from the CrashMe application. The special case when you know that a certain application will crash but don’t know under which circumstances can be handled in at least two ways:

a) Configuring the Windows Error Reporting to collect a customized dump when an application crashes. Here is a sample how to configure CDB to get dumps when any application crashes in Windows:

Set the following registry values of the key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug

Debugger = "C:\Debuggers\cdb.exe" -p %ld -e %ld -g -c "~*k;.dump /mfh /u C:\Dumps\crash.dmp;.kill;qd"

Auto = “1”

Notes:

-“C:\Debuggers\cdb.exe” is the path to the CDB debugger, which might vary depending on the installation path of Debugging Tools for Windows.

-%ld will be provided by Windows and represents the PID of the process where the 2^nd chance exception occurs.

- After the crash is collected, the crashing process is killed (this is specified by passing a custom command to CDB using the –c switch).

b) Monitoring for a certain process to start and to crash:

adplus.exe -crash -pmn notepad -o C:\Dumps

The parameter passed to adplus in –pmn is the name of the process to monitor and the parameter after –o is the directory where the DMP files will be collected.

What to do when the crashing process uses a custom resiliency mechanism

Microsoft Office uses a custom resiliency mechanism for handling exceptions. It automatically calls Doctor Watson and disregards the Windows Error Reporting.

To work around this, the easiest thing to do is to rename the registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PCHealth\

ErrorReporting\DW\Installed

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PCHealth\

ErrorReporting\DW\_Installed

This way Office will not be able to pass the exceptions to DW and Windows will be able to handle them. Please be careful with this change as it is not a supported scenario. The Office Dr. Watson buckets are very useful for collecting mini dumps from customers and identifying critical problems (so please click on Send next time you see a “Send”/”Don’t send” dialog popping up).

Create Smart Document solutions in .NET Framework 2.0

Costin Boldisor — Sun, 10 Jan 2010 15:06:00 GMT

Creating a managed Smart Document project or Add-in for Office 2003 is a bit of a headache because of the many configuration problems one might encounter.

Here is a short summary of the prerequisites of such a project on a target system:

What to install

Microsoft Office 2003 with the following features enabled:

Microsoft Office Excel -> .NET Programmability Support
Office Tools ->Microsoft Forms 2.0 .NET Programmability Support
Office Tools -> Smart Tag .NET Programmability

Why?

There are some DLLs in the following folder which are required for Office to be able to load managed solutions:

c:\Program Files\Microsoft Office\Office11

Explanation

Office 2003 uses the OTKLoadr.dll library to load Visual Studio customizations, smart documents and smart tags.

What to install

Office 2003 PIAs (Primary Interop Assemblies)

Why?

This will add the Smart Tag Library to the GAC

Explanation

Managed code that implements the ISmartDocument interface requires a reference to the Microsoft.Office.Interop.SmartTag namespace.

See: http://msdn.microsoft.com/en-us/library/aa195478(office.11).aspx

Download for Office 2003 PIA:

http://www.microsoft.com/downloads/details.aspx?familyid=3c9a983a-ac14-4125-8ba0-d36d67e0f4ad&displaylang=en

What to install

VSTOR 2005 SE

Why?

The VSTO runtime automatically installs the newest version of OTKLOADR.DLL and additionally adds the CLR Lockback registry key ([HKEY_CURRENT_USER\Software\Classes\Interface\{000C0601-0000-0000-C000-000000000046}]). This enables Office 2003 to run .NET 2.0 code, as initially Office 2003 could not do this.

Explanation

Download for VSTOR 2005 SE

http://www.microsoft.com/downloads/details.aspx?familyid=f5539a90-dc41-4792-8ef8-f4de62ff1e81&displaylang=en

Alternately, one can install the KB907417, which includes the newest version OTKLoadr, and set up the registry key manually.

What to install

Extensibility fix

Why?

Adds the Extensibility.dll to the GAC.

Explanation

This is included in the extensibilityMSM,msi file from KB 908002. http://support.microsoft.com/kb/908002

The IDTExtensibility2 interface is required for all add-ins and automation projects (it includes events like loading, unloading, updating)

http://msdn.microsoft.com/en-us/library/extensibility.idtextensibility2.aspx

What to install

.NET Framework Runtime 1.1

Why?

See the explanation for “FullTrust”

Explanation

Download:

http://www.microsoft.com/downloads/details.aspx?familyid=262D25E3-F589-4842-8157-034D1E7CF3A3&displaylang=en

What to install

.NET Framework 2.0

Why?

Obviously, the code is written in .NET 2.0 and requires the framework

Explanation

Download:

http://www.microsoft.com/downloads/details.aspx?familyid=0856EACB-4362-4B0D-8EDD-AAB15C5E04F5&displaylang=en

What to install

The installed DLL should be added to the “FullTrust” zone for both .NET 1.1 and .NET 2.0

Why?

Automation code that is executed via the OTKLoadr appears to follow the permission guidelines of .NET 1.1. However, as the solution most likely contains code whose call stack go through the .NET 2.0 Framework (e.g. a simple MessageBox.Show()) , the code needs to be trusted also by the .NET 2.0 Framework

Explanation

Add the DLL to the Full Trust zone using the CASPol tool from the .NET Framework:

C:\Windows\Microsoft.NET\Framework\v2.0.50727\ and

C:\Windows\Microsoft.NET\Framework\v1.1.4322\

CASPol.exe –ag My_Computer_Zone –url “C:\Users\[Username\Local Settings\Application Data\Microsoft\Schemas\[NameSpace]\[Name].DLL” FullTrust

Here is a sample Visual Studio 2008 project which creates a very basic Smart Document DLL in .NET 2.0. It also contains the manifest and a sample schema:

SmartDocument.ZIP

WinDBG Tutorial - Part 3

Costin Boldisor — Fri, 18 Sep 2009 18:02:00 GMT

Just short piece of information about Remote Debugging.

Remote Debugging actually means connecting the debugger on one computer to a process on another computer.

Typical scenario: PC 1 has a program that needs to be debugged. PC 2 does the debugging.

Advantages:

- PC 1 might not have symbols and sources for debugging (e.g. PC 1 is user of application, PC 2 is application vendor)

- PC 1 can be installed in a virtual machine

- PC 2 can debug kernel crashes

There are multiple ways of connecting the two PCs (which is normal, as multiple transport protocols are supported), but here is a quick-start for the most typical: via TCP/IP.

Steps:

1. Setup PC 1 as debugging Server:

If the WinDBG is already started on PC 1, open the Command window and write:

.server tcp:port=4001

If WinDBG is not opened, start it from the command prompt with the options:

\:>Windbg.exe -server tcp:port=4001

2. Connect PC 2 to PC 1:

Open WinDBG from command prompt with arguments:

\:>Windbg.exe windbg.exe -remote tcp:server=\\157.58.253.236,port=4001

Note: The value written in red are only examples. The IP address has to be the one of PC 1 and the port should be one of the free ports of PC 1.

3. Attach the debugger on PC 1 to a process, open a Dump or whatever. When a debugger is attached on PC 1, PC 2 will be able to debug that process as if it were a local process.

This is only one of the numerous ways of doing remote debugging. There is always the Help -> Search option in WinDBG that you can check for advanced options.