Jeffrey Wall's WebLog

ESE links

Jeffrey Wall — Mon, 05 Nov 2007 20:47:00 GMT

ESE API at MSDN: http://msdn2.microsoft.com/en-us/library/ms684493.aspx

New features in ESE for Vista/Server 2008: http://msdn2.microsoft.com/en-us/library/aa964797.aspx

ESE overview http://technet.microsoft.com/en-us/library/aa998171.aspx

Wikipedia article: http://en.wikipedia.org/wiki/Extensible_Storage_Engine

SQL Server 2005 Compact Edition Overview

Jeffrey Wall — Mon, 05 Nov 2007 20:40:00 GMT

A good overview of SQL Server 2005 Compact Edition

http://msdn2.microsoft.com/en-us/library/bb380177.aspx

Windows Contacts viewer in MCE

Jeffrey Wall — Mon, 11 Dec 2006 20:04:00 GMT

Mobileware.net has released a contacts viewer for MCE:
http://mobilewares.spaces.live.com/Blog/cns!78533A1A2E078194!163.entry

Some screen shots:
http://www.mobilewares.net/mce/bscscreens.htm

CLSID_OEEnvelope

Jeffrey Wall — Thu, 02 Nov 2006 18:23:00 GMT

Several 2007 Microsoft Office applications provide an API that lets a mail client interface with 2007 Office applications when the API is implemented by the mail client. This API provides an integrated experience for sending a copy of a 2007 Office document. If the default mail client implements the API, you can use the Microsoft Office Outlook 2007 mail envelope experience directly in 2007 Office applications.

http://support.microsoft.com/kb/926453

Windows Mail Channel 9 video

Jeffrey Wall — Mon, 24 Apr 2006 19:00:00 GMT

Windows Mail video link:
http://channel9.msdn.com/Showpost.aspx?postid=116711

Windows Contacts API

Jeffrey Wall — Mon, 24 Apr 2006 18:50:00 GMT

Windows Vista includes significant changes to the Windows Address Book.

The WAB MAPI APIs are still supported:
http://windowssdk.msdn.microsoft.com/library/default.asp?url=/library/en-us/WinContacts/wincontacts/wc_entry_point.asp

And there are new APIs in the SDK:
http://windowssdk.msdn.microsoft.com/library/default.asp?url=/library/en-us/WinContacts/wincontacts/wc_entry_point.asp

Double Buffer

Jeffrey Wall — Wed, 22 Sep 2004 19:27:00 GMT

In a project planning meeting recently, I heard a coworker say "Double Buffer". I figured that he meant a performance technique for animation or networking:

http://www.codeproject.com/java/javadoublebuffer.asp

or like:

http://www.usenix.org/publications/library/proceedings/osdi99/full_papers/pai/pai_html/node20.html

But he actually meant doubling the time estimate for each task in our project. I couldn't stop myself from immediately suggesting that he read the book 'Critical Chain':

http://www.amazon.com/exec/obidos/tg/detail/-/0884271536/104-9443640-3586327

Or for a quicker overview:

http://www.processimpact.com/articles/rainyday.pdf

Defrag API C# wrappers

Jeffrey Wall — Tue, 14 Sep 2004 01:15:00 GMT

For my own amusement, I decided to wrap the NT Defrag APIs in C#.

It isn't great, but... I'll give it the classic dev guarantee of "it works on my machine". http://www.thenetworkadministrator.com/top20.htm

The MSDN documentation uses lots of very scary words like "IOCTL" and "DeviceIoControl" and the such. Hopefully, I've abstracted some of that away...

First, you can get a bitmap of the cluster usage on the device...

static public BitArray GetVolumeMap(string DeviceName)

Then you can get a map of a file - virtual clusters and logical clusters

static public Array GetFileMap(string path)

Finally, you can move sections of a file around on a volume...

static public void MoveFile(string deviceName, string path, Int64 VCN, Int64 LCN, Int32 count)

Here's the whole thing, nearly guaranteed to break your file system if used. :-)

//
// a set of simple C# wrappers over the NT Defragmenter APIs
//

//
// Refrences

//
// http://www.sysinternals.com/ntw2k/info/defrag.shtml
//
// msdn how-to
// ms-help://MS.MSDNQTR.2004JUL.1033/fileio/base/defragmenting_files.htm
// http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/defragmenting_files.asp
//
// FSCTL_GET_VOLUME_BITMAP
// http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/fsctl_get_volume_bitmap.asp
//
// interesting structures...
// FSCTL_MOVE_FILE
// FSCTL_GET_RETRIEVAL_POINTERS
// RETRIEVAL_POINTERS_BUFFER
// FSCTL_GET_RETRIEVAL_POINTERS
//
// DeviceIoControl
// http://msdn.microsoft.com/library/default.asp?url=/library/en-us/devio/base/deviceiocontrol.asp
//

using System;
using System.Diagnostics;
using System.Collections;
using System.Runtime.InteropServices;

namespace defraglib
{
public class IOWrapper
{

        //
        // CreateFile constants
        //
        const uint FILE_SHARE_READ = 0x00000001;
        const uint FILE_SHARE_WRITE = 0x00000002;
        const uint FILE_SHARE_DELETE = 0x00000004;
        const uint OPEN_EXISTING = 3;

const uint GENERIC_READ = (0x80000000);
const uint GENERIC_WRITE = (0x40000000);

        const uint FILE_FLAG_NO_BUFFERING = 0x20000000;
        const uint FILE_READ_ATTRIBUTES = (0x0080);
        const uint FILE_WRITE_ATTRIBUTES = 0x0100;
        const uint ERROR_INSUFFICIENT_BUFFER = 122;

        [DllImport("kernel32.dll", SetLastError = true)]
        static extern IntPtr CreateFile(
            string lpFileName,
            uint dwDesiredAccess,
            uint dwShareMode,
            IntPtr lpSecurityAttributes,
            uint dwCreationDisposition,
            uint dwFlagsAndAttributes,
            IntPtr hTemplateFile);

[DllImport("kernel32.dll", SetLastError = true)]
static extern int CloseHandle(IntPtr hObject);

        [DllImport("kernel32.dll", SetLastError = true)]
        static extern bool DeviceIoControl(
            IntPtr hDevice,
            uint dwIoControlCode,
            IntPtr lpInBuffer,
            uint nInBufferSize,
            [Out] IntPtr lpOutBuffer,
            uint nOutBufferSize,
            ref uint lpBytesReturned,
            IntPtr lpOverlapped);

        static private IntPtr OpenVolume(string DeviceName)
        {
            IntPtr hDevice;
            hDevice = CreateFile(
                @"\\.\" + DeviceName,
                GENERIC_READ | GENERIC_WRITE,
                FILE_SHARE_WRITE,
                IntPtr.Zero,
                OPEN_EXISTING,
                0,
                IntPtr.Zero);
            if ((int)hDevice == -1)
            {
                throw new Exception(Marshal.GetLastWin32Error().ToString());
            }
            return hDevice;
        }

        static private IntPtr OpenFile(string path)
        {
            IntPtr hFile;
            hFile = CreateFile(
                        path,
                        FILE_READ_ATTRIBUTES | FILE_WRITE_ATTRIBUTES,
                        FILE_SHARE_READ | FILE_SHARE_WRITE,
                        IntPtr.Zero,
                        OPEN_EXISTING,
                        0,
                        IntPtr.Zero);
            if ((int)hFile == -1)
            {
                throw new Exception(Marshal.GetLastWin32Error().ToString());
            }
            return hFile;
        }

        /// <summary>
        /// Get cluster usage for a device
        /// </summary>
        /// <param name="DeviceName">use "c:"</param>
        /// <returns>a bitarray for each cluster</returns>
        static public BitArray GetVolumeMap(string DeviceName)
        {
            IntPtr pAlloc = IntPtr.Zero;
            IntPtr hDevice = IntPtr.Zero;

            try
            {
                hDevice = OpenVolume(DeviceName);

Int64 i64 = 0;

GCHandle handle = GCHandle.Alloc(i64, GCHandleType.Pinned);
IntPtr p = handle.AddrOfPinnedObject();

                // alloc off more than enough for my machine
                // 64 megs == 67108864 bytes == 536870912 bits == cluster count
                // NTFS 4k clusters == 2147483648 k of storage == 2097152 megs == 2048 gig disk storage
                uint q = 1024 * 1024 * 64; // 1024 bytes == 1k * 1024 == 1 meg * 64 == 64 megs

                uint size = 0;
                pAlloc = Marshal.AllocHGlobal((int)q);
                IntPtr pDest = pAlloc;

                bool fResult = DeviceIoControl(
                    hDevice,
                    FSConstants.FSCTL_GET_VOLUME_BITMAP,
                    p,
                    (uint)Marshal.SizeOf(i64),
                    pDest,
                    q,
                    ref size,
                    IntPtr.Zero);

                if (!fResult)
                {
                    throw new Exception(Marshal.GetLastWin32Error().ToString());
                }
                handle.Free();

                /*
                object returned was...
          typedef struct
         {
          LARGE_INTEGER StartingLcn;
           LARGE_INTEGER BitmapSize;
           BYTE Buffer[1];
         } VOLUME_BITMAP_BUFFER, *PVOLUME_BITMAP_BUFFER;
                */
                Int64 StartingLcn = (Int64)Marshal.PtrToStructure(pDest, typeof(Int64));

Debug.Assert(StartingLcn == 0);

pDest = (IntPtr)((Int64)pDest + 8);
Int64 BitmapSize = (Int64)Marshal.PtrToStructure(pDest, typeof(Int64));

Int32 byteSize = (int)(BitmapSize / 8);
byteSize++; // round up - even with no remainder

IntPtr BitmapBegin = (IntPtr)((Int64)pDest + 8);

byte[] byteArr = new byte[byteSize];

Marshal.Copy(BitmapBegin, byteArr, 0, (Int32)byteSize);

                BitArray retVal = new BitArray(byteArr);
                retVal.Length = (int)BitmapSize; // truncate to exact cluster count
                return retVal;
            }
            finally
            {
                CloseHandle(hDevice);
                hDevice = IntPtr.Zero;

                Marshal.FreeHGlobal(pAlloc);
                pAlloc = IntPtr.Zero;
            }
        }

        /// <summary>
        /// returns a 2*number of extents array -
        /// the vcn and the lcn as pairs
        /// </summary>
        /// <param name="path">file to get the map for ex: "c:\windows\explorer.exe" </param>
        /// <returns>An array of [virtual cluster, physical cluster]</returns>
        static public Array GetFileMap(string path)
        {
            IntPtr hFile = IntPtr.Zero;
            IntPtr pAlloc = IntPtr.Zero;

            try
            {
                hFile = OpenFile(path);

Int64 i64 = 0;

GCHandle handle = GCHandle.Alloc(i64, GCHandleType.Pinned);
IntPtr p = handle.AddrOfPinnedObject();

uint q = 1024 * 1024 * 64; // 1024 bytes == 1k * 1024 == 1 meg * 64 == 64 megs

                uint size = 0;
                pAlloc = Marshal.AllocHGlobal((int)q);
                IntPtr pDest = pAlloc;
                bool fResult = DeviceIoControl(
                    hFile,
                    FSConstants.FSCTL_GET_RETRIEVAL_POINTERS,
                    p,
                    (uint)Marshal.SizeOf(i64),
                    pDest,
                    q,
                    ref size,
                    IntPtr.Zero);

                if (!fResult)
                {
                    throw new Exception(Marshal.GetLastWin32Error().ToString());
                }

handle.Free();

                /*
                returned back one of...
     typedef struct RETRIEVAL_POINTERS_BUFFER {
     DWORD ExtentCount;
     LARGE_INTEGER StartingVcn;
     struct {
         LARGE_INTEGER NextVcn;
      LARGE_INTEGER Lcn;
        } Extents[1];
     } RETRIEVAL_POINTERS_BUFFER, *PRETRIEVAL_POINTERS_BUFFER;
    */

Int32 ExtentCount = (Int32)Marshal.PtrToStructure(pDest, typeof(Int32));

pDest = (IntPtr)((Int64)pDest + 4);

Int64 StartingVcn = (Int64)Marshal.PtrToStructure(pDest, typeof(Int64));

Debug.Assert(StartingVcn == 0);

pDest = (IntPtr)((Int64)pDest + 8);

// now pDest points at an array of pairs of Int64s.

Array retVal = Array.CreateInstance(typeof(Int64), new int[2] { ExtentCount, 2 });

                for (int i = 0; i < ExtentCount; i++)
                {
                    for (int j = 0; j < 2; j++)
                    {
                        Int64 v = (Int64)Marshal.PtrToStructure(pDest, typeof(Int64));
                        retVal.SetValue(v, new int[2] { i, j });
                        pDest = (IntPtr)((Int64)pDest + 8);
                    }
                }

                return retVal;
            }
            finally
            {
                CloseHandle(hFile);
                hFile = IntPtr.Zero;

                Marshal.FreeHGlobal(pAlloc);
                pAlloc = IntPtr.Zero;
            }
        }

        /// <summary>
        /// input structure for use in MoveFile
        /// </summary>
        private struct MoveFileData
        {
            public IntPtr hFile;
            public Int64 StartingVCN;
            public Int64 StartingLCN;
            public Int32 ClusterCount;
        }

        /// <summary>
        /// move a virtual cluster for a file to a logical cluster on disk, repeat for count clusters
        /// </summary>
        /// <param name="deviceName">device to move on"c:"</param>
        /// <param name="path">file to muck with "c:\windows\explorer.exe"</param>
        /// <param name="VCN">cluster number in file to move</param>
        /// <param name="LCN">cluster on disk to move to</param>
        /// <param name="count">for how many clusters</param>
        static public void MoveFile(string deviceName, string path, Int64 VCN, Int64 LCN, Int32 count)
        {
            IntPtr hVol = IntPtr.Zero;
            IntPtr hFile = IntPtr.Zero;
            try
            {
                hVol = OpenVolume(deviceName);

hFile = OpenFile(path);

                MoveFileData mfd = new MoveFileData();
                mfd.hFile = hFile;
                mfd.StartingVCN = VCN;
                mfd.StartingLCN = LCN;
                mfd.ClusterCount = count;

                GCHandle handle = GCHandle.Alloc(mfd, GCHandleType.Pinned);
                IntPtr p = handle.AddrOfPinnedObject();
                uint bufSize = (uint)Marshal.SizeOf(mfd);
                uint size = 0;

                bool fResult = DeviceIoControl(
                    hVol,
                    FSConstants.FSCTL_MOVE_FILE,
                    p,
                    bufSize,
                    IntPtr.Zero, // no output data from this FSCTL
                    0,
                    ref size,
                    IntPtr.Zero);

handle.Free();

                if (!fResult)
                {
                    throw new Exception(Marshal.GetLastWin32Error().ToString());
                }
            }
            finally
            {
                CloseHandle(hVol);
                CloseHandle(hFile);
            }
        }
    }

    /// <summary>
    /// constants lifted from winioctl.h from platform sdk
    /// </summary>
    internal class FSConstants
    {
        const uint FILE_DEVICE_FILE_SYSTEM = 0x00000009;

const uint METHOD_NEITHER = 3;
const uint METHOD_BUFFERED = 0;

const uint FILE_ANY_ACCESS = 0;
const uint FILE_SPECIAL_ACCESS = FILE_ANY_ACCESS;

        public static uint FSCTL_GET_VOLUME_BITMAP = CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 27, METHOD_NEITHER, FILE_ANY_ACCESS);
        public static uint FSCTL_GET_RETRIEVAL_POINTERS = CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 28, METHOD_NEITHER, FILE_ANY_ACCESS);
        public static uint FSCTL_MOVE_FILE = CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 29, METHOD_BUFFERED, FILE_SPECIAL_ACCESS);

        static uint CTL_CODE(uint DeviceType, uint Function, uint Method, uint Access)
        {
            return ((DeviceType) << 16) | ((Access) << 14) | ((Function) << 2) | (Method);
        }
    }

}

Defrag APIs

Jeffrey Wall — Tue, 14 Sep 2004 00:36:00 GMT

I just recently re-discovered the NT file defrag APIs...

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/defragmenting_files.asp

And I found a great defragmenter...

http://www.whitneyfamily.org/Hacks/?item=Defrag

Longhorn Communications Wiki

Jeffrey Wall — Mon, 14 Jun 2004 21:08:00 GMT

One of the PMs on the team has put together a wiki on Channel9

http://channel9.msdn.com/wiki/default.aspx/Channel9.LonghornCommunications

Sort communications by person

Jeffrey Wall — Thu, 06 May 2004 17:40:00 GMT

From: http://www.pcworld.com/news/article/0,aid,116005,00.asp

Future OS will track all your contacts regardless of the type of communication

Chief among Longhorn's capabilities is a communications history from which you can manage everything, says Paul Erickson, development manager for Windows communications. From this central location, you'll be able to sort and filter messages by various themes such as name of sender or date of conversation, he says.

For example, every time you get an e-mail from a particular person, a copy of that message ends up in the history file under the person's name. The next time you prepare to communicate with that person, Longhorn will automatically call up your message history. Later, if you use other methods to communicate with the same person, the OS will tie the resulting messages together so you can see all of them.

Communication History RSS Longhorn sample

Jeffrey Wall — Thu, 06 May 2004 00:37:00 GMT

The communication history team build a small sample RSS application for the WinHec build.

For each entry in the RSS feed the sample creates a Message WinFS item that appears in communication history.

http://longhorn.msdn.microsoft.com/lhsdk/sampledocs/desktopprogramming/shell_connect_commhistoryrss.aspx

Communication History integrated with caller ID

Jeffrey Wall — Thu, 06 May 2004 00:25:00 GMT

http://www.microsoft.com/presspass/features/2004/may04/05-03WinHec.asp

In addition to the hardware, the software was integrated with the process so that users could get rich caller information such as their communications history with the caller (for example, via a pop-up window on-screen with all the voicemail and e-mail history from that person), based on the caller ID. While some of the elements themselves weren't new, "Athens" was unique in bringing together the software, hardware, peripherals and the design to pull everything together.

Communication History at WinHec questions

Jeffrey Wall — Wed, 05 May 2004 20:38:00 GMT

The dev manager on my team did a presentation at WinHec today. The presentation covered some of the work that we are doing around the communications experience in Longhorn.

In the Q&A there were some good questions asked about Communication History that were well answered...

Q: Given the functionality around Communication History, how does this relate to Outlook and Outlook Express?

A: First and foremost, we are providing a platform for communication applications to write to a common store using common schemas. This simplifies the job for ISVs – as they don’t have to all build separate stores, schemas and APIs – they’re built-in in Longhorn. Communication history is the default UX to view logs about a person. And as the data is available to all apps, future versions of Outlook Express, Outlook and other communication apps can expose not only mail messages but voicemails, telephone logs, faxes, IM logs, etc. as well.

Q: How does data in Communication history work with Exchange?

A: Communication history shows data aggregated from multiple sources – data from mail servers sync’ed to WinFS would be one of those sources.

Q: Does Communication history meet archiving requirements for financial services industry?

A: We can’t speculate on whether the default Windows UX will meet the stringent logging requirements of the financial services or military customers. This is a platform and all the right hooks (WinFS APIs) are available for ISVs to build solutions that do logging.