TSF Aware

In an earlier post on keyboards, I talked briefly about text service categories.  I'd like to talk more about categories.

TSF will make sure that at most one text service in any category is enabled at any given time.

So, for example, you can enable one text service with GUID_TFCAT_TIP_KEYBOARD, one text service with GUID_TFCAT_TIP_SPEECH, and one text service with GUID_TFCAT_TIP_HANDWRITING, and all three text services will be active at the same time.

But if you enable another text service with GUID_TFCAT_TIP_KEYBOARD, the first one will be deactivated. 

If you don't want this to happen, then you should call RegisterCategory with your own category GUID.

If you're building a text service DLL, you almost certainly don't want to use Visual Studio 2008's compiler.  The problem is that Visual Studio 2008 uses a new C Runtime Library, and if you build your text service with Visual Studio, your text service likely won't load in all applications.  (Plus, you would have to redistribute the C Runtime Library.)

What to do instead?

Well, I would recommend installing the Vista (or XP) DDK and use the DDKWizard instead.  The DDK comes with its own C/C++ compiler that uses the C Runtime Library that ships with the OS (and won't cause problems with other applications), and the DDKWizard will let you use all of Visual Studio's awesome capabilities.

I recently had two people ask me the same question:

"Why can't I insert more than one character into a composition on Notepad?"

It's actually a bit more complicated than that, since this behavior only appears to happen on Windows XP with a US English text service.  (Japanese text services appear to work correctly.)

I just had a chance to figure out what was going on here, and, since it doesn't seem to be documented anywhere else, I thought I'd post the answer so that the next poor sod doesn't have to spend eternity scratching his head wondering why he can't get it to work.

After a longish debugging bout, and a bunch of searching through the XP source code, I have the answer.

The answer is that there's a Text Event Sink attached to the TSF-unaware context, and when the context changes, this event sink looks to see if the changed text has the GUID_PROP_COMPOSING property attached to it.  If it doesn't, it terminates the composition.

So, if you want to have your text services insert more than one character into a composition, you need to make sure that your text has the GUID_PROP_COMPOSING property set to 1.

The code to do that would look like this:

BOOL CTextService::_SetCompositionComposing(TfEditCookie ec, ITfContext *pContext)
{
    ITfRange *pRangeComposition;
    ITfProperty *pComposingProperty;
    HRESULT hr;

    // the composition requires a range and the context it lives in
    if (_pComposition->GetRange(&pRangeComposition) != S_OK)
        return FALSE;

    hr = E_FAIL;

    // get our the display attribute property
    if (pContext->GetProperty(GUID_PROP_COMPOSING, &pComposingProperty) == S_OK)
    {
        VARIANT var;
        // set the value over the range
        var.vt = VT_I4;
        var.lVal = 1;

        hr = pComposingProperty->SetValue(ec, pRangeComposition, &var);

        pComposingProperty->Release();
    }

    pRangeComposition->Release();
    return (hr == S_OK);
}

When you're ready to finalize your composition, you should clear the property over the composed text by calling ITfProperty::Clear().

Note that none of the sample text services do this.

If you've tried to use the modified version of Scintilla that I described in my MSDN article, you will find that the zipped sources don't actually have the changes that I made.  That was my fault; when I was packaging the sources, I had two versions of ScintillaWin.cxx around, and I picked the newer one, which (sigh) was the wrong one.  Anyway, here's a link to the correct version of ScintillaWin.cxx with TSF support.

I got caught by this recently.  ITfCompartmentEventSink::OnChange means what it says.  If you repeatedly store the same value into a global compartment, the event sinks will not fire.  If you store a different value into the compartment, the event sinks fire just fine.

I've been working with compartments recently, and I've run across a few 'features' that tripped me up.  I figure if I've run across them, others have too.

Although MSDN says that you can put integers, BSTRs, and interface pointers into a compartment, you can not store interface pointers or strings into a global compartment (that you get from ITfThreadMgr::GetGlobalCompartment).  You can only store integers and empty variants.

Side note:  I would be very cautious about storing interface pointers into a (local) compartment, as well.  If an application has multiple UI threads (e.g., Explorer), TSF will load multiple instances of the text service (one instance per thread).  When each instance retrieves the interface pointer from an event sink and tries to call it, the interface pointer is properly marshalled across the apartment boundaries (COM marshals the IUnknown in the variant returned from ITfCompartment::GetValue), but you do have a proxy in the way, and calls across the interface pointer will be quite slow.

Text Services Framework assumes that your text service follows a particular processing path.  If your text service doesn't conform to these assumptions, then your programming job will be more complicated.  (Not impossible, just more complex.)  The text service samples on MSDN also follow these assumptions, but they aren't explicitly stated anywhere (that I know of).   I've mentioned some of these assumptions in previous articles, but I thought I'd bring them together in one post.

Text Services makes the following assumptions:

1. Your service must perform all changes to a context or range object within an edit session.  Text Services Framework enforces this assumption through the use of edit cookies.
2. Your service should not assume that it is possible to request a synchronous edit session.  (I discussed this here.)
3. Your service should track focus changes between applications and between controls within an application.  This means that your text service must install event sinks for ITfThreadFocusSink and ITfThreadMgrEventSink.
4. Your text service should use compositions to handle partially formed input. 

This last assumption is the big one.  It can cause problems for text services that aren't keyboard-related (speech, for example).

The problem is that TSF handles the (admittedly, very difficult) job of interacting with non-TSF aware applications entirely through compositions.  Once you close the composition, TSF assumes that you're completely finished with that piece of input.

Unfortunately, it's hard to tell beforehand when you're done with a piece of dictation.  SAPI will tell you when it's recognized a piece of text, obviously, but, ideally, once you've dictated some text, you would like to be able to correct it.  That requires that you leave the composition open.

In an application that isn't TSF-aware, though, you need to close that composition as soon as you can (it's bad form to have large open compositions; most IMEs have compositions that are a few characters in size).

So there's a tradeoff here.  Dictation in Windows Vista currently closes the composition as soon as the text is recognized.  (In fact, it doesn't use compositions at all.)   That works fine for TSF-aware applications, but causes problems with TSF-unaware applications.  In particular, once you've dictated some text, you can't correct it by voice.  That's why Windows Speech recognition makes you confirm every dictation into a TSF-unaware application. 

I received an interesting email the other day asking about how to get the character code from the parameters passed to the ITfKeyEventSink::OnKeyDown method.

The answer is that most keyboard related text services only work with a particular keyboard layout, and the text service manages the mapping from virtual key codes to character codes.

It is actually surprisingly difficult to write a keyboard-related text service that is keyboard-layout agnostic, as there isn't a public API that exposes the details of the keyboard layout.

"What about ToAscii/ToUnicode (and the related APIs)", you say?  Well, unfortunately, those APIs don't actually work across all languages.  In particular, they don't deal well with dead keys and obscure shift states (AltGr, SGCAPS, etc.), and, of course, fail utterly with languages like Japanese, Chinese, and Korean.  Michael Kaplan has an extensive set of articles about using ToUnicode (summarized here).

If you want to write a text service that is completely agnostic as to keyboard layouts, then, most likely, you probably don't want to use ITfKeyEventSink.  What can you use to monitor keyboard changes?

So far, the best recommendation I have is to use ITfTextEditSink::OnEndEdit, and if either of the following two conditions are true, then you can conclude that a keyboard made the changes: 

1) Call ITfEditRecord::GetTextAndPropertyUpdates to see if it contains a range with the property GUID_PROP_COMPOSING. TSF will set this property on text that is part of a composition.

2) Check if the context that you get from OnEndEdit has one or more ITfCompositionView objects within it.  Here's some sample code to demonstrate:

hr = pContext->QueryInterface(IID_ITfContextComposition,
                              (void **)&pContextComposition);
if (hr == S_OK)
{
    IEnumITfCompositionView *pEnumCompositionView;

    hr = pContextComposition->EnumCompositions(&pEnumCompositionView);
    if (hr == S_OK)
    {
        ITfCompositionView *pCompositionView;

        while (pEnumCompositionView->Next(1, &pCompositionView, NULL) == S_OK)
        {
            ITfRange *pRange;
            hr = pCompositionView->GetRange(&pRange);
            if (hr == S_OK)
            {
                // Do Stuff Here
                pRange->Release();
            }
            pCompositionView->Release();
        }
        pEnumCompositionView->Release();
    }
    pContextComposition->Release();
}

Another useful, but underutilized, group of functions in TSF are those relating to Input Scopes.   Input Scopes allow an application to define the sorts of things that are expected in this document (edit control, etc.).  For example, the Internet Explorer 7 address bar has an input scope of IS_URL | IS_DEFAULT | IS_ENUMSTRING - meaning that the address bar expects the input to be either a URL, one of the entries in its enumeration, or just plain anything (IE will default to searching for the string you typed). 

Input scopes were originally created for the Tablet PC, and a lot of the MSDN documentation is specifically related to the Tablet PC.  However, other input handlers (for example, speech) use input scopes as well to improve recognition.   For example, Windows Speech Recognition has a custom URL language model that is used when a control specifies the IS_URL input scope.  This significantly improves recognitions on URLs that are part of the language model.

The functions are very easy to use, if you have an hwnd for your control.  If you only want to set a single input scope (for example, IS_DIGITS for a numeric control), then SetInputScope is the function for you.  If you want to set multiple input scopes, or you want to use one of the more exotic input scopes like IS_PHRASELIST or IS_REGULAREXPRESSION (which return application-defined strings), then you will want to use SetInputScopes, which allows you to pass an array of input scopes, along with other data.  Finally, Windows Vista adds SetInputScopes2, which is a variant of the SetInputScopes function, but which takes a string enumerator object instead of an array of strings.

One thing to take note is that if you call SetInputScope (or one of its siblings), you need to call SetInputScope(hwnd, IS_DEFAULT) before the window is destroyed.  MSDN mentions this in the remarks; I want to emphasize it.

If your control doesn't have an hwnd, or if you want to have multiple input scopes per document, the situation is a bit more complicated.   MSDN isn't very clear about how to make input scopes work in TSF-aware applications.  In the description of the ITfInputScope interface, MSDN says

A TSF-aware application does not call SetInputScope directly, but rather implements either ITextStoreACP or ITfContextOwner to get a pointer to ITfInputScope.

What this bit means is that TSF-aware applications need to implement input scopes via an application property.  In particular, you need to implement ITextStoreACP::RequestSupportedAttrs, and ITextStoreACP::RetrieveRequestedAttrs.  If you want to have multiple input scopes per document, then you also need to implement ITextStoreACP::RequestAttrsAtPosition.  Note that the documentation refers to TS_ATTRIDs; these are typedefs for GUIDs!  The TS_ATTRID that you need to support for input scopes is GUID_PROP_INPUTSCOPE  (you may, of course, add support for other TS_ATTRIDs). 

Retrieving input scopes is fairly straightforward.  MSDN has a nice code snippet describing how to retrieve the ITfInputScope interface here; once you have it, use ITfInputScope::GetInputScopes to retrieve the list of input scopes, and the other methods return auxiliary data for exotic input scopes.

If you need the string enumerator (i.e., you want to support IS_ENUMSTRING), then you need to get the ITfInputScope2 interface, instead of ITfInputScope; all you need to do is QI the application property for IID_ITfInputScope2 instead of IID_ITfInputScope.   The MSDN documentation for ITfInputScope2 is incorrect; ITfInputScope2 extends ITfInputScope, not IUnknown (and it's only available on Windows Vista).   

The Input Scope enumerations are defined here. 

One interface that I hadn't paid much attention to in TSF is ITfContextKeyEventSink.  What does this let you do?  Why, it lets you inspect keyboard input for a particular context (or document).  This lets you do some really fun things, like redirect keyboard input from the current control and put it into another control.  An example of this can be seen in the sample text service with candidate list, which, while it doesn't really show how to use the ITfCandidateList interface (boo!), does show how to redirect keystrokes from the main document to the candidate window.  Most of the fun is in CandidateList.cpp.

In the function CCandidateList::_StartCandidateList, the service creates a second context for the document, pushes it onto the context stack, then installs a ITfContextKeyEventSink on the new context (and a layout sink on the original context, so that the candidate window can track layout changes).

Then, in the functions CCandidateList::OnTestKeyDown and CCandidateList::OnTestKeyUp, the service intercepts the incoming keystrokes, and the functions CCandidateList::OnKeyDown and CCandidateList::OnKeyUp redirect the keystrokes to the candidate window. 

I've received a few emails asking about debugging Text Services.  I figured if two people actually went to the trouble of sending an email, that there's enough demand for a post.

But first, a little diversion:  It turns out that Visual Studio 2005 SP1 ships with an updated version of the C Runtime library, and, by default, adds an entry to the application manifest to use that new library.  Unfortunately, this can cause serious problems for text services - in particular, the text service DLL often won't load.  I work around this problem by turning off manifest generation in the linker - either on the command line, with /manifest:no, or by going to the project property page, then selecting Configuration Properties/Linker/Manifest File in the tree view, and setting 'Generate Manifest' to 'No'.   In general, you should minimize your use of the C Runtime library anyway, as it's more stuff that gets loaded into every process.

Now, on to debugging.  There are two options:  tracing, and real debugging.  You can use OutputDebugString and DBMon in your text service to trace the flow of execution without any problems, but for serious problems, like hangs, or crashes, you'll need to use a debugger.  I've never been able to debug a text service running on the same machine as the debugger, mostly because the TSF Manager needs to use your (stopped) text service in order to give keyboard focus back to the debugger.  Instead, you will either need a second machine, or a virtual machine like Virtual PC.  (Microsoft gives this away for free, and also publishes trial virtual disks that you can use for evaluation.)

Once you have installed your new text service on your other machine, you can use remote debugging to debug problems in your text service.  I've used the visual studio debugger with great success, but it can have some problems, particularly when you need to debug an application that needs to run elevated on Windows Vista.  In those situations, I use cdb and windbg (cdb on the test machine, and windbg on the local machine).  

To be honest, I've gotten used to the power of windbg (and cdb), and I use that almost exclusively.  But windbg isn't the friendliest debugger in the world, so I want people to know that visual studio works just fine.

Once you have your debugger set up, you can launch a process on your other machine (e.g., wordpad, notepad, Microsoft Word, etc.), and set a breakpoint at your dll entry point (or at the entry point of your choice, for example, CMyTextService::Activate).   Visual studio's syntax for setting a breakpoint in a specific DLL is a little baroque (e.g., {,,mydll}MyFunction) , and I find windbg's syntax (e.g., mydll!MyFunction) a little easier to use.

I don't have anything better to blog about right now (I'm in the middle of reviews and designs), so I thought I would share a bit of whimsy - notably, this steampunk version of Star Trek. If you're familiar with A Trip to the Moon, you will find this deeply funny.  Hat tip to BT.

(If you do have suggestions, please contact me.)

Another thing I didn't talk about in my article was how to make sure your rich text edit controls are based on RichEdit 4.1 (which has TSF support). 

You need to do two things:

1)  change your window class name from RICHEDIT_CLASS to MSFTEDIT_CLASS, and

2)  LoadLibrary("msftedit.dll") instead of LoadLibrary("richedit.dll").

That's it!

In my article in MSDN, I mention that there are some easy ways to enable dictation support in controls that don't normally support dictation.  All the methods I described assume that the control has a window.  There is a way to enable dictation support for windowless rich text edit controls, assuming that you are using the RichEdit 4.1 implementation of ITextServices. 

I did not include this method in my article because I didn't want to include all the details about ITextServices, how you get an instance, and so forth.  I assume that if you already have an existing windowless rich edit control that uses ITextServices, you already know all that.  (If you don't, there's a quick primer available at CodeProject; note that I haven't actually used this code, and cannot vouch for it.)

Anyway, if you have an ITextServices interface pointer, then you can enable TSF support (and, by extension, dictation support), by calling ITextServices::TxSendMessage(EM_SETEDITSTYLE, SES_USECTF, SES_USECTF).

I hope that more richedit users will enable TSF support, so that users will have an easier time using dictation.

You might have noticed that I haven't really talked about text stores on this blog.  That's because I've been working on an article for MSDN Magazine on just that very subject.  It's in the July 2007 issue, and I just got my copy in the mail, so if you're a subscriber, you can read all the gory details about how to add TSF support to your controls and applications, complete with a working example.

 The article isn't currently available online yet - I'll add a link to the article as soon as it's available.

Update:  The article is now online, and you can find it here.