TSF Aware : Text Services

Why you need to actually test instead of just looking at the source code

Eric Brown — Fri, 15 May 2009 21:43:12 GMT

In a previous post, I mentioned that you need to set the GUID_PROP_COMPOSING property across text in a composition, or else Windows XP will terminate the composition.

That’s true.

I then provided a code snippet to set that property.

That snippet is wrong.

It turns out that you can’t set the GUID_PROP_COMPOSING property via ITfProperty::SetValue. If you try, you’ll get a TF_E_READONLY error (even if you are in a read-write edit session), which can cause great consternation.

Instead, you need to call ITfComposition::ShiftStart and/or ITfComposition::ShiftEnd to set the bounds of the composition after changing the text in your composition.

These calls update the GUID_PROP_COMPOSING property internally.

Sorry for the confusion.

Where are the TSF Samples?

Eric Brown — Wed, 22 Oct 2008 23:57:41 GMT

Many people have emailed me asking about the TSF samples on MSDN. They’re supposed to be on MSDN code gallery, although they don’t appear to be there.

They are, however, part of the Windows SDK. After installation, you'll find them in %programfiles%\Microsoft SDKs\Windows\v6.1\Samples\winui\Input\tsf.

There are actually more samples in the Windows SDK than were on MSDN, including some examples of how to write a text store, as well as how to interact with TSF in UILess mode.

Visual Studio 2008 issues

Eric Brown — Fri, 18 Jan 2008 01:52:49 GMT

If you're building a text service DLL, you almost certainly don't want to use Visual Studio 2008's compiler. The problem is that Visual Studio 2008 uses a new C Runtime Library, and if you build your text service with Visual Studio, your text service likely won't load in all applications. (Plus, you would have to redistribute the C Runtime Library.)

What to do instead?

Well, I would recommend installing the Vista (or XP) DDK and use the DDKWizard instead. The DDK comes with its own C/C++ compiler that uses the C Runtime Library that ships with the OS (and won't cause problems with other applications), and the DDKWizard will let you use all of Visual Studio's awesome capabilities.

Things the Documentation left out, part N

Eric Brown — Sat, 22 Dec 2007 18:00:00 GMT

I recently had two people ask me the same question:

"Why can't I insert more than one character into a composition on Notepad?"

It's actually a bit more complicated than that, since this behavior only appears to happen on Windows XP with a US English text service. (Japanese text services appear to work correctly.)

I just had a chance to figure out what was going on here, and, since it doesn't seem to be documented anywhere else, I thought I'd post the answer so that the next poor sod doesn't have to spend eternity scratching his head wondering why he can't get it to work.

After a longish debugging bout, and a bunch of searching through the XP source code, I have the answer.

The answer is that there's a Text Event Sink attached to the TSF-unaware context, and when the context changes, this event sink looks to see if the changed text has the GUID_PROP_COMPOSING property attached to it. If it doesn't, it terminates the composition.

So, if you want to have your text services insert more than one character into a composition, you need to make sure that your text has the GUID_PROP_COMPOSING property set to 1.

I had previously written some code to manage this property. However, you cannot explicitly set the GUID_PROP_COMPOSING property -

ITfProperty::SetValue() will return TF_E_READONLY if you do.

What you do need to do is use the ITfComposition::ShiftStart and ITfComposition::ShiftEnd methods to move the start and ending points of your composition to cover the text. These methods will update the GUID_PROP_COMPOSING property directly.

~~The code to do that would look like this:~~

BOOL CTextService::_SetCompositionComposing(TfEditCookie ec, ITfContext *pContext)
{
    ITfRange *pRangeComposition;
    ITfProperty *pComposingProperty;
    HRESULT hr;

    // the composition requires a range and the context it lives in
    if (_pComposition->GetRange(&pRangeComposition) != S_OK)
        return FALSE;

~~hr = E_FAIL;~~

    // get our the display attribute property
    if (pContext->GetProperty(GUID_PROP_COMPOSING, &pComposingProperty) == S_OK)
    {
        VARIANT var;
        // set the value over the range
        var.vt = VT_I4;
        var.lVal = 1;

~~hr = pComposingProperty->SetValue(ec, pRangeComposition, &var);~~

pComposingProperty->Release();
}

pRangeComposition->Release();
return (hr == S_OK);
}

~~When you're ready to finalize your composition, you should clear the property over the composed text by calling ITfProperty::Clear().~~

Note that none of the sample text services do this.

ITfCompartmentEventSink::OnChange means what it says...

Eric Brown — Mon, 05 Nov 2007 18:00:00 GMT

I got caught by this recently. ITfCompartmentEventSink::OnChange means what it says. If you repeatedly store the same value into a global compartment, the event sinks will not fire. If you store a different value into the compartment, the event sinks fire just fine.

Compartment Values

Eric Brown — Fri, 02 Nov 2007 17:00:05 GMT

I've been working with compartments recently, and I've run across a few 'features' that tripped me up. I figure if I've run across them, others have too.

Although MSDN says that you can put integers, BSTRs, and interface pointers into a compartment, you can not store interface pointers or strings into a global compartment (that you get from ITfThreadMgr::GetGlobalCompartment). You can only store integers and empty variants.

Side note: I would be very cautious about storing interface pointers into a (local) compartment, as well. If an application has multiple UI threads (e.g., Explorer), TSF will load multiple instances of the text service (one instance per thread). When each instance retrieves the interface pointer from an event sink and tries to call it, the interface pointer is properly marshalled across the apartment boundaries (COM marshals the IUnknown in the variant returned from ITfCompartment::GetValue), but you do have a proxy in the way, and calls across the interface pointer will be quite slow.

The usual path

Eric Brown — Mon, 08 Oct 2007 17:00:01 GMT

Text Services Framework assumes that your text service follows a particular processing path. If your text service doesn't conform to these assumptions, then your programming job will be more complicated. (Not impossible, just more complex.) The text service samples on MSDN also follow these assumptions, but they aren't explicitly stated anywhere (that I know of). I've mentioned some of these assumptions in previous articles, but I thought I'd bring them together in one post.

Text Services makes the following assumptions:

Your service must perform all changes to a context or range object within an edit session. Text Services Framework enforces this assumption through the use of edit cookies.
Your service should not assume that it is possible to request a synchronous edit session. (I discussed this here.)
Your service should track focus changes between applications and between controls within an application. This means that your text service must install event sinks for ITfThreadFocusSink and ITfThreadMgrEventSink.
Your text service should use compositions to handle partially formed input.

This last assumption is the big one. It can cause problems for text services that aren't keyboard-related (speech, for example).

The problem is that TSF handles the (admittedly, very difficult) job of interacting with non-TSF aware applications entirely through compositions. Once you close the composition, TSF assumes that you're completely finished with that piece of input.

Unfortunately, it's hard to tell beforehand when you're done with a piece of dictation. SAPI will tell you when it's recognized a piece of text, obviously, but, ideally, once you've dictated some text, you would like to be able to correct it. That requires that you leave the composition open.

In an application that isn't TSF-aware, though, you need to close that composition as soon as you can (it's bad form to have large open compositions; most IMEs have compositions that are a few characters in size).

So there's a tradeoff here. Dictation in Windows Vista currently closes the composition as soon as the text is recognized. (In fact, it doesn't use compositions at all.) That works fine for TSF-aware applications, but causes problems with TSF-unaware applications. In particular, once you've dictated some text, you can't correct it by voice. That's why Windows Speech recognition makes you confirm every dictation into a TSF-unaware application.

What to do when you push a key

Eric Brown — Fri, 05 Oct 2007 17:00:01 GMT

I received an interesting email the other day asking about how to get the character code from the parameters passed to the ITfKeyEventSink::OnKeyDown method.

The answer is that most keyboard related text services only work with a particular keyboard layout, and the text service manages the mapping from virtual key codes to character codes.

It is actually surprisingly difficult to write a keyboard-related text service that is keyboard-layout agnostic, as there isn't a public API that exposes the details of the keyboard layout.

"What about ToAscii/ToUnicode (and the related APIs)", you say? Well, unfortunately, those APIs don't actually work across all languages. In particular, they don't deal well with dead keys and obscure shift states (AltGr, SGCAPS, etc.), and, of course, fail utterly with languages like Japanese, Chinese, and Korean. Michael Kaplan has an extensive set of articles about using ToUnicode (summarized here).

If you want to write a text service that is completely agnostic as to keyboard layouts, then, most likely, you probably don't want to use ITfKeyEventSink. What can you use to monitor keyboard changes?

So far, the best recommendation I have is to use ITfTextEditSink::OnEndEdit, and if either of the following two conditions are true, then you can conclude that a keyboard made the changes:

1) Call ITfEditRecord::GetTextAndPropertyUpdates to see if it contains a range with the property GUID_PROP_COMPOSING. TSF will set this property on text that is part of a composition.

2) Check if the context that you get from OnEndEdit has one or more ITfCompositionView objects within it. Here's some sample code to demonstrate:

hr = pContext->QueryInterface(IID_ITfContextComposition,
                              (void **)&pContextComposition);
if (hr == S_OK)
{
    IEnumITfCompositionView *pEnumCompositionView;
    hr = pContextComposition->EnumCompositions(&pEnumCompositionView);
    if (hr == S_OK)
    {
        ITfCompositionView *pCompositionView;
        while (pEnumCompositionView->Next(1, &pCompositionView, NULL) == S_OK)
        {
            ITfRange *pRange;
            hr = pCompositionView->GetRange(&pRange);
            if (hr == S_OK)
            {
                // Do Stuff Here
                pRange->Release();
            }
            pCompositionView->Release();
        }
        pEnumCompositionView->Release();
    }
    pContextComposition->Release();
}

Input Scopes

Eric Brown — Tue, 10 Jul 2007 17:00:00 GMT

Another useful, but underutilized, group of functions in TSF are those relating to Input Scopes. Input Scopes allow an application to define the sorts of things that are expected in this document (edit control, etc.). For example, the Internet Explorer 7 address bar has an input scope of IS_URL | IS_DEFAULT | IS_ENUMSTRING - meaning that the address bar expects the input to be either a URL, one of the entries in its enumeration, or just plain anything (IE will default to searching for the string you typed).

Input scopes were originally created for the Tablet PC, and a lot of the MSDN documentation is specifically related to the Tablet PC. However, other input handlers (for example, speech) use input scopes as well to improve recognition. For example, Windows Speech Recognition has a custom URL language model that is used when a control specifies the IS_URL input scope. This significantly improves recognitions on URLs that are part of the language model.

The functions are very easy to use, if you have an hwnd for your control. If you only want to set a single input scope (for example, IS_DIGITS for a numeric control), then SetInputScope is the function for you. If you want to set multiple input scopes, or you want to use one of the more exotic input scopes like IS_PHRASELIST or IS_REGULAREXPRESSION (which return application-defined strings), then you will want to use SetInputScopes, which allows you to pass an array of input scopes, along with other data. Finally, Windows Vista adds SetInputScopes2, which is a variant of the SetInputScopes function, but which takes a string enumerator object instead of an array of strings.

One thing to take note is that if you call SetInputScope (or one of its siblings), you need to call SetInputScope(hwnd, IS_DEFAULT) before the window is destroyed. MSDN mentions this in the remarks; I want to emphasize it.

If your control doesn't have an hwnd, or if you want to have multiple input scopes per document, the situation is a bit more complicated. MSDN isn't very clear about how to make input scopes work in TSF-aware applications. In the description of the ITfInputScope interface, MSDN says

A TSF-aware application does not call SetInputScope directly, but rather implements either ITextStoreACP or ITfContextOwner to get a pointer to ITfInputScope.

What this bit means is that TSF-aware applications need to implement input scopes via an application property. In particular, you need to implement ITextStoreACP::RequestSupportedAttrs, and ITextStoreACP::RetrieveRequestedAttrs. If you want to have multiple input scopes per document, then you also need to implement ITextStoreACP::RequestAttrsAtPosition. Note that the documentation refers to TS_ATTRIDs; these are typedefs for GUIDs! The TS_ATTRID that you need to support for input scopes is GUID_PROP_INPUTSCOPE (you may, of course, add support for other TS_ATTRIDs).

Retrieving input scopes is fairly straightforward. MSDN has a nice code snippet describing how to retrieve the ITfInputScope interface here; once you have it, use ITfInputScope::GetInputScopes to retrieve the list of input scopes, and the other methods return auxiliary data for exotic input scopes.

If you need the string enumerator (i.e., you want to support IS_ENUMSTRING), then you need to get the ITfInputScope2 interface, instead of ITfInputScope; all you need to do is QI the application property for IID_ITfInputScope2 instead of IID_ITfInputScope. The MSDN documentation for ITfInputScope2 is incorrect; ITfInputScope2 extends ITfInputScope, not IUnknown (and it's only available on Windows Vista).

The Input Scope enumerations are defined here.

Fun with Contexts

Eric Brown — Thu, 28 Jun 2007 02:08:53 GMT

One interface that I hadn't paid much attention to in TSF is ITfContextKeyEventSink. What does this let you do? Why, it lets you inspect keyboard input for a particular context (or document). This lets you do some really fun things, like redirect keyboard input from the current control and put it into another control. An example of this can be seen in the sample text service with candidate list, which, while it doesn't really show how to use the ITfCandidateList interface (boo!), does show how to redirect keystrokes from the main document to the candidate window. Most of the fun is in CandidateList.cpp.

In the function CCandidateList::_StartCandidateList, the service creates a second context for the document, pushes it onto the context stack, then installs a ITfContextKeyEventSink on the new context (and a layout sink on the original context, so that the candidate window can track layout changes).

Then, in the functions CCandidateList::OnTestKeyDown and CCandidateList::OnTestKeyUp, the service intercepts the incoming keystrokes, and the functions CCandidateList::OnKeyDown and CCandidateList::OnKeyUp redirect the keystrokes to the candidate window.

Debugging a Text Service (and Visual Studio problems)

Eric Brown — Tue, 19 Jun 2007 21:09:12 GMT

I've received a few emails asking about debugging Text Services. I figured if two people actually went to the trouble of sending an email, that there's enough demand for a post.

But first, a little diversion: It turns out that Visual Studio 2005 SP1 ships with an updated version of the C Runtime library, and, by default, adds an entry to the application manifest to use that new library. Unfortunately, this can cause serious problems for text services - in particular, the text service DLL often won't load. I work around this problem by turning off manifest generation in the linker - either on the command line, with /manifest:no, or by going to the project property page, then selecting Configuration Properties/Linker/Manifest File in the tree view, and setting 'Generate Manifest' to 'No'. In general, you should minimize your use of the C Runtime library anyway, as it's more stuff that gets loaded into every process.

Now, on to debugging. There are two options: tracing, and real debugging. You can use OutputDebugString and DBMon in your text service to trace the flow of execution without any problems, but for serious problems, like hangs, or crashes, you'll need to use a debugger. I've never been able to debug a text service running on the same machine as the debugger, mostly because the TSF Manager needs to use your (stopped) text service in order to give keyboard focus back to the debugger. Instead, you will either need a second machine, or a virtual machine like Virtual PC. (Microsoft gives this away for free, and also publishes trial virtual disks that you can use for evaluation.)

Once you have installed your new text service on your other machine, you can use remote debugging to debug problems in your text service. I've used the visual studio debugger with great success, but it can have some problems, particularly when you need to debug an application that needs to run elevated on Windows Vista. In those situations, I use cdb and windbg (cdb on the test machine, and windbg on the local machine).

To be honest, I've gotten used to the power of windbg (and cdb), and I use that almost exclusively. But windbg isn't the friendliest debugger in the world, so I want people to know that visual studio works just fine.

Once you have your debugger set up, you can launch a process on your other machine (e.g., wordpad, notepad, Microsoft Word, etc.), and set a breakpoint at your dll entry point (or at the entry point of your choice, for example, CMyTextService::Activate). Visual studio's syntax for setting a breakpoint in a specific DLL is a little baroque (e.g., {,,mydll}MyFunction) , and I find windbg's syntax (e.g., mydll!MyFunction) a little easier to use.

What's a Keyboard?

Eric Brown — Wed, 30 May 2007 17:00:00 GMT

The Text Services Framework makes a number of assumptions when you register your text service as a keyboard text service (i.e., your text service calls RegisterCategory(<clsid of your text service>, GUID_TFCAT_TIP_KEYBOARD, <clsid of your text service).

First, TSF assumes that exactly one keyboard text service can be active at a time. If the user activates one keyboard text service, all other keyboard text services will be disabled.

Second, TSF assumes that keyboard text services must monitor two compartments:

GUID_COMPARTMENT_KEYBOARD_DISABLED, and GUID_COMPARTMENT_KEYBOARD_OPENCLOSE;

and that keyboard text services should monitor GUID_COMPARTMENT_KEYBOARD_INPUTMODE_CONVERSION and GUID_COMPARTMENT_KEYBOARD_INPUTMODE_SENTENCE.

The sample text service tells you to monitor these; unfortunately, it doesn't really describe why you need to monitor those compartments, nor does it tell you what these compartments really mean.

The input mode compartments (GUID_COMPARTMENT_KEYBOARD_INPUTMODE_CONVERSION and GUID_COMPARTMENT_KEYBOARD_INPUTMODE_SENTENCE) contain conversion flags that your text service can use to control the mappings from keystrokes to characters. The other two compartments control when the text service should interpret the incoming keystrokes.

When GUID_COMPARTMENT_KEYBOARD_DISABLED is non-zero, it means that all keyboard processing should be disabled, and all keystrokes should be passed through without alteration. This compartment often has a non-zero value when a text service wishes to display its own candidate UI.

When GUID_COMPARTMENT_KEYBOARD_OPENCLOSE is non-zero, it means that the keyboard is 'open' and that keystrokes should be processed normally. If the compartment value is zero, it means that the keyboard is 'closed', and that all keystrokes should be passed through without alteration.

The main difference between these two compartments is that GUID_COMPARTMENT_KEYBOARD_DISABLED applies to a context, and GUID_COMPARTMENT_KEYBOARD_OPENCLOSE applies to an entire thread (and therefore to all the documents and contexts owned by that thread).

Since GUID_COMPARTMENT_KEYBOARD_OPENCLOSE applies to an entire thread, many text services choose to set up an compartment event sink on this compartment in order to cache the compartment value. That way, you can avoid querying the thread compartment inside your keyboard processing routines. You could also set up a sink on GUID_COMPARTMENT_KEYBOARD_DISABLED, but you would have to disconnect and reconnect the sink when the document or context changes (including focus changes). The cost of manipulating the sinks may outweigh the cost of caching the value.

Generating Candidates from a Text Service

Eric Brown — Tue, 29 May 2007 17:00:00 GMT

So how do you create the candidates from within the text service? Well, the first thing you need is the original input, and you need to know what text was created from the original input. Almost all text services (there might be a few that don't do this) place the created text in the document, and attach the original input as a property associated with the created text.

Now, when you want to create the candidates for a given range, you get your property data from the range, look at the original input, and generate candidates based on the original input. Of course, you need to be in an edit session to do this.

Once you've finished creating the candidates, you can show them (outside an edit session) in some sort of modeless dialog or popup. Once you're done, you can replace the old text with the new text.

Generating Candidates from an Application

Eric Brown — Fri, 25 May 2007 21:43:00 GMT

Kirby left a comment to my post on candidates asking when a text service should create an ITfCandidateList.

The answer is that when the text service wants to show candidates (via a preserved key or other mechanism), it should show its modal UI, and quite possibly may wish to push a new context so that the modal UI can be keyboard enabled (the sample does this).

The thing to note here is that ITfCandidateList is not involved here.

So when does the text service use ITfCandidateList? When the application wants to show candidates.

How does this work? Well, it's pretty complicated, but I'll try to outline the steps here. (Get ready, it's going to get a little wild...)

The big blocks

1) Application gets ITfFnReconversion interface.

2) Application gets target range (ITextStoreACPServices::CreateRange comes in very handy here).

3) Application calls ITfFnReconversion::QueryRange to transform the target range.

4) Application calls ITfFnReconversion::GetReconversion to get the list of candidates.

5) Application displays candidate list (e.g., in context menu, dialog, what have you).

6) Application calls ITfCandidateList::SetResult() when the selection is selected.

7) Application releases candidate list, reconversion interface, and does other cleanup.

Alternatively, if the application wants to let the text service manage the UI, it can call ITfFnReconversion::Reconvert at step 4 and step out of the way. (Note that not all text services implement this, as this function must not wait for the UI to be dismissed before returning.)

An excellent way to test this is with Microsoft Word. Word supports reconversion, and will show the candidates in the context menu when you right-click on a word.

Getting ITfFnReconversion

This is a two-step process. First, you get the system function provider, and then you get the reconversion pointer (error checking is omitted):

ITfFunctionProvider* ipProvider;

hr = ITfThreadMgr::GetFunctionProvider(GUID_SYSTEM_FUNCTIONPROVIDER, &ipProvider);

IUnknown* ipConv;

hr = ipProvider->GetFunction(GUID_NULL, IID_ITfFnReconversion, &ipConv);

ITfFnReconversion *ipConversion;

hr = ipConv->QueryInterface(IID_ITfFnReconversion, &ipConversion);

What should QueryRange do?

ITfFnReconversion::QueryRange needs to modify the range to cover the entire range of text contained by the conversion token. An example is useful here, as that last sentence didn't even make sense to me. The Speech text service does correction by words, and all words must be part of a single recognition.

Therefore, the Speech text services's implementation of QueryRange alters the range of text to contain entire words that are part of a single recognition.

If the range starts in the middle of a word with recognition data, QueryRange shifts the start of the range to the start of a word. If the range starts in text that doesn't have recognition data associated with it, it moves the start point to a point where it does have recognition data, or to the end of the initial range. If it runs off the end, it sets pfConvertable to FALSE, and returns NULL.

If it does find some recognition data for the start of the range, it adjusts the end of the range in the same manner.

But wait, you say. How can QueryRange do all that when it doesn't have an edit cookie? Well, it gets one in the usual manner - it (say it with me) schedules an edit session to do all that work. OK, you say, how can I get my data back from an asynchronous edit session? Well, you're in luck - you're guaranteed to be able to schedule a synchronous edit session from ITfFnReconversion::QueryRange.

What should GetReconversion do?

ITfFnReconversion::GetReconversion needs to compute the candidates from a target range (that should be aligned with a conversion token). Like QueryRange, most of the work will need to be done from an edit session, and also like QueryRange, you are guaranteed to be able to schedule a synchronous edit session from within GetReconversion. How you go about computing the candidates is discussed in my next post.