Murray Sargent: Math in Office

Pasting Bitmaps into Text

2013-04-30T04:42:00Z

In the (almost) old days, when you paste a bitmap into a rich-text editor, the editor would give it to OLE (Object Linking and Embedding) to figure out what to do with it. But these days, OLE isn’t always available, so it’s more general to paste them using the Windows Imaging Component (WIC) mentioned in the alpha-channel post and in the RichEdit 8.0 Image Support post. For example, you might want to paste an image from MS Paint or from the handy Snipping Tool. These programs offer bitmaps on the clipboard, although they don’t offer jpg’s or png’s. The bitmaps are in the memory-format Device Independent Bitmap (clipboard formats DIB and DIBV5). As such they are “raw” bitmaps without a bitmap file header. This is a problem for WIC since WIC only accepts bitmaps that have the correct bitmap file format as well as containing valid image data. Other image formats like png’s and jpg’s have headers identifying which codec to use, and WIC depends on this information.

Fortunately DIBs are easily converted to the bitmap file format by prepending the correct bitmap file header to the DIB data. WIC then happily accepts the result. As described in Device Independent Bitmap, a bitmap file header consists of fourteen bytes. You just have to choose the right ones! They are given in the following table

Byte IDs	Purpose	Value
0—1	Type	“BM” (little endian)
2—5	Size of file	Size of DIB + 14
6—7	Reserved	0
8—9	Reserved	0
10—13	Offset of image data	14 + size of DIB header

Here since “BM” is little endian, byte 0 is ‘B’ and byte 1 is ‘M’. The size of the DIB header is given by the first four bytes of the DIB. These bytes make up a little-endian 32-bit integer.

A different kind of bitmap file format is the ico format. The WIC documentation lists this format as one with a native WIC codec. The format provides for one or more icon bitmaps and later versions included png’s and alpha channels. The different bitmaps typically have different sizes, which is handy for accommodating various screen sizes and resolutions. Unfortunately, the native WIC codec can only decode ico files. For some reason, it cannot encode them, so if you paste one into your document, you can see it on screen but can’t save it. It’s tempting to write an ico codec that can encode as well as decode, but since nowadays it’s common to use regular images like png’s for icons, maybe a better ico codec doesn't have high priority.

The Alpha Channel

2013-03-16T20:29:00Z

One of our testers inserted a Microsoft logo image into RichEdit and saw an inky black rectangle. Ignoring any connotation that this might be a bad omen (!), I proceeded to study image formats in Wikipedia and in MSDN. (Not that I hadn't already scoured such documentation). Pretty soon I started to suspect that the image, which is a png (portable network graphics), might have an alpha channel. Such a channel isn't something from outer space; it's a channel that lets you blend an image with the background or make portions of the image transparent. When a pixel's alpha-channel value is transparent (0), the pixel's color is typically black. If so and the pixel's alpha-channel value is ignored, the pixel renders as black, because the pixel is not treated as transparent.

More generally, nonzero alpha-channel values blend the image pixel colors with the background pixels. The alpha-channel value α gives the degree to which the image pixel is opaque. For byte values, α/255 of the pixel's red, green, and blue are combined with 1 - α/255 of the background pixel's red, green, and blue, respectively. So α = 0 implies that the image pixel color isn't used; only the background pixel is seen. Unless you ignore the alpha-channel value.

Not all image formats can have an alpha channel. For example, jpeg images can contain EXIF metadata that gives information about the image, such as time and place where the picture was taken, the kind of camera used, and the exposure settings. But jpegs cannot have an alpha channel. Meanwhile png images can have an alpha channel, but cannot have EXIF metadata. Png's can have other kinds of metadata.

In testing various png’s, we found that a number of them displayed as a completely black rectangles. The png specification shows how to know if an alpha channel is present. Specifically, a png begins with the 8 bytes 89 50 4E 47 0D 0A 1A 0A followed by the IHDR chunk. Note that hex 50 4E 47 are the ASCII values of PNG. The IHDR chunk has the content

Chunk count (13)	4 bytes
Type (IHDR)	4 bytes
Width	4 bytes
Height	4 bytes
Bit depth	1 byte
Color type	1 byte
Compression method	1 byte
Filter method	1 byte
Interlace method	1 byte
CRC	4 bytes

If bit 2 of the color type is nonzero, the image has an alpha channel.

The Microsoft logo png shown later in this post starts with the bytes (each consecutive pair of hexadigits gives a byte) 89504E470D0A1A0A 0000000D 49484452 000000FA 0000002D 08 06. These bytes have the meanings

png header	Chunk length	IHDR	Width	Height	Bit depth	Color type
89504E470D0A1A0A	0000000D	49484452	000000FA	0000002D	08	06

Bit 2 of the color type 06 is 1, so there is an alpha channel. In fact, 06 means “Truecolour with alpha”.

Using GDI, it’s easy to display the image with its built-in (alpha-channel) transparency. The AlphaBlend() function has a BLENDFUNCTION argument that lets you say an image has an alpha channel. Check the image’s pixel format (call IWICBitmapFrameDecode::GetPixelFormat()) and note whether the GUID returned corresponds to a bitmap with an alpha channel. If so, use a BLENDFUNCTION saying the image has an alpha channel.

Here’s a picture of the png logo on a yellow background and another on a red background. The red and yellow backgrounds are displayed because the png is transparent in those areas.

Direct 2D (D2D) doesn’t support all bitmap formats, so in converting from a WIC bitmap to a D2D bitmap you need to choose carefully. For example, the premultiplied bitmap format GUID_WICPixelFormat32bppPBGRA is supported by D2D, but I wasn’t able to get the straight GUID_WICPixelFormat32bppBGRA format to work correctly. With D2D that format always displays as a black rectangle. Maybe someone noted that the premultiplied alpha-channel bitmap is more efficient and therefore chose not to support the straight alpha-channel bitmap. In contrast, GDI handles all formats that I tried.

Mystery of the black rectangles solved!

RichEdit 8 Performance Improvements

2013-02-22T19:27:00Z

This post describes a couple of performance improvements introduced in RichEdit 8: 1) a more efficient display tree, and 2) a faster rich-text formatting mechanism. Performance is always of interest, partly to make the user experience more enjoyable (animations should be smooth, not jerky, and who wants to wait for things to happen?) and partly to save battery power. RichEdit 8 also has performance improvements when multiple stories are used, something I’ll describe in a future post. In contrast the DWrite path introduced in RichEdit 8 is slower than the GDI path primarily because it handles default OpenType features. The features improve the typography, but they do take time to process. In fact, OneNote disables this extra processing, favoring speed over typographic elegance. We hope to improve the performance for elegant typography in time.

RichEdit’s display tree for a multiline display consists of an array object called a CArray that contains CLine objects. The CLine object is a recursive structure used to cache display information about the text lines and table rows to be displayed on screen and other output devices. A CLine can represent a single line, or it can point to a CArray of CLine’s for table cells in a table row, or to a CArray of CLines for a math paragraph or for the lines in a table cell. A large file requires many CLine’s so in principle the CLine structure should be as small as possible given constraints of what data is needed for scrolling and displaying the text.

Over the years, various metric variables have been added to the CLine structure in order to support enhanced display capabilities. Specifically Page/TableServices (PTS) needed three extra metrics, ideal layout (device independent, possibly subpixel) required five extra metrics, and OfficeArt (used for PowerPoint, Smart Art, and other clients) required four. Having all of these in the CLine penalizes simpler clients such as WordPad and even clients like OneNote when it doesn’t enable PTS. Accordingly RichEdit 8 stores the extra metrics in a property bag which is empty for simple cases like WordPad and is filled with just the metrics needed for the more elaborate models. For simple plain text, all lines have the same height and the same descent. Removing these values from the CLine would make it even smaller, but this feature isn't implemented in RichEdit 8. Another improvement would be to bypass measuring lines altogether if line wrapping is disabled. This could be handy for scanning through large plain-text data files.

The second performance improvement speeds up character and paragraph formatting changes and also speeds up updates of the display arrays referred to in the first improvement. To understand this improvement, consider that when a user types characters, the characters are inserted into a text buffer gap. This improves typing performance because the buffer memory only has to be moved when the buffer fills up. The text buffer gap has been a RichEdit feature since Version 1.0. The character and paragraph format arrays (CArray’s of format runs) and the CArray’s of CLine’s also have always had buffer gaps, but up to RichEdit 8.0, their gaps were located at the ends of the CArray’s. So when a character format change is made in the middle of the array as during script itemization and a format run is added or deleted, a block move occurred. Similarly when you delete or type enough in the middle of a document to cause the number of CLine’s to change, a block move was needed. To speed this up, the CArray buffer gap was made moveable so that changes always occur in the gap. For C++ aficionados, the CArray is actually a template class, so that it can be reused for many different kinds of objects, such as the CLine and the format runs.

RichEdit 8.0 Touch Support

2013-01-29T19:24:00Z

Windows 8 introduced substantially better touch facilities than those available in Windows 7. One cool thing is the new touch on-screen keyboard, which is enabled if you have a touch screen. To see it, swipe in from the right side, touch the cog icon for settings, touch the Keyboard, and then touch “Touch keyboard and handwriting panel”. This displays the touch keyboard across the bottom of the screen. As is typical with on-screen keyboards, this one shows upper-case letters when the shift key is pressed, alternate keys when an alt key is pressed, and selecting a different language keyboard displays the appropriate keys for that language. A new feature is that if you keep pressing a key, it may display a set of alternative keys around the original. For example if you press the a key, you’ll see a whole group of accented a keys, which you can then move to and insert. Similarly if you press the = key, you see a display of ≡ ¬ ≠ ≈ ≉, which you can insert. Imagine, in principle you could design a math keyboard with all the Unicode math characters this way! This blog post doesn’t pursue this exciting possibility, but instead describes how a programmer can take advantage of some touch functionality added to RichEdit 8 in Windows 8.

Touch facilities such as zooming with pinch and expand (reverse pinch) are typically handled by the client using transforms. RichEdit itself has been generalized to handle the selection and grippers on Windows 8 touch devices. These features are available to clients who want to offer touch capabilities with standard Windows apps using msftedit.dll. For Windows RT apps based on the XAML RichEditBox and TextBox controls, which also use msftedit.dll, touch facilities are handled by the XAML infrastructure.

Specifically, sending an EM_SETEDITSTYLEEX message with the SES_EX_MULTITOUCH extended edit style enables or disables the touch features. They are disabled by default, for which touch input is simulated by mouse messages. The new messages EM_SETTOUCHOPTIONS and EM_GETTOUCHOPTIONS can be used to control whether the selection handles (grippers) are displayed.

Windows 8 has gripper controls, which are a pair of transparent windows displaying gripper visuals and sending touch manipulation events back to the host, e.g., RichEdit. RichEdit 8 supports the following touch operations:

Smart caret placement on tap (snaps to the nearest word boundary)
Dragging the gripper extends or adjusts the selection
Tap on selection or gripper as well as Press and Hold brings up the context menu

When read-only mode is active, the behavior is modified so that a tap on a link activates the link and a tap on text selects a word.

In addition to combining touch with keyboards as in the touch on-screen keyboard, one can combine touch with a stylus. Touch isn’t nearly as geometrically precise as a stylus. The two together can offer special functionality beyond what either can offer alone. For example, a touch can emulate a control key like ctrl or alt, while the stylus pinpoints the location to be altered/interrogated. Multitouch is already being used widely in zoom/expand, scrolling and other ways. Imagine the number of possibilities with multitouch and a stylus! People who are musically inclined and gamers have the dexterity to perform all sorts of amazing interactions. These possibilities are beyond what’s supported by RichEdit 8, but are certainly exciting prospects for the future.

Program Annotations

2012-12-31T19:50:00Z

Notation is a favorite topic of mine being at least partly inspired by Bertrand Russell, who once wrote, “A good notation has a subtlety and suggestiveness which at times make it seem almost like a live teacher…and a perfect notation would be a substitute for thought.” An earlier post on notation deals with the linear format for mathematics used in Microsoft Office. The present post deals with annotations used to help code-proofing tools verify the correctness of computer programs, especially those written in C++.

The vast majority of programs in both Windows and Microsoft Office is written in C++. This choice benefits performance substantially, but C++ can act like a double-edged knife: you can cut with it, but it can cut you too! Accordingly people have written programs like Prefast that analyze programs for various kinds of errors, such as buffer overflows and null pointer dereferences, that cause crashes and security leaks. The analysis is greatly improved if annotations are included that define how the variables in the code are designed to be used. A great variety of annotations have been used over the years, but the current set, called SAL (source code annotation language) seems to be stabilizing and is used in both Office 2013 and Windows 8. This stabilization greatly aids development of programs like RichEdit, which are shipped in both environments.

While the benefits of SAL are great in improving program reliability, the annotations add substantial notational overhead that can slow down a programmer’s understanding of the code. The simple annotations like _In_ and _In_opt_ are easy enough to read and understand, but as the annotations have become more and more detailed, they have become increasingly opaque and unwieldy, such as _Out_opt_cap_post_count_(c, d) (don’t ask!). For example, this shows up in the RichEdit code base in the function header

HRESULT CMathFont::GetMathGlyphVariant (

PCLSMATHSTYLE pstyle, // IN: Math style

USHORT gindex, // IN: Glyph index

MATH_EXTENSION extType, // IN: Extension type

LONG lDimension, // IN: Measurement requested

USHORT cMaxComponent, // IN: Maximum number of glyph components

BOOL * pfVariantFound, // OUT: Returns true iff a variant is found

USHORT * pcComponents, // OUT: # components in resulting construct

_Out_opt_cap_post_count_(cMaxComponent, *pcComponents)

USHORT * piComponents, // OUT: List of component glyph IDs

_Out_opt_cap_post_count_(cMaxComponent, *pcComponents)

LONG * plOffsets) // OUT: List of glyph offsets

This function is used to get one or more glyphs used to display the input glyph at a particular size, such as a large integral sign. One can say that even without the _Out_opt_cap_post_count_(c, d) annotations this is a complicated function prototype, but with the annotations it’s particularly hard to read. And the function prototype is missing SAL annotations for the other argument variables (the comments explain the usage, but the code analysis tools don’t understand the comments).

While the annotations are included primarily for the code analysis tools, they do provide useful information for programmers if they can understand them and they should be available for inspection by programmers. Furthermore a programmer had to add them in the first place. They are not added automatically, although their absence is flagged by the code analysis tools.

So how can we get around this quandary of needing the annotations but not always wanting them staring us in the face? Two additions to the Visual Studio’s C++ editing environment would help: 1) tooltips that explain the annotations in plain English, and 2) the ability to suppress their display. Visual Studio has long been able to collapse/expand the bodies of if() statements. It could similarly provide the ability to collapse/expand the SAL annotations.

RichEdit 8 Zoom Support

2012-11-16T22:18:00Z

In this day and age with touch pinch and expand zooming of screens, zoom is everywhere! So you might wonder how to zoom the contents of a RichEdit control. There are two ways: 1) maintaining the display width and rewrapping the text as necessary to fill the client rectangle, and 2) zooming everything including the display width. The first way can be called rewrapped zoom and is enabled by sending an EM_SETZOOM message or by providing an appropriate display height in the ITextHost::TxGetExtent callback. Rewrapped zoom can be accomplished by RichEdit itself, requiring a minimum of host effort. It’s useful notably in that no panning is needed to read the text. For example, the Amazon Kindle offers rewrapped zoom to allow readers to view text at a larger size without panning.

The second way can be called unrewrapped zoom and has to be implemented in the host since the client RECT zooms along with the contents. This is the typical zoom method used on cell phones, tablets, and in many Windows applications such as OneNote and Word. It requires panning for larger zoom sizes when the text lines no longer fit within the window width. It uses a six-factor 2D transform and the host has to “unzoom” mouse coordinates so that RichEdit hit testing works correctly. For the traditional Windows GDI environment, the hosts sets the zoom factor by calling SetWorldTransform(). For the relatively new D2D environment, the host sets the zoom factor by calling ID2D1RenderTarget::SetTransform(). Both calls pass the same 2×3 transform matrix, although the matrices have different names. The host can also implement rewrapped zoom by unzooming the display width appropriately, although it’s easier to let RichEdit handle rewrapped zoom.

Up through RichEdit 8, both kinds of zoom work in GDI mode. In RichEdit 8, unrewrapped zoom works in D2D mode; earlier versions of RichEdit don’t have a D2D mode. RichEdit 8 doesn’t rerasterize images when the zoom factor changes, so images can become blurry even when they have high native resolution.

EM_SETZOOM provides the zoom numerator in wparam and the zoom denominator in lparam. Alternatively the numerator is given by the height returned by the ITextHost::TxGetClientRect() and the denominator by the vertical size returned by ITextHost::TxGetExtent. The latter is in HIMETRIC units (0.01 mm), and is converted to pixels for the zoom denominator. The zoom factor is incorporated into the horizontal and vertical display device resolution factors when GDI is being used to display text. These factors are set to EMUs per inch (914400/inch) for the D2D/DWrite mode, and hence RichEdit 8 doesn’t offer rewrapped zoom in D2D mode. Hopefully someday!

Inserting Tables Using the Keyboard

2012-10-13T00:05:00Z

This post describes a little trick that lets you insert a table into RichEdit and into Word just using the keyboard. Admittedly in Word, you can use the handy table tool on the Insert tab with a mouse or with a keyboard. But in WordPad and the Windows RT RichEditBox, you don’t have a table tool.

So here’s the trick: on a line of its own type a plus (+) followed by as many dashes (-) as needed for the width of the first cell, followed by a plus to end that cell. If you only want a one column table, just hit the Enter key and you have a one cell table. To have more cells in the row, type more dashes and pluses, being sure to end the line with a plus. For example, hitting the Enter key at the end of the line

+-----------------------+---------+-------------------------------------+

inserts the table row

When you have as many cells as you want, to get another row, put the insertion point at the end of the row and hit the Enter key. Alternatively put the insertion point in the last cell and hit the Tab key. If the insertion point isn’t in the last cell of the table, a Tab advances to the next cell and a Shift+Tab backs up to the previous cell. When the insertion point is in the last cell of a table, the Tab key inserts another row. So pretty quickly you can enter a simple table with arbitrary numbers of cells, rows, and cell sizes. You can use the mouse to change the width of the columns.

You can also change the alignment of a table by selecting the whole table including the end of row marks and hitting ctrl+e or ctrl+r for centering and right-justifying the table, respectively. Ctrl+L left justifies the table. In Word there are a number of other cool tools such as the one to autofit the contents of a table (select the table, right-mouse click on it and choose the Autofit option).

RichEdit 8.0 Accessibility

2012-09-14T20:24:57Z

An important part of a computer’s user interface is to provide for user interaction alternatives to the usual sight-oriented methods. Such capability requires programmatic access to what’s on the screen so that screen reader programs can express the content using voice or other means. Microsoft UI Automation (UIA) provides such access. A valuable side benefit is that test scripts can use UIA to test the content of a wide variety of programs without having to understand the individual program object models.

RichEdit 8 has an implementation of UIA that exposes most objects in a RichEdit instance. This is done via the UIA Text Pattern and includes basic character and paragraph formatting, images, OLE objects, math zones, tables, hyperlinks, and the text inside or associated with these objects. The implementation is showcased in WordPad on Windows 8 and provides a popular (inside Microsoft) example of how to implement UIA for text applications. All UIA value providers are read only. For example, you can read out a URL’s text but cannot change it. To change it programmatically, you can use RichEdit’s Text Object Model (TOM) or the Microsoft Text Services Framework.

If you implement a windowless RichEdit control, you hook up its UIA tree by calling ITextServices::QueryInterface() for an IRicheditWindowlessAccessibility interface and then call IRicheditWindowlessAccessibility::CreateProvider() with an IRawElementProviderWindowlessSite to get the UIA standard base interface IRawElementProviderSimple.

You can examine the complete UIA tree using the Microsoft SDK inspect.exe. The raw text for a RichEdit instance is given for the “RichEdit Control” document node. This raw text includes the structure characters defining any table-row start and end delimiters and cell markers, as well as the math structure characters for math objects like fractions. A math-zone object is treated as an image with a text string giving the math in the linear format. That format gives a pretty good idea of the math content especially if it’s the way the user enters math. Tables are shown as a tree that can be expanded to show each cell’s plain text.

The UIA text range is similar to the original TOM ITextRange and the TOM specification influenced some of the UIA text range choices. Unlike ITextRange2, the UIA text range doesn’t provide access to math text semantics. In this connection, it could be handy if UIA added an ITextMathProvider interface to aid in communicating math text verbally. Alternatively a UIA custom property could be defined that returns the MathML for math zones. MathML has become an important format in math accessibility. Only one property would be needed. Its UIAutomationPropertyInfo structure would contain the property GUID, the UIAutomationPropertyInfo::pProgrammaticName = L”MathML” and the UIAutomationPropertyInfo::type = UIAutomationType_String. One could be more specific and use L”Presentation MathML”, but since it’s part of a text pattern, a client would probably assume it’s Presentation MathML and not Content MathML.

An advantage of a built-in math provider is that the internal Office representation of math is a bit closer to what one would say in speaking the math. For example, the internal (and external OMML) representation uses an n-ary object for integrals, summations, and the like, so a reader immediately knows the kind of operation involved. This is simpler than MathML’s use of <msub>, <msup>, <msubsup>, <mover>, <munder>, and <munderover>, all of which represent n-ary expressions if their base is an n-ary operator. But appropriate programming can parse the MathML and there are readers that make MathML accessible, so a MathML UI custom property is an easy way to math-enable UIA. For MathML-unaware UIA clients, the RichEdit 8.0 image object with a math linear format string is a good fallback.

Thanks go to Rick Sailor, who implemented RichEdit 8.0's UIA support and answered many questions about it.

RichEdit Spell Checking, Autocorrection and Prediction

2012-08-31T15:32:39Z

RichEdit has provided support for client spell checking (TOM object model and temporary character formatting—see tomApplyTmp) and autocorrect (see EM_SETAUTOCORRECTPROC) for many years. But it has been the RichEdit client’s responsibility to access the spell-checking and autocorrection components, except for the built-in math autocorrect option. For clients like OneNote and Outlook, such a responsibility is a small part of the app. But for little programs, accessing proofing components can be arduous. This would be especially true for small apps written using Windows RT.

Accordingly, RichEdit 8 was enhanced to be able to access the Windows 8 spell-checking and autocorrection components directly. The traditional Windows client using msftedit.dll on Windows 8 enables this functionality by sending an EM_SETLANGOPTIONS message with one or more of the options IMF_SPELLCHECKING, IMF_TKBAUTOCORRECTION, and IMF_TKBPREDICTION. The client also needs to enable the Text Foundation Services by sending an EM_SETEDITSTYLE message with the flags SES_USECTF, SES_CTFALLOWEMBED, SES_CTFALLOWSMARTTAG, and SES_CTFALLOWPROOFING.

In the Windows RT RichEditBox, which uses a rich-text RichEdit control, spell checking is enabled by default. The Windows RT TextBox, which uses a plain-text RichEdit control, also can have spell checking, but spell checking is disabled by default. You can change the setting using the Properties pane. Bogdan Radakovic, a developer in Windows 8, integrated the proofing features into RichEdit 8.

To try the proofing features out, I fired up Visual Studio 12 and created a new C++ project for the “Windows Store” using the “Blank App (XAML)” template. I dragged a RichEditBox from the Toolbox into the design view for the MainPage.xaml file. The RichEditBox starts out really small, so I expanded it to a reasonable size for typing text and built and ran the project. To my delight, my typing was proofed and autocorrected very nicely! The keyboard language is used to select the current proofing tools. So I switched to a German keyboard and typed in some text. Sure enough, the German proofing tools are used, which is very cool (I included the German proofing tools in my US Windows 8 installation). But if I type German using an English keyboard, I see lots of words with red-squiggle underlines. The proofing tools do not attempt to guess the language on the fly. For most people, this is a reasonable approach, especially since language detection isn’t infallible. Possibly in the next version, language detection will be offered as an option.

Office Adopts New Windows Display Technology

2012-07-29T18:58:00Z

Office 2013 has undergone a substantial shift to a relatively new display facility, Direct2D, and a new text facility, DirectWrite. These are the display facilities that are used on Windows Phone 8, the new Windows RT slates, and optionally on Windows 7 & 8. Up through Office 2010, the Windows Graphics Device Interface (GDI) and complex script support by Uniscribe were used. RichEdit 8.0 interfaces to both kinds of display technologies, but a given RichEdit instance uses only one of them. This post describes some of the advantages of using the new display facilities along with some of the problems we encountered in supporting them. One of the challenges was ensuring that math display wasn’t broken in the Direct2D/DirectWrite world. Another was to implement higher-level BiDi protocols using DirectWrite, which was designed to support the Unicode Bidi Algorithm (UBA) only.

Direct2D and DirectWrite are modern components that can take advantage of graphics acceleration hardware. DirectWrite supports better text placement using subpixel, device independent layout along with ClearType or other antialiasing methods. So in principle, it’s a big win to move forward to these components. Meanwhile GDI and Uniscribe have been around for years and have been taught to handle all sorts of nonobvious problems. It’s not easy to produce new code with their generality. Furthermore, RichEdit needs to be able to use either DirectWrite Version 1.0, which ships with Windows 7, or the more powerful DirectWrite Version 2.0 that ships with Windows 8. As a result, it was a bit of a tour de force to generalize RichEdit to interface smoothly to these three different display models. The factorization we’ve done has simplified getting RichEdit onto other platforms as well, such as the Android and iOS operating systems.

As an example of GDI tricks, if you specify the charset in creating a font object, GDI will ensure that you get a font that handles that charset. Admittedly charsets cover only a subset of the world’s languages (no Indic, Syriac, etc.), but they do cover many important languages. Another trick is if a character is an end-user-defined character (EUDC) in the Unicode Private Use Area, GDI will ensure that you see a glyph by searching through registered EUDC fonts. These characters are not defined in the Unicode Standard, so you can’t use them reliably for text interchange. But they are popular in CJK locales and a given machine may have fonts with the glyphs that the user wants.

Uniscribe was developed alongside LineServices to support both the UBA and rich-text higher-level protocols. This approach is important for RichEdit rich-text instances, since they use the keyboard and math higher-level BiDi protocols. In addition, digit substitution support requires breaking out numeric runs, which Uniscribe does. DirectWrite creates runs giving the same script value to alphabetic characters and numeric digits that are considered by context to be part of that script. The method we found to identify the digits in an Arabic run given by DirectWrite, for example, is that they are given a BiDi level of 2, whereas the strong RTL characters have a BiDi level of 1. We’ve dreamt up a number of other ways to ensure that Latin letters are rendered LTR (except in plain text with an RLO override embedding). And then you get little problems such as two Thai characters surrounding the Arabic comma U+060C. Since U+060C is considered common to several scripts, DirectWrite happily puts it in the Thai script run. But a Thai font cannot handle it. Font fallback is one way to see a glyph; RichEdit breaks the run and uses an Arabic font.

For the new immersive environment on slates and the Windows Phone 8, not only are GDI and Uniscribe absent, so are the functions handled by the venerable user.dll and OLE dll’s. The user.dll program library includes the OS SendMessage, MessageBox, CreateWindow, etc. In the immersive environment, RichEdit is only available in windowless mode. The client can still send RichEdit messages via the ITextServices::TxSendMessage() method. The advantage of dropping user.dll and the OLE dll’s is the relative simplicity of the model and the correspondingly reduced size. But doing so omits significant functionality, at least in the initial version. At the same time, the touch functionality is dramatically improved on Windows 8 and in the immersive environment.

OneNote 2013 is the first Office application that has a fully immersive version, thanks, in part, to the fact that RichEdit 8.0 can run in a fully immersive environment. OneNote 2013 has the math display facility and you can edit math using alt+= to insert math zones and the linear format to enter equations. One problem with OneNote 2013 is that for some zoom factors, horizontal lines are snapped vertically to the wrong pixel boundary. There wasn’t enough time to fix this problem for OneNote 2013, so you may notice some square roots with misaligned bars.

Another side effect of these huge changes is that we have had no time to improve the Office math editing and display facility for Office 2013. The main accomplishment math-wise is not to lose any math functionality, aside from misaligned bars in OneNote. Hopefully next time we can add some cool math features.