Murray Sargent: Math in Office

The Presentation MathML 3.0 specification provides for RTL math zones. In fact, it allows a dir = “ltr” or “rtl” attribute on the top level <math> element as well as on <mrow>, <mstyle> and token elements like <mi>. Except in rare cases, only the <math> direction need be specified, since all the elements inside have the same directionality (see Section 3.15 of the MathML 3.0 specification). The specification has now undergone Last Call status and so we need to have implementations of the new features. Accordingly I’m interested in implementing at least part of the RTL functionality, namely RTL math zones.

First, consider what an LTR math zone is. This is what Word 2007 and the Office 2010 applications implement. It does have RTL text whenever Arabic or standard Hebrew characters appear adjacent to one another. But all operators and other “neutral” characters are considered to be “strong LTR”, that is, they are displayed to the right of the character that precedes them. This can be quite different from a display that obeys the Unicode Bidirectional Algorithm. A sequence of digits is always displayed LTR, regardless of the character that precedes it even outside math zones and according to the Unicode bidi algorithm. Inside LTR math zones a sequence of digits is displayed to the right of the character that precedes it even if that character is Arabic. According to the Unicode bidi algorithm, a number following an Arabic character is displayed to the left of the Arabic character in both LTR and RTL paragraphs. Inside embedded normal text in a math zone, the usual rules for bidi text are followed. Note that except for such text, the math-zone bidi rules are much simpler than those of the Unicode bidi algorithm, which gets quite tricky in complicated scenarios.

Perhaps you noticed the term “standard Hebrew characters” above. By this I mean all Hebrew characters except the four Hebrew letter-like math symbols ALEF SYMBOL, BET SYMBOL, GIMEL SYMBOL, and DALET SYMBOL (U+2135..U+2138). These symbols are strong LTR characters, unlike their HEBREW LETTER counterparts located in the Unicode Hebrew block (U+0590..U+05FF).

Analogously in an RTL math zone and in the absence of directional overrides, operators and other neutrals are treated as strong RTL characters. A sequence of digits is still displayed LTR, but it appears on the left of the character that precedes it even if that character is Latin. Sequences of Arabic and standard Hebrew letters are RTL as usual. At least that’s how I think a typical RTL math zone should be displayed.

This description of math-zone directionality is somewhat simplified compared to the generality encountered in the real world. To see some of the special cases that can happen, please read the papers by Azzeddine Lazrek:

http://www.ucam.ac.ma/fssm/rydarab/doc/unicode/amassf.doc

http://www.ucam.ac.ma/fssm/rydarab/doc/unicode/amasl.pdf

http://www.ucam.ac.ma/fssm/rydarab/doc/unicode/amdsl.pdf

http://www.ucam.ac.ma/fssm/rydarab/doc/unicode/others.pdf

http://www.ucam.ac.ma/fssm/rydarab/doc/communic/unicodem.pdf

http://www.w3.org/TR/arabic-math/

http://www.ucam.ac.ma/fssm/rydarab/

The following review papers are excellent sources for overviews of RTL math:

http://en.wikipedia.org/wiki/Modern_Arabic_mathematical_notation

http://www.ima.umn.edu/2006-2007/SW12.8-9.06/activities/Lazrek-Azzeddine/MathArabIMAe.pdf

When writing the Math in Office 2010 post back in July, I could just imagine the disappointment various people would have when they discovered no mention of equation numbering. After getting math into PowerPoint, equation numbering had been the most often requested feature. Since PowerPoint 2010 now has the math facility, equation numbering has risen to the top of the wish list. Note that there is a way to display and manage equation numbering in Word as described in the work of Dong Yu. To get a feel for a native numbering facility, I implemented a prototype in RichEdit that interfaces to the underlying Page/TableServices (PTS) math handler’s equation numbering facility. This post describes that approach. It may not be the one we ship someday, but it does work pretty well. First I describe how equation numbers are represented in the file format and in memory and then consider equation number management.

The file format is one area a new feature has to consider. It would be nice not to change the file format for equation numbers, since it makes backward compatibility just that much harder to deal with. For example, it would be very convenient if equation numbers from a future version of Word would display reasonably well in Word 2010 or Word 2007, admittedly with small modifications to those versions. So I examined the nooks and crannies of the Office math format, OMML, for a math object that could be used to contain an equation number.

Lurking in that object space is a “no-op” phantom. To understand what this object is and why no one is likely to have used it, we need to know what a phantom object is. The phantom is characterized by five Boolean flags: 1) zero ascent, 2) zero descent, 3) zero width, 4) show, and 5) transparent. You can read about it in the post on MathML and Ecma Math (OMML) and in Sec. 3.17 of the linear format paper. The “zero” flags are handy for suppressing any combination of three object dimensions. Such suppression is called “smashing” by Donald Knuth. You can “smash” the descent of a character with a descender like y so that it has the same size as one without a descender like x. To facilitate entering smashes, we have the control words \asmash, \dsmash, and \hsmash to smash the ascent, descent, and width, respectively, of the phantom argument. Smashes have the show flag on, since the idea is to show the argument, but give it one or more zero dimensions. If you have the show flag off, you have a true phantom, in that space is taken up by the argument, but nothing is displayed. The fifth flag, the transparent flag, means that the argument is treated by the surrounding environment as if no object were there. So an equals sign inside a transparent phantom has math spacing appropriate for an equal sign outside a math object, apart from changes imposed by smashing various dimensions.

A no-op phantom is one that displays its argument as if the phantom weren’t even there: the argument has its true dimensions and it is transparent. Since the no-op phantom doesn’t serve any purpose, we might as well give it one: house an equation number. That works for the file format. Now let’s see how it works for actual display in a document.

The PTS math handler which ships with Office 2007 and Office 2010 has a number of callbacks for equation numbers. We just need to answer the questions appropriately and presto; the equation number will be displayed accordingly. To make sophisticated choices such as placing the numbers on the left hand side of equations instead of the right hand side, one needs to have some document properties. Ignoring such generality for the moment, we tell the PTS math handler to display the equation number flushed to the right on the last line of an equation. An equation may take up several lines and the PTS math handler can center the equation number vertically, if desired. Secondly, we place the no-op phantom as if it were an equation itself directly following the equation to be numbered. As such the no-op phantom is the only thing inside a soft paragraph, separated from its equation by a Shift+Enter (ASCII VT character). The PTS callbacks needed to be generalized slightly to deal with this special kind of “equation”, but it’s straightforward.

The last thing to consider is how to manage equation numbers, namely insert, delete, and edit them. Section 3.21 of the linear format paper explains how to type in an equation number, namely enter the equation followed by a # (U+0023) followed by the desired equation number text and type a [Shift]+Enter. The numbers can be edited in place. But underneath one needs renumbering and synchronization with inline equation number references. Such management could be accessed via a context-menu or math-zone acetate drop down menu. Insert and Delete would work with a single click and edit would open a dialog with options such as whether to include chapter and section numbers and whether the resulting entries should be separated by periods or hyphens. Word would likely treat the equation numbers as bookmarks so that links to them would automatically show the current number. Alternatively, the insert/delete/edit command handler could update all inline no-op phantoms to agree with the corresponding no-op phantoms inside math paragraphs.

MathML doesn’t have the concept of a math paragraph, but when embedded in a parent format, the math paragraph can be emulated fairly easily. Basically a math paragraph is just one or more math zones separated by soft paragraph breaks. So the equation number for a displayed MathML math zone (<math>…</math>) would follow the math zone. MathML has an <mphantom>, which suppresses the display of its argument, but keeps its size. Since we want to show equations numbers, <mphantom> isn’t useful for housing an equation number. <mpadded> has the other attributes of the OMML <phantom> except for transparent and if it just displays its argument with no changes, it could be used to house an equation number. In addition, it’s legal to add any attributes that one wants to a MathML element provided the attributes are in a private namespace, so one can be quite precise as far as a given family of implementations go. To be interoperable, MathML would have to add its own way of representing equation numbers, but hasn’t done so in MathML 2.0 or 3.0.

This post is a companion to Automatic RichEdit Hyperlinks. As stated in that post, RichEdit has two kinds of hyperlinks, automatic hyperlinks (autoURLs) and friendly name hyperlinks. A friendly name hyperlink has a name, which is displayed, and a hidden instruction part that contains the actual URL. Such hyperlinks are commonly used when an author wants to display an informative name for a link rather than the URL itself. The present post describes RichEdit’s implementation of friendly name hyperlinks. It’s aimed at programmers, so if you’re not so inclined, you may want to skip it.

A friendly name hyperlink is essentially a field with two parts: an instruction part containing the URL and a result part containing the name. In fact that’s the way it appears in RTF, which has the syntax {\field{\*\fldinst {HYPERLINK "..."}}{\fldresult{...}}}.

In RichEdit, the hyperlink field entity is represented by character formatting effects, as contrasted to delimiters which are used to structure math objects. As such, these hyperlinks cannot be nested, although in RichEdit 5.0 and later they can be adjacent to one another. The whole hyperlink has the character formatting effects of CFE_LINK and CFE_LINKPROTECTED, while autoURLs only have the CFE_LINK attribute. The CFE_LINKPROTECTED is included for the former so that the autoURL scanner skips over friendly name links. The instruction part, i.e., the URL, has the CFE_HIDDEN attribute as well, since it’s not supposed to be displayed. The URL itself is enclosed in ASCII double quotes and preceded by the string “HYPERLINK “. Since CFE_HIDDEN plays an integral role in friendly name hyperlinks, it cannot be used in the name.

For example, in WordPad, which uses RichEdit, a hyperlink with the name MSN would have the plain text

                HYPERLINK "http://www.msn.com"MSN

The whole link would have CFE_LINK and CFE_LINKPROTECTED character formatting attributes and all but the MSN would have the CFE_HIDDEN attribute.

In RichEdit 4.1, the only way to insert a friendly name hyperlink is to read in the corresponding RTF. For the example above, this RTF could be as simple as

{\rtf1{\field{\*\fldinst{ HYPERLINK " http://www.msn.com"}}{\fldresult{ MSN} }}}.

Reading in the RTF can work well, but it’s convenient to have a more programmatic approach. Accordingly RichEdit 6.0 added the ITextRange2::SetURL(BSTR bstr) method, which uses the bstr as the URL for the range of text selected by the ITextRange2. The text in the bstr needs to start and end with ASCII double quotes. The word HYPERLINK is inserted in front of the URL by the SetURL() method. RichEdit 7.0 allows you to remove the link status from a friendly name hyperlink by calling SetURL() with a NULL bstr or one that has only the start and end quotes, signifying an empty string.

RichEdit doesn’t allow the CFE_LINKPROTECTED attribute to be set directly by programs. Maybe it will allow it someday after enough testing is completed to ensure that things cannot go awry.

As for autoURLs, the RichEdit client enables hyperlink notifications (EN_LINK) by sending RichEdit the ENM_LINK flag in the mask included with the EM_SETEVENTMASK message. In RichEdit 5.0 and later, the client can enable tooltips displaying the URLs by sending the EM_SETEDITSTYLE message with the SES_HYPERLINKTOOLTIPS (8) flag.

The original RichEdit Versions post covered RichEdit versions 1.0 through 6.0, since 6.0 was the latest version at the time. RichEdit 7.0 will ship with Office 2010, so here’s an update describing what that version adds. Most additions involve math editing/display and play a central role in the math features of OneNote 2010, PowerPoint 2010, Excel 2010, and other OfficeArt applications. Specifically RichEdit is responsible for interfacing to the LineServices and Page/TableServices math handlers, the math font library, and for a significant part of the math user interface in those applications.

The RichEdit team also has spent considerable time working in the OfficeArt and mso (a large Office shared library) code bases, both for math implementation and for extending/maintaining OfficeArt text. The latter is responsible for the text you see in PowerPoint and also in text boxes in Excel and in other Office applications except for Word. Word renders the text in its own text boxes and uses RichEdit only for dialogs and for the math linear format input method (formula autobuildup, manual buildup, and linearization). The RichEdit team spent a fair amount of time supporting the Windows 7 and WinCE versions of RichEdit. May they too have math someday!

Several entries below feature client access via TOM2 (text object model, version 2), so they can be used by clients other than RichEdit just as formula autobuildup and linearization are used by multiple applications. This provides a start in factoring the RichEdit code base into pieces that can be used individually by other programs.

RichEdit 7.0 Additions:

·         MathML/OMML reader and writer (client access via TOM2)

RichEdit has two kinds of hyperlinks, automatic hyperlinks (autoURLs) and friendly name hyperlinks. As its name suggests, the autoURL is automatically recognized by RichEdit as a hyperlink and is displayed as a URL. A friendly name hyperlink has a name, which is displayed, and a hidden instruction part that contains the actual URL. The present post describes autoURLs; a later post will describe friendly name hyperlinks.

The first autoURLs appeared in RichEdit 2.0, which shipped with Office 97, and have the usual web form, such as, http://www.msn.com. The permitted URL protocols were http:, file:, mailto:, ftp:, https:, gopher:, nntp:, prosper:, telnet:, news:, wais:, and outlook:. To include spaces in the URL, the whole URL needed to be enclosed in an angle bracket pair as in <http://www.xxx.com/fun computing>. RichEdit 3.0, which ships with Windows 2000 up through Windows 7, added the capability to recognize URLs of the form www.msn.com and ftp.unicode.org. RichEdit 4.1, which ships with Windows XP up through Windows 7, added friendly name hyperlinks as well as autoURLs of the form \\word\richedit2\murrays. RichEdit 7, which ships with Office 2010, adds recognition for spaces in URLs without needing enclosure in <>. It also adds recognition of telephone numbers, drive-letter paths, email addresses, and URLs enclosed in ASCII double quotes "". It makes all of these recognitions optional, since you might not want to recognize, for example, phone numbers, or you might want to recognize telephone numbers exclusively.

The recognition is dynamic, fast, and displayed with underline and a blue text color. The autoURL notifications can be sent to the client application by user actions such as typing the Enter key or clicking the left mouse button. Unlike in Word, a particular autoURL cannot be disabled, short of disabling the autoURL recognition functionality in general. This hasn’t been a problem for the most part, but it might be nice to be able to turn off trailing portions of an autoURL that are included incorrectly by RichEdit 7.0’s URL space recognition feature.

To enable or disable autoURL recognition in a RichEdit control, send the control the message EM_AUTOURLDETECT with lparam = 0 and wparam = 1 or 0, respectively. When autoURL recognition is enabled, mouse movement over a link or clicking on a link sends an EN_LINK notification to the client. In RichEdit 5.0, lparam = 1 enables East Asian URL recognition. In RichEdit 7.0 wparam can have any combination of the following values: AURL_ENABLEURL (1), AURL_ENABLEEMAILADDR (2), AURL_ENABLETELNO (4), AURL_ENABLEEAURLS (8), and AURL_ENABLEDRIVELETTERS (16). AURL_ENABLEEAURLS is a preferred alias for the lparam = 1 way of enabling East Asian URL recognition. The idea is that lparam could eventually be used to point to a client structure specifying more information, such as additional protocols.

In memory, autoURLs are identified by the CFE_LINK character formatting attribute. You can retrieve this attribute using the EM_GETCHARFORMAT. This approach can be cumbersome, so RichEdit 6.0 added support for the tomLink unit in the TOM (Text Object Model) ITextRange::StartOf(), EndOf(), and Expand() methods, and RichEdit 7.0 adds tomLink support in the ITextRange::Move(), MoveEnd(), and MoveStart() methods.

The Office math ribbon has a few examples of matrices, but you might like to be able to enter a lot more kinds of matrices and enter them substantially faster. For this you can use the linear format (see Sec. 3.9). For example, a 2x2 matrix is entered by \matrix(…&…@...&…), where the ellipses are the contents of the matrix elements. As in TeX, &’s separate matrix elements. Rows are separated by @’s.

You don’t have to enter anything in the matrix elements if you’re willing to enter them explicitly later. For example to enter a 4x4 empty matrix you can type \matrix(&&&@@@). This is missing the &’s for the second through fourth rows. But the build-up machinery automatically generates a rectangular matrix with each row containing the same number of elements as the row with the maximum number of elements. So in the 4x4 \matrix(&&&@@@) case, the first row has four elements and therefore each of the three subsequent rows has four elements as well.

The matrices don’t have to be square. A six-element, single row empty matrix is entered by\matrix(&&&&&) and a five element, single column empty matrix is entered by \matrix(@@@@).

Often you want parentheses surrounding the matrices. You can enter them explicitly as in (\matrix(&&@@)), or in Office 2010, you can use TeX’s control word \pmatrix(&&@@), which is slightly faster to enter. A little later this year, I’ll update the linear format paper to include the additions that appear in Office 2010.

The maximum number of elements is 254. So you can enter a 15x15 matrix and a 127x2 matrix without error. If anyone needs larger matrices, I’d like to hear about it. We don’t want to allow arbitrarily large matrices because unrealistic test scenarios might grind the machine to a virtual halt.

Imagine typing alt+= in PowerPoint, OneNote, Excel, and, of course, Word and Outlook to enter a math zone and then type a^2+b^2=c^2<space> to see the Pythagorean theorem beautifully typeset on your screen! Or some way more complicated equation, equally beautifully typeset. You don’t have to wait much longer as the people getting the Office 2010 Technical Preview this week can attest.

We’ve added the math editing and display of Word 2007 and Outlook 2007 to PowerPoint 2010, Excel 2010, and OneNote 2010 and improved the model a bit. For example, now the tooltips for the symbol and operator galleries display what you can type to enter these characters from the keyboard. It’s going to be so much fun to demo Office math using PowerPoint live! A slide deck generated with PowerPoint 2010 displays well with earlier versions of PowerPoint. You just can’t edit the slides with the earlier versions since those versions don’t understand math zones. In addition, Office 2010 running on Windows 7 has a very powerful new input method: recognition of hand written equations. This works with all math-enabled applications, but it is streamlined with OneNote 2010, as you might guess. If you’re taking lecture notes in a math, science, or engineering course, enjoy! Also install the math graphing-calculator add-on, which already works with Word 2007.

Adding math support to PowerPoint, OneNote and Excel text boxes hasn’t been easy. Word 2007’s math facility is a tour de force, from both display and user-interface points of view. Achieving the same display in the other applications was facilitated by using RichEdit and Page/Table/Line Services for math display and that was already working in Office 2007 (except for RichEdit 6.0’s lack of math-paragraph support). But the math user interface, which I’ve documented in a series of posts to this blog, is far from trivial. The devil is in the details and Word 2007 does an amazing job. Now all the applications behave similarly in math zones. Another area that required lots of effort is handling the various file formats and cut/copy/paste.

Our approach for Office 2010 hasn’t been to improve on what Word 2007 does, aside from a couple of small exceptions. Bringing the other apps into parity with Word 2007 was about all that one could hope for, given the relatively short development cycle, unforeseen documentation demands, the new web application support, backward compatibility, and the ODF file format support we added to both Office 2010 and 2007. Also we didn’t want to extend OMML or MathML to handle new features. There are a few improvements, such as, OneNote 2010 supports automatic arguments, and all apps except Word and Outlook support the invisible times as an equation line break as well as automatic build up of extended math autocorrect strings. For example, if you type \binomial<space> into a PowerPoint math zone,  you immediately see the built-up (“Professional”) binomial-theorem equation, whereas in Word/Outlook you still need to choose the Professional option to build up the linear-format version of that equation. Conversely some Word/Outlook features haven’t been implemented in the other applications, such as custom matrix column spacings and alignments. These features might be helpful, notably for uses of matrices outside mathematics.

There’s nice MathML import/export support, which is documented here (I’ll do a separate post about that documentation). In particular, when the ODF file formats are used in Office 2010 applications, the math content is represented using MathML 2.0. And you can copy MathML to/from math engines such as Mathematica and Maple.

The bottom line is that if you need math editing and display, whether as a student or as a professional, Office 2010 on Windows 7 is a very compelling combination. If you have problems or suggestions, please feel free to post a comment on this blog or email me.

In Windows 7, WordPad has undergone many improvements even though it uses RichEdit 4.1+ for editing and display. Time and time again, the excellent Hyderabad team responsible for enhancing the Windows 7 WordPad requested very reasonable extensions to RichEdit 4.1, extensions that have been included in later versions of RichEdit. They’ve also requested features that aren’t in the latest RichEdit (7.0) which ships with Office 2010, but that’s another story. One of their many requests has to do with limiting numbered paragraphs to a maximum number of 255. Why not 1000 or 10000? The reason for the small numbering maximum is that RichEdit 4.1 doesn’t cache paragraph number offsets and calculating the offsets on the fly can require substantial execution time. Without the limit, the RichEdit team had bugs claiming that RichEdit was literally hanging. Interestingly enough, later versions of RichEdit don't have this limit. Let's see why.

When the original OneNote was created in the Office 2003 time frame, it used RichEdit 5.0 and needed some special Page/Table Services (PTS) features.  PTS is a component library complementing LineServices. PTS is responsible for laying out pages and tables, whereas LineServices is responsible for laying out individual lines. PTS requires “names” for each paragraph (soft or hard), so Sasha Gil of the RichEdit team added special “NMP” runs for the purpose. Fortuitously, this run infrastructure provides a convenient place to cache paragraph number offsets and allowed us to remove RichEdit’s numbering maximum of 255.

Ironically OneNote has always handled paragraph numbering itself, but nevertheless RichEdit numbering has benefitted by OneNote’s need for PTS. Bidirectional (BiDi) layout involving Hebrew and/or Arabic text and math processing also benefit since the NMP runs greatly speed up paragraph boundary navigation. In fact RichEdit 6.0 and later versions automatically enable NMP runs whenever numbered paragraphs, BiDi and/or math are present in a document even if PTS itself isn’t enabled.

A footnote to this saga is some sage advice given to me by Eliyezer Kohen, the leader of the original teams that developed Microsoft’s TrueType implementation, the OpenType specification, LineServices, and PTS. I was designing and implementing RichEdit’s binary format for, you guessed it, the first version of OneNote (OneNote 2010 has switched to a different format). Eliyezer strongly recommended that RichEdit’s binary format tag paragraph boundaries explicitly, even though RichEdit didn’t formally need such tagging. He was absolutely right. From both structural and performance standpoints, knowledge of paragraph boundaries is very valuable in text editing and display.

The next version of Windows, Windows 7, includes a cool applet called the Math Input Panel. This applet lets you enter mathematical text using a pen or a mouse. It recognizes what you enter and displays the result using a special private version of RichEdit 6. It also lets you copy the results to Word, Mathematica, or any other application that reads Presentation MathML.

Many people may find that writing equations by hand with this applet is the easiest and fastest way to enter them into a computer. Since I’ve made similar claims for linear format entry in Word 2007, we decided to have a race. I chose nine equations from theoretical physics and we started entering. The person entering via hand writing beat me by a nose, but had two errors, whereas I had none. But really we both won, since we demonstrated that we could enter equations into Word 2007 remarkably fast.

You type Alt+= or click the Insert ribbon Equation button, and presto! You’ve inserted an empty math zone place holder that states “Type equation here.” in the language you’re using. Then you type a^2+b^2<space> and you see a²+b², except in better typography. The empty math zone place holder seems simple and intuitive, but behind the scenes a fair amount of software is handling a variety of situations that probably don’t come to mind when you do the things above.

For example, if you cursor away from the math zone, either by using the arrow keys or the mouse, the empty math zone place holder remains; it’s just not selected anymore. If you save the file and reopen it, you’ll see the place holder just where you inserted it. If you select the place holder, possibly with text surrounding it, and copy paste it somewhere else in the document, you’ll see it pasted at the target location.

Furthermore if you return to the place holder, either using the arrow keys or the mouse, the place holder will automatically be reselected. So if you then type something, what you type replaces the place holder. Yet another characteristic: if you type Backspace in a math zone until the last character is deleted, the empty math zone place holder reappears and is selected. Naturally if you type Backspace yet one more time, the place holder is deleted, just as it is if you type the Delete key.

Now what if you type Alt+= immediately before or after an unselected empty math zone place holder? You guessed it! Instead of inserting a new place holder, the current one is reselected. There are other variations that Word 2007 doesn’t get quite right: if you select a bunch of text including one or more empty math zone place holders and then type Alt+=, you’d probably expect the included empty math zone place holders to be deleted, since they’re inside a math zone and nested math zones aren’t permitted. But Word 2007 doesn’t catch this relatively unlikely scenario (it doesn’t crash either, thank goodness!)

The bottom line is that the seemingly simple idea of having a place holder for inserting some math isn’t as straightforward as you might expect.

What do to in the future? The empty math zone place holder is clearly a useful math user interface feature, even if it is tricky to implement correctly. In addition, particularly for educational purposes, one might like to have other place holders within nonempty math zones to ask the user to type, for example, the numerator of a fraction or the integrand of an integral. Such place holders also could be copied and saved to file and reselected by moving the insertion point into them.

A number of character formatting properties are treated differently in a math zone than they are in ordinary text. These include underline, strikeout, math font face and size, subscript and superscript. This post discusses how these properties differ in math zones, sometimes to the surprise of users.

For starters, the ordinary subscript and superscript attributes are not allowed in math zones, since they are insufficient to handle the more elaborate kinds of scripts found in mathematics. In particular they are only one level; a subscripted superscript cannot be represented with them, nor can a superscript be aligned over a subscript. On a more subtle level, the special “cut-in” kerning of math fonts like Cambria Math isn’t available in nonmath fonts. In principle, it would be possible to support the Ctrl+= hot key for subscript and the Ctrl+Shift+= hot key for superscript along with the corresponding ribbon tools, but it might be confusing and simple linear format commands like e^x for superscripting the x are easier to type in spite of their substantially increased generality.

The underline and overstrike attributes are automatically extended throughout the math zone. We tried allowing them to apply on the usual per glyph basis, but that can lead to visual chaos, with underlines up and down throughout a multilevel equation. So to give feedback to the user, we draw a uniform underline and/or a strikeout throughout the math zone. If you want to underline a particular expression, use the underbar math object (linear format \underbar(…), where … is what you want to underline. This measures the … and draws a uniform underline under it . Similarly \overbar(…) draws a bar above the … . If you want to strike through an expression put it in a rectangle using \rect(…) and use the context menus to choose the horizontal strike option. You also need to hide the rectangle borders, since these are shown by default.

Onto the math font. Only one is allowed in a given math zone and it must be the same size throughout that math zone. Furthermore if the math font can display a character, it must be displayed by the math font and not by some other font, unless the character is marked as "Normal text". The only cases where other fonts are allowed in a math zone is for characters not supported by the math font, such as Chinese and Indic characters, or for Normal text embedded in the math zone.

At first these restrictions to a single math font and size seem overly cautious. Users love to use multiple fonts for one purpose or another. But the restrictions are there for very good reasons. Mathematical typography has extensible entities such as fraction bars, the line over the radicand of a square root, and a myriad bracket types that grow to fit their arguments. There are many font-size depended parameters that support the varioushorizontal and vertical placements in mathematical expressions. One needs to have a uniform size throughout the math zone to render these things properly. Also mathematical expressions just look better when displayed consistently with the same font. However, one case where it might be handy to use another font on occasion is when the math font doesn’t support the full Unicode math character set, which is the case with Cambria Math. The STIX font does have glyphs for all the math characters. Hopefully it will support the other features of a math font needed for high-quality math typography.

You might want to call attention to some math characters within a math zone. Changing the math style isn’t what you need, since that would change the meaning of the variables. One method that I’ve used over the years in lectures is to distinguish variables by color. For example, three-mode laser light has the frequencies ω₁, ω₂, and ω₃, in order of increasing frequency. To emphasize their relationships, I’d color them and their ray lines as red (ω₁), green (ω₂), and blue (ω₃), since in the visible light spectrum red has lower frequency (longer wavelength) than green, which, in turn, has lower frequency than blue. The laser mode frequencies were much closer together in frequency than those colors, but people understood what was meant. In fact, color is sufficiently important for such purposes that MathML has foreground and background color attributes. OMML doesn’t, but should. Word uses embedded WordProcessingML to represent color in math zones, but this isn’t as interoperable as having color attributes directly in OMML.

RichEdit paragraph indents are based on the original RichEdit 1.0 definitions. These differ somewhat from Microsoft Word’s definitions and hence from TOM’s (RichEdit’s Text Object Model), which were developed in collaboration with the Word team.  In addition, the RTF file format uses Word’s definitions, naturally. RichEdit’s paragraph indents can be set using the EM_SETPARAFORMAT message and retrieved using EM_GETPARAFORMAT both of which use either the PARAFORMAT or PARAFORMAT2 structures. The differences in these definitions have led to considerable confusion over the years, so this post is dedicated to explaining the relationships. These definitions differ, in turn, from those used in math paragraphs, as discussed in recent blog posts.

The PARAFORMAT[2] structure has the indent variables

                LONG    dxStartIndent;

                LONG    dxRightIndent;

The PARAFORMAT2 structure also contains the variable

                WORD   wNumberingTab;

Meanwhile Word, TOM, and RTF have the variables (in TOM terminology) FirstLineIndent, LeftIndent, RightIndent, and ListTab. They look similar to the corresponding PARAFORMAT2 variables, but the first two have different meanings and hence the confusion, since RichEdit clients typically end up using a mixture of the APIs.

First consider cases having a zero value of wNumberingTab, which equals the TOM ListTab. Then the first line is indented by LeftIndent + FirstLineIndent. Meanwhile in PARAFORMAT notation, the first line is indented by PARAFORMAT::dxStartIndent. Subsequent lines in the paragraph are indented by the TOM LeftIndent, while in PARAFORMAT notation, they are indented by PARAFORMAT::dxStartIndent + PARAFORMAT::dxOffset. Summarizing the TOM/PARAFORMAT relationships, we have

Here the third relationship follows from the first two, but it’s included for clarity.

Now consider cases with nonzero wNumberingTab or equivalently ListTab. This value is the width reserved for the list (numbering) text on the first line of a numbered/bulleted paragraph provided its value exceeds both dxOffset (−FirstLineIndent) and the width of the list text itself. In particular, if ListTab < −FirstLineIndent, ListTab has no effect. Since FirstLineIndent is negative in most scenarios (e.g., when you type the Ctrl+Shift+L hot key to turn on a bulleted list), FirstLineIndent often determines the space allocated to the numbering text instead of ListTab. But for a positive FirstLineIndent, ListTab determines the space so long as it’s wider than the numbering text. Summarizing these relationships, the start of text on the numbered line is indented by (using PARAFORMAT2 variables)

                dxStartIndent  + max(list text width, wNumberingTab, dxOffset)

or equivalently with TOM variables

                LeftIndent + FirstLineIndent + max(list text width, ListTab, −FirstLineIndent)

RTF control words for these variables are \fiN (FirstLineIndent), \liN (LeftIndent), \riN (RightIndent), and \pnindentN (ListTab). Here \pnindent is a control word in the original Word list notation. Later control words are described in the RTF specification under the heading “List Tables”.

The OMML (Office math markup language) specification has been improved in response to the great feedback we received from the ISO reviews. It’s part of the full OpenXML documentation Version 2 and is very handy to have if you’re working with OMML. The full OpenXML documentation itself is also very useful if you’re working more generally with OpenXML.

To get the documentation, click on http://www.ecma-international.org/publications/standards/Ecma-376.htm. Then click on ECMA-376 2nd edition Part1 to start a 50 MB download of the file entitled “ECMA-376, Second Edition, Part 1 - Fundamentals And Markup Language Reference.zip”. Browse the zip file and copy “Part 1: Fundamentals and Markup Language Reference”, a 34 MB pdf file to somewhere convenient on your disk drive. Then open that file with the Adobe Reader. In the left pane of the Reader, click on the Bookmarks icon to navigate this huge file.

To get to the OMML documentation, click on SharedMLs Reference Material, and then on Math. You’ll see three entries, “Table of Contents”, “Elements”, and “Simple Types”. You can click on the “Elements” to see each OMML element referenced. Clicking in turn on one of them, you get the full documentation on that element. Similarly you can investigate the "Simple Types".