What type to use for code page values

Sorting It All Out — Sun, 29 Apr 2007 10:21:41 GMT

I was asked via the Contact link: Why does CharPrevExA/CharNextExA take a WORD for code page, whereas

re: Is CharNextExA broken?

Michiel — Fri, 20 Apr 2007 14:13:25 GMT

First rough solution:

1. IsDBCSLeadByteEx is documented as returning non-zero if it's a lead byte. Make it return the number of characters following for a lead byte. (By definition 1 for DBCS)

2. Update its documentation to say "A lead byte is the first byte of a character sequence in a double-byte character set (DBCS) or multibyte character set (MBCS) for the code page."

3. CharNextExA uses the returned number of characters from IsDBCSLeadByteEx.

This approach takes advantage of the fact that UTF-8 was designed explicitly to determine the number of bytes in a character sequence from its lead byte.

CharPrevExA is harder, as IsDBCSLeadByteEx has to return 0 for both single-byte characters and the non-lead-bytes. And you can't decrement twice without risking to underrun a buffer (the caller must obviously make sure there is one preceding character, but not two). Hence, only one byte is available. If that's a single-byte character, we're done, else we search back until we find the first lead byte. The remaining problem here is determining single-byte characters.