Welcome to MSDN Blogs Sign in | Join | Help

Browse by Tags

All Tags » System.Text   (RSS)
Tuesday, August 18, 2009 10:30 AM

What is Title Case?

Disclaimer: I'm not an English teacher (that's my mom), so I'm sure my description of title casing in English probably has exceptions/variations. Title casing has an interesting history in computer programming. Programmers like to use CamelCase to make
Posted by shawnste | 2 Comments
Monday, June 01, 2009 5:38 PM

Writing "fields" of data to an encoded file.

The moral here is "Use Unicode," so you can skip the details below if you want :) A common problem when storing string data in various fields is how to encode it. Obviously you can store the Unicode as Unicode, which is a good choice for an XML file or
Posted by shawnste | 2 Comments
Friday, June 08, 2007 7:20 PM

Why can't we strip the diacritics?

We have some "best-fit" behavior which we generally consider to be "bad". Any loss of data is generally a bad thing, so we recommend storing data in Unicode (so you don't lose anything). Assuming you can't use Unicode, why is it so bad to just make everything
Posted by shawnste | 5 Comments
Thursday, June 07, 2007 1:19 PM

Encoder/Decoder Encoding fallbacks fail after 2GB of data has been converted

We have an unfortunate bug in .Net v2.0+ that causes encoding or decoding of more than 2GB of data to fail. That's a lot of data, but it still shouldn't do that. This is a problem with our built in fallbacks. Ironically, if you encounter bad bytes then
Posted by shawnste | 0 Comments
Thursday, May 03, 2007 1:29 PM

How do I get HKSCS 2004 characters from Big-5 in .Net?

Well, that's pretty tricky. We provide the Microsoft Character Code Conversion Routines For HKSCS-2004 functions, but those are intended for use with unmanaged code. The fundemental problem is that these "HKSCS" characters were in use prior to the assigment
Posted by shawnste | 3 Comments
Tuesday, May 01, 2007 11:13 AM

Please avoid UTF-7

UTF-7 inherently some of the security issues that concern people about encodings. For example, by shifting in & out of the base64 mode one can create multiple representations of the same string, enabling spoofing and other problems. UTF-7 is primarily
Posted by shawnste | 1 Comments
Tuesday, March 20, 2007 12:58 PM

Some Reasons to Make Your Application Unicode

[Updated Mar 30 2007: Mike pointed out errors which I've corrected] Many applications are "still" ANSI and can't handle Unicode. We (Microsoft) have even released non-Unicode applications reasonably recently. even though we should know better. In particular
Posted by shawnste | 3 Comments
Tuesday, March 13, 2007 11:29 AM

A History of Code Pages or What Made Code Page XXXX (or many other computer things) The Way It Is?

Disclaimer: This is mostly my conjecture, so I could be completely wrong about some of this, but it seems plausible to me. I’m aiming for the general concepts here, not to start a discussion about the specific details of the history of code pages. Taking
Posted by shawnste | 1 Comments
Monday, November 06, 2006 3:52 PM

Expected names of Microsoft Windows "ANSI" Code Pages (Encodings)

I was asked about our use of the windows "ansi" code page names, as used in things like MIME types, http content-type tags, etc. Each "code page" has a name that most accuratly round trips back to the same code page, which I've listed as the "preferred
Posted by shawnste | 0 Comments
Thursday, October 12, 2006 4:00 PM

Example of overriding your own Encoding.

Previously I wrote about the Best Way to Make Your Own Encoding , but didn't include an example, so today I'm including an example of a replacement Encoding. I also included an EncoderFallback example, which replaces unknown characters with numerical
Posted by shawnste | 7 Comments
Attachment(s): EncodingSample.cs
Friday, September 29, 2006 3:06 PM

Best Way to Make Your Own Encoding

Martin recently asked what the best way to roll his own encoding in .Net 2.0, in particular can you override Encoding/Encoder/Decoder, or should he write his own StreamWriter. #1 is, of course, to use Unicode :), but apparently Martin doesn't have that
Posted by shawnste | 3 Comments
Filed under:
Tuesday, July 18, 2006 10:41 AM

Encoding.GetEncodings() has a couple "duplicate" names

The Microsoft.Net v2.0 Encoding.GetEncodings() method returns a complete list of supported encodings, uniquely distinguished by code page. Note that in general I consider the code page number to be a poor way to exchange code page information since its
Posted by shawnste | 4 Comments
Wednesday, June 21, 2006 5:38 PM

What's with Encoding.GetMaxByteCount() and Encoding.GetMaxCharCount()? Part 2

A little over a year ago I wrote What's with Encoding.GetMaxByteCount() and Encoding.GetMaxCharCount()? to address the question "Why does GetMaxCharCount(1) for my favorite Encoding return 2 instead of 1." (Short answer is that the Decoder/Encoder could
Posted by shawnste | 1 Comments
Filed under:
Friday, June 16, 2006 3:07 PM

Change to Unicode Encoding for Unicode 5.0 conformance

The behavior for UTF8Encoding, UnicodeEncoding and UTF32Encoding has changed in Windows Vista to conform better to the Unicode 5.0 requirements for Unicode Encodings. [23 July 2007: Now this behavior has also been made to .Net 2.0 with MS07-040 update
Posted by shawnste | 3 Comments
Thursday, January 19, 2006 12:16 PM

Best Fit in WideCharToMultiByte and System.Text.Encoding Should be Avoided

Windows and the .Net Framework have the concept of "best-fit" behavior for code pages and encodings. Best fit can be interesting, but often its not a good idea. In WideCharToMultiByte() this behavior is controlled by a WC_NO_BEST_FIT_CHARS flag. In .Net
Posted by shawnste | 6 Comments
More Posts Next page »
 
Page view tracker