Barry Gervin posts some interesting information he derived about Datasets. While it's true that datasets are serialized as XML (passed around that way between tiers, etc), they aren't represented that way in memory...

http://objectsharp.com/blogs/barry/posts/284.aspx