The Wayward WebLog : XML

Xen meets its Omega

mattwar — Wed, 14 Jul 2004 16:11:00 GMT

I got a surprise in my inbox this morning. An email from Erik Meijer read:

The impossible has happened: X# became Xen, Xen became Comega, and Comega has shipped.

http://www.research.microsoft.com/research/downloads/default.aspx.

Enjoy

Matt

X#: There and back again

mattwar — Fri, 26 Mar 2004 19:43:00 GMT

It looks like X# might just see the light of day afterall, as the foundation of a even more ambitious research language from Microsoft. This article actually gets most of all the facts right. Except it doesn't mention my brilliance anywhere.

X# == Xen == C Omega == X Omega

If you have a good idea for a new c# based language, just make up an appropriate sounding name and add it to the list!

Matt

XML: Schema Breeds Intolerance

mattwar — Thu, 25 Mar 2004 06:31:00 GMT

I was at an offsite a while ago where a bunch of us got together to talk about some technology thing, some potential product or what not, that may not see the light of day for a while. I had a chance to give a presentation on some of my forays into data query languages. Of course, there were other presentations too, and mine was not the first, so as I sat there admiring the Hotel’s wireless net connection, googling for random words, thinking about work related things, and paying full attention to the current talk by making use of my many co-processors, out of the corner of my eye I notice a slip of paper with tiny square stickers. They were our parking validation stickers, left by the hotel management, which is an important thing if you don’t want to get stuck paying to get your car out at the end of the day.

Now I have a whimsical mind that favors pedantic puns, and so it is often spinning extra cycles in the background on odd word associations. One of the reasons my wife often asks what’s on my mind is that she knows there’s usually something brewing, and of course she is right because quite often for apparently no reason I will just start spouting some obscure joke or quip. She’s a good sport and tries to laugh at most of them, though it also makes her wonder how much time I actually spend trying to dream them up, but that’s not how it works at all.. They just sort of pop up out of thin air. I’m telling you it’s the co-processors. They are always churning out stuff.

Anyway, my bank of x86’s apparently slid the word ‘validate’ over to meet the word the ‘data’ in a mental re-enactment of a bad episode of the electric company and I get spun in the direction of thinking about XML Schemas. Now, just tell me the truth, wouldn’t you? Of course you would. That’s the first thing that comes to mind when thinking about data validation. You do validate your data, don’t you? I mean, you wouldn’t want just any old data to come slinking in off the street and set up shop in your application? Would you?

To tell you the truth, I’m not sure anymore. I mean, there are plenty of good reasons to have schema’s around. They describe your data layout. For things like WSDL, they give you a ‘crisp’ contract for describing how data is passed between interfaces, etc. You can feed them into tools that do code generation to build you proxies and data structures. But this is all really just static stuff you do with schemas. It’s not really runtime stuff.

You probably don’t use the schema information itself at runtime. If your app is sucking in data using the XmlReader or the DOM, then your app is probably structured around the data you expect anyway. It probably reads and recognizes particular tags in particular namespaces. It probably queries into documents looking for those very specific names. So what if there might be stuff inside that your app does not recognize. Isn’t that the whole point of XML in the first place? Doesn’t it allow you to build applications that are resilient to change and that rare unexpected integer? What good does it do to actually pay the overhead of validating your data stream against the actual physical schema when your app is only going to grovel over what it understands anyway?

It seems that getting overly anal about schema just leads to making apps brittle, and that should be a bad thing. We should work on making apps more tolerant of each other. Live and let live, I say.

Matt

Navigating the XPath of Life

mattwar — Thu, 18 Mar 2004 06:14:00 GMT

Brian Noyles blogs about the XPathNavigator, and how it has improved his life.

I spoke last night at the Central Pennsylvania Users Group in Harrisburg, PA. They have a great group and about 35 folks turned out despite snow and nasty weather. Judy Calla is the group lead and gave a nice little beginners talk on debugging and error handling in .NET applications. I then jumped in with a talk on querying XML data in .NET.

The primary function of the XPathNavigator is to act as a common plug-in point for xpath/xslt query processing. It is the query equivalent to the XmlReader. We built the XPathDocument pretty late in the game right before .Net framework 1.0 shipped. It was basically a perf thing. We were beating ourselves up trying to get better perf out of the DOM, but were reaching the limit to what we could do with that model. It was when we realized that most of what was slowing us down was baggage from the DOM like sequences of text nodes and entity expansion, things that had no representation in the XPath model. The XPathDocument is optimized for xpath query because it gets rid of anything that doesn't fit, so there is no impedence mismatch.

It also didn't hurt to assign identity to each node in increasing order right a load time. This makes it incredibly fast to keep the nodes straight, especially when wacky coders start using strange axes. Because the XmlDocument was editable, it couldn't keep a zero cost ordering scheme. Let's hope the new XPathDocument stays as fast, even with its new editing capability.

See, I can actually post something that links to someone else's blog. Who knows, yours may be next.

Matt

XML: Getting dirty with BMX

mattwar — Wed, 10 Mar 2004 19:48:00 GMT

Years ago there was a time when I thought XML was evil. That was back when the whole idea was pretty new to my ears. The only introduction I’d had to it was a few examples being shown around by Adam Bosworth when he’d come to our group and try to convince us to redo ADO and SQL server to support XML as a transport. Of course, I thought it was a really bad idea at the time. After all, we had just gone through a lot of pain enabling Unicode in the transport layer and we were getting a lot of flak for it. No one liked the downside that it took more bandwidth.

But Adam persisted, and I became notorious. He would show up at a meeting trying to convince our dev manager or product unit manager and see me there along with everyone else. “Oh, it’s you again,” he would snarl at me as he entered the room.

Adam would remind us of all the benefits of XML, how it was unstructured but discoverable. At the time I wondered how this was really any different than HL-7, but mostly I was skeptical how we would benefit by replacing a binary format that we owned on both ends. Still, I was an engineer, so as an engineer I tend to try to solve problems. So I took what Adam described as the strong points of XML and was hell bent on developing an equivalent binary format that worked instead.

I put a lot of effort into trying to make certain that it could both capture hierarchy and be fully discoverable and even contain tags, but at the same time, if your data was highly regular then the additional bandwidth cost would be nominal. When I presented this gem to Adam and his team it was immediately shot down. Apparently, I didn’t get it. XML was text. Andrew Layman assured us that the W3C or someone was busily working on an official binary format, and that it was a heated debate. We certainly shouldn’t waste any effort duplicating that or trying to invent or own proprietary form of XML.

So I put the spec aside and forgot about it. Eventually I even lost the files due to a mishap and so no record was left that I even attempted it except for the idea still floating around in my head. Of course, a few years later, a strange twist of fate had me developing parts of the XML frameworks for .Net. By that time, I had changed my mind. I had forgotten about the bandwidth problem, as I just didn’t care about it anymore. Pipes got fatter. Size was not really an issue.

Yet, then came the long death-march, the stabilization period for version 1.0 of the frameworks. We were fixing bugs like crazy and doing tons of perf work. We met the goals we were shooting for, but I always thought we could do more. At this time I started up an internal effort called ‘TenX’. TenX was focused on taking performance to the next level. The idea was to throw out any pre-conceived notions about how to solve the particular problems that plagued us. The goal was to do miraculous things that would improve performance by an order of magnitude. Through this effort I encouraged most of our top developers to think outside the box and find new solutions. Maybe the solutions would only solve edge cases, but even edge cases are useful to someone.

One idea I had was to improve the speed of the XML parser. Sure, I made many attempts at re-writing the parser itself, and one day I’ll even share a faster parser with you. But I also had this crazy idea that if the XML was already mostly parsed, it would scream. You see, I knew that much of the cost of parsing came from doing various validations and fumbling over characters one by one. If I could throw out most of that, maybe I could double or triple the speed of reading it in. So I went back to the old binary XML idea. If I could invent a format that was easier to parse than XML, it would be a big win.

So I revived BMX and went to work making a prototype that I could put to the test. What I came up with was very different than the original. It is actually just a tokenized stream, because decoding tokens is extremely fast and why mess with the text itself? The big news is BMX encoded files read faster than XML text, approximately 10 times faster!

I’m happy to be able to share this with everyone. Feel free to criticize my programming skills if you’d like. There is extra stuff in there that I was experimenting with but not entirely germane. There are also many functions that are stubbed off with not-implemented-yet exceptions.

Take a look at the code. It's here all crammed into a single file for your downloading pleasure. All the comments with foul language have been removed.

Enjoy

Matt

The XML Conspiracy

mattwar — Wed, 11 Feb 2004 07:37:00 GMT

I really shouldn't be telling you this. It will probably get me into a lot of trouble. Still, its not like a lot of people are reading this, so I might as well open up for once and tell you the straight scoop. XML was a hoax. This is real. Not many people know this.

Years ago, before I came to work at Microsoft, I used to be one of those people that liked to grumble about technology, especially things about the operating systems I didn't like. Others at the company where I worked were liked minded, so often we sat around at lunch and griped about whatever was on our minds. Now, it wasn't always griping about lame hardware or software, sometimes we griped about what we disliked about our own jobs. Ya, I know, hard to believe. We'd often get into debates over coding styles and programming languages. Whenever someone brought up a new programming language they knew about, I aways fought back and said things like “LISP could do that years ago. What's so new?” These debates went on for hours.

One day a very particular topic arose. Many of us had just read in PC Week about a change Intel was making to their 486 processor, to bring out a lower end model. Apparently, they took a little laser and zapped out the math co-processor. So in effect, you got a 486 chip with all the modern instructions, you just got sucky math. I think they called it the 486SX, where SX stood for 'sucks.' Anyway, I recall being livid about it at the time, and that's when I had my unfortunate slip of tongue. I shouted, red faced, “They might as well bring out a low end version of LISP. Sure, just burn out the execution engine!“ That got a few laughs from most everyone. Except this one guy, Paul. Well, to tell the truth, I don't recall anytime when he did laugh. He was one of those guys who was always over-serious, over-passionate about any topic. He just looked at me and waited for the laughter to die down, and then he said, “what do you mean?“

So I guess I just started to ramble off an explanation. I just made it up on the spot.

“You know,“ I said. “Take out all of the operators.“

“Why would you do that?“ This was one clueless dude.

“So you'd just have the data. Now wouldn't that be dumb.“

You see, one of the really cool things about LISP is that code and data were stored in the same representation. LISP had this really simple form used to describe both. A LISP program was basically a bunch of nested lists of identifiers. Each list was surrounded by a pair of parentheses. If you imagine back to those really ugly examples of type casts done in C programs where you just had parens going everywhere, then you'd start to see what a LISP program looked like. It was a mess to the untrained eye, but to me it was beautiful. It could describe everything.

Why this was so ridiculous to me, was at that time the computer industry was just then coming to terms with the new idea of object-oriented programming. The whole benefit was being able to couple behavior with data. The holy grail of the entire industry was a phantom chase after something that already existed in its purest form; LISP. So you can see the joke. LISP without code was just plain ridiculous. It was like rolling back the clocks. We might as well go back to computers with punch cards.

But Paul never got the joke. He was one of those guys in the office that just never understood the humor in a Farside comic. He thought I was being serious. He would come back to me every now and then and asked more questions about code-less LISP. So I strung him along. Everyone thought it was a good gag. Still even Paul eventually started to doubt what you would use this codeless LISP for, so in spirit of a good prank I kept feeding him tempting justifications.

“Look,” I would tell him. “Even though there's no code there, you can still manipulate the data using another language. Its a perfect text representation of information, and you don't even have to build a parser, it comes built into the codeless LISP engine.”

Sometimes I carry a joke too far. One day I will tell you about the satellite tracking system, or my brother will since he seems to read my posts a lot and he was in on that one. I wanted to show Paul the magic of a truly running codeless lisp engine. So I took the public domain XLISP source code and just sort of yanked out all the operators, and made the 'quote' function the default. Then I slapped a couple of exported functions into it and built it as a link library.

“See,“ I told Paul when I was done, “all you have to do is link up to the XLISP library, and you can feed it your files at runtime, what you get back is this tree structure and a bunch of functions for manipulating the tree.“ I had some sample 'data' in a '.XLP' file. It always makes things seem more 'real' when you go to extreme levels of detail like making up a new file extension.

I wrote a sample program that showed him that I could 'read' the data w/o first knowing what its structure is. “Wow,“ was all he could say. And even the sample program was a joke. There was no such thing as not-needing to know what the structure was. So what if I could 'read' it into memory. My code still had to know what to expect.

Eventually, I just couldn't take it anymore. Even I had a limit on how far I could dupe someone, so I just stopped. I didn't try to convince him it was a joke. He wouldn't have believed me. So I just stopped talking about it.

Years later, I was working at another company when I got a call from Paul. He had tracked me down through another co-worker that I had kept in contact with. Apparently, he wanted to know if I still had that sample program, and the codeless XLISP. I had to dig it up off a 5 1/4, but I found it. Apparently Paul was now working with this guy named Tim, and Tim was even nuttier than Paul. Tim had thought this data-without-code thing was a great idea. They were starting up some 'standards body' to manage its design.

I sent them the code, but I made them promise to leave me out of it and to call it something else. After all, one day they'd eventually see how stupid it all was and try to blame me.

But I digress

Matt ;-)

The XML Pull

mattwar — Sun, 08 Feb 2004 06:03:00 GMT

What's all the hubbub about Microsoft's XML patents? It's not as if the company is trying to patent XML itself. You'd think by all the noise being stirred up over it that Microsoft was trying to steal milk from babies. And believe me, they are not. In fact, Microsoft GIVES milk to babies. When my son was still a baby he frequently received free milk from Microsoft. It's right there in the coolers with the free Coke. On days that I brought him to work, if I was a little busy entering loopholes for viruses into the operating system he might take it upon his own initiative to crawl down the hall to the kitchenette where he would grab himself one or two cartons of delicious, all American FREE milk.

Now you might be wondering if he stuck to just milk alone, because believe it or not, there is free juice as well; apple, orange AND berry. It would be a fair assumption to make, since children tend to like juice as much as they like milk. But Charlie, he likes his milk, and he likes it warm. My wife fooled him one day when he asked for hot-cocoa. Instead she just gave him warm milk. He was a pleased as punch. Which is sometimes difficult to do when there is no actual punch involved. So that leads you to the next logical question. Just how does he operate the microwave to heat his own milk? Luckily, he was not often the only baby in the kitchenette getting milk. Sometimes when a group of three or more are there together, they team up and build a human (albiet babyish) ladder up to the top of the counter. Charlie was a smart little guy at nine months. He knew and could use over twenty different sign-language signs and could operate the VCR. Believe me, pressing the 'cook 1 more minute' button on the microwave was easy to teach him. We usually let him do it at home anyway, and that one has many more buttons. You should have seen when he tried to defrost an ice-cream sandwich. It was a hoot.

Sometimes, watching him figure things out for the first time gave me ideas. For example, when he first figured out that the nesting boxes had other boxes inside, and so on. It brought a smile to his face, as well as mine. I immediately saw the simplicity in the recursion, and that led me to a big break-through in the design of .Net XSLT engine. I began to see signs in other toys that he played with. The DUPLO was a dead give away. Interfaces between components should be interlocking. You should have been there when the light-bulb went off in my head. He was pulling a rope tied to a small wooden wagon. The answer seemed obvious. The .Net XML components should rely on pull-model parsing. Any baby knew it was easier to pull than to push. All XML components should interlock amongst each other using a standard pull-model abstraction. It was brilliant. I began to wonder if he was somehow trying to communicate this all to me and more.

I knew what I had to do. I went to work and the rest is history. The .Net frameworks shipped with Charlie's innovative ideas. I knew that one day he should reap the reward for what he had contributed to computer science and the world. I wrote up patents for these XML related technologies and that was that.

Of course, I was duped. The fine print in my employment agreement spelled it all out. When I put my John Hancock to the document, I ignorantly agreed to hand over all my inventions to the company. They got the patents from me without even a struggle. Sure they deposited a small sum into my bank account, but all I really got left with was a few black-lucite cubes in my office with the names of the patents pending. My son was barely a year old, and already he had gotten scr****d by the MAN.

Uh, so maybe they did take milk from a baby afterall.

But I digress

Matt