William Adams' WebLog : A Bit Short

Scheming with Schema

wadams — Fri, 27 Feb 2004 06:00:00 GMT

Pokemon; Ed, Edd, and Eddy; Masters of the Universe; Zoids.

These are the cartoons you get to watch, either because you woke up at 6am, or so happen to be watching at 9pm like I am.

Why am I watching TV right now? Because I was typing some schema code while sitting on my bed in my room, and my battery said it was getting low. My laptop's power cord is downstairs plugged in next to the couch in front of the TV, so here I sit and type.

Laptops are great because they're powerful enough to actually get some programming done these days. I don't have to be sitting at my desk any more.

Speaking of sitting at my desk, as my laptop sat on my desk last week, I wiped out the various source trees, and internal installations of Visual Studio, and loaded Visual Studio 2003 Professional. I debated a long time between this particular release, and putting the latest Whidbey release on my machine. I ended up with this stock version because it's what everyone in the public has. I want to do some coding samples, and I want those samples to be accessible to more people than Whidbey insiders.

In our group, we spend most of our time looking forward. VS 2003 represents code that we first released in 2002, which was originally designed in 1999/2000. But, what does it look like? I was part of the team which this stuff was designed. It seemed like it was all so terrific at the time.

The two most primary design centers were XmlReader and XmlNavigator. Other pieces of the puzzle include DOM, XSLTTransform and the like. XmlSchema came later in the cycle as the XML Schema standard was ratified. That's a whole other story that I'll have to relate some time.

What I'm looking for in particular is how to use the XmlSchema Object. I'm mainly concerned with how I can turn my newly inferred book schemas into working program code for a simple librarian application. There's plenty of fertile ground for this application. I'll let you know how it goes.

In the meanwhile, I was actually trying to use the Schema Object Model, and all I can say is... I'm sorry.

The conclusion I've come to is, using the C# editor to create schemas is not quite the right tool.

What I mean is this. When your schema looks something like this:

<?xml version="1.0"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:group name="testGroup">
  <xs:choice>
    <xs:any namespace="##any"/>
  </xs:choice>
  </xs:group>
  <xs:element name="myElement" >
  <xs:complexType>
    <xs:choice maxOccurs="unbounded">
      <xs:group ref="testGroup" />
    </xs:choice>
  </xs:complexType>
  </xs:element>
</xs:schema>

It's probably easier to write the following:

        string xsd = "example.xsd";

        FileStream fs;
        XmlSchema schema;
            fs = new FileStream(xsd, FileMode.Open);
            schema = XmlSchema.Read(fs, new ValidationEventHandler(ShowCompileError));

            schema.Compile(new ValidationEventHandler(ShowCompileError));

In this case, Notepad has been used as the schema editor, and C# simply loads the result. To create this schema programatically is a lot more involved, and not really that fun. That method is good for tools that are generating schema on the fly. It's not really the interface that I would recommend for programmers who are consuming schemas.

So, maybe it's not that bad. You don't have to use the API to create the schemas, but what about reading it?

Wha? Is that Samurai Jack? Nope, X-Men. Sorry, gotta go!

I'll get back to you.

Posting pictures probably presents problems

wadams — Thu, 19 Feb 2004 17:19:00 GMT

So, I removed them. I'll have to find a better location than MSN to put them so that I can link to them without causing problems.

More random thoughts

wadams — Wed, 18 Feb 2004 07:31:00 GMT

As I was contemplating de treasuring various of my computer books last night, I began to get nostalgic, and I suffered a bit of panic. What if what I wanted was NOT available on the internet? So, I went look.

A very long time ago, I compiled a set of C++ routines and classes that were rather esoteric in nature. You know, the typical library of sort, search, containers, random number generators and the like. I thought for sure I had the library on disk somewhere. I did, but it was formatted as a BFS filesystem (BeOS file system). So, until I get my BeBox plugged in again, I can't really access that disk. That's another story.

I thought I remembered the name, and I'm pretty sure I know my own name, so I went to my little Google search bar and typed in “William Adams bebits ftp”. After some grubbing around, I found it here:

http://linux.inf.elte.hu/ftp/beos/development/samples/support_kit/obsolete/bebits.zip

It's taylored for the BeOS, which is extinct, but only for the thread and spinlock classes. The rest is fairly generic. It might even be interesting to resurrect it as a C# library, but I think that will largely be covered by generics in Whidbey.

Looking back at that library is interesting to me because it reminds me of the different design choices, constraints, and assumptions I was operating under at the time. Things like, C++ compilers did not have reliable template implementations, or STL was not quite standard as yet. Things I wouldn't think about today with the far superior MS C++ implementation. But, they were my assumptions at the time.

I looked at the random number generator that I had in there. Then I thought, “where did I get that code?”. Digging back through my library, I remembered that it came from this book: Data structures and problem solving in C++

http://www.cs.fiu.edu/~weiss/#adspc++2

That book sells for almost $100 in its second edition. How much will I get for my first edition copy? Probably about $3 at the used book store.

But, as I had hoped, the source code from the book can be found here:

http://www.cs.fiu.edu/~weiss/adspc++2/code/Random.cpp

So, I can safely detreasure this book from my library, as painful as it might be. By the way, this particular implementation of a LCG RNG is pretty good because it has a period of 2(32)-1, and it visits all of the numbers. See if your local random number generator does as good, you may or may not be impressed by what you find.

Looking at the stack of books at my feet, I see a lot of history. I also see history being digitized in the age of the internet. Progress is informed by look backward, but is only made safely by looking forward. I will probably detreasure these classics, but I will probably keep the Knuth book for old time's sake.

I did go looking into a few plastic storage boxes I have. I found the entire series of Commodore PET user's club magazines from volume 1, issue 1 onward! That represents magazines that are about 25 years old, and the oldest computer magazines in my library. I can't detreasure them because they'd have zero 'value' today. I think they're either headed for eBay, the Boston Computer Society, or the Smithsonian. Who knows, perhaps I'll preserve them for my daughter so she can dispose of them when I'm old and gray.

For Arpan: To 'Osborne' your product, look here: http://oldcomputers.net/osborne.html at the bottom where it talks about the Executive and Vixen releases.

Random What? Mersenne who?

wadams — Tue, 17 Feb 2004 07:30:00 GMT

What do these books have in common?

CRC Standard Mathematical Tables and Formulae, Numerical Recipes in C, Seminumerical Algorithms, Applied Cryptography, The Twofish Encryption Algorithm

For starters, they are all published earlier than 1999, mostly before 1996, and in one case, 1969. They all cost about $50 apiece. They all have information that was probably at one point or another classified as munitions by the US government. In my computing history, they have all led to my understanding of encryption, random numbers, and other esoteric subjects required to do some basic computing tasks.

Why do I look at them now? Well, this is one of those cases where all the information they contain can be found on the internet in one form or another. So, it's time to de Treasure them. But, before they go, I thought I'd look at some of the bookmarks that I had made and see what I was interested in almost 10 years ago.

From the CRC, I find a bookmark on the page headed:

7.5 RANDOM NUMBER GENERATION

7.5.1 METHODS OF PSEURANDOM NUMBER GENERATION

Linear Congruential generators.

Turns out, you can generate pseudo random numbers by a simple formula. Pseudo random because they're not random at all. You can predict exactly what the next number will be based on the current state of the calculation, and the initial state of the calculation.

Xn = aXn-1 + b (mod M)

That is, a 'random' number Xn can be generated by taking Xn -1, multiplying by 'a', adding 'b' and modulus 'M'. It works out nicely because it's a simple multiplication, and add, and some other little thing at the end. Really fast stuff. Depending on what numbers you pick for a, b, M , and the initial seed, you can generate these numbers rippin fast on modern computing hardware.

What's a Mersenne prime? I don't know, I'm sure you can look it up on the internet. It turns out that if you use one of these as the 'M', you get really fast computations.

One concern with LCG RNGs (Linear Congruential Random Number Generators) is the periodicity. That is, how many times can you generate a number before the sequence begins to repeat itself? The trick is in the numbers you choose.

There are plenty of tables for what a, b, and M should be

a b M Source

7(5) 0 2(31)-1 Park-Miller

131 0 2(35) Neave

16333 25887 2(15) Oakenfull

3432 6789 9973 Oakenfull

171 0 30269 Wichman-Hill

For those who are in the know, I'm sure these names mean much, and a hush goes over the audience when they speak. For most of us though, the intricacies of LCG constants are lost in the bowels of the rand() function. Most people neither know, nor care about the details. And that my friends is the beginning of all our troubles with writing secure software.

What about shift register generators, Lagged-Fibonacci generators, and chained LCGs? What kind of random numbers are you consuming in your applications? Do you care about the distribution of the lower bits? Do you care that your numbers may overflow before the modulus? Do you care that you are losing some precision when you're flipping between int and float?

These are little tidbits from the CRC. A book chock full of tables, formulas, and the like. I used the CRC when I was in high school to look up log tables. And I'm sure it's been in existance in various forms for a couple hundred years before that.

Donald Knuth goes over the top in “Seminumerical Algorithms”. Knuth spends an unprecedented 177 pages on the subject of random numbers, random sequences, and anything esoteric with the word 'random' in it. Can you believe it? 177 pages? Surely the spec, docs, code, and tests for the typical rand() function do not take up that much space. I'd guess they typically take up no more than two pages worth of text layed end to end.

What's the point? I don't know, I guess I'm just being a little random. How random you can't possibly know unless you understand where I started, and under what conditions I change the subject.

As I de treasure these books, I can't help but think that most programmers using modern programming languages today, aren't likely to amass such a library, nor understand the esoteric little pieces embodied in the deep literature. Maybe we've gone beyond the point of having to care about such things. To a certain extent, this is true. The general computing public does not need to care about the multipliers that are used in the LCG found deep within our rand() implementation. But, I hope the systems programmer who wrote that code sure understood what they were doing, and I hope they didn't de treasure their library of knowledge too soon.