Mike Christensen: Web Dev Guy : General

Castle ActiveRecord

mikechr — Tue, 18 Mar 2008 10:07:00 GMT

Lately, I’ve been playing around with Castle ActiveRecord tolearn about this particular technology. For those of you who are unaware, Castle ActiveRecord is an open sourceimplementation of the ActiveRecord pattern written in .NET and built onNHibernate.

The ActiveRecord pattern represents rows in a table asinstances of a class (with column values as instance methods) while providingaccess to the table itself as static methods of the class.

For example, a database table named Users might berepresented by a class called User. The User class would have a static method called FindRow which takes ina numeric value corresponding to the primary key of that table. User.FindRow(5) would return aninstance of a User object, which would have various properties such as Name.

The benefit of this is it allows developers to access andmanipulate data in a database while providing a layer of abstraction from theactual database syntax. This codecan be database agnostic and hide the user from SQL queries and what not. It also allows simplifying databasecode and being able to perform basic database operations in one or two lines ofcode.

One of the benefits of Castle ActiveRecord is you can defineyour entire database schema using .NET code and even have the engine create thedatabase for you. For example, tocreate a User table, we could use the following code:

[ActiveRecord(“Users”)]

public class User : ActiveRecordBase<User>

{

string name;

[Property(NotNull = true)]

public string Name {get {return name;} set {name = value;}}

}

Using various attributes, you can specify database typemappings, primary keys, foreign constraints, indexes, etc. When you’re done, you can instructActiveRecord to create the database using a given connection string.

We can now create a row in this table with the followingcode:

User user = new User();

user.Name = “Mike”;

user.Create();

This eliminates the need to use ADO.NET or construct SQLstatements or stored procedure calls.

One interesting thing you can do is specify the relationsbetween tables. If I had an Ordertable, I can specify a User that placed the order:

Public class Order

{

….

[BelongsTo(“Name”, NotNull = true)]

public User PlacedBy {get {return _placedby;}}

You’ll notice this method actually returns a User object, notjust an int or a guid or what not. When an Order row is queried, Castle ActiveRecord joins in the Userstable and also populates the PlacedBy instance of the order. You can enable a feature called “lazyloading” which will not query the Users table until the property getter isaccessed. This would prevent thegeneration of queries with lots of joins.

So far, I’ve found Castle ActiveRecord to do a great jobhandling simple database operations for me, however it definitely has itsdrawbacks. I’ve found it lackingin creating any sort of complicated database schema, as it doesn’t seem to beable to create indexes on foreign keys or allow the user to specify clusteredvs non-clustered indexes. However,it can also generate a SQL script to create your database and output to a fileso you can make changes before provisioning your database. For more complicated scenarios, I wouldnot use Castle ActiveRecord for database provisioning.

There were a few instances where I wanted to be able toreturn only certain columns from the database and control exactly what tablesget join’ed in. It seems CastleActiveRecord wants to return all columns all the time, or no columns and generate new SELECTstatements every time you access a property. It would be nice if we could simply specify what columns wewant to query for, and the remaining properties would remain null. For these scenarios, CastleActiveRecord can create an IDbCommand and you can fire off your own storedprocedure or SQL statement. I’vefound using a mix of regular ADO.NET and ActiveRecord calls for the simplerstuff is a great approach for more complicated applications. The ActiveRecord base class also allows you to override various methods, and pretty much any part of the system can be rewritten.

One thing that has disappointed me is it seems support fortransactions is completely broken! I can create a transaction like so:

using(TransactionScope t = new TransactionScope())

{

//Do some DB calls

t.VoteCommit();

}

In theory, if the code crashes, the transaction will neverbe committed and the Dispose method of TransactionScope will roll thetransaction back. However, when Itried this, I found the SQL code isn’t getting run and no SQL errors would begenerated until the Transaction is disposed, thus making it impossible to rollback the transaction due to database errors. On thesupport forums, I found several other people who ran into this problem but noone had a solution for it yet.

Overall, I think this technology is a great model for dataaccess using .NET applications and works perfectly for web applications aswell. It seems they need to ironout a few minor problems, but they have a large following and an active supportcommunity. It’s also open sourcewith a very flexible license, which would allow large applications to run ontheir own customized version of Castle ActiveRecord. I think the pattern is very solid and at the very least,should give you some ideas for creating a database access layer from scratch for yourweb based applications. I’d bevery interested in hearing from anyone using Castle ActiveRecord in theirapplications and what they like and don’t like about it.

For more information, visit: http://www.castleproject.org/activerecord/index.html

Mike

Housing prices

mikechr — Tue, 12 Sep 2006 04:42:00 GMT

When I moved to the Seattle area over eight years ago, I knew I wanted to make this place my home. Since it was clear my residence would be permanent, it seemed like a good idea to buy something rather than rent. Back then, I found a nice condo for around $130,000 in the Redmond area. At a decent interest rate, a 30 year fixed rate mortgage with a 20% down payment was about $680/month. It was a no brainer! I was paying almost $900/month for an apartment, and had I not moved they were increasing my rent to $1090. I could have a bigger place for cheaper, and build equity I could later use as a down payment on a house. I also owned the inside walls and could do stuff like remodel the kitchen, paint, install hardwood floors, etc. Back then, the only reasons most people continued to rent were 1) they didn’t plan on being in the area very long or 2) they had lots of debt or credit problems preventing them from getting a home loan.

Now days, it’s much different. My condo is worth about $240,000 and it’s considered a pretty low end place. Most of the condos in the Kirkland area are $300k-$400k, and if you want down town Seattle you’ll pay even more. With interest rates starting to climb back up, it’s harder to justify paying $2,000/month or more for a condo. Gone are the days of using a condo as a “stepping stone” between an apartment and a house. The concept of “owning is cheaper than renting” is gone, which scares away most people – especially those who aren’t too concerned about their financial future.

It seems the solution to this over the past couple years has been to keep lowing the interest rates until people can afford houses. But has this just made the problem worse? Most people look at a house in terms of what it costs them per month. I have x dollars per month that I can afford as a house payment. What does this buy me? As interest rates drop, the value of the house that money can buy me goes up. This increases the buying power of the average potential home owner. With so many people moving into the area over the past few years and more buyers than sellers, this allows people to offer more money on a house and causes the market value of houses to go up. We’ve been seeing this for a few years now, and other states like California and New York are also no stranger to this. When the point comes where people are having a hard time affording a house, banks do stuff they’ve never done before. We have 40 and 50 year mortgages now. We have 1-year adjustable rate mortgages. We have interest only loans. Banks do whatever it takes to lower your monthly payments to attract your business, all while housing prices continue to increase.

Those who cannot afford to buy a home have to rent. Land lords have to raise rents to afford their mortgages, so rents are going up as well preventing people from saving any money. In San Diego, a lot of apartment owners are getting the idea to turn their cheap apartments into low cost condos. By low cost, I mean only $400,000 or so. This lowers the availability of apartments thus driving the cost of other rental units up as well. This ends up pushing poorer people out of the area.

So what’s the effect? Those who can get their foot into to the real estate market continue to get richer while those who are forced to rent get further and further behind. Is this creating a wider social gap between the rich and the middle class? What can be done about this? Obviously, raising interest rates would be a problem. You don’t want the house you bought a year ago for $500k to all of a sudden only fetch you $300k because none of the potential buyers can afford that mortgage. Is this simply capitalism at work and a normal aspect of our economy, or do all Americans have the right to be able to afford a house some day?

I was very lucky to “get in” to the housing game before all this non-sense started, and due to the appreciation of my condo I was able to make a nice 20% down payment on a home and have some extra money left over to invest. I think I made a good decision to buy something seven years ago, but what about those who are just starting their lives now? Some people push themselves to their financial limits to buy the cheapest possible shack and become “house poor”. Yes, they have their house but they have to live off dry noodles every night for dinner.

I was talking to a real estate agent friend of mine and she said the majority of her clients buying their first homesare getting the down payments from their parents, or a family member. Does this lead to a world where social status is inherited?

I’m very interested in any comments you guys have.

Mike

Housing adventures

mikechr — Thu, 07 Sep 2006 00:45:00 GMT

I must apologize for my recent lack of blogging, but I’ve been most busy attempting to become a home owner. This is a dream I’ve had for years now, but it was put on hold for a few years due to the most rediculous litigation surrounding my condo assocation. One day, the association realized they were in $9 million dollars in debt and this was a problem. Due to this unfortunate financial circumstance, my monthly home owners dues went from $117 to $330. Since the corporate charter said they could not increase the dues by more than 10% a year, they were immediately sued by all sorts of people. The prices of the condos in this complex plummeted and I was forced to stay unless I wanted to take a $20,000 loss. After a few years, the association eventually won all the lawsuits (really good lawyers?) and the prices are finally starting to get back up to the same level as other similar condos in the area.

I decided to resume my dream of owning a house, however I became very interested in building it myself. That way, I could get it just the way I wanted and use high quality materials. I wanted something small and nice which is hard to find. So I started looking into building a house – the first step is, of course, to find land.

Now, there’s a difference between a piece of land and a “lot”. Land is simply any piece of dirt someone is willing to sell you. This land might be on a slope, covered with brush, or be in some way totally unable to support any type of construction. Of course, you don’t know until you actually do a feasibility study, which could costs thousands. Lots would already be prepped and ready to build on. A lot is tough to find unless you want a microscopic little tenth of an acre parsel in the middle of some yuppyville housing community. Land around here goes for around $120,000 for anything decent. Figure another $80,000 for leveling, escavation, clearing, permits, environmental studies, bribing public officials, etc. So I figured I’d be looking at $200k before I could even start building anything. So I called about 6 real estate agents looking for someone who would help me buy land. None of them returned my phone calls. It seems about 60% of land deals fall through and no one wants to get in that business. In fact, the only people that build houses anymore are insanely rich people or major housing development corporations who build .1 acre lot yuppyville homes with the cheapest materials and give you a list of about 3 options to choose from. This is not my thing. So once you have the land, then what? Well, you can buy a manufactured home. This is by far the cheapest way to go. You get these homes from companies such as Hiline Homes (www.hilinehomes.com) and they show up one day on the back of a rather large truck. Those go for about $100k for a decent one, mind you they have incredibly undesirable floor plans. If you want something nicer, you can go with a cute little log home from someone like Lindal Cedar Homes (www.lindal.com). I love these! These things show up on another rather large truck in pieces and you hire someone to put them together for you. The cost is around $110/sf. So figure $200,000 for a good size home. However, this is just the outer “shell” of a house. The walls, ceilings, doors, little metal bracket thingies, etc. Once you’re done with that you’d have to hire a general contractor to fill in the innards which might cost you another $100k. Third option - if you really want something nice, you have to hire an architect to design you something from scratch and have it custom built. Custom built houses in this area go for around $300/sf. So figure a nice little 2,000sf home would run you $600,000 just for construction. Adding up these numbers makes building a house really push the budget of any average person. So I decided to give up on this and look for an existing house that I liked. One problem, I wanted a basement.

Why a basement? Well, where else are you gonna hide all the dead bodies? Actually, being a movie buff I really want to build a home theater. It’d be great to have a projector mounted on the ceiling, with a nice 100” screen, accoustic tile on the ceilings, carpetted walls all in a room you could get pitch black. Since it’s underground, you wouldn’t have to worry too much about noise. And also as a guitar player – well you get the point.

Here’s the problem with that idea. All the basements around here are daylight basements in Kirkland (not to knock on Kirkland, it’s a fine city but split entry houses are sooooo 70s.) A daylight basement really doesn’t adapt well to my dream of having a home theater where I can crank up “The Mummy” and not have to worry about neighboring mobs showing up on my porch wielding pitch forks. After looking at dozens of crappy houses, I decided to scrap the idea of having a basement (maybe someday) and start to look out east of Redmond. Out there, you can still get a good amount of land and a decent place without having to give up your first born son.

Eventually, I found a great house on a big lot that I absolutely loved. It has an awesome kitchen, great back yard for parties, plenty of space and a living room wired for home theater stuff. I’ll be moving in next weekend if all goes well, and fixing up my condo to put it on the market before the holiday season.

One thing about looking for homes is it really helps you figure out what you like and what you don’t like in a house. Walking through houses I absolutely hated was a great exercise and a most valuable experience. I started to get really good at narrowing down what kind of house I wanted and what features were the most important, and I eventually found out that having a basement was not as huge of a requirement as I previously thought and wasn’t worth sacrificing other characteristics I wanted. Having a big lot and a house that didn’t look like everything else on the street was far more important in the end. One thing I did was take pictures of every house I visited and started to compile notes on what I liked and what I didn’t like. In somewhat of a "binary search" fassion, I was able to narrow down exactly what I wanted in a house.

Hopefully when the dust settles, I’ll be able to blog more again. Stay tuned for my semi-left wing rant on housing prices (I didn’t want to make this entry too long). That’s it for now!

Mike

Yay! Higher gas prices!

mikechr — Sat, 29 Jul 2006 00:56:00 GMT

Now that I've caught your attention with my post title, let me explain. Unlike most people, I'm a fan of high gas prices. I'm sure there's a point where I'd start walking to work, but the extra 50 cents per gallon costs me maybe $20 dollars per month. It's $20 dollars I could use for other stuff perhaps, but it's not the end of the world and it's not nearly as bad as the thousands I spend on a poorly managed social security program. Now, there are of course the people who 1) fill up their tanks more than I do and spend much more than $20 dollars extra per month, or 2) the people who are working their butts off to barely make ends meet. And, of course, gas prices affect the price of a lot of other stuff. This is painful, but I maintain that gas is simply not a requirement to survive (unlike water and oxygen.) There will come a point where Americans simply won't agree to pay for it anymore - which brings me to my argument.

Though it's unfortunate, we happen to live in a country that doesn't do anything about a problem until it actually becomes a problem. Now I don't like it, but that's the mentality we have as a people. So, these high gas prices have started to really make a difference in the priorities of consumers. People are looking for more gas efficient vehicles, which creates demand, which creates a market. Various car companies are spending lots more money on Hybrid technologies, which keep getting better and better. More money is being spent on alternative fuel sources such as this cool little vehicle:

http://www.teslamotors.com/

It's a fully electric car that does 0-60 in 4 seconds and can drive 250 miles on a full charge. It can't replace my current car yet, as I like being able to do long road trips, but this is a huge breakthrough on previous electric cars that had a range of about 60 miles. Hopefully this sort of innovation will result in a huge breakthrough in energy storage (better laptop batteries, cell phone batteries, etc.) There will some day be a car that's cheap, does not run off gas, is quick and easy to "recharge", and drives like a Viper - and higher gas prices is going to drive that sort of innovation. It's either that or we wait until we run out of oil completely.

So bring on the high gas prices, it hurts now but if it results in a better world I'm all for that.

Mike

Great ASP.NET site

mikechr — Thu, 29 Jun 2006 03:25:00 GMT

Here’s a great site for developers that aggregates a bunch of other ASP.NET blogs, making it easy to find new information all on a single site. Check it out!

http://www.dotnetslackers.com/

Question of the day...

mikechr — Tue, 27 Jun 2006 04:50:00 GMT

Here’s a hypothetical situation; you’re sucked into a worm hole and transported back to the year 1300 A.D. in Medieval Europe. You didn’t bring back your cell phone, or laptop, or any “more recent” invention. What advantages would you have over society at that time? Would you be able to survive, and would you be able to influence the course of history?

More importantly, could you build anything [from scratch] that hadn’t been invented at the time?

I was thinking about this and it occurred to me that there’s very little I’d be able to do. I’m pretty sure, given enough time, I could generate electricity using magnetite or lodestone as magnets, and a coil of copper wire made from copper ore. This would be something that hadn’t been seen before, but perhaps wouldn’t be useful unless I could harness it for something, perhaps light or heat. Making a light bulb might be possible given enough time, though it took Thomas Edison quite a while to make one that lasted. I’m not sure if I could do it.

How about a steam engine? It’s a simple concept, but manufacturing the parts might take quite a bit of effort. Provided you got the electrical current thing going, creating an electric motor might be the next logical step since it’s basically the opposite of an electric generator. However, for this to be useful to anyone you’d want a way to store energy when you can’t generate it directly. Creating a “wet cell” battery might be fairly easy. What could you build to harness large amounts of current, such as a windmill or water turbine?

What else would be practical?

Any ideas?

Mike – Web Dev Guy/Modern Technology Specialist

More about type descriptors

mikechr — Wed, 31 May 2006 02:18:00 GMT

The other day, one of my readers posed a question regarding my recent post “Data Binding to Custom Objects” – The question was essentially “Why are you writing all this code? There’s much simpler ways to do the same thing.” Well, the reader is correct. There are many ways to do this same thing. We’ll go over that in a second. First, let me take a shot at defending myself. All over the web, there’s thousands of web sites and articles, blogs and newsgroups, discussion lists and bulletin boards all about “how to do stuff in .NET.” This is great, and it helps you find the information you’re looking for to solve the problem that you currently have in front of you in your day to day jobs. However, the truly great programmers are the ones who know what’s going on behind the scenes. Managed languages such as Java, VB and C# get a lot of criticism from the more “pure programmers” who prefer to work more low level with C and assembly. Some claim there’s an entire generation of programmers who will never write their own memory manager, never create a linked list class from scratch, don’t know how a hash table works internally, don’t know how pointers really work, and don’t know the difference between memory allocated on the stack and memory allocated on the heap. I would disagree and say that it’s not the language itself that’s directly responsible for this new “generation” of programmers, but the complexity of the APIs they depend on. If you took the .NET framework out of C#, you’d have nothing more than a object-oriented language that still relies on the developer to construct these sorts of collection objects and design algorithms. This is no different than the plethora of ready made objects MFC provides for C++ Windows programmers. A language is simply an abstract way of expressing an idea, but those more familiar with the internal workings of a language will be the ones who can exact their ideas in a clear and more precise manner. Just because the .NET framework comes with a Hashtable class doesn’t mean you shouldn’t know how to write your own, and just because the .NET framework can take care of property descriptors for you doesn’t mean you shouldn’t learn how they work.

This new generation is not because of lazy programmers, or programmers who didn’t learn the basics. Programmers in general will learn what they need to do their job, and no matter how low level you get, there’s always a level of abstraction. Assembly programmers said this about C, C programmers said this about VB. Until you want to write everything in pure machine code, running in ring 0, on your own CPU design that you soldered together in your basement, you’re relying on a framework that makes your job easier. What’s to blame is this concept of “rapid application design” which VB started. It became critical to toss together these quick little applications in a fraction of the time it would take with more low level languages. IDEs started doing the boring work for you, which is great as long as you know what’s going on. Which leads me back to my original point – there’s plenty of places on the net to find out how to do something, this blog is dedicated to explaining how things works underneath. There’s only so far you will get memorizing what APIs to use where, but I promise you it’s always worth the time to really dig into something and learn what’s going on, for when you need to modify how something behaves or figure out how to fine tune something, you’ll have a pretty good idea of how to do that.

So use my article as a learning tool, this is how the data binding code works internally and this is how ADO.NET works inside. Now, if you’re simply wanting to bind a collection of simple objects to a grid, you can rely on “default type descriptors” rather than generating your own custom type descriptor as described in my article.

Take the following class:

class Company

{

private string _name;

private string _address;

public string Name { get { return _name; } }

public string Address { get { return _address; } }

public Company(string name, string address)

{

_name = name;

_address = address;

}

It couldn’t be simpler, it has two string properties and a constructor. I can bind it to the Data Grid like so:

protected void Page_Load(object sender, EventArgs e)

{

List<Company> list = new List<Company>();

list.Add(new Company("Microsoft", "123 Main"));

list.Add(new Company("IBM", "333 Main"));

Grid.DataSource = list;

Grid.DataBind();

}

My grid doesn’t change, it still binds to the “Name” and “Address” property of the object. Internally, the grid still asks List<> for an enumerator, which List<> provides. Each “Company” object in the list doesn’t provide a custom type descriptor, so a default type descriptor is used. This type descriptor will look for a public method matching the names of the bound fields. Note this must be a method, not a property (if I mark _name public and bind to that, I’d get a runtime error.) This works great for the simple case, but those who used this method without first writing their own custom type descriptor would have no idea why this works, or how they can manipulate or extend this system to do exactly what they wanted. Suppose later on, I wanted to provide some dynamic array of properties, such as items created by the user, or retrieved from a database. I’d be totally lost and never know how to create my own custom type descriptors. Hopefully some people out there got some use from my post, and there will be some better, more efficient code out there because of my efforts.

Thanks for the comments,

Mike Christensen – Web Dev Guy

My thoughts on Monad, the new Windows command line shell

mikechr — Fri, 28 Apr 2006 11:32:00 GMT

Today I decided to install Monad, the new command shell for Windows and I gotta say - this thing is awesome! It's pretty much the most confusing shell I've ever seen, but it combined everything I like from DOS, Bash, Amiga, and the Visual Studio "immediate window".. Basically this thing sits around and creates managed objects in memory and returns instances of those objects.. Those instances can be echo'ed to the command line (ToString is called), or piped into another object.. You can start with simple expressions such as:

1+1

And get back "2"... You can also declare variables such as:

$x = 1+1

You can even call into managed objects such as:

$curdate = [DateTime]::Now

$curdate is actually a manged System.DateTime object, so you can do stuff like $curdate.Date right on the command line.

You can also create an instance of a COM object, such as Internet Explorer:

$ie = new-object -com internetexplorer.application

And then call methods off that such as:

$ie.navigate2("http://www.msn.com/")

The scripting language is pretty nuts and is like nothing I've seen before, but seems to be pretty powerful. However, if you get lost, you can create "CmdLets" which run in process and can be written in managed code. You can easily create CmdLets in Visual Studio, compile them as a DLL, and then call "Add-MshSnapin" on that DLL to register it within the shell. They're easy to debug by attaching to the msh.exe process. When I create a new CmdLet, I specify a verb name (such as Get or Set) and a noun name. I can define parameters for my command as properties of the class. When I run my command from the command line, it's actually creating an instance of that class in memory. If I pipe that command into something else, the actual object is being transferred in memory, not text. Using Reflection, you can also get "tab" command line completion for parameters and what not. You can "capture" all the CmdLets you have active in your shell to an .mcf file so you can easily load them later. In fact, you can create your entire own shell (CmdLets, aliases, script files, etc) and compile the whole thing as an EXE. You can override just about anything you want on the shell, and then just run that EXE as your own person shell with all your customizations that's targetted to your specific need. There's also no reason your shell needs to be command line based. I can make a GUI application that runs on top of Monad as well.

Another great feature is a total abstraction within the file system. Before, you were used to interacting with current file system objects and navigating through folders on the file system. Now, you can do all that, as well as navigate using other providers. Various "MshDrives" are installed, and you can see them by typing "Get-MshDrive". By default, you have drives for mounted disk drives, the registry, ActiveDirectory, Certificates, environment variables, and a few other things. This means you can type:

cd HKCU:\

And you're now in your HKEY_CURRENT_USER registry node. You can type "dir" to see everything and interact with objects just as if they were files.

You can create your own drives as well. I can type:

New-MshDrive -name WinDir -MshProvider FileSystem -root C:\WINNT

And now the WinDir:\ drive is mapped to C:\WinNT. This makes it easy to create shortcuts to long paths.

Since every command uses the same parser, there's various intrinsic parameters that work any any command. For example, I can add the -whatif parameter to show what my command would have done without actually doing anything. I don't really have to worry about parsing parameters on the command line. For those of you who like wildcards, they're way better now. I can pretty much use full regular expressions and search for items or commands. Since everything is an object that exposes properties, I can do stuff like "group" the files by their manufacturer, or search for any dll made by NVidia. I can also search for all processes that take up more than 5 megs of RAM.

Anyway, I could spend hours going on about this and I've only been using this for a day. I've decided to "hide" cmd.exe and use the new shell exclusively, thus forcing myself to get better with it. For a better explanation by the guy who actually designed the thing, go check out:

http://channel9.msdn.com/ShowPost.aspx?PostID=25506

And for an actual demonstration:

http://channel9.msdn.com/Showpost.aspx?postid=25915

If any of you are using this, I'd love to hear your comments and any tips and tricks you've learned so far. I'm a complete newbie and am still doing a lot of exploring.

Thanks,

Mike

Too many tiers? Writing efficient multi-tier web applications.

mikechr — Thu, 06 Apr 2006 07:53:00 GMT

A while ago, we were all convinced that dividing our applications up into multiple tiers was the way to go. This was great because it allowed you to scale up the parts of your application that might represent a bottleneck, and have more control over what resources were allocated to what tier. Windows DNA was built on this concept, as well as the whole “Duwamish” architecture. The problem is, this introduces the possibility of a very “sloppy” implementation of this ideal that creates a much larger bottleneck. This is especially true when using slower mechanisms to transport data across tiers.

Suppose you have a SQL Server backend that pretty much spends its time running stored procedures and returning back datasets. You’ve written a stored proc for anything anyone might want to do. You have a middleware component that connects to your SQL backend, and this provides a programmable API to access your SQL data via web services. This abstracts your actual data store from anyone who wants to program against it, and third parties can simply call into managed APIs to get the data they need. No one needs to run some SQL statement or stored proc that might change in the next version. There are no SQL injection attacks, and other problems with front ends running SQL commands. Life is great.

Then, you have a front end written in ASP.NET. This front end calls into your web services to get the data it needs. The problem comes along when the front end (web application) is too tightly coupled with the business layer (web services). The web application might call several of these web methods during various stages of rendering a page. It might call some APIs to validate the user’s credentials, and then call a web method that returns a dataset to bind to a control, also get some menu data, etc. For each of these web service calls, a stored procedure is run. Now, we pretty much have a 1 to 1 mapping between a web service call and stored procedure call. For large data sets, a high percentage (like 40-50% of the page rendering time is simply spent deserializing these datasets across the wire. This is especially inefficient when the web server is running on the same physical machine as the business layer. ASP.NET will open up a socket, connect to IIS, “POST” in a SOAP envelope, get the results, deserializer this back into a DataSet, then return a reference to this object. While this is happening, the thread serving your web page request is blocked waiting for this result (unless you’re using synchronous web service calls, but that’s a huge challenge to design properly.) Yes, it’s true “remoting” can solve some of these problems, as well as Indigo. If you have more users hitting your site than you have available threads, this can cause all sorts of perf problems. Mean while, until the next web service call comes along, your business layer machine is sitting around twiddling its thumbs.

When this architecture was designed, the ideal situation was the higher tiers (meaning the data layer and business layer) were also the lesser used tiers. The business layer would load in all the data it needed to run, cache it and providing an in-memory representation of the data to work with. At an appropriate time, the data could be written back to the database. The presentation layer would request the data it needed from the business layer and then render it into a form suitable to display to the user. In other words, calling into your business layer wouldn’t necessarily mean a call into your data layer unless new data was needed.

For load balancing reasons, everyone seems to want their middleware components to be stateless and not have to worry about caching data and data being out of sync with the database.

Using proper abstraction means that a single call doesn’t drill down into 80 different layers every time it’s called, otherwise why even have this abstraction? The same works for “abstracting” your application into logical layers. If every call is going to drill down into every single layer every time, why have the layers?

I think a better approach for certain applications, especially those who have very simple business logic (ie: just grab some data from SQL Server) is to write business objects as sort of a “data adapter.” My data adapter knows how to get my data from a SQL connection and provide an API for me to read and manipulate it. My data adapter should be in the form of a DLL that my web application links to. The DLL knows how to connect to the SQL Server that it’s configured to use. Multiple instances of my web server, each with their “data adapters” connecting to a cluster of SQL back-ends, can be running to scale up to the demands of my users. This is still a layer of abstraction, but doesn’t have a massive bottleneck marshalling the data across the wire between tiers. If my front end is super dependent on this data access API, each call should be as lightning fast as I can make it, and that means running it in the same process so I can just return a live pointer to my object.

I could also write a separate web service that also links to my “data adapter” DLL, but provides a SOAP based interface to my API. My application doesn’t need to use this, but third parties that wish to program against my backend can easily use this interface. There’s really no reason for my own application to be accessing my own business layer via SOAP, it’s inefficient and results in blocking threads and dragging down the CPU. The amount of effort it takes to marshal a large chunk of data across SOAP is simply not worth the extra scalability that I’m not convinced I’d really have anyway. In fact, connecting two tightly coupled tiers that are both owned by me and controlled entirely by me should not be talking in an extensible human readable, 7bit protocol – especially if I’m doing this 30 times for every page I serve up.

If you’re building such an application, I’d be interested in hearing your comments as far as architectural decisions and scalability, and if you’ve done any stress testing with large amounts of data, where your bottlenecks are. It’s my opinion that flattening out these tiers into a single process running ASP.NET code and an adapter that can connect to a data backend is the way to go, especially when using SOAP. I’d love to hear your comments!

Mike

Welcome!

mikechr — Sat, 01 Apr 2006 05:00:00 GMT

Back in the early days of web development, web pages were simple. They used HTML and “web developers” typed in that HTML using powerful text editing programs such as Notepad. Most of these files were static files stored on a web server, and were rendered out to the client very quickly. Soon after, CGI was invented. This was pretty cool too. An HTML request could be handled by an EXE program, and the web server would pass in a string of parameters or a path to a file containing those parameters. Combined with HTML forms, this made web pages much more useful. Life was good in those days.

Today, we have “web applications.” A Web Application is a program that your PM wants to look and feel like a normal Windows application, but goes completely against the entire design of HTML. Web Applications are littered with script, dynamic content that changes without refreshing the page, behaviors, and bizarre hackery that fetches new data from the server without refreshing the page. Content is almost never static, and even the frameworks are designed to mimic standard Windows form based programming. This style of software design is half programming, half web design, and half voodoo. Developers who don’t understand the stateless architecture of HTTP are usually lost, and I’ve still yet to meet a programmer who understands both web design and computer science.

This blog is dedicated to all things related to Web Applications, and specifically those built on the ASP.NET framework which is the most un-web like thing I’ve ever seen. I’m a strong believer in good architecture and understanding how things work under the covers, so I put strong emphasis on doing things “the right way” and avoid hacks whenever possible.

Enjoy!

Mike Christensen: Web Dev Guy