From The Depths

Better Late than Never

It's been over a year since my last post so I figured its just the right time to write a new one.  As you have most likely heard, the Microsoft Business Framework project I had been working on for several years was cancelled.  Now I'm back in the Business Solutions Division instead of the Developer Division.  Along with the re-org, I switched teams to the Business Logic team.  I'm now longer a database guy.  Ah well, I had a good run.  In the Business Logic team, we'll be responsible for providing an infrastructure for allowing business application developers to declare and execute their business logic.  It's a nice change of pace and I've got a lot to learn.

My Philosophy on Software Design

I like to think of software design as cracking a code.  You keep working on the design over and over until you've finally cracked the code and an elegant design emerges.  I believe the solution exists out "there" and I just have to find it.  Albeit there is more than one solution but some are more elegant than others.  The act of searching for it is software design. 

I've been working on a design the past few days both at work and sometimes at home while in bed (my brain just doesn't like to take a break sometimes).  Last Saturday morning, I think I finally cracked the code.

Questions on Concurrency Conflict Resolution

There's been an e-mail thread going around in an internal Microsoft Business Framework (MBF) discussion list of the best way to handle concurrency conflicts.  Someone brought up the novel idea of asking users what they would like to happen.  So that's what I'm doing.

Currently in MBF we support two modes of concurrency which you can specify for each entity type: at the entity level and the property level.  Entity level concurrency will ensure that when you save an entity, all the property values of that entity must match what exists in the database in order for the save to succeed.  Property level concurrency will ensure that when you save an entity, only those properties which have been modified must match what exists in the database in order for the save to succeed.

Developer Questions:

1. Are there other variations or modes such as timestamp that you would like to see?
2. What kind of information do you want returned when a conflict exception occurs?
3. In what ways do you want to be able to resolve a conflict?  For example, should there be a way to force the save to overwrite the database values?

User Questions:

1. Part of MBF involves UI generation.  How do you want to interact with the UI to resolve conflicts?
2. In order to avoid user interaction, would something like a user-defined rule-based method of resolving conflicts be useful?

Any feedback you can give will help the MBF team out.  Thanks.

The Good and The Bad of Dogfood

When first becoming a Microsoft employee, I was excited to get to see and use many of our products before they were released to the public.  Most people love to be the one of the first at trying out brand new software.  After a while, the novelty wore off, however.  When using pre-beta daily build versions of software like Visual Studio and Longhorn, you will see the not-so-pretty defects of incomplete code.  Of course, this is expected and, with each new build, things get a little better.  So over time you see a gradual improvement in the software.  Now compare this to being a regular consumer that doesn't work at Microsoft.  They jump from one version of Windows to the next without seeing all the unsightly defects that occurred along the way.  I miss that feeling.  Ah well, back to work.

Distinguishing Compositions and Associations

For all the application developers out there, this is a request for comments.  In business apps there are generally two types of relationships: compositions and associations.  Compositions are tightly-coupled relationships where the composition's lifetime is tied to its parent.  Associations are loosely-coupled reference relationships.  In designing a prescriptive framework such as the Microsoft Business Framework, it may be beneficial to the application developer to distinguish compositions and associations from each other while coding in a way that does not involve referring to the documentation.  In other words, how can I tell that when I navigate from Order to Customer (as in Order.Customer) that I am referring to an association rather than a composition? 

So two questions for everyone:

1. How useful would it be to be able to distinguish between compositions and associations when coding against those relationships?
2. What ideas do you have to help make that distinction?

Search Engine Game Show

Sometimes it's crazy the things you think of when you wake up in the middle of the night.  Last night was one of those nights.  About 3am, I woke up, and then... Search Engine Game Show.  What an idea!  Have a trivia game show where the contestants have a computer with an Internet connection.  They're allowed to use the Internet to answer the question.  Whoever can answer the question first, wins that question.  This lets you have extremely obscure questions that nobody would really know without doing some surfing.  For example, who was the first U.S. Supreme Court justice from Kansas?  Uh... I don't know but I can find out.  Ah.. David J. Brewer.  Maybe throw in a bonus question if someone gets the question right that they have to answer immediately (3 seconds).  So if they navigated to a good web page with that content, they could find it quickly.  It could have some trivia questions which makes you break out the calculator as well.  How many square furlongs is Yellowstone National Park?  A show like this would display good search techniques to the viewers.  You could actually broadcast the contestants' monitors to viewers on TV to show what they're doing.  Ok, so maybe this show would be incredibly boring.  But what kind of ideas do you expect when waking up at 3 in the morning?

Gimme a Buddy

Microsoft has begun a new program called Microsoft ISV Buddy Program.  “This program enables an ISV to connect and build a 1-on-1 relationship with a Microsoft employee.” (http://msdn.microsoft.com/isv/)  I signed up to be matched with an ISV.  While I haven't been paired with one yet, I'm excited to begin.  Being able to have an ongoing relationship with a vendor will be a fun experience.  This enables developers such as myself to get some feedback from “the real world” which is great to have.  It should provide value to both “buddies”.  The ISV will have a personal contact within Microsoft for answers to questions and requests.  I am glad that they have put an escalation process into place.  I don't claim to know everything and my ISV may have a question I don't know, so it's good to know that I am able to escalate the question in a process where someone can help out.  I think this program is a great initiative by Microsoft and can't wait to get started.

It's That Time of the Year Again

It's annual performance review season again at Microsoft.  My favorite part of the annual review is to look at the changes to the form that were made.  This year, objectives are no longer called objectives.  They're called commitments.  That seems to be just a step up in intensity.  Each year they need to step it up, I think.  Soon it will be “Things I need to get done if I want to continue working here” or “Tasks to do in order to prevent foreclosure on my house”.  Now that's steppin' up the intensity.  But seriously, annual reviews do force us to lift our heads out of work and look down the road for the coming year.  What do you really want to accomplish in your job in the next year?  Not only that but you have to look through the past year and describe how you did on your goals, er, objectives, ... I mean, commitments.  It can be tough to toot your own horn for some people.  And not so tough for others.  I used to struggle at it when I was younger in job interviews.  But it's just one of those things that's necessary in order to be successful in your job.  Use it when it's necessary is the key, I think.  Modesty is still a good thing.

SQL Server Yukon DTS: Success in cleaning CRM data

I was just reading an article at TechnologyEvaluation.com about the difficulty in maintaining data quality for CRM applications.  The extremely volatile nature of CRM activity leads to degradation in the data such as:

• Customer details that are incorrect or inconsistent with other data
• Duplicate records
• Multiple database synchronization problems

Duplicate records is noted as being the most common issue.  This can be caused, for example, by spelling variations resulting in duplicate records for the same customer.  As I was reading this I recalled what I had learned about the new Data Transformation Services (DTS) feature of SQL Server Yukon.  Yukon's DTS will include data cleaning technology such as fuzzy lookup and fuzzy grouping.  Fuzzy lookup involves an input row of data whose columns are mapped to the columns of a given table of existing data.  So you may have an existing Customer table and now you have a new candidate row to insert into the Customer table.  Fuzzy lookup involves scanning the candidate row's values and comparing them against existing data to check for similarities so that the data can be “fixed up” so that they are consistent.  For example, if the candidate Customer's company name is equal to “Micrsooft” and it finds that there exists a current Customer with a company name of “Microsoft”, it will classify that as a high-scoring match all allow the user to take the appropriate actions either manually or through an automatic action.  Fuzzy grouping allows you to find fuzzy duplicates in a given result set in a similar way to fuzzy lookups.  This allows you to fix up data that already exists.  As you can see, these DTS features match well with solving the data quality issues of CRM.  I'm sure Yukon will be taken advantage of in this sector.

 This posting is provided "AS IS" with no warranties, and confers no rights.

Metadata Mapping in O-R Mapping Technology

In my previous post I described the benefits of using Object-Relational Mapping (O-R Mapping) technology in regards to the encapsulation of change.  In this post and several subsequent posts I'm going to delve into some of the basic designs used in O-R Mapping technology.  The first deals with the actual mechanism of mapping objects to relational tables and object fields to table columns.  Most commercial O-R Mapping products use metadata mapping as the mechanism that describes the relationship between an object and its data source so this is what I will focus on in this post. 

Metadata mapping involves a developer defining mappings between an object and its data source in some static form separate from the code which processes that metadata to produce CRUD operations.  The form in which this metadata can be stored could be XML, a database, or even directly within source code.  The inherent hierarchical structure and non-compiled nature of XML lends itself to be the most useful of these forms.  A convenient method of using XML is to create your mappings through source code using a set of APIs which creates a map object of the same type that is used by the system which consumes the metadata.  The object is serialized to XML when creating the mappings then deserialized by the system when it needs to read the XML. 

The more functionality supported by the O-R Mapping technology, the more complex the metadata mapping structure.  For example, if the O-R Mapping technology supports inheritance in its objects, there needs to be a way to express the inheritance in the metadata mapping.  Obviously, commercial O-R Mapping products aim to have significant functionality causing the metadata mapping to be quite complex.  Because of this mapping tools are developed to generate these metadata mappings.  This allows the developer to work through a UI rather than manipulating whichever underlying form was chosen for the metadata mapping.

Mapping is one of the first things focused on when developing an O-R Mapping product.  The information contained in the metadata will most likely change over time during development as more edge cases are found.  It is important to focus on developing a robust design in the metadata as a lot of change occurring in this area causes churn due to alterations of the old mappings in order to reflect the new design.

In my next post I will describe the different methods of mapping inheritance hierarchies in a relational data source.

Final thought:
You are only as good as your metadata.

Object-Relational Mapping Systems Encapsulate Change

Why have an object-relational mapping (ORM) system?  Because change happens.  There's no getting around it.  I'll concede that ORM would not be necessary if you built a piece of software or data source which never had to be updated, extended, customized, upgraded, or have a new technology adopted.  Such a piece of software, of course, wouldn't last too long. 

At its core, ORM is a separation of concerns in relation to the application that uses it.  The application programmer need only be concerned with objects while the ORM is concerned with how the objects relate to the data source.  I'd like to go through each of the points raised above and explain how an object-relational mapping system solves these issues. 

Updating

ORM encapsulates changes in the data source.  In order to improve data source performance, a developer may wish to consolidate the sources of a particular inheritance hierarchy into one source.  (I'll deal with the different ways of mapping a class inheritance hierarchy to data source in a later post.)  This change can be made within the ORM without effecting the code or functionality of the application whatsoever.

Extending/Customizing

I view extending functionality to be an internal practice and customizing functionality to be a practice done externally.  Essentially, ORM effects them in the same way by isolating the changes that need to be made.  Of course, enabling external customization has some extra infrastructure involved, but this is separate from the discussion of ORM.  Classes can be added to an existing inheritance hierarchy without effecting existing application code. 

Upgrading

Upgrading to a new version of a data source has the potential for breaking changes to occur.  By encapsulating the communication with the data source in ORM, the breaking changes will not spread to application code.

New Technology

New features in a data source can be utilized more effectively by hiding their interfaces from application code with ORM.  For example, the next version of SQL Server, code-named “Yukon”, allows you to use CLR-defined objects in SQL Server through enhanced functionality of UDTs.  The new syntax used to manipulate these UDTs can be completely hidden with ORM and never exposed to application code.  This allows modifications to be made by ORM developers without impacting higher layers. 

It can be seen that ORM encapsulates the change that occurs when dealing with such a dynamic system as the data source. 

Welcome - From the Depths

Intro

Welcome to my blog from the depths of Microsoft.  While I am as low as you can go in the org chart in that I have no one reporting to me, that doesn't make my role any less important.  I am a software design engineer in the Developer Division (DevDiv) at Microsoft working on the Microsoft Business Framework.  In the 1.x versions of Microsoft Business Framework I worked on the Entity Persistence team which is an object-relational mapping subsystem.  This allows you to persist, query, and delete objects to a relational data source.  Today, I am working on the integration of Microsoft Business Framework to run on top of ObjectSpaces, Microsoft's general solution to object-relational mapping.

About Me

So let me tell you a little about myself that's not related to work.  I enjoy doing digital photography.  My gear includes the following: Canon 10D digital camera, Canon EF 24-70mm f/2.8L USM zoom lens, Canon EF 70-200mmf/2.8L IS USM zoom lens, and a Canon EF 50mmf/1.8 II standard lens.  I mainly do landscape and wildlife photography.  I love to travel to all kinds of locations in the National Park System.  Parks that I've visited include Glacier NP, Pipestone NM, Mount Rushmore, St. Louis Arch, Theodore Roosevelt NP, Waterton NP in Alberta, Yellowstone NP, Devil's Tower NM.  I'm planning on visiting Isle Royale NP one of these days.  I actually have the map of it posted in my cubical for inspiration.  Of course, you can't find good photo opportunities without doing some hiking and/or snowshoeing which I also love doing. 

Blog Topics Covered

What topics can you expect to be written about in this blog?  I plan on writing about the standard concepts of object-relational mapping solutions, SQL server programming, C# programming, and maybe a little about some tips to make cross-team/cross-site collaboration work go smoothly.