Welcome to MSDN Blogs Sign in | Join | Help

Oslo CTP update; Data Modeling Design Patterns in M

As modelers, one of the things we're doing all the time is looking for patterns - trying to distinguish what in each model is truly unique to the domain from that which is more broadly applicable, and then either using or adapting existing patterns or harvesting new patterns to put in our back pocket.  The January Oslo CTP update (woohoo!!) on the Oslo Dev Center includes an initial set of data modeling design patterns.  These patterns, described and illustrated in M, include patterns for modeling enumerations and relationships, including closed type-based enumerations, open extent-based enumerations, and one-to-many, many-to-many and one-to-one relationships.  

While this may seem like pretty basic stuff you'll likely find some or all of it useful, particularly if you're new to M and coming at it and repository modeling with an OO rather than relational mindset.  The patterns all have code samples so you may find it useful to have the patterns open while you're coding.

We'll add more patterns as our experience grows and the language evolves and is more fully implemented.  In the pipeline are patterns for vertical partitioning, including patterns that help when mapping a generalization hierarchy in a conceptual model to a relational design.   You can see some early exploration of these kinds of patterns in the domain schemas included in the CTP.

As always, we'd love to get your feedback.  If you have any comments on the patterns and the usefulness of this kind of material let us know.  And if you have any broadly useful patterns in your back pocket that might map nicely into M they would be great to hear about also.

 

Posted by Bill Gibson | 0 Comments
Filed under: , ,

Domain Modeling

My focus within the Olso team is on domain modeling – creating models for specific problem domains using the Oslo modeling platform’s languages and tools.  Let me describe why we think of this as more than just data modeling.

At the center of an Oslo domain is a data model authored in M. The primary purpose of this model or schema is to describe repository storage, defined in terms of extents. This domain schema defines data integrity constraints and may also define reusable data definitions or types, as well as computed values that describe interesting ways to retrieve and compute data from the model.  The schema is compiled into SQL, resulting in tables, views, constraints and functions which are loaded into a SQL Server database primed with some simple Repository patterns.  The result is a simple and easy-to-query and otherwise normal SQL database, which can be queried using any SQL data access API or tools.

Surrounding this data model and considered part of the domain may be any number of domain specific languages, which if developed using Oslo may be textual, authored in MGrammar, or visual, authored with Quadrant.  These DSLs define ways in which domain information (instance data) can be created or presented.  Any number of textual and visual languages may be defined for the same domain, each customized to provide some specific perspective.  A domain can also reference other domains, so models and languages may also reference (and in due course, embed) other models and languages.

Accompanying each DSL may be utilities that map or import/export language 'documents' to and from repository storage.  For textual DSLs described in MGrammar simple transformations can be expressed in the grammar itself - more complex transformations need to be described separately.  A key Oslo scenario will involve cataloging and potentially creating/editing pre-defined artifacts that might otherwise be created and/or consumed externally by other tools.

Collectively, the domain schema, textual and visual DSLs, and their accompanying import/export tools define what we think of as a domain from an Oslo standpoint and will typically be evolved as a unit.  This whole-domain perspective is an important part of the way we think about modeling with Oslo and strongly influences how you can expect the Oslo platform to be developed.

You can see some early examples of domain schemas included with the CTP on the Oslo Dev Center.  Take a look if you can.

Next up, data modeling design patterns in M... 

Posted by Bill Gibson | 1 Comments
Filed under: ,

Getting Back in the Saddle (although no longer on a Whitehorse)

Long time no blog!  In the meantime I've changed jobs within Microsoft; I've been working for the last several years on the modeling platform known as “Oslo” which 'came out' at PDC in October. You can find out more about Oslo at the Oslo Dev Center

I was amazed to find my blog still accessible after being dormant for so long.  While it was with some nostalgia that I freshened up the links to the Whitehorse tech notes below, you should follow this link to get the latest on Visual Studio Team System Architect Edition

Posted by Bill Gibson | 0 Comments
Filed under:

Whitehorse Architecture TechNotes Published on MSDN

Phew!  A set of Visual Studio Team System Tech Notes have now been published on MSDN, including some fifteen new articles I've written on topics related to the Whitehorse architecture tools.  These should all be easy to read and are inter-linked, to make them an easy to browse.  Hopefully they will provide some useful background and some new insights into the Distributed Systen Designers. 

Below are links to each of my notes, ordered in what is probably a good sequence in which to read them. 

  • TN_1104: Understanding the System Definition Model
  • TN_1114: The Four Layers of Systems in the System Definition Model
  • TN_1100: Understanding Applications and the Application Diagram
  • TN_1105: Why Class Libraries are not shown on an Application Diagram
  • TN_1109: Understanding Systems and the System Designer
  • TN_1101: Understanding and Using the Default System
  • TN_1110: Using Systems to Represent Services in a Service Oriented Architecture
  • TN_1112: System Portfolio Management
  • TN_1102: Designing Substitutable Web Services
  • TN_1113: Representing Connections to Manual, Physical and Other Kinds of System
  • TN_1111: Top-down System Design
  • TN_1108: Connecting Applications to Web Services via Class Libraries
  • TN_1106: Copying Endpoints vs Creating Endpoints from WSDL
  • TN_1103: Web Service Endpoint Name Propagation
  • TN_1107: Understanding, Using and Creating Toolbox Prototypes
  • Why do ASP.NET WebService and ASP.NET WebApplication look alike on the Application Designer?
  • Posted by Bill Gibson | 1 Comments
    Filed under: ,

    On shadow applications and agile development

    Randy Miller has published a paper on agile development that discusses the use of 'shadow' applications in an agile modeling and development context, in which he talks in somewhat veiled terms about the use of the Whitehorse design tools.  The term shadow application emerged from discussions Randy and I had at Tech Ed earlier this year as we developed the top-down design approach described in my earlier article.  In the article I show how you can use a shadow application as a placeholder for the behavior of a system and thus design systems top-down. In his paper, Randy take this idea further and suggests the shadow application is actually written or generated, consistent with the code-oriented agile development philosophy.
    Posted by Bill Gibson | 0 Comments
    Filed under:

    Logical vs. Physical Architectural Modeling Concerns

    We've been having hallway discussions about some of the dimensions of modeling application architectures.  SDM adopts a particularly physical perspective, as it focuses on the deployment packaging stack of concerns.  Its focus is on resources (things like assemblies, config, XML files etc.) that are composed into independently deployable applications, that are composed into systems, which are then composed into still higher level systems.  SDM, does allow you focus on tangible software entities and express their behavior - very CBD-like.  But of course there are other perspectives, and particularly troublesome to some is the absence in the current designers of a logical perspective, in which you can define what an application does, independently of describing how it does it or how that implementation is packaged.  The discussion turned to the role of namespace in that logical perspective, specifically to the question, are namespaces the primary logical organizing perspective. I wrote some comments in an email on the subject and have elaborated on them here and invite comment.

    While namespaces are clearly not a packaging concept as they can cut across assemblies, they are not necessarily the best or only way to organize the elements in a logical architecture.  Instead the notion of a cross-cutting and entirely logical organization of function is proposed.  The term "module" was suggested some months ago by a colleagure for this notion.  Thus a module is an arbitrary collection of function.  A module could be a collection of methods, one class, many classes; may be coincident with an assembly, cross assemblies or contain assemblies.  Modules can also describe data. Early on in your design, you might think about modules but go no further than giving a name and describing the role.  A classic block structure architecture diagram might be used to show the way modules interact, either depicting a specific instance of an architecture or a pattern before any details exist of the behavior of each module. 

    You might assign namespaces to modules – although the naming of types/classes, while important, is not intriniscally linked to the logical structuring of software.  Namespaces might reflect the logical structure and you might be tempted to conflate the two, such that you use namespaces when really you arethinking about modules.  At some point in your design process as you refine your module designs and start to define classes, creating namespaces will become important.  And at some point in your design packaging considerations will become important, when you will specify applications, frameworks, and systems.  With your packaging decisions you harden the partitioning of the functions in your modules and make communication and binding decisions.  

    Thus there are module, namespace and packaging dimensions, These represent orthogonal but related concerns which inform each other.  Different design strategies will have you tackle these aspects: physical structure (systems, applications, frameworks), logical structure (modules) and implementation (classes with namespaces) in different amounts at different times – being able to reason about all three as and when you choose will be the most flexible and should be what we aim for in tools.  Associating a namespace with a module or even a package may make good sense when the purpose of the module/package and the goals of the namespace align but the two are not intrinsically linked. 

    Posted by Bill Gibson | 1 Comments
    Filed under:

    ARCast on model-driven development

    An interesting initial round on this model-driven development podcast, including comments from Jack Greenfield from our team.  Will be fun to watch this one develop...  As someone who was involved in what we believed to be one the big successes of the 80's and 90's so called CASE era, I always find it a little irritating to hear about all the failures, so it was interesting to hear some positive comments about the lessons to learn from that period.  From my experience, what we were doing at Texas Instruments was successful for a long time, in large part because the problem domain we tackled was limited and the patterns well recognized-- we were targeting large scale integrated database transaction processing systems. In effect, while we never described it in these terms, what we developed was a domain specific language - or at least a family of languages.  And as one of the contributors mentioned, the approach relied on the assumption that you never modified a line of generated code - there were lots of user exits but you never touched the code.  We even implemented a feature that deleted the source code after it had been compiled to save space :-)  The approach we used then would be much harder to make successful now, when frameworks are so much richer and the problem domains and application model so much more varied. 
    Posted by Bill Gibson | 2 Comments

    Kick the VSTS tires...

    Now you can try out Team System in a hosted environment without the hassle of downloading and installing.  Check out Rob Caron's post for details.

    Posted by Bill Gibson | 1 Comments
    Filed under:

    Going dark...

    I've not lost enthusiasm for this blogging thing, its just that I've been asked to convert some of my blog entries into MSDN TechNotes and write others afresh to accompany the Visual Studio launch, so I've been busy with notes/articles in work on the following topics among others:

    Understanding the SDM Type System
    The Four Layer Model of Systems in SDM
    Understanding Applications in Application Designer
    When and how to use the Default System
    The Role of System Designer
    Using Systems to Define Services
    Designing Substitutable Services
    Understanding Creating and Using Toolbox Prototypes
    Top-Down Design (published here already)
    Why no Class Libraries on the Application Designer (blogged here already)

    I will post the finished articles here and/or links to them on MSDN. 

    Posted by Bill Gibson | 1 Comments

    Top-down System Design Article posted

    I have posted an article on top-down system design using the System Designer.  This describes and illustrates a little-known technique for defining systems and applications top-down. 
    Posted by Bill Gibson | 1 Comments

    Developer? Architect? All hail the Devarchitect!

    No doubt you know that Microsoft is very persona driven.  A persona is a fictitous person that embodies the characteristics of a class of user - useful to us when working through product usage scenarios.  Mort is probably the most famous of Microsoft's development personas.  Whitehorse has its personas too - you'll hear us chatting about Andrew, the Enterprise Architect and Vernon the Application Architect as if we had a drink with them in a bar the night before. 

    However, I think we're missing a persona which is a variation of the application architect - I don't have a name for him/her yet - but I think this persona is a Devarchitect.  Devarchitects think of themselves as architects (and may have that job title) - they are intimately involved in defining the function, structure, and design of systems, including performance, scalability, reliability, interoperability, etc - but they are also deeply involved in developing aspects of those systems, perhaps writing and testing early prototypes or core elements, often in the early days of a project so that the technical architecture and its execution characteristics can be proven, as well as digging in to deliver some of the more complex components as the project ramps up.  A devarchitect rolls up his sleeves and gets his hands very dirty.  Many high-end developers are devarchitects or aspire to be.  Devarchitects often don't want to be anything else - they don't want to get out of touch with development and the details of the technology.  And in some cases their organizations can't afford the luxury of full-time architects. 

    At TechEd during a focus group I recently met several people who fitted this description well.  The profile of one of them confirmed this - he identified his primary job responsiblity as architect, and profiled his work with tasks associated with an architect role occupying 20% of his time and those associated with a (lead) developer occupying 70%.

    Does anyone want to suggest a name for our devarchitect?

    Posted by Bill Gibson | 12 Comments

    Understanding SDM: Systems and the Four Layer Model

    The Whitehorse Distributed System Designers are based on SDM - the System Definition Model.  SDM offers a simple model for representing computer systems that can help in many parts of the design, deployment and management space.  I'll try over an occasional series of posts to introduce some of the key SDM concepts and how they play together.  I'll also show how we can extend these to do some neat things in the application architecture modeling space.

    At the heart of SDM is the notion of a 'system'.  Let's look at a definition of system that I find useful.

    A system is a configurable collection of resources that can be independently deployed .  If the resources in a system need to access resources of another system they do so by communicating via endpoints on the systems. 

    A system can be atomic or composite; a composite system is composed of other systems. I'll explore system composition, resources and endpoints further in other posts, this post will focus on the notion of system hosting.

    One system is considered to host another system if it provides infrastructural services to the hosted system.  Services can be thought of as infrastructural if the services are provided to many systems of the same type.  Introducing the hosting relationship allows the model to be layered, so that different concerns can be addressed in different layers.  For example, the requirement of an ASP.NET application on the services provided by IIS is an infrastructural hosting relationship.  Similarly IIS is hosted on Windows which in turn is hosted on an appropriate physical system.  It wouldn't be useful to have to include models of IIS and Windows and all the other systems that exist in those layers every time you wanted to model an application so we model those separately and then host a model at one level on the model of another. This reflects the reality that one tends to think of configuring standard system 'platforms' on which other systems are installed.  Most organizations use a small number of standard configurations of hardware systems on which they install standard configurations of OS and networking systems on which they run standard configurations of application hosting systems on which they run standard configurations of their business applications.

    We typically think about there being four layers of systems: application systems, application hosting systems (such as IIS, SQL Server), operating and network systems (such as Windows), and hardware systems (such as a Dell or HP server, Cisco router etc.).  In reality there are no hard boundaries and there may well be more than four layers of interest, but it is often useful to think about these four general categories of systems, and it certainly helps people 'get' the overall model.

    Different roles/people at different stages are interested in defining, deploying and managing each of the layers.  Information about one layer can be provided to those involved in defining another layer to assist in validating that systems at one level will successfully deploy on a specific configuration of systems at the next level.  The Distributed System Designers, for example, support the definition of models of the application layer and the application hosting layer and enable deployment validation - where the configuration of an application is verified against a configuration of application hosting software with the goal of increasing the likelihood of successfully deploying the application.   

    Posted by Bill Gibson | 1 Comments
    Filed under: , ,

    Does Application Designer support operations defined on interfaces?

    Another question that keeps cropping up is whether we support viewing and editing Web service operations when these are defined on interfaces.  It usually comes up in the form of the question, I've built a Web service and defined the operations using attributes on methods on an interface, but the operations don't show up in the Web Service Details window when I click on the endpoint.  What's wrong? 

    Application designer doesn't support display of the operations where the WebMethod attributes are defined on methods on an interface.  The Web method attributes must be placed on the concrete class referenced by the asmx file for them to appear as operations in the Web Service Details window.  The endpoint does appear on the diagram and you can navigate from it to the class in code, but we don't walk up the inheritance chain to look for attributes on the interface, so we don't see the operations.  Asmx platform support for interfaces was only introduced into the VS 2005 release late in cycle, too late for us to respond with effective tool support.  We would have needed to support both viewing and editing operations and handle several complex use cases around mixed definitions.  It simply wasn't possible to address this in the time available and ensure the quality level we wanted so we had to cut the feature.

    We recognize the value of using interfaces which are a cornerstone of the Windows Communication Foundation (Indigo) service model.  So, expect to see tool support for WCF fully incorporate interface support as part of a contract-driven service development approach.  We'll be able to talk more about this as our plans crystallize.

    And on this topic, one observation is that if using a contract-driven approach in which a WSDL contract is viewed as the master, then it is not so critical that a CLR interface exists.  Using the Application Designer's support for creating web service endpoints from WSDL you can construct the web service implementation class from the WSDL contract directly.  While using interfaces undoubtedly helps some aspects of the engineering process it is not as critical when a WSDL doc is considered the master definition.  And note also that you can use interfaces in your design it is just that the Web service-related attributes must be defined on the concrete class for Appplication Designer to recognize them, and any edits in the designer are applied to the methods on the class and not the interface. 

    Posted by Bill Gibson | 1 Comments

    Application Designer: copying endpoints vs. creating them from WSDL

    There are two ways you can copy the specification of a Web service endpoint using Application Designer, which vary in subtle ways that you can leverage. You can simply use copy/paste - see my earlier post, or, if the source application has been implemented, you can use Create Web Service Endpoint from WSDL (right click on the target application and then reference the URL of the source application's endpoint).  In both cases you get what we think of as a shallow copy - that is, no implementation code is copied.  The differences are in how each approach handles complex types referenced by the operations.  In the case of copy/paste, the operation signatures only are copied and no new types are created in the target application.  If you create the endpoint from the WSDL, then the WSDL and any referenced XSDs are processed and new CLR types are generated for any complex types referenced in the message definitions.  If the target application is unimplemented, you cannot see or reference these new types until code is generated, but rest assured they are there! 

    Copy/paste of an endpoint takes the view that the implementer of the target application will provide the definition of the parameter and return types out of band.  You can define the types from scratch, copy the type definitions from the source or more interestingly, reference the actual type definitions used in the source application.  This is likely not a good idea if the types are defined in the source Web service project itself, but may be very useful if you define the types in a separate project/assembly, which is arguably best practice.

    The other approach, creating the endpoint from WSDL works rather differently.  This works the same way as WSDL.exe /server and will create a 'fresh set' of types in the target application based on the types in the WSDL and imported XSDs.  This approach echoes the SOA tenet of sharing schema and not types - although obviously that is directed primarily at avoiding implementation dependencies between consumers and providers of a service - not between providers. 

    It can be argued that we could have gone further with copy/paste and created references in the target application to the projects/assemblies that contain the referenced types.  There proved to be several technical obstacles to doing this, not least that the referencing mechanisms in Visual Studio were only designed for creating references from projects and were not accessible to us - in many cases the target application will not have been implemented and there is no project.  Given that the workaround is to simply add the reference yourself to the project once implemented we thought this was a reasonable course.  And this approach recognizes that you may not want to create the tie between the service implementations anyway.

    In any case, with the two techniques available you have a choice.  And choice is a good thing :-)

    One other point to note is that you can create endpoints from the WSDL of External Web Service endpoints, which don't otherwise support copy/paste.

    Posted by Bill Gibson | 1 Comments

    Why the name, Whitehorse?

    A little bit of trivia.  Whitehorse is the capital of the Yukon Territory in Canada.  That made sense as a name back in the days when the group's mission was largely SQL Server data tools.  Times changed and our mission changed but the name stuck. 

    Whitehorse is a great place to live... 

    We later adopted the Uffington white horse as our 'logo' - this is very cool, it's amazing to see modern art that dates back to 1000 BC!

    Posted by Bill Gibson | 1 Comments
    More Posts Next page »
     
    Page view tracker