MSDN Utopia : Architecture

Design: Architecture Frame Considerations (Part 2)

Salva Patuel — Tue, 10 Feb 2009 19:58:00 GMT

In this second instalment we are going to review another set of frames. This guideline should help you to understand what is involved on each one of them as well as considerations and known patterns.

You can find the first part following this link.

The second part looks like this:

The following guideline can help you to identify the key issues on each area of the frame and some recommendations that we usually share with our customers in the ADC team (Application Development Consultant ). Note that this list is not exhaustive and are general suggestions that can help you on your design process.

Configuration Management
The message here is clear, we need to move configurable options to a configuration file so the user can change it without recompiling the application. Yes, is quite clear but the problem is that developers usually overlook how this information is accessed and secured. We have seen many times user names and passwords on these files without any security! Not only have that, with time these configuration files grow to unmanageable levels that jeopardized the application execution.

Key Issues:

· Lack of or incorrect configuration information

· Sharing configurations around farms

· Not securing sensitive configuration information (exposing passwords/identities)

· Lack of security around the configuration files

Consider the following guideline:

· Use IIS sharing configuration features in order to deploy configuration files over a farm.

· Consider using least-privileged process and service accounts.

· Categorize the configuration items into logical sections, this method can help you secure different areas (features like configuration delegation in IIS are very helpful in this scenario)

· Encrypt sensitive information in your configuration store

· Restrict the access to your configuration information

· Provide a separate administrative application to deal with configuration files.

Pattern	Brief
Provider	Implement a component that exposes an API that is different from the client API in order to allow any custom implementation to be seamlessly plugged in.

Coupling and Cohesion
When you are designing components for your application you should ensure that are highly cohesive and loose coupling is used across layers. Coupling refers to the dependency between components and cohesion relies on the functionality that the component provides.

Key Issues:

· Incorrect grouping of functionality

· No clear separation between objects

· Tight coupling across layers

Consider the following guideline:

· Design for loose coupling between layers.

· Consider using abstraction to implement loose coupling between components.

· Consider using common interface definitions

· Use dependency inversion to decouple dependencies, depending on abstractions rather than concrete components.

· Divide your application functionality into logical layers; if you are dealing with customers you shouldn’t include your message binding properties on the same object.

· Design for high cohesion. If a component says “Customers” it should provide only customer functionality.

· While loose coupling requires more code, the benefits include a shortened dependency chain and simplify the building process.

Pattern	Brief
Composite view	Combine individual views into a composite view.
Inversion of Control	Populate any dependencies of objects on other objects or components that must be fulfilled before the object can be used by the client application.
Template View	Implement a common template view, and derive or construct views using the template view.
Two-Step View	Transform the model data into a logical presentation without any specific formatting, and then convert that logical presentation into the actual formatting required.
View helper	Delegate business data-processing responsibilities to helper classes.

Data Access
Separating the data access layer (DAL) is an important decision that improves the application maintainability and extensibility. This will allow you to easily change the repository or the logical structure of your data model without affecting the application functionality. It is recommended to implement an entity model to translate physical designs into business ready entities.

Key Issues:

· Per-user authentication and authorization when not needed

· Chatty database calls interfaces

· Data access code embedded with business logic

Consider the following guideline:

· Avoid coupling your application model to the database schema. Use abstraction models like the entity framework.

· Enforce data integrity in the database, not through data layer code.

· Move business logic to the application layer, avoid long stored procedures as they make applications harder to migrate and control.

· Open connections as late as possible and release them as early as possible.

· Avoid accessing the database from different layers, the DAL should be the only component that knows anything about the repository.

Pattern	Brief
Active record	Include a data access object within a domain entity.
Data mapper	Implement a mapping layer between objects and the database structure that is used to move data from one structure to another while keeping them independent.
Data Transfer Object	An object that stores the data transported between processes, reducing the number of method calls required.
Domain Model	A set of business objects that represents the entities in a domain and the relationships between them.
Query Object	An object that represents a database query.
Repository	An in-memory representation of a data source that works with domain entities.
Row data gateway	An object that acts as a gateway to a single record in a data source.
Table data gateway	An object that acts as a gateway to a table or view in a data source and centralizes all of the select, insert, update, and delete queries.
Transactional Script	Organize the business logic for each transaction in a single procedure, making calls directly to the database or through a thin database wrapper.
Table module	A single component that handles the business logic for all rows in a database table or view.

Exception Management
It is important to track all the application exceptions to make the system more reliable and to properly capture application defects. A crash can be very costly for customers and is not something that you want to re-experience in order to get more information. A proper exception management should provide you with enough information when the fault occurs, this should include as much information as possible in order to reproduce the scenario.

Key Issues:

· Unstable state exposure

· Revealing sensitive information to the user

· Using exceptions to control logic flow

· Inaccurate/non-existence logging information

Consider the following guideline:

· Do not reveal sensitive information in exception messages and log files. I have seen many websites showing the full stack when an error occurs!

· Let exceptions flow if they are unexpected, catching everything without proper logging may provide a false end user experience that can lead to corrupted entries.

· Design an appropriate exception propagation strategy.

· Send the information back to a central repository when possible to analyze trends when large user based is expected.

· Log information on common repositories, like log files, event logs or external web services.

Pattern	Brief
Exception Shielding	Prevent a service from exposing information about its internal implementation when an exception occurs

Layering
Proper layering design allows you to separate the functionality into different areas of concern. This will help you to group functionality and components that will make your system more predictable and able to scale out. Layers should have a well defined communication flow, for example, layer A can talk with layer B but not the opposite scenario.

Key Issues:

· Incorrect grouping of components within layers

· Not following layering and dependency rules

· Not considering physical boundaries

Consider the following guideline:

· They should be used as logical groups, for example, user interface, process layer, application layer, data layer.

· Components within a layer should be cohesive.

· Consider physical boundaries when designing layers

· Use interfaces to define each layer, this will allow proper communication and different deployment models.

· If you are running web applications consider using message-based interfaces to provide a stateless user interface.

· Provide a facade pattern to the process and application layers.

Pattern	Brief
Remote Facade	Creates a high-level unified interface to a set of operations or processes in a remote subsystem to make that subsystem easier to use, by providing a course-grained interface over fine-grained operations to minimize calls across the network.

The next part will complete the set of considerations, watch this space!

Design: Architecture Frame Considerations (Part 1)

Salva Patuel — Mon, 01 Dec 2008 13:53:00 GMT

At Microsoft we are working hard in putting together the different components that are involved deciding the architecture approach. One of these components is the architecture frame, which together with the architecture style and application type creates the architectural baseline for modern design. This guideline is targeting architects and not developers, therefore the topics and recommendations cover high level decisions.

One of the most common situations when we sit down during reviews is that the team starts choosing the technology first without analysing what they really need. They usually end up in a model that fails to deliver value to the business and trigger a set of failures around the qualities of service. In order to avoid this, a proper review of the architecture frame can help architects and developers to foresee potential problems and chose the right technology and pattern based on the real execution context. The frame is pictured below with the section covered in part 1.

Authentication & Authorization
This area is one of the most misunderstood concepts in the frame, as many architects confuse these concepts as they security has been a complex area to understand. Remember that the authentication process validates the user credentials and the authorization involves the roles and actions that the user has on the system. Designing a good strategy on this area helps to improve the security and reliability of the application.

Key Issues:

· Lack of authentication across trust boundaries, as this leaves your application vulnerable to spoofing attacks, dictionary attacks and session hijacking.

· Granular or improper authorization can leave your application expose to vulnerabilities like information disclosure, data tampering and elevation of privileges.

Consider the following guideline:

· The most important consideration is performing a STRIDE analysis that may help to identify risks.

· Identify your trust boundaries; authenticate and authorize users and calls across trust boundaries. Some calls may need to be authenticated on the client and the server (mutual authentication).

· If your system involves multiple applications based on different user subset consider using a single sign-on strategy

· Do not store passwords in plain text; instead consider using hash codes or proper encrypted algorithms.

· Do not transmit passwords in plain text.

· Protect resources based on identity, groups or roles.

· Use role based authorization for business decisions.

· Use resource based authorization for system auditing.

· Use claims based authorization when you need to support federated environments or complex rights models.

· Consider using a trusted subsystem for access to back-end services to maximize the use of pooled connections.

· If the presentation and business layers are on the same machine and you need to keep the original caller’s ACL permission consider using impersonation. On the other hand, if they are on different machines consider using delegation. Note that this may affect performance.

· Implement IP filtering to protect your business layer.

· Consider the implications of using different trust settings for executing service code.

· Ensure that secure protocols such as SSL are used with basic authentication.

· Use secure mechanisms such as WS security with SOAP messages

· Run services under the most restrictive account that is appropriated

Pattern Map:

Pattern	Brief
Data Confidentiality	Use message-based encryption to protect sensitive data in a message.
Data Integrity	Ensure that messages have not been tampered with in transit.
Data Origin Authentication	Validate the origin of a message as an advanced form of data integrity.
Exception shielding	Prevent a service from exposing information about its internal implementation when an exception occurs
Federation	An integrated view of information distributed across multiple services and consumers.
Replay protection	Enforce message idempotency by preventing an attacker from intercepting a message and executing it multiple times
Validation	Check the content and values in messages to protect a service from malformed or malicious content

Caching
This is the developer’s favourite, we have seen many applications implementing caching patterns without considering why and where it is needed. Poorly designed caching strategy can lead to degradations on performance and responsiveness; on the other hand, good caching design can really speed up responsiveness and contributes to the concurrency and efficiency factor.

Key Issues:

· Caching invalid content

· Caching sensitive data

· Incorrect choice of caching store

Consider the following guideline:

· You should use caching to optimize reference to lookups and avoid network round trips.

· Do not cache volatile data.

· Try to load the cache asynchronously to avoid call delays.

· Consider ready-to-use objects when storing them in the cache instead of raw database data.

· If you need to cache sensitive data use encryption as a simple memory dump can reveal it.

· If your application is deployed in a farm avoid using local caches that need to be synchronized, instead consider using transactional resource manager or a product that supports distributed caching.

· Do not depend on data still in your cache, as it may have been removed.

Pattern Map:

Pattern	Brief
Dependency Cache	Use external information to determine the state of the data stored in a cache.
Page Cache	Improve the response time for dynamic pages that are accessed frequently

Communication
This area covers how the components on different layers will communicate with each other, the mechanism depends on the deployment scenario but the key issues are usually similar.

Key Issues:

· Incorrect choice of transport protocol

· Chatty communications

· Failure to protect sensitive data

Consider the following guideline:

· When crossing physical boundaries consider using message based communications, on the other hand, for crossing logical boundaries you should use object based communication.

· If you are not concern about the order of the messages consider using asynchronous communication to unblock processing or UI threads

· Consider using message queuing to queue messages for later delivery to cover network interruptions. Remember that that this pattern supports transacted message delivery and supports reliable once-only delivery.

· Consider net pipes for inter-process communication, TCP for local networks and HTTP based protocols for external links.

· Determine how to handle unreliable or intermittent communication.

· Validate endpoint addresses in messages.

· Decide if a message communication needs to be two-way or one-way

Pattern Map:

Pattern	Brief
Intercepting Filters	A chain of composable filters that implement common pre-processing and post-processing tasks.
Pipes and Filters	Route messages through pipes and filters that can modify or examine the message as it passes through the pipe.
Service Interface	A programmatic interface that other systems can use to interact with the service.
Duplex	Two-way message communication where both the service and the client send messages to each other independently, irrespective of the use of the one-way or the request/reply pattern.
Fire and Forget	A one-way message communication mechanism used when no response is expected.
Reliable sessions	End-to-end reliable transfer of messages between a source and a destination, regardless of the number or type of intermediaries that separate endpoints.
Request Response	A two-way message communication mechanism where the client expects to receive a response for every message sent.
Channel Adapter	A component that can access the application's API or data and publish messages on a channel based on this data, and can receive messages and invoke functionality inside the application.
Message bus	Structure the connecting middleware between applications as a communication bus that enables them to work together using messaging.
Messaging broker	A component that connects messaging systems and replicates messages between these systems.
Point to point channel	Send a message on a Point-to-Point Channel to ensure that only one receiver will receive a particular message.
Pub/Sub channel	Create a mechanism to send messages only to the applications that are interested in receiving the messages without knowing the identity of the receivers
Command Message	A message structure used to support commands
Event Message	A structure that provides reliable asynchronous event notification between applications.
Competing consumer	Set multiple consumers on a single message queue and have them compete for the right to process the messages, which allows the messaging client to process multiple messages concurrently
Durable subscriber	In a disconnected scenario, messages are saved and then made accessible to the client when connecting to the message channel to provide guaranteed delivery.
Idempotent receiver	Ensure that a service will only handle a message once
Message dispatcher	A component that sends messages to multiple consumers.
Messaging gateway	Encapsulate message-based calls into a single interface in order to separate it from the rest of the application code.
Messaging Mapper	Transform requests into business objects for incoming messages, and reverse the process to convert business objects into response messages.
Polling consumer	A service consumer that checks the channel for messages at regular intervals.
Selective consumer	The service consumer uses filters to receive messages that match specific criteria.
Service activator	A service that receives asynchronous requests to invoke operations in business components.
Transactional client	A client that can implement transactions when interacting with a service.

Composition
This part of the frame is very important an it has a significant impact in the user experience. The composition is the process used to define how the user interfaces are structured in order to provide a consistent look and feel.

Key Issues:

· Cooperating UI modules are coupled by dependencies making development, testing and maintenance more difficult.

· Dependency changes between modules forces code recompilation and module redeployment.

· Dynamic UI layout update and dynamic modules loading difficult due hardcoded dependencies.

Consider the following guideline:

· Use abstraction patterns when possible to avoid issues with dependencies between components.

· Consider creating templates with placeholders, a good approach may be using the template view pattern to compose dynamic screens to ensure reuse and consistency.

· Consider composing views from reusable modular parts like user controls. The composite view pattern is a good approach to build a view from atomic component parts.

· Avoid using dynamic layout as they are difficult to maintain.

· If you need to allow communication between presentation components consider using the pub/sub pattern, this will lower the coupling between components and improve testability.

Concurrency and Transactions
Designing good concurrency and transaction models is a key part of the architecture as it directly affects performance (specially concurrency). Identifying the right requirements in order to select optimistic or pessimistic models can be a critical decision. There are plenty of documentation and best practices around this area but from the architectural perspective we should worry about the general issues that are system may face.

Key Issues:

· Nonexistent or poor protection for concurrent access to static data

· Deadlocks caused by improper locking

· Not choosing the right data concurrency model.

· Long running transactions that hold locks on data

· Using pessimistic/exclusive locks when not necessary

Consider the following guideline around transactions:

· Wrap business critical operations in transactions

· Use connection based transactions when accessing a single data source

· Don’t be afraid of using transaction scopes to manage transactions on multiple data sources.

· Avoid holding locks for long periods.

· If transactions are not an option consider the use of compensation methods to revert the data to the previous state.

Consider the following guideline around concurrency:

· Remember that static data is not thread safe.

· Updates to shared data should be mutually exclusive; this will prevent two threads from attempting to update the data at the same time.

· Locks should be scoped at a very fine grained level, the idea is to lock only when is necessary and release it as soon as possible.

· If you need to update static fields use the check-lock-check pattern as other thread may have changed the value between your check and your lock.

· Never apply a lock to a type definition or the current instance. Using these constructs can lead to deadlock issues that are difficult to locate. Use private static objects as locking reference instead.

· Consider the use of the synchronization support provided by collections when working with shared ones.

Pattern Map:

Pattern	Brief
Capture transaction details	User DB objects such as triggers and shadow tables to record changes to all tables belonging to the transaction.
Optimistic offline lock	Ensure that changes made by one session do not conflict with changes made by other session.
Pessimistic offline lock	Prevents conflicts by forcing a exclusive lock
Coarse Grained lock	Lock a set of related objects with a single lock
Transaction script	Organize the business logic for each transaction in a single procedure, making calls directly to the database or through a thin database wrapper.
Implicit Lock	Use framework code to acquire locks on behalf of code that accesses shared resources.

On the next blog entry we are going to explore the second subset of considerations using the architecture frame.

You can find part 2 here.

Design: SOA Taxonomy, an object oriented approach

Salva Patuel — Tue, 02 Sep 2008 14:05:00 GMT

An essential part of the SOA Governance is to properly identify the requirements in order to introduce or upgrade a service within your SOA model, this is commonly known and service and integration planning within the SOA lifecycle. This is all good in theory but the nightmares start when architects are trying to put it in practice.

As an application development consultant here at Microsoft we constantly find chaos in the SOA first attempts, the main reason behind this is the lack of services organization. This not only brings problems to the model that are expensive to change but does not shows business value to the stakeholders, as they cannot see any ROI in the implementation. As a result, the efforts are abandoned and the architectural model remains broken.

How do we help our customers to solve this issue? We usually take them a level higher within the abstraction layer, why do you need SOA? Which are the problems that you usually have? Which are the costs involved in changing your system? We really need to understand our driven forces that lead us to a new architectural paradigm.

But what about designing the services model? This is another area where confusion arises as architects ask themselves where they should start and how. The answer to this is enclosed on the services taxonomy. There are many categorizations and naming around taxonomy but I want to present a recommendation based on the Microsoft services experience.

We define four main categories that really help us to define the high level model. Let me explain each one of them.

· Entities Services: The entities encloses your business entities, these are independent actors within your business model. The entities do not provide business benefit acting alone as they don’t have the expertise around which other entities are around. They should be completely independent and should own the data repository. A good example of an entity service will be customers. Customers are nothing more than what the name says, it has and owns all the information about the entity “customer” but knows nothing about your business.

· Capabilities Services: This is the crucial part of your business; this is what you do to add value to your services. The capabilities know very well the entities and capabilities context where it works, as interacts and consumes services to deliver value. An important thing to understand is that they know about the capability but do not control the entities that consumes, this means that if a capability needs to alter an entity it should use the entity service responsible for that. The capability also owns its own repository and controls all the business logic associated with it. A good example of a capability is an order service. The order service will know everything about placing an order and will need entities in order to generate one, for example, the order will need the customer information that comes from the entity service.

· Application services: These services supports your business model, this means that provides auxiliary services to your entities and capabilities not directly associated with your system. This usually involves wrappers to third party application or bridges to other systems. A good example may be a service wrapper to a Remedy application.

· Bus services: This taxonomy groups all the support and infrastructure services that allow your entities and capabilities to work within the infrastructure policies. For example the security services that validate users can be grouped in this taxonomy.

As you see, with this categorization you can start to add services to your model. Where do you start? The easiest way to do it is asking yourself what your business does, what is the essence of it. Let’s imagine that you are the responsible for designing the SOA model for a large supermarket. The amount of services that you will need is massive but if I ask you about the essence of your business which will be the answer? Believe it or not this sounds easy but architects that are too embedded in the daily duties cannot properly answer the question. My model would be:

Look how simple it is, you just join customers with products and generate orders. Now, if I ask you to add these services in the taxonomy model you now know how to do it.

Remember that upon I have presented orders as the linking service between the entities this does not suggest hierarchy. All the services are at the same level as they can be accessed by an ESB (Enterprise service bus) or ISB (Internet service bus), or even simply your process layer. What is more, what you can do is apply the same model when you enter into the service design scope; this is for example when you design the internal taxonomy of “Orders”. This service will have again a process layer, entities and capabilities, but now only around the orders scope. Does this sound familiar? Well, it may, as it is a common model applied to the object oriented design.

Internally, these new entities and capabilities can be isolated components that are responsible for a solely function, for example, an internal process can be the workflow that triggers a new order and a capability may be altering the order value.

This can show you that applying the SOA governance model around taxonomy can help you and your developers to properly design services and object oriented components properly isolated and encapsulated, as is the same mindset. What is more, the WCF team is working in a “in-process” binding that will allow you to use a WCF communication with objects that today are part of your main process and tomorrow may develop into independent services just changing the configuration file! Isn’t it beautiful?

If your organization is facing these problems and you want Microsoft to support you and help you around your SOA initiative don’t hesitate the contact the “Application Development Consultant team” in Microsoft (http://www.microsoft.com/uk/adc)

Architecture: Multi-channel with WPF

Salva Patuel — Tue, 08 Jan 2008 03:35:00 GMT

As architects, we should see the world with tendencies oriented eyes, being aware of the different signals that the market shows behaving like explorers. Sometimes is a difficult task that demands us going the extra mile – I like to see a good architect as that one that can properly filter what is important and what is not. The latest trends on the consumer space are showing us that we should care about the multichannel experience. On this blog entry I am going to focus on the user experience and how affects the architectural decisions.

We should have a common programming model to develop user interfaces, but at the moment is a bit challenging with many technologies around, making very difficult to bring platforms and formats together maintaining the same consistent user experience. Some attempts using the web had brought some success but at a high maintenance cost with some unpleasant stories. This is one of the main reasons why I am a big supporter of WPF. Having a common language that can be rendered on different platforms and modalities is helping us to decide which next user experience technology we should include in our roadmaps. With a single Ux investment stream we can keep the developers team focused reducing the learning curve and optimizing the production, with the benefits of reducing cost when we need to bring a new channel into the equation. Remember that a good architect should look into this other factors that constrain the business, rather than the usual appetite for new technologies.

But nothing comes cheap; as Microsoft is pushing for these technologies we are facing challenges on the implementation side. The multiple channels today require a different sub-set of the render engine (embedded in a framework subset), making our jobs a little harder. The challenge is to find the thin line that divides the flexibility with the complexity, dividing what we share and what is custom code.

This is a good place to put emphasis in design patterns, those patterns mostly avoided not for the complexity but fear. If we put it in context is easy to separate the service layer from the user layer but things get a little more complicated when architects and developer needs to code the user layer. The most common approach is to build it as a monolithic application (from the VB times), as the separation has been done using the typical three tier model. Well, let me tell you that we are facing new challenges where this model is not applicable anymore. Consumers are demanding new ways to interact with the application; what is more, enterprises will come to you arguing why they should use certain platform to run your application. Your nightmare will not end there, operations - finally taking into consideration – will come to you demanding features to support and maintain the system that you are architecting. As I have introduced WPF as a convenient technology you still can shoot on your foot. This is where the patterns like MVP can really help you to separate the business logic of your user layer application.

A contemporary approach that I usually coach is the mindset of layer bubbling. This concept starts from the forest correctly dividing the layers, using maybe SOA approaches to promote reusability and stability (as many other qualities of service). Once you have finished the forest approach you should take each layer and treat them using the same pattern, this means that your user layer will have a view (Ux), a controller or presentation (App) and your model (Data), this approach will help you to reduce the amount of custom code that you will need for the multi channel approach.

With this type of approach you will be able to asses which parts of your applications and data will be stored where, furthermore, how the information is shared. In our specific case, the model will be the proxy interface that retrieves information from our services, then the user interface workflow will be coded separately, as it should not be compromised by the model neither the view (how is presented). In the past this model was hard to implement due the fact of the view code, coding in win forms, html, mobile was very expensive and hard to support. XAML comes to the rescue.

Another important task as an architect is to envision the future, how this architecture model will stand in time. Well, I can foresee the platforms coming together at some point, reducing the amount of gaps on the rendering engine. It will take longer to have a common framework for all these channels but at least a uniform WPF render engine will do the work. Today we have some items that are supported on SilverLight, others in WPF desktop and XBAP (and mobile quite soon), but I can see standardization in the near future.

This little architectural roadmap has shown as how important is the use of the two main architectural pillars: An explorer and an envisioner. Put them in practice!

Design: Task Parallel Library explored

Salva Patuel — Mon, 12 Nov 2007 00:11:00 GMT

As some of you may know, the threadpool code of the .NET is helping many developers to use multiple threads on their applications, increasing “sometimes” the responsiveness and delegating all the switching responsibility. But, for those less fuzzy that like to know how it really works you usually find that the code is not the best threadpool library. The internal reason for it is mostly related to inherit code. If we really have to write it again we will do it in a completely different way but this is a story for another post, what I really want to focus today is in the inner workings of the task parallel library, that come to complement a important part of it : Organized parallelism.

I am not planning to explain what is TPL, if you need this information I will recommend this basic article that will introduce you to the library here. The idea is to show you how the internal architecture works and some considerations on the use of it, as it not a magical library, you can still write bad code on it.

Let’s start with the general architecture, as we can see on the first figure the TPL sits just above the CLR, both teams has been working together in order to coordinate the implementation. The TPL library contains the task scheduler, this is available for extensibility. The best examples of this extensibility are the Parallel framework and PLINQ.

Let’s start from the top of the stack; the parallel FX publishes a set of methods that allows developers to access the library (note that is static (shared VB))

Parallel.For (start, end, delegate(int param) { a[param] = a[param]*a[param]; });

The method will invoke the TPL library, assigning the parameters and the delegate to execute (note that this can be a lambda expression as well!). The interesting bit comes at this stage when the request enters in the main queue. The library evaluates the amount of threads that will be required for the task, trying to find the optimal amount based on amount of cores and length of the task. Is even possible to decide to use a single thread in a sequential mode, this scales well as the developers does not need to know how many cores will have the final users, and the application will behave differently if the system is scaled in.

In order to avoid a bottleneck in the queue consumption by the threads the TPL implements a queue per thread. The main queue distributes equally the tasks across the available threads and each thread starts to execute the delegates that are waiting on the individual queues, this is a common pattern applied in OpenMP. In theory this works fine but as we know, the cores are not always available for the threads; therefore one thread maybe finishes the queue earlier than the other threads. This situation is not optimal therefore the queue stealing model is applied. When the thread does not has any other task on the queue it will start to query the neighbour queues looking for tasks. If it finds more tasks it will remove them from the busy queue respecting the order, optimizing the processor time. As you have guessed by this time, the threads are not based on the threadpool, as they have core affinity.

This execution model does not guarantees the execution order, as if you have 1000 tasks in a 4 core machine all of them should have 250 tasks, but if a thread finishes earlier it will consume other tasks. This is an important consideration if you need to respect certain order on the execution, as I said at the beginning, it is important to use this extensions when you are completely sure about the task independence. The same applies if the delegates use shared memory, a contention is possible on this scenarios that will limit the scalability.

The final architectural consideration is regarding exceptions. In a normal sequential “for” if an exception is detected the loops stops, with the parallel library this becomes a little more difficult.

If an exception occurs in one of the threads the TPL will bubble up the exception and it will inform the other threads that the execution needs to be cancelled. This happens automatically, but the TPL cannot guarantee that some of the tasks are executed after the exception. Now, what happen if two threads throw an exception at the same time? The parallel library throws a special exception type that contains both exceptions. This is something important to consider.

To summarize this blog I must confess that I am quite excited about the potential of this library in the future, is only the first version but I can see the future of parallel computing in a single place. If we think about the next step....mmm...maybe including grid computing... J I better stop here.

UPDATED: New Microsoft Parallel Computing Developer Section (including CTP download: http://msdn2.microsoft.com/en-us/concurrency/default.aspx)

Design: Ephemeral cache explored

Salva Patuel — Mon, 06 Aug 2007 16:19:00 GMT

Many of .net developers are familiar with the ephemeral model included in the garbage collector. The idea of using generations in order to manage the lifespan of the objects is extremely helpful when you need to control the life of an object. Incidentally, this is applicable to custom cache models where content expiration is an issue.

There are many ways to control the content expiration on a cache and all of them bring to the table pros and cons. Most of the challenges come from the cache size and how the expiration can be controlled using time stamped data. Time structures are expensive and bulky, and browsing the content in order to find the expired ones can be a painful tasks. Many techniques have evolved in order to solve the problem like date indexing or date bucking, but they require careful control over the manipulation and sometimes even index persistence.

The main problem arises when we manipulate a big cache, if we use time stamped a secondary thread, responsible for removing garbage, will analyse the list of entries. If the entry time stamp does meet the expiration criteria the entry is removed. This not only can take resources, also affects the storage fragmentation.

The same concept of the garbage collection can be also applied to our cache service, if we split the cache in generations we can control the cache quality, moving commonly content to a more secure and less intrusive list. When a new query arrives the cache service will retrieve the information from the resources (i.e. a DB). The call is responded and a secondary process inserts the content on the generation 0 cache. This generation tends to grow rapidly as all the call responses are cached. As this cache will grow very quickly using time stamp expiration will be very expensive.

Now, most of this cache will be loaded in memory just because a random single request read from a record that maybe will not be used for the rest of the cache lifespan. This is a real waste of resources. Applying cleaning processes to the cache will leave it very fragmented, reducing the performance. If we want to differentiate commonly used data and non common we can implement a new generation, where the data accessed on the generation 0 is promoted to the generation 1. The second cache will be smaller and will have more useful information, making the first choice for a new query.

We can see on the graphic that the call first go to the older generation, if the result is not there it will go to the first generation. Once is found the entry can be promoted. With this model the first generation can be purged quite often, as is information only requested once (note that an access counter can also be added to the generations adding rate rules like “you need 5 requests in order to be promoted”). What is more, the model can be correctly designed to support more generations keeping a healthy cache with accurate data loaded.

As the higher generations contain valuable information, it is a good idea to persist those caches, otherwise when the service is restarted we are going to lose the valuable tracking information. I tend to use file storages on generation 1 and even DB storage for generation 2. This means that the most valuable generation (2) will be able to be loaded by other caches in our distributed environment. This increases efficiency as the caches will contain only valuable information.

I have personally tried this model on large cache (4 million entries) and the performance and resource consumption are balanced with the response time. The volatile cache is dropped by slides avoiding fragmentation and can be triggered by time or even by resource pressure. If you need more information regarding the implementation do not hesitate to contact me.

Architect: SOA services lifecycle

Salva Patuel — Fri, 13 Jul 2007 16:11:00 GMT

Beyond the natural MSF lifecycle where all software development should fit in, there is a higher level of process lifecycle abstraction that relates to the service. Services are one of the most important backbones of a SOA model, where the business capabilities and activities interact with the entities to provide the solution response.

From the architectural point of view, services born, live and die. In order to use a common definition we are going to divide it in three states: expose, compose and consume. The architect is responsible to govern and manage this lifecycle with a clear visibility of the current status of each individual service. This will help to empower reusability and to have a better control when change occurs, what is more, is the recommended path to migrate architectures that contain legacy monolitic systems in order to reach the dynamic level on the application platform optimization model.

Expose: This first step consist in analysing the business drivers (processes) that are going to be addressed. This helps to indentify the necessary services that will solve the process requirement. With the services indentified the reuse (or new green field) services are exposed.

Compose: In this stage the serivces are composed depending on the type of process, if this requires an orchestration of many services this is the place to do it. This stage will incorporate the necessary knowledge to use the business services.

Consume: The client applications are able to discover and use the services based on the processes that govern them.

The lifecycle granularity can be further exploded into specific activities that will categorize the service in a specific domain. This domain has specific implications and the architect will need to embark into the different tasks and responsibilities that each step brings. This is not a fixed model; organizations can tailor it to fit their structure but is a good starting point to avoid missing the plot and rushing services into the model.

The picture shows how the architect is involved in both areas, the solution framework and the operations framework. This approach has worked quite well in many implementations here at Microsoft. If the architect only stays aware on the solution section it will miss the global view, how the services are performing and which ones are candidates for decommission or replacement. When a new service request arrives, the taxonomy of the model should help to identify which services are available; this is part of the analysis step. At this stage we can review the current entities, capabilities and activities and their functional domain. Is the request pushing for a completely new concept that the current services can not supply? This is the question to answer.

Once the service has been scoped and designed following the MSF guideline the building of the service is triggered as well as the testing and provisioning. I always like it to have a list with all my service listed, organized in a domain based ontology that gives me the full picture of what is going on. A graphical representation is always useful to visualize the services used by the different processes, I know that is hard to keep it up to date, but trust me, it really worth it.

Design: How good SOA can help you dealing with multi-cores

Salva Patuel — Mon, 09 Jul 2007 17:59:00 GMT

Unless you have been living under a rock for the last 5 years you should know about the multi-core architecture that the new processors bring to our world. Well, processors hardly can go any faster with the current technology therefore we need to start to have parallel thoughts...

Well, you are about to design a solution and you are facing the challenge “How I can take the most of the multi-core processor that my server has” Sometimes is not easy to take advantage of this extra horse due the lack of parallel design skills, I have seen many attempts to achieve this with more headaches than solutions.

After working on several projects where we were trying to improve the performance of certain components I came across multiple solutions but there is one tendency to solve this problem that comes within the SOA shadow. Thinking about services can help you to divide the pieces of functionality that can be parallelized; this will provide a natural processor optimization with the incorporation of different processes to get the maximum output from the multi-core environment.

I have seen many times a SOA model that includes a service multi layer derived from an ontology model, where the hierarchy drives to the conclusion that some services must be componentized within a single service to reduce complexity. The problem of this merge is that sometimes the developers fail to divide the components (adding dependencies) or they limit the multi threading capabilities of the hardware layer.

In the figure above we can see the service hierarchy that encapsulates service C1 and C2, when they are implemented in a single process the services needs to properly handle the threading model and the communication model, using memory synchronization patterns. This adds locking and latency problems that reduce the performance and scalability when the service is scaled in.

If we decide to divide our services in different process and use the common contract as communication layer we can solve the problem reducing complexity and isolating the threads. This helps you to reduce the component coupling and scales in better and cheaper. There is no need for synchronizating shared resources as the process isolation will dela with the individual heaps.

What is more, the model can have processor affinity to control the load on the hardware, this can became more common in a 4x4 architecture where 16 processors can give you not only the service isolation, also the hardware. As software architects need to deal with teams without the proper parallelism skills, this type of approach can help you reducing project risk and delivers a proper set of services to the business processes that allows you more flexibility if a new process needs the C1 service directly.