MSDN Utopia : Design

Working with memory mapped files in .NET 4

Salva Patuel — Mon, 08 Jun 2009 19:47:00 GMT

I have been exploring different new features that come with the .NET 4, beyond the most popular ones like dynamic types and covariance; I was interested in performance enhancements. For this reason I am going to publish a couple of blog entries were I explore these different features.

Memory mapped files may sounds alien to the managed code developer but it has been around for years, what is more, it is so intrinsic in the OS that practically any communication model that wants to share data uses it behind the scenes.

So what is it? A memory mapped file allows you to reserve a region of address space and commit physical storage to a region (hmmm, sounds like virtual memory, isn’t it?) but the main difference is that the physical storage comes from a file that is already on the disk instead of the memory manager. I will say that it has two main purposes:

· It is ideal to access a data file on disk without performing file I/O operations and from buffering the file’s content. This works great when you deal with large data files.

· You can use memory mapped files to allow multiple processes running on the same machine to share data with each other.

The memory mapped file is the most efficient way for multiple processes on a single machine to communicate with each other. What is more, if we check other IPC methods we can see the following architecture:

Impressive, isn’t it? Now you have the power of this technology available on the System.IO namespace (instead of using the Pinvoke approach).

Now let’s quickly explore how it works. We have two types of memory mapped files models, one model is using a custom file, this can be any file that the application accesses it and the other one using the page file that we are going to share it with the memory manager (this is the model that most of the technologies above use).

Let’s explore the custom file model. The first thing that we need to do is to create a FileStream to the file that we are going to use, this can be an existing file or a new file (keep in mind that you should open this file as shared, otherwise no other process will be able to access it!). With the stream in place, we can now create the memory mapped file. Let’s see an example:

using System.IO.MemoryMappedFile;

MemoryMappedFile MemoryMapped = MemoryMappedFile.CreateFromFile(

new FileStream(@"C:\temp\Map.mp", FileMode.Create), // Any stream will do it

"MyMemMapFile", // Name

1024 * 1024, // Size in bytes

MemoryMappedFileAccess.ReadWrite); // Access type

I have use one of the simplest constructor, we define the stream to use and we provide a name. The object needs to know the size of it in bytes and the type of access that we need. This will create the memory mapped file but to start using it we will need a map view. We can create one using the following syntax:

MemoryMappedViewAccessor FileMapView = MemoryMapped.CreateViewAccessor();

This map covers the file from the first byte until the end. If we need now to write or read information from it we just call the map view methods with the correct offset.

int Number = 1234;

FileMapView.Write(0, Number);

FileMapView.Write<Container>(4, ref MyContainer);

We can see that we can write built in types or custom types with the generic version. The good thing about the memory mapped files is persistence, as soon as you close it the contents will be dumped on the disk, this is a great scenario for sharing cached information between applications.

Now if we want to read from it, the other process needs also to create a memory mapped file, we can use the other static initialize that opens an existing one or creates one if it does not exist.

MemoryMappedFile MemoryMapped = MemoryMappedFile.CreateOrOpen(

new FileStream(@"C:\temp\Map.mp", FileMode.Create), // Any stream will do it
"MyMemMapFile", // Name

1024 * 1024, // Size in bytes

MemoryMappedFileAccess.ReadWrite); // Access type

Create the map view and read it:

using (MemoryMappedViewAccessor FileMap = MemoryMapped.CreateViewAccessor())

{

Container NewContainer = new Container();

FileMap.Read<Container>(4, out NewContainer);

}

That’s it, really simple isn’t it? Now, there is a small drawback with this approach and is related to the size of the memory mapped file. If you don’t know in advance you will need to create a large file
“just in case”, this can be a very large file with a lot of wasted space, it would be nice to have the ability to reserve the space instead of committing all of it, isn’t it?

In order to solve this issue you can use the page file, this has the advantage of allowing you to commit data on the fly but introduces another issue: you don’t own the file and the map will last until the last handle is destroyed. But think about it, for certain scenarios this is absolutely valid.

Now if we revisit the sample we will need to change some initialization parameters, for this particular one I will use the full constructor so I can introduce some other features that are also applicable to the custom files one.

MemoryMappedFileSecurity CustomSecurity = new MemoryMappedFileSecurity();

MemoryMappedFile PagedMemoryMapped = MemoryMappedFile.CreateNew(

@"Salvador", // Name

1024 * 1024, // Size

MemoryMappedFileAccess.ReadWrite, // Access type

MemoryMappedFileOptions.DelayAllocatePages, // Pseudo reserve/commit

CustomSecurity, // You can customize the security

HandleInheritability.Inheritable); // Inherit to child process

The memory mapped file security allows you to customize who or which process can have access to the resource, this can be quite important when you want to protect sensitive information and you don’t want other processes changing the file map. You can explore that object and see all the different settings that you can change. The reference is here. If we explore the construction of the memory mapped file we can see that there is no stream, we just name the resource. This will create an association between a section of the file based on the size and the map name, this is how both processes will access the file. Note as well that I am setting the property “DelayAllocatePages”, this implements a pseudo reserve/commit model where we will use the space once is needed. Finally, the other interesting parameter is the handle inheritance model, this will allow to share this resource with a child process if is required.

The access to the file uses the same syntax as the previous example, remember that if you close the memory mapped file this will be non accessible, this issue catches many developer.

Finally, another interesting area is the creation of multiple map views, these can work on the same memory mapped file accessing different areas of the files. This will allow you to properly protect the content and allowing you to synchronize the access.

You can do this defining different offsets and lengths when you create your map view.

MemoryMappedViewAccessor WriteMap = MemoryMapped.CreateViewAccessor(0, 1024,
MemoryMappedFileAccess.ReadWrite);

MemoryMappedViewAccessor ReadMap = MemoryMapped.CreateViewAccessor(1025, 1024,
MemoryMappedFileAccess.Read);

Now you can enjoy the power of this high performance data sharing model on your applications when you compile them with .NET 4.0! Note that this blog is based on beta 1 information, I will check the post once is released.

Design: Architecture Frame Considerations (Part 2)

Salva Patuel — Tue, 10 Feb 2009 19:58:00 GMT

In this second instalment we are going to review another set of frames. This guideline should help you to understand what is involved on each one of them as well as considerations and known patterns.

You can find the first part following this link.

The second part looks like this:

The following guideline can help you to identify the key issues on each area of the frame and some recommendations that we usually share with our customers in the ADC team (Application Development Consultant ). Note that this list is not exhaustive and are general suggestions that can help you on your design process.

Configuration Management
The message here is clear, we need to move configurable options to a configuration file so the user can change it without recompiling the application. Yes, is quite clear but the problem is that developers usually overlook how this information is accessed and secured. We have seen many times user names and passwords on these files without any security! Not only have that, with time these configuration files grow to unmanageable levels that jeopardized the application execution.

Key Issues:

· Lack of or incorrect configuration information

· Sharing configurations around farms

· Not securing sensitive configuration information (exposing passwords/identities)

· Lack of security around the configuration files

Consider the following guideline:

· Use IIS sharing configuration features in order to deploy configuration files over a farm.

· Consider using least-privileged process and service accounts.

· Categorize the configuration items into logical sections, this method can help you secure different areas (features like configuration delegation in IIS are very helpful in this scenario)

· Encrypt sensitive information in your configuration store

· Restrict the access to your configuration information

· Provide a separate administrative application to deal with configuration files.

Pattern	Brief
Provider	Implement a component that exposes an API that is different from the client API in order to allow any custom implementation to be seamlessly plugged in.

Coupling and Cohesion
When you are designing components for your application you should ensure that are highly cohesive and loose coupling is used across layers. Coupling refers to the dependency between components and cohesion relies on the functionality that the component provides.

Key Issues:

· Incorrect grouping of functionality

· No clear separation between objects

· Tight coupling across layers

Consider the following guideline:

· Design for loose coupling between layers.

· Consider using abstraction to implement loose coupling between components.

· Consider using common interface definitions

· Use dependency inversion to decouple dependencies, depending on abstractions rather than concrete components.

· Divide your application functionality into logical layers; if you are dealing with customers you shouldn’t include your message binding properties on the same object.

· Design for high cohesion. If a component says “Customers” it should provide only customer functionality.

· While loose coupling requires more code, the benefits include a shortened dependency chain and simplify the building process.

Pattern	Brief
Composite view	Combine individual views into a composite view.
Inversion of Control	Populate any dependencies of objects on other objects or components that must be fulfilled before the object can be used by the client application.
Template View	Implement a common template view, and derive or construct views using the template view.
Two-Step View	Transform the model data into a logical presentation without any specific formatting, and then convert that logical presentation into the actual formatting required.
View helper	Delegate business data-processing responsibilities to helper classes.

Data Access
Separating the data access layer (DAL) is an important decision that improves the application maintainability and extensibility. This will allow you to easily change the repository or the logical structure of your data model without affecting the application functionality. It is recommended to implement an entity model to translate physical designs into business ready entities.

Key Issues:

· Per-user authentication and authorization when not needed

· Chatty database calls interfaces

· Data access code embedded with business logic

Consider the following guideline:

· Avoid coupling your application model to the database schema. Use abstraction models like the entity framework.

· Enforce data integrity in the database, not through data layer code.

· Move business logic to the application layer, avoid long stored procedures as they make applications harder to migrate and control.

· Open connections as late as possible and release them as early as possible.

· Avoid accessing the database from different layers, the DAL should be the only component that knows anything about the repository.

Pattern	Brief
Active record	Include a data access object within a domain entity.
Data mapper	Implement a mapping layer between objects and the database structure that is used to move data from one structure to another while keeping them independent.
Data Transfer Object	An object that stores the data transported between processes, reducing the number of method calls required.
Domain Model	A set of business objects that represents the entities in a domain and the relationships between them.
Query Object	An object that represents a database query.
Repository	An in-memory representation of a data source that works with domain entities.
Row data gateway	An object that acts as a gateway to a single record in a data source.
Table data gateway	An object that acts as a gateway to a table or view in a data source and centralizes all of the select, insert, update, and delete queries.
Transactional Script	Organize the business logic for each transaction in a single procedure, making calls directly to the database or through a thin database wrapper.
Table module	A single component that handles the business logic for all rows in a database table or view.

Exception Management
It is important to track all the application exceptions to make the system more reliable and to properly capture application defects. A crash can be very costly for customers and is not something that you want to re-experience in order to get more information. A proper exception management should provide you with enough information when the fault occurs, this should include as much information as possible in order to reproduce the scenario.

Key Issues:

· Unstable state exposure

· Revealing sensitive information to the user

· Using exceptions to control logic flow

· Inaccurate/non-existence logging information

Consider the following guideline:

· Do not reveal sensitive information in exception messages and log files. I have seen many websites showing the full stack when an error occurs!

· Let exceptions flow if they are unexpected, catching everything without proper logging may provide a false end user experience that can lead to corrupted entries.

· Design an appropriate exception propagation strategy.

· Send the information back to a central repository when possible to analyze trends when large user based is expected.

· Log information on common repositories, like log files, event logs or external web services.

Pattern	Brief
Exception Shielding	Prevent a service from exposing information about its internal implementation when an exception occurs

Layering
Proper layering design allows you to separate the functionality into different areas of concern. This will help you to group functionality and components that will make your system more predictable and able to scale out. Layers should have a well defined communication flow, for example, layer A can talk with layer B but not the opposite scenario.

Key Issues:

· Incorrect grouping of components within layers

· Not following layering and dependency rules

· Not considering physical boundaries

Consider the following guideline:

· They should be used as logical groups, for example, user interface, process layer, application layer, data layer.

· Components within a layer should be cohesive.

· Consider physical boundaries when designing layers

· Use interfaces to define each layer, this will allow proper communication and different deployment models.

· If you are running web applications consider using message-based interfaces to provide a stateless user interface.

· Provide a facade pattern to the process and application layers.

Pattern	Brief
Remote Facade	Creates a high-level unified interface to a set of operations or processes in a remote subsystem to make that subsystem easier to use, by providing a course-grained interface over fine-grained operations to minimize calls across the network.

The next part will complete the set of considerations, watch this space!

Design: SOA Taxonomy, an object oriented approach

Salva Patuel — Tue, 02 Sep 2008 14:05:00 GMT

An essential part of the SOA Governance is to properly identify the requirements in order to introduce or upgrade a service within your SOA model, this is commonly known and service and integration planning within the SOA lifecycle. This is all good in theory but the nightmares start when architects are trying to put it in practice.

As an application development consultant here at Microsoft we constantly find chaos in the SOA first attempts, the main reason behind this is the lack of services organization. This not only brings problems to the model that are expensive to change but does not shows business value to the stakeholders, as they cannot see any ROI in the implementation. As a result, the efforts are abandoned and the architectural model remains broken.

How do we help our customers to solve this issue? We usually take them a level higher within the abstraction layer, why do you need SOA? Which are the problems that you usually have? Which are the costs involved in changing your system? We really need to understand our driven forces that lead us to a new architectural paradigm.

But what about designing the services model? This is another area where confusion arises as architects ask themselves where they should start and how. The answer to this is enclosed on the services taxonomy. There are many categorizations and naming around taxonomy but I want to present a recommendation based on the Microsoft services experience.

We define four main categories that really help us to define the high level model. Let me explain each one of them.

· Entities Services: The entities encloses your business entities, these are independent actors within your business model. The entities do not provide business benefit acting alone as they don’t have the expertise around which other entities are around. They should be completely independent and should own the data repository. A good example of an entity service will be customers. Customers are nothing more than what the name says, it has and owns all the information about the entity “customer” but knows nothing about your business.

· Capabilities Services: This is the crucial part of your business; this is what you do to add value to your services. The capabilities know very well the entities and capabilities context where it works, as interacts and consumes services to deliver value. An important thing to understand is that they know about the capability but do not control the entities that consumes, this means that if a capability needs to alter an entity it should use the entity service responsible for that. The capability also owns its own repository and controls all the business logic associated with it. A good example of a capability is an order service. The order service will know everything about placing an order and will need entities in order to generate one, for example, the order will need the customer information that comes from the entity service.

· Application services: These services supports your business model, this means that provides auxiliary services to your entities and capabilities not directly associated with your system. This usually involves wrappers to third party application or bridges to other systems. A good example may be a service wrapper to a Remedy application.

· Bus services: This taxonomy groups all the support and infrastructure services that allow your entities and capabilities to work within the infrastructure policies. For example the security services that validate users can be grouped in this taxonomy.

As you see, with this categorization you can start to add services to your model. Where do you start? The easiest way to do it is asking yourself what your business does, what is the essence of it. Let’s imagine that you are the responsible for designing the SOA model for a large supermarket. The amount of services that you will need is massive but if I ask you about the essence of your business which will be the answer? Believe it or not this sounds easy but architects that are too embedded in the daily duties cannot properly answer the question. My model would be:

Look how simple it is, you just join customers with products and generate orders. Now, if I ask you to add these services in the taxonomy model you now know how to do it.

Remember that upon I have presented orders as the linking service between the entities this does not suggest hierarchy. All the services are at the same level as they can be accessed by an ESB (Enterprise service bus) or ISB (Internet service bus), or even simply your process layer. What is more, what you can do is apply the same model when you enter into the service design scope; this is for example when you design the internal taxonomy of “Orders”. This service will have again a process layer, entities and capabilities, but now only around the orders scope. Does this sound familiar? Well, it may, as it is a common model applied to the object oriented design.

Internally, these new entities and capabilities can be isolated components that are responsible for a solely function, for example, an internal process can be the workflow that triggers a new order and a capability may be altering the order value.

This can show you that applying the SOA governance model around taxonomy can help you and your developers to properly design services and object oriented components properly isolated and encapsulated, as is the same mindset. What is more, the WCF team is working in a “in-process” binding that will allow you to use a WCF communication with objects that today are part of your main process and tomorrow may develop into independent services just changing the configuration file! Isn’t it beautiful?

If your organization is facing these problems and you want Microsoft to support you and help you around your SOA initiative don’t hesitate the contact the “Application Development Consultant team” in Microsoft (http://www.microsoft.com/uk/adc)

Design: Classic stored procedures or Entities?

Salva Patuel — Fri, 16 May 2008 16:01:00 GMT

With the new entities framework to be released in the next service pack some of our customers and team members are starting to discuss what is the correct pattern. I wanted to extract my position around the choices.

I am big supporter of having options, in this case I don’t consider one better than the other one, the stored procedure option can be the best choice for one scenario and LINQ, with all the different flavours including Entities, for another one.

You can go all the way separating process layers using stored procedures that may sound perfect for your scenario. In this scenario the architects and developers must maintain the complexity inherit from this approach as the data is usually quite tight to the processes. This adds risk to the projects making them more complicated to maintain (you need to involve the DBA for schema changes!). Some people may argue that this is the most performance approach but I am a strong believer that usually performance is not well understood. What is fast enough for an application? Do you need to go to the extreme performance when it may not be needed, sacrifying functionality or simplicity? If that is the case SP approach can be the best for you.

Now on the other side, I love the idea of separating the physical model from the logical model (so I can get rid of the DBAs! J). Business entities are much better represented with the entity framework and gives developers another layer to abstract the schema complexity, using strongly type models that can help detecting errors at early stage. Yes, there is a performance penalty price to pay, but maybe the functional benefits really help the business to develop quality software reducing the project risk. Have you measure how slower is it? You will be surprise with the results if you correctly design the entities.

The LINQ story is quite powerful as well, it is true that you have to consider learning curve and the community is taking longer than expecting to digest it. But with LINQ to entities things are starting to get more sense and I can imagine more people jumping into it.

What I think is cool? ADO.NET Data services (do not confuse with SQL Server Data Services! ), as is a great story for another cool technology like Silverlight. In this model you have your physical database completely abstracted from the model as you use the entity framework, you can use LINQ to query it and then post it using REST. This will send the results using XML, transformed to a struct using Silverlight Data services directly binding the results to your grid! That’s what I call functional abstraction.

To conclude, I still believe that it depends on the scenario, the beauty of this is to have options.

Design: The Sync framework

Salva Patuel — Tue, 15 Jan 2008 16:01:00 GMT

In first place let me answer the question why do we need a sync framework. The current connected scenario present different challenges for developers, the technologies are there but all of them are different. If you want to synchronize your pictures with your computer you will use the photo sync technology, if you want to share music you will use a custom application, and if you want to share data you may use replication technologies just to mention some examples! Things become a little more complicated if you want shareable repositories like back-up models. Today the industry is taking us to the connected devices environment adding wireless and USB connectors to everything (remember the promise of an intelligent washing machine?). I believe is time to have a common framework that we can use to normalize this approach, this is time for the sync framework!

The framework comes as a separate installation as is at the moment of this article just a CTP, for more information about the basics you can check here. In this article I am going to talk about the internal architecture and how this can be used by designers in order to build scalable synchronization models aligned with the Microsoft roadmap.

The framework architecture is based on service providers, at the moment Microsoft is offering file services, ADO.NET services and SyncFeed (for RSS) providers. All these providers are used by the synchronization framework that is responsible for synchronizing them via the metadata information.

The application initializes the services that is going to use and then will use the SyncAgent to synchronize the local and the remote provider. The sync agent consumes the sync providers in order to synchronize the knowledge between the local and the remote provider. The agent publishes progress objects that can inform the client application about the status of the synchronization (quite handy for progress bars). As the synchronization runs on the callers thread this is a good candidate for a background thread to enhance the user experience.

An interesting feature of the file provider is the ability to act based on defined properties, for example, we can set that after any delete or change due synchronization the files will go to the recycle bin, using the following option:

FileSyncOptions.RecycleDeletes | FileSyncOptions.RecycleOverwrites

The filters can be configured using the FileSyncScopeFilter allowing the developer to define which files to include on the synchronization. Upon these options are published for the file provider each of the other ones will expose custom properties to set up the provider, including vertical and horizontal filters on database tables (similar to the replication model).

Different approaches can be taken when dealing with the synchronization direction, supporting a client server model, mesh or peer to peer. This will depend on the participant types:

· Full Participant: These participants will host the framework and will be responsible for triggering the synchronization.

· Partial Participant: These participants are dummy storages, which receive and synchronize information from a full participant but do not host the framework (either is aware of it). For examples, USB drives, cameras, mp3 players, remote folders

· Simple Participant: These participants provide data but do not store it, for example an RSS feed or a webservice that provides enumerable data. Again, a full participant is needed to host the framework.

The direction of the synchronization is based on the following parameters:

SyncDirection.UploadAndDownload
SyncDirection.Upload
SyncDirection.Download
SyncDirection.DownloadAndUpload

As we can see, the framework is a powerful option that can help to use a single synchronization pattern for the different models. If the implementation is a success I don’t see a reason why current replication and synchronization models like SQL replication can be based on this model, look at what happened to WF and BizTalk J.

Regarding the internal works of the framework, at the moment is in CTP and is likely to change. The current implementation is also accessible through unmanaged code as the file service provider for example uses COM to interact with the file system, not the same story for the ADO.NET provider, but at the end that is the beauty, independent providers using a common framework.

Architecture: Multi-channel with WPF

Salva Patuel — Tue, 08 Jan 2008 03:35:00 GMT

As architects, we should see the world with tendencies oriented eyes, being aware of the different signals that the market shows behaving like explorers. Sometimes is a difficult task that demands us going the extra mile – I like to see a good architect as that one that can properly filter what is important and what is not. The latest trends on the consumer space are showing us that we should care about the multichannel experience. On this blog entry I am going to focus on the user experience and how affects the architectural decisions.

We should have a common programming model to develop user interfaces, but at the moment is a bit challenging with many technologies around, making very difficult to bring platforms and formats together maintaining the same consistent user experience. Some attempts using the web had brought some success but at a high maintenance cost with some unpleasant stories. This is one of the main reasons why I am a big supporter of WPF. Having a common language that can be rendered on different platforms and modalities is helping us to decide which next user experience technology we should include in our roadmaps. With a single Ux investment stream we can keep the developers team focused reducing the learning curve and optimizing the production, with the benefits of reducing cost when we need to bring a new channel into the equation. Remember that a good architect should look into this other factors that constrain the business, rather than the usual appetite for new technologies.

But nothing comes cheap; as Microsoft is pushing for these technologies we are facing challenges on the implementation side. The multiple channels today require a different sub-set of the render engine (embedded in a framework subset), making our jobs a little harder. The challenge is to find the thin line that divides the flexibility with the complexity, dividing what we share and what is custom code.

This is a good place to put emphasis in design patterns, those patterns mostly avoided not for the complexity but fear. If we put it in context is easy to separate the service layer from the user layer but things get a little more complicated when architects and developer needs to code the user layer. The most common approach is to build it as a monolithic application (from the VB times), as the separation has been done using the typical three tier model. Well, let me tell you that we are facing new challenges where this model is not applicable anymore. Consumers are demanding new ways to interact with the application; what is more, enterprises will come to you arguing why they should use certain platform to run your application. Your nightmare will not end there, operations - finally taking into consideration – will come to you demanding features to support and maintain the system that you are architecting. As I have introduced WPF as a convenient technology you still can shoot on your foot. This is where the patterns like MVP can really help you to separate the business logic of your user layer application.

A contemporary approach that I usually coach is the mindset of layer bubbling. This concept starts from the forest correctly dividing the layers, using maybe SOA approaches to promote reusability and stability (as many other qualities of service). Once you have finished the forest approach you should take each layer and treat them using the same pattern, this means that your user layer will have a view (Ux), a controller or presentation (App) and your model (Data), this approach will help you to reduce the amount of custom code that you will need for the multi channel approach.

With this type of approach you will be able to asses which parts of your applications and data will be stored where, furthermore, how the information is shared. In our specific case, the model will be the proxy interface that retrieves information from our services, then the user interface workflow will be coded separately, as it should not be compromised by the model neither the view (how is presented). In the past this model was hard to implement due the fact of the view code, coding in win forms, html, mobile was very expensive and hard to support. XAML comes to the rescue.

Another important task as an architect is to envision the future, how this architecture model will stand in time. Well, I can foresee the platforms coming together at some point, reducing the amount of gaps on the rendering engine. It will take longer to have a common framework for all these channels but at least a uniform WPF render engine will do the work. Today we have some items that are supported on SilverLight, others in WPF desktop and XBAP (and mobile quite soon), but I can see standardization in the near future.

This little architectural roadmap has shown as how important is the use of the two main architectural pillars: An explorer and an envisioner. Put them in practice!

Design: Ephemeral cache explored

Salva Patuel — Mon, 06 Aug 2007 16:19:00 GMT

Many of .net developers are familiar with the ephemeral model included in the garbage collector. The idea of using generations in order to manage the lifespan of the objects is extremely helpful when you need to control the life of an object. Incidentally, this is applicable to custom cache models where content expiration is an issue.

There are many ways to control the content expiration on a cache and all of them bring to the table pros and cons. Most of the challenges come from the cache size and how the expiration can be controlled using time stamped data. Time structures are expensive and bulky, and browsing the content in order to find the expired ones can be a painful tasks. Many techniques have evolved in order to solve the problem like date indexing or date bucking, but they require careful control over the manipulation and sometimes even index persistence.

The main problem arises when we manipulate a big cache, if we use time stamped a secondary thread, responsible for removing garbage, will analyse the list of entries. If the entry time stamp does meet the expiration criteria the entry is removed. This not only can take resources, also affects the storage fragmentation.

The same concept of the garbage collection can be also applied to our cache service, if we split the cache in generations we can control the cache quality, moving commonly content to a more secure and less intrusive list. When a new query arrives the cache service will retrieve the information from the resources (i.e. a DB). The call is responded and a secondary process inserts the content on the generation 0 cache. This generation tends to grow rapidly as all the call responses are cached. As this cache will grow very quickly using time stamp expiration will be very expensive.

Now, most of this cache will be loaded in memory just because a random single request read from a record that maybe will not be used for the rest of the cache lifespan. This is a real waste of resources. Applying cleaning processes to the cache will leave it very fragmented, reducing the performance. If we want to differentiate commonly used data and non common we can implement a new generation, where the data accessed on the generation 0 is promoted to the generation 1. The second cache will be smaller and will have more useful information, making the first choice for a new query.

We can see on the graphic that the call first go to the older generation, if the result is not there it will go to the first generation. Once is found the entry can be promoted. With this model the first generation can be purged quite often, as is information only requested once (note that an access counter can also be added to the generations adding rate rules like “you need 5 requests in order to be promoted”). What is more, the model can be correctly designed to support more generations keeping a healthy cache with accurate data loaded.

As the higher generations contain valuable information, it is a good idea to persist those caches, otherwise when the service is restarted we are going to lose the valuable tracking information. I tend to use file storages on generation 1 and even DB storage for generation 2. This means that the most valuable generation (2) will be able to be loaded by other caches in our distributed environment. This increases efficiency as the caches will contain only valuable information.

I have personally tried this model on large cache (4 million entries) and the performance and resource consumption are balanced with the response time. The volatile cache is dropped by slides avoiding fragmentation and can be triggered by time or even by resource pressure. If you need more information regarding the implementation do not hesitate to contact me.

Design: How good SOA can help you dealing with multi-cores

Salva Patuel — Mon, 09 Jul 2007 17:59:00 GMT

Unless you have been living under a rock for the last 5 years you should know about the multi-core architecture that the new processors bring to our world. Well, processors hardly can go any faster with the current technology therefore we need to start to have parallel thoughts...

Well, you are about to design a solution and you are facing the challenge “How I can take the most of the multi-core processor that my server has” Sometimes is not easy to take advantage of this extra horse due the lack of parallel design skills, I have seen many attempts to achieve this with more headaches than solutions.

After working on several projects where we were trying to improve the performance of certain components I came across multiple solutions but there is one tendency to solve this problem that comes within the SOA shadow. Thinking about services can help you to divide the pieces of functionality that can be parallelized; this will provide a natural processor optimization with the incorporation of different processes to get the maximum output from the multi-core environment.

I have seen many times a SOA model that includes a service multi layer derived from an ontology model, where the hierarchy drives to the conclusion that some services must be componentized within a single service to reduce complexity. The problem of this merge is that sometimes the developers fail to divide the components (adding dependencies) or they limit the multi threading capabilities of the hardware layer.

In the figure above we can see the service hierarchy that encapsulates service C1 and C2, when they are implemented in a single process the services needs to properly handle the threading model and the communication model, using memory synchronization patterns. This adds locking and latency problems that reduce the performance and scalability when the service is scaled in.

If we decide to divide our services in different process and use the common contract as communication layer we can solve the problem reducing complexity and isolating the threads. This helps you to reduce the component coupling and scales in better and cheaper. There is no need for synchronizating shared resources as the process isolation will dela with the individual heaps.

What is more, the model can have processor affinity to control the load on the hardware, this can became more common in a 4x4 architecture where 16 processors can give you not only the service isolation, also the hardware. As software architects need to deal with teams without the proper parallelism skills, this type of approach can help you reducing project risk and delivers a proper set of services to the business processes that allows you more flexibility if a new process needs the C1 service directly.