Grigori Melnik: Thoughts on Agile Software Engineering and Beyond

Microsoft Enterprise Library 6 - Wave 2 Release

Grigori Melnik MSFT — Fri, 24 May 2013 22:49:00 GMT

Last month we shipped the new versions of Microsoft Enterprise Library and Unity. They've been very well received. Today we are pleased to add the following guidance assets to enhance the set of reusable components:

Reference Implementation (RI) – an end-to-end sample application that showcases the use of various application blocks from Enterprise Library.

Note the RI comes in 2 versions – for EntLib v5.0 and for EntLib6. This was intentional as we have never produced an RI for v5.0 and by doing so we were also able to guide you through a migration process.
It’s highly recommended you review the installation guide packaged with the RI first.

Enterprise Library 6 binaries – full set (includes all binaries and doesn’t require any ports to be open to pull additional dependencies via NuGet) – this is in response to the requests of the customers in constrained network environments.
New Unity Developer’s Guide with code samples.
Updated Enterprise Library Developer’s Guide (preview) with code samples.
Enterprise Library Quickstarts, including:

Semantic Logging: Flush on Error Trigger.
Semantic Logging: Alarm Flood Throttle.
Reconfiguring Logging Application Block at runtime (via a verbosity setting in an appSetting in the configuration file).
Reconfiguring Semantic Logging Application Block at runtime (via a service setting for a Windows Azure role).

Unity Quickstarts:

StopLight showcases basic container usage such as type mapping and resolving with attributes.
Event Broker showcases the use of a custom extension to wire up events instead of wiring them up through code.

Updated Config tool VSIX package for Visual Studio 2012

Addresses an issue of side-by-side editing of v5 and v6 solution properties if you have both installed .

Updated Unity reference documentation (preview).
Updated Enterprise Library reference documentation (preview).

As a reminder, we also have a Migration Guide, which we shipped earlier, to help you move your app from using EntLib v5 to v6.

We hope these additional materials will help you learn EntLib and Unity and get quickly up to speed in using them.

In p&p we consider the written guidance that complements our code to be as important as the code itself. We spend a lot of time and effort making sure that it’s technically accurate, as well as engaging to read. We appreciate your patience while we work on polishing and finalizing the developer’s guides and making them available as books (both printed and eBooks). We’ll gratefully accept your feedback on any of the above. We are also seeking EntLib and Unity users willing to share their stories as case studies to be included in the guides. If you are interested in contributing a short story, please contact me for details.

Happy learning, happy experimenting, happy coding!

Dealing with Windows Azure Storage transient faults

Grigori Melnik MSFT — Wed, 22 May 2013 23:22:37 GMT

Preamble

When an application uses a service, errors can occur because of temporary conditions such as intermittent service, infrastructure-level faults, network issues, or explicit throttling by the service. These types of error occur more frequently with cloud-based services, but can also occur in on-premises solutions. Very often, if you retry the operation a short time later (maybe only a few milliseconds later) the operation may succeed. These types of error conditions are referred to as transient faults. Transient faults typically occur very infrequently, and in most cases, only a few retries are necessary for the operation to succeed.

The Transient Fault Handling Application Block (Topaz) lets developers make their applications more resilient by adding robust transient fault handling logic. It encapsulates information about the transient faults that can occur when you use the following services in your application:

SQL Database
Windows Azure Service Bus
Windows Azure Storage
Windows Azure Caching Service

This blog post focuses on Windows Azure Storage.

Retries in Windows Azure Storage Client

The Windows Azure Storage Client Library has supported retries for a while as an integral part of the API. Instead of wrapping each API call with a call to one of Topaz's RetryPolicy methods, you would set the storage client’s RetryPolicy property and execute the action directly. In fact, you would get retries by default, so in order to use Topaz's retries you had to explicitly override the default retry policy in the storage client.

Using built-in support for retries, if such support is available, is usually more appealing than using external support like Topaz's: code is terser, and the API has more information about the actual operations it retries so it can be more effective. With external retry support an entire operation needs to be retried, while retries managed by the API can retry specific requests within a single operation. Also, when the API might perform additional work after it has returned a result (for example, if it returns an IEnumerable that might result in new requests to a service to pull new results while enumerating) an external retry would need to include not only the API call but the processing of the results.

Why would you want to use Topaz's retries instead of the built-in retry support from the Windows Azure Storage Client Library?

The built-in retry support prior to Windows Azure Storage Client v2 had two shortcomings:

It was relatively hard to extend, since it was based on delegates rather than classes.
It only allowed you to specify how long to wait but not which exceptions should be retried; t that decision was still made by the API. You could refuse a retry for exceptions that the client library considered should be retried, but you couldn't force a retry for exceptions that the client library had already vetoed.

With v2 the support for retries was overhauled to address these two shortcomings: it is class based now, and the new IRetryPolicy implementations are responsible for determining what exceptions should be retried. Because of this change, our recommendation now is that you use the built-in support for retries if you're using v2 of the Windows Azure Storage Client Library.

Does this mean that Topaz does not support Windows Azure Storage Client Library v2?

Absolutely not. Topaz supports retries for the new versions of the client library, as well as previous versions. You can use it if you wish, particularly if you want to benefit from Topaz's configuration support. It's just that as the Windows Azure Storage Client library evolved Topaz's support became less necessary so the recommendation changed.

Can you show me the difference in usage?

Here are examples of what using Topaz and the built-in retries look like. In both cases the initialization of retry policies can be separated from performing the requests, and many requests might share the same retry configuration.

Using Topaz

// setup retries
var retryPolicy =
    new RetryPolicy<StorageTransientErrorDetectionStrategy>(
        new ExponentialBackoff(3, TimeSpan.FromMilliseconds(10),
            TimeSpan.FromMilliseconds(10000), TimeSpan.FromMilliseconds(200)));

client.RetryPolicy = new NoRetry();

// execute operation with retries
try
{
foreach (var container in retryPolicy.ExecuteAction(() =>
client.ListContainers().ToArray()))
{
Console.WriteLine(container.Name);
}
}
catch (Exception e)
{
Console.WriteLine("Exception occurred: " + e.Message);
}

Note how the IEnumerable of containers is converted to an array as part of the operation to retry (which might result in some overhead) to ensure that all requests to the Windows Azure Storage services are retried.

Using built-in retry policies:

// setup retries
client.RetryPolicy = new ExponentialRetry(TimeSpan.FromMilliseconds(200), 3);

// execute operation with retries
try
{
foreach (var container in client.ListContainers())
{
Console.WriteLine(container.Name);
}
}
catch (Exception e) {
Console.WriteLine("Exception occurred: " + e.Message);
}

Many thanks to Fernando Simonazzi for contributing to this post.

Just released - Microsoft Enterprise Library 6

Grigori Melnik MSFT — Fri, 26 Apr 2013 00:55:00 GMT

Five months ago we formulated our vision for the new version of Enterprise Library. Now we are delivering on it. I’m excited to announce the latest release of Microsoft Enterprise Library: version 6.

What is Enterprise Library?

Enterprise Library is made up of application blocks, each aimed at managing specific crosscutting concerns. Crosscutting concerns are those tasks that you need to accomplish in several places in your application. When trying to manage crosscutting concerns there is often the risk that you/different team members will implement slightly different solutions for each task at each location in your application, or that you will just forget them altogether. Writing entries to a system log file or Windows Azure table storage, dealing with transient error conditions and validating user input are typical crosscutting concerns. While there are several approaches to managing them, the Enterprise Library application blocks make it a whole lot easier by providing generic and configurable functionality that you can centralize and manage.

Enterprise Library application blocks are standalone. They work well together, but you only have to get the ones that you need. They are also customizable and extensible, so you can extend them to provide what you need in your specific contexts. You can choose to use it as a seedwork and grow your own library, which you can later reuse and sell. We ship under MS-PL, so this is allowed.

What are the main themes for this release?

Simplifying the library all around
Embracing semantic logging
Increasing resiliency to errors
Enhancing Unity type registration
Supporting Windows Store apps (Unity, Topaz)
Streamlining programmatic configuration of all blocks
Integrating with other technologies (ASP.NET MVC and ASP.NET Web API)
Improving ease of learning, ease of experimentation (fast start), and ease of use

What’s in the box?

New block

Semantic Logging Application Block (v1.0)

New programmatic configuration
Updated 7 blocks:

Data Access Application Block
Exception Handling Application Block
Logging Application Block
Policy Injection Application Block
Transient Fault Handling Application Block
Validation Application Block
Unity Application Block/DI Container (v3.0)

Configuration console (largely unchanged since v5)

As usual, we are shipping not only the binaries, but also the source code and test cases (unit, integration). To see the details of what have changed and deployment instructions, see the Release Notes.

The diagram below depicts all application blocks and their dependency on the Common package (needed to support configuration tool experience) and optional interblock dependencies.

Additionally, in the next couple of weeks we’ll be shipping a reference implementation and several Quickstarts to showcase how the Enterprise Library can be used in certain scenarios.

How to get it?

Our primary shipping channel for code is NuGet. There are 27 NuGet packages. Search for Unity or EntLib inside NuGet Package Manager.

Additionally, we’ve made all deliverables available as self-extractable zip packages via Microsoft Download Center.

Our guidance deliverables are available on Codeplex today. After post-production work is completed, they will be published on MSDN. You will also be able to order the hardcopies of the Developer’s Guides or get them in PDF, Epub or Mobi formats.

How to get started?

Whether you are experienced with Enterprise Library or new to it, we have guidance to help you make the most out of this release:

Dependency Injection with Unity – new Developer’s guide (preview on Codeplex)
Updated Developer’s Guide to Enterprise Library (preview on Codeplex)
Updated reference documentation (preview on Codeplex)
Reference implementation (v5 and v6) (available in May)
Reference API (on MSDN)
Quickstarts (available in May)
New Migration Guide to help you move from v5 to v6
Videos (on Channel 9)

The table below will help you orientate:

Do I have to upgrade?

Strictly speaking, no. If you are happy with Enterprise Library 5.0, you can continue using that version. If the new and improved features of Enterprise Library 6.0 appeal to you, then you would want to migrate. Note: Enterprise Library 6.0 targets .NET4.5 framework only. Enterprise Library 5.0 works fine on .NET3.5, .NET4.0 as well as .NET4.5. So, in order for you to move to .NET4.5, you don’t have to upgrade.

How to get help and provide feedback?

Many people provided feedback on our vision, our backlog and CTPs. We thank you all.

If you’d like to provide further feedback, please post it via the Codeplex forum. This is where you can also get support. We have a dedicated sustained engineering team monitoring the forum regularly. To report a bug, use online Issue Tracker.

If you have a story of how your team leverages EntLib and would like to share it with the broader community, please contact us. We’ll be happy to work with you on a case study.

Happy coding!

Unity Configuration - Registration by Convention

Grigori Melnik MSFT — Tue, 12 Mar 2013 22:03:26 GMT

To make container setup easier and less verbose, we are adding a new feature in Unity 3.0 – registration by convention.

What is the Registration by Convention feature?

This feature (also referred to as auto-registration) is intended to simplify your type registrations. Rather than specify each type mapping individually (lots of container.RegisterType<IFoo, Foo>()) , you can direct the Unity container to scan a collection of assemblies and then automatically register multiple mappings based on a set of rules that identify the types and their locations.

The following code shows example usage of the current implementation of registration by convention:

using (var container = new UnityContainer()) {

container.RegisterTypes(

AllClasses.FromAssembliesInBasePath(),

MappedTo.MatchingInterface,

WithName.Default,

WithLifetime.ContainerControlled);

}

What conventions are supported?

A convention must specify which available types should be mapped in the Unity container as well as how those type mappings are defined. Currently, the available conventions work by specifying the following rules:

1. Identify the list of assemblies to scan for types to be registered: you can provide your own list of assemblies, use only currently loaded assemblies, or use assemblies that reside in the base path of your application. You can further filter the list of types by names, suffixes or other predicates by using LINQ queries or extension methods. This first step results in a list of types to potentially register with the container. By the way, by default, all system assemblies will be skipped – you have to opt-in for them to be included. Note: you don’t need to start from our helper method for loading types. You can use any method to obtain an enumeration of types, even as simple as an array of literal types.

2. Optionally, specify which types (typically interfaces or abstract classes) you want the container to resolve to each of the types identified in step 1. Currently, conventions include:

a. Create mappings where the following naming convention is followed: type Foo and interface IFoo.

b. Map each type to all of the interfaces it implements.

c. Map each type to all of the interfaces it implements where those interfaces are in the same assembly as the type.

3. Optionally, specify whether to use a named registration. In this case Unity uses the type name as the name of the registration.

4. Optionally, specify which lifetime manager to use for the registrations.

5. Optionally, identify any members of the type that should be injected.

What about Windows Store Apps?

The registration by convention will also be supported by Unity for Windows Store Apps with some limitations enforced by the framework, such as you won’t be able to scan the loaded assemblies and you won’t be able to scan a particular folder, but you can assemblies packaged with the app.

What other specific conventions should we support?

As always, we seek your feedback on the new features. As we are finalizing the implementation of the registration by convention, it would be helpful to get your thoughts on the following:

1. Do you see a requirement for out-of-the-box automatic creation of mappings for abstract types as well as interfaces?

2. Do you see a requirement for out-of-the-box support for scanning other locations for assemblies? E.g. arbitrary folders.

3. In scanning the assemblies in some location, would you want to have a way of annotating specific assemblies with an optional attribute?

4. Do you see any requirement for context specific conventions? Such as ASP.NET MVC, WCF, etc. Or is Unity auto-wiring sufficient in these cases?

5. If you are providing injection members, some of those will be associated with the “from” type, but most will be associated with the “to” type. Do you see the need to be able to specify more fine-grained injection members, or is the support for “to” types sufficient?

Embracing Semantic Logging

Grigori Melnik MSFT — Thu, 07 Feb 2013 21:34:00 GMT

UPDATE 4/25/2013: The Semantic Logging Application Block final version is released. Get it via NuGet.

UPDATE 2/14/2013: The Semantic Logging Application Block CTP is now available.

In the world of software engineering, every system needs to log. Logging helps to diagnose and troubleshoot problems with your system both in development and in production. This requires proper, well-designed instrumentation. All too often, however, developers instrument their code to do logging without having a clear strategy and without thinking through the ways the logs are going to be consumed, parsed, and interpreted. Valuable contextual information about events frequently gets lost, or is buried inside the log messages. Furthermore, in some cases logging is done simply for the sake of logging, more like a checkmark on the list. This situation is analogous to people fallaciously believing their backup system is properly implemented by enabling the backup but never, actually, trying to restore from those backups.

This lack of a thought-through logging strategy results in systems producing huge amounts of log data which is less useful or entirely useless for problem resolution.

Many logging frameworks exist today (including our own Logging Application Block and log4net). In a nutshell, they provide high-level APIs to help with formatting log messages, grouping (by means of categories or hierarchies) and writing them to various destinations. They provide you with an entry point – some sort of a logger object through which you call log writing methods (conceptually, not very different from Console.WriteLine(message)). While supporting dynamic reconfiguration of certain knobs, they require the developer to decide upfront on the template of the logging message itself. Even when this can be changed, the message is usually intertwined with the application code, including metadata about the entry such as the severity and entry id.

Thus, the discrete event payloads get flattened and the unstructured or semi-structured logging messages become the norm. Take a look at the following section of a typical log file:

This is what typical string-and-property-bag-based logging produces. While it may contain useful info such as the level (priority) of the log event and its source, the log message itself is totally flattened into a string. In order to parse it, knowledge of the format/template is required and that format/template must have been consistently applied in the application. This is especially true, if the log data needs to be post-processed (aggregated, filtered etc.) This pre-requisite knowledge gets lost if you are using unstructured logging frameworks today. The messages highlighted in pink and light blue demonstrate the compatibility problem, where you must actually know both templates. This problem is exacerbated when the developers are long gone; the IT/Pros are faced with the hassle of interpreting these kinds of messages, many of which don’t even use intuitive names.

Also, the code to generate log entries and make calls through logging framework facades, often pollutes the business logics and make the core code less readable.

Imagine another world, where the events get logged and their semantic meaning is preserved. You don’t lose any fidelity in your data. Welcome to the world of semantic logging. Note, some people refer to semantic logging as “structured logging”, “strongly-typed logging” or “schematized logging”.

Take a look at the next example. Here I took an event from the Event Log.

It may appear that this is no different than the event message discussed earlier. This is because the General tab formats the event data into a human readable string. It flattens it, but it’s an artifact of the tool, and it only displays this way because I, the reader, chose to view it this way. Underneath the raw message in structured format is preserved. Switching to Details, I can see that along with some system metadata, I get the event specific discrete payload, which corresponds to the message displayed earlier (I used color highlighting to emphasize how different parts of the event got used to form the message):

We hit the hassle of dealing with unstructured logging when working on the Tailspin sample app for the Autoscaling Application Block (“Wasabi”) a year ago. All visualizations for monitoring (demoed here, starting 00:16:23) would have been so much simpler to implement, had we had semantic logging in place.

The technology to enable semantic logging in Windows has been around for a while (since Windows 2000). It’s called ETW – Event Tracing for Windows. It is a fast, scalable logging mechanism built into the Windows operating system itself. As Vance Morrison explains, “it is powerful because of three reasons:

1. The operating system comes pre-wired with a bunch of useful events

2. It can capture stack traces along with the event, which is INCREDIBLY USEFUL.

3. It is extensible, which means that you can add your own information that is relevant to your code.

I’m not going to get into the details of ETW and its architecture. This MSDN article does a good job with that. What I’d like to emphasize here is that ETW is based on a different logging paradigm where you must create a class representing an event (an ETW Event Provider). Historically, this has been a non-trivial task. It also involved foreign, unnatural to .NET developers concepts (like authoring manifests) and required tedious steps. This could be one of the reasons the ETW adoption was slow (our research confirms that).

To address this, the System.Diagnostics.Tracing.EventSource type was added in the .NET Framework 4.5. It makes writing ETW providers easier and much more natural to the way .NET developers typically work. In an nutshell, developers specify an EventSource subsclass, where each of its methods defines an event and its signature defines the event’s discrete payload. This separates the code that defines events from the code that raises the events, providing the application-specific instrumentation API. Events published through methods in an EventSource class are published to ETW, and there is also the possibility of adding other destinations for events using custom Event Listener implementations. Vance Morrison provides an in-depth explanation of what kind of problem the EventSource class is solving and how. I highly recommend you watch his video tutorial on generating your own events with EventSources and the companion blog entry.

While the EventSource provides a great idiomatic way for .NET developers to author ETW events, it doesn’t come with any of the familiar destinations for logging (rolling file, database, etc.). ETW uses its own destinations such as .etl files, and uses different tools for parsing and analyzing log data.

This is where the new Semantic Logging Application Block comes in – it makes it easier to incorporate the EventSource functionality and to manage the logging behavior of your system. It makes it possible to inherit most of the structure from the ETW pipeline, including event metadata and payload, to write log messages to multiple destinations, such as flat files, databases, or Windows Azure table storage, and to control what your system logs by setting filters and logging verbosity. Note, that the event flattening problem that I have described with the unstructured logging, may still be an issue when using the file-based destinations of the Semantic Logging Application Block. But the flattening does not occur where the logging request takes place, it occurs when the log message is written and that’s the core enabler of semantic logging.

We’ve designed the Semantic Logging Application Block for .NET developers. The block is intended to help you move from the traditional, unstructured logging approach (such as that offered by the Logging Application Block) towards the semantic logging approach offered by ETW. The Semantic Logging Application Block enables you to use the EventSource class and semantic log messages in your applications without moving away from the log formats you are familiar with. Importantly, you do not need to commit to how you consume events when developing business logic; you have a unified application-specific API for logging and then you can decide later whether you want those events to go to ETW or alternative destinations. In the future, you can easily migrate to a complete ETW-based solution without modifying your application code: you continue to use the same custom EventSource class, but use ETW tooling to capture and process your log messages instead of using the Semantic Logging Application Block event listeners.

If with a traditional logging infrastructure, such as that provided by the Logging Application Block, you might record an event exhibited in the previous example as shown in the following code sample:

myLogWriter.Write(
    new LogEntry {
        Severity = TraceEventType.Information,
        EventId = 5315,
        Message = string.Format(
            CultureInfo.CurrentCulture,
            EventResources.NextPolicyProcessing,
            username,
            nextAttemptTime,
            nextAttemptTimeUnitName),
        Categories = { "Policy Processing" }});

Using the Semantic Logging Application Block, you would record the same event by:

MyCompanyEventSource.Log.NextPolicyProcessing (username, nextAttemptTime, nextAttemptTimeUnit);

where the NextPolicyProcessing() is defined separately in the MyCompanyEventSource like so, for example:

[Event(5315, Level = EventLevel.Information, Message = "Next processing for {0} will be attempted in {1} {2}", Keywords = Keywords.PolicyProcessing)]
internal void NextPolicyProcessing(string username, int nextAttemptTime, TimeUnit nextAttemptTimeUnit)
{
...
}

Notice how with the Semantic Logging Application Block you are simply reporting the fact that some event occurred that you might want to record in a log. You do not need to specify an event ID, a priority, or the format of the message when you write the log message in your application code. This approach of using strongly typed events in your logging process provides the following benefits:

You can be sure that you format and structure your log messages in a consistent way because there is no chance of making a coding error when you write the log message.
It is easier to query and analyze your log files because the log messages are formatted and structured in a consistent manner.
You can more easily parse your log data using some kind of automation.
You can more easily consume the log data from another application. For example, in an application that automates activities in response to events that are recorded in a log file.
It is easier to correlate log entries from multiple sources.

Of course, this requires you to author the MyCompanyEventSource class (subclass of the EventSource). But it makes sense for the developers to do so. It puts the effort in the right place: instead of putting all the effort in the parsing of the flat messages (and often failing at that), you put the effort into structuring your events properly in the first place. Besides, we are thinking of providing some additional aiding tools (snippets and/or FxCop rules) to make authoring of event methods even easier.

You’ll be able to use the Semantic Logging Application Block to create an out-of-process logger. This will help to reduce the logging overhead in your LOB applications because most of the processing of log messages takes place in a separate application. This is especially useful if you have a requirement to collect high volumes of trace messages from a production system. The out-of-process logger included with the Semantic Logging Application Block can also be dynamically reconfigured simply by editing a configuration file, enabling you, for example, to easily change the level of logging on the fly without touching your production system.

As we release the CTP of the Semantic Logging Application Block, we’ll demonstrate how to construct your own EventSources and route your logging to various destinations. This blog post is intended to describe the type of problem we are trying to solve and to entice you to think about the semantic logging paradigm.

In conclusion, I encourage you to read Vance Morrison’s tutorials and familiarize yourself with the EventSource. Start thinking about your logging in the new way. The Semantic Logging Application Block we are building is going to be a stepping-stone for you to get from a traditional, unstructured logging approach (such as that taken by the Logging Application Block or log4net), to a modern, semantic approach as provided by ETW.

Join us at the patterns & practices symposium 2013

Grigori Melnik MSFT — Thu, 06 Dec 2012 21:59:10 GMT

Join my team for our flagship event – the patterns & practices symposium 2013. It will take place on Microsoft campus, Jan 15-17, 2013.

Don’t miss the opportunity to see my colleagues and I speak on the latest projects coming from p&p and industry experts discussing the latest trends in software engineering. We have a fantastic roundup of speakers. Where else would you get to meet in person and hear ScottGu, Scott Hanselman, Greg Young, Adam Steltzner and Felicity Aston – all at one event?

The symposium agenda includes three tracks: The Cloud, The Client and The Engineering. Check out the details and register now. We look forward to seeing you and interacting with you over these three exciting days.

Enterprise Library 6.0 - Vision/Scope

Grigori Melnik MSFT — Thu, 08 Nov 2012 21:50:14 GMT

TL;DR version

This post is an overview of the direction the patterns & practices team is taking with the next version of Microsoft Enterprise Library. Feel free to scroll down for the specific product backlog themes or just visit our Uservoice site.

Exploration and preparation

After a couple of months of exploration, we have finalized our plans for the next major version of Microsoft Enterprise Library. We have analyzed a sea of information from various sources, including:

numerous discussions with subject-matter experts and EntLib users
discussions with the members of the advisory board
discussions with various Microsoft product groups
suggestions and votes on our Uservoice site
IssueTracker items
MSDN, Codeplex and StackOverflow forums
suggestions from our sustained engineering and Premier support teams
review of remaining stories from the previous product backlog (that didn’t get done in v5.0)
team brainstorming, categorizing, filtering
responses to our blog posts
comments in social media over the past 18 months
direct email communications with the product owner and members of the dev team
observations of users in action
log analysis (popularity of certain parts of Enterprise Library based on number of downloads and page views of the corresponding pages on MSDN)
user ranking and commenting on MSDN articles
satisfaction exit survey data
logging scenarios survey data
conversations with //Build attendees
downloads and comments on the EntLib related videos on Channel9.

We thank everyone who provided their thoughtful feedback.

We have now created a new Advisory Board, presented our vision to the leadership team, got business consensus on the themes and an approval to go ahead with the project. Our dev team is formed and we are excited about the project.

The Vision – simplification throughout

Here’s the formal vision for Microsoft Enterprise Library 6.0:

For developers, solution architects, DevOps and IT Pros building, extending and maintaining LOB systems on Microsoft platforms (.NET Framework 4.5 and Windows Azure), Microsoft Enterprise Library 6.0 will provide guidance and reusable components designed to encapsulate recommended practices for end-to-end application development in .NET which facilitate ease of use, consistency, and extensibility.

Let me share with you a couple of laptop stickers that I designed for EntLib5.0 and 6.0. They are meant to epitomize this project vision.

Here’s the one for 5.0 (on the left).

As you can see the key message here was to position EntLib as a developer accelerator meant to do heaving lifting for you. The Enterprise Library design has traditionally been guided by the principles of:

Consistency: Application blocks are written with and used in common patterns
Extensibility: Pluggable extension points
Ease of use: Config tool, tons of docs, simple interfaces, labs, webcasts
Integration: Work well together or separately

The vision of EntLib blocks addressing common-cross cutting concerns and promoting code reuse hasn’t changed. We still want to encourage devs to focus less on the plumbing and more on what matters to the stakeholders – business logic and overall user experience. However, the order of the guiding design principles has changed. Our primary consideration now (as can be seen from the stated vision above) is the ease of use and the fast start.

The metaphor for v6.0 is this one on the right.

As you can see, what we are envisioning is to make the Enterprise Library much lighter. This means deprecating the blocks that are not relevant anymore due to the modern capabilities of the platform itself (Caching, Security and Crypto), removing lots of historic cruft which we were carrying forward to support backward compatibility, and generally simplifying the underlying architecture. For example, there are currently four different ways to get your Enterprise Library objects. This is not needed – so we’ll simplify this and offer 1 or 2 recommended approaches. Simplification also means minimizing the number of interblock dependencies. We are going to emphasize that each application block can be used standalone. The vision is that if you download the Logging Application Block package, you will not need to marry into an extended family of other packages (like EnterpriseLibrary Common -> Unity Interception -> Unity -> Service Location). It will be just one package with an optional dependency on the configuration package (if and only if you want to use declarative configuration).

We are also making a substantial investment in tackling the configuration complexity. Our ultimate goal is to make it so simple, straightforward, and intuitive that no configuration tool is required at all. This is while assisting DevOps and IT/Pros in deploying, managing, and supporting their systems that are built with EntLib application blocks. This is a very ambitious goal. We feel with your feedback we’ll be able to pull it off. I will be writing a separate post dedicated specifically to our plans around configuration.

One other important aspect of the new release is that we plan to target both on-prem and cloud naturally, with no tension. This includes the configuration and all application blocks (including the Transient Fault Handling Application Block and the Autoscaling Application Block). This is also worth a separate post.

Target personas

These are our primary target personas with the corresponding goals:

Devs new to EntLib:

– Get going fast!

Alumni EntLib devs who are not using it today

– Reignite my passion for EntLib, make me see the value-adds; debunk the myth of all-or-nothing deal.

Current loyal EntLib fans

– Provide smooth migration experience, continuity and support.

DevOps and IT Pros

– Simplify deployment and operations by reducing the chance of errors.

Product backlog themes

Currently, our product backlog is organized by the following broad themes:

Fast start & learnability
Configuration
Architectural refactoring
Semantic Logging (new block!)
Generalizing the Transient Fault Handling Application Block (“Topaz”)
Autoscaling Application Block (“Wasabi”) improvements
Unity improvements
Logging Application Block improvements
Validation Application Block improvements

Individual user stories can be found on the Uservoice.

Target framework

Let me make it clear: Enterprise Library 6.0 is primarily about the back-end story. We will target the .NET 4.5 framework and all platforms that .NET 4.5 is supported on, including:

Windows 8 Desktop (no Windows RT except for Unity)
Windows 7 (32-bit and 64-bit)
Windows Server 2012 (64-bit)
Windows Sever 2008 R2 (64-bit)
Windows Server 2008 SP2 (32-bit and 64-bit)

Cloud-specific features will target the latest Windows Azure SDK.

Unity 3.0 is the only application block which will have a WinRT version for now. We have already released the preview of it. During this project, we’ll do the proper test pass, security review and other quality gates.

How can you help?

Please continue to comment/vote on use stories on our Uservoice site. We definitely appreciate your input and would like our backlog priorities and sprints to continue to be informed and driven by your feedback.

If you feel strongly about engaging with us more closely, consider joining our advisory board. We are looking for both experience EntLib users and enthusiastic novices.

Topaz now supports the new Task-based Async Pattern

Grigori Melnik MSFT — Tue, 09 Oct 2012 19:19:20 GMT

We’ve released a new version of the Transient Fault Handling Application Block (“Topaz”) which supports the new Task-based Asynchronous Pattern using the Task Parallel Library (TPL). This simplifies the process of retrying asynchronous calls significantly.

The new block is available via NuGet. Check out the Release Notes.

Special thanks to Stephen Toub for the code review and feedback.

Clarifying logging scenarios

Grigori Melnik MSFT — Fri, 14 Sep 2012 18:55:38 GMT

The patterns & practices team is exploring the space of application development for Windows App Store and Windows 8. We are thinking of building lean application blocks (reusable components) to help accelerate your development. If you are building or planning on building this type of app, please engage with us and help us understand your logging scenarios.

This is the first survey in the series of our public consultations to help direct our exploration. Depending on your feedback and related priorities we will produce an initial backlog which we will also invite the community to comment/vote on later.

Please take a few minutes to provide your input and also invite others to do the same: http://www.surveymonkey.com/s/loggingscenarios

Thank you!

Enterprise Library vNext - Simplified!

Grigori Melnik MSFT — Thu, 23 Aug 2012 00:44:00 GMT

TL;DR edition

We are simplifying EntLib. Next version targets Windows 8/Windows Server 2012 and .NET4.5. Help shape the release. Provide input via Uservoice.

Full edition

It’s been over 2 years since the last major release of Microsoft Enterprise Library. That release (v5.0) was the most successful release to date with over 1.5 million downloads and many teams using it in their systems development today. I regularly get feedback and testimonials from our users. Those are inspiring. We have also shipped two integrations packs – one for Silverlight and another one for Windows Azure.

Original mission

When the original Enterprise Library was envisioned back in 2004, the target audience included developers, solution architects, and operations specialists, who build, extend, and maintain .NET line-of-business systems that typically need to be deployed widely and need to interoperate with other applications and systems. The vision was to provide guidance and reusable components designed to encapsulate recommended practices for .NET application development which facilitate consistency, extensibility, ease of use, and integration. These components (which we call “application blocks”) address common cross-cutting concerns (like logging, validation, data access, error management), sparing you, the developer, from building your own plumbing (though many would consider that to be the most fun) and instead focus on what matters the most to your customers – the business logic. Thus, Enterprise Library is meant as a development accelerator.

The vision hasn’t changed.

Path forward

Over time, Enterprise Library evolved and grew by adding new application blocks and new features to the existing blocks. We kept the public APIs mostly backwards compatible with the previous versions of EntLib (going as far back as v2.0). As a result, EntLib became large and arguably more complex than it needs to be. In our efforts to simplify EntLib, we’ve removed certain features in v5.0 (such as WMI support) and announced deprecation of others in the future releases (such as the Caching Application Block). A year ago, I wrote a blog post On Deprecation elaborating on why deleting code is one of my favorite things. I encourage you to read it to better understand our approach to graceful deprecation.

The time has come to think about the future of Enterprise Library and Unity. In vNext, we’d like to revisit all the scenarios EntLib addresses and see whether they still make sense to be included. The .NET Framework has advanced big time in the past couple of years. Thus, we plan to deprecate many pieces that are not relevant anymore. In addition to the Caching Application Block, the Cryptography Application Block and the Security Application Block are likely candidates. We don’t want to be confined by the historical cruft. We will also do a sweep of all providers for other blocks and see which ones make sense to keep. This means that even the remaining blocks will be reimagined and possibly rearchitected, not simply ported. On the other hand, we’ll evaluate which blocks from the integration packs make sense to generalize. In the world of distribution and asynchrony, it’s likely that the Transient Fault Handling Application Block will play a much more prominent role and become more general purpose, not just focusing on the technologies and services offered by Windows Azure.

The way we envision Enterprise Library v6.0 and Unity 3.0 is that they will be targeted at Windows 8 and Windows Server 2012 developers that are using .NET framework 4.5. We would like to take advantage of all the new powerful features .NET 4.5 has to offer (including the new asynchronous features, new communication APIs, improved portable class libraries, instrumentation, etc.) as well as the full power of the Windows 8 runtime and application model. Keep in mind that EntLib 5.0 has been fully tested on .NET 4.5 already. You can use it now with confidence. We have also recently released a port of Unity to .NET4.5 and WinRT.

The guidance will embody recommendations and patterns of design for scalability, high availability, and high perf. Another important theme will likely be the unification and simplification of the configuration throughout. We recognize this is a non-trivial task.

Call to action

Considering the above, I’d like to issue an open call to action. Help us scope and shape the next release of Enterprise Library and Unity. Where should we focus our efforts? What stories should we address? Unleash your ideas. Don’t feel constrained by the current blocks or features. Think of an ideal world where all your wishes could be granted and the types of scenarios you really wished were supported. My call goes to our raving fans and loyal users, as well as to the avid critics. If you are annoyed by any of the EntLib features, make your criticism constructive – suggest improvements! We are not only listening, but also acting.

You can provide your input in a variety of ways:

Respond/comment/vote via UserVoice. Do not assume we are looking for new feature requests only. If there’s something in the existing EntLib you really want to see moving forward, please submit that as a separate UserVoice story. Also, please don’t be biased by the existing submissions –UserVoice displays them in the order of the most votes, but we suggest you review the entire list.
Comment to this blog post.
If you prefer, send me an email directly at grigori dot melnik at microsoft dot com.

Look forward to your suggestions!

Update [27-Aug-2012]: Call for Advisory Board Members

I describe how our Advisory Board operates here. We are lookig for both subject matter experts and passionate novices. If you are interested in being an advisor and helping us shape the next releases of Enterprise Library and Unity, answer this call.