Grigori Melnik: Thoughts on Agile Software Engineering and Beyond

Powered by EntLib

gmelnik — Fri, 13 Nov 2009 12:30:00 GMT

When speaking about or demoing EntLib, I often get asked which Microsoft products Enterprise Library is used in. There are several dozen internal Microsoft projects (that we know of) that take advantage of Enterprise Library. However, many people are particularly interested in commercial products that use it. Today I am able to speak about some of them.

This Monday the Microsoft Exchange team announced the public launch of Exchange Server 2010. There are many innovations in it, which you can read about on the Microsoft Exchange Teamblog. The reason I mention it here, however, is not only to congratulate the Exchange team, but also to proudly share the news that Enterprise Library and Unity container are used by Exchange Server 2010.

In addition, the Enterprise Service Bus (ESB) Toolkit 2.0, which is now part of Microsoft BizTalk Server 2009 also uses Enterprise Library 4.1 and Unity 1.2

Through the course of my interactions with many customers, I’ve compiled an extensive list of companies that use Enterprise Library (including 50 Fortune 500 companies). Unfortunately, I do not have permission to share many of their names. However, you can find some of the case studies published here. In addition, you can explore Microsoft Solution Finder and search for projects with “Enterprise Library” as a keyword.

These endorsements are a clear indication of the value that Enterprise Library provides to Microsoft product groups, our customers and partners.

I would also like to use this opportunity to invite customers who would like to share their story, and be profiled on our patterns & practices development center page, to contact me.

Dependency Injection pattern in APIs

gmelnik — Fri, 13 Nov 2009 00:44:20 GMT

My colleague and friend Chris Tavares, who is the Dev Lead on Enterprise Library and Unity, published an insightful article in MSDN magazine on the usage of DI in APIs, which I highly recommend. It is based on our recent experience of doing architectural refactoring of the Enterprise Library. It outlines our design objectives and the approach taken to meet them. It concludes with lessons learnt and key considerations when designing your own API or framework.

Read Chris’s article here.

Zen and the Art of Managing Crosscutting Concerns – EntLib Dev Guide Preview

gmelnik — Fri, 13 Nov 2009 00:44:00 GMT

A new public preview release is now available on Codeplex. It now includes the following chapters, revised and updated for V5.0:

Chapter 1 – Intro. Meet the Librarian
Chapter 2 – Much ADO about Data Access
Chapter 3 – Error Management Made Exceptionally Easy

Your comments are invited.

Stay tuned for more.

Grigori at PDC

gmelnik — Wed, 11 Nov 2009 13:54:22 GMT

I’ll be at PDC2009 in LA next week. I’ll be happy to meet with EntLib and Unity users as well as critics, to discuss new features of EntLib 5.0, preview the new config experience and also solicit suggestions for post v5 work.

If you’d like to discuss acceptance testing and test automation strategies, I’d be interested in that too.

Look for me at the p&p booth Mon-Thu or ping me to set specific time to meet.

See you there.

p&p Summit 2009: What’s coming in Enterprise Library 5.0

gmelnik — Fri, 16 Oct 2009 13:51:06 GMT

Yesterday Bob Brumfield and I gave a talk on EntLib5.0 at p&p summit in Redmond . We’ve discussed the underlying architectural refactoring and previewed the following improvements:

Data accessors that will help you easily consume sproc results when using Data Access Application Block – instead of a data reader you get an IEnumerable back and can do all LINQ magic on it.
Logging Block performance improvements due to optimizations using predefined timestamp patterns mapped to high-speed formatting implementations and new “local” tokens for context info (e.g. localMachine, localAppDomain) that we cache.
Improved config experience epic which included:

programmatic config support via fluent interface
face lift of the config tool to make it more intuitive (especially when displaying relationships), simpler (by removing a lot of design time dlls) and better extensible (by simplifying the way config for custom handlers/blocks) can be plugged into the config tool)

Honouring validation attributes in System.ComponentModel.DataAnnotations
Getting EntLib objects in various ways :

via a single point of entry EnterpriseLibraryContainer.Current
or injected by the container

We also demoed container independency story which allows to switch the underlying container. Out of the box, you get EntLib shipped with Unity, but if you prefer to use another DI container, now you can easily switch over.

We are planning several webcasts to share these demos.

In the meantime, feel free to check out our recent bits here and provide feedback.

Removing WMI support from Enterprise Library

gmelnik — Tue, 13 Oct 2009 20:00:31 GMT

In our quest to simplify EntLib’s codebase, we are considering removing WMI2 support in v5. This is based on recommendation by our Advisory board, as well as lack of usage and interest from the user community and the high development complexity to maintain WMI2 integration (the code itself as well as non-trivial environmental setup on dev and test machines).

If this is something you care deeply about, please scream now. We are waiting for any scenarios that could convince us to keep WMI2 support of EntLib objects till the end of October.

Note: as part of our instrumentation story, we will continue to maintain EventLog and perf counter updates.

Enterprise Library reaches 2 million downloads!

gmelnik — Wed, 09 Sep 2009 02:49:46 GMT

Enterprise Library has a broad reach and adoption. No wonder, since it embodies many of the proven .NET development practices and ships with the code, tests and hands-on labs – lots to reuse and lots to learn from. The EntLib community has welcomed more than 2 million downloads worldwide! At least 10% of Fortune 500 companies use Enterprise Library (that’s who we know of). In addition, I’ve noticed that employers now list specifically EntLib skills in their developer job ads on monster and workopolis.

To celebrate the 2M milestone, our p&p colleagues surprised us with a cake. A slice goes to everyone who has ever worked on application blocks or the library itself (one day I’ll be able to do a better job and acknowledge each and everyone of you!) as well as to all community members who have used EntLib, commented on it, reported bugs, suggested improvements, written extensions. It’s been five years since this effort began and the tremendous success of EntLib today is largely due to all of you! Thank you! We look forward to continuing collaboration with you.

Don’t forget to check out our Codeplex development site where you can see what’s coming next (in v5) and download most recent code drops.

EntLib/Unity Trainers

gmelnik — Sat, 15 Aug 2009 01:50:00 GMT

If you are offering industrial or academic courses or workshops that use Enterprise Library or Unity, I hope you are already taking advantage of our updated Hands-On Labs. I would like to know what else we could do to better support you and the learners.

Please reply to this post or ping me directly. You may also want to send me your syllabus and we’ll consider profiling you under the Learning section of our Codeplex site.

- Grigori

Showcasing patterns & practices’ goodness

gmelnik — Fri, 14 Aug 2009 07:54:00 GMT

These are some of the events I’ll be presenting at in the next couple of months:

Patterns & Practices Conference Ukraine, Kiev, Sep 16, 2009 (www.pnp.in.ua)
Patterns & Practices Summit Russia, Moscow, Sep 22, 2009 (www.pnpsummit.ru)
Patterns & Practices Summit Redmond, Oct 12-16, 2009 (www.pnpsummit.com)
TechEd EMEA, Berlin, Nov 9-13, 2009 (www.msteched.com)

Whether you are using Enterprise Library or any other p&p asset, I’d be happy to chat with you about your experience. Look forward to it!

Unity 2.0 container update

gmelnik — Thu, 16 Jul 2009 21:18:56 GMT

Our latest work on Unity 2.0 is available on Codeplex (see change set 28516). This is the first work we’ve done on Unity as part of the EntLib5.0 effort.

Major changes in this drop include:

ObjectBuilder2 DLL is now gone. The code is now completely contained in the Unity assembly, and you won't need to reference OB2 anymore in your projects using Unity. Over time the ObjectBuilder stuff is going to lose a lot of it's generality and be just the underpinnings for the container.
We're taking a first cut at fixing our exception messages. All the extra BuildFailedException’s that ended up in the inner exception tree are now GONE. Instead, you get a top level ResolutionFailedException, and the InnerException is the actual exception that caused resolution to fail. And the exception message has been reformatted significantly to hopefully get a better description of what the container was doing, and how it ended up where it did, when the error occurred. This is in response to user feedback to provide better error messaging in Unity. We consider this to be a significant improvement, but still need your feedback. In particular, we look forward to specific suggestions about how we could change the messages to be even more descriptive and intuitive. Please reply to this post or via Codeplex discussion thread.
Silverlight projects are now updated to Silverlight 3.

Acceptance Test Engineering Guide, volume 1 BETA2 release

gmelnik — Wed, 01 Jul 2009 03:30:00 GMT

Today I am pleased to announce the release of the beta2 of the first volume of our Acceptance Test Engineering Guide. We are getting closer to “done”.

As it became apparent that the primary reader personas for each part in the original guide (that was released as beta1) were different and the size of the guide was approaching 800 pages, we decided to split it into 3 volumes. It allows us to ship Volume I sooner while finishing Volume II and III. Incremental delivery for the win!

Based on the first round of reviews, Volume I has undergone some serious rewriting (especially, the Gating model, which is now called the Acceptance Process Model) and the entire new chapters were added (Part II – Perspectives on Acceptance).

Here’s the current structure of the guide:

Volume I provides an overview of the acceptance process and how acceptance testing and other key practices fit into the process. This volume is intended to be read from beginning to end. It is subdivided into three main parts:

Part I – Thinking about Acceptance explains six mental models that are useful when thinking about the acceptance process.

Chapter 1 The Acceptance Process
Chapter 2 Decision-Making Model
Chapter 3 Project Context Model
Chapter 4 System Requirements Model
Chapter 5 Risk Model
Chapter 6 Doneness Model

Part II – Perspectives on Acceptance describes the acceptance process from the perspectives of key stakeholders in two different kinds of organizations: the Information Technology Department in a business and the Product Development Company. Most readers involved in the acceptance process should find some commonality with at least one of the roles describes.

Chapter 7 Business Lead’s Perspective
Chapter 8 Product Manager’s Perspective
Chapter 9 Test Manager’s Perspective
Chapter 10 Development Manager’s Perspective
Chapter 11 User Experience Specialist’s Perspective
Chapter 12 Operations Manager’s Perspective
Chapter 13 Solution Architect’s Perspective
Chapter 14 Enterprise Architect’s Perspective
Chapter 15 Legal Perspective

Part III – Accepting Software introduces the practices that are necessary for planning the acceptance process, for performing acceptance testing and for improving the acceptance process.

Chapter 16 Planning for Acceptance
Chapter 17 Assessing Software
Chapter 18 Managing the Acceptance Process
Chapter 19 Streamlining the Acceptance Process

Volume II is a collection of what we call thumbnails that describe the practices introduced in Volume I in more detail. A thumbnail is a short overview of a practice that explains what it is, when you may want to use it, the risks that it mitigates, and an overview of how to perform the practice. Thumbnails also include a list of references to papers, books, and other resources that provide more complete descriptions of the practices in question. The main purpose of a thumbnail is to describe a topic well enough to provide an overview, serve as a mental reminder for someone who has used the practice on how to do it, and give someone unfamiliar with the practice enough information about the practice and its applicability to determine if they want to learn more about it. Some of these topics and practices have entire books written about them that describe the concepts in greater detail and depth than this guide could possibly do. Volume II is intended to be used as a reference; most readers will not read it from beginning to end.

Volume III. This volume is a collection of sample artifacts generated by applying different practices in a fictional but realistic situation for the fictional company Global Bank. These artifacts are embedded in a series of case studies of what the Global Bank team may have produced while building the application. The case studies provide some context to the individual artifacts. They also provide cross-references to the practices described in Volume II. The artifacts are intended to be used as way to learn more about how to perform a practice; they can also be used as templates for your own artifacts.

As always, we look forward to your feedback! Feel free to submit it through the Codeplex site (Issue Tracker or Discussion Forum) or simply email it to me.

Enterprise Library 5.0: Architectural refactoring complete

gmelnik — Fri, 26 Jun 2009 01:19:45 GMT

The highest priority story in the EntLib5 backlog is ARC01: Architectural refactoring, which includes the use of a dependency injection container for wiring your EntLib objects instead of the one-off ObjectBuilder factories and DI container independence (to allow other containers such Spring.NET, Castle/Windsor, StructureMap, Ninject etc. to be used instead of Unity if you prefer so). This work is now complete. For details of this architectural update, please see an easy-to-read summary by Bob Brumfield. We also touch on this in the channel9 video.

As usual, the updated bits are available for preview on Codeplex - http://entlib.codeplex.com/SourceControl/ListDownloadableCommits.aspx

Video on what’s coming in EntLib 5.0

gmelnik — Fri, 26 Jun 2009 01:01:11 GMT

In this channel9 video, Chris and I give a 10,000 ft view on what’s coming in EntLib5.0. For details, consult our product backlog.

Thoughts on improving performance of the Logging Application Block

gmelnik — Fri, 19 Jun 2009 20:16:44 GMT

EntLib5 team is working on improving performance of the logging block (story LAB02 in our product backlog). We’ve identified that message formatting in particular takes substantial amount of time based on our own performance tests and customer reports. So, we are considering alternatives. One of them is improving performance of the formatting operations and another one is doing formatting and logging asynchronously, thus offloading the formatting work to a separate thread and improving perceived performance. Note, this discussion is not about optimizing the message formatting mechanism, which we may address separately.

Our intention is to implement these improvements on the opt-in basis to preserve the default behavior of the Logging Block consistent with the previous versions of the EntLib.

Here are the possible limitations we’ve identified when doing formatting and logging asynchronously:

1. When using the Logging Block, it is possible to utilize non-EntLib-specific .NET trace listeners (such as console, debug, eventlog, etc.). These trace listeners use TraceEventCache object to obtain context information, including thread-specific information, which we cannot control. This means that performing the logging operation in a thread different than the originating one may result in the log entry that has inaccurate context information.

2. Besides, the TraceEventCache also supplies the call stack, which we can force to be pre-computed in the originating thread. However, this turns out to be very expensive and likely be a waste since not all trace listeners would require it. Options include:

a. Pre-compute always

b. User elects to pre-compute via configuration

c. Add logic to the block to determine whether pre-compute is necessary based on the configuration of the trace listeners

Note, this is not the call stack you would care about when logging exceptions through the LoggingHandler of the Exception Handling Application Block. In this scenario, the actual message (together with the call stack) gets formed before a call to the Logging Block is even made.

3. With this approach, there is no guarantee that all log entries are effectively logged. For example, if a log entry posted just before the application terminates, it’s possible that the logging thread may not complete its processing.

We’d like to understand which logging characteristics you care about. Among the following, please select top 3 that matter to you:

A. Maintain the chronological order of log entries

B. Avoid depletion of the thread pool

C. Minimize the lag time between post of the log entry and its persistence (this would reduce the risk of losing posted entries)

D. Start throttling to avoid unbound memory consumption

E. Ability to provide custom logging strategies (such as async logging) to be plugged-in

F. Ability to elect which categories/trace listeners use async or sync logging mechanism

Based on relative importance of these choices, we will select a design direction ranging from using the thread pool to implementing more sophisticated thread and data management.

With your votes (feel free to reply to this post or send Grigori an email), please describe the context driving your choices and any other comments or concerns we may have overlooked.

We would appreciate your rapid feedback – if you care for this story, please do not wait with your response.

Taking User Experience Seriously

gmelnik — Thu, 14 May 2009 09:14:51 GMT

As I have mentioned in my previous posts, over the years, Enterprise Library grew large (it now includes 9 application blocks), and arguably more complex than it really needs to be. With v5, we are focusing on enhanced user experience, simplicity, and learnability. We are tackling these goals from several dimensions: architectural refactoring to simplify our code base and make it easier for users to extend EntLib; improving our config experience through the config tool; and writing more approachable guides. Let me describe these in a bit more detail.

Simplifying the underlying architecture

The creation of EntLib objects is, on the surface, pretty simple. You call into the appropriate factory, stuff happens, the current configuration is read, and the appropriate instances are returned to you. Of course, the aforementioned “stuff” gets rather complex. Every EntLib object requires the object itself, a Data class describing the configuration, a custom factory class, and (usually) an assembler class, plus a bunch of attributes scattered around to wire them all up together. Maintaining all these moving parts can be challenging to the uninitiated, and this is a major barrier to people who want to build their own blocks or extensions.

At its heart, the ObjectBuilder infrastructure (and the factories and assemblers, etc.) was really a very specialized dependency injection framework. With the basic idea of DI moving into the mainstream, we took the opportunity to revisit this design. We are replacing all our custom object creation infrastructure with a dependency injection container plus methods that properly configure the container. The net result is the removal of a lot of moving parts, which simplifies the codebase significantly. You no longer have to wade through quite so much infrastructure to get to the real meat of what Entlib is doing. And switching to a full-featured DI container also gives us much more flexibility, both in Entlib and in how it interacts with your applications.

Usability of the config tool experience

The config tool (available as a standalone application, or as an integrated configuration editor with Visual Studio starting with EntLib v3) was a major step forward in creating, modifying, and validating application block settings. No one wants to manually tweak xml configuration. The tool has not undergone any major changes since v1. It uses a tree-based interface that, according to some anecdotal evidence, users find not very intuitive. There also seem to be some conceptual barriers in connecting various parts. We would like to understand the current user experience and improve it in v5, specifically in regards to:

Approachability and usability. Important goals are increasing efficiency of use and reducing pain points associated with common tasks.
Extensibility is a key feature of EntLib. The new design for the configuration tool must be robust and flexible enough to support multiple types of user customization and also scale with future EntLib releases.

In order to do this methodically, we are partnering with seasoned user experience researchers (Kelly Franznick and Jen Amsterlaw of Blink Interactive) and executing on the following plan:

1) Baseline usability testing of the v4.1 config tool. This is needed to identify areas for improvement, and as a standard to compare to the new tool, once it is designed and implemented. We are targeting .NET developers who are fairly new to EntLib (1-3 months of experience) and IT Operations/Admins who are new to EntLib. By asking users to perform specific tasks in a controlled and moderated setting, we are focusing on specific areas of concern, including processes that require conceptual understanding to assess learning curve issues. We are currently doing this at TechEd in LA. If you are interested in participating, stop by the p&p booth on the exhibit floor (blue area).

2) User research. This will involve interviewing existing end users to understand their goals, motivations, and perceptions of the overall EntLib experience. Together with the results of the baseline usability testing, the user research will inform the next phase, which will focus on interaction design.

3) Interaction design. During this phase, three distinct concepts will be designed to reflect the major workflows of the EntLib improved config. These will be presented in the form of wireframe prototypes.

4) Prototype usability testing will test these prototypes.

Based on the results of these studies, we will analyze the feasibility of implementing the best prototype and build the new tool. I will keep the community updated about how these activities are going.

Learnability

Our current set of docs includes over 1000 pages. We understand that no one has time to read all of that information, some of which has more details that most users need. To address this, we are experimenting with a new style of documentation — Developer Notes. These are written in a more informal, conversational style, are task based and are meant to be more approachable. The key here is for users to be able to quickly find out how to do common tasks with EntLib (thus, we are limiting the size of the guide to be under 200 pages total). Chapter 1. Meet the Librarian and Chapter 3. Error Management Made Exceptionally Easy are available for preview and public commenting.

The first 50 people who provide thoughtful feedback will receive a complete copy of the book once it is out.

Hope these three initiatives give you a good indication of how seriously the EntLib team takes user experience. If you have other suggestions, we welcome them as well.

Back to TechEd crowd now…