Grigori Melnik: Thoughts on Agile Software Engineering and Beyond : plans

Video on what’s coming in EntLib 5.0

gmelnik — Fri, 26 Jun 2009 01:01:11 GMT

In this channel9 video, Chris and I give a 10,000 ft view on what’s coming in EntLib5.0. For details, consult our product backlog.

Enterprise Library 5.0 … and we’re off!

gmelnik — Mon, 20 Apr 2009 20:56:37 GMT

We have officially started our development work on EntLib5.0. Earlier I’ve shared our product backlog, which was largely driven by you, the community. Our initial plan is to ship in early 2010.

If you want to stay tuned with what we’re doing, the best way is to keep an eye on the home page of our community site at http://codeplex.com/entlib. Here, you can view recent posts aggregated from individual team members’ blogs, participate in online discussions, get the most recent bits of code, see file bugs/issues, and access other useful resources.

As you probably know, we run our projects in agile fashion (XP@Scrum). For those wondering, yes, we do perform TDD, pairing (I kid you not! – even distributed pairing), automated acceptance testing, refactoring, daily standups, regular retrospectives, customer proxy demos, and iteration planning; plus frequent releases, continuous integration, collective code/content ownership, low-fi information radiators, and progress tracking. We are working in two-week iterations and will be doing regular source drops on Codeplex.

Now it’s time to introduce the mighty team. You will recognize some familiar faces (many of us have worked on one or more of the previous EntLib releases) and also see some new ones. I am privileged to work with this group of accomplished professionals. I give you… <drum roll please>:

from left to right: David Hill (architect), Alex Homer (documentation and user experience lead/writer), Chris Tavares (dev lead/architect), Bob Brumfield (dev/architect), Fernando Simonazzi (dev/architect), François Tanguay (system test/dev), Nelly Delgado (release lead), Grigori Melnik (product owner/program manager). Not shown are: Rohit Sharma (test lead), RoAnn Corbisier (technical writer/content coordinator), Dennis DeWitt (technical writer), Carlos Farre (perf test consultant), Mani Krishnaswami (test), Nicolas Botto (system test/dev), Olaf Conijn (dev), Mr X (security test).

For the kickoff iteration, we were able to get the core of the team together in Redmond. The level of energy was remarkable! This team has all necessary expertise, diversity of perspectives, and desire to deliver even more goodness and value with EntLib v5.0.

You should realize that, in reality, the team is way bigger. It includes 20 members of the advisory board who we meet with on a regular basis (every 2-3 weeks). These experts help steer our direction. I must say that they are quite direct and honest, and are not shy in pushing back on any of our ideas that don’t make sense to them or do not provide sufficient value for the buck. The advisory board is representative of our target audience with experts from large and small companies, from a wide variety of domains (such as retail, transportation, government, insurance, finance, military, health care, education, media, IT services, and more).

We are also leveraging much of the learning from the previous four major releases of EntLib. Therefore, we appreciate everyone who has ever been a part of the EntLib team. We are smarter today because of you!

In addition, seven members of the patterns & practices directorate (John deVadoss, Ade Miller, Ajoy Krishnamoorthy, Don Smith, Javed Sikander, Mohammad Al-Sabt, and Steve Elston) blessed our project. Their trust and support in our abilities to self-organize and self-manage are real tokens of confidence.

All other teams at patterns & practices are involved and provide their feedback informally on the daily basis. We also interact with many members of other Microsoft product groups via regular workshops, brownbags, and internal discussion forums (with 300+ members).

I would also like to acknowledge our sustained engineering team from Avanade, who have been doing a terrific job in providing online technical support.

And, of course, you – the community! Anyone who ever participated in our surveys, submitted requests/bug reports through Codeplex, shared their EntLib extensions via EntLibContrib, participated in the customer workshops, came to our booth at the conventions (TechEd, PDC, p&p summit, etc.), or in any other way provided feedback or suggestions has contributed to this development effort. For that, we salute and thank you! We build EntLib for you and with you!

We all look forward to yet another exciting ride developing EntLib v5.0!

Enterprise Library 5.0: Product Backlog prioritization results

gmelnik — Thu, 16 Apr 2009 21:50:00 GMT

Some two weeks ago, we exposed the preliminary product backlog for Enterprise Library 5.0 and invited the community to vote on the features that matter to them most. And what a response we got! Over 2,600 people viewed the backlog and 981 cast their votes, with 230 troopers persevering to the very end. 18 members of the EntLib Expert Advisory Board also voted. In this post I would like thank you all for taking the time to complete the survey, and I want to share the results with you.

In retrospective, every previous EntLib release had a theme:

1.0 = Integration of application blocks
2.0 = .NET 2.0 update
3.0 = Validation, PIAB, Block Factory
4.0 = Dependency Injection with Unity, WMI 2.0

Over the years, EntLib grew larger, and arguably more complex than it really needs to be. The community has spoken loud and clear that they want EntLib to be easier to learn and to use. We have taken this feedback seriously, and have decided to focus this release on enhanced user experience, simplicity, and learnability. The working slogan is “Simplicity for the Win!”.

So, getting back to the product backlog. Let me explain the methodology we used to analyze the data. Stories voted as Priority 1 got 20 points, Priority 2 got 19 points, Priority 3 got 18 points, and so on, with Priority 20 getting only 1 point. Votes from our team of expert advisors’ carried double weight. Then, for each story, we calculated an arithmetic sum (the total number of points), and - using these numbers - ranked all of the stories (obviously the story with the most points was ranked as #1).

Based on past experience, we didn’t want to plan our development execution in the strict natural order of story rank, because this would require juggling various blocks within each iteration, and potentially create silos of experts around each block. Instead, we wanted an iteration to have a theme that the entire team can focus on. Therefore, we decided to carry out affinity mapping of stories for each block, and calculate their group weightings (ignoring all stories with ranks 50 and higher, since those automatically made our P3 bucket and so are not likely to be done). We then stack ranked the P1 groups (colored green in the following table) and we plan to address them in the stack ranked order, with the individual stories inside the group still ranked according to the original scores. Note that learnability and user experience stories will be addressed in parallel with all the development stories throughout the entire project, and -therefore - those groups are listed near the top of the backlog.

Total points	Rank	Description (T-shirt size)
*Architecture*
2910	1	ARC01 : Architectural update: use DI container instead of one-off Object Builder factories; DI container independence (XL)
2164	2	ARC02 : Simplification of the codebase (identify redundancies, obsoletes etc.) (XL)
1898	6	ARC03 : Reduction of the number of assemblies (XL)
*Learnability*
2082	4	LEARN03: Notebook-style 150 page book for Enterprise Developers (XXL)
1400	12	LEARN02: Notebook-style 150 page book for Architects (XXL)
1338	14	LEARN08: Updated Quickstarts (XL)
976	22	LEARN09: Updated Hands-on Labs (XXL)
590	37	LEARN10: EntLib Overview Video (L)
524	42	LEARN07: Migration guide v.4.1 ->5.0 (L)
*User Experience*
1876	8	UX05: Config tool facelift - towards a more intuitive and easier to use UI (XL)
1718	10	UX03: Config IntelliSense for Unity and EntLib in XmlEditor (M)
1610	15	UX01: Better error reporting/messaging throughout (M)
1010	28	UX02: Debugging Visualizer for Unity (L)
744	36	UX04: Config IntelliSense for Unity in XmlEditor (M)
*Data Access*
2652	3	DAAB01: DAAB & LINQ basic integration (i.e. execute a query on a database, get back an object that you can do LINQ on) (M)
2012	9	DAAB02: Async ADO.NET support (M)
1390	16	DAAB04: No swallowing of SQL exceptions (S)
*Logging*
2006	7	LAB02: Async logging (text formatting done asynchronously) to cut down on load on primary thread (M)
1100	17	LAB03: Automatically purging old files by the rolling trace listener (S)
*Tool support*
1516	13	TOOL02: Support of Unity configuration (M)
*Unity*
1072	19	UIN01: Generalized interceptor (XXL)
1108	20	U06: Clearer error reporting (M)
1042	23	U05: Unity-MEF Integration (e.g. unityContainer.Resolve<MefPart>()) (M)
896	24	U03: Support for passing arguments to the Resolve method (M)
*Config*
1962	11	CFG01 : Config decentralization (support for config stored in multiple sources) (S)
1276	18	CFG05: Making Unity configuration less verbose (M)
1072	21	CFG07: Unity config auto-registration: expanded conventions and helper classes to reduce need for explicit configuration (M)
804	34	CFG02: Improved Config API (XL)
850	38	CFG03: Support for different sections of config in different media (not just files) (XL)
*Validation*
2438	5	VAB01: Integration with WPF (XL)
1020	25	VAB04: Decorating LINQ objects with Validation attributes (M)
944	26	VAB02: Integration with/assistance in implementing IDataErrorInfo (S)
840	33	VAB07: Validation Block refactoring to allow using Unity to resolve Validators (M)
570	40	VAB12: Inheritance-aware object validator (M or ?)
*Installability/Uninstallability*
858	29	INS01: Installer allows you to pick which specific blocks to install instead of the whole package (M)
864	31	INS03: EntLib installers to be included as part of installers of other custom products (merge modules) (M)
*Extensibility*
1028	30	EXT01: Simplifying writing extensions and plugging their custom configs into the config tool (XXL)
*PIAB*
784	32	PIAB01: Implement PIAB interface that takes an interceptor type (to leverage new Unity interceptors) (S)
718	35	PIAB03: New handlers (incl. RequiresTransaction and Call Forwarding) (S)
*Resource management*
996	27	RAB03: Resource Application Block (a provider to get assorted resources from various media/sources) (XXL)
*Localizability*
834	39	LOC01: Localizability of EntLib assemblies (L)
752	41	LOC02: Localizability of exception messages and templates (S)

Stories in P2 (yellow in the following table) and P3 (red) categories are listed according to their ranks, and not group ranks. Should time permit, we will address the P2 stories in the order shown.

P2
614	43	LEARN01: Discoverability and context/dependencies EntLib poster (L)
556	44	LEARN11: EntLib5.0 New Features Video (L)
682	45	EHAB02: Parameterized templates for exceptions (M)
572	46	TOOL01: Type picker improvements (sped up search, better generics UI) (M)
516	47	U04: Container introspection (S)
590	48	LEARN12: Unity Overview Video (L)
500	49	VAB06: Simple validating field only if the value is not null (S)
590	50	VAB13: Support for recursive data structures (M)
496	51	U02: Allow a mapping from a named registration to the "default" one (S)
506	54	VAB08: Enabling Validation Block to be used with 3rd party object relational mappers (M)
482	55	VAB09: Honouring validation attributes defined in System.ComponentModel.DataAnnotations (S)
468	59	LEARN04: Notebook-style 100 page book for Operations/IT management (XXL)
394	65	VAB03: ArgumentValidationException.ToString() show the validation results (S)
P3
500	52	UIN02: Caching of matching policies (M)
608	53	DAAB03: Fix: validation of sprocs parameters – do not validate (S)
496	56	LAB04: Authenticated Email Trace Listener (S)
430	57	CFG06: Support other config schemas for Unity config (e.g. XAML-based config)
402	58	U01: ResolveAll to return unnamed registration too (S)
390	60	LAB01: ETW sink (?, potentially XXL)
422	61	LEARN14: Unity Extensibility Guide (XL)
424	62	VAB16: Additional validator: Validator that checks that an enum contains a value that maps onto its defined values (S)
404	63	INS02: Installer ships both debug and release versions of the DLLs (S)
546	64	CFG04: Support for multiple pieces of config for a single block in multiple places (XXL)
412	66	EHAB01: Default post-handling action to ThrowNewException instead of NotifyRethrow (S)
378	67	TOOL04: Web-based config tool (L)
372	68	CACHE01: Adding a reason to the class that informs a user why their object was removed from the cache (S)
409	69	LEARN15: Guidance on environmental overrides in Unity (S)
409	70	LEARN15: Unity& MEF decision tree/matrix (M)
344	71	VAB14: Provide NullValidator to properly "ignore nulls" instead of the "Or composite" (S-M)
390	72	CACHE02: Cache backing store targeting ESENT (L)
352	73	VAB15: Additional validator: Number of decimal places validator (S)
292	74	PIAB02: Add non-generic overloads (S)
344	75	VAB05: Simplifying checking for positive / negative values (S)
282	76	LEARN05: Migration guide v.2.0 ->5.0 (XL)
266	77	VAB11: Better localization support (i.e. the first time the validator is used, the locale for the messages is defined) (M)
270	78	VAB18: RelativeDateTimeValidator update to compare against UtcNow instead of Now (S)
238	79	VAB10: Honouring MetadataType attribute (M)
238	80	TOOL03: Block invocation from the configuration designer (e.g. Configure a block, Right-click, Copy “Code to Invoke”, and then paste it elsewhere) (S)
234	81	TOOL05: Easier manipulation of validation trees (potentially, drag&drop) (M)
238	82	VAB17: Validator that compares against a single specific value (S)
196	83	LEARN06: Migration guide v.3.1 ->5.0 (XL)
162	84	LEARN13: Unity Extensibility Video (L)

The initial request for a Business Rules Application Block (which would help developers to take advantage of the WF business rule engine without creating a workflow) has been evaluated, and we solicited additional feedback from the advisors and the community. We didn’t see much convergence on the scenarios, and concluded that - at this point - we would not be playing in this area.

With regard to the platform we will target, originally our plan was to target binaries to version 4.0 of the .NET Framework. However, our advisors and the community pushed back on this. The result from 1,229 votes is shown in the following graph:

This data is significant enough for us to adjust our plans to meet the community needs, and so we will target the binaries to version 3.5 SP1 of the .NET Framework. We will also carry out our test path on version 4.0 of the .NET Framework to ensure compatibility.

A similar question was asked about the framework version to which we should target Unity. The graph below shows the result of the votes, and so the decision is similar to that for EntLib 5.0. We will be targeting version 3.5 SP1, but also testing on version 4.0.

DAAB and Entity Framework: possible marriage?

gmelnik — Mon, 09 Feb 2009 22:38:03 GMT

First of all, thanks to all people who have provided their inputs to the EntLib5.0 backlog (please keep them coming!). One story seems to be mentioned way too many times. It is related to DAAB integration with Entity Framework.

We’ve looked at this a little bit internally and we do not see any opportunity or value in this marriage. We might be missing something here. For those, who had asked for this, could you please provide more details about how this integration would work and what values you’d derive from it?

Enterprise Library 5 – what’s on your wish list?

gmelnik — Tue, 27 Jan 2009 08:55:40 GMT

We’ve started our planning of the Enterprise Library 5. This is your chance to send us suggestions, comments, screams... These may include ideas for new blocks, new providers for existing blocks, integration with new .NET Framework 4.0 features, improvements to the design-time experience, performance improvements, support of specific deployment/operations scenarios, documentation/training support etc.

I am going to use the same technique Tom employed several years ago soliciting feature requests for EntLib3. It seemed to work quite well then. So, imagine you have a sum of money that you can invest in developing EntLib 5. Say $100 (or should I index the amount to be more like a stimulus package? :) In any case, when suggesting your features, decide how much of that $100 you want to spend for each feature. You can split $100 any way you want. If there is only one feature you want to spend all your money on, you can do it too. Please stay within the budget. Also, be specific – explain your scenario in detail, provide some context, make your case potentially appealing to other users of EntLib. Do remember, however, that EntLib has been and will be a general purpose library of reusable components that are architecture- and application domain-agnostic.

Once we gather your feedback, we will publish the list of feature requests and invite you to prioritize them.

Feel free to post your suggestions as comments on my blog or simply email me. Look forward to your propositions!

Thanks!

What is Enterprise Library team currently up to?

gmelnik — Fri, 12 Dec 2008 01:10:00 GMT

After shipping Enterprise Library v4.1, the team is focusing on the ease-of-use and learnability aspects of both EntLib and Unity. Our top stories of the current backlog include the following:

Unity for Silverlight [Done - 12/14/2008]
Comprehensive Enterprise Library FAQs [Done - 2/10/2009]
Validation Block Hands-on Lab [Done - 1/3/2009]
Interception Hands-on Lab [Done - 3/18/2009]
Unity Extensibility documentation [In Dev]
New video on Unity (level 200)
New video on Unity Extensibility (level 400)
Customer satisfaction survey analysis [Done]

These are all things we plan on doing prior to starting Enterprise Library v5 research and planning work early next year.

Microsoft Enterprise Library – What Do Customers Want ?

gmelnik — Wed, 10 Dec 2008 05:15:00 GMT

As the Enterprise Library product owner and now program manager, I frequently talk to users. Despite the fact that we position the Enterprise Library as coded guidance, I consistently hear the sentiment that it is perceived as a product. As one customer unequivocally put it, “We don’t care what you call it. As far as we are concerned, Enterprise Library is no different from .NET Framework. Therefore, we expect three things: quality, support and continuity.” Let me address these in this posting.

Quality

The quality bar of the Enterprise Library is very high. When producing a release, our team goes through similar quality gates any other Microsoft product will have to go through. Furthermore, since we ship the source code (which is part of the guidance), our key objective is for the code to embody the good and proven practices of designing and building line-of-business systems and components on the platform. A lot can be learned from the code itself as well as the full suite of unit tests that we ship.

Today, Enterprise Library is used in most if not all industries (incl. financial, insurance, manufacturing, retail, healthcare, transportation, etc). It is also used by governments and non-profit organizations. By mid next year we expect the total number of EntLib downloads to reach 2 million. I do recognize the problems with such statistics (it may not accurately illustrate the actual number of EntLib users, since many people have probably downloaded EntLib dozens of times; on the other hand, there are many enterprise customers who download it only once, validate the release according to their own operational acceptance criteria, sometimes modify it, and then disseminate to thousands of developers worldwide). The Net Promoter Score for EntLib is 54% (!) That’s super high [see my upcoming post on NPS]. Speaking of which, if you haven’t filled out the EntLib customer satisfaction survey, we would appreciate if you took a few minutes to do so. At the recent TechEd EMEA conference, I’ve asked the audience how many people were using EntLib. I was so thrilled that I got my camera out and took a couple of photos. It’s amazing, I only counted a few people who didn’t raise their hands.

EntLib is also used internally. In fact, internal adoption of the Enterprise Library is increasing. This is very encouraging. As someone who always fought the Not-Invented-Here-Syndrome, I am pleased to see that other groups at Microsoft are making use of EntLib goodness. Importantly, it’s not only Microsoft IT (our internal IT org) that uses EntLib. Pieces of EntLib are shipping as parts of actual shrink-wrapped products.

Support

We’ve continuously provided support via forums (http://www.codeplex.com/entlib/Thread/List.aspx, http://www.codeplex.com/entlib/WorkItem/List.aspx, http://www.codeplex.com/unity/Thread/List.aspx). Besides the core development team, we also have a dedicated sustained engineering team continuously monitoring the forums and answering questions. In fact, by the most recent version of EntLib (4.1) we have resolved all open EntLib issues that had 5 votes or more.

In addition, customers were always able to receive appropriate support through Microsoft Premier Services (though it was treated as customer-written code). We also announced partnership with Avanade to provide support and sustained engineering (starting Nov 1, 2008). For more information, see http://www.avanade.com/entlib/. Our vision is to develop an ecosystem of partners providing both support and building extensions to the Enterprise Library. One example of such extensions is the distributed cache provider for Caching Application Block (via NCache – see http://www.alachisoft.com/ncache/ncache_express.html).

Continuity

When talking about continuity, I believe there is no doubt that Microsoft heavily invested in the development of the Enterprise Library (7 releases in three years). We intend to maintain that heartbeat of producing at least one major release a year. We now have a strong, well-gelled team that we intend to preserve and expand as we move forward. My next post will talk about our current work and also our plans in terms of v5.

Summary

I hope the above explanations demonstrate that Microsoft Enterprise Library is here to stay. Evidence from the field tells us clearly that a great number of customers is using EntLib and deriving value from it. This is what keeps us energized and going. We’ve had a great collaboration with many of you in the community and we look forward to more of it. We are committed to our mandate: making you succeed on the platform. Enjoy the ride!

P.S. For those of you who feel bad about using EntLib for free, we’ll take your feedback as payment :) Thank you in advance.