Marty Wasznicky - The "Event Driven Business"

Well....I guess it was time.  My last day at Microsoft was November 30th.   Over 6 years working with the BizTalk team.....I'm a little sad to go.  I really loved the BizTalk development world so this was a hard decision for me.  Between working on customer projects, creating the Virtual Technical Specialist program, driving the business, conducting my training classes, and building the Microsoft ESB Guidance......I guess I kept myself pretty busy.  However, everyone has to think about growth.....and I'm no different I guess.  I worked with a lot of really smart people, Jean-Emile, Lee Graber, David Stucki, Kartik, John Taylor….well the list goes on.   I'm going to miss everyone.   However, I'll continue to develop ESB technologies in my new role at Neudesic. 

I've joined their Neuron team to lead up development.  If you haven't heard, Neuron is a pure WCF based ESB, the brain child of David Pallman, also formally of Microsoft's WCF product team.  I'm pretty excited about it.  We really have an opportunity to solve a lot of real world problems in short order.  I've worked with Neudesic for almost 6 years while at Microsoft.  They really have an amazing talent pool.  It was certainly one of the things that compelled me to accept the role.  Lots of smart people to work with!  I’m looking forward to that.  They probably have more experience building and solving these integration problems then most SIs…considering they were one my go to partners for the last 6 years for BizTalk related work.   We’ll tap into all that solution experience to develop Neuron into a world class ESB.

Who knows….maybe I’ll actually start blogging J

Cheers!

Well....the project is humming a long....more cool things we're doing.  Here's an email I sent out to some folks in the community

Marty Wasznicky
Regional Program Manager – BizTalk Server
Connected Systems Division
Microsoft Corporation

MCSE, MCSD, MCDBA, CNE, MCTS in BizTalk 2004

http://blogs.msdn.com/martywaz

333 S. Grand Ave., Suite 3300, Los Angeles, CA 90071

É213.806.7484È310.980.5495 šmartin.wasznicky@microsoft.com

BizTalk Server Webcasts:

http://msdn.microsoft.com/biztalk/Community/webcasts/default.aspx

BizTalk Server 2006 Virtual Labs:

http://www.microsoft.com/technet/prodtechnol/biztalk/virtual-labs.mspx

BizTalk Server 2006 Trial Download:

http://www.microsoft.com/biztalk/evaluation/trial/default.mspx

BizTalk Server 2006 Home Page:

http://www.microsoft.com/biztalk

Wow......I can't even remember the last time I blogged.   I've been living on an airplane; it seems like for years now.  I guess an update is in order.  I'm still working for Microsoft....in the Connected Systems Division.  That's the place in Redmond where we build things like BizTalk, Host Integration Server and WCF.  I still cover the West Region for Microsoft in Regards to BizTalk....as well as a bit of the Northeast Region.  I still run the BizTalk Virtual TS program....a program I designed in SoCal back in 2004....but I've been able grow it across the nation where it now contributes to almost a third of the US BizTalk business.  We've even got seeds of it growing in Canada and EMEA...pretty cool.

Most recently though I've been heads down on building the Microsoft ESB Guidance for BizTalk....a project I started about a year ago from the work that Lukas Cudrigh (Microsoft NorCal), Brian Loesgen (Neudesic) and I started at a major healthcare company (Kaiser).  You can read all about it on Brian's blog (http://geekswithblogs.net/bloesgen/archive/2006/11/03/96062.aspx).

We released the first build of this last October at our Microsoft Redmond SOA event.  As you can imagine....it took off like wild fire!  We set up a Microsoft email alias for those interested in it.  I got inundated with several hundred requests for it in a matter of months!

Short story, we were able to convince the powers that be that this was the way to go.  I got the green light to move forward and we are now partnering with the Patterns and Practices team to revise and complete the Guidance to work BizTalk Server 2006 R2 (which we expect to release in September).  The Guidance itself, we expect to ship shortly after R2, probably at the beginning of October this year.  The ESB Guidance will be an official branded release by the Patterns and Practices team. 

On the Patterns and Practices side, Don, Dmitri, Rajat, Alex and the team are keeping us honest and the quality bar high (I think Rajat spent his entire weekend just testing our latest drop).  On the development side, I'm the architect and development lead for a small pack of Partner based resources (Sogeti, Magenic, CTS and Neudesic).  We're a small but passionate group and, together, I think we're going to deliver one of heck of a package to the community.

As we continue to develop the features and dream of new things on the horizon, I'll try to keep my ideas posted here.  In the meanwhile, I’ve posted below the  latest email I sent to the community at large regarding our latest CTP release of the ESB Guidance on codeplex (www.codeplex.com/esb ).  If you’re interested, you can also peruse through the current documentation on Alex’s site, http://www.daveandal.net/articles/esbdocs/

Cheers! 

BizTalk Server 2004 has a very cool architecture that allows it to balance the processing of a workload across many BizTalk Servers when they are configured to function as a group. When we use this "Group", or "Farm" configuration, all the BizTalk Servers share the same configuration and message box database. As workload increases or, for high availablity, we can push "BizTalk" applications (in the form of .Net assemblies) out to additional servers in the farm. In most cases, "pushing out" really just means "GAC'ing" the custom BizTalk application assemblies on the target BizTalk Servers within the farm.

What does this architecture give us?  If the Orchestrations fail on one server, they can easily start up where they left off on the other server as all the state is managed in the database (not local machine queues).  All we need to do is make sure that the application assemblies are GAC'd on all the servers we want to fail over to.

Today, people either have to do that manually or or use a third party tool to GAC assemblies.  

But lets dream for a moment....wouldn't it be cool if we had a tool that would allow us to manage all the assemblies deployed within the BizTalk group?  Imagine,  that we could use this tool to list out all the assemblies registered in the BizTalk Management database, then check to see if they've been deployed to other servers within the BizTalk group.  Then, when we find one that wasn't GAC'd on a specific server, we could press a button and wholla (did I spell that right :) ), the assemblies get deployed and GAC'd on the remote server and we didn't even have to leave our desks.

Well...this is possible now because of a BizTalk Escalation Engineer that works for Microsoft Premier Services, Rick Caudle.  He wrote the BizTalk Assembly Checker, which does exactly what I described above. 

When we launch the tool, we'll see a view that allows us to specify the SQL Server that the Management database is located on and connect to it.  Once we connect, it will list all the BizTalk Servers and deployed application assemblies registered in the BizTalk Group.  The tool allows us to select each server and check to see the assemblies are deployed.

If we need to copy and GAC the assemblies on one of the servers, we just click the GAC Tool button which brings up the dialog box below:

Check whether we want to copy the assemblies locally and GAC them and click the OK button.....the tool does the rest: 

You can download the tool as well as the latest build source code from here.

Installing BizTalk Server 2004 sometimes entails installing Windows Sharepoint Services, specifically if you want to use Business Activity Services (Trading Party Mgmt, BAM portal, etc..).   However, I see more and more folks configuring it improperly, hence the BizTalk Server installation fails when trying to install our BAS site or updating the infopath templates.

The product group is pretty good about updating the Installation Guide for BizTalk Server 2004.   In fact, you can find the latest published guide (February 05) at http://www.microsoft.com/technet/prodtechnol/biztalk/biztalk2004/deploy/install_guide.mspx. 

In the guide it reads:

To configure Windows SharePoint Services

1. If it is not already open, open SharePoint Central Administration. On the Start menu, point to Programs, point to Administrative Tools and then click SharePoint Central Administration.
2. On the SharePoint Central Administration page, under Server Configuration, select Configure virtual server for central administration.
3. On the Configure Administrative Virtual Server page, select Use an existing application pool if one already exists. If this is your first Windows SharePoint Services configuration, select Create a new application pool. Enter an application pool name. For example, SpsAdminAppPool.
4. Select a Configurable security account and use a local administrator. Click OK. You will be prompted to restart IIS at this point. Click OK.
5. After IIS restarts, open SharePoint Central Administration. On the SharePoint Central Administration page, under Server Configuration, select Set configuration database server.
6. On the Set Configuration Database Server page, enter values for Database server name and SQL Server database name. Next, specify your database connection type. Click OK.
7. On the SharePoint Central Administration page, under Virtual Server Configuration, select Extend or upgrade virtual server. The Windows SharePoint Services Virtual Server List page appears.
8. On the Windows SharePoint Services Virtual Server List page, if there are no servers in the list, click the Go to the complete list link. The complete list appears.
9. Select the server you want to extend.
10. Click Extend and create a content database.
11. On the Extend and Create Content Database page, under Application Pool enter an Application pool name. For example, SpsWebSiteAppPool.
12. Select a Configurable security account and use a local administrator.
13. Click OK.

By default, Windows SharePoint Services includes the root of the IIS default Web site as a managed path. To create a virtual directory on a Windows SharePoint Services-extended virtual server for your own use (for example, for a BizTalk Server receive location or for a Web service application), you must exclude the URL for your virtual directory from the Windows SharePoint Services managed paths. Alternatively, if you have not configured a top-level Windows SharePoint Services site on the root of the IIS Web site, you may remove the root as a managed path.

To configure Managed Paths in Windows SharePoint Services

1. If it is not already open, open SharePoint Central Administration. On the Start menu, point to Programs, point to Administrative Tools and then click SharePoint Central Administration.
2. Click the Configure virtual server settings link.
3. Click on the server that you extended.
4. Under Virtual Server Management, click on Define managed paths.
5. Under Included Paths select the (root) checkbox and click the Remove selected paths link.
6. Click the Windows SharePoint Services link to return to the SharePoint Central Administration site.

For information about how to configure managed paths from the command line, see the topic "Managing Paths" in Microsoft Windows SharePoint Services Administrator's Guide at http://go.microsoft.com/fwlink/?LinkId=18110.

So, you can see, there's a few things you have to do to configure it properly.   In the past, I've taken all this and just put it in a batch file to simplify the process.   You'll find the text of the batch file below.  I literally just run this instead of running the setup independently and doing all the configuration manually.   I avoid common mistakes this way.  

You'll see I declare some variables at the top of the batch file.  These set the path of the WSS installation executable, the server and port which WSS is being installed on, the user id, password and email address that the WSS service will run under (make sure they are members of the local IIS_WPG group), the database server in which to create the content and configuration database, ports and the name of the configuration database to create.

Next, I basically execute the following steps:

1. Install WSS using the remote Sql option

2. Remove front page extensions from the web site I'm installing WSS on

3. Configure the WSS admin site and the application pool for it, including the credentials it runs under

4. Create the WSS configuration database

5. Extend the virtual server on the default site and set credentials

6. Lastly, I delete the root as a managed path under WSS.

Cheers!

@SET WSSEXE          = C:\WUTemp\Platform\WSSV2\STS2Setup_1033
@SET WSSSERVER   = bts04dev01:80
@SET WSSUSERID    = waz-msft\WSSAdminSvc
@SET WSSPWD         = pass@word1
@SET WSSEMAIL      = WSSAdminSvc@waz.msft.com
@SET DBSERVER      = BTS2004sql01
@SET FPPORT           = 80
@SET ADMINPORT   = 8080
@SET WSSCONFIGDB = WSSConfig

%WSSEXE%\SETUPSTS.EXE remotesql=yes /qb

"c:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\50\BIN\owsadm.exe" -o fulluninstall -p %FPPORT%

"c:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\60\BIN\stsadm" -o setadminport -p %ADMINPORT% -admapcreatenew -admapidname adminvs -admapidtype configurableid -admapidlogin %WSSUSERID% -admapidpwd %WSSPWD%

"c:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\60\BIN\stsadm" -o setconfigdb -ds %DBSERVER% -dn %WSSCONFIGDB%

"c:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\60\BIN\stsadm" -o extendvs -url http://%WSSSERVER% -ownerlogin %WSSUSERID% -owneremail %WSSEMAIL% -apcreatenew -apidname defaultvs -apidtype configurableid -apidlogin %WSSUSERID% -apidpwd %WSSPWD%

"c:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\60\BIN\stsadm" -o deletepath -url http://%WSSSERVER%

We've been at it again.  If you haven't seen these, check out these new white papers:

• BizTalk Server 2004: A Messaging Engine Overview. This document describes the architecture and internal workings of BizTalk Server 2004 as a messaging engine.
• BizTalk Server 2004 and Web Services. This paper discusses how BizTalk Server 2004 supports the use of Web services in building solutions.
• Transactions Across BizTalk Server 2004. This document details transactional characteristics of BizTalk Server 2004 and how they apply across components, such as messaging, pipeline, and orchestration. It also describes how to construct business processes to guarantee message delivery.
• Working With BizTalk Adapter for SQL Server. This document provides general knowledge about the BizTalk Adapter for SQL Server. It gives suggestions about when to use the SQL transport and how to retrieve, insert, update, and delete data. It also includes code samples

A lot of folks heard at the Tech-Ed Keynote address on June 7 that we will have a joint launch of Visual Studio 2005, SQL Server 2005 and BizTalk Server 2006 on November 7, 2005.

Please do NOT confuse this with the RTM (Release to Manufacturing) date of BizTalk Server 2006.  BizTalk Server 2006 is still scheduled to RTM during the 1st Quarter of 2006.   November is just when the Joint  Launch "Party" will take place

I just posted a pretty cool sample of how you could call a map dynmically in C#.   Very nice.  However, it has a drawback (besides being officially unsupported).  

Specifically, the way I used TransformBase in the sample will result in regenerating a dynamic assembly for the xslt every time the code is executed. We'd run out of memory very quickly.

Here's the solution.  One of our engineers, Yossi Levanoni, created the Metadata classes that basically cache everything there is to cache for any of the XLANG artifacts, including maps.  

By replacing these lines of code:

    Assembly mapAssembly = Assembly.LoadWithPartialName("EAISchemas");
    Type mapTypeClass = mapAssembly.GetType("EAISchemas.MapToReqDenied");
    
    TransformBase map = (TransformBase)System.Activator.CreateInstance(mapTypeClass);
    XslTransform transform = map.Transform;
    XsltArgumentList argList = map.TransformArgs;

With this :

   using Microsoft.XLANGs.RuntimeTypes;

    XslTransform transform = TransformMetaData.For(typeof(EAISchemas.MapToReqDenied)).Transform;
    XsltArgumentList argList = TransformMetaData.For(typeof(EAISchemas.MapToReqDenied)).ArgumentList;

Will result in the maps being pulled from our Cache rather than from the Assembly each and every time....

Pretty cool stuff!

Every wonder how you could possibly reuse the XSLT that we store in BizTalk maps?   Every wonder how you could pick and chose which map to execute in your code from the compiled BizTalk Assembly.   Well there is a way!  The type containing the map xslt is devired from TransformBase contained in Microsoft.XLANGs.BaseTypes.  We can use that object to extract the xslTransform and the xslArgumentList objects from the BizTalk Map Type residing in the assembly and pass them to TransformMessage.   Pretty cool.   This allows you to resuse BizTalk maps and call them dynamically as you see fit.  For example, imagine the case of dynamic send ports where you also need to determine the map execution model dynamically or through a set of rules.  Remember though, this is completely Unsupported by Microsoft.

I put together a simple project from the shipped EAISolutions tutorial.  I added the EAIUtilies project to it which has the code for this.  This code is then called within a message assignment shape within the EAIOrchestration project to create a new message from the executed map.  Thanks Ashwani for helping me troubleshoot.

Download the project ..Check it out..

sing System;
using System.Xml;
using System.Xml.Xsl;
using Microsoft.XLANGs.BaseTypes;
using System.Reflection;
using System.IO;
using System.Text;
using System.Xml.XPath;

namespace EAI.Utilities
{
 /// <summary>
 /// Summary description for Class1.
 /// </summary>
 [Serializable()]
 public class MapHelper
 {
  public MapHelper()
  {
  }
  public static XmlDocument GetMappedDoc(Microsoft.XLANGs.BaseTypes.XLANGMessage msg)
  {

   XmlDocument xmlDocOutputMsg = new XmlDocument();
   XmlDocument xmlDocInputMsg = null;
     
   try
   {
    xmlDocInputMsg = (XmlDocument)msg[0].RetrieveAs(typeof(XmlDocument));

    Assembly mapAssembly = Assembly.LoadWithPartialName("EAISchemas");
    Type mapTypeClass = mapAssembly.GetType("EAISchemas.MapToReqDenied");

    TransformBase map = (TransformBase)System.Activator.CreateInstance(mapTypeClass);
    XslTransform transform = map.Transform;
    XsltArgumentList argList = map.TransformArgs;
   
    xmlDocOutputMsg = TransformMessage(transform,argList,xmlDocInputMsg);
      
   }
   catch (Exception exc)
   {
    System.Diagnostics.EventLog.WriteEntry("mapHelper",exc.InnerException.ToString());

   }
   return xmlDocOutputMsg;

  }
  private static XmlDocument TransformMessage(XslTransform xslt,XsltArgumentList args,XmlDocument sourceDoc)
  {
   XmlNodeReader sourceRoot=new XmlNodeReader(sourceDoc);
   XPathDocument srcXPath=new XPathDocument(sourceRoot);
   MemoryStream stm = new MemoryStream();

   XmlTextWriter writer = new XmlTextWriter(stm,System.Text.Encoding.Unicode);

   xslt.Transform(srcXPath,args,writer,null);

   stm.Flush();
   stm.Seek(0,SeekOrigin.Begin);
   
   XmlTextReader destReader;
   XmlDocument destDoc;

   try
   {

    destReader= new XmlTextReader(stm);
    destDoc = new XmlDocument();
    destDoc.Load(destReader);
    return destDoc;

   }
   finally
   {
   }
  }
 }
}

Remember, during the course we went over App Domain configuration for BizTalk.  The key to creating App Domains with BizTalk, and hence being able to use configuration files with your project is to edit the BTSNTSvc.exe.config file located in the BizTalk program directory.  The next key is to make sure you put the right sections in the right spot within the config file, else the BizTalk service will never start again (I ran into this myself once).

Why would you use them?  Well, here's a couple of reasons:

4) Internally, XLANG can use app-domains to host different versions of the XLANG runtime (e.g., Voyager vs. Pathfinder)

Here's a sample of the edits to the BizTalk config file.  Remember, the configSections comes first, and the sections it defines comes last....all the new additions are in RED.

<?xml version="1.0" ?>
<configuration>
 <configSections>
  <section
  name="xlangs"
  type="Microsoft.XLANGs.BizTalk.CrossProcess.XmlSerializationConfigurationSectionHandler, Microsoft.XLANGs.BizTalk.CrossProcess" />
 </configSections>

    <runtime>
        <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
            <probing privatePath="BizTalk Assemblies;Developer Tools;Tracking;Tracking\interop" />
        </assemblyBinding>
    </runtime>
   
    <system.runtime.remoting>
   
        <channelSinkProviders>
            <serverProviders>
                <provider id="sspi" type="Microsoft.BizTalk.XLANGs.BTXEngine.SecurityServerChannelSinkProvider,Microsoft.XLANGs.BizTalk.Engine" securityPackage="ntlm" authenticationLevel="packetPrivacy" />
            </serverProviders>
        </channelSinkProviders>
   
        <application>
            <channels>
                <channel ref="tcp" port="0" name="">
                 <serverProviders>
                     <provider ref="sspi" />
                        <formatter ref="binary" typeFilterLevel="Full"/>
                    </serverProviders>
                </channel>
            </channels>
        </application>
    </system.runtime.remoting>
    <xlangs>
  <Configuration>
   <AppDomains AssembliesPerDomain="10">
    <DefaultSpec SecondsIdleBeforeShutdown="5" SecondsEmptyBeforeShutdown="5">
    </DefaultSpec>
    <AppDomainSpecs>
        <AppDomainSpec Name="MyAppDomain">
            <BaseSetup>
                <ConfigurationFile>C:\Projects\Projects\AppDomainConfig\MyAppDomainConfiguration.config</ConfigurationFile>
            </BaseSetup>
        </AppDomainSpec>
    </AppDomainSpecs>
    <PatternAssignmentRules>
     <PatternAssignmentRule AssemblyNamePattern="AppDomainConfig.*" AppDomainName="MyAppDomain" />
    </PatternAssignmentRules>
   </AppDomains>
  </Configuration>
     </xlangs>
</configuration>

Lastly, here's the sample we walked through in class.  Yossi Levanoni put this together back in September of 03'.  

convenient chm file.  Its updated to SP1.  Enjoy.

I can't believe I see more and more recent BizTalk implemenations in production WITHOUT SP1.   With all the issues that we fixed in SP1, any new BizTalk Server 2004 project should simply never go into production without SP1.   I can't stress that enough.  The product team did phenomenal work raising the bar on the quality of this service pack.   It can be downloaded here. 

Additionally, I would also recommend that you contact premier support and ask for the following POST SP1 fixes, if you encounter any of the issues described below...which you undoubtedly will.

Here's the powerpoint decks we used for some of the extended sessions.  Some of these have added sections on all the perf tuning recommendations we layed out.  There's quite a bit that you have to consider when working in a farm scenario.   Also, you'll find some best practice around flat file, and the Flat File project I walked you through building.  In all you'll find the following:

1. Advanced Orchestration

2. Rules Engine

3. Building Pipelines

4. Messaging and Flat File best practice

5. Installation of farms and best practice and tuning parameters

Here are the decks, and here is the project we walked through to demonstrate parsing Flat Files...