Richard Murillo on Software Development

Building Performant Line of Business Applications with LINQ to SQL

Richard Murillo — Tue, 01 Sep 2009 21:45:00 GMT

The Scenario

Throughout the article, I will refer to the AdventureWorks database and use the following hypothetical customer scenarios:

Customer browses list of all products
Customer checks status of order through web portal showing all previous orders; detail for first order shown

Each step in the scenario requires different entities in different configurations; to keep explanations brief we will only use the Customer, SalesOrderHeader, SalesOrderDetail, and Product entities. The basis for which you tune is based on the business objects for each bullet of the scenario, starting with minimizing what data you really need.

Minimizing your Payload

The most straightforward way to accomplish item two of the scenario is to retrieve all orders for a given customer and display them in a table on the portal. The user clicks some header information (like the order identifier) and is taken to a detail page. Simple enough—the relationships of the entities and how the context traverses them is unchanged. The customer object is loaded, and when the page requests the order data for rendering, that data is also retrieved from the system. The fundamental issue involved here is we have to make two trips to the database to retrieve the entities: one for the customer, the other for their orders. How can this be made more efficient?

Data shaping with DataLoadOptions

The LINQ to SQL framework allows the developer to specify how eager or lazy the DataContext should be in loading related entities and materializing them. By default related entities (as specified in a 1:1 or 1:* relationship) are materialized "on demand", or lazily, when the property wrapping those entities is invoked. Conversely, a DataContext may also be configured to eagerly load related entities in an effort to reduce the amount of queries and data materialized for a given query (Microsoft Corporation, 2007) by loading all related when an entity is materialized and various configurations in between.

There is no absolute rule for configuring the DataContext one way or another, but is rather a function of what the object for the data is. An example of the differing configurations is evident in the various scenarios above. In scenario one, the customer does not need to know anything about their previous orders while placing a new order. In scenario two, the customer needs their order header information, but not the details. In scenario three, we need to know the customer, the order header information, and the detail.

The mechanism to control this is called DataLoadOptions and can be associated with a DataContext to control entity materialization aggressiveness.

Projections and change tracking

Additionally a projection could be made as a result of a series of joins to provide a flattened, read-only view of the data. Unlike Entities, which have identities and can be modified, projections cannot be persisted and are represented in denormalized views.

Comparing LINQ to SQL performance with ADO.NET

ADO.NET maintains only the transfer and fundamental materialization of CLR types to represent data and not classes representing the entities themselves. With less overhead the raw speed shows fairly impressive numbers at the cost of the development convenience.

Method	Median	Mean	Standard Deviation	Minimum	Maximum
DLINQ: no change tracking/compiled/projection	3407.861	3387.762	200.316	3073.884	3644.475
ADO.NET: DataReader	3461.938	3543.22	232.798	3453.19	4240.387
DLINQ: no change tracking/compiled	4978.377	5018.974	130.961	4949.884	5407.276
DLINQ: no change tracking	4978.478	5010.796	100.207	4940.606	5296.035
DLINQ	5381.326	5398.57	54.385	5340.671	5543.181
DLINQ: compiled	5396.588	5462.503	154.085	5351.099	5786.155
ADO.NET: DataSet	6198.333	6276.153	162.064	6146.066	6604.432

Table 1 Scenario 1: read only list of products

The table shows results for what we might expect, about 3.5 milliseconds to load the products from the database using an ADO.NET DataReader and about 6.2 milliseconds using an ADO.NET DataSet. The data also shows that while a simple one entity LINQ to SQL object graph does not perform better than an ADO.NET DataReader, a projection that bypasses the identity cache performs just as well. For comparison let us examine the code required to perform the action, one as the traditional ADO.NET DataReader, the other as the LINQ to SQL projection.

List<object[]> productValues = new List<object[]>(); 
using( var connection = new SqlConnection( LinqToSqlPerf.Properties.Settings.Default.AdventureWorksConnectionString ) ) 
{ 
    using( var command = new SqlCommand( "SELECT [t0].[ProductID], [t0].[Name], 
            [t0].[ProductNumber], [t0].[MakeFlag], [t0].[FinishedGoodsFlag], [t0].[Color], 
            [t0].[SafetyStockLevel], [t0].[ReorderPoint], [t0].[StandardCost], 
            [t0].[ListPrice], [t0].[Size], [t0].[SizeUnitMeasureCode], 
            [t0].[WeightUnitMeasureCode], [t0].[Weight], [t0].[DaysToManufacture], 
            [t0].[ProductLine], [t0].[Class], [t0].[Style], [t0].[ProductSubcategoryID], 
            [t0].[ProductModelID], [t0].[SellStartDate], [t0].[SellEndDate], 
            [t0].[DiscontinuedDate], [t0].[rowguid], [t0].[ModifiedDate] FROM 
            [Production].[Product] AS [t0]" ) ) 
    { 
        command.Connection = connection; 
        connection.Open(); 

        var dr = command.ExecuteReader( CommandBehavior.CloseConnection ); 

        while( dr.Read() ) 
        { 

            var productRowValues = new object[ dr.FieldCount ]; 
            dr.GetValues( productRowValues ); 
            productValues.Add( productRowValues ); 
        } 

        connection.Close(); 
    } 
}

Figure 1 Scenario 1 product retrieval as ADO.NET DataReader

The resulting implementation is 11 lines of code with an embedded TSQL statement.

using( var context = new DataClasses1DataContext() ) 
{ 
    context.DeferredLoadingEnabled = false; 
    context.ObjectTrackingEnabled = false; 

    var productsProjection = from product in context.GetScaledDownProductsFast() 
    select new 
    { 
        ProductID = product.ProductID, 
        Name = product.Name, 
        Number = product.ProductNumber, 
        Color = product.Color, 
        ListPrice = product.ListPrice, 
        Size = product.Size, 
        Weight = product.Weight, 
        Style = product.Style 
    }; 

    var products = productsProjection.ToArray(); 
}

Figure 2 Scenario 1 product retrieval as LINQ to SQL projection

The resulting implementation 5 lines of code without an embedded TSQL statement.

For the second scenario, we require joins on all four tables and require read only views.

Method	Median	Mean	Standard Deviation	Minimum	Maximum
DLINQ: data load options (details)/compiled	1313.25	1433.731	255.303	1285.668	1975.581
*DLINQ: data load options (details)/no change tracking/compiled*	1362.623	1405.075	158.995	1302.991	1874.912
*DLINQ: data load options (details)/no change tracking/compiled/cache*	1554.027	1659.795	284.601	1392.884	2289.98
*ADO.NET: DataReader*	1614.67	1694.006	319.316	1379.959	2373.989
*ADO.NET: DataSet*	1940.488	2072.922	306.141	1830.352	2871.116
DLINQ: data load options (headers)/compiled	6112.38	6213.204	472.648	5784.032	7345.794
DLINQ: compiled	8458.7	8693.848	494.015	8070.511	9690.908
DLINQ: data load options (headers)	8657.555	8685.617	135.128	8404.756	8931.598
DLINQ	9148.816	9371.009	509.871	8833.747	10461.56
*DLINQ: data load options (details)/no change tracking*	12069.17	12212.82	252.35	11975.24	12714.18
*DLINQ: data load options (details)/no change tracking/cache*	12176.1	12330.98	335.642	12136.32	13297.72
DLINQ: data load options (details)	12821.73	12903.43	394.513	12476.51	13517.67

Table 2 Scenario 2: Read only customer and order entities

The lines in Table 2 in grey show load options optimized for read-only scenarios, with the traditional DataSet implementation included for reference. Given the data for both scenarios several important observations can be made:

Compilation of a query on a simple object graph has little impact to the performance of the query
The object graph depth at which optimizations are performed matters significantly
Not registering objects with the change tracker affects overall performance significantly
Optimized correctly, LINQ to SQL performance meets expectations

In the first observation, we see that for scenario the second query compilation positively affects the performance of data loading such that when the query is compiled into a variable, along with any methods required, for later use. When the resulting expression is invoked, the cost for preparing the object graph and associated methods has already been performed (some elements of the query can be changed for variables), but no jitting is performed, resulting in gains of over 85.6% when compared to a straight LINQ to SQL implementation.

Additionally we find that the level that the data load options are placed in the object graph dramatically affects the performance of the query. The leading LINQ to SQL scenario has optimizations placed on the relationship between the Customer entity and the SalesOrderDetails entity such that when a Customer entity is loaded so are the related sales order headers, the order details, and the resulting products. Table 2 shows the outcome of placing the load optimization on the details resulting in a performance increase over other LINQ to SQL methods with different or no optimizations.

In the third observation, we also see that in instances where the context is not tracking object changes (and results are read-only) the code can execute faster, although in some instances not significantly. For the first scenario, toggling the change tracking feature resulted in about 7.5% performance gain. However, we also see that changing the query to use a projection, reducing the amount of data materialized, in addition to disabling change tracking, resulted in further gains—consistently exceeding the ADO.NET DataReader performance.

As another point of comparison, the two fastest implementations from ADO.NET and LINQ to SQL are below.

object[] customerValues = null; 
List<object[]> salesHeaderValues = null; 
List<object[]> salesDetailValues = null; 
List<object[]> salesDetailProductValues = null; 

using( var connection = new SqlConnection(AdventureWorksConnectionString ) ) 
{ 
    using( var command = new SqlCommand( "SELECT [t0].[CustomerID], [t0].[AccountNumber], [t0].[rowguid], [t0].[ModifiedDate] FROM [Sales].[Customer] AS [t0] WHERE [t0].[CustomerID] = @p0" ) ) 
    { 
        command.Parameters.AddWithValue( "@p0", 29475 ); 
        command.Connection = connection; 
        connection.Open(); 

        var dr = command.ExecuteReader( System.Data.CommandBehavior.CloseConnection ); 
        customerValues = new object[ dr.FieldCount ]; 
        
        while( dr.Read() ) 
        { 
            dr.GetValues( customerValues ); 
        } 
        connection.Close(); 
    } 
} 

using( var connection = new SqlConnection(AdventureWorksConnectionString ) ) 
{ 
    using( var command = new SqlCommand( "SELECT [t0].[SalesOrderID], [t0].[RevisionNumber], 
            [t0].[OrderDate], [t0].[DueDate], [t0].[ShipDate], [t0].[Status], 
            [t0].[OnlineOrderFlag], [t0].[SalesOrderNumber], [t0].[PurchaseOrderNumber], 
            [t0].[AccountNumber], [t0].[CustomerID], [t0].[SubTotal], [t0].[TaxAmt], 
            [t0].[Freight], [t0].[TotalDue], [t0].[Comment], [t0].[rowguid], 
            [t0].[ModifiedDate] FROM [Sales].[SalesOrderHeader] AS [t0] WHERE 
            [t0].[CustomerID] = @x1" ) ) 
    { 
        command.Connection = connection; 
        command.Parameters.AddWithValue( "@x1", customerValues[ 0 ] ); 
        connection.Open(); 

        var dr = command.ExecuteReader( System.Data.CommandBehavior.CloseConnection ); 
        salesHeaderValues = new List<object[]>( 16 ); 
        while( dr.Read() ) 
        { 
            var salesHeaderRowValues = new object[ dr.FieldCount ]; 
            dr.GetValues( salesHeaderRowValues ); 
            salesHeaderValues.Add( salesHeaderRowValues ); 
        } 
        connection.Close(); 
    } 
} 

using( var connection = new SqlConnection(AdventureWorksConnectionString ) ) 
{ 
    using( var command = new SqlCommand( "SELECT [t0].[SalesOrderID], 
        [t0].[SalesOrderDetailID], [t0].[CarrierTrackingNumber], [t0].[OrderQty], 
        [t0].[ProductID], [t0].[UnitPrice], [t0].[LineTotal], [t0].[rowguid], 
        [t0].[ModifiedDate] FROM [Sales].[SalesOrderDetail] AS [t0] WHERE 
        [t0].[SalesOrderID] = @p0" ) ) 
    { 
        command.Connection = connection; 
        command.Parameters.AddWithValue( "@p0", salesHeaderValues[ 0 ][ 0 ] ); 
        connection.Open(); 

        var dr = command.ExecuteReader( System.Data.CommandBehavior.CloseConnection ); 
        salesDetailValues = new List<object[]>( 16 ); 
        while( dr.Read() ) 
        { 
            var salesDetailRowValues = new object[ dr.FieldCount ]; 
            dr.GetValues( salesDetailRowValues ); 
            salesDetailValues.Add( salesDetailRowValues ); 
        } 
        connection.Close(); 
    } 
} 

using( var connection = new SqlConnection(AdventureWorksConnectionString ) ) 
{ 
    using( var command = new SqlCommand( "SELECT [t0].[ProductID], [t0].[Name], 
            [t0].[ProductNumber], [t0].[MakeFlag], [t0].[FinishedGoodsFlag], [t0].[Color], 
            [t0].[SafetyStockLevel], [t0].[ReorderPoint], [t0].[StandardCost], 
            [t0].[ListPrice], [t0].[Size], [t0].[SizeUnitMeasureCode], 
            [t0].[WeightUnitMeasureCode], [t0].[Weight], [t0].[DaysToManufacture], 
            [t0].[ProductLine], [t0].[Class], [t0].[Style], [t0].[ProductSubcategoryID], 
            [t0].[ProductModelID], [t0].[SellStartDate], [t0].[SellEndDate], 
            [t0].[DiscontinuedDate], [t0].[rowguid], [t0].[ModifiedDate] FROM 
            [Production].[Product] AS [t0] WHERE [t0].[ProductID] = @p0" ) ) 
    { 
        command.Connection = connection; 
        command.Parameters.AddWithValue( "@p0", salesDetailValues[ 0 ][ 4 ] ); 
        connection.Open(); 


        var dr = command.ExecuteReader( System.Data.CommandBehavior.CloseConnection ); 
        salesDetailProductValues = new List<object[]>( 16 ); 
        while( dr.Read() ) 
        { 
            var salesDetailProductRowValues = new object[ dr.FieldCount ]; 
            dr.GetValues( salesDetailProductRowValues ); 
            salesDetailProductValues.Add( salesDetailProductRowValues ); 
        } 
        connection.Close(); 
    } 
}

Figure 3 Scenario 2 using ADO.NET DataReader

Much like the previous example, the ADO.NET version is more verbose and includes embedded TSQL statements for data retrieval, totaling nearly 50 lines of code.

public static readonly Func<AdventureWorksDataContext, int, Customer> CustomerWithSalesOrderDetails = 
CompiledQuery.Compile( 
    ( AdventureWorksDataContext context, int identifier ) => 
            context.Customers.Single( c => c.CustomerID == identifier ) 
    ); 

using( var context = new DataClasses1DataContext() ) 
{ 
    var dlo = new DataLoadOptions(); 
    dlo.LoadWith<Customer>( c => c.SalesOrderHeaders ); 
    dlo.LoadWith<SalesOrderHeader>( soh => soh.SalesOrderDetails ); 
    dlo.LoadWith<SalesOrderDetail>( sod => sod.Products ); 

    context.ObjectTrackingEnabled = false; 
    context.DeferredLoadingEnabled = false; 
    context.LoadOptions = dlo; 

    customer = CustomerWithSalesOrderDetails(context, 29475); 
    headers = customer.SalesOrderHeaders.ToArray(); 
    header = headers.First(); 
    details = header.SalesOrderDetails.ToArray(); 
    products = details.Select( sod => sod.Products ).ToArray(); 
}

Figure 4 Scenario 2 using LINQ to SQL

Conversely, the same result is achieved with 14 lines of code with LINQ to SQL.

Caching

To this point, all of the tests and observations have been made against contexts with short lifetimes and in our scenarios, it has worked fine. However, in some other scenarios having to load fairly static data repeatedly may produce unnecessary overhead and slow the performance of the application down.

To optimize for this, LINQ to SQL includes an identity cache such that when an entity is retrieved it is automatically inserted into this cache. If you run another query where results overlap with an entity that has already been materialized in the same context, the entity is returned from the identity cache instead of the source data store except if using a projection.

Let us extend the second scenario such that the customer calls into a call center to check the status of the order, where an operator uses a Windows application to view/update the order on behalf of the customer. Given that, operators are reusing the same reference data repeatedly (such as product information, country, region, state, and province); it seems inefficient to reload the data for every screen that requires it.

Custom Object Pooling

To work around these issues, a crude object pool was created using the HttpRuntime.Cache object to store reference information. Additionally some modifications were also made to the objects to accommodate lazy lookups of reference data without violating the integrity of the developer experience.

[Association(Name="SalesOrderDetail_Product", Storage="_Products", ThisKey="ProductID", OtherKey="ProductID", IsUnique=true, IsForeignKey=false)] 
public Product Product 
{ 
    get { return _Product.Entity; } 
    set { 
            var previousValue = _Product.Entity; 
            if ((previousValue != value) || (_Product.HasLoadedOrAssignedValue == false)) 
            { 
                SendPropertyChanging(); 

                if ((previousValue != null)) 
                { 
                    _Product.Entity = null; 
                    previousValue.SalesOrderDetail = null; 
                } 

                _Product.Entity = value; 
                if ((value != null)) 
                { 
                    value.SalesOrderDetail = this; 
                } 

                SendPropertyChanged("Product"); 
            } 
        } 
}

Figure 5 Entity relationship for LINQ to SQL

public Product Product2 
{ 
    get { return EntityCache.GetOrAdd<Product>( p => p.ProductID == ProductID ).FirstOrDefault(); } 
    set { 
            SendPropertyChanging(); 

            if( value == null ) 
            {
                ProductID = 0; 
            }
            else 
            {
                ProductID = value.ProductID; 
                SendPropertyChanged( "Product2" ); 
            } 
        }
}

Figure 6 Entity relationship for cache

The EntityCache type, where the data is loaded from a context and stored in the local cache object, performs the work. The property accessors were updated on the entity to use the different means of managing the entity relationships due to issues with associating entities from multiple contexts together. Since LINQ to SQL really translates strongly typed entity properties to foreign keys, the keys are manipulated manually in the updated property.

The data showing the overhead of such an operation is shown in Table 2, where the caching method is 12% slower, but results in less data retransmitted over the network (each product in this case is about 1KB) and still performs the same or better than a straight data reader. While the performance degradation over 1KB of data is hardly persuasive, applied to our extension of scenario 2 where operators are loading the same reference data repeatedly may result in significant performance gains.

Building Line of Business Applications with LINQ to SQL

Richard Murillo — Wed, 26 Aug 2009 19:49:00 GMT

Introduction

The LINQ framework offers a basic pattern of sequence operators, allowing for primitive behavior such as filtering, grouping, and joining over collections and types. The support of the high level query comprehension syntax in C# 3.0 is paramount to developing an application quickly and allows for transparent development practices for developers where querying an in-memory collection or a remote database works the same.

While LINQ to SQL provides necessary services for radically simplifying how developers write applications, it has some shortcomings.

Why use LINQ to SQL?

To provide a sustainable model for future support, an object model was created to match the business entities used by the application. Having dependencies on tables and views in our relational database, the objects required “mapping” from the relational world to the object world, which was achieved using LINQ to SQL.

LINQ to SQL provides a simple mapping between tables, columns, and relationships to classes, fields, and collection properties (for relationships) that allow for “ease of use” programming at the cost of composability. Composability was lost due to the nature of databases and the limitations of LINQ to SQL such that relational databases relate tables via foreign key relationships where individual entities need not concern themselves with relationships (such as Customers to Orders); however, when mapping tables to classes the relationships are defined in advance.

Further, the use of classes mapped to tables allow for database abstraction and suit the needs of rapid application development such that the framework ensures correct database create, read, update, and delete statements are issued after changes are made to objects through the use of change tracking mechanisms.

LINQ to SQL Architecture Limitations

LINQ to SQL does not have a good story with distribution and concurrency such that new application design patterns for “cloud computing” do not translate well. Entities can be detached, attached, and moved around various layers in applications but are not truly portable (Strahl, 2007) and are really only for single-tier or partitioned multitier-esque solutions.

Multi-threading

LINQ to SQL (or LINQ for that matter) does not natively support distributed parallel queries. LINQ queries are translated to SQL statements and objects are materialized when required from the data stream. By design the LINQ to SQL DataContext class is not thread-safe and was not meant to be shared, however, compiled queries are thread safe and can be used with separate data contexts.

Forward only (reference) data

It is not uncommon for applications to include varying amounts of reference data used to assist the user in making a decision. This can commonly be seen in contact card scenarios where address details are bound to combo boxes (such as states or provinces and countries or regions).

The general guidance for using LINQ to SQL is to use short lived data contexts, where the data context is discarded after each unit of work operation. The fundamental issue encountered with this approach is entities that have data stored in the database and are materialized cannot be attached or associated to a new data context and therefore must be loaded with each new data context. The approach is valid and introduces a trivial amount of overhead when loading small or moderate amounts of data. However, in several instances our application can have large amounts of data loaded and bound, producing a noticeable delay when loading records.

To work around the short data context lifetime an object pool was created to store the values loaded from the database. All classes that used the reference data were modified to exclude the specific 1:1 foreign key attributes (effectively making a simple property) for LINQ to SQL. The contents of the property retrieved the appropriate value from the object pool, based on the foreign key identity, during retrieval and likewise updated the foreign key value during a set operation.

Concurrency

Through the use of time stamps on rows in our relational database, the system makes use of optimistic concurrency. When LINQ to SQL detects a concurrency conflict, an exception is raised where the list of conflicting objects can be retrieved.

Issues with time stamps

While using time stamps an issue arose that caused significant issues with the application:

1. Multiple entities with time stamp properties are updated in a context

2. During change submission conflicts occur for some of the entities but not all

3. The raised change conflict exception is caught and the conflicts are resolved

4. Change submission is requested now that conflicts are resolved and an exception is raised indicating the generated time stamp has been updated

The fundamental issue is one buried inside the LINQ to SQL framework—after processing the first change submission the properties are updated on the entities that are not in conflict. When the exception is raised the time stamp property is not reset to its original (due to the failure on submission) and left in a modified state. (Parra, 2008)

There are multiple workarounds for this issue:

1. Do not use time stamps for concurrency or disable automatic synchronization. Automatic synchronization can be disabled for update scenarios (or disabled completely) by changing a parameter on the property’s attribute. If synchronization is disabled for update operations, the issue is avoided; however, the time stamp property is uncoordinated with the database (Kulkarni, 2008), (Parra, 2008). This can be resolved in several different ways:

a. Resolve the concurrency conflict by merging the entity. A concurrency conflict exception is raised on the next call to submit changes and can be handled by merging the changed values with the database values (effectively refreshing the time stamp while keeping the user changes).

b. Proactively refresh the entity after the update operation. Once the update operation has completed issue a refresh for the entity to coordinate the time stamp values (Parra, 2008).

c. Discard the DataContext that materialized the entity. After the changes submit successfully the data context can be disposed. Future queries for the entity are performed in another context and will contain the correct value from the database (Kulkarni, Lifetime of a LINQ to SQL DataContext, 2008).

2. Manually update the entity’s time stamps to their original values. After the conflict exception is caught and the entities in conflict are resolved, the entities that were also not in conflict must have their time stamp properties updated.

Refreshing entities

LINQ to SQL offers a succinct means to refresh materialized entities, either singularly or in a collection; however, some behavioral anomalies were uncovered that lead to issues with the application—specifically issues with “deep refresh” or refresh of related entities:

1. The refresh method only works for objects that have already been materialized. If additional foreign keys have been added to the database they are not loaded on a refresh call--membership is fixed once the related entities are loaded (Kulkarni, LINQ to SQL Tips 6: How to refresh given DataContext won't stomp on your objects, 2008).

2. To load additional foreign keys a new select statement must be issued. New entities are added to the collection during a new selection; however, existing entities are not updated due to LINQ to SQL object pooling. A refresh is required of the entities to ensure they are up to date (Kulkarni, LINQ to SQL Tips 9: Understanding DataContext's internal caching, 2008).

This became an issue in our application where we have a single entity that contains a dozen foreign key relationships. Performing a deep refresh (having refreshed all data from all related entities from the database) issues dozens of queries to the database (specifically it is the number of related entities multiplied by two).

Query Throughput and Scalability

Our application employs the use of several static queries to retrieve data in various portions of the application (examples: work in progress, recently completed work, etc.). Upon execution of the query the results are bound to a list where the user can interact with them. The fundamental issue we had was with large amounts of objects being materialized on query execution (greater than 500). The queries took large amounts of time to complete (due to materialization overhead) and either timed out or the clients ran out of memory. In instances where objects were successfully materialized the users would encounter out of memory errors due to a memory leak with the DataContext object after two to four hours of regular usage.

To resolve this issue the application used stored procedures to return the required data quickly and bind to the list box for interaction, invoking object materialization when the user navigated to the item for interaction. The resulting read only list did not provide all of the rich interaction features the entities provided (such as updating data for multiple items in the list when it is updated in a detail screen), however, the resulting query performance was much more acceptable and netted a big usability gain. The issue with memory leaking still persisted but under normal usage conditions was not noticeable.

Concluding Remarks

LINQ to SQL provides the ability to expose database objects as they exist in the relational world as .NET objects without layers of abstraction. Providing an easy mechanism to expose LOB data at the cost of some performance, developers must focus on balancing materialized objects and their lifetimes. Our LOB tools were able to effectively accomplish this by minimizing the amount of tracked entities materialized by leveraging stored procedures for bulk data retrieval and LINQ to SQL for specific data edits resulting in a seamless experience to the end user.

Works Cited

Kulkarni, D. (2008, April 27). Lifetime of a LINQ to SQL DataContext. Retrieved from Dinesh's Cyberstation: http://blogs.msdn.com/dinesh.kulkarni/archive/2008/04/27/lifetime-of-a-linq-to-sql-datacontext.aspx

Kulkarni, D. (2008, May 23). LINQ to SQL Tips 6: How to refresh given DataContext won't stomp on your objects. Retrieved from Dinesh's Cyberstation: http://blogs.msdn.com/dinesh.kulkarni/archive/2008/05/23/linq-to-sql-tips-6-how-to-refresh-given-datacontext-won-t-stomp-on-your-objects.aspx

Kulkarni, D. (2008, July 1). LINQ to SQL Tips 9: Understanding DataContext's internal caching. Retrieved from Dinesh's Cyberstation: http://blogs.msdn.com/dinesh.kulkarni/archive/2008/07/01/linq-to-sql-tips-9-understanding-datacontext-s-internal-caching.aspx

Parra, S. (2008, June 2). Feedback: Linq. Conflict resolving records with timestamps. Retrieved from Microsoft Connect: http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=348901

Strahl, R. (2007, August 15). LINQ to SQL and Disconnected Entities Follow-up. Retrieved December 7, 2007, from Rick Strahl's Web Log: http://www.west-wind.com/weblog/posts/135659.aspx

Influencing Executives

Richard Murillo — Thu, 18 Jun 2009 01:03:40 GMT

Preparing Your Idea

The first real step in preparing a proposal with a greater chance of success is finding a subject you’re passionate about. Even if you present all the data to support your idea, your lack of passion for the subject will give you the disadvantage when asking an executive to take a risk on you and your idea. Once you have that, you need to ensure you’re not duplicating effort occurring elsewhere in the industry or within your own company. Once you’re in the clear on that you must refine your idea to make it crisp.

Executives take risks on people, make it count. Having clearly and concisely articulated your vision, the same must be done for the ROI associated with implementing your vision. Often ideas undertaking major changes are not specifically actionable (or at least cannot be easily explained and understood in 30 seconds), so they must be decomposed. In doing so the constraints of organization budget become less of a factor—individual actionable items can be accomplished in various stages to accomplish the larger vision. Additionally, if you’ve clearly explained the value of your vision, it is understood by the sponsor, and it is the right thing for the company to do, there is a greater chance you will get funding.

Making the Pitch

At this point you should be well prepared for your meeting. Ensure you can clearly articulate your vision in less than 30 seconds (elevator pitch) and remember to bring the passion. Data is optional, passion is not. It also good to note the clear separate between what you like to have and what is needed to be successful.

If in the event your sponsor does not approve your proposal, don’t give up. In the meeting debrief, consider the following questions:

Is the vision clear?
Is this really the right thing for the company?
Has the distinction between what you would like and what you need been clearly made?
Is there history of you not delivering results?
Is the scope granular enough so it can be accomplished?

Don’t give up. Iterate on your idea and your materials and try again. If your vision is clear and sufficient ROI present, your chances of getting approval are high.

Reporting Status and Delivering Results

Once you have funding it is now important that you deliver regular status updates and deliver. You may find that your executive sponsor wishes to be involved in your project—indulge them. Let them feel your passion and excitement by being involved. However in any circumstance, ensure that when you deliver your status that it is done so clearly and often, typically being less than 1/2 page. It is also important that mistakes made during your project are surfaced quickly and that responsibility and remediation is clearly evident. Surfacing mistakes made can potentially save large amounts of time, effort, and money to other team members or organizations within your company and will be appreciated by your sponsor.

It is also important to think of your project plan, and its artifacts, as results delivered for a “blue chip” company, even if yours is not. Blue chip results are based out of lean project plans, where results are consistent, of high quality, and are widely accepted. Much like companies of the same name, projects executed in this manner can weather changes in economic conditions and build reputations of stability and reliability.

Easy String to Resource file refactoring

Richard Murillo — Tue, 10 Feb 2009 21:47:00 GMT

During the development of internal tools we often have to stop working on features and clean up after ourselves to ensure that strings and such are embedded in the appropriate resource files and not hard coded into our applications. This can be a very tedious and time consuming process. Luckily there is a refactor extension that makes this very easy.

Resource Refactoring Tool

Resource Refactoring Tool provides developers an easy way to extract hard coded strings from the code to resource files.

Features for Resource Refactoring Tool

Works with C#, VB.Net languages. Supports all project types that ships with Visual Studio 2005 including web sites and web application projects.

A preview window to show changes.
Finds other instances of the text being replaced in the project automatically.
Lists existing resources by their similarity level to the text being replaced.
Automatically replaces hard coded string with a reference to resource entry.

More information at http://www.codeplex.com/ResourceRefactoring

Utilizing memoization and Dijkstra’s Fibonacci algorithm to compute large values

Richard Murillo — Wed, 15 Oct 2008 21:49:00 GMT

More of an acute fascination than anything else I expanded my use of memoization for computation to use a more efficient means of calculating Fibonacci sequences for values of n greater than 40 (previous Fibonacci example takes several minutes to compute value at 45 and is fairly unusable beyond that). To perform the computing of Fibonacci I utilized Dijkstra's algoritm for graph search pathing (algorithm details can be found in EWD654 "In honor of Fibonacci").

Dijkstra's algorithm for Fibonacci:

D(2n) = (2D(n-1) + D(n))D(n)
D(2n-1) = D(n-1)² + D(n)²

This algorithm in C#:

public static Func Dijkstra = n =>
{
     if( n < 2 )
          return n;

     double half = ( n % 2 == 0 ) ? n / 2 : ( n / 2 ) + 1;

     double p1 = Dijkstra( half );
     double p2 = Dijkstra( half - 1 );
     double result = default( double );

     if( n % 2 == 0 )
          result = ( 2 * p2 + p1 ) * p1;
     else
          result = Math.Pow( p2, 2 ) + Math.Pow( p1, 2 );

     return result;
};

Utilizing the memoization technique the amount of computations performed drops significantly. To compute the value of 50 the following computations are performed:

F(50) = F(25) and F(24)

F(25) and F(24) = F(13) and F(12)

F(13) and F(12) = F(7) and F(6)

F(7) and F(6) = F(4) and F(3); F(3) and F(2)

F(4) and F(3) = F(2) and F(1)

F(3) = F(2) and F(1)

F(2) = F(1) and F(0)

F(1) and F(0) are 1 and 0

Utilizing this technique the number of computations required is 14. At the cost of some space the computations become faster as the graph becomes denser.

Technorati Tags: C#,code,tech,software,algorithms

Significantly speeding up computations with smart predicates

Richard Murillo — Wed, 15 Oct 2008 21:13:48 GMT

There is a technique that allows optimization of repeated calculation results for a set of inputs called memoization. This technique is particularly helpful when working with numeric computations, pathing, such as for tree searches, etc. where the time taken to traverse the result set is expediential. What's more interesting is that the technique is applied on top of the System.Func object allowing for transparent optimization of those delegate types. Here is the method that I use to perform the optimization.

public static class SystemMemoizationExtension 
{ 
    private static object lockObject = new object(); 
    public static Func Memoize( this Func func ) 
    { 
        Dictionary t = new Dictionary(); 
        return n => 
        { 
            if( !t.ContainsKey( n ) ) 
            { 
                lock( lockObject ) 
                { 
                    if( !t.ContainsKey( n ) ) 
                    { 
                        var result = func( n ); 
                        t.Add( n, result ); 
                    } 
                } 
            } 
            return t[ n ]; 
        }; 
    } 
}

To utilize the method simply assign the result of the Memoize method back to your delegate as shown in the example below.

Func<int, int> Fibonacci = null; 
//Recursive Fibonacci algorithm
Fibonacci = n >=n <= 1 ? n : Fibonacci(n – 1) + Fibonacci(n-2); 
//Optimize lookups
Fibonacci = Fibonacci.Memoize();

While this is an overly simplistic example of recursion optimization you can see that by calling the code once then again using the same input that a dramatic improvement is made performing the computation (an input for F(n) where n = 42 takes several seconds to compute, while the subsequent call takes sub seconds). The technique itself is useful for reducing the traversal time for any top-down parsing.

Technorati Tags: C#

Extensions I found useful while developing with WPF and LINQ to SQL

Richard Murillo — Wed, 15 Oct 2008 20:54:11 GMT

When I observe developers writing solutions using LINQ to SQL or Entity Framework, I see a number of conversions occurring time and time again. Below are some of the more trivial utility method extensions I used when developing solutions based on LINQ to SQL and WPF—simple as they are I found them useful in a variety of situations.

Extension to remove data from a collection using a predicate.

public static void Remove<T>( this ICollection<T> data, Func<T, bool> predicate )
{
     if( predicate == null ) 
     {
          data.Clear(); 
     } 
     else 
     { 
         foreach( T item in data.Where( predicate )) 
         { 
               data.Remove( item ); 
         } 
     } 
}

Converting an enumerable to an ObservableCollection for data binding

public static ObservableCollection<T> ToObservableCollection<T>( this IEnumerable<T> collection ) 
{ 
    List<T> list = new List<T>( collection.Count() ); 
    using( IEnumerator<T> enumerator = collection.GetEnumerator() ) 
    { 
        while( enumerator.MoveNext() ) 
        { 
            list.Add( enumerator.Current ); 
        } 
    } 
    return new ObservableCollection<T>( list ); 
}

The same but for a list

public static ObservableCollection<T> ToObservableCollection<T>( this List<T> collection ) 
{ 
    return new ObservableCollection<T>( collection ); 
}

Software development outsourcing and client proximity

Richard Murillo — Sat, 08 Mar 2008 22:09:13 GMT

I had a recent comment come in that I wanted to share:

I was wondering what your take was on Consulting firms. Although it is not offshore outsourcing, in some sense, it is still outsourcing their projects to firms. I totally agree with you regarding the lack of communication and planning to execute and deliver projects when attempting to have someone work on a project 2000 miles away from you.

The short of it is there is still risk in outsourcing no matter the proximity, however, I believe that there are three key tenets that facilitate failure in the situation:

Spoken and written language comprehension
Cultural awareness and differences
location, location, location

Communication plays a fundamental role in project success, whether it is with the local project team, virtual team members, or external parties. I believe the biggest challenges we face with traditional overseas outsourcing revolves around this, however I have experienced similar challenges with outsourced work with American vendors for different reasons all together, so there's a project risk either way.

Fundamentally it comes down to a risk management decision for the client. I have found that groups and companies that do not have their own crisp corporate governance, operational and infrastructure policies are more often than not unsuccessful in outsourcing (root cause is uncontrolled chaos undermining communication plans and data management policies). In a vendor/client model there are a few things we have exercised to be successful: emphasis on information security, a clear and well understood communication protocol, and frequent audits to ensure policy compliance. Above all of that there's also the associated business risk of the outsourcing deal that needs to be managed up front. In either "near" or "far" outsourcing, the recognition of a failing deal is pivotal to shifting risk back on to the vendor.

Visual Studio 2008 Released

Richard Murillo — Mon, 26 Nov 2007 23:05:37 GMT

After months of waiting, I can finally say that Visual Studio 2008 was released last week. There are hundreds of new features, language enhancements, and technology integrations in this version. To help you get your head around what is new in Visual Studio 2008 and .NET Framework 3.5, get the training kit from Microsoft download center.

Some new features that I am excited about:

Visual Studio performance improvements
AJAX integration with javascript intellisense
Cider integration (WPF visualizer)
Multi-core builds
Windows Server 2008 / SQL Server 2008 support (TFS)
Vista UAC compliance
Continuous integration
Web Access
Integration of Database Edition (VSTS SKU)
T-SQL Static code analysis
SQL Dependency tree visualization
Schema reporting for documentation
T-SQL refactoring
New designers for SOA scenarios
Hotpathing (tell me where perf in my application sucks)
LINQ

Why moving work offshore fails

Richard Murillo — Fri, 20 Apr 2007 19:19:00 GMT

Not to say that all offshore projects fail, I have both been a contributor and leader in projects that have been a great success and those that have been complete failures. Each project had its own reasons for failing and were defined by the business (e.g. why are you sending it offshore to begin with) that typically include cost savings, quality, increased turnaround in deliverables, and customer satisfaction--all of which are key drivers for moving work offshore, but in doing so there are key areas your business must pay close attention to.

According to Gartner, businesses will spend more than $50 billion USD on offshore and "near-shore" outsourcing by 2007 and many projects will fail because of poor planning. Gartner also maintains that there are benefits achieved by those businesses that successfully outsource their non-core processes, however, rewards will not be instant.

Reasons for Failure

Often times, the effort and time involved in communicating with offshore team members and maintaining that relationship is underestimated. In my experiences with offshore teams to date, there has often been a lack of key items to complete awarded work effectively (like infrastructure, soft skills, planning). Additionally, coordinating between the teams requires longer hours, detailed planning, and cultural training—all of which are more expensive when working with offshore than with your own staff and can, as I have observed here, lead to lower morale and reduced deliverable output. Gartner also observes that lack of productivity in the offshore is also an issue for several reasons including high staff turnover and skill levels, especially in highly competitive markets such as Bangalore and Hyderabad, India.

Much like the dot com days, new programmers coming into the field of work are inexperienced, attaining only the necessary core skills for them to receive a job in this field and often struggle with ambiguities in specification or shifting directives. As such, the teams typically do not operate with clear processes and depend on the competence and heroics of those more experienced and not on the use of proven processes. In spite of the chaos, these teams often produce usable work products; however, they frequently exceed the budget and schedule. More often than not these teams over commit, abandon any established process in times of chaos, and may not be able to repeat past successes again.

What's more, senior executives are not involved to keep strategy on track and morale high—they are only in the picture when a significant escalation occurs or to sign a new deal. As I mentioned in my previous entry, in order for an offshore deal to succeed there needs to be a good level of communication between all parties. Requirements, goals, and expectations have to be defined clearly and in detail. Your onshore managers need to explain to the coordinating staff why the work has been sent offshore and what benefits are expected.

More often than not, the cultural differences will come into play creating havoc for the project; classic incarnations of this include not questioning authority and just pressing forward by the offshore team. All too many times do we find out late that guidance or requirements had been ignored for cross cutting to please the schedule rather than announcing a slip.

How can Visual Studio Team System help?

Gartner advises companies that plan on offshoring work to figure out their IT process maturity and identify gaps in your process. As previously mentioned, it is important to set all expectations clearly up front with your vendor. When using Visual Studio 2005 with Team Foundation Server, several mechanisms out of box enable teams to work effectively in these environments:

Process: out of the box, Visual Studio Team Foundation server includes MSF for CMMI Process Improvement Level 3 and MSF for Agile Development work item templates. By utilizing the templates and the process behind them, teams can effectively work across physical boundaries with increased confidence and transparency, allowing software development activities to be predictable and success repeatable.
Communication: Visual Studio Team System facilitates the transparency between individuals and teams with work items, a shared team portal, integrated change management, and a common data repository. The availability of information, and insight into an individual's progress, creates a more unified work environment regardless of physical location. Project managers can stay informed on an individual's progress without having to visit each individual—having real time information about each individual's work and their progress allows project managers to create precise schedules and report more accurately to management
Productivity: utilizing the common repository, managers and leads can answer common questions such as: What's in the current build that QA can test today; are requirements being met; are my teams adhereing to quality standards; is the product ready. Further, it provides the single team portal for integrating source code, issue tracking, project plans, vision statements and others that are critical assets to a project team.

Outsourced Projects using Visual Studio Team System

Richard Murillo — Tue, 03 Apr 2007 23:59:00 GMT

Having been included in two separate teams where our outsourced projects don't go as well as expected I cannot help but to take a step back and ask why things are consistently failing with separate teams, and separate organizations.

The commonalities in both projects were fixed-bid (i.e. fixed price, features and time) using a modified waterfall model (to account for geographic locations). I know—you say the waterfall model doesn't work. Be that as it may organizations revert to this model and are thus handicapped; unable to make adjustments to the changing business and unable to make definitive decisions on how to improve their process, but more on that later. Other commonalities included high level and low level designs, unit test cases, test cases, etc.

I have heard that groups similar to mine are having great success with outsourced projects, and I myself have also been involved with projects that were executed to great success, however I assert that for us there are areas of improvement. At a high level, I can sum it up in one word: process.

Where have my two projects failed? IMHO it is in the analysis of the requirements. The requirements had the vendors pegged implementing features on product technologies they were unfamiliar with—some churn in requirements and with new technology. Traditionally engineering teams would perform proof of concepts and experiments to validate design, and clickable prototypes to share with customers to stabilize requirements (business and technical). While some of that was done here, I still see where it could have been improved. In both instances, we started with a solid vendor relationship that quickly begins to fall apart due to lack of trust. IMHO the biggest areas we are lacking are:

Trust between teams
High SNR in communication (i.e. direct, candid communications on issues, risk and status)
Close coordination between said teams

IMHO in both projects are considered high risk when examining the vendor skill set, staffing model, and experience with projects of this complexity. In most situations you can apply the Parento principle (AKA 80-20 rule) where 80% of staff is offshore while 20% is onsite for R&D phases, moving to a 90-10 planned iterative model for build and less onsite still for stabilization.

In both projects the offshore team encountered additional issues during implementation that require workarounds for resolution, R&D, or a complete shift in approach. On both projects we tried to mitigate risk by having frequent check-ins with the offshore team and checking progress on a tight basis (one was almost daily, the other bi-weekly). Alas when significant and unexpected delays arose the relationship fell to pieces and in response we put in-house staff on the project to sure it up and get it going in the right direction.

Lessons Learned

Complete as many iterations as you can: That is, don't disrupt forward progress on the project for the sake of getting a work product from the team, but do get deliverables as often as it makes sense. Some of our projects get some weekly, others monthly.
Use collaborative tools to increase confidence in deliverables: Your in-house staff does not have to wait until the end of the formal delivery cycle to perform review or check in with the offshore team. Utilize tools like LiveMeeting Net Meeting, Windows Live Messenger, etc. to increase collaboration
Code review, code review, code review: DO NOT UNDERESTIMATE THIS. A majority of issues we found on one project were found in code review by in-house staff familiar with the business rules. In both projects the adherence to offshore code reviews prior to sending to our in-house staff was not maintained, resulting in many bugs raised later in the cycles. In the reviews we took code that was a representative sample (chosen at random, but was a deep slice) and did a review. Additionally we looked at key strategic areas for performance bottlenecks and security to ensure code conventions, standards, and requirements were being met
Integrate as frequently as possible: Our teams use branches to logically separate the project or feature into areas for work. As a matter of practice, the more frequently these branches can be integrated the faster you can identify and resolve defects.
Where possible, leverage frameworks: Use application blocks (Enterprise Library), Guidance Automation (Web Client Software Factory, Smart Client Software Factory) to solve common problems. Where possible, create your own frameworks and reuse them on projects to reduce the amount of custom code required to be written, which decreases development time and improves quality.
Use Design Patterns: These patterns illustrate common ways to solve problems. Being standard, they can convey meaning across geographical boundaries. Further, by implementing patterns maintenance complexity is reduced and documentation and clarity is improved. A different vendor in a different country, maybe even in-house staff, may need to understand and modify the code—using a proven design allows ramp up to easily occur.
Use useful metrics to measure quality and progress: Build metrics into SLAs and look at meaningful metrics, such as defect density, bug reactivations, code churn, code coverage (unit test or otherwise), project velocity, etc.

Leveraging Visual Studio Team System

I have found that VSTS handles the needs of geographically dispersed development teams (and their inherent challenges) in a variety of ways:

"Better" communication and coordination: Team Foundation Server provides the ability to track work items (tasks, bugs, issues, requirements, risks, change requests) in addition to source code and documentation. The architecture, utilizing web services and cache servers, is optimized for distributed teams working over slow or unreliable connections. The product also allows for customizable notification of events to give you the clarity you need from the projects and help improve onsite/offshore communications
"Better" status reports: Coupled with the data warehouse maintaining statistics and work item history, reports can be generated to give you insight in to some of the metrics I mentioned above. The platform is also extensible, utilizing SQL Analysis Services (SSAS) and SQL Server Reporting Services (SSRS) to allow team members to gather information with various levels of granularity.
"Better" code quality: the product also includes a unit testing framework that works with the centralized build and reporting portions to give indications of code churn, unit test coverage, and pass rates. Check in policies can be configured to check if the code compiles, run static code analysis, even run selected tests

Each one of these allow the onsite team to ensure compliance with corporate standards and gain trust in confidence in the offshore team's ability to deliver a quality, stable product.

Team Foundation Server Power Tools (tfpt) 1.2 Released

Richard Murillo — Wed, 21 Feb 2007 22:49:12 GMT

Today the Team Foundation Server team announced the release of version 1.2 of Team Foundation Power Tools (formally known as Power Toys). What are the Power Toys you say? They're extra functionality delivered out of band for Team Foundation Server including some really useful stuff (get changes for changeset, consolidated annotation of a file, command line history, and rollback to name a few).

In the latest release they have included support for Windows Vista (woo-hoo!) and several new tools:

[from announcement]

Workspace Command (tfpt.exe) - Use the workspace command for additional workspace operations not supported in the currently shipping Team Foundation Server command-line tool (tf.exe).
Treeclean Command - Use the treeclean command to see and optionally delete files in the current directory and all subdirectories that are not under version control.
Process Template Editor - A tool that provides a user interface for authoring work item types and some of the associated process template components.
Check-in Policy Pack - A set of custom policies that address often expressed customer needs. For example, the Custom Path Policy allows you to filter the specific items upon which another policy acts.
Build Test Tools Task - An MSBuild task that allows you to run unit tests by specifying the DLLs or a file name pattern in the TfsBuild.proj file instead of using .vsmdi files to specify which tests to run.

One thing that I am very excited for is the new path policy (included in the Check-In Policy Pack). This allows policies to be applied to specific paths in team foundation server (i.e. don't check for valid unit tests on a directory with 3^rd party source code). The other policy I'm excited for is the work item query policy—my project teams have several builds operating concurrently within the same team project and sometimes a private build is executed and picks up work items and associations not belonging to it specifically creating false relationships and throwing off some of our reporting.

To download version 1.2, go to the Microsoft Download center at http://www.microsoft.com/downloads/details.aspx?familyid=7324c3db-658d-441b-8522-689c557d0a79&displaylang=en Help for the new tools can be found on the TFPT Forums at http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=930&SiteID=1

Additional information on the tools in depth can be found on Rich's Developer Tools Blog

Getting my Sandcastle Team Build targets file to work with Sandcastle September CTP

Richard Murillo — Tue, 31 Oct 2006 20:45:00 GMT

A few things before we get started:

You will need the HTML Help Workshop installed
September CTP of Sandcastle
My September CTP Sandcastle Targets File (attached)
This file needs to be copied to your %programfiles%\MSBuild directory

Next you will need to make some modifications to the standard configuration file (I am using the VS2005 style)

Make a copy of %programfiles%\Sandcastle\Presentation\vs2005\configuration\sandcastle.config
Change references of
1. ..\..\ to %programfiles%\sandcastle\
2. ..\cpref_reflection to %programfiles%\sandcastle\examples\cpref_reflection

Update your TFSBuild.proj to include the following:

Add a new Import tag referencing the sandcastle targets file
Add a new target called DocumentCLR
<Target Name="DocumentCLR">
<CallTarget Targets="SandcastleDocument" ContinueOnError="false" />
<MakeDir Condition="!Exists('$(BinariesRoot)\Documentation')" Directories="$(BinariesRoot)\Documentation" ContinueOnError="false" />
<Copy SourceFiles="$(SandcastleWorkingDirectory)\Output\$(SandcastleDocumentationName).chm" DestinationFiles="$(BinariesRoot)\Documentation\$(SandcastleDocumentationName).chm" ContinueOnError="false"/>
</Target>
Add an additional target that overrides the AfterCompile target
<Target Name="AfterCompile">
<CallTarget RunEachTargetSeparately="true" Targets="DocumentCLR" ContinueOnError="false"/>
<OnError ExecuteTargets=" OnBuildBreak;"/>
</Target>
Update your TFSBuild.rsp file to include some configuration
/p:SandcastleDocumentationName=NameOfYourCHM
/p:SandcastleConfiguration="C:\Path_TO_MODIFIED_CONFIG\sandcastle.config"
/p:SandcastleDocumentationOutput=$(BinariesRoot)\Documentation
/p:SandcastleDependencies="%windir%\Microsoft.NET\Framework\v2.0.50727\*.dll,\\lcabuildvm01\3rdParty\Dependencies\*.dll,%windir%\assembly\GAC_MSIL\Microsoft.VisualStudio.QualityTools.UnitTestFramework\8.0.0.0__b03f5f7f11d50a3a\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll"
/p:SandcastleHelpGeneration=CHM

Reusable MSBuild Sandcastle Targets File

Richard Murillo — Tue, 31 Oct 2006 03:24:00 GMT

Based on a past post, I have an external targets file you can place in your $(MSBuildExtensionsPath) folder (attached in zip).

You can call it like this:

</Target>

</Target>

The following is configuration from my msbuild response file:

/p:SandcastleDocumentationName=MyProjectDocumentation

/p:SandcastleConfiguration=C:\YOUR\CONFIG\sandcastle.config

/p:SandcastleDocumentationOutput=$(BinariesRoot)\Documentation

/p:SandcastleDependencies="%windir%\Microsoft.NET\Framework\v2.0.50727\*.dll,C:\YOUR\Dependencies\*.dll,%windir%\assembly\GAC_MSIL\Microsoft.VisualStudio.QualityTools.UnitTestFramework\8.0.0.0__b03f5f7f11d50a3a\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll"

/p:SandcastleHelpGeneration=CHM

Note: tested with build 2.0.2426.28539

VSTF Feature Request

Richard Murillo — Wed, 25 Oct 2006 19:43:14 GMT

There have been several instances when working with multiple teams that they need to build different build configurations on the same build machine. For example, we have a central build type that encapsulates the compilation of the bits, database installation and unit testing, and packaging of the compiled bits into an MSI. We also have build configurations that separate each. With OOB functionality you get an error message with the details of a soap exception explaining that only one build can be run at a time, which is understandable, but what about queuing?

To get around this issue, I built a queuing web site that can be used to invoke builds instead of through the team explorer. The code uses the thread pool to wait for the active build to complete then starts its build task