Towards Web 3.0

PDC2009 @ LA – Day 2 Coverage – Keynotes and Announcements Summary

2009-11-19T13:20:27Z

Steven Sinofsky (President, Windows Division), Scott Guthrie (VP, Microsoft Developer Division) and Kurt DelBene (Sr. VP, Office Business Productivity Group) were the top speakers and headliners for today’s (day# 2) keynote at PDC2009 in Los Angeles, CA today. The PDC is historically quite important for announcements, this year was no different.

· Biggest applause of the day# 2 comes from Sinofsky when he mentioned about a partnership project with Acer where it puts its own team through the process of actually building a laptop computer, just to see how one is built - what laptop engineers actually go through. In learning the system that Acer goes through, Microsoft built its own limited editions for PDC'09 laptops. They (i.e. Acer machine with Microsoft’s preferred software image, resistive multi-touch, accelerometer) will be giveaways to all PDC attendees!!

· In terms of pure wow factor from the announcements, Silverlight 4.0 steals the show! The big news today is Silverlight, specifically the beta of version 4.0. Silverlight continues its march towards providing more and more of the functionality of .net, almost full WPF and Windows. Silverlight 4.0 is incredible, that’s what you’ll hear from anyone that watched the day 2 keynote and demos.

(Background, Silverlight 3 released just over a year ago and here we are with yet another release full of features that our developer community has been asking for. This beta release is a developer release.)

Background from PDC2009 Day # 1

· Everything is about 3 screens (desktop, phone and tv) and the cloud.

Day 1 focus – Backend i.e. Azure and cloud based services
Day 2 focus – Office, IE, Silverlight & Windows

· Emphasis on IE + Silverlight for all 3 screens – desktop, phone and TV.

There was quite a bit of developer emphasis with Silverlight in contrast to Windows 7. Windows 7 is great but the difference between Silverlight and Windows is continuing to shrink. Silverlight 4.0 also even supports multi-touch as well as lots more direct hardware access.

Announcements from Keynotes

· Public beta will include all features demonstrated today, tooling support for VS 2010. Available for download now at http://silverlight.net/getstarted/silverlight-4-beta/.

Final release of Silverlight 4 shipped first half of next year.

· Office 2010 and SharePoint 2010 public betas go live today.

· Office Mobile Clients for Windows Mobile 6.5 betas available today in Windows Mobile Marketplace.

· Outlook Social Connector -- . Get social networking in Outlook with people info, history, activities; SharePoint 2010 Provider, Windows Live Provider and a plug-in involving partners such as LinkedIn. Hence, enabling information from individuals' social organizations and networks to be displayed in a meaningful context in Outlook. This is “SDK based approach/ ability to make providers” implying that developers will be expected to integrate and deploy it.

· Promise of discussion on Windows Mobile at MIX’10 – March 15-17, 2010 in Las Vegas.

· The announcement that got faded due to other announcements was XRM for Microsoft Dynamics. It is one of the more silent but promising pieces of Microsoft’s cloud technology.

oXRM is Microsoft’s only fully multitenant application platform as a service technology that runs on and off premises.

· Other betas released today are

Microsoft Project 2010 Beta
Microsoft Visio 2010 Beta
Also announced: PowerPivot for Excel 2010 – CTP version is available - http://www.powerpivot.com/
Download URL: http://www.microsoft.com/2010

Details

Silverlight

· Silverlight now on 45% of the world’s internet-connected devices (up from 33% in the summer)

· Silverlight 4.0 under trial by Bloomberg, National Instruments, Siemens (medical diagnostic imaging)

· Silverlight will be used this Winter for Victoria’s Secret Fashion Show and Winter

· Silverlight 4.0 key focus – Media, Business Applications, Beyond the Browser

· Silverlight 4.0 offers Webcam & Microphone support on the machine (including raw access), multi-cast streaming and offline DRM support

· Silverlight 4.0 introduces support for printing, rich text, clipboard access, rights click, mouse wheel, implicit styles, drag and drop, html hosting (including content as brush)

· Silverlight 4.0 includes compile REST protocol enhancements; improved WCF support (such as TCP channel support) and RIA

· Visual Studio 2010 Silverlight support - WYSIWYG Design Surface for XAML

· Better intelli-sense, Improvements for Data Binding, Layout and Styles, Services Integration

· Silverlight 4.0 offline capabilities includes Windowing APIs, Notification pop-ups, HTML hosting and Drop Target

· Silverlight 4.0 offline ‘elevated’ model includes: Custom Windows Chrome, Local File System, Cross-Site Network, Keyboard in Full Screen Mode, Hardware Device Access, COM Automation of local objects (using the dynamic keyword in C#) and location APIs.

Ability to build trusted applications that run outside the sandbox on Windows and the Mac, the key is that user consent dialog is provided automatically.
For example:
Office 2010 calendar can be queried, Pivot-Charts brought up from Excel
Multi-touch features with basic features, zoom in, zoom out.

· Silverlight 4.0 - Twice as fast, 30% faster startup and offers new profiling support; Transfer data 600% faster using internal transfer protocols instead of HTTP.

· Silverlight 4.0 is supported on Google Chrome.

· Silverlight 4.0 is still under 5MB to install.

· It will ship the Silverlight 4.0 Facebook integration demo as a reference sample

· •70% of voted-for Silverlight 4 features (including 9 of top 10) included

· View the list of the new features at

o http://channel9.msdn.com/learn

· Silverlight can use client-side object model to talk to SharePoint 2010

Silverlight Key Demos

· Webcam application that captures live video, can do live effects with bulge, distortion. Integration with Twitter enables the result to be live-integrated into Twitter profile, so a picture just taken becomes the user's Twitter icon.

· Open-source library for barcode reading, can immediately look up the prices of any object scanned from the barcode scanner, pulls up distributors or retail sellers.

· Demo of Vancouver Olympics site with Silverlight player, with instant seek.

· Rich text control that ships in Silverlight 4.0: Arabic, Hebrew, Kanji character sets all within the text editor. Custom context menu after right-click. Can paste and insert text, pictures, and DDE-like controls into a Silverlight app, such as a Data Grid control

· from Excel. Can cut and paste from Data Grid control back into Excel.

· Text can be dragged and dropped from a browser to the application. Print Preview works, including with custom Print Preview dialogs. Silverlight 4.0 will now write directly to the printer, has a print API.

· HTML control is hosted within Silverlight app. HTML image can be converted into a brush -- the entire HTML page can be used as a brush, so that the page can be converted into, say, a jigsaw puzzle, the pieces of which can be juggled around the screen.

· Brian Goldfarb is demonstrated an S4 application that utilizes Facebook, but which uses its own chrome to develop a real, custom app on top of custom Facebook apps. Can also take advantage of COM automation to right-click data from Facebook, then add to the Outlook calendar. Access to the Outlook inbox, with virtual wall on the right "to contextualize who I'm talking to." Photos dragged-and-dropped from the outside can be loaded into the application live, then tagged, prior to being sent to Facebook.

Internet Explorer (IE)

· First IE9 news: Three weeks into the project, the team is focused on the following areas:

Standards: Acid3 and new emerging standards like HTML5
JavaScript performance
Hardware-accelerated DirectWrite/Direct2D Graphics & Text

· IE9 already on 32/100 on Acid3 test, up from 20/100 on IE8

· CSS selectors test, using CSS3.info - passed 572 out of 578 (a variation of the SlickSpeed test) for CSS selectors used in rendering.

· IE rendering engine will support rounded borders in CSS.

Rendering engine will use hardware-acceleration in DX9 mode (not DX10), using Direct2D. Highly resolved text with much resolved clarity.
Sub-pixel positioned text using DirectWrite. Zooms used to be jittery in GDI, nice and smooth as we move to Direct2D. Smooth realignments - "changes without you having to do anything different with your site."
Maps re-rendering will use 60 fps rather than 2 or 3, by moving to Direct2D from GDI.

· Lot of new APIs in Windows 7, and IE will take advantage of these APIs.

· Videos of demos will soon be available on Microsoft Channel 9.

IIS Media Capabilities

IIS media tool, new version will enable streaming of media directly to Apple iPhone. Video can be encoded once using smooth streaming, target clients using the iPhone.

Windows

Telemetry Data

Sinofsky mentioned about the lessons learned at Microsoft about being responsible about how to disclose information about the product. "You should expect us to have learned that lesson about responsible disclosure, and to continue as we move forward.”

The value of the "Send Feedback" button, learning from clients what drivers were loaded, whether the installation of drivers and services were successful. Software Quality Monitor is designed to be opt-in for customers, but the customers were "opted in automatically" during the preview and beta.

· Lots of telemetry data from the Windows development cycle – even stuff like number of presses on Start button and Aero Snap/Shake uses.

· The audio of the audience at the last PDC was analyzed – best reaction was to the new Windows 7 slider control for UAC levels.

There was also a silly Video about how Microsoft programmers are held directly responsible for the errors of their ways, by way of a kind of "agony chair" that shocks, stuns, or stabs the individual developers discovered using the Watson logs to have been responsible for a specific problem. Filled with entertainment-only disclaimer.

Windows Demos:

· Various demos of W7 new hardware-supporting features – touch, sensors, hardware-accelerated encoding, DirectX 11, etc.

· Angiulo demonstrated the differences between DirectX 11 processing power and DX10, mainly by means of offloading much of the computing power from the CPU to the GPU.

Demos of moving thousands of "star" objects simultaneously in a simulated galaxy formation, with gravity and physical forces between them, all in a 40+ Gflops operation running on a $400 graphics card rather than a $15,000 computer.

This year if you did not have the opportunity to join the Microsoft PDC event personally then I hope that these notes will help you. Also, Keynotes were streamed live both on November 17th and November 18th. If you would like to watch them on-demand they are being made available on the www.microsoftpdc.com site soon. One last thing: if you haven’t watched the keynote, take the time to watch it soon because I promise, it’s worth it. As Silverlight is one of my focus points I can recommend you to watch the Silverlight 4 part at the very least.

Also, credit to the following post where Tim has as provided a comprehensive blog on Silverlight 4.0 features:

http://timheuer.com/blog/archive/2009/11/18/whats-new-in-silverlight-4-complete-guide-new-features.aspx

Hope that you have enjoyed this post!

Rajan Dwivedi

PDC2009 @ LA – Day 1 Coverage – Keynotes and Announcements Summary

2009-11-18T09:42:40Z

Microsoft kicked off Professional Developer Conference (PDC) 2009 with a keynote by Ray Ozzie and Bob Muglia.

· Ray started with the theme for "Three Screens and a Cloud”, with focus on the back end today.

· Expect news on client software (Silverlight and IE) to be covered tomorrow i.e. Nov-18, 2009.

You can visit http://microsoftpdc.com for continuing coverage from the Channel 9 team as well as for Day 2 Keynote featuring Scott Guthrie.

· Starting from 10:30AM PST each day the Channel 9 team will interview conference presenters, technical leaders, industry luminaries, partners and the Channel 9 community. If you or your customers are unable to attend PDC09 in person, please tune in to the live smooth-streaming broadcast (powered by IIS7 and Silverlight) by visiting http://microsoftpdc.com.

Lots of big news and interesting announcements but the big word is “Azure” and "Fabric". First of all, the idea of a fabric for running distributed applications is not new, but its applicability to better explaining the vision of running apps on premises and off premises using a consistent framework is new and is a good step forward. Here are the highlights of the morning's key announcements for Day 1 (Nov 17, 2009), with links to more information in several areas.

· Windows Azure - Microsoft's Cloud Computing platform was officially launched. It will be available for production use beginning January 1, 2010.

· Microsoft Pinpoint - A new online marketplace for Microsoft partners to offer applications.

· Microsoft Codename “Dallas” - A place where developers can find and license commercial and public data from the cloud.

· Windows Server AppFabric - A set of application server technologies including formerly code-named "Dublin" and “Velocity”. These features will simplify and enhance the use of Windows Workflow Foundation (WF) and Windows Communication Foundation (WCF).

· ASP.NET MVC 2 - The first beta release of MVC v2 was announced and made available to developers. – Download from http://asp.net/mvc.

The following are the details:

Windows Azure

· Windows Azure Launch Date: Windows Azure will officially launch 01/01/10 (January 1, 2010) and, as a little sweetener for New Year, the 1st month will be free.

Nov 17 2009 - Feature complete
Jan 1, 2010 - Production
Feb 1, 2010 - Customer billing begins

· Features that go live today:

Single Sign-on
Project templates out-of-box in Visual Studio 2010
Multiple sizes of VMs
FastCGI (Enabling extremely flexible binding with Zend Framework, MySQL, Java, php, eclipse)

· Entity Group Transactions added to Azure tables, integrated content delivery network

· Support for pushing popular downloads to the edge.

· XDrive - Azure Storage blobs that are mountable as NTFS drives

Landing Site: https://www.microsoft.com/windowsazure/

SQL Azure

· A true database as a service, simply create a new database when you need a new one (don’t need to think about memory, disaster recovery, etc. It is all automatic)

· Support for ADO.NET, ODBC, JDBC

· Cloud-based authentication service for cloud-leveraged database in SQL Azure, through SQL Server Management Studio.

Landing Site: https://www.microsoft.com/windowsazure/developers/sqlazure/

Customers that Go Live today!

Select customers are taken into production starting today, some are

· WordPress running on Azure - Matt Mullenweg, founder of WordPress (parent company Automatic), shares the story with MySQL and Apache on Windows Azure.

· Kelly Blue Book, running on Azure - Kelley Blue Book demonstrates a Silverlight app for an online car finder. Needs a second data center for failover, "only using it a couple of hours per week," and the cloud model provides a cost-effective, flexible solution.

· OddlySpecific.com (team from http://www.ICanHazCheeseburger.com), launching today; running on SQL Azure and Windows Azure. When it gets popular, Azure specifications can be modified instantaneously.

Microsoft Project Sydney

Announced today Microsoft Project Sydney will enable you to connect existing in-house/on-premise services with those running in the cloud on Azure-based instances. It also allows you to connect direct to on-premise SQL Server from cloud roles. The goal is to make Azure an integral part of corporate/IT environments.

Microsoft Pinpoint

Announcing Microsoft Pinpoint, integrated into Azure Developer Portal, Microsoft Partner Network, in the midst of integrating into Microsoft Portal for IT. The idea here being a sharing of customers searching for services, and searches searching for customers.

"It's difficult for me to overstate the importance of common catalogs...a place where anyone can see some stunning network effects for the things found to be most popular to the community. Online catalogs aren't just about apps, they can be places to discover the most popular and useful data." – Ray Ozzie

Landing Site: http://www.pinpoint.com/en-US/

Microsoft Project Dallas

A CTP of a "game-changing new subsystem" code-named Dallas, an open catalog and marketplace for commercial and public data, with a uniform discovery mechanism, binding mechanism, a uniform licensing model for easy joining and recombining.

"By delivering data as a service, our aspirations are that Dallas could catalyze a whole new wave of experimentation by developers."

It serves as an “information and brokerage service” designed to allow access to commercial and reference data from various entities. The following are among the data providers in the early going:

· Associated Press

· NASA

· National Geographic

· UN

The aim is to create Data As A Service (DAAS) and let data become a commodity that catalyzes a whole new way of remixing and presenting data. There was a public data demonstration from NASA that included 3D images of landscapes taken by the Mars Rovers. Everyone used 3D glasses during the demo.

Dallas data feeds will be discovered through Pinpoint. For example, infoUSA has interesting data about businesses, some of it shows reviews by users.

Key benefits includes:

· Taking the friction out of discovering, exploring and using data

· Subscriptions to data consumption

· Explore the dataset, REST-based data consumption

· Import into Excel through PowerPivot

· OpenData protocol (ADO.NET Data Services = Open Data); Build a service proxy from Dallas

· Apps and innovation around public data

· Eliminate inconsistency in data formats (CSV, XML, etc.); - “Dallas” brings consistency to the data format

Landing Site: http://www.pinpoint.com/en-US/Dallas

Windows Server AppFabric

Announced the release of a new application server called Windows Server AppFabric that goes into Beta 1 today. It is a set of integrated technologies that make it easier to build, scale and manage web and composite applications that run on IIS. It has 3 main parts:

· Caching

· Workflow Management

· Service Management

Muglia announced it as a "platform for building scale-out, high-tier services" enabling developers to concentrate on core functionality, fielding out the failover part of the operation to Microsoft. Database cache is kept entirely in the cloud.

· a set of integrated, high-level application services that enable developers to more easily deploy and manage applications spanning both server and cloud

· a platform for building scale-out middle-tier services such as WCF, WF, database cache, automatic failover and scale-out

· underlying technologies include “Dublin” for hosting and “Velocity” for caching – the combination provides “high-speed access, scale, and high availability to application data”.

· .NET Services is now referred as “AppFabric Service Bus” and “AppFabric Access Control”

You can see more info and some AppFabric samples here:

http://msdn.microsoft.com/en-gb/windowsserver/ee695849.aspx

Tailspin Demo

This morning, in Bob Muglia’s keynote at the PDC, Cameron Skinner presented a demo focused on some of the most exciting new features coming in Visual Studio 2010, .NET Framework 4, and the server platform. The demo was built to align with the following release announcements: ASP.NET MVC 2 beta, Windows Identity Foundation RTM, and the beta of the new Windows Server AppFabric.

Do check it out, it is a great way of familiarizing with the new features in VS2010 and our new server wave! You can now download the Tailspin Travel application from Codeplex.

http://tailspintravel.codeplex.com

Miscellaneous Topics

Vivek Kundra speaks live from Washington

Vivek Kundra, speaks live from Washington. He invited attendees to look at http://beamartian.jpl.nasa.gov to see the public data collected by the Jet Propulsion Laboratory -- also over 100,000 data sets from the EPA on toxic waste.

Vivek Kundra also demonstrated the Azure-based mobile job finder application this morning running on an iPhone. It enables job seekers to use full data center contents from Microsoft's back end, to find jobs sorted by category, and also by seekers' personal GPS locations. "Data applications for the good of the republic," says Kundra.

C++ and Azure

Developer crowd favorite Don Box demonstrated low-level programming techniques (including assembler macros) that leverage Azure services. C++ demonstration (not C#) to show how standardized this can be.

Seesmic.com

Loic LeMeur, Founder & CEO of Seesmic, took the stage. He demonstrated a Silverlight-based application which collects together feeds from individual users in social networks, plus RSS feeds, plus Twitter feeds..

· Shipping today Seesmic for Windows - very smooth, very fast!

· Taking advantage of Win7; sensor for location

· It is #1 request, Seesmic for Windows will be a platform for plug-in development

· Community needs Seesmic to be available on every device

· It will ship Silverlight version soon

Funny Video

The job of giving identity to the cloud concept has been left to Bob Muglia, whose annual funny video this morning wasn't too far off the mark from reality. In the video, he is serving as a "life coach" for the cloud (a guy dressed in a puffed-up pillow costume), who is suffering an identity crisis and he acts as a kind of personal, spiritual counselor for "the cloud". Here "The Cloud" is trying to find a purpose for itself, and Muglia's advice is that the cloud can be anything and everything it wants to be...whatever that is. Then The Cloud takes Muglia's advice.

Windows Live

Unlike last PDC, PDC2009 Day 1- there was no news on Windows Live. You may have to wait for MIX '10 in Las Vegas.

Articles and Resources

Here is the press releases from today’s announcements:

· Microsoft Cloud Services Vision Becomes Reality With Launch of Windows Azure Platform

· From the Cloud to the Crowd: NASA and Microsoft Ask Citizen Scientists to “Be a Martian”

· On-Premises or in the Cloud, One Consistent Composite Application Experience for Developers

More New Downloads & Resources

· SQL Server Modeling CTP Nov 2009 (Formerly Codename "Oslo")

· Sync Framework PowerPack for SQL Azure

· SQL Server 2008 R2 November CTP Reporting Services Add-in for Microsoft SharePoint Technologies 2010

· Windows Azure Platform Training Course

Hope it helps if you could not attend or watch/track the developments at PDC2009 in LA.

Choosing a Collection Class

2009-10-29T23:30:40Z

I see quite often customer asking what is the right type of collection for specific purpose and performance overhead versus flexibility. This is again a passionate topic and it’s easy to get into debate with your colleagues especially during the peer code review processes.

Choosing the right Collections class must be done carefully. Using the wrong collection can unnecessarily restrict how you use it.

Consider the following questions:

Do you need temporary storage?

If yes, consider Queue or Stack.
If no, consider the other collections.

Do you need to access the elements in a certain order, such as first-in-first-out, last-in-first-out, or randomly?

The Queue offers first-in, first-out access.
The Stack offers last-in, first-out access.
The rest of the collections offer random access.

Do you need to access each element by index?

ArrayList and StringCollection offer access to their elements by the zero-based index.
Hashtable, SortedList, ListDictionary, and StringDictionary offer access to their elements by the key of the element.
NameObjectCollectionBase and NameValueCollection offer access to their elements either by the zero-based index or by the key of the element.

Will each element contain just one value or a key-singlevalue pair or a key-multiplevalues combination?

One value: Use any of the collections based on IList.
Key-singlevalue pair: Use any of the collections based on IDictionary.
Key-multiplevalues combination: Consider using or deriving from the NameValueCollection class in the Collections.Specialized namespace.

Do you need to sort the elements differently from how they were entered?

Hashtable sorts the elements by the hash code of the key.
SortedList sorts the elements by the key, based on an IComparer implementation.
ArrayList provides a Sort method that takes an IComparer implementation as a parameter.

Do you need fast searches and retrieval of information?

ListDictionary is faster than Hashtable for small collections of ten items or less.

Do you need collections that accept only strings?

StringCollection (based on IList) and StringDictionary (based on IDictionary) are in the Collections.Specialized namespace.
Generic List<string>

Hope this is useful to you when you make the choice about the collections!

.NET - Domain Neutral Assemblies

2009-05-06T02:25:43Z

Recently few of my colleagues enquired about Domain Neutral Assemblies esp. when to build and implement them due to lack clarity around them. I did research and thought to share here so that other folks can discover the information too.

Here is some background before we get into the topic. An AppDomain is a unit of code isolation. You must load an assembly into an application domain before you can execute the code it contains. Running a typical application causes several assemblies to be loaded into an application domain. The way an assembly is loaded determines whether its just-in-time (JIT) compiled code can be shared by multiple application domains in the process, and whether the assembly can be unloaded from the process.

By definition, a domain neutral assembly is an assembly that lives across multiple app-domains. Essentially the host hints CLR that assemblies can be domain neutral. Hence, CLR will share those assemblies as much as possible across multiple app-domains after jitted only once. The jitted code, as well as various runtime data structures (used to represent the assembly in memory) like MethodTables, MethodDescs, will be shared across app-domains. In many ways, this sharing of runtime data is analogous to the way the operating system shares static code pages for DLLs that are loaded by multiple processes. This cuts down unnecessary duplication. However we still get some "duplication" since the data segments are duplicated in order to provide the appearance to our code that the AppDomains have each got their own copy of the assembly. When we load an AppDomain as domain-neutral, all of it's referenced assemblies have to be loaded domain-neutral too. This set of assemblies is referred to as an assemblies binding closure. Well, if the binding closures are identical, then all is well and good. However, if the binding closures are different, the CLR will create two copies of our domain-neutral assembly in SharedDomain. Any future AppDomains using this shared assembly will have their binding closures evaluated, and will get a reference to the appropriate copy (or a new copy made).

Domain neutral assemblies are stored in a shared area generally referred as SharedDomain. SharedDomain is simply a repository for domain neutral assemblies. There is no code execution in SharedDomain. Code in domain neutral assemblies is executed in user app-domains. Domain neutral assemblies cannot directly access domain bound assemblies. This constraint is enforced automatically by CLR. Please also make sure you understand that domain neutral assemblies cannot be unloaded, even though all the app-domains using those assemblies have been unloaded. They're never unloaded because the SharedDomain only unloads on process exit.

The runtime host determines whether to load assemblies as domain-neutral when it loads the runtime into a process. There are two ways for a host to specify this hint. One way is through hosting API CorBindToRuntimeEx. Another way is decorating your Main method with LoaderOptimizationAttribute. For managed applications, apply the LoaderOptimizationAttribute attribute to the entry-point method for the process, and specify a value from the associated LoaderOptimization enumeration. For unmanaged applications that host the common language runtime, specify the appropriate flag when you call the CorBindToRuntimeEx Function method. Please note that an assembly can be domain neutral if and only if it is in GAC, and all the assemblies in its transitive binding closure are all in GAC.

Domain Neutral Assemlies help in performance gain in multi app-domains scenario in middle tier server application scenarios. If an assembly is loaded domain-neutral, all application domains that share the same security grant set can share the same JIT-compiled code (but not its data), which reduces the memory required by the application. FYI, access to static data and methods is slower for domain-neutral assemblies because of the need to isolate assemblies. The extra internal logic slows down the call. Hence, the performance of a domain-neutral assembly is slower if that assembly contains relatively higher amount of static data or static methods that are accessed frequently. When you decide whether to load assemblies as domain-neutral, you must make a tradeoff between reducing memory use and other performance factors.

Some useful links:

Hope this helps.

Great Debate: Store Procedure vs. Dynamic SQL

2009-04-28T23:54:43Z

This is one of those topics that always generate a lot of heat among developers, architects and DBAs etc. People typically take a very strong stance on one approach over the other.

As most of the IT situations, the answer to this question is : “It Depends”. It really depends upon the type of your application and what the application is supposed to do. We are only talking about OLTP applications here…for BI applications where heavy data churning is required, it is always advisable to keep the code in the DB layer to use SET based logic to do your processing logic or use an ETLM tool for the processing logic.

It depends on whether you are doing things dynamically (lots of ad-hoc queries really aren’t a good match for sprocs), it depends on how your org is run (do you have locked down DBs and strong DBAs, or do the “enterprise architects” with an interest in ORMs who win every argument). It depends on whether the app is read-heavy or write-heavy, on whether the DB is strict 3rd normal or de-normalized for reporting reasons. It depends on whether the DB schema makes any sense at all to devs (was it created for this application, or is it some hodge-podge that was created 15 years ago and is now used for 200 different reasons). It depends on a lot of things.

Store Procedure Advantages

Although you can do most of the things a stored procedure can do with simple ad hoc Transact-SQL statements, stored procedures have a number of advantages over ad hoc queries, including

Execution plan retention and reuse
Query auto-parameterization
Encapsulation of business rules and policies
Application modularization
Sharing of application logic between applications
Access to database objects that is both secure and uniform
Consistent, safe data modification
Network bandwidth conservation
Support for automatic execution at system start-up
ENCRYPTION option

Needless to say secondary benefits include:

Syntax checking (pseudo-compiled i.e. a kind of normalized query tree as it saves SQL resources)
Object dependency model
Schema buckets
Consistent monitoring
Permissions and Limitations
Ability to view exec plans
You can use sp_usage to generate stored procedure usage info
Set environmental options (e.g., NOCOUNT, LOCK_TIMEOUT, user options, @@NESTLEVEL, @@TRANCOUNT, SET XACT_ABORT, SET CURSOR_CLOSE_ON_COMMIT, SET TEXTSIZE, SET IMPLICIT_TRANSACTIONS etc.) and Built-In protection such as max allowed nested is 32 levels deep
Dash board reports
Temporary procedures (# for current connection, ## for global)
Replace your complex SQL statements with a single stored procedure

Strategic Advantage

Clear choice between Top Heavy vs. Bottom Heavy Architecture
Migration of core business logic and IP is easy as SQL has not changed much over decades vs. programming languages (COM to .Net). How many apps still need to be completely re-written from VB6 to VB.NET?
Intentional use of indexes to cover the query
Proven, almost no learning curve (compare to new technologies like LINQ/ADO.NET EF)
No exposure of database design schema in applications (encapsulation)
Bandwidth savings!! No SQL over the wire repeatedly.
DBA Control and Optimizations (Throttling and saving resources, Increased data security and integrity: One can secure the tables for direct access and only grant privileges on the stored procedures/packages.
By using bind variables (parameterized queries), SQL injection mitigate
Single Vendor DB

Let me elaborate on some key points:

Network Bandwidth

As the procedures/functions are stored in the database there is no need to transfer the code from the clients to the database server or to transfer intermediate results from the server to the clients. This results in much less network traffic and again improves scalability.

Assume that the application server(s) and the database server(s) are separate servers. Since the source code is actually stored on the database server, the application only needs to send the name and the parameters for executing it and this in turn reduces the amount of data that is sent to the server. When you use embedded SQL or dynamically generated SQL through an ORM, then the full source of commands must be transmitted and in a scenario where there is a lot of concurrent activity going on and the load on the system requires a lot of users, this can very soon become a performance bottleneck. This can be mitigate in part by a judicious usage of views as well.

Abstraction Layer (Encapsulation)

SPs help in separating the business logic from data manipulation logic. Since the interface to the application remains the same, changes done internally to the stored procedures/packages do not affect the application and in turn leads to easy deployment of changes.

Unit of Work (Batching)

It offers simplified batching of the commands. Since stored procedures/packages are meant to do a “Unit of Work”, this procedural logic can be simple to maintain and offers additional advantages like making use of the rich feature functionality that the database engines provide. SQL is a SET based language and using SET based procedural logic is the easiest and most performant way of dealing with the data. With every new release of Oracle, SQL Server or DB2 LUW, new features are being introduced in PL/SQL, T-SQL and/or SQL/PL which makes handling of different types of requirements very easy in the DB layer code.

Adhoc Work (Temporary SPs)

Temporary procedures are useful when you want to combine the advantages of using stored procedures such as execution plan reuse and improved error handling with the advantages of ad hoc code. Because you can build and execute a temporary stored procedure at run-time, you get the best of both worlds. For the most part, sp_executesql can alleviate the necessity for temporary procedures, but they're still nice to have around when your needs exceed the capabilities of sp_executesql.

Data-Centric Tracking

Code Instrumentation and tracing logic can be built in very easily using the stored procedures. This is one thing that we implemented for one of our clients recently. We created a table which had a list of the DB code that was being used in the schema and this table had a trace_flag column in it which could have 4 different values:

0 (no code instrumentation),
1 (log the sequence of events),
2 ( log the sequence of events and the time taken by those SQL statements),
3 ( log the sequence of events + the time taken + the execution plan from that point of time - since the execution plan can very easily be different at the time of execution under a load scenario vs when you actually run it separately)
4 (Start the trace - example: starting 10046 level 12 trace in the case of Oracle).

Using this logic, code instrumentation and troubleshooting production issues became very easy. One could then run reports against the data that was logged and present it to the end user or the support personnel. Code instrumentation can be done in the application tier as well using the same guidelines (or using logging blocks like MSFT logging block in .Net) and a good programmer would always instrument their code. However, for the DB code, this code instrumentation becomes a lot more easier to implement.

Store Procedure Challenges

Applications that involve extensive business logic and processing could place an excessive load on the server if the logic was implemented entirely in stored procedures. Examples of this type of processing include data transfers, data traversals, data transformations and intensive computational operations. You should move this type of processing to business process or data access logic components, which are a more scalable resource than your database server.

Maintenance and the agility of your application becomes an issue when you must modify business logic in T-SQL. For example, ISV applications that support multiple RDBMS should not need to maintain separate stored procedures for each system. If your application runs on multiple RDBMS, example: You are a vendor and you need to provide your product that runs on Oracle, SQL Server and DB2 LUW in order to expand your customer base, then in that scenario, you have to code or put fixes in for three different code bases. Not only that, you need to have proper staffing to ensure that the code written is optimal since the locking and concurrency mechanisms are very different between these RDBMS. Also, the language used by all these “big 3″ is very different as well.

Writing and maintaining stored procedures is most often a specialized skill set that not all developers possess. This situation may introduce bottlenecks in the project development schedule. We have seen client shops which offload all of their CRUD operations on the DB tier - as a result they end up with one-liner stored procedures and if you have say 200 objects, you now have 200 * 4 (select/insert/update/delete) stored procedures or one procedure per object with the flag option to indicate the operation and need to code the procedure to use dynamic SQL in order to take into account the conditional parameter logic. Maintenance becomes a nightmare in this case. Also, developers/DBAs sometimes get carried away with this approach and forget that SQL is a set based language - one example is that using this scheme, a client shop was doing purges of the data and kept calling the delete procedure by passing in the ID (surrogate Key) value when they actually could have purged and archived the data using a SET based logic. Procedures are supposed to do unit of work - having one liner procedures and invoking that many calls in a batch does not yield any benefit. In addition, it has to un-necessarily incur the cost of checking permissions and plan associated with that one liner procedure - the cost is albeit a very miniscule one.

Parsing of strings is not one of the forte’s of the DB languages - that code is better suited in the application tier unless you start using CLR or Java code.

Another thing to remember is when people point out that an advantage of stored procedures is that the code is always pre-compiled, that is not always true, and there can be scenarios that can lead to re-compilation. Also, if proper bind variables are being used for the queries built using an ORM, it serves the same purpose (i.e. has the same advantage as that of a parsed/compiled stored procedure query) since the plan for that SQL statement is parsed and compiled. Depending upon a lot of factors (cache sizes, concurrent load, statistics updates etc.), that plan may or may not be available the next time the call gets made.

Needless to say Versioning issues, Max Size 128MB, Max Parameters 102, Lack of Documentation - Naming Convention and Headers.

Before we wind up, let us also review compilation mechanism as people assume many things here.

First Time Execution Mechanics

When you execute a stored procedure for the first time, it's compiled into an execution plan. This plan is not compiled into machine code or even byte codes, but is pseudo-compiled in order to speed execution. "Pseudo-compiled" means that object references are resolved, join strategies and indexing selections are made, and an efficient plan for executing the work that the procedure is to carry out is rendered by the SQL Server query optimizer. The optimizer compares a number of potential plans for performing the procedure's work and selects the one it thinks will cost the least in terms of total execution time. It bases this decision on a number of factors, including the estimated I/O cost associated with each plan, the CPU cost, the memory requirements, and so on.

Conclusion

So, the bottom line is that one needs to carefully evaluate which portion(s) of the application really belong as stored procedure/packages. In applications that work on volume of data and do bulk data processing, it is always advisable to have that logic in stored procedures/packages that reside on the DB layer so that one can take advantage of the SET based approach of the SQL language.

One can also use ORMs (Object Relational Mappers) to prepare their data access strategy - one can then extend it to make calls to the database stored procedure (if so desired in certain situations), have externalized SQL statements, have dynamic SQL statements being prepared by the ORM etc.. One just has to make the right decision depending upon the application and the requirements that are needed.

A good mix of an ORM plus DB code is usually the best compromise to get the best of both the worlds. Deciding what goes where is the key and though there are guidelines on that, it can vary from application to application depending upon the requirements.

Data Strategy based on ADO.NET 3.5

2009-04-15T18:09:00Z

Often most of my customer seek guidance on data strategy based on ADO.Net technologies stack such as ADO.NET DAL, LINQ and Entity Framework and the Data Services.

The choices for developers/architects are:

1) Traditional ADO.Net Provider Model

Reduces OR mapping impedance mismatch through (typed) dataset
Tightly coupled with store/RDBMS (Like name of table columns and SP Parameters)
Sweet Spot: Hand coded Data Access Layer (DAL)/ Folks who don’t like ORMs (or Dynamic SQL within App Tiers)

2) LINQ to SQL

Suitable for TPH model (Table per Hierarchy)
Provides abstraction and compile time checks
Sweet Spot: RAD/Prototype/Proof-of-Concept (PoC)
Limitation: No further advancement planned

3) Entity SQL

Dynamic query creation
Independence from storage/RDBMS
ESQL based on underlying Entity Data Model
Limits: Read Only, Text-Based, Reader based (EntityDataReader)
Sweet Spot – use it when you have multiple databases, leverage your SQL like skills

4) LINQ to Entities

Supports TPH, TPT (Table per Type), TPCT (Concrete Types) inheritance and O/R mappings
Strongly typed, compile time verification, Intelligence
Uses Entity Client Provider under the covers but returns collection (IEnumerable)
Powerful LINQ lambda expressions, operators and dynamic criteria
Sweet Spot: LINQ and Dynamic SQL lovers

5) Object Services (Entity Framework)

Full CRUD
Choice of Entity SQL, LINQ or EDMContext
Fully materialized and strongly typed objects (Entity Type and Entity Sets aka Containers)
Sweet Spot: No Data Access Layer required, First Class Entities, Heavy-Lifting by Entity Framework
Limits: Thin Object Layer required to process basic entities as per business logic, transaction and concurrency management challenges, SQL hints, Diagnostics is difficult (like source of SQL Query)

5) Data Services (Astoria)

Ability to consume EDM or LINQ-to-SQL models
Any Store + (interface to support update semantics for ADO.NET Data Services) implementation
Provides platform for exposing data over HTTP as a service for various clients such as AJAX/Silverlight/.Net and non-MSFT applications
REST Style resource model for full CRUD
Uses minimalistic formats to represent data, and supports AtomPub/JSON format to accommodate as many clients.
Extensibility through pipeline (Query and Change Interceptors) and Service Operations
Granular security at entity level
IIS based security for service level such as SSL and Auth/Z

Store Procs can be imported as Functions and can be invoked over the model directly (both LINQ to SQL and EF supports it).

It’s perfectly fine to have a mixed strategy that addresses your needs in an existing project. When starting a new project, you can make clear choices such as discarding dataset/adapter based approach.

Hope that this brief summary is useful to you in order to quickly compare various options offered by ADO.NET for data access.

MIX 2009: Day 2 Summary

2009-03-20T17:50:16Z

The keynote on Day# 2 was all about Internet Explorer 8 and passion for design by Deborah Adler about the prescription drug containers that finally adopted by national pharmacy chain (Target).

Dean Hachamovitch, General Manager for the Internet Explorer team, released the final build of IE8.0 (Theme: Faster, Easier and Safer).

We used real-world data to build IE8 – hundreds of millions of users, 200+ data points with the products, millions of user sessions, hundreds of hours of usability labs, and dozens of in-home studies.
Available in 25 languages for Windows Vista, Windows XP, Windows Server 2003, and Windows Server 2008 within x86 and x64 editions.
Performance has been a major area of investment in Internet Explorer 8, as this performance testing video demonstrates. IE8 is faster than other browsers on many real-world sites. Even the JavaScript engine itself is 70% faster than the one shipped in IE7.
In IE8, we now have tab crash isolation – when one tab of the browser fails, it doesn’t bring down the entire browser.
New features includes Full CSS 2.1 Support, accelerators, web slices and provider model for Visual Search.
- Great opportunities for developers for interoperability, extensibility and integrating their sites/tools.
- There are over 1,200 accelerators, web slices and visual search providers out there already today, including Digg, ESPN, OneRiot, Amazon, Sina, TaoBao, and Yahoo.
- Get some at http://ieaddons.com
IE blocks over a million phishing and malware attacks a month. (Check more at: http://www.nsslabs.com/anti-malware/browser-security)
Visit the IE8.0 MSDN Developer Center and the IE8 Team Blog. Watch the great video on internet history for yourself!

Deborah Alder (*Designer’s inspirational story- start video around the middle) - a successful design experience begins by

Having a love affair with your customer and really digging into your customer's needs.
Bringing your design skills to bear in solving those needs humanly and humanely.

More about Deborah at https://content.visitmix.com/2009/speakers/default.aspx#DeanHachamovitch

Regards,

Rajan Dwivedi
Premier Services, Microsoft

MIX2009 Announcements Summary

2009-03-20T00:30:00Z

For those of you that did not get a chance to attend the MIX09 conference, or watch any of the live streams I wanted to provide you with a summary of all the announcements made yesterday morning.

Bill Buxton announced the theme for MIX this year is “Return on Experience,” and that is precisely what he focused on during the opening minutes of keynote.

Scott Guthrie
“The Gu” came on stage after a funny video which featured him dancing, getting his hair fixed and passion for red polo shirt. Major announcements in web space include:

The preview of Expression Blend 3 is available for download today!
SuperPreview is a new tool that is part of Expression Web 3 that will allow web developers to comprehensively test their web pages for cross-browser compatibility.
The ASP.NET MVC 1.0 framework shipped as of today.
A series of improvements for ASP.NET 4.0, including:
- Enhanced web form development
- Integration of ASP.NET MVC and AJAX
- Distributed caching
Enhancements to Visual Studio 2010 for web developers, including:
- New and enhanced tools for JavaScript, AJAX and JQuery development
- SharePoint developers become first class citizens in the IDE with new development tools for MOSS
- New publishing and deployment tools, including the ability to keep multiple web.config files specific to a deployment environment, i.e. development, test, staging and production
Microsoft Web Platform Installer 2 beta provides you with the ability to get all the tools and technologies for developing web applications for the ASP.NET platform in one place.
Microsoft Commerce Server 2009 available today.
Customer-driven enhancements to the Windows Azure Service Platform, including FastCGI, PHP and .NET full trust, SQL Data Services will become a true relational database in the cloud and commitment to ship Windows Azure in 2009!

Silverlight 3.0 highlights:

Silverlight 3 will provide cross-platform support for hardware acceleration.
Silverlight 3 will include the H.264, AAC and MPEG-4 codecs - it will also include a raw bit-stream AV API which will allow developers to create custom codecs in managed code.
Silverlight 3 includes enhanced logging capabilities for managing app analytics (monetization and SEO)
IIS Media Services, as a free add-on, will enable any IIS7 web server to provide smooth AV streaming (key features includes advanced logging, bit-rate throttling and edge caching)
Few of the slides for Silverlight 3 listed the following feature enhancements:
- GPU acceleration and hardware compositing
- Perspective 3D, essentially moving 2D objects in the UI in a 3D space
- An API for bitmap images and pixels
- Shader effects
- Hardware acceleration for Deep Zoom
- “Deep linking,” which is the ability for a user to link to a specific place inside a Silverlight application
- Navigation and search engine optimization
- Clear Type Fonts
- Assemblies/Library caching support
- More than 100+ controls available
Silverlight 3 will ship with the native ability to run outside the browser. This means a big challenge to Adobe AIRsJ
Olympics in Vancouver (2010) will be streamed live using Silverlight platform.
The Silverlight 3 installer package is SMALLER (40K) than the previous version.

Netflix Case Study - Kevin McEntee, VP, Web Engineering shared the stage with ScottGu

Netflix acknowledged moving away from own (proprietary) player to Silverlight. (Great platform for having great video players!!)
Silverlight‘s Adaptive Media streaming works!! (including Mac - Safari and Firefox)
Cross Platform is “a reality” using Silverlight!! (Cross Browser, Cross Platform and On-Devices such as Xbox)
Installers are Evils but Silverlight offers sweet experience!!
Silverlight’s DRM abilities for content protection is great!!

Hope this helps in case you have limited time to watch many hours of streaming to mine golden nuggets :=))

Best Regards,
Rajan Dwivedi
Premier Services, Microsoft