Microsoft project code named "Velocity"

Announcing Windows Server AppFabric Beta2 Release

2010-03-02T04:48:00Z

We are excited to announce that the Beta 2 release of Windows Server AppFabric is now available for download via http://msdn.microsoft.com/appfabric . This is the Windows Server AppFabric build that works with the RC (Release Candidate) build of Visual Studio 2010/.NET 4. Windows Server AppFabric is a set of application services focused on improving the speed, scale, and management of Web, Composite, and Enterprise applications. This release represents our “feature complete” milestone. That is, it contains all the features that we plan to ship in Windows Server AppFabric v1 by Q3 of 2010

Some of the new AppFabric "Caching" features added in this release are –

Added PowerShell cmdlets for configuring cache clusters. The commands are New-CacheCluster, Add-CacheHost, Get-CacheClusterInfo, Register-CacheHost, Remove-CacheCluster, Remove-CacheHost, and Unregister-CacheHost. These commands give you granular control of cluster deployment.
The Cache service now throttles when it is under memory pressure. When it is in the throttling state the service will allow all “read” and “delete” requests but will reject all “write” and “update” operations. This prevents the service from engaging in excessive paging. Throttling events get logged to the Event Log and allow administrators to free available memory, add nodes to the cluster, or change the expiration or deletion policy of cached items, for example.
The Cache service now logs information to two ETW channels. The Admin channel is enabled by default and collects all events like Start Cache Service and Stop Cache Service and any unhandled exceptions. The Operational channel is disabled by default. When it is enabled it captures all warnings from the Cache service. The Event Viewer shows the collected data under Applications and Services Logs -> Microsoft -> Windows -> Application Server – System Services.

Also checkout http://blogs.msdn.com/endpoint/archive/2010/02/26/windows-server-appfabric-beta-2-available.aspx for more on Hosting and Service Management features in this release.

We are excited about our Beta 2 release and would love to hear what you think.

Thanks

Windows AppFabric Caching Team

PDC talk

2009-12-04T02:14:00Z

http://microsoftpdc.com/Sessions/FT26

My PDC talk on AppFabric Caching

Announcing "Windows Server AppFabric"

2009-11-18T04:10:00Z

At PDC 2009 we have announced roadmap and details on release of Microsoft project codenamed "Velocity" under Windows Server AppFabric, to download latest bits and more details please check out following links.

On-Premises or in the Cloud, One Consistent Composite Application Experience for Developers

http://www.microsoft.com/presspass/features/2009/nov09/11-17PDCAppFabric.mspx?rss_fdn=Top%20Stories

Windows Server AppFabric

http://msdn.microsoft.com/en-us/windowsserver/ee695849.aspx

Thanks

Microsoft project codenamed "Velocity" Team

Griddynamics benchmark on Velocity

2009-09-30T21:11:00Z

Benchmark showcases Microsoft’s Project Code-named “Velocity” ability to Scale Web Application

Grid Dynamics, a Microsoft partner, created three sample applications, designed to be typical use cases for data caching, and ran extensive benchmarking tests to evaluate their performance characteristics. The applications were a blogging engine, demonstrating the most basic features of Velocity caching technology; a simple e-commerce website, demonstrating Velocity’s capabilities for managing session state, and; a market data application, demonstrating Velocity’s event processing capabilities. Building these applications with the Velocity project’s in-memory application cache platform achieved superior, linear scalability when compared to the same applications built without Velocity.

Click Here to download the full White Paper

About Grid Dynamics

Grid Dynamics is the global leader in scaling mission-critical systems. Our .NET Scalability Practice offers solutions that enable customers to develop and scale applications with the Velocity caching platform, Windows HPC Server 2008, the Azure Services Platform, and other Microsoft .NET technologies

Learn more from:

http://griddynamics.com/velocity.html

http://griddynamics.com/partners/microsoft.html

PowerShell Scripting for Microsoft project code named "Velocity"

2009-06-29T15:22:00Z

Windows PowerShell allows you to invoke C# code directly from the PowerShell scripts. This offers a powerful way to “script” cache operations such as puts/gets etc. This can be used to prime the cache as part of an automated setup or to perform other management tasks such as exploring the cache etc. This blog shows some examples on how you can invoke the "Velocity" APIs directly from PowerShell scripts. Note that I will use product and "Velocity" interchangeably .

Let us see how we can do it step by step in this write up. Windows PowerShell let's you create .NET objects, you can create object with new-object cmdlet like this

[System.Random]$rand =new-object System.Random

Before creating any object make sure that the assembly for that class is loaded by PowerShell.

You can check the assemblies loaded by PowerShell by running this on PowerShell.

[System.Threading.Thread]::GetDomain().GetAssemblies()

By default, PowerShell loads a bunch of assemblies for you so that you can start working with the base types, and some common functions. If you want to use the functionality in an assembly that is not already loaded, you will have to load that assembly yourself.

you can load by doing something like this on PowerShell prompt

[Reflection.Assembly]::LoadFrom("CacheBaseLibrary, Version=1.0.0.0, " `

+ "Culture=neutral, PublicKeyToken=b77a5c561934e089")

It will be good idea to use PowerShell-based cache administration tool which will be installed as part of product installation. Let us now see how we can create DataCacheFactory object in a PowerShell Script.

[Microsoft.Data.Caching.DataCacheServerEndpoint ]$server =new-object

Microsoft.Data.Caching.DataCacheServerEndpoint("machineName", 22233, "DistributedCacheService")

[ Microsoft.Data.Caching.DataCacheServerEndpoint[]]$serverarray = new-object Microsoft.Data.Caching.DataCacheServerEndpoint[] 1

$serverarray[0]=$server

[Microsoft.Data.Caching.DataCacheFactory ]$cacheFactory = new-object Microsoft.Data.Caching.DataCacheFactory($serverarray, $true, $false)

Once we have created this DataCachefactory we can use it as we do it in any C# client, what I mean is that you can call GetCache, GetDefaultCache etc. on this DataCacheFactory. In the following code

I am creating a cache with a random name and going the use it with the DataCacheFactory created above by calling GetCache on it. Creating a region and putting a key and value in it. At last a get

call is getting the value for key we inserted by calling get on cache we created, after that we are testing the value for its correctness.

[System.random]$rand =new-object System.random

$cacheName = $rand.next(100,134)

#Creating a Named cache in product cluster

New-Cache -CacheName $cacheName -NotExpirable true -Eviction LRU -TTL 1400 -Secondaries 0 -NotificationsEnabled true -Verbose

start-sleep -seconds 60

[Microsoft.Data.Caching.DataCache]$cache= $cacheFactory.GetCache($cacheName)

$regionName = $rand.next(22,127)

$cache.CreateRegion($regionName, $true)

start-sleep -milliseconds 15000

$key=$rand.next(45,292)

$value=$rand.next(83,120)

$cache.put($key, $value, $regionName)

$item = $cache.get($key, $regionName)

# Testing the value we got is what we put

if ($value -eq $item)

{

write-output("Test Passed")

}

In the above code snippet you can see that is helping you in many ways. Firstly you are able to mix cache administration cmdlets with the client code, secondly it is a script and you get all powers of script here. We will see how a Cache Administrator can solve some problems that he he/she may face.

1) Strating a product Cluster, Creating a Cache, Creating some regions and filling some data before giving it for other use.

# Starting Cluster

Start-CacheCluster

start-sleep -seconds 15

$errRegionAlreadyExists = [Microsoft.Data.Caching.DataCacheErrorCode]::RegionAlreadyExists

# Random Class I will use it to create random values for cachename, region name, keys and values

[System.random]$rand =new-object System.random

$cacheName = $rand.next(1,999)

#Creating a new cache using a cache administration cmdlet

write-output ($cacheName )

New-Cache -CacheName $cacheName -NotExpirable true -Eviction LRU -TTL 1400 -Secondaries 0 -NotificationsEnabled true -Verbose

start-sleep -seconds 60

# Product Exception handling block, this will handle all product exception in this script

trap [Microsoft.Data.Caching.DataCacheException]

{

write-error $("TRAPPED: " + $_.Exception.Message);

write-error $("TRAPPED: " + $_.Exception.ErrorCode)

if($_.Exception.ErrorCode -eq $errRegionAlreadyExists)

{

$regionName = $rand.next(1,200)

$cache.CreateRegion($regionName, $true)

}

if($_.Exception.ErrorCode -eq 12011)

{

exit

}

continue;

}

# Creating Cache Factory with one DataCacheServerEndpoint

[Microsoft.Data.Caching.DataCacheServerEndpoint]$server =

new-object Microsoft.Data.Caching.DataCacheServerEndpoint("machinename", 22233, "DistributedCacheService")

# This code Microsoft.Data.Caching.DataCacheServerEndpoint[] 1 creates a array of one

[Microsoft.Data.Caching.DataCacheServerEndpoint[]]$serverarray = new-object Microsoft.Data.Caching.DataCacheServerEndpoint[] 1

$serverarray[0]=$server

[Microsoft.Data.Caching.DataCacheFactory] $cacheFactory = new-object Microsoft.Data.Caching.DataCacheFactory($serverarray, $true, $false)

# This block will handle any kind of exception in this script if it not of product type

trap [Exception]

{

write-error $("TRAPPED: " + $_.Exception.GetType().FullName);

write-error $("TRAPPED: " + $_.Exception.Message);

write-error $("TRAPPED: " + $_.Exception.StackTrace);

write-error $("TRAPPED: " + $_.Exception.Source);

write-error $("TRAPPED: " + $_.Exception.TargetSite);

continue;

}

# Getting the cache object in here which we created earlier in this script.

[Microsoft.Data.Caching.DataCache]$cache= $cacheFactory.GetCache($cacheName)

$regionName = $rand.next(22,127)

# Creating region with a random name, which is in this case a integer

$cache.CreateRegion($regionName, $true)

start-sleep -milliseconds 15000

$key=$rand.next(4,292)

$value=$rand.next(8,120)

$cache.add($key, $value, $regionName)

# Putting data in the Region we created above

for ($i=1;$i -le 100;$i+=1)

{

$key=$rand.next(4,292)

$value=$rand.next(8,120)

$cache.put($key, $value, $regionName)

}

In the code above you can see how product exceptions being handled, you can also choose try-catch style in PowerShell Script.

Function TryCatch() {

&{#Try Block

throw (new-object Microsoft.Data.Caching.DataCacheException)

}

#Catch Block

trap [Microsoft.Data.Caching.DataCacheException] {

Write-Host " $_.Exception"

Write-Output "Failure!"

continue

}

References:

http://blogs.msdn.com/powershell/archive/2006/04/25/583273.aspx

http://www.microsoft.com/windowsserver2003/technologies/management/powershell/default.mspx

http://blogs.msdn.com/wriju/archive/2008/07/01/how-to-find-public-key-token-for-a-net-dll-or-assembly.aspx

Thanks,

Abhijeet Bhattacharya

(Microsoft project code named “Velocity” team)

Troubleshooting “Velocity” Series – III (Continued)

2009-06-29T08:18:00Z

How to control ETW sessions and collect traces

In this section I will describe how to work with an ETW sink. ETW stands for “Event Tracing for Windows”. It is a high speed tracing facility provided by the OS. It uses buffering and logging mechanism implemented in the OS kernel. The buffers used are written to the disk. The user can later retrieve these events in a human-readable format. Please follow these steps to subscribe to Microsoft project code named “Velocity” ETW sink.

· For the first time logging metadata has to be registered for the machine. ‘Logs’ folder (under the installation folder: Microsoft Distributed Cache\V1.0\Logs) contains the following files:

Ø Provider.man - manifest file for events for Microsoft Windows Vista and above.

Ø Provider.mof - manifest file for events for downlevel (Microsoft Windows XP, Microsoft Windows Server 2K3).

Ø ProviderUninstall.mof - manifest file to uninstall “Velocity” provider from WMI repository.

Ø ProviderGUID.txt - text file which contains the GUID for “Velocity” session provider and is used internally.

Moving/deleting these files can break the functionality and is not supported.

From command prompt:

Ø Downlevel users would have to do “mofcomp Provider.mof”

Ø Users having Microsoft Windows Vista and above would have to do “wevtutil im Provider.man” (this requires elevation).

· For an ETW sink, logs would go to ETW session and can be retrieved by command line utilities. Sessions can also be controlled in the same way.

· The utilities required are ‘tracelog.exe’ and ‘tracerpt.exe’. These are standard Windows utilities and are publicly available.

· Start collecting logs: first, an ETW session has to be started with a definite log level. Logs generated would be pumped to it. The log level can be changed dynamically.

tracelog -start <sessionName> -f <logFile> -guid ProviderGUID.txt -level <level>

o sessionName: name of the ETW session.

o logFile: file in which the logs would go to.

o level: the desired Log level.

§ 2 - Error

§ 3 - Warning

§ 4 - Information

§ 5 - Verbose

o To disable logging, log level should be given as 1.

· Change log level: it can be changed dynamically by

tracelog -enable <sessionName> -guid ProviderGUID.txt -level <level>

· Stop collecting logs: now, the ETW session needs to be stopped.

tracelog -stop <sessionName>

· Trace dump can be viewed by : tracerpt <logFile> -y

Ø Downlevel users would get a .csv file.

Ø Users having Microsoft Windows Vista would get an .xml file. To get a .csv file, the switch ‘-of CSV’ would have to be added. Command then would be “tracerpt <logFile> -of CSV -y”.

This was all about ETW sink on a cache host. However, if you want this functionality on the client side, you need to copy Provider.man, Provider.mof, ProviderUninstall.mof and ProviderGUID.txt from the ‘Logs’ folder (under the installation folder: Microsoft Distributed Cache\V1.0\Logs) onto the client machine before executing the steps mentioned above.

Thanks,

Amit Kumar Yadav

(Microsoft project code named “Velocity” team)

Troubleshooting “Velocity” Series – III

2009-06-24T15:26:00Z

How to use “Velocity” logging framework

Microsoft project code named "Velocity" provides the ability to trace events on the cache client and cache host. These events are captured by enabling log sinks. A sink represents a valid destination to which logs or events can be emitted. “Velocity” supports three log sinks:

· Console sink - emits events to the console.

· File sink - emits events to a text file.

· ETW sink - emits events to the OS buffers which are written to the disk.

See related sections in the documentation for more details:

Log Sink Settings: http://msdn.microsoft.com/en-us/library/dd187434.aspx

How to: Set Log Sink Levels (XML): http://msdn.microsoft.com/en-us/library/dd169209.aspx

How to: Set Log Sink Levels (Code): http://msdn.microsoft.com/en-us/library/dd187351.aspx

How to change log file generation format

By default file sink emitted log files are generated hourly and use the following file format ‘dd-hh’. Hence you would see log files named with current date and hour and a new file would be generated every hour. For example, DCacheTrace[1371]13-2, DCacheTrace[1371]13-3, DCacheTrace[1371]13-4 and so on.

Now to increase or decrease the granularity of log file generation, you would need to change the above mentioned time format. Say for instance:

· If you need log files per minute, you would want to change it to ‘dd-hh-mm’.

· For a 24-hour format, change it to ‘dd-HH-mm’.

· For AM/PM display, change it to ‘dd-hh-mm tt’.

Now I will list out the steps required to change the time format. Note that these settings are applicable to a single cache host since changes are done locally.

· Open DistributedCache.exe.config (located in the installation folder).

· Go to <fabric> <section name=”logging” path=”” > tag.

· Add the following entry in “sinks” collection.

· The path in ‘sinkParam’ should be taken from ‘location’ attribute in the <log> tag.

· The portion after ‘/’ in ‘sinkParam’ represents the time format. Change it as per your requirements.

· Log level can be specified in ‘defaultLevel’. This represents the same property as ‘logLevel’ in the <log> tag.

· Note that these settings override the ones defined in the <log> tag.

For more details, refer to http://msdn.microsoft.com/en-us/library/dd187434.aspx

In the next post, I will talk about the ETW sink.

Thanks,

Amit Kumar Yadav

(Microsoft project code named “Velocity” team)

Troubleshooting “Velocity” Series - II

2009-06-22T15:43:00Z

How to ensure you have correct client libraries

This section is more important for people who are upgrading from CTP2 to CTP3. Since Microsoft project code named “Velocity” bits are not backward compatible, while upgrading the cache hosts, “Velocity” clients have to be upgraded as well. There are four client libraries:

1. ClientLibrary.dll

2. CacheBaseLibrary.dll

3. CASBase.dll

4. FabricCommon.dll

Even if these are updated, since the client needs references to only ClientLibrary.dll and CacheBaseLibrary.dll, in a scenario where by some chance only these two libraries are updated and the client still has older CASBase.dll and FabricCommon.dll, user would run into issues (has been reported earlier by multiple folks). The fact that file and product versions for the second two libraries are 1.0.0.0 for both CTP2 and CTP3 makes troubleshooting even more difficult. Hence you need to make sure that the correct set of assemblies has been copied by checking that their ‘Date modified’ timestamp is same in both locations (copied from, copied to).

Next time we’d be discussing “Velocity” logging framework usage.

Thanks,

Amit Kumar Yadav

(Microsoft project code named “Velocity” team)

Troubleshooting “Velocity” Series

2009-06-17T15:15:00Z

Hi,

Hope all of you are having great time caching your data with Microsoft project code named “Velocity”. This post is first in the series of some practices and guidelines to follow in order to make best use of “Velocity” and for troubleshooting any common issues you may face. I also recommend that you see the "Velocity" readme file and the product help file for other known issues. You can find the code_name_velocity_readme.txt file on the "Velocity" download page.

How to work with common “Velocity” exceptions

The occurrence of any undesired behavior in “Velocity” cache requests results in DataCacheException being thrown to the client application. So the application using “Velocity” should be capable of catching them, recovering and taking appropriate actions, for instance, retrying a retry-able error or falling back to the persistent data store.

A “Velocity” client application should check for the exception’s ErrorCode field. These are explained in the documentation in detail. There are error codes like DataCacheErrorCode.KeyDoesNotExist, DataCacheErrorCode.RegionDoesNotExist, etc. which point to logical issues and represent definite failures. However, there also exist some error codes which may come due to cluster’s temporal state of re-alignment as a result of a cache host coming up, going down, named cache getting created/deleted etc. These are explained here and should be taken care of.

1. DataCacheErrorCode.RetryLater : this error code represents transient incapability of the cluster to serve a request. Application can retry the request.

For instance, this would be thrown when the cluster is undergoing a configuration change or there are not enough cache hosts in the cluster to perform a Put/Add.

In the next release, this error code would be accompanied by an error sub-status which would tell the exact reason for this failure. For example - primary cache copy not found, secondary cache copy not found to update, etc to name a few.

2. DataCacheErrorCode.Timeout : this error code is thrown when the client doesn’t receive a response from the server within a specified timeout. For instance, this would be thrown when there is a glitch in underlying communication between client and the server.

However, there is a possibility that the request may have been processed by the server, so application logic should take care of that while retrying.

To clarify a bit more, suppose client does

cache.CreateRegion(“Foo”, false);

and gets a Timeout error code. Trying again may result in an exception with code DataCacheErrorCode.RegionAlreadyExists.

If the deployment mode is chosen as ‘Simple’, there is one more error code which the application should take care of - DataCacheErrorCode.CacheServerUnavailable. This is thrown when a “Velocity” client is not able to reach a particular cache host. This may be possible if that server is down or there is some network related failure.

Next in the series is ‘How to ensure you have correct client libraries’.

Thanks,

Amit Kumar Yadav

(Microsoft project code named “Velocity” team)

EF Query caching with Velocity

2009-06-17T00:03:00Z

Jarek has written a blog and a sample of integrating caches into EF. One integration sample is using Velocity CTP3 bits. Take a look at http://blogs.msdn.com/jkowalski/archive/2009/06/11/tracing-and-caching-in-entity-framework-available-on-msdn-code-gallery.aspx

Sample code at http://code.msdn.microsoft.com/EFProviderWrappers