Speaking of which...

Changing file dates and other more uncommon properties in PowerShell

JohanSt — Thu, 29 Dec 2011 15:17:47 GMT

There are some file properties that aren’t too easy to change. Neither in the command line or in Windows Explorer. One that I’ve wanted to change personally quite often is the created and changed dates of a specific file, or a batch of files.

Background

I frequently use my Windows Phone (currently a Nokia Lumia 800) for listening to audio books. I’ve found that the best way to do so is to change the category of the files to “podcast” and they’ll show up as podcasts and I won’t have to keep track of where I was, etc. since Zune will do that for me. Zune uses the creation date to order the files in a podcast, and some times the creation dates of my files may not be entirely correct. There are a lot of shareware programs that will allow you to do batch updates on files and change these properties, but it’s actually a lot easier to do it yourself in PowerShell

How to fix this

If you have the files sorted the way you want alphabetically and want to order them by creation date in the same way, then this is a pretty easy way to do it.

$files = dir
for($i=0;$i -le $files.GetUpperBound(0); $i++)
{
($files[$i]).CreationTime = (get-date).AddSeconds($i)
($files[$i]).LastWriteTime = (get-date).AddSeconds($i)
}

Naturally you could alter other properties as well simply pass one of the objects into get-method to see what’s available:

Name                      MemberType     Definition
----                      ----------     ----------
Mode                      CodeProperty   System.String Mode{get=Mode;}
Create                    Method         System.Void Create(System.Security.AccessControl.DirectorySecurity director...
CreateObjRef              Method         System.Runtime.Remoting.ObjRef CreateObjRef(type requestedType)
CreateSubdirectory        Method         System.IO.DirectoryInfo CreateSubdirectory(string path), System.IO.Director...
Delete                    Method         System.Void Delete(), System.Void Delete(bool recursive)
Equals                    Method         bool Equals(System.Object obj)
GetAccessControl          Method         System.Security.AccessControl.DirectorySecurity GetAccessControl(), System....
GetDirectories            Method         System.IO.DirectoryInfo[] GetDirectories(string searchPattern), System.IO.D...
GetFiles                  Method         System.IO.FileInfo[] GetFiles(string searchPattern), System.IO.FileInfo[] G...
GetFileSystemInfos        Method         System.IO.FileSystemInfo[] GetFileSystemInfos(string searchPattern), System...
GetHashCode               Method         int GetHashCode()
GetLifetimeService        Method         System.Object GetLifetimeService()
GetObjectData             Method         System.Void GetObjectData(System.Runtime.Serialization.SerializationInfo in...
GetType                   Method         type GetType()
InitializeLifetimeService Method         System.Object InitializeLifetimeService()
MoveTo                    Method         System.Void MoveTo(string destDirName)
Refresh                   Method         System.Void Refresh()
SetAccessControl          Method         System.Void SetAccessControl(System.Security.AccessControl.DirectorySecurit...
ToString                  Method         string ToString()
PSChildName               NoteProperty   System.String PSChildName=.gimp-2.6
PSDrive                   NoteProperty   System.Management.Automation.PSDriveInfo PSDrive=C
PSIsContainer             NoteProperty   System.Boolean PSIsContainer=True
PSParentPath              NoteProperty   System.String PSParentPath=Microsoft.PowerShell.Core\FileSystem::C:\Users\j...
PSPath                    NoteProperty   System.String PSPath=Microsoft.PowerShell.Core\FileSystem::C:\Users\jstraar...
PSProvider                NoteProperty   System.Management.Automation.ProviderInfo PSProvider=Microsoft.PowerShell.C...
Attributes                Property       System.IO.FileAttributes Attributes {get;set;}
CreationTime              Property       System.DateTime CreationTime {get;set;}
CreationTimeUtc           Property       System.DateTime CreationTimeUtc {get;set;}
Exists                    Property       System.Boolean Exists {get;}
Extension                 Property       System.String Extension {get;}
FullName                  Property       System.String FullName {get;}
LastAccessTime            Property       System.DateTime LastAccessTime {get;set;}
LastAccessTimeUtc         Property       System.DateTime LastAccessTimeUtc {get;set;}
LastWriteTime             Property       System.DateTime LastWriteTime {get;set;}
LastWriteTimeUtc          Property       System.DateTime LastWriteTimeUtc {get;set;}
Name                      Property       System.String Name {get;}
Parent                    Property       System.IO.DirectoryInfo Parent {get;}
Root                      Property       System.IO.DirectoryInfo Root {get;}
BaseName                  ScriptProperty System.Object BaseName {get=$this.Name;}

Well, that’s basically it. Enjoy!

/ Johan

System.Text.EncoderExceptionFallback

JohanSt — Thu, 01 Dec 2011 13:32:00 GMT

Today I found the following e-mail in my inbox:

Hello According to this article, http://blogs.msdn.com/b/johan/archive/2008/01/31/using-windbg-hunting-exceptions.aspx written by you, you write that there are three "ever-present exceptions", I can see why they have to be on the heap, but why is 790ff624 3 36 System.Text.DecoderExceptionFallback 790ff5d8 3 36 System.Text.EncoderExceptionFallback Also always there? I always see them on the heep when I do a debug session. I have always discarded them. According to Tess, they are "not real exceptions", refering to that a print of all exceptions on the heap throws an error when the are printed. I think that the are put on the heap when the process is created, like the other three. But why? Why are they so important? I DEMAND an answer, because I love you and your blog :) I am just joking. I have written to you because I think that you know your ways around the CLR and the heap.

I thought the answer might interest others as well, so I decided to answer the e-mail here.

This is not an Exception

The reason why these are on the heap and are not possible to examine using !PrintException (!pe) is because they’re actually not Exceptions at all. They show up when you run !dumpheap -type Exception because they contain the word “Exception” in their name, but that’s it. The System.Text.DecoderExceptionFallback object for example derives from the DecoderFallback object. It’s a fail-safe mechanism that is used when you can’t class that is used then you can’t complete the decoding conversion.

So what is a Decoder Fallback?

There are different fallbacks you can use. You can use the DecoderReplacementFallback that will replace any byte sequence that it can’t decode with a predefined string. For example; the decoder comes across a segment it can’t convert. You’ve defined a DecoderReplacementFallback and it replaces the segment with “[unknown]”. Execution will then continue.

If you had instead defined a DecoderExceptionFallback a DecoderFallbackException (notice the difference?) would be thrown when the decoder came across the segment that it couldn’t convert and execution would stop after that.

Cheers! / Johan

Changing Site Settings through code in SharePoint 2010

JohanSt — Tue, 13 Sep 2011 13:44:00 GMT

It’s been quite a while since I had the time to post. A customer dropped me this question today and I thought it’d be interesting to share.

He wanted to change the “Allow rules to specify another site as a target location”-setting (seen below) through code and needed to know if this could be done.
I hadn’t tried this before, but thought it’d be fun to try.

Finding the correct key

I suspected that the property could be found in Web.AllProperties, but I had no clue as to what the key might be called, so I simply iterated through all the keys twice. First with the setting enabled and secondly with the setting disabled. I wrote the keys to a label, and simply pasted them into Notepad so I could compare the two listings. The code I used to do this was quite simple:

foreach (String key in Web.AllProperties.Keys)
{     lblKeys.Text += key + "<BR>";
}

Having done this it was quite easy to identify the key I needed. It turned out it was called “_routerenablecrosssiterouting”. I recommend using the same approach if there is another setting you wish to change.

The final code

Now that I had the key name I was basically done. I used the following line to read the setting:

myString = (Web.AllProperties["_routerenablecrosssiterouting"]).ToString();

This is the code I used to change it:

Web.AllowUnsafeUpdates = true;
Web.AllProperties["_routerenablecrosssiterouting"] = "False";
Web.Update();

Since I was using a GET-request in my little sample I had to set AllowUnsafeUpdates = true

I’ve seen other people asking how to accomplish this on the net, so hopefully this post will be of use to you.

/ Johan

Problems running sos-commands

JohanSt — Fri, 04 Mar 2011 10:04:00 GMT

Yesterday I tried opening up a standard dump file from a 64-bit Windows 2008 Server. It opened up just fine in windbg, but when I tried executing a command I got the following:

0:000> !dumpheap -stat
The garbage collector data structures are not in a valid state for traversal.
It is either in the "plan phase," where objects are being moved around, or
we are at the initialization or shutdown of the gc heap. Commands related to 
displaying, finding or traversing objects as well as gc heap segments may not 
work properly. !dumpheap and !verifyheap may incorrectly complain of heap 
consistency errors.
Error requesting GC Heap data
Unable to build snapshot of the garbage collector state

This can happen if both the .NET 4.0 and the .NET 2.0 runtime are loaded in the same process. I actually tried using .loadby on both Framework versions and neither of them returned with an error

0:000> .loadby sos mscorwks
0:000> .loadby sos clr

The easiest way to resolve this is to run the following command:

0:000> .cordll -I mscorwks.dll -N -ve

After that, simply reload the extensions you want to use and you should be fine.

Updating the Smart tag panel of a WebControl requires closing and reopening the panel

JohanSt — Thu, 02 Sep 2010 08:23:00 GMT

The problem

I came across the following scenario a while back: A customer was creating a custom WebControl in the Visual Web Designer. The control had a smart tag panel with an option that they wanted to behave like a radio button. I.e. The text says “Enabled”, you click it and it changes to “Disabled”. The problem was that the panel didn’t refresh. Even if they called DesignerActionUIService.Refresh(). They had to close the panel and reopen it for the change to become visible.

If they instead hosted the same code in a WinForms control then the panel was refreshed immediately. So in effect the code below would update immediately if designing a WinForms component, and require closing and reopening the panel if designing a WebForms panel.

public bool LockColors
{
    get
    {
        return colLabel.ColorLocked;
    }
    set
    {
        GetPropertyByName("ColorLocked").SetValue(colLabel, value);

        // Refresh the list.
        this.designerActionUISvc.Refresh(this.Component);
    }
}

Cause

Simply put, the Web Developer doesn’t subscribe to the DesignerActionUIStateChange event, so this is currently by design.

Resolution

You can fix this by implementing your own event using code like this:

public UserControlDesignerActionList(ISourceControlledItem scItem)
        : base(((IDesigner)scItem).Component)
    {
        this.designerActionUISvc = GetService(typeof(DesignerActionUIService))
as DesignerActionUIService;
 
        if (this.Component is Control) {
            this.designerActionUISvc.DesignerActionUIStateChange += 
new DesignerActionUIStateChangeEventHandler(OnDesignerActionUIStateChange);
        }
    }
 
    void OnDesignerActionUIStateChange(object sender, DesignerActionUIStateChangeEventArgs e) {
        if (e.ChangeType == DesignerActionUIStateChangeType.Refresh) {
            ElementDesigner ed = ElementDesigner.GetElementDesigner((Control)this.Component);
            IWebElementDesigner wed = (IWebElementDesigner)ed;
            if (wed != null) {
                wed.UpdateView();
            }
        }
    }

In order for this to work you need to add a reference to Microsoft.Web.Design.Client.dll, which is located at C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\Microsoft.Web.Design.Client.dll as well as in the Global Assembly Cache.

Licensing

The main issue with this approach is that of licencing. If you intend to redistribute this component to a customer you may redistribute the .dll to that customer if they already own a valid license of that same .dll. Any component you design that need this change to work would require a copy of Visual Studio to be at all usefull, but you never know...

The best approach would probably be to rely on the version in the GAC and not distribute the .dll at all.

Using SSL without an Internet connection

JohanSt — Tue, 02 Feb 2010 12:39:14 GMT

Problem

If you have a web application that uses SSL and this application, for some reason, is hosted on an IIS that doesn’t have Internet access you will eventually find that any call to the server over SSL will take a lot of time to reply. It doesn’t matter if it’s ASP.NET, classic ASP or even CSS. As long as the request is made via SSL the response-time will skyrocket.

This isn’t a very common situation, since normally the IIS is connected to the Internet. HoweverI’ve come across this “in the field” where clients were routed internally to the IIS through a proxy, so it does happen.

Cause

This is a variation of the problem described in this article. The IIS will want to go on-line and verify that the certificate has not been revoked. In order to do this is requires a working Internet connection. Otherwise it will wait until the request times out and then service the request. When the next SSL-request comes along it will attempt to go on-line again, time out, and so on.

If you've had your server connected to the Internet during testing, then you will most likely not come across this problem straight away, since the response from the lookup in the certificate revocation list (CRL) will be cached. This makes it harder to pinpoint the root cause, unless you know where to look.

Resolution

The easiest way to resolve this is to skip CRL checking. This is done by setting the metabase property CertCheckMode to 1.

Create a blank .txt file on the server and paste the following code into it:

Set oWeb = GetObject("IIS://localhost/W3SVC")
oWeb.CertCheckMode = 1
oWeb.SetInfo
Set oWeb = Nothing

Running this script will change the property so that revocation checking is no longer done. If you want to switch back to the default value, then you should set CertCheckMode to 0 instead.

/ Johan

Make sure you have an Internet connection when using signed assemblies

JohanSt — Thu, 12 Nov 2009 11:08:22 GMT

A customer called in the other day and told me that his web application took a long time to start. My initial thought was (off course) that it was a matter of the classic slow-to start web services I’ve written about earlier.

(What to do about the slow startup of web services)

As I learned more about his problem I found that this was more than just a simple matter of pre-compilation and keeping your w3wp.exe alive.

The Cause

The customer was using authenticode signed assemblies in his web application. When that application calls the assembly for the first time it will want to go on-line and check that the certificate is still valid. So, what if this is an internal server with no Internet access? Well, then we have a problem. The process will spend quite some time trying to check the certificate revocation lists. Eventually it will give up, but for my customer this meant that it hung for at least 20 seconds upon startup.

The Resolution

To resolve this I’d chose one of the following three options:

Grant Internet access to the server

This is will obviously resolve the problem, but may not always be possible.

Review the need for Authenticode signing

If you own the assemblies then you might want to consider not using Authenticode signing at all. A Strong Named Assembly might be sufficient. For more information on Strong Named Assemblies and how to create them, please see the following article on MSDN: How to: Sign an Assembly with a Strong Name

Disable Signature Verification

If all else fails you can disable Signature Verification by adding the following to your machine.config or application.exe.config:

Identifying the problem using windbg

Okay, so if we suspect that this is the problem, how can we verify this?

I got a memory dump from my customer, so I’ll use that to demonstrate.

First of all you want to make sure the dump is taken as the application pool hangs upon the first request. Getting a dump right after the problem has occurred will usually do us no good at all. We need to get the dump as the problem is actually ocurring. Having made sure the dump is fine I open up the dump in windbg and load sos.dll. I then use the !aspxpages command to see the pending requests currently on the server. !aspxpages dumps all the HttpContexts found on the heap so we get a list of all currently executing requests, as well as recently finished ones that haven’t been garbage collected yet.

0:000> .load C:\Windows\Microsoft.NET\Framework\v2.0.50727\sos.dll
0:000> !aspxpages
Going to dump the HttpContexts found in the heap.
Loading the heap objects into our cache.
HttpContext    Timeout Completed     Running ThreadId ReturnCode   Verb RequestPath+QueryString
0x01b41ad4        0            no        11 Sec     XXX        200       /App/
0x01b75f48      110 Sec        no        11 Sec     XXX        401   GET /App/Default.aspx
0x01b7eb40    19200 Sec        no        11 Sec      19        200   GET /App/Default.aspx
Total 3 HttpContext objects

So, the request on thread 19 is the one we want. As we can see in the ThreadId column it is the only active request. Let’s jump to that thread and investigate what it is doing.

0:000> ~19s
eax=01510000 ebx=00000000 ecx=03e2d348 edx=776d9a94 esi=03e2d07c edi=00000000
eip=776d9a94 esp=03e2d034 ebp=03e2d0a4 iopl=0         nv up ei ng nz ac pe cy
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000             efl=00000297
ntdll!KiFastSystemCallRet:
776d9a94 c3              ret
0:019> !clrstack
OS Thread Id: 0x11cc (19)
ESP       EIP
03e2eb8c 776d9a94 [PrestubMethodFrame: 03e2eb8c] App.BasePage.Page_Load(System.Object, System.EventArgs)
03e2eba0 6aada7ff System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr, System.Object, System.Object, System.EventArgs)
03e2ebb0 6c392594 System.Web.Util.CalliEventHandlerDelegateProxy.Callback(System.Object, System.EventArgs)
03e2ebc4 6c38ba84 System.Web.UI.Control.OnLoad(System.EventArgs)
03e2ebd8 6c38bac3 System.Web.UI.Control.LoadRecursive()
03e2ebf0 6c387b74 System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)
03e2ed48 6c3877a4 System.Web.UI.Page.ProcessRequest(Boolean, Boolean)
03e2ed80 6c3876d1 System.Web.UI.Page.ProcessRequest()
03e2edb8 6c387666 System.Web.UI.Page.ProcessRequestWithNoAssert(System.Web.HttpContext)
03e2edc4 6c387642 System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)
03e2edd8 012b02a6 ASP.text.ProcessRequest(System.Web.HttpContext)
03e2ede8 6c38db16 System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
03e2ee1c 6c36132c System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
03e2ee5c 6c95531f System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)
03e2ee60 6c94b704 [InlinedCallFrame: 03e2ee60]
03e2ef00 6c93613d System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext)
03e2ef70 6ca0a7a2 System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
03e2ef74 6ca0a58f [InlinedCallFrame: 03e2ef74]
03e2f4c8 01092314 [NDirectMethodFrameStandalone: 03e2f4c8] System.Web.Hosting.UnsafeIISMethods.MgdIndicateCompletion(IntPtr, System.Web.RequestNotificationStatus ByRef)
03e2f4d8 6ca0a839 System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
03e2f55c 6ca0a58f System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
03e2f65c 01092314 [ContextTransitionFrame: 03e2f65c]

So we’re in the Page_Load event of the page in question, but apart from that !clrstack doesn’t tell us much. Let’s take a look at the native callstack instead using the kb-command.

0:019> kb20
ChildEBP RetAddr Args to Child
03e2d030 776d9254 76adc244 000006d8 00000000 ntdll!KiFastSystemCallRet
03e2d034 76adc244 000006d8 00000000 03e2d07c ntdll!ZwWaitForSingleObject+0xc
03e2d0a4 74256247 000006d8 00003a98 00000000 kernel32!WaitForSingleObjectEx+0xbe
03e2d0d8 74251f7c 0141f2e4 00000002 00202005 cryptnet!CryptRetrieveObjectByUrlWithTimeout+0x1a5
03e2d1c8 742527a1 0141f2e4 00000002 00202005 cryptnet!CryptRetrieveObjectByUrlW+0xcc
03e2d268 742591be 00000000 0141f2e4 00000002 cryptnet!RetrieveObjectByUrlValidForSubject+0xa0
03e2d2c8 74252556 00000000 00000000 00000001 cryptnet!RetrieveTimeValidObjectByUrl+0x15c
03e2d374 74253fc1 00000082 0141f2d8 00000000 cryptnet!CTVOAgent::GetTimeValidObjectByUrl+0x178
03e2d45c 74253d98 00000003 018aa590 00000002 cryptnet!CTVOAgent::GetTimeValidObject+0x4a8
03e2d48c 74253428 00000003 018aa590 018c8220 cryptnet!FreshestCrlFromCrlGetTimeValidObject+0x2d
03e2d4d0 74259283 00000003 018aa590 018c8220 cryptnet!CryptGetTimeValidObject+0x58
03e2d54c 74253b1c 00000003 018aa590 018aa590 cryptnet!GetTimeValidCrl+0x2cb
03e2d590 74253992 018aa590 018c8220 03e2d5c8 cryptnet!GetBaseCrl+0x34
03e2d63c 757f77d5 00000001 00000001 00000001 cryptnet!MicrosoftCertDllVerifyRevocation+0x163
03e2d6c8 757f7641 00000001 00000001 00000001 crypt32!I_CryptRemainingMilliseconds+0x2aa
03e2d74c 757f7a26 00000001 00000001 00000001 crypt32!CertVerifyRevocation+0xd4
03e2d7f8 757f7838 01899590 00000000 00000000 crypt32!CChainPathObject::CalculateRevocationStatus+0x2d0
03e2d83c 75806c44 01899590 018c3018 01870fc0 crypt32!CChainPathObject::CalculateAdditionalStatus+0x152
03e2d904 758069a5 01899590 018aa590 01870fc0 crypt32!CCertChainEngine::CreateChainContextFromPathGraph+0x23e
03e2d93c 7580be14 018aa590 018aff94 01870fc0 crypt32!CCertChainEngine::GetChainContext+0x46
03e2d974 749172ba 01421338 018aa590 018aff94 crypt32!CertGetCertificateChain+0x72
03e2d9dc 749170fc 40000001 00000000 03e2da10 wintrust!_WalkChain+0x1ae
03e2da18 7491363b 00000000 03e2dbdc 03e2db74 wintrust!WintrustCertificateTrust+0xb9
03e2db30 7491346c 00000000 03e2dbe4 00000000 wintrust!_VerifyTrust+0x253
03e2db54 64025b1b 00000000 03e2dbe4 03e2db74 wintrust!WinVerifyTrust+0x50
03e2dbf8 716b7f4d 014443f8 00000000 00000003 mscorsec!GetPublisher+0xe4
03e2dc50 7145ed3b d60caabc 0188c4a8 00603c48 mscorwks!PEFile::CheckSecurity+0xcb
03e2dc78 7145ec84 d60caa64 00000000 0188c4a8 mscorwks!PEAssembly::DoLoadSignatureChecks+0x3a
03e2dca0 7145f0ca 01879f00 00000000 0188c4a8 mscorwks!PEAssembly::PEAssembly+0x109
03e2df3c 7145f1c5 0188c4a8 00000000 00000000 mscorwks!PEAssembly::DoOpen+0x103
03e2dfd0 71459062 0188c4a8 00000000 00000000 mscorwks!PEAssembly::Open+0x79
03e2e134 71456ace 03e2e15c 00000001 00000000 mscorwks!AppDomain::BindAssemblySpec+0x247

Here we see that we’re currently trying to check the certificate revocation list. This is done on a separate thread so this thread is currently doing nothing but waiting. Our next step would be to find this thread, but first we can find out the name of the .dll that we’re trying to verify. Look at the first argument to GetPublisher.

0:019> du 014443f8
014443f8 "C:\Windows\Microsoft.NET\Framewo"
01444438 "rk\v2.0.50727\Temporary ASP.NET "
01444478 "Files\scstt\57ae16cd\2f8637d5\as"
014444b8 "sembly\dl3\078f3d05\00fba208_c03"
014444f8 "6ca01\Microsoft.Practices.Enterp"
01444538 "riseLibrary.Logging.DLL"

So it’s a Microsoft .dll that is part of the Enterprise Library. Good to know. Now let’s find the thread that is attempting to retrieve the revocation list by running ~*kb and scanning through the threads until we find this one:

0:021> kb2000
ChildEBP RetAddr Args to Child
0427d7f8 776d9254 75502283 0000070c 00000001 ntdll!KiFastSystemCallRet
0427d7fc 75502283 0000070c 00000001 0427d824 ntdll!ZwWaitForSingleObject+0xc
0427d83c 75501fc8 0000070c 00000714 00000001 mswsock!SockWaitForSingleObject+0x19f
0427d928 77801693 00000001 00000000 0427d9d0 mswsock!WSPSelect+0x38c
0427d9a8 72a733e5 00000001 00000000 0427d9d0 ws2_32!select+0x494
0427e1ec 72a730fd 00000714 0427e210 72a72e88 winhttp!ICSocket::Connect_Start+0x3b0
0427e1f8 72a72e88 01828250 72a737bc 00000000 winhttp!CFsm_SocketConnect::RunSM+0x42
0427e210 72a7382b 018b0c60 00000000 00000000 winhttp!CFsm::Run+0x20
0427e234 72a7467d 01828250 018ccf30 0427e258 winhttp!DoFsm+0x2a
0427e244 72a74646 0000ea60 00000005 00000020 winhttp!ICSocket::Connect+0x32
0427e258 72a742b5 0000ea60 00000005 0000ea60 winhttp!ICSocket::Connect+0x13
0427e2a4 72a74038 01434458 0427e2c8 72a72e88 winhttp!HTTP_REQUEST_HANDLE_OBJECT::OpenConnection_Fsm+0x471
0427e2b0 72a72e88 01434458 00000000 00000000 winhttp!CFsm_OpenConnection::RunSM+0x37
0427e2c8 72a7382b 018b0c60 00000000 00000000 winhttp!CFsm::Run+0x20
0427e2ec 72a74625 01434458 018ccf30 0427e324 winhttp!DoFsm+0x2a
0427e2fc 72a745bc 00000000 00000000 018b0c60 winhttp!HTTP_REQUEST_HANDLE_OBJECT::OpenConnection+0x2f
0427e324 72a74526 0141f638 0427e348 72a72e88 winhttp!HTTP_REQUEST_HANDLE_OBJECT::MakeConnection_Fsm+0x9b
0427e330 72a72e88 0141f638 00000000 00000000 winhttp!CFsm_MakeConnection::RunSM+0x37
0427e348 72a7382b 018b0c60 00000000 00000000 winhttp!CFsm::Run+0x20
0427e36c 72a743c1 0141f638 018b0c60 014343c0 winhttp!DoFsm+0x2a
0427e3ac 72a74339 014343c0 0427e3d0 72a72e88 winhttp!HTTP_REQUEST_HANDLE_OBJECT::SendRequest_Fsm+0x93
0427e3b8 72a72e88 014343c0 018ccf30 00000000 winhttp!CFsm_SendRequest::RunSM+0x37
0427e3d0 72a7382b 018b0c60 00000000 00000000 winhttp!CFsm::Run+0x20
0427e3f4 72a73db0 014343c0 018b0c60 01888820 winhttp!DoFsm+0x2a
0427e414 72a76f71 00000000 0427e438 72a72e88 winhttp!HTTP_REQUEST_HANDLE_OBJECT::HttpSendRequest_Start+0x2af
0427e420 72a72e88 01888820 00000001 00000000 winhttp!CFsm_HttpSendRequest::RunSM+0x4c
0427e438 72a73c3e 018b0c60 00000000 00000000 winhttp!CFsm::Run+0x20
0427e480 72a73b8b 01888820 018ccf30 00000000 winhttp!StartFsmChain+0xcf
0427e4c4 72a739d8 018ccf30 00000000 00000000 winhttp!HttpWrapSendRequest+0x18c
0427e544 742599b4 018ccf30 00000000 00000000 winhttp!WinHttpSendRequest+0x19b
0427f5e4 74259741 005f7bc0 018ccf30 01898e60 cryptnet!InetSendAuthenticatedRequestAndReceiveResponse+0x56f
0427f6ec 7425921d 005f7bc0 01898e60 00202005 cryptnet!InetSendReceiveUrlRequest+0x2c8
0427f724 742523e9 01898e60 00000002 00202005 cryptnet!CInetSynchronousRetriever::RetrieveObjectByUrl+0x5f
0427f75c 742521a8 01898e60 00000002 00202005 cryptnet!InetRetrieveEncodedObject+0x64
0427f7b8 74256350 01898e60 00000002 00202005 cryptnet!CObjectRetrievalManager::RetrieveObjectByUrl+0xbb
0427f810 76ad4911 00000000 0427f85c 776be4b6 cryptnet!CryptRetrieveObjectByUrlWithTimeoutThreadProc+0x67
0427f81c 776be4b6 01898db0 4e080acf 00000000 kernel32!BaseThreadInitThunk+0xe
0427f85c 776be489 742562f7 01898db0 ffffffff ntdll!__RtlUserThreadStart+0x23
0427f874 00000000 742562f7 01898db0 00000000 ntdll!_RtlUserThreadStart+0x1b
0:021> du 01898e60
01898e60 "http://crl.microsoft.com/pki/crl"
01898ea0 "/products/CodeSignPCA2.crl"

This is the thread. To finally verify that it is in fact attempting to download the Certificate Revocation List we can check the value of the first argument to cryptnet!CInetSynchronousRetriever, InetRetrieveEncodedObject and CObjectRetrievalManager::RetrieveObjectByUrl or the second argument to cryptnet!InetSendReceiveUrlRequest as I did above.

Conclusion

In this case the customer chose to disable Signature Verification completely rather than granting Internet access to the server.

If you found the windbg-portion of this post hard to understand I can recommend a closer look at my Debugging School (In the menu to the left). Begin with Getting started with windbg - Part I and you should be set to go in no time.

/ Johan

Investigating Locks

JohanSt — Fri, 09 Oct 2009 16:49:56 GMT

Consider the following scenario:

You have an ASP.NET application which intermittently responds sluggishly. As the problem occurs memory usage is about average, as is CPU usage, but still certain pages respond slower and slower. The machine acts just as if it is under heavy load, but judging from the CPU it isn't. In fact CPU usage might even decrease rather than increase. When trying to access non .NET content such as plain text/html or images you usually find that response times are just as fast as ever.

When you find that your application has hung in a low CPU state it is usually because of one of the following reasons (in order of likelihood based on my experience) :

External resources responding slowly. E.g. a database query or AD lookup is taking a long time to complete and so the application is simply waiting for another server to respond.
One of the threads of the application has entered a critical section and now several other threads are waiting to enter the same critical section.
A classic deadlock. Thread A waits for thread B which waits for thread C which waits for thread A.

Quite often you'll see a mixture of items 1 and 2. A thread enters a critical section, calls a remote database, or similar, and all other threads attempting to enter the same critical section will have to nicely stay in line until the struggling database has responded. Still, it is quite possible that the critical section is waiting for a local event to finish, but then you'd usually see a higher CPU load on the machine. (Though not necessarily in the w3wp.exe process.)

What are Critical Sections?

In case you've forgotten or simply not worked that much with critical sections it might be necessary to refresh your memory on this.

A Critical Section is a segment of code that you do not want to be executed more than one at a time. In order to ensure this you use the lock keyword to ensure that a certain block of code is able to complete with no interruption from any other threads. To illustrate this I've created the following code sample:

public class FileIO
{
    private static object myLock = new object();

	public FileIO()
	{
	}

    public string performLockOperation()
    {
        string strResult = "The Lock Operation started on " + System.DateTime.Now.ToLongTimeString();
        lock (myLock)
        {
            // Let the thread sleep for 10 seconds
            // This way we simulate a File IO operation that takes some time to complete.
            System.Threading.Thread.Sleep(10000);
        }
        strResult += " and ended on " + System.DateTime.Now.ToLongTimeString();
        return strResult;
    }

    public string performNoLockOperation()
    {
        string strResult = "The No Lock Operation started on " + System.DateTime.Now.ToLongTimeString();
        System.Threading.Thread.Sleep(10000);
        strResult += " and ended on " + System.DateTime.Now.ToLongTimeString();
        return strResult;
    }

}

Let's take a quick look at the parts of this class that I've written, called FileIO:

myLock

The class has a static variable called myLock that I will be using to identify the lock I'm about to create. This is why the variable is static. Otherwise I'd be creating new locks for each instance of the FileIO class leaving the lock operation completely redundant. Now that I use a static variable the lock spans all instances of the FileIO class.

constructor

The constructor is empty. Move along. Nothing to see here.

performLockOperation

This is the most interesting part of the class. It is the function we'll be using to actually reproduce the potential problems. The function returns a string which will contain information about when the call to the function was made and when it completed. We store this data in a string called strResult which we will eventually be returning as the result of the function call.

Having declared this variable and put the initial time of the function call in it we then create our lock. This is simply done with the following syntax:

lock (object)
{
}

Any code within this block will first make sure that there is not already a lock taken on this object. If there is, then they will pause execution and wait for the lock to become available before continuing execution.

Inside our lock we then simulate a slow File IO operation by letting the thread sleep for 10 seconds.

After that we append the current time to the result and return it.

performNoLockOperation

This function is identical to the previous one, except this time we do not have a critical section. This function is included just for comparison.

The potential problem

Now, let's get down to business. If we have a web page that makes a call to the performLockOperation-function and that page is requested at the same time by four different clients. What would be the response times?

The answer is 10, 20, 30 & 40 seconds.

Here's why:

The thread handling the first request enters the critical section, effectively locking it for everyone else. When it is complete 10 seconds have passed due to the Sleep-call.
One of the other threads enters the critical section once the first request is finished. It has already waited for 10 seconds, and the thread will now sleep for an additional 10 seconds.
As the second thread finishes one of the remaining two enters the critical section. It has now waited for 20 seconds and will sleep for an additional 10.
The final request is able to enter the critical section after having waited for 30 seconds. Once it is complete 40 seconds have passed.

If we'd been using the performNoLockOperation instead then all requests would have responded after 10 seconds.

Now, I know that 10 seconds is an exceptionally long time to let the thread sleep, but this was all for illustration purposes. Imagine instead that the sleeping time was 1 second and the number of requests was 200...

Troubleshooting this issue

Okay, so what would it look like when this problem occurs? Let's have a look. If you haven't looked at my posts on getting started with Windbg, then I suggest reading them before continuing. (Part I & Part II)

First of all I fire up WinDbg, then I attach to the w3wp.exe. I then use "~*e!clrstack" to look at the managed threads of the w3wp.exe application. Here is a snippet of what I get:

OS Thread Id: 0xce0 (24)
Child-SP         RetAddr          Call Site
000000000447e0f0 000007ff001e13d2 FileIO.performLockOperation()
000000000447e170 000007feee323ec9 _Default.Page_Load(System.Object, System.EventArgs)
000000000447e1b0 000007fee7956aea System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr, System.Object, System.Object, System.EventArgs)
000000000447e1e0 000007fee794d2c4 System.Web.Util.CalliEventHandlerDelegateProxy.Callback(System.Object, System.EventArgs)
000000000447e210 000007fee794d322 System.Web.UI.Control.OnLoad(System.EventArgs)
000000000447e250 000007fee79498ac System.Web.UI.Control.LoadRecursive()
000000000447e2a0 000007fee7948db0 System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)
000000000447e370 000007fee7948cdb System.Web.UI.Page.ProcessRequest(Boolean, Boolean)
000000000447e3d0 000007fee7948c70 System.Web.UI.Page.ProcessRequest()
000000000447e430 000007ff001e0c09 System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)
000000000447e490 000007fee7950117 ASP.default_aspx.ProcessRequest(System.Web.HttpContext)
000000000447e4c0 000007fee791449b System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
000000000447e570 000007fee7ffbd41 System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
000000000447e610 000007fee7fed132 System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)
000000000447e7a0 000007fee7fcf599 System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback)
000000000447e7f0 000007fee80f5344 System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext)
000000000447e910 000007fee80f65bb System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
000000000447ea80 000007fee80f4994 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
000000000447eae0 000007fef72701ea DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)
000000000447f310 000007fee80f5424 DomainNeutralILStubClass.IL_STUB(IntPtr, System.Web.RequestNotificationStatus ByRef)
000000000447f3f0 000007fee80f65bb System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
000000000447f560 000007fee80f4994 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
000000000447f5c0 000007fef727043b DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)
OS Thread Id: 0x15ec (25)
Unable to walk the managed stack. The current thread is likely not a 
managed thread. You can run !threads to get a list of managed threads in
the process
OS Thread Id: 0x1958 (26)
Child-SP         RetAddr          Call Site
0000000004c1dd10 000007ff001e13d2 FileIO.performLockOperation()
0000000004c1dd90 000007feee323ec9 _Default.Page_Load(System.Object, System.EventArgs)
0000000004c1ddd0 000007fee7956aea System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr, System.Object, System.Object, System.EventArgs)
0000000004c1de00 000007fee794d2c4 System.Web.Util.CalliEventHandlerDelegateProxy.Callback(System.Object, System.EventArgs)
0000000004c1de30 000007fee794d322 System.Web.UI.Control.OnLoad(System.EventArgs)
0000000004c1de70 000007fee79498ac System.Web.UI.Control.LoadRecursive()
0000000004c1dec0 000007fee7948db0 System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)
0000000004c1df90 000007fee7948cdb System.Web.UI.Page.ProcessRequest(Boolean, Boolean)
0000000004c1dff0 000007fee7948c70 System.Web.UI.Page.ProcessRequest()
0000000004c1e050 000007ff001e0c09 System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)
0000000004c1e0b0 000007fee7950117 ASP.default_aspx.ProcessRequest(System.Web.HttpContext)
0000000004c1e0e0 000007fee791449b System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
0000000004c1e190 000007fee7ffbd41 System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
0000000004c1e230 000007fee7fed132 System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)
0000000004c1e3c0 000007fee7fcf599 System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback)
0000000004c1e410 000007fee80f5344 System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext)
0000000004c1e530 000007fee80f65bb System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
0000000004c1e6a0 000007fee80f4994 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
0000000004c1e700 000007fef72701ea DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)
0000000004c1ef30 000007fee80f5424 DomainNeutralILStubClass.IL_STUB(IntPtr, System.Web.RequestNotificationStatus ByRef)
0000000004c1f010 000007fee80f65bb System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
0000000004c1f180 000007fee80f4994 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
0000000004c1f1e0 000007fef727043b DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)
OS Thread Id: 0x1e28 (27)
Child-SP         RetAddr          Call Site
000000000492dcf0 000007ff001e13d2 FileIO.performLockOperation()
000000000492dd70 000007feee323ec9 _Default.Page_Load(System.Object, System.EventArgs)
000000000492ddb0 000007fee7956aea System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr, System.Object, System.Object, System.EventArgs)
000000000492dde0 000007fee794d2c4 System.Web.Util.CalliEventHandlerDelegateProxy.Callback(System.Object, System.EventArgs)
000000000492de10 000007fee794d322 System.Web.UI.Control.OnLoad(System.EventArgs)
000000000492de50 000007fee79498ac System.Web.UI.Control.LoadRecursive()
000000000492dea0 000007fee7948db0 System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)
000000000492df70 000007fee7948cdb System.Web.UI.Page.ProcessRequest(Boolean, Boolean)
000000000492dfd0 000007fee7948c70 System.Web.UI.Page.ProcessRequest()
000000000492e030 000007ff001e0c09 System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)
000000000492e090 000007fee7950117 ASP.default_aspx.ProcessRequest(System.Web.HttpContext)
000000000492e0c0 000007fee791449b System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
000000000492e170 000007fee7ffbd41 System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
000000000492e210 000007fee7fed132 System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)
000000000492e3a0 000007fee7fcf599 System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback)
000000000492e3f0 000007fee80f5344 System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext)
000000000492e510 000007fee80f65bb System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
000000000492e680 000007fee80f4994 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
000000000492e6e0 000007fef72701ea DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)
000000000492ef10 000007fee80f5424 DomainNeutralILStubClass.IL_STUB(IntPtr, System.Web.RequestNotificationStatus ByRef)
000000000492eff0 000007fee80f65bb System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
000000000492f160 000007fee80f4994 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
000000000492f1c0 000007fef727043b DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)
OS Thread Id: 0x1c38 (28)
Child-SP         RetAddr          Call Site
0000000004d0e0b0 000007ff001e13d2 FileIO.performLockOperation()
0000000004d0e130 000007feee323ec9 _Default.Page_Load(System.Object, System.EventArgs)
0000000004d0e170 000007fee7956aea System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr, System.Object, System.Object, System.EventArgs)
0000000004d0e1a0 000007fee794d2c4 System.Web.Util.CalliEventHandlerDelegateProxy.Callback(System.Object, System.EventArgs)
0000000004d0e1d0 000007fee794d322 System.Web.UI.Control.OnLoad(System.EventArgs)
0000000004d0e210 000007fee79498ac System.Web.UI.Control.LoadRecursive()
0000000004d0e260 000007fee7948db0 System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)
0000000004d0e330 000007fee7948cdb System.Web.UI.Page.ProcessRequest(Boolean, Boolean)
0000000004d0e390 000007fee7948c70 System.Web.UI.Page.ProcessRequest()
0000000004d0e3f0 000007ff001e0c09 System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)
0000000004d0e450 000007fee7950117 ASP.default_aspx.ProcessRequest(System.Web.HttpContext)
0000000004d0e480 000007fee791449b System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
0000000004d0e530 000007fee7ffbd41 System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
0000000004d0e5d0 000007fee7fed132 System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)
0000000004d0e760 000007fee7fcf599 System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback)
0000000004d0e7b0 000007fee80f5344 System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext)
0000000004d0e8d0 000007fee80f65bb System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
0000000004d0ea40 000007fee80f4994 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
0000000004d0eaa0 000007fef72701ea DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)
0000000004d0f2d0 000007fee80f5424 DomainNeutralILStubClass.IL_STUB(IntPtr, System.Web.RequestNotificationStatus ByRef)
0000000004d0f3b0 000007fee80f65bb System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
0000000004d0f520 000007fee80f4994 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
0000000004d0f580 000007fef727043b DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32

So, what does this tell us?

Well, obviously we have four threads that call my function FileIO.performLockOperation() from Page_Load.

Please note that we don't have any additional callstack, and since the performLockOperation-function was named by me it could just as well have been named "foo". In order to find out that these threads are actually waiting for a lock we need to look at the native callstack using the kb-command. Let's take a look at thread 24:

0:029> ~24kb
ntdll!NtDelayExecution+0xa
kernel32!SleepEx+0x84
mscorwks!EESleepEx+0x2d
mscorwks!Thread::UserSleep+0x71
mscorwks!ThreadNative::Sleep+0xf9
App_Code_tvh3usvo!FileIO.performLockOperation()+0x8a
App_Web_uywrbugj!_Default.Page_Load(System.Object, System.EventArgs)+0x32
System_Web_RegularExpressions_ni!System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr, System.Object, System.Object, System.EventArgs)+0x19
System_Web_ni!System.Web.Util.CalliEventHandlerDelegateProxy.Callback(System.Object, System.EventArgs)+0x2a
System_Web_ni!System.Web.UI.Control.OnLoad(System.EventArgs)+0x84
System_Web_ni!System.Web.UI.Control.LoadRecursive()+0x42
System_Web_ni!System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)+0x97c
System_Web_ni!System.Web.UI.Page.ProcessRequest(Boolean, Boolean)+0xa0
System_Web_ni!System.Web.UI.Page.ProcessRequest()+0x5b
System_Web_ni!System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)+0xf0
App_Web_uywrbugj!ASP.default_aspx.ProcessRequest(System.Web.HttpContext)+0x9
System_Web_ni!System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()+0x257
System_Web_ni!System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)+0xab
System_Web_ni!System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)+0x501
System_Web_ni!System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback)+0x72

Apparently we entered FileIO.performLockOperation, and then called sleep, (which is what we're doing in this repro in order to simulate a slow IO operation.) So, what do the other threads look like? Here's thread 26:

0:029> ~26kb
ntdll!ZwWaitForMultipleObjects+0xa
kernel32!WaitForMultipleObjectsEx+0x10b
mscorwks!WaitForMultipleObjectsEx_SO_TOLERANT+0xc1
mscorwks!Thread::DoAppropriateAptStateWait+0x41
mscorwks!Thread::DoAppropriateWaitWorker+0x191
mscorwks!Thread::DoAppropriateWait+0x5c
mscorwks!CLREvent::WaitEx+0xbe
mscorwks!AwareLock::EnterEpilog+0xc9
mscorwks!AwareLock::Enter+0x72
mscorwks!AwareLock::Contention+0x1fb
mscorwks!JITutil_MonContention+0xdf
App_Code_tvh3usvo!FileIO.performLockOperation()+0x7f
App_Web_uywrbugj!_Default.Page_Load(System.Object, System.EventArgs)+0x32
System_Web_RegularExpressions_ni!System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr, System.Object, System.Object, System.EventArgs)+0x19
System_Web_ni!System.Web.Util.CalliEventHandlerDelegateProxy.Callback(System.Object, System.EventArgs)+0x2a
System_Web_ni!System.Web.UI.Control.OnLoad(System.EventArgs)+0x84
System_Web_ni!System.Web.UI.Control.LoadRecursive()+0x42
System_Web_ni!System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)+0x97c
System_Web_ni!System.Web.UI.Page.ProcessRequest(Boolean, Boolean)+0xa0
System_Web_ni!System.Web.UI.Page.ProcessRequest()+0x5b

This is one of the threads waiting for the critical section. It then found that it was unable to get a lock and so began waiting for the critical section on thread 24 to end.

Resolving the problem

This is the tricky part. There is no quick-fix for this. We can use the debugger to identify the bottleneck, but after that we'll need to review our design in order to get this working smoothly. A couple of generic things to try would be the following:

Review what you're doing inside the critical section. Is it really necessary to perform all of the operations you're doing, or would it be possible to do some/all of them outside the scope of the critical section?
Identify the reason for the lock taking so much time. Can it be resolved by load-balancing or clustering the resource we're waiting for?
If we're waiting for another server, make sure the physical infrastructure is okay. Are network response times within the expected parameters or do you need to fix your wiring, router, etc?
Are all calls to the critical section taking the same amount of time, or are there certain calls that take an exceptional amount of time? Perhaps most calls to the function are just fine, but intermittently a call comes along that tries to get everything from the AD, database, etc. In order to investigate this I'd use the commands outlined in Getting started with windbg -part II. Identify the thread owning the lock and investigate the callstack and stack objects using !clrstack and !dso. Use !dso and !do to query the stack objects and try to find the SQL query or AD query that is causing the thread to take such a long time to finish. Is the query significantly different from the ones waiting to execute, then this is most likely our culprit.

Post Script

A quick thought on locks and their limits:

Locks do not scale well. A lock can only be made within a specific process, so if you're using a web farm, or in any other way more than one w3wp.exe, then you will not be able to lock all processes. In other words. Server 1 can get a lock, and so can Server 2. Both will happily execute their code independent of each other. For products such as SQL server you'd deal with this in your SQL query, but if you're performing other operations, such as File IO you might have to work around this by using the same approach as Ms Word. Create a lock file in a shared space, and before you enter your critical section you simply check if the file exists:

You are entering your critical section. Look for lock file in shared location
If file exists, then wait
If/when file doesn't/no longer exist, create file and enter critical section
Make sure you have solid exception handling within the critical section so that you under no circumstances leave the critical section without removing the lock file.
When done, remove lock file

/ Johan

Problems with Flash-content in the WebBrowser control

JohanSt — Thu, 06 Aug 2009 17:59:34 GMT

A recent case I worked on involved the following scenario.

Problem description

The client was building a kiosk-like application. I.e. a WinForms application with the WebBrowser control slapped onto it. The application worked fine for the most part, but they'd found that when viewing video via facebook they were only able to view the video the first time. Any additional attempt to view the video would fail with a custom error from the facebook application stating that the video was either restricted or no longer available.

This only happened when viewing the page via the WinForms application. Using IE directly worked just fine. We also found that this problem occurred with both IE7 and IE8 installed, so it seemed not to be related to the browser version.

Other sites using the Adobe Flash player to display video, such as MSN or Youtube worked just fine.

Troubleshooting

Using the Developer tools included in IE8 I found that the facebook application displays a placeholder image with a play-button. Once that button is clicked the .swf player is loaded via client-side script. The .swf player then downloads the .mp4 that is to be displayed and divides it into suitable chunks.

Fiddler traces revealed that the mp4 was downloaded when using IE but not when using the WinForms application. No errors were reported, instead the request was simply never made.

We removed all code behind. All that remained was the On_Load event that set a starting page for the WebBrowser control, still we were unable to successfully load the movie.

Adding facebook to the trusted zone changed nothing.

Using the COM-version of the WebBrowser control still reproduced the problem.

I then suspected restrictions in cross-domain access or similar, (facebook uses a lot of different domains,) but couldn't find anything conclusive.

Finally I stumbled upon the root cause.

Cause

The problem is due to a bug within the Adobe Flash-player: http://bugs.adobe.com/jira/browse/FP-256

The problem occurs when dynamically trying to display a cached .swf application with an external interface through script. So, another quick repro would be the following (Copied from the Adobe bug report):

Create a C# application with an embedded WebBrowser control.
Load a HTML page with no content.
Via JavaScript, add a SWF with external interface to the page via an object tag. (At this point I see server traffic, requesting the SWF.)
From JavaScript, access the external interface of the SWF. Works OK.
Browse to a second web page.
Browse back to the empty one.
Via JavaScript, add the SWF again. Cannot contact the external interface. (There is no server traffic, indicating that the SWF is already being cached.)

Solution

While waiting for a fix from Adobe the only way to circumvent this is to manually remove the cached .swf. Article 262110 describes in detail how you can do this. Unfortunately this can be a bit tricky some times. In the sample repro above you'd resolve it by simply removing all .swf items from the cache upon page-load, but this wouldn't work in the facebook situation, since it might use the same control repeatedly without refreshing the page in between. Depending on your level of control over the target platform it might be easier to simply turn of caching.

/ Johan

Adding Server ROles to an IIS installation on an RODC

JohanSt — Tue, 05 May 2009 17:10:15 GMT

In a recent case I was involved with I stumbled upon something quite interesting. The customer had set up a brand new Windows 2008 Server running IIS, and made it into a Read Only Domain Controller (RODC). Having done this they now wanted to add the FTP-server Role to the existing IIS installation and found that they couldn't.

The reason for this is that the RODC is quite picky about what server roles and features it will allow you to install. I recommend looking at the following articles, but they still don't cover everything.

Applications That Are Known to Work with RODCs
http://technet.microsoft.com/en-us/library/cc732790.aspx

Planning for Application Compatibility with RODCs
http://technet.microsoft.com/en-us/library/cc731746.aspx

The main lesson to be learned is that although the application is listed as known to work with RODCs that doesn't necessarily mean it installs well once the server has been promoted to RODC. When it comes to FTP you have two valid options to get this working:

Install the FTP server before promoting the server to RODC
Install the FTP Service 7.5 for IIS7. This is a stand-alone installer and works just fine.

/ Johan

Creating a game for the Zune

JohanSt — Tue, 31 Mar 2009 17:34:00 GMT

I've finally had the time to sit down and create a simple game for my Zune. Rather than running off and creating a complex game requiring a lot of AI, etc I wanted to create a simple application so that I could focus on the XNA-specifics. I decided to make a Code Breaker game, also known as Master Mind. It's a pretty straight-forward game. There is no increasing difficulty curve and no fancy AI to be written, so it seemed like an excellent first attempt.

Prerequisites

Before I could start I needed to install the necessary software. I already had Visual Studio 2008 and the Zune Software. Actually you don't need the full Visual Studio 2008. You can use the completely free Visual C# 2008 Express Edition instead.

You do need to sign up as a creator in the XNA Creators Club. Having done that, download the XNA Game Studio and install it. Now you're set to go.

Creating content

I gathered a few sound effects and whipped up some graphics for my project. I decided that this is what I wanted the game to look like:

The background would be customizable. I wanted to be able to cycle through the available pictures on your device.

The architecture

I decided on a fairly simple architecture for my game. In my main class (Game1.cs) I deal with the necessities. E.g. I load the sound effects that will be used throughout the application, and I create the GraphicsDeviceManager that will be used to draw to the screen. Apart from that I have two classes that I use. MainGame and MainMenu. As their names imply they either show a menu or the game screen. User input is handled in the main class and it then calls methods in the relevant classes. E.g. if the menu is shown and the user clicks on the Zune "squircle" then a menu item has been selected and the event should bubble to the MainMenu class and not to the MainGame class.

I do more or less the same thing in the Draw-method of the main class. Depending on what is shown on screen individual methods of the two classes are called. And the screen is updated accordingly.

Zune features

I made sure to add at least some of the Zune-specific features.

Pictures

Like I've already mentioned I added support for getting backgrounds from the pictures on the device.

Music

I also let the player choose a playlist to be running in the background. (Anything already playing when the game is started will continue to play until the player actually changes this setting) I could have given full access to each individual song, but building an interface for that would have been quite ridiculous. Playlists only seemed like a good idea.

Touch interface

After some usability testing I found that in order to get this game working properly I needed to decide if I should incorporate the touch/sweep interface of the Zune 2.0 devices, or if I should settle for "proper" clicks only. I decided on the latter. Mainly because I still needed to rely on the center click, and it was easy to accidentally brush away to the side when trying to click. It also ment that this application would work with all Zunes.

Other design choices

In the beginning I let the player change colors by clicking up/down and going to the next "peg" by clicking right/left.

This turned out to be a bit complicated. It was quite easy to accidentally click up or down as well which would mean that you not only moved to the next peg, but also changed the previous pegs color. I ended up using the following key settings.

Click Right - Move Right
Click Left - Move Left
Center Click - Change color / Click button
Play / Pause - Change color "back"
Back - Menu

Installing

If you want to install and run this little game you need to go through the motions filed under Prerequisites above. Open the project in Visual Studio 2008 and dock your Zune. You should then be able to deploy the release version or debug the application directly on the device.

Download

You can download the project here

Additional screenshots

Here are some additional screenshots of the game:

I hope you enjoy it!

/ Johan

Internet Explorer 8 Released today

JohanSt — Thu, 19 Mar 2009 15:57:00 GMT

Today we release Internet Explorer 8. You'll be able to download it later today from http://www.microsoft.se/ie8

/ Johan

Glimpsing 10 years into the future

JohanSt — Mon, 09 Mar 2009 13:58:27 GMT

Stephen Elop, the president of Microsoft's Business Division recently held a presentation where he allowed the viewers to glimpse 10 years into the future. It's amazing to look at a "real" interpretation of the future interfaces and technologies rather than the standard hollywood stuff. - You know the holographic projections that for some strange reason still use slow-typing green text on a black background. The video is really inspirational, and what I really liked was how Stephen, after showing the video, went through all the technologies show in the clip and discussed how they could be accomplished.

If you're in a rush you can skip the first part of the presentation and jump to directly to the video at about 14½ minutes. But I really recommend watching the whole thing, especially the final rundown where he discusses the underlying technologies.

/ Johan

Running a scheduled PowerShell script on a pre-Vista OS

JohanSt — Thu, 05 Mar 2009 13:00:50 GMT

It's been fun to see the reception that PowerShell has gotten. People are using it for tons of different stuff and the ingenuity in some of the scenarios is quite impressive. There is one thing, however, that you should be aware of. PowerShell is not an ideal candidate for execution through the Task Scheduler.

Problem

A long-running PowerShell script is being executed using the Task Scheduler. At apparently random intervals the script will fail with a System.Management.Automation.PipelineStoppedException.

Cause

Looking up the exception on MSDN might not shed any light on the situation unless you know what is happening.

The exception thrown when a cmdlet or a Windows PowerShell provider attempts to write to the pipeline or perform a number of other Windows PowerShell operations after the pipeline has been terminated. The pipeline could have been terminated before the call was made or during the call.

When the Task Scheduler executes a PowerShell script it needs to use the .NET Framework. When running the .NET Framework it uses the console session (Session 0). This problem occurs when someone is logged on to the root console (either physically or using Terminal Services) and logs off while the script is executing.

Repro

You can easily reproduce this by creating a small script that sleeps for a while. Schedule the script to run immediately and then log off. To illustrate I've used the following script:

function Main()
{
     Start-Transcript "C:\Test\Psjob.txt"
     'Start-Sleep'
     Start-Sleep (2*60)
     'Ready'
     Stop-Transcript
}

Main

When correctly executed this script will generate a transcript that looks like this:

**********************
Windows PowerShell Transcript Start
Start time: 20090305111600
Username : DOMAIN\SYSTEM
Machine : MACHINENAME (Microsoft Windows NT 5.2.3790 Service Pack 2)
**********************
Transcript started, output file is C:\Test\Psjob.txt
Start-Sleep
Ready
**********************
Windows PowerShell Transcript End
End time: 20090305111800
**********************

However, if I log off the console while the script is executing I get the following transcript:

**********************
Windows PowerShell Transcript Start
Start time: 20090305114300
Username : DOMAIN\SYSTEM
Machine : MACHINENAME (Microsoft Windows NT 5.2.3790 Service Pack 2)
**********************
Transcript started, output file is C:\Test\Psjob.txt
Start-Sleep
**********************
Windows PowerShell Transcript End
End time: 20090305114340
**********************

As you can see the transcript is properly ended as PowerShell shuts down, but if you look at the output in the middle you'll see that we never write "Ready" to the file. This is because the script was prematurely terminated.

Resolution

The quick and easy resolution is to upgrade to Vista, Windows Server 2008 or later, since as of Windows Vista the console session is no longer running in Session 0.

PowerShell - Managing Feature Delegation in IIS7

JohanSt — Tue, 16 Dec 2008 16:51:54 GMT

Perhaps you've read my earlier post on advanced feature delegation using PowerShell and the IIS 7.0 PowerShell Provider. This could actually be called Part II.

The previous post was based on a quick question from a premier customer. He is a never-ending source of intriguing questions and I always enjoy his "just one thing"-type of inquiries. They often force you to think a little extra. Anyway, the other day he came with a question, which (as always) seemed simple enough at a first glance. The goal was to set Feature Delegation for SMTP E-mail And Forms Authentication to "Read Only".

Now, most basic configuration topics have been covered in countless posts, and at first I thought that this shouldn't be so hard. There is, however, a major difference. The SMTP E-mail and Forms Authentication settings are set in the root web.config. Not in applicationhost.config.

If we open up and compare web.config before, and after we change the setting, we'll notice that the change made by the IIS Manager is adding the following into the root <configuration>-block:

Okay, so using what we already know from the old post we should put together a call that looks something like this:

Set-WebConfiguration "/System.Net/mailSettings/smtp" -value @{<something... overrideMode="Deny", maybe?>} -PSPath IIS:\

Please note that this is NOT the correct syntax. It is close, but no cigar. What we need to do is:

Correctly set the overrideMode. The above sample is not the way to do it.
Make sure the changes are saved to web.config

So, the final, (and correct,) call to the Set-WebConfiguration cmdlet would look like this:

Set-WebConfiguration //System.Net/mailSettings/smtp -metadata overrideMode -value Deny -PSPath MACHINE/WEBROOT

The Forms Authentication call would look like this:

Set-WebConfiguration //System.Web/authentication -metadata overrideMode -value Deny -PSPath MACHINE/WEBROOT

I've searched quite extensively for samples on this and so far I've found none. So I thought someone should write it down. :)

Happy Holidays! / Johan

Office Automation

JohanSt — Tue, 14 Oct 2008 16:15:30 GMT

A very common scenario that keeps sprouting new heads like a hydra is Office Automation. Let me start by saying that this is not supported.

There is a KB-article number 257757 discusses this, and clearly states the following:

Microsoft does not currently recommend, and does not support, Automation of Microsoft Office applications from any unattended, non-interactive client application or component (including ASP, ASP.NET, DCOM, and NT Services), because Office may exhibit unstable behavior and/or deadlock when Office is run in this environment.

Why is this not supported?

The crucial thing to consider is the fact that Microsoft Office is designed to be an end-user, single client product. Automating Microsoft Office in a client application, using the identity and security context of the logged on client is supported, but unattended execution is not.

There are a number of things that can go wrong and article 257757 lists most of them. For arguments sake, let's consider the following scenario:

Your server application starts up and uses CreateObject to create an instance of MS Word. Word tries to read the settings of the current client and since your application is using the Network Service account this presents an immediate problem. There are, for example, certain methods that rely on a default printer being installed. All this can lead to serious problems. You then need to monitor the server, since Office might show a modal dialogue for some reason. The "install on first use" feature of MSI might also kick in, prompting the client to install additional features. All this would hang the current thread. Also, running one instance of Word, might be okay, but what happens when you get 100 more or less concurrent requests and each request starts up an instance of Word? MS Word is not a server, it's an excellent piece of single-client software.

So what should I do instead?

Actually, most of your problems can be resolved using HTML. I've seen applications where lines and lines of complex code were used for generating an Excel document, when it could just as well have been made using a standard HTML-table. Excel would have no problems whatsoever reading a table like the one below:

<table>
    <tr>
        <td><b>Person</b></td>
        <td><b>Age</b></td>
    </tr>
    <tr>
        <td>Pete</td>
        <td>30</td>
    </tr>
    <tr>
        <td>Claire</td>
        <td>40</td>
    </tr>
    <tr>
        <td>Average age</td>
        <td>=AVERAGE(B2:B3)</td>
    </tr>
</table>

As you can see all formatting, and even the functions, would be interpreted correctly.

Obviously Word is great with HTML as well, so most formatting issues can quite easily be resolved using this approach.

So how do I do this?

The solution is quite easy. In order to generate an Excel document all you need to do is to create a table with the data you want, and add the following two lines to the Page_Load event of your page:

Response.ContentType = "application/vnd.ms-excel";
Response.AddHeader("Content-Disposition", "attachment; filename=Data.xls;");

The first line, sets the returned ContentType to Excel, and the second changes the filename of the returned file. This means that the client will be prompted to Open/Save the file, and the filename will be set to "Data.xls" rather than "Default.aspx" (or whatever your original document may be called.)

If you wanted to return a word document you'd set the ContentType to "application/msword" instead.

One little thing to consider is the fact that by default you're probably adding a lot of redundant information to your webpage. You might want to remove all excessive HTML from the page so that you remove all unnecessary headers, stylesheets, viewstate, etc. For an example, please consider the code below:

Response.Clear();
Response.Buffer = true;
Response.ContentType = "application/vnd.ms-excel";
Response.AddHeader("Content-Disposition", "attachment; filename=Data.xls;");
this.EnableViewState = false;
System.IO.StringWriter oSw = new System.IO.StringWriter();
System.Web.UI.HtmlTextWriter oHtml = new System.Web.UI.HtmlTextWriter(oSw);
Table oTable = new Table();
for (int ctr = 1; ctr <= 10; ctr++)
{
    TableRow oRow = new TableRow();
    TableCell oCell1 = new TableCell();
    oCell1.Text = ctr.ToString();
    TableCell oCell2 = new TableCell();
    oCell2.Text = (ctr*10).ToString();
    oRow.Cells.Add(oCell1);
    oRow.Cells.Add(oCell2);
    oTable.Rows.Add(oRow);
}
oTable.RenderControl(oHtml);
Response.Write(oSw.ToString());
Response.End();

Security issues in Office 2007

If you're running Office 2007 you might come across the following error message:

"The file you are trying to open, 'Data.xls', is in a different format than specified by the file extension. Verify that the file is not corrupted and is from a trusted source before opening the file. Do you want to open the file now?

This is completely by design. In this example Excel has noticed that though the file is named .xls it does in fact contain html, so it's warning us about this inconsistency. We might bypass this by actually changing the file extension to htm or, perhaps saving the data as comma separated values (csv) which Excel also supports, but both options would be opened directly in the browser rather than passed on to Excel, so this is a design consideration we have to take into account.

There is really no way of bypassing this security feature other than manually disabling it, client-side, using a registry hack. (The key is HKCU\Software\Microsoft\Office\12.0\Excel\Security\ Add a DWORD named xtensionHardening and set it to 0.) For obvious reasons this is not something that is generally recommended.

Other options

There are also third-party products available for generating MS Office-compatible documents, but there are also some articles in the knowledge base on the subject. For example this little gem which can easily be adopted to ASP.NET as well:

How to use ASP to generate a Rich Text Format (RTF) document to stream to Microsoft Word

/ Johan

PowerShell - Advanced configuration editing in IIS7

JohanSt — Thu, 02 Oct 2008 16:28:18 GMT

I've written a lot of PowerShell posts lately and here's another one. :-)

I got a question from one of the account managers if it was possible to alter the FTP Authorization Rules for a specific folder on his IIS.

The appcmd for the operation was

appcmd.exe set config "FTPFolder" -section:system.ftpServer/security/authorization /+"[accessType='Allow',users='*',roles='*',permissions='Read, Write']" /commit:apphost

He wanted to know if there was a PowerShell equivalent. Sure, you could use the appcmd directly from powershell, but if you're going to use the command line for everything, then what's the use of PowerShell?

The section in applicationhost.config that we want to edit is the following:

System Specs

As you might have noticed from the screenshots already he was Running Windows 2008 with FTP 7 installed. He also had the IIS 7.0 PowerShell Provider installed.

Troubleshooting

There are a lot of cool things you can do with the WebSites using the IIS7 PowerShell provider. Below is a sample copied from iis.net. To use it properly, go to an IIS directory such as IIS:\DemoSite\DemoApp:

$winAuth = Get-WebConfiguration -filter /system.webServer/security/authentication/windowsAuthentication
$winAuth.enabled = $false
$winAuth | set-Webconfiguration -filter /system.webServer/security/authentication/windowsAuthentication -PSPath IIS:\ -location "DemoSite/DemoApp"

So judging from the sample above we should simply have to set anonymousAuthentication.enabled = $true or something like that. Unfortunately this isn't the case. If you try to access the section of applicationhost.config that we're attempting to edit (see above) you will see that we don't have any applicable properties for the object. $ftpAuth = Get-WebConfiguration -filter /system.ftpServer/security/authorization will not throw any exceptions, but using TAB to cycle through the properties of $ftpAuth will show us no immediate properties or methods of use.

Solution

The solution in this scenario is to use the Add-WebConfiguration cmdlet. The applicationhost.config can easily be translated to a valid Add-WebConfiguration call:

Add-WebConfiguration "/system.ftpServer/security/authorization" -value @{accessType="Allow";users="?";permissions=3} -PSPath IIS:\ -location FTPFolder

The only thing that sticks out as being out of the ordinary is the permissions=3 setting. Why isn't it "Read, Write"? Actually, passing "Read, Write" as a parameter will not work. It will leave the permissions setting blank. "Read" or "Write" only will work, but not both together. I've tried figuring out if there's a valid way of passing both arguments, but so far I've drawn blanks. I've tried putting them in an array {"Read";"Write"}, passing them as "ReadWrite", "Read;Write", "Read+Write" etc. but the only way I've found so far is to pass the value 3 which is the obvious sum of the two enum values for Read and Write.

Later! / Johan

PowerShell - Editing permissions on a file or folder

JohanSt — Wed, 01 Oct 2008 15:52:49 GMT

I got the following question from a reader the other day:

I've been trying to figure out how to change permissions on a folder in PowerShell. I've looked at the Get-Acl and Set-Acl, but I can only use them to copy the settings from a pre-existing object. How do I manually configure permissions?

This is actually a quite common question, so I thought I'd write a quick post on the subject.

Get-Acl and Set-Acl

To quote the PowerShell documentation "Get-Acl gets the security descriptor for a resource, such as a file or registry key." while "Set-Acl changes the security descriptor of a specified resource, such as a file or a registry key." In other words; if you want Folder_A to have the exact same permissions as Folder_B, then you simply copy the Access Control List (ACL) of Folder_B and "paste" it onto Folder_A.

$Acl = Get-Acl "C:\Folder_B"
Set-Acl "C:\Folder_A" $Acl

So far, so good.

Changing the ACL

Okay, so you want to change the ACL. Here's some sample code for how to do that:

New-Item -type directory -path C:\MyFolder
$Acl = Get-Acl "C:\MyFolder"
$Ar = New-Object  system.security.accesscontrol.filesystemaccessrule("username","FullControl","Allow")
$Acl.SetAccessRule($Ar)
Set-Acl "C:\MyFolder" $Acl

So, first we create a new folder. We then copy the ACL of that folder. We then create a new AccessRule that gives "username" full control. We then add this AccessRule to the ACL, and finally we reapply the new, altered ACL to the folder.

If we wanted to we could also have used $Acl.RemoveAccessRule($Ar) or possibly $Acl.RemoveAccessRuleAll() as well.

/ Johan

PowerShell - Automatically organizing your mp3-collection

JohanSt — Tue, 23 Sep 2008 14:53:23 GMT

Is your mp3-collection perfectly sorted? I know mine wasn't. I thought I'd address that while creating a pretty good demo-script to show some basic filtering, script structures, etc. Since I also wanted the script to extract the metadata from every file in my collection I had to use the Shell.Application Com-object as well. All in all I think it turned out pretty well.

NOTE:

This script is provided completely as is. It is not an official product in any way and I take no responsibility for any harm it may cause.

Please note that certain metadata editors that integrate directly into the Shell may cause some severe confusion in the metadata, so I'd recommend disabling them before trying. Off course I would also recommend making a backup copy of the music library before surrendering it to the mercy of PowerShell.

If you're new to PowerShell and haven't done so already I'd recommend looking at my earlier posts on the subject:
PowerShell - An introduction, Part I
PowerShell - An introduction, Part II

The script

Param([String] $Folder)
$INVALIDCHARS = [System.IO.Path]::GetInvalidPathChars() + "/", "\", "*", "?", ":"
$MUSICATTRIBS = "*.m4a", "*.m4b", "*.mp3", "*.mp4", "*.wma", "*.flc"
$PLAYLISTATTRIBS = "*.m3u", "*.zpl"
$ALLATTRIBS = $MUSICATTRIBS + $PLAYLISTATTRIBS
$PLAYLISTSFOLDER = "Playlists"
$objShell = New-Object -ComObject Shell.Application
$iTotalFiles = 0
$iCurrentFile = 0

function checkFolderName($FolderName)
{
	# Make sure the folder doesn't contain any invalid characters

	if(!$FolderName) { $FolderName = "Unknown" }
	$INVALIDCHARS | % {$FolderName = $FolderName.replace($_, "")}
	return $FolderName
}



function MoveFiles($startDir)
{
	Write-Host "Indexing playlists..."
	
	# Do a recursive DIR in the starting directory, include everything with the attributes of a playlist.
	# Filter the result by excluding everything that is a Container (folder)
	$dirResult = get-childitem -force -path $startDir -recurse -include $PLAYLISTATTRIBS | where{! $_.PSIsContainer}
	if($dirResult)
	{
		Write-Host "Moving playlists..."

		foreach($dirItem in $dirResult)
		{
			move-item $dirItem.FullName ($PLAYLISTSFOLDER + "\" + $dirItem.Name)
		}
	}
	Write-Host "Indexing music..."

	# Do a recursive DIR in the starting directory, include everything with the attributes of a music file.
	# Again we filter the result by excluding everything that is a Container.
	$dirResult = get-childitem -force -path $startDir -recurse -include $MUSICATTRIBS | where{! $_.PSIsContainer}
	
	# Make a note of how many hits we got, so we can show the progress to the client.
	$iTotalFiles = $dirResult.Count
	if($dirResult)
	{
		foreach($dirItem in $dirResult)
		{
			# Up the counter for how many files we've processed so that we can show progress
			$iCurrentFile +=1

			# Get the metadata for the file
			$fileData = getMP3MetaData($dirItem.FullName)
			
			# Find the path where we the song should be stored
			$ArtistPath = $fileData["Album Artist"]
			if (!$ArtistPath) { $ArtistPath = $fileData["Artists"] }
			if (!$ArtistPath) { $ArtistName = "Unknown" }

			# Make shure it's a valid path
			$ArtistPath = checkFolderName($ArtistPath)
			$AlbumPath = checkFolderName($fileData.Album)
			$ArtistPath = join-path $startDir $ArtistPath 
			$AlbumPath = join-path $ArtistPath $AlbumPath

			# Check if the file should be moved
			if($dirItem.DirectoryName -ne $AlbumPath)
			{
				if(!(test-path $ArtistPath)) # If the Artist folder doesn't exist
				{
					MKDIR $ArtistPath | out-null
				}
				if(!(test-path $AlbumPath)) # If the Album folder doesn't exist
				{
					MKDIR $AlbumPath | out-null
				}
				move-item $dirItem.FullName ($AlbumPath + "\" + $dirItem.Name)
			}

			# Show progress
			$percentage = ([int](($iCurrentFile / $iTotalFiles)*100))
			cls
			Write-Host "$percentage% ($iCurrentFile files of $iTotalFiles) complete"
		}
	}
}


function removeEmptyFolders($startDir)
{
	# Get all folders and subfolders
	$dirResult = Get-childitem $startDir -recurse | where{$_.PSIsContainer}
	if($dirResult)
	{
		foreach($dirItem in $dirResult)
		{
			# If the folder is empty (Doesn't contain any music or playlists) it should be deleted
			if(isfolderempty($dirItem.FullName))
			{
				# Check if the path is still valid, we may have deleted the folder already recursively.
				if(Test-Path $dirItem.FullName)
				{
					remove-item -path $dirItem.FullName -force -recurse
				}
			}
		}
	}
}

function removeOtherFiles($startDir)
{
	# Get all non-music files (except *.jpg-files) and remove them 
	$dirResult = Get-childitem $startDir -recurse -exclude ($ALLATTRIBS + "*.jpg") | where{! $_.PSIsContainer}
	if($dirResult)
	{
		foreach($dirItem in $dirResult)
		{
			if(Test-Path $dirItem.FullName)
			{
				# Make sure the file still exists, and hasn't been deleted already
				remove-item -path $dirItem.FullName -force
			}
		}
	}
}

function isFolderEmpty($folderPath)
{
	# Search for any remaining music items in the folder.
	if(Test-Path $folderPath)
	{
		$dirChildren = get-childitem -force -path $folderPath -recurse -include $ALLATTRIBS | where{! $_.PSIsContainer}
		if($dirChildren -ne $null)
		{
			return $false
		}
		else
		{
			return $true
		}
	}
	else
	{
		# No use trying to delete the folder if it doesn't exist
		return $false
	}
}

function getMP3MetaData($path)
{
	# Get the file name, and the folder it exists in
	$file = split-path $path -leaf
	$path = split-path $path
	$objFolder = $objShell.namespace($path)
	$objFile = $objFolder.parsename($file)
	$result = @{}
	0..266 | % {
		if ($objFolder.getDetailsOf($objFile, $_))
		{
			$result[$($objFolder.getDetailsOf($objFolder.items, $_))] = $objFolder.getDetailsOf($objFile, $_)
		}
	}
	return $result
}

# MAIN FUNCTION

# If no argument was passed, see if anything was piped to the function instead.
if (!$Folder)
{
	$input | % { $Folder = $_ }
}

# Check if the path is valid, if not reset it to ""
if ($Folder) { if (!(Test-Path -path $Folder)) { $Folder = "" }}

# Since no valid path was given, prompt the user for a proper path.
while (!$Folder)
{
	"Please enter the full path to the folder where you wish to rearrange your MP3's"
	$Folder = Read-Host
	if ($Folder) { if (!(Test-Path -path $Folder)) { $Folder = "" }}
}

# Make sure we have a folder to store the playlists in.
$PLAYLISTSFOLDER = join-path $Folder $PLAYLISTSFOLDER
if(!(test-path $PLAYLISTSFOLDER)) # If the Playlists folder doesn't exist
{
	MKDIR $PLAYLISTSFOLDER | out-null
}

# Now that everything is ready, begin rearranging the files.
MoveFiles($Folder)
removeEmptyFolders($Folder)
removeOtherFiles($Folder)

Running the script

So, how do you run this script?

It's quite easy. The first step would be to copy it into notepad save it with a .ps1 extension and open up PowerShell

Execution policy

If this is the first time you try to execute a script from PowerShell you will most likely have to change the execution policy for PowerShell. I'd recommend setting it to RemoteSigned. This (obviously) means that all remote scripts must be signed or PowerShell will refuse to execute them. To change the execution policy simply type:

Set-ExecutionPolicy RemoteSigned

That's it.

If you're uncertain what your execution policy setting is you simply type:

Get-ExecutionPolicy

Okay, having changed that we can now execute the script. If you saved the script in a folder that is in your system path you only need to type the filename of the .ps1 file. (The .ps1 is optional) So, if you saved it as RearrangeMP3s.ps1 you'd write RearrangeMP3s and press enter

Breakdown

Okay, so let's take a look at the separate parts of the script.

Initial variables and constants

First we declare the parameters we're going to use. In this case it's just one: $Folder.

After that we have some constants. You may want to change $MUSICATTRIBS or $PLAYLISTATTRIBS to include, or remove, extensions based on your needs. $PLAYLISTSFOLDER is a constant which indicates in which subfolder you wish to store your playlists.

Finally we have some variables. $objShell is used for file system operations, and rather than creating a new object all the time I'd rather have a global one. $iTotalFiles and $iCurrentFile are two integers used to create a progress indicator.

Functions

After the initial variables come the sub functions called by the main function. To see what each individual function does I'd recommend looking at the comments. I've prioritized readability over minimum number of keystrokes, so yes, instead of writing:

foreach($dirItem in $dirResult)
{
	move-item $dirItem.FullName ($PLAYLISTSFOLDER + "\" + $dirItem.Name)
}

I could have written

$dirResult | % { mv $_.FullName ($PLAYLISTSFOLDER + "\" + $_.Name) }

But in this case I think that would have been counterproductive.

The main function

Finally we have the main function. It is written last in the script. Why is that? Well, actually PowerShell is the most sequential language you'll ever come across. It will not bother to check the entire script for function declarations. Instead it will go through the script one line at a time. This means that the following script will not work:

SayHello

function SayHello
{
	Write-Host "Hello World"
}

When you call SayHello in the sample above the function has not yet been declared, so PowerShell will throw an exception in your face saying that SayHello is unrecognized.

Footnote

There is actually a supported way of achieving the very same thing, but where's the fun in that?

Open up Windows Media Player
Go to Options -> Library and tick the checkbox for "Rearrange music in rip folder"
Go to the "Rip Music"-tab
Make sure the path is correct in the "Rip music to this location"-section
Click OK
Select all files in the library
Right-click and choose the "Advanced Tag Editor"-option
Do not change anything. Simply click Apply
Windows Media Player will now rearrange all the files using your Rip settings.

/ Johan

PowerShell - An introduction, Part II

JohanSt — Mon, 25 Aug 2008 14:22:12 GMT

This is a continuation of part I. If you haven't read it I suggest at least going through the summary at the end of the post.

Working with Drives

In your basic command prompt you have the normal file system drives that you use to access your hard-drive, floppy, DVD, USB-stick, network shares etc. PowerShell uses this established interface to access other data sources as well. The following providers are accessible by default through drives in PowerShell 1.0:

Registry (HKLM: or HKCU:)
Certificate store (Cert:)
Environment variables (Env:)
Aliases (Alias:)
Functions (Function:)
Variables (Variable:)

When you install extensions for like the one fore IIS, etc. you usually get another drive accessible where you can look at the current webapplications and check their properties.

Navigating

Selecting one of the PowerShell drives is easy. It is almost like in MS-DOS.

PS C:\>CD HKLM:
PS HKLM:\> DIR

You can use the same commands as you'd do in MS-DOS, such as CD, DIR, MKDIR, etc.

To get a certain property for a registry key, (or a regular folder for that matter) you use the Get-ItemProperty cmdlet.

PS HKLM:\> CD system
PS HKLM:\system> CD CurrentControlSet\Control
PS HKLM:\system\CurrentControlSet\Control> Get-ItemProperty CrashControl

PSPath           : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\system\CurrentControlSet\Control\CrashControl
PSParentPath     : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\system\CurrentControlSet\Control
PSChildName      : CrashControl
PSDrive          : HKLM
PSProvider       : Microsoft.PowerShell.Core\Registry
AutoReboot       : 1
CrashDumpEnabled : 2
Overwrite        : 1
LogEvent         : 1
DumpFile         : C:\Windows\MEMORY.DMP
MinidumpDir      : C:\Windows\Minidump

All in all it's more or less the same working with Aliases or Environment variables. Note the following difference in output, though:

PS Env:\> Dir Env:PROCESSOR_ARCHITECTURE

Name                           Value
----                           -----
PROCESSOR_ARCHITECTURE         AMD64

PS Env:\> $Env:PROCESSOR_ARCHITECTURE
AMD64
PS Env:\>

Did you note how putting a $-sign before the variable name gave us the value of the variable instead of a complete object? This brings us quite nicely to the subject of variables.

Variables

Variables aren't that tricky to work with. They do not need to be pre-initiated. They do, however, require a $ to be identified as variables. Otherwise you get the following error message:

PS C:\> $Name = "Johan"
PS C:\> Name = "Johan"
The term 'Name' is not recognized as a cmdlet, function, operable program, or script file. Verify the term and try again.
At line:1 char:5
+ Name <<<< = "Johan"
PS C:\>

To display the value of a variable you simply type the name of the variable and PowerShell kindly returns the contents

PS C:\> $Name
Johan

When it comes to strings, you can choose for yourself if you want to use double quotes or single quotes, but there is one minor difference:

PS C:\> $str1 = "Hello"
PS C:\> $str2 = "World!"
PS C:\> $strCombo1 = "$str1 $str2"
PS C:\> $strCombo2 = '$str1 $str2'
PS C:\> $strCombo1
Hello World!
PS C:\> $strCombo2
$str1 $str2
PS C:\>

As you can see, a variable within double quotes will be evaluated, while within single quotes it will be interpreted literally.

Apart from strings you can also store integers and dates. Integers do not need delimiters and dates are most easily created using the Get-Date function

PS C:\> $birthday = 11/15/1973

Arrays

Declaring Arrays is a breeze. PowerShell will automatically Split any comma-separated sequence of values so creating a arrays is as easy as this:

PS C:\> $Colors = "Red", "Green", "Blue"

The arrays are also dynamic by default, so adding values to them is a simple matter as well.

PS C:\> $Colors += "Cyan", "Magenta", "Yellow"

Creating two-dimensional arrays are done by declaring each row of the array as a one-dimensional array and then adding them to another array:

PS C:\> $arr1 = 1, 2, 3
PS C:\> $arr2 = "A", "B", "C"
PS C:\> $arr3 = $arr1, $arr2
PS C:\> $arr3[0][0]
1
PS C:\> $arr3[1][1]
B
PS C:\>

User input

Finally, to get dynamic user input you can use the Read-Host cmdlet.

$Name = Read-Host "What is your name?"

Summary

Okay, so to summarize we have now covered the following:

PowerShell uses system drives to access the registry, environment variables, etc.
Variables are preceded with a $
To access the value of a variable, simply type it's name.
Arrays are comma-separated lists of variables
User input can be handled by the Read-Host cmdlet

Next post will probably cover some script logic such as if-clauses, looping, etc. After that I think it will be time to post some sample scripts.

/ Johan

Back in business

JohanSt — Fri, 22 Aug 2008 18:00:00 GMT

Okay, after a summer break that no doubt got a little longer than expected I am now I'm now back in the saddle again.

Next post will most likely be a follow-up on the PowerShell post I wrote a while back. Stay tuned.

/ Johan

The first thing you should do after setting up an IIS7-server

JohanSt — Thu, 29 May 2008 15:14:58 GMT

As you're probably well aware IIS7 relies heavily on a master configuration file named ApplicationHost.config. Now, instead of using a history file like IIS6 did, IIS7 will check this file every 2 minutes to see if it's changed. If the file has been updated, then a backup copy of the old .config will be stored in the inetpub\history-directory.

Configuration history

This is really cool and a very useful feature. There's only one little caveat in my opinion and that's the fact that by default IIS7 will only save up to 10 backups. All the other backups will be discarded. This means that if you're spending 20 minutes fiddling with the settings in ApplicationHost.config you will quite likely have overwritten all your old backups. Now, if something goes wrong at that point and you decide to spend another 20 minutes trying to fix this problem by further editing the ApplicationHost.config you will surely have overwritten the backups.

In a worst case scenario you might even have gone from Healthy Server to Unstable Server to Server Down and you now find that you aren't even able to restore it to the Unstable state.

My suggestion is to increase the default number of backups to 100, or why not 1000? You can then feel certain that any backups taken aren't overwritten when you try to undo the very same changes. Here's how you do it:

Go to %windir%\system32\inetsrv\config\
Open applicationHost.config in notepad
Locate the system.applicationHost section group
Add the following item to the section group:
<configHistory maxHistories="100" period="00:02:00" />
(Naturally the maxHistories value can be whichever value you prefer)
Save

To restore one of the previous configurations, simply go to inetpub\history\cfgHistory_[nnnnnnnnnn] and copy the config from there, or use appcmd. (See below)

Backup using appcmd

Now that we've done this there is one more thing I'd strongly recommend doing:

Open up a command-line with administrative priviliges
Go to %windir%\system32\inetsrv
Run "appcmd add backup "[Name of your backup]""

What you've done now is to create a manual backup of your current config. It's always good to have a decent (and persistent) restore point to return to.

If you wish to restore this backup you simply use the following syntax: "appcmd restore backup "[Name of your backup]"". You can also delete a backup using the "delete" keyword, or you can list all backups, including the ones generated by the Configuration history, like this: "appcmd list backup"

Okay, so now we have made sure we have some extensive automatic backups in place which might save us a lot of headaches later on.

Later! / Johan

PowerShell – An introduction, Part I

JohanSt — Wed, 28 May 2008 14:53:15 GMT

Over the past months I’ve been holding a couple of workshops on PowerShell. It’s quite an interesting topic, but not something I originally thought would be suitable for this blog and its web-development theme. Still, having given this a bit more thought I decided that there are so many useful features and cmdlets that could be of great value when hosting a web site or two that I should reconsider.

This first post will be a brief introduction for those who’ve never touched PowerShell before. Later posts will probably be more hands-on and less theoretical. I’ll gladly receive any thoughts or comments you may have.

Installing

PowerShell can be downloaded from the Microsoft web site. Simply point your browser to http://www.microsoft.com/powershell and you’ll be set in no time. Apart from the executables you’ll find a lot of useful links, like the one to the PowerShell team blog and the IIS 7.0 PowerShell Provider.

Basic syntax

Okay, having installed PowerShell we’re now ready to try some basic commands.

Echoing text

Any object not used in an assignment or operation will be echoed to the screen. This means that if we simply type a string like this:

PS C:\>“Hello World”
Hello World

It will be displayed on the screen. (Without the quotes.) The same goes for numbers, so by typing an equation you’ll be given the result:

PS C:\>1 + 1
2

If you type in an array PowerShell will echo it one line at a time:

PS C:\>“foo”, “bar”, “bletch”, “fum”
foo
bar
bletch
fum

Basically you could say that any object will, unless told otherwise, be displayed on screen using its default formatting. The default formatting is defined by the object itself.

Calling executables

Running an executable is more or less as easy as in the regular command shell. At least if the .exe is in the PATH. For any executable not in the PATH you'll need to write the full path of the .exe for it to successfully execute. This may seem like a hassle, but this behavior is by design and was implemented in order to prevent you from accidentally executing an .exe.

Cmdlets

Cmdlets (pronounced "command-lets") are PowerShell-specific commands. They follow a strict naming standard of verb-noun, (for example "get-date" or "set-location").

Get-Help

This a command that you'll use quite a lot. Get-Help <cmdlet> will display some generic information about the cmdlet in question. Using various switches such as "-detailed", "-examples" or "-full" you can then get more detailed information on the cmdlet in question. For example:

Get-Help Get-Date -examples

This will show you the generic information on get-date as well as some sample code.

Set-Location

Set-Location is the PowerShell equivalent to the old MS-DOS command "CD". Some examples are:

Set-Location ..
Set-Location C:\Windows\
Set-Location D:

Aliases

Actually you can write CD in PowerShell too. By default PowerShell will set up an alias for you that points to the Set-Location cmdlet. This means that CD is just another name for Set-Location. If you want you can create your own aliases using the New-Alias cmdlet:

New-Alias gh Get-Help

This creates an alias named gh that points at Get-Help. If you type Get-Help gh PowerShell will be smart enough to show you the documentation for the Get-Help cmdlet.

Object orientation

One of the cool things about PowerShell is that it is object oriented. This means that all the cmdlets return objects, which can, in turn, be passed on to another cmdlet. To illustrate this, let’s look at the DIR-command.

Using DIR

You can use DIR in PowerShell, but it is actually an alias for the get-childitem cmdlet. How can you tell? Use the get-command cmdlet:

PS C:\> get-command dir

CommandType     Name                                                Definition
-----------     ----                                                ----------
Alias           dir                                                 Get-ChildItem

Since DIR in PowerShell is not the same thing as DIR in MS-DOS that means you can’t use all the switches you might be used to, for example DIR /w will not work.

So, what happens when you type DIR and press enter?

Well, first of all the get-childitem cmdlet is run. It looks in your present working directory, creates a collection of all the items there, and returns it. You can think of this collection as a big Excel Spreadsheet if you like.

Once the collection is returned to PowerShell it will be displayed on screen. This is because the default behavior for PowerShell when it comes to any object is to display it on screen. We saw this earlier with strings, integers and arrays.

The collection will be displayed using a default presentation. This does not mean you will see all the information the object contains. If I run Get-ChildItem (DIR) on a folder I’ll get the standard information. Like so:

PS C:\Test> dir

    Directory: Microsoft.PowerShell.Core\FileSystem::C:\Test

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---        2008-05-28     11:25      45568 Document.doc
-a---        2008-05-26     10:00    8993629 Satisfaction.mp3
-a---        2008-05-28     11:25       1599 Text.txt

Even though I get only text on screen the actual object returned is much more complex. I can demonstrate this by piping the object into another cmdlet.

Piping

A simple MS-DOS piping example would be type file.txt | more. Actually you can use more in PowerShell as well, but let’s look at another, more interesting cmdlet that we could pipe the object into; Format-List.

Format-List

By default the output of Get-ChildItem is formatted in a table, but let’s say we want it as a list instead? No problem. Simply use the following syntax:

PS C:\Test> dir | format-list

    Directory: Microsoft.PowerShell.Core\FileSystem::C:\Test

Name           : Document.doc
Length         : 45568
CreationTime   : 2008-05-28 11:22:53
LastWriteTime : 2008-05-28 11:25:26
LastAccessTime : 2008-05-28 11:25:26
VersionInfo    :

Name           : Satisfaction.mp3
Length         : 8993629
CreationTime   : 2008-05-28 11:24:33
LastWriteTime : 2008-05-26 10:00:46
LastAccessTime : 2008-05-28 11:24:33
VersionInfo    :

Name           : Text.txt
Length         : 1599
CreationTime   : 2008-05-28 11:22:53
LastWriteTime : 2008-05-28 11:25:35
LastAccessTime : 2008-05-28 11:22:53
VersionInfo    :

We still have the default information, only presented in a different manner. Still, there is a lot more information we could get, by using the –property switch. By simply using the following syntax: dir | format-list –property * we get all the properties of each and every item in the directory. The output is quite extensive:

PS C:\Test> dir | format-list -property *

PSPath            : Microsoft.PowerShell.Core\FileSystem::C:\Test\Document.doc
PSParentPath      : Microsoft.PowerShell.Core\FileSystem::C:\Test
PSChildName       : Document.doc
PSDrive           : C
PSProvider        : Microsoft.PowerShell.Core\FileSystem
PSIsContainer     : False
Mode              : -a---
Name              : Document.doc
Length            : 45568
DirectoryName     : C:\Test
Directory         : C:\Test
IsReadOnly        : False
Exists            : True
FullName          : C:\Test\Document.doc
Extension         : .doc
CreationTime      : 2008-05-28 11:22:53
CreationTimeUtc   : 2008-05-28 09:22:53
LastAccessTime    : 2008-05-28 11:25:26
LastAccessTimeUtc : 2008-05-28 09:25:26
LastWriteTime     : 2008-05-28 11:25:26
LastWriteTimeUtc : 2008-05-28 09:25:26
Attributes        : Archive

PSPath            : Microsoft.PowerShell.Core\FileSystem::C:\Test\Satisfaction.mp3
PSParentPath      : Microsoft.PowerShell.Core\FileSystem::C:\Test
PSChildName       : Satisfaction.mp3
PSDrive           : C
PSProvider        : Microsoft.PowerShell.Core\FileSystem
PSIsContainer     : False
Mode              : -a---
Name              : Satisfaction.mp3
Length            : 8993629
DirectoryName     : C:\Test
Directory         : C:\Test
IsReadOnly        : False
Exists            : True
FullName          : C:\Test\Satisfaction.mp3
Extension         : .mp3
CreationTime      : 2008-05-28 11:24:33
CreationTimeUtc   : 2008-05-28 09:24:33
LastAccessTime    : 2008-05-28 11:24:33
LastAccessTimeUtc : 2008-05-28 09:24:33
LastWriteTime     : 2008-05-26 10:00:46
LastWriteTimeUtc : 2008-05-26 08:00:46
Attributes        : Archive

PSPath            : Microsoft.PowerShell.Core\FileSystem::C:\Test\Text.txt
PSParentPath      : Microsoft.PowerShell.Core\FileSystem::C:\Test
PSChildName       : Text.txt
PSDrive           : C
PSProvider        : Microsoft.PowerShell.Core\FileSystem
PSIsContainer     : False
Mode              : -a---
Name              : Text.txt
Length            : 1599
DirectoryName     : C:\Test
Directory         : C:\Test
IsReadOnly        : False
Exists            : True
FullName          : C:\Test\Text.txt
Extension         : .txt
CreationTime      : 2008-05-28 11:22:53
CreationTimeUtc   : 2008-05-28 09:22:53
LastAccessTime    : 2008-05-28 11:22:53
LastAccessTimeUtc : 2008-05-28 09:22:53
LastWriteTime     : 2008-05-28 11:25:35
LastWriteTimeUtc : 2008-05-28 09:25:35
Attributes        : Archive

As long as something is returned it is possible to daisy-chain additional cmdlets onto the previous one. We could, for example, add | more to the command above to divide it into smaller chunks. We could use the Out-File cmdlet to save the result to a textfile, etc.

Summary

What have we covered so far?

PowerShell is object-oriented
By default any object will be displayed on screen, so if you want to know what 2+2 equals, simply type 2+2.
You can call standard .exe applications, etc from PowerShell, but they either need to be in the PATH, or you need to write the full path to them.
PowerShell-specific applications are called cmdlets (“Command-lets”)
Most MS-DOS commands work, but they’re usually aliases to their PowerShell equivalent, so you might not be able to use all the switches you’re used to.
You can use the | to daisy-chain cmdlets together, such as dir | format-list | more

We still haven’t covered things like functions, variables, creating .NET objects, etc. I also plan to write a few samples showing how to use the cool cmdlets to get information from computers on the network, looking for specific files, exporting to Excel, printing, etc. As always I’d be happy to receive feedback and suggestions on things to cover.

/ Johan

Why doesn’t the GC kick in when the worker process is inactive?

JohanSt — Tue, 13 May 2008 16:39:48 GMT

I got the following question in my Getting started with windbg – post and I thought it might be worth posting the replies in a separate article:

Hi Johan,

about those threads with an ID of XXXX, should they go away after certain amount of idle time like 2 minutes? I am trouble shooting an application while the application should sit idle (because of no stimulation), however it is still using quite some CPU time and and threadpool threads, where it should not. So I created an adplus dump when it is sitting idle, and I found that there are many threads with ID XXXX, and I created a dump file again after 15 minutes, again, it still have the exact same XXXX threads. All those threads are completion port threads, and I wonder why they did not get recycled?

Also, when doing a '~*e !clrstack', most of the worker threads and completion port threads are showing "Failed to start stack walk: 80004005". Is there a way to show the stack for those threads, because those are the threads I am interested in.

Thank you!

The GC will be triggered when you’re allocating memory. If your application is inactive, then so is the Garbage Collector. Even if you’re making a few, random allocations they still may not be enough to trigger a GC, so this is not at all unexpected.

The “Failed to start stack walk: 80004005”-error is displayed when the thread did contain a managed stack, but no longer does.

/ Johan

What to do about the slow startup of web services

JohanSt — Wed, 02 Apr 2008 18:05:16 GMT

Due to the architecture of web services and web applications they can be quite slow to start. For example on my Windows 2003-box the initial localhost-call to a simple “Hello World!”-web service takes approximately 8 seconds, while the next request is more or less immediate.

Why is this?

This isn’t news, really. One of the first things we learned in the early beta stages of ASP.NET was that the first request would take a little extra time due to the Just In Time Compilation (JIT). This was a major change from the classic ASP architecture where everything was interpreted rather than compiled. Upon the first request the .aspx / .asmx file will be compiled (JIT’ed) into Microsoft Immediate Language (MSIL) and the resulting .dll will be moved to it’s proper location.

This compilation will occur every time the application pool starts causing a significantly longer response time on the first request compared to the following requests.

Run-time performance over startup performance

It all comes down to prioritizing run-time performance over a quick startup. Which, in my humble opinion, is a sound choice. There are, however, situations where you may feel that this is to your disadvantage. The other day I got a question from a customer who had a web service that was called quite infrequently and to him this meant that with every other request response times would be horrendous, simply because the application got recompiled.

For each case I get I usually do a quick search on the problem description. I don’t really expect to find the solution this way, since I can only assume that the customer has tried this as well. Instead it will likely present me with a list of troubleshooting steps that the customer has already tried as well as possible solutions that didn’t fit. (Off course I still need to verify this with the customer).

This time I didn’t really need to do this since I was pretty sure what the cause was. Still, I was curious to see how common this problem was. I honestly didn’t think this was a very common scenario, but my quick search on the web revealed the contrary. The few hits that I quickly browsed through suggested writing an application that would ping the web service in order to keep it alive. I must admit I find this quite inventive, but I really don’t think it’s the best approach. :)

What to do

So, what is to be done about this? Well, I’ve already mentioned that there are better solutions than pinging the web service. As I told my customer there are basically two good options:

Precompile the assemblies
Change the settings for automatic recycling on the worker process

Precompiling the assemblies

There is a nice little tool called Native Image Generator. (ngen.exe) It will compile the assembly into native images and installs them into the native image cache. The syntax is pretty straight-forward: ngen.exe install [assembly] The problem is that once you’ve changed the assembly you’d have to precompile it again making updates a bit tedious.

There is also a service-version of the ngen called the Native Image Service. More information about ngen can be found at http://msdn2.microsoft.com/sv-se/magazine/cc163808(en-us).aspx

Changing the recycling settings

The reason why my customer was experiencing this problem was because his application pool would time out and recycle itself due to the default settings in the IIS manager. By default the worker process will recycle after 20 minutes of inactivity, so if no one has pinged your application in that time the next request will cause the application to recompile, resulting in a longer response time. By tweaking this setting in the IIS manager to something more suitable to the current rate of requests, like 40 minutes or maybe even 60 he should also be able to find a nice balance. Actually, considering the low load it would probably be a good idea to turn this setting off completely. Here’s how:

Open up IIS Manager
Locate the Application Pool in question
Right-click it and select “Properties”
Go to the “Performance”-tab
Change, or disable the “Idle timeout” setting

/ Johan