ERROR MESSAGE:

This request operation sent to net.pipe://localhost/iisconfigurator did not receive a reply within the configured timeout (00:01:00).  The time allotted to this operation may have been a portion of a longer timeout.  This may be because the service is still processing the operation or because the service was unable to send a reply message.  Please consider increasing the operation timeout (by casting the channel/proxy to IContextChannel and setting the OperationTimeout property) and ensure that the service is able to connect to the client.

 

EXCEPTION DETAILS:

System.TimeoutException was unhandled

Message=This request operation sent to net.pipe://localhost/iisconfigurator did not receive a reply within the configured timeout (00:01:00).  The time allotted to this operation may have been a portion of a longer timeout.  This may be because the service is still processing the operation or because the service was unable to send a reply message.  Please consider increasing the operation timeout (by casting the channel/proxy to IContextChannel and setting the OperationTimeout property) and ensure that the service is able to connect to the client.

  Source=mscorlib
StackTrace:
Server stack trace:
at System.ServiceModel.Dispatcher.DuplexChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at IConfigurator.Deploy(String roleId, WebAppModel webAppModelPath, String roleRootDirectory, String sitesDestinationRootDirectory, String diagnosticsRootDirectory, String roleGuid, Dictionary`2 globalEnvironment)
at Microsoft.WindowsAzure.Hosts.WaIISHost.Program.Main(String[] args)
InnerException:

 

 

Root cause & problem description:

 

1. WaAppAgent.exe starts the Full IIS web role by starting WaIISHost.exe process which loads the Web Role DLL

 2. Due to role being a full IIS host web role, IISconfigurator.exe process also starts along with WaIISHost.exe

 3. IISconfigurator.exe looks the service definition file and finds <Sites> section along with PhysicalDirecotry if set, as below:

 

  <Sites>
<Site name="Web" PhysicalDirectory="appdir">
<Bindings>
<Binding name="Endpoint1" endpointName="Endpoint1" />
</Bindings>
</Site>
</Sites>

 

4. Now IISconfigurator.exe starts checking each file in PhysicalDirecotry for security setup and *if the overall process is taking more than 60 seconds*, it exceeds the default timeout is set for 60 seconds.

 5. This timeout cause the WaWebHost.exe to terminate.

 6. After 2 minutes, WaAppAgent.exe process restarts the WaIISHost.exe and everything happens again from step 1-5.

 

If you are getting this exception, the main issue here is that you have so many files in the Physical directory to be taken care by IISConfigurator.exe, which exceed the 60 seconds default time.  In my test scenario in small VM IISConfigurator.exe took about 90 seconds to process 19000+ files in physical folder. I also found the threshold is about ~15000 files to be processed by IISConfigurator.exe in 60 seconds time. I don’t expect thousands of static files in virtual directories specific to web role. The cloud architecture is designed to move static content out of Windows Azure VM so if you have really thousands (15000+) of files in your web site, it is the time to move static content outside the web role. 

 

Solution(s):

To solve this problem there are two workarounds as below:

 

1. Running Legacy HWC Web Role:

 If you don’t have to use functionalities provided by full IIS role i.e. Multiple web sites, Virtual Directories etc, you can remove the <Sites> section in your Service Definition file as below:

 

    <!--<Sites>
<Site name="Web">
<Bindings>
<Binding name="Endpoint1" endpointName="Endpoint1" />
</Bindings>
</Site>
</Sites>-->

After that, you can repackage the solution and redeploy it. 

 

2. If you must need Full IIS Role then you can use the following workaround:

a)       Please copy all the files (or most of the files) in your virtual directories folder into ZIP files.

b)       When the Role starts expand the ZIP file in OnStart() function

c)        IISConfigurator.exe runs before role start so having thousands of files in ZIP will not cause the total time to exceed 60 seconds, and there will be not exception