Hosting TestController and TestAgents on Azure

Hosting TestController and TestAgents on Azure

Rate This
  • Comments 4

We have recently seen a surge in the number of users trying out the TC TA based testing on Azure. Since this product has a long history which predates Azure, or any other cloud service for that matter, there are some additional not-so-obvious steps needed to get the setup up and running on Azure. In this post I will list all the required steps in detail.

 

Summary:

A typical setup on Azure will host some of the components like TestController, TestAgents, TFS on Azure, while the rest of them like VS/MTM Client will be hosted On-premise. While doing this, there are a few pre-reqs that are required to be done for various components to talk to each other. Once you do the needful to enable all communication channels, everything else works as-is.

Note that similar steps can be followed if the configuration is different than the one discussed. (like when Client is also on Azure, or TC is present On-premise)

 

PreReqs:

Connecting Azure with On-Prem:

  1. When you are hosting components across On-prem and Azure, a VPN must be configured between the On-prem and Azure network so that the 2 networks are connected. Follow this link for detailed instructions.
  2. If the On-Premise machines are domain joint, you will need to add an IPSec exemption so that the bidirectional communication can succeed. This can be avoided if you can unjoin the On-premise machine from the domain and join it to a workgroup instead.

 

TestController:

  1. HostName Resolution settings: Once your machines are setup you will be able to refer to other machines using IP Address in the TestAgent and TestController Configuration tools. However in some cases, this might not be sufficient and you also require hostname based usage. To achieve this you can either setup a name resolution service, or edit machine's hosts.ini file to explicitly add static mappings for IP-Name pair. Typically since the second approach is simpler, you can follow the below steps to do this. Hosts required to be resolved for TestController : All TestAgents, TFS and Client machines.
    1. Open the file %systemdrive%\Windows\System32\Drivers\etc\hosts.ini. For each of the above hosts required to be resolved, add an entry in a format like this : <ip addr> <hostname>
    2. For each of the above machines, add 2 entries – one with the hostname and other with its FQDN name (ex. devmachine.abcdomain.blah.blah.com)
    3. Note that if dynamic IP allocation is setup in VPN configuration, the machine's IP address will get changed everytime it connects to the network. Thus in scenarios like machine reboots the bindings need to be updated everytime. (You can write a small windows startup script to do this automatically). This wont happen in static IP allocation.
    4. For testing purpose ping all the required components using both hostname as well as FQDN name. Now onwards both hostname as well as IP addresses can be used in the configuration tools etc, however the recommendation is to go with hostname service.

  2. Firewall settings:
    1. Enable the firewall settings to "notify when any app is blocked" from "Control Panel\System and Security\Windows Firewall". This may not be set by default, and firewall might silently block the connection request flowing from TestAgent to TestController.
    2. Enable "file and printer sharing" and exempt it from firewall for the network profile currently in use (for ex. Domain/Public/Private). Sometimes the exception is not present for all the network profiles and might end up blocking the connection requests. This can be done through "Control Panel\System and Security\Windows Firewall".
    3. Expose the right set of TCP ports through your Azure Cloud service by configuring an end point for all your components. For ex. when TestController machine is on Azure, port 6901 (default port) must be exposed out through the machine’s cloud service.

  3. Timeout configuration: Once TC TA are able to talk to each other and you are now trying to run an automated test run on the agents, sometimes due to slow network (especially when your TestController and TestAgents are spread across OnPremise and Azure), the testrun initialization might time out. To prevent this, increase the following timeouts specified in QTController.exe.config to thrice their default values : AgentConnectionTimeoutInSeconds, AgentInitTimeout, AgentSyncTimeoutInSeconds, AgentCleanupTimeout. This file can be found in the Test Controller machine : <TestController Install Location>/Common7/Ide/QTController.exe.config.

 

TestAgent:

  1. HostName Resolution settings: As mentioned in the previous section, setup name resolution entries on the TestAgent machine in the below way. Hosts required to be resolved for TestAgent : TestController machine.
    1. Open the file %systemdrive%\Windows\System32\Drivers\etc\hosts.ini. For each of the above hosts required to be resolved, add an entry in a format like this : <ip addr> <hostname>
    2. For each of the above machines, add 2 entries – one with the hostname and other with its FQDN name (ex. devmachinename.abcdomain.blah.blah.com)
    3. Note that if dynamic IP allocation is setup in VPN configuration, the machine's IP address will get changed everytime it connects to the network. Thus in scenarios like machine reboots the bindings need to be updated everytime. (You can write a small windows startup script to do this automatically). This wont happen in static IP allocation.
    4. For testing purpose ping all the required components using both hostname as well as FQDN name. Now onwards both hostname as well as IP addresses can be used in the configuration tools etc, however the recommendation is to go with hostnames.

  2. Firewall settings:
    1. Enable the firewall settings to "notify when any app is blocked" from "Control Panel\System and Security\Windows Firewall". This may not be set by default, and firewall might silently block the connection request flowing from TestAgent to TestController.
    2. Enable "file and printer sharing" and exempt it from firewall for the network profile currently in use (for ex. Domain/Public/Private). Sometimes the exception is not present for all the network profiles and might end up blocking the connection requests. This can be done through "Control Panel\System and Security\Windows Firewall".

 

TFS:

  1. HostName Resolution settings: As mentioned in the previous section, setup name resolution entries on the TFS machine in the below way. Hosts required to be resolved for TFS : Client and TestController machine.
    1. Open the file %systemdrive%\Windows\System32\Drivers\etc\hosts.ini. For each of the above hosts required to be resolved, add an entry in a format like this : <ip addr> <hostname>
    2. For each of the above machines, add 2 entries – one with the hostname and other with its FQDN name (ex. devmachinename.abcdomain.blah.blah.com)
    3. Note that if dynamic IP allocation is setup in VPN configuration, the machine's IP address will get changed everytime it connects to the network. Thus in scenarios like machine reboots the bindings need to be updated everytime. (You can write a small windows startup script to do this automatically). This wont happen in static IP allocation.
    4. For testing purpose ping all the required components using both hostname as well as FQDN name.

  2. Firewall settings:
    1. Enable the firewall settings to "notify when any app is blocked" from "Control Panel\System and Security\Windows Firewall". This may not be set by default, and firewall might silently block the connection request flowing through TFS.
    2. Enable "file and printer sharing" and exempt it from firewall for the network profile currently in use (for ex. Domain/Public/Private). Sometimes the exception is not present for all the network profiles and might end up blocking the connection requests. This can be done through "Control Panel\System and Security\Windows Firewall".

 

VS/MTM Client:

  1. HostName Resolution settings: As mentioned in the previous section, setup name resolution entries on the Client machine in the below way. Hosts required to be resolved for Client : TFS and TestController machine.
    1. Open the file %systemdrive%\Windows\System32\Drivers\etc\hosts.ini. For each of the above hosts required to be resolved, add an entry in a format like this : <ip addr> <hostname>
    2. For each of the above machines, add 2 entries – one with the hostname and other with its FQDN name (ex. devmachinename.abcdomain.blah.blah.com)
    3. Note that if dynamic IP allocation is setup in VPN configuration, the machine's IP address will get changed everytime it connects to the network. Thus in scenarios like machine reboots the bindings need to be updated everytime. (You can write a small windows startup script to do this automatically). This wont happen in static IP allocation.
    4. For testing purpose ping all the required components using both hostname as well as FQDN name. Now onwards both hostname as well as IP addresses can be used in the VS/testsettings etc, however the recommendation is to go with hostnames.

  2. Firewall settings:
    1. Enable the firewall settings to "notify when any app is blocked" from "Control Panel\System and Security\Windows Firewall". This may not be set by default, and firewall might silently block the connection request flowing from Client to TestController.
    2. Enable "file and printer sharing" and exempt it from firewall for the network profile currently in use (for ex. Domain/Public/Private). Sometimes the exception is not present for all the network profiles and might end up blocking the connection requests. This can be done through "Control Panel\System and Security\Windows Firewall".

 

Hope this helps those of you wanting to move your test infrastructure to Azure Cloud.

Note that if you are doing load testing in the cloud using the test infrastructure, we strongly recommend you to use Load Testing in Cloud with Visual Studio Online 

Ajay

 

(Editor’s Note from Charles Sterling)

A couple of additional things to check:

1. Make sure and check out the article: http://social.msdn.microsoft.com/Forums/en-US/df043823-ffcf-46a4-9e47-1c4b8854ca13/troubleshooting-guide-for-visual-studio-test-controller-and-agent?forum=vststest 

2. Often when you go through this process you will be using shadow accounts.  Important note: MAKE SURE YOUR SHADOW ACCOUNTS NOT JUST HAVE MACHINE ACCESS BUT ALSO MATCH A TFS ACCOUNT.

3. In many cases your IPSEC policies will block the use of Shadow accounts.  Make sure you get a machine exemption….Inside of Microsoft these servers are obtained by requesting a “boundary server” account from our Ops folks

4. The specific ports needed to be opened:

  1. netsh advfirewall firewall add rule name="RPC" dir=in action=allow service=any enable=yes profile=any localport=rpc protocol=tcp
  2. netsh advfirewall firewall add rule name="RPC-EP" dir=in action=allow service=any enable=yes profile=any localport=rpc-epmap protocol=tcp
  3. netsh advfirewall firewall add rule name="Port 135 TCP" dir=in action=allow service=any enable=yes profile=any localport=135 protocol=tcp
  4. netsh advfirewall firewall add rule name="Port 135 UDP" dir=in action=allow service=any enable=yes profile=any localport=135 protocol=udp
  5. netsh advfirewall firewall add rule name="Port 136 TCP" dir=in action=allow service=any enable=yes profile=any localport=136 protocol=tcp
  6. netsh advfirewall firewall add rule name="Port 136 UDP" dir=in action=allow service=any enable=yes profile=any localport=136 protocol=udp
  7. netsh advfirewall firewall add rule name="Port 137 TCP" dir=in action=allow service=any enable=yes profile=any localport=137 protocol=tcp
  8. netsh advfirewall firewall add rule name="Port 137 UDP" dir=in action=allow service=any enable=yes profile=any localport=137 protocol=udp
  9. netsh advfirewall firewall add rule name="Port 138 TCP" dir=in action=allow service=any enable=yes profile=any localport=138 protocol=tcp
  10. netsh advfirewall firewall add rule name="Port 138 UDP" dir=in action=allow service=any enable=yes profile=any localport=138 protocol=udp
  11. netsh advfirewall firewall add rule name="Port 139 TCP" dir=in action=allow service=any enable=yes profile=any localport=139 protocol=tcp
  12. netsh advfirewall firewall add rule name="Port 139 UDP" dir=in action=allow service=any enable=yes profile=any localport=139 protocol=udp
  13. netsh advfirewall firewall add rule name="Port 445 TCP" dir=in action=allow service=any enable=yes profile=any localport=445 protocol=tcp
  14. netsh advfirewall firewall add rule name="Port 445 UDP" dir=in action=allow service=any enable=yes profile=any localport=445 protocol=udp
  15. netsh advfirewall firewall add rule name="Port 6901 TCP" dir=in action=allow service=any enable=yes profile=any localport=6901 protocol=tcp
  16. netsh advfirewall firewall add rule name="Port 6901 UDP" dir=in action=allow service=any enable=yes profile=any localport=6901 protocol=udp
  17. netsh advfirewall firewall add rule name="Port 6910 TCP" dir=in action=allow service=any enable=yes profile=any localport=6910 protocol=tcp
  18. netsh advfirewall firewall add rule name="Port 6910 UDP" dir=in action=allow service=any enable=yes profile=any localport=6910 protocol=udp
Leave a Comment
  • Please add 7 and 5 and type the answer here:
  • Post
  • I'm glad you wrote this as I'm looking into testing using Azure VMs.  What you be your recommendation for topology if TFS, VS and MTM client are on premise so only test agents and possibly controllers are deployed in Azure?  Essentially, the only decision point is where to put the controllers.  I've heard that distributed testing with controllers on premise can be unreliable.  What are the trade-offs running controllers on premise vs controllers on Azure?

    Thanks

    bwr

  • If you want to optimize on test execution's performance, you need to keep the build drop location, test controller and test agents close to each other. For your case, my best suggestion would be to keep controller on azure. However if build is also on azure, the execution should be the fastest.

  • thanks Ajay for the valuable information!!

  • For Automated Testing Windows Application, Is it possible to host TFS, TestController, Test agent etc., on a single azure machine. I am just evaluating build-deploy-test scenario. The automation should build, deploy in the same system and run the tests in the same system.

Page 1 of 1 (4 items)