I work for E&D systems Test team. We have been working towards building test automation framework for E&D projects. In our quest to build reliable test automation we were in the process of evaluating several tools to achieve our larger objective of faster , better and cheaper ways of providing business solutions. That's when VSTS 2010 happened as one stop shop for all automation requirements
We walked across to IDC VSTT team and offered to dog food VSTS 2010 for our projects.
Following features of VSTS 2010 (MTR and Camano) are quite interesting to try out. I am sure you will love it :-)
1. Manual Test Runner (MTR) - a record and playback of web controls with human interaction helping us to automate the UI test cases which drastically reduced the time spent invested in Regression testing.
2. MTR also helps to create a detailed bug report (records tester's own actions, video of the tester's desktop during execution, and a historical debug log with details of the execution)
3. Test case management works better with introduction of Microsoft Test Manager (code name: Camano):
- Camano provides integrated test case management to create, organize and manage test cases across both the development and test teams
- Provides good traceability (Requirements,Test cases,Bugs are connected) helping to achieve good test coverage
- With Camano we are able to combine our test cases as suites or test runs and get them executed thro MTR and get the bugs logged from there.
While we used the tool, we came up interesting bugs and feature feedback to further enhance the product. Excited to be a part of VSTT TAP program as we are among few folks who get into know all latest cool features of VSTS 2010 first :-)
When VSTS 2010 is released worldwide, we are definitely more ready and look forward to its successful launch to Microsoft customers!!!
In my previous blogs, we discussed in detail about InfoPath's dynamic parameters and how to extract and parameterize them. In this blog we shall talk about some of the challenges we face while we migrate our Performance scripts to different environment.
Moving the VSTS scripts from server to another server is not a seamless experience. Let me share some of our key learning's in this front. Our scripts stopped working when migrated to a different server base. When we tried analyzing the requests we found some parameters apart from web server names needed to be parameterized.
Let’s look into the requests for a better understanding,
”Web servers’ parameterization” feature in VSTS parameterizes only the server name in the script and not the contents in the string body, event log or in Query string
String body
Server on which Scripts were recorded:
8;0;{{MyValue1}};{{myvalue2}};0;;urn%3Aschemas-microsoft-com%3Aoffice%3Ainfopath%3ATaskForm%3A-myXSD-2008-01-04T05-38-11;https%3A%2F%2Ftestserver1.com%2F_layouts%2FFormServer.aspx%3FXsnLocation%3Durn%3Aschemas-microsoft-com%3Aoffice%3Ainfopath%3ATaskForm%3A-myXSD-2008-01-04T05-38-11;;https%3A%2F%2Ftestserver1.com%2FDynamics%2FAxapta%2F{{SiteName}};;1;0;0;162;1;633421497999791000;;{{myvalue3}} 0;V1_I1_D15;; Review%20completed%20and%20reviewed%20files%20attached%3Ay 0;V1_I1_S:17_I1_C1;;true 0;V1_I1_T23;0;comments 1;CTRL14_5;V1_I1;
New Server:
8;0;{{MyValue1}};{{myvalue2}};0;;urn%3Aschemas-microsoft-com%3Aoffice%3Ainfopath%3ATaskFormV1-1%3A-myXSD-2007-07-02T05-38-11;https%3A%2F%2Ftestserver2.com%2F_layouts%2FFormServer.aspx%3FXsnLocation%3Durn%3Aschemas-microsoft-com%3Aoffice%3Ainfopath%3ATaskFormV1-1%3A-myXSD-2007-07-02T05-38-11;;https%3A%2F%2Ftestserver2.com%2FDynamics%2FAxapta%2F{{SiteName}};;1;0;0;162;1;633579187166585000;;{{myvalue3}}
Differences in parameters are highlighted, so it’s always better if we can parameterize as much as possible in our request. This could simply be done by using our extraction rules as mentioned earlier, for server names (https%3A%2F%2Ftestserver1.com) or any other parameters found in string body.
For instance:
8;0;02e85855-4790-4ce7-afe0-382465ac5740;b395b396-33ab-43ce-8325-3ec299f462d3:ver:1.0.0.3994;0;%2FDynamics%2FAxapta%2F{{sitename}}%2FInstanceu20Library%2FInstanceForm-{{instancename}}.xml;;https%3A%2F%2Ftestserver1.com%2FFormServerTemplates%2FArtDefn.xsn;;https%3A%2F%2Ftestserver1.com%2FDynamics%2FAxapta%2F{{sitename}};https%3A%2F%2Ftestserver1.com%2FDynamics%2FAxapta%2F{{sitename}}%2FInstanceu20Library%2FForms%2FAllItems.aspx;0;1;0;162;1;633552605014752000;;AxSBhHlXoLz18RGKBqtie6KBv62gO3n2eJJUR9dJITdmPOlFEcQlMvCBnPblYs1aMnjdLuJwVQZbJ9mU/XMleg==|633552605014595593 1;btnpostIntiationUpdate;V1_I1_S1_I1;
Could be parameterized in the below fashion to avoid any rework on the script
8;0;{{Myvalue1}};{{Myvalue2}};0;%2FDynamics%2FAxapta%2F{{sitename}}%2FInstanceu20Library%2FInstanceForm-{{instancename}}.xml;;{{source}}%2FFormServerTemplates%2F{{contenttype}}.xsn;;{{source}}%2FDynamics%2FAxapta%2F{{sitename}};{{source}}%2FDynamics%2FAxapta%2F{{sitename}}%2FInstanceu20Library%2FForms%2FAllItems.aspx;0;1;0;162;1;633552605014752000;;{{Myvalue3}} 1;btnpostIntiationUpdate;V1_I1_S1_I1;
We can find that server name is also parameterized as {{source}} in String body.Now we can be sure that even if the server name is changed, the parameters in the String body, event log and query string are also changed accordingly as they are derived from the earlier responses.
If you are new to InfoPath scripting please go thru my previous blogs to understand the fundamentals. This blog focus more on debugging the VSTS scripts.
Does your web test playback fails even after adding necessary extraction rules? What if some requests are missing? How can we identify which requests are missing? Here again Fiddler helps us.
Each request has a post back counter (highlighted in green below) as discussed earlier. The post back counter will follow a sequence in the script, if we find there is a break we can be sure there is some request missing.
Let’s see with an example:
8;0;e8f1110f-9a7c-4c9a-8d5b-c139fd561a55;337818e1-961a-4dde-b6d9-dee88bf242bf:ver:1.0.0.4039;0;%2FDefinitionLibrary%2FWFDefinition-{{DefinitionName}}.xml;;https%3A%2F%2Ftest.com%2FFormServerTemplates%2FArtDefinition.xsn;;https%3A%2F%2Ftest.com;https%3A%2F%2Ftest.com%2FDefinitionLibrary%2FForms%2FAllItems.aspx;0;1;0;23;1;633576714887154000;;T96l+tfY4xvMXmhNhkLxP3vMr8FnMvOd+9fFJjwA0IqejPnGQBtzfW+R4wOyIoFlZsM0cUVnsuG6oXxvYMyQeg==|633576714885747485 1;btnCreateWFTemplate;V1_I1_S1_I1;
8;2;e8f1110f-9a7c-4c9a-8d5b-c139fd561a55;337818e1-961a-4dde-b6d9-dee88bf242bf:ver:1.0.0.4039;0;%2FDefinitionLibrary%2FWFDefinition-{{DefinitionName}}.xml;;https%3A%2F%2Ftest.com%2FFormServerTemplates%2FArtDefinition.xsn;;https%3A%2F%2Ftest.com;https%3A%2F%2Ftest.com%2FDefinitionLibrary%2FForms%2FAllItems.aspx;0;1;0;137;1;633576714887154000;;IqCL2kRkh37dM4cSboBgG5ujVWdGmQxI9wE5wshOG/GXZsEhaIO7ZgigrEensblSV1CMytADJYvhrnpuQwtT4Q==|633576715314960741 0;V1_I1_S1_I1_R4_I1_R1_I1_D3;;Frank
If you look at the highlighted post back counter parameters, you find the sequence is broken. We can be sure the request with the post back counter 1 is missing. If we have given appropriate comments while recording it will give us a clue as to which request is missing. So how can we go about adding the missing request to our VSTS script?
Figure 7 below shows the recorded upload script where the upload request is missing. Now we have to look into Fiddler for the request.We can copy the request from Fiddler, paste it in our VSTS script and add request parameters as form post parameters
Recorded Web test for a simple file upload
Upload Request added to VSTS script from fiddler request
Form post parameters added to upload request
The script will now work fine for upload.
In my previous blog, we saw the methods to identify the dynamic parameters in an InfoPath form’s response. Now lets understand how to parameterize them using appropriate extraction rules
Every response becomes the input for every subsequent request. So, in order to get the value for dynamic parameters we need to extract the same from the previous response. Let’s take a simple example and analyze further.Let’s pick the following request from VSTS and parameterize it:
8;0;1893c270-9550-46bf-9599-89b18f493678;02490e79-9ee0-4319-a860-d96c965b2ddb:ver:1.0.0.447;0;;urn%3Aschemas-microsoft-com%3Aoffice%3Ainfopath%3ATaskFormV1-3%3A-myXSD-2007-07-02T05-38-11;https%3A%2F%2Ftest.com%2F_layouts%2FFormServer.aspx%3FXsnLocation%3Durn%3Aschemas-microsoft-com%3Aoffice%3Ainfopath%3ATaskFormV1-3%3A-myXSD-2007-07-02T05-38-11;;https%3A%2F%2Ftest.com%2FDynamics%2FAxapta%2Fupload4mbsite1;;1;0;0;162;1;633614592099519000;;uQQHHhbr8PDmvX7aDrE8fpll0JlVZHTW3og3Yp3cK5oZrYjuRFjCP1U77zafB+6Jd60wKIfoUiXJ8+qaR8iODA==|633614592096706480 0;V1_I1_D15;;Handoff%20incomplete%2FFiles%20Missing%3An 0;V1_I1_D15;;Final%20PDF%20Review%20Completed%20-%20Approved%20for%20Release%3Ay 0;V1_I1_S17_I1_C1;;true 1;CTRL14_5;V1_I1;
Let’s parameterize the first dynamic parameter i.e. editing session id (1893c270-9550-46bf-9599-89b18f493678).
Here is where Fiddler gives us a hand. Let’s try searching for this parameter from the previous responses (Figure 3), examining only in headers and body. We get the matching response carrying the Editing Session ID highlighted in green.
Extracting the editing session id from response using appropriate filters
Having found the session id, we need to extract it using extraction text rule in VSTS. So create a context parameter called “value1” in VSTS with the following extraction rule:
Starts with= -1,false,false]],false],-1,false,false]],false,[0,0]],-1,false,false],0,
Ends with= “
Refer to the above figure for more information.
Adding Context parameter to the web test
Similarly we need to parameterize all dynamic parameters in every request. The typical parameterized request looks like:
8;0;{{value1}};{{value2}};0;;urn%3Aschemas-microsoft-com%3Aoffice%3Ainfopath%3ATaskForm%3A-myXSD-2008-01-04T05-38-11;https%3A%2F%2Ftest.com%2F_layouts%2FFormServer.aspx%3FXsnLocation%3Durn%3Aschemas-microsoft-com%3Aoffice%3Ainfopath%3ATaskForm%3A-myXSD-2008-01-04T05-38-11;;https%3A%2F%2Ftest.com%2FDynamics%2FAxapta%2F{{SiteName}};;1;0;0;162;1;633421601380894000;;{{value3}} 0;V1_I1_D15;;{{Approval Status}} 0;V1_I1_S17_I1_C1;;true 0;V1_I1_T23;0;Dloaded%20a%20doc 1;CTRL14_5;V1_I1;
Are you facing issues while trying to record and playback InfoPath forms using VSTS web test? I would like to share some the tricks and techniques we used while conducting performance testing for our applications which used browser enabled InfoPath forms extensively.
When we perform any action in the InfoPath form it is recorded in the form of an event log entry or string body using JavaScript. When the form is submitted, or when logic must be performed on the server, the browser sends the event log to the server. The server plays back the actions in the event log to recreate the state of the form. Finally, the updated state of the form is returned to the browser and the browser refreshes its view of the data. InfoPath forms have two types of postback – partial postback and full page postback. Partial postback transmits less data and is comparatively more efficient. Full page post back happens when there is a view switching. For more details refer to http://www.codeplex.com/ipfswebtest
Let’s review one of our performance scripts to understand this post back. In this below example, on click of create button, a view switching happens and results in a full postback whereas when we try changing one of the attributes in the form (such as choosing some drop down data), a partial postback happens.
When a partial postback occurs, the event log is sent to the server using a POST request to Postback.FormServer.aspx and the requests are in the form of String Body. When a full page post back occurs, the event log, along with some other data, is sent to the server using a POST request to FormServer.aspx. The response from the server contains HTML and JavaScript to render the updated view of the data.
Let’s look at a typical request, captured during the VSTS recording:
8;0;506381d7-fa68-4ba8-8fa3-eaa136676275;02490e79-9ee0-4319-a860-d96c965b2ddb:ver:1.0.0.447;0;;urn%3Aschemas-microsoft-com%3Aoffice%3Ainfopath%3ATaskFormV1-3%3A-myXSD-2007-07-02T05-38-11;https%3A%2F%2Ftest.com%2F_layouts%2FFormServer.aspx%3FXsnLocation%3Durn%3Aschemas-microsoft-com%3Aoffice%3Ainfopath%3ATaskFormV1-3%3A-myXSD-2007-07-02T05-38-11;;https%3A%2F%2Ftest.com%2FDynamics%2FAxapta%2Fdownload4mbsite1;;1;0;0;162;1;633602516488353000;;oXXTzwbYir0m0iauGkpgL3FbLIGHYv2hti2/GZJlh4v1C46b61ESXg2cKfAbpk+n7PgD7ezwZ3bN3q9QZxgTuw==|633602516488353471 0;V1_I1_D15;;HandOff%20Approved%20and%20Translated%20files%20attached%3Ay
These looks complicated at first glance unless we go more into it and understand what each parameter corresponds to. Let’s categorize this way to simplify things:
<Parameter0> “;” <PostbackCounter> “;” <EditingSessionId> “;” <SolutionId> “;” <Parameter4> “;” <Parameter5> “;” <Parameter6> “;” <Parameter7> “;” <Parameter8> “;” <Parameter9> “;” <Parameter10> “;” <Parameter11> “;” <Parameter12> “;” <Parameter13> “;” <Parameter14> “;” <Parameter15> “;” <Parameter16> “;” <Parameter17> “;” <Canary>;<Parameter18>
When you do a playback of the above request it will fail as some of the parameters are dynamic. Now the challenge is to find which parameters are dynamic. We have a tool called Fiddler which can be used record the scenarios and compare the same with that of VSTS. Record the scenario in similar fashion using Fiddler .Try comparing same request from VSTS and fiddler
Request from VSTS:
8;0;3a2a936e-73c7-447f-9f7b-95b728068f46;b395b396-33ab-43ce-8325-3ec299f462d3:ver:1.0.0.3994;0;%2FDynamics%2FAxapta%2F2FBulk_PISite %2FInstanceu20Library%2FInstanceForm- Multi_instance12008-10-02T043732290.xml;;https%3A%2F%2Ftest.com%2FFormServerTemplates%2FArtDefn.xsn;;https%3A%2F%2Ftest.com%2FDynamics%2FAxapta%2FBulk_PISite;https%3A%2F%2Ftest.com%2FDynamics%2FAxapta%2F2FBulk_PISite %2FInstanceu20Library%2FForms%2FAllItems.aspx;0;1;0;162;1;633546595550842000;;JawjD/T9yExN3MFXdt28ubn2IyV9Sc6WJsgv4UxfyXg40EKStkMmWvFCYZBY0ZXpb0YwRV+8JGKxmTPzlFxVxw==|633546595550529792 1;btnpostIntiationUpdate;V1_I1_S1_I1;
Request from Fiddler:
8;0; dbfd9bc6-9a35-4342-9bf6-5926d9cfc565;337818e1-961a-4dde-b6d9-dee88bf242bf:ver:1.0.0.3994;0;%2FDynamics%2FAxapta%2F2FBulk_PISite %2FInstanceu20Library%2FInstanceForm-Multi_instance32008-10-16T003011850.xml;;https%3A%2F%2Ftest.com%2FFormServerTemplates%2FArtDefn.xsn;;https%3A%2F%2Ftest.com%2FDynamics%2FAxapta%2F2FBulk_PISite;https%3A%2F%2Ftest.com%2FDynamics%2FAxapta%2F2FBulk_PISite %2FInstanceu20Library%2FForms%2FAllItems.aspx;0;1;0;162;1;633546595550842000;; vFIuHMmlR+l9Oi1UOMQSwE+59BS+9v0w1b0xU2bf9TG1QkGyqjfYV3bn4xzXQLcmkiMx0WNy+VzyXmbPwc0tEA==|633564624538972226 1;btnpostIntiationUpdate;V1_I1_S1_I1;
As you can see, the highlighted sections (Editing Session id, Solution id, Canary value) differ from time to time. In simple terms, every time we record the same scenario we have different values in these places. This confirms what all the dynamic parameters are.
