Learn to use Visual Studio, Visual Studio Online, Application Insights and Team
Foundation Server to decrease rework, increase transparency into your application and increase the rate at which
you can ship high quality software throughout the application lifecycle
One of our premier test partners Testhouse (Shivaram Patil and Roshin Jacob) has been working with Hosam Kamel(Microsoft regional technology solution professional and ALM Ranger) for performance testing one of our mutual enterprise customers. The project was focused on using Visual Studio to do a performance and load testing for Seibel application. In this blog post Hosam does a GREAT job of summarizing their key learning and challenges faced during load testing an enterprise scale Siebel CRM application.
Hosam thanks for sharing this information and writing up the post!
Siebel Load Testing using Visual Studio: Recording and Playback
Introduction to Siebel CRM
Authoring Load Testing
Challenges of authoring Siebel load testing
Challenges of playback recorded tests
Best Practices & additional tips
Siebel CRM provides customer relationship management platform for businesses of all types and sizes. It is part of the Oracle suite of CRM solutions. Siebel CRM also offers extensive business analytics, and can be deployed on premise or as a Web-based, on-demand solution. Oracle's data base functionality is the basis for the application.
Load testing is the process of putting demand on a system or device and measuring its response. Load testing is performed to determine a system’s behavior under both normal and anticipated peak load conditions. It helps to identify the maximum operating capacity of an application as well as any bottlenecks and determine which element is causing degradation. When the load placed on the system is raised beyond normal usage patterns, in order to test the system's response at unusually high or peak loads, it is known as stress testing. Ensuring Siebel application quality and performance is critical but also challenging. There are many different applications and business transaction flows to test. Siebel’s highly distributed application architecture means there are many integrated components to consider, from both a functionality and performance perspective. And periodic Siebel application & infrastructure upgrades and patches need to be managed and result in ongoing testing.
Load testing Siebel application using Visual Studio can help you ensure that your application will perform and scale under real user workloads once deployed in production. It can help you identify and address any critical bottlenecks responsible for degradation prior to deployment.
Automated load testing of Siebel application has its own challenges due to the highly dynamic nature of Siebel applications.
· We recommend being very careful while recording the Siebel CRM. Some of the requests were not captured by Visual Studio web recorder and hence we decided to use “Fiddler” utility and then export the captured traffic as Visual Studio Web Test. (Reference 1, Reference 2)
· Understanding Seibel events and parameters are very important you may refer to the following reference to understand more about those events and parameters http://docs.oracle.com/cd/E14004_01/books/PortalFrame/PortalFrameDelConExtWebApps26.html
· Siebel requests are dynamic and manual parameterization if often required to replace the hard-coded data values to make Web Tests work as expected. Identifying these dynamic parameters is very tedious and time consuming.
Due to the highly dynamic nature of Siebel applications, the recorded tests are highly expected to fail while you do play back them, hereunder the major error you may expect and how to fix them.
· Root cause:
How to fix:
· How to fix:
Making the scripts work is tougher job and hence we recommend to thoroughly checking the request that are failing and understand the causes. It’s always best to have a Siebel consultant working with you in understanding how Siebel web client communicated with the server. We found few of the dynamic parameters like row id, lead reference are automatically created by Siebel server whenever you create a lead or campaign. So, it’s recommended to have an understanding on the Siebel architecture for better troubleshooting of these dynamic values and understanding the error codes of Siebel application.
Along with the tips document in Seibel reference here, hereunder a few tips as well:
· If the business process takes lot of time to execute, it’s recommended to increase the timeout for that particular request. Check with application owners for the benchmarking. Increasing the timeout values may provide inaccurate performance measurements.
· Following are the values that need to be manually correlated. There can be many other such values depending on the business process you plan to record. These are the most common one’s we found out when creating the scripts for Lead creation, response capture, bulk lead upload and campaign journey business process:
On rare occasions, these parameters needs to be correlated too:
This value usually remains the same throughout the script. But if the Script is lengthier or more applets/views are visited in the scenario, then correlating this value would do well.
Visual Studio provide a great tool for testing to ensure production performance and scalability provides constant, step, or goal-based load patterns for load testing working on enterprise scale with an unlimited number of test nodes, including web applications with support for an unlimited number of virtual browsers.
Hosam Kamel is a regional technology solution professional working for MEA Center Of Expertise specializing in Visual Studio ALM focusing on helping software professionals and organizations build better applications and solutions using Microsoft Application Lifecycle Management technologies, practices and tools working with development teams eliminate traditional silos between development, testing, and project management to establish cohesive processes with the Visual Studio ALM tools.
Hosam is also an active Visual Studio ALM Ranger contributing to a lot of the ALM Ranger projects.
Thank you Hosam for all the kind support in making this success story.
Good one .. is there any example where you have used the same scripts with minimal changes and tested Sales Force?
Testhouse , one of our UK Gold ALM Partners, have written up how they used Visual Studio load tests against