How to anticipate or better off avoid performance related "surprises" during load and stress testing?

Apply performance engineering practices throughout SDLC (Software Development LifeCycle). Here are major talking points, tools, resources, and further reading.

Performance Engineering Frame

In order to focus and streamline Performance Engineering tasks throughout the development lifecycle ask the following questions

• How to cache data?
• How to handle communications?
• How to handle concurrency?
• How to handle components’ coupling/cohesion?
• How to perform data access?
• What algorithms to use?
• How to handle exceptions?
• How to handle resource management?
• How to handle state management?

More info - Performance Frame. Pay special attention to performance threats and performance vulnerabilities.

Development phases and related activities

• Envision Phase
  • Identify requirements and objectives.
  • Conduct performance awareness training.
• Architecture and Design Phase
  • Performance Modeling
  • Architecture Design Guidelines
  • Architecture Design Review
• Implementation/Coding Phase
  • Compile and distribute performance coding guidelines.
  • Conduct performance code inspection
• Testing Phase
  • Conduct load testing
  • Conduct stress testing

Tools

• Guidance Explorer (create guidelines documents, inspection checklists)
• p&p Practices Checker – performance (conduct code inspection)
• WCAT (generate load/stress)
• Ajax View (measure during development)
• SQL Server Profiler (measure during load/stress tests)
• Microsoft Network Monitor 3 (helpful in conjunction with 3rd party tools like NetworkSmart from LeadByte)
• Fiddler (measure during development and load/stress tests)
• IIS Logs (examine during development and testing)
• .Net Profiler of your choice (measure during development)
• Perfmon (measure during development and load/stress tests)

Related materials

• Baking Performance Into the Life Cycle
• Performance Testing Guidance for Web Applications
• Improving .NET Application Performance and Scalability
• patterns & practices Performance Testing Guidance
• Do You Really Need A Distributed Architecture?