Narendra Parihar's Blog

Windows 8 Modern App Testing

2013-01-01T21:42:23Z

Is your team building Windows 8 Modern App for customers? As part of QA team let me assure you it’s a good thing to be in that project! But we ready to get exposed to new things to test, its lot of interesting stuff for QA community to learn and quickly come to terms of changing software trends.

Testing a Windows 8 Modern Applications is interesting and happening space. Having tested couple of these applications made me write this post to share some of the things which are different and need more attention than normal testing approach.

As much as tester is required to understand the intended functionality of the Win8 apps, they need to understand Win 8 Modern App principles themselves thoroughly to really test a true Win 8 app than just any HTML5/XAML application.

Watch here about Windows 8 Modern Application guidelines, you will find many bugs by just validating against these guidelines during tests.

Testing data in your application would be specific to requirements, however some of the important things to consider during different phases of development are listed below in tabular format. This list focuses on QA tasks which are important or different than conventional application testing.

	QA Tasks
Requirements	Ask do we really need Win 8 Modern App?
	Why not other solutions?
	Who is the Customer base ? Do they have or will have Windows 8 in future?
	Influence the requirements which are really appropriate to show in a Windows 8 App
	Get Performance (concurrent/simultaneous) requirements
	Security requirements, who can access, how to download/Install and data related security
	User Experience Requirements

Design/Code	Go through material available online to see if design is in right path?
	Why HTML5 or XAML as solution technology for Windows 8 Application?
	UX design is most critical to review
	Validate UX and Design against Modern app principles
	Plan for Automation early, Microsoft Test Manager(MTM) 2012 part of Visual Studio 2012 offers in built automation functionality
	Plan for White Box testing, structure based testing and Code Coverage for your testing aka Test Coverage against Code

System Testing	Other than normal things which are part of test strategy below are some important things or different things which comes into equation when testing a Windows 8 App
	MTLM for exploratory testing for modern apps
	Test in Landscape, Portrait mode
	Performance Testing: Visual Studio Load Tests with Unit Tests added to them are useful
	Windows8 Application Tool Kit is good tool to run against your app
	Touch and Click, test for both operations
	Application Bar
	Navigation Bar
	Resolutions supported
	How does App works when projecting to a monitor or screen?
	Logging for all errors/crashes are very important, as Operating System itself is evolving, so log every Crash you see with detailed logs to track
	Will App work like Outlook meaning can it sync from different computers?
	Live Tile / Notifications
	Uninstall/Install
	Immersive Content
	Snap feature
	Different Hardware like desktop, Slates, Tablets, Surface, Monitor mode, extended display, 32 bit/64 bit
	Tile size, all options you get when right click app tile
	Various Charms and gestures supported
	Keyboard support, can user do everything in application without Touch as well
	Application Permissions like sharing, document library etc..

Hope this would help QA community to test Windows 8 Modern Apps better and delight customers!

Happy New Year and Wish you happy reading.

-Naren

Views or Opinions expressed in this blog represent only author and not any company or organization or person.

Regression Testing

2012-04-19T22:21:00Z

Regression testing defination and scope has always fascinated me even after 8 years in Testing World. What really amazes me is it hasnt evolved a lot since last time someone defined defination of regression testing some 20 years back. Software testing is one field where evolution has been really slow and stuck at books of my parent's generation.

So i started to think back and re-looked at defination of Regression Testing, top of world's top search engines bring up this page as first result (Bing and Google).

http://en.wikipedia.org/wiki/Regression_testing

I looked at history of page and it was last edited on 29-March-2012.

Lets look at defination...

Regression testing is any type of software testing that seeks to uncover new software bugs, or regressions, in existing functional and non-functional areas of a system after changes, such as enhancements, patches or configuration changes, have been made to them.

The intent of regression testing is to ensure that a change, such as a bugfix, did not introduce new faults.^[1] One of the main reasons for regression testing is to determine whether a change in one part of the software affects other parts of the software.^[2]

Common methods of regression testing include rerunning previously run tests and checking whether program behavior has changed and whether previously fixed faults have re-emerged. Regression testing can be used to test a system efficiently by systematically selecting the appropriate minimum set of tests needed to adequately cover a particular change.

I for some reason dont agree to this defination for reasons explained further, this sounds like assuming many things which have evolved in test space.

- Testing technique is mostly Blackbox

-Tester does not knowledge of internal system in terms of they dont look into code to know dependency of objects or dont do any impact analysis while writting Test Plan

- They wait till Regression test cycle at end of Test Phase to identify if system have any new bugs or regression bugs

I feel these assumptions are not so true nowourdays, with Test organizations moving towards more of SDET profile and more focus on White Box techniques like code reviews by Test team, Code Coverage techniques, Peer reviews, Inspections at all stages; testers are better equipped to analyze and make decision which areas to regress in the application rather than regressing the entire application. Let me try to explain what i mean with visual diagram.

Lets say your application consists of below modules from A to F, you had regression suite with say 300 test cases which have coverage for all areas, and it costs 10 days to run that regression suite.

Now in next release C has been modified to C++, which impacts A and F, due to indirect impact or data flow lets say it also impacts B not less than A and F. So below diagram shows areas which are impact due to this change in yellow or light yellow.

Now if we go by definations which had assumptions i listed earlier, i would still need same effort and resource for regression testing.

Here comes problem, i dont agree to that concept. I feel now with tools, techniques, more SDET skills, i know that only A, B and F are affected due to change in this release and it has no what so ever impact on E and D. So I should be testing only A, B and F in regression testing, which is going to save time for me and make my project manager more happy :-)

Ofcourse my approach assumes that you have the skills and knowledge to identify A, B, F as impacted areas with confidence in first place but thats where we are heading isnt it.. Testers are getting more skilled day by day with lots of focus on technical skills and domain knowledge.

We should get better at doing regression testing with evolution of testing space, get smarter and bring in more productivity every time we do something, do your regression cycles faster and target right areas which would yield potential bugs rather than executing same regression suite which was never re-visited and keep on taking standard days forever!!!

-Naren

Views or Opinions expressed in this blog represnt only author and not any company or organization or person.

Test Automation failures

2012-03-29T04:22:00Z

Every now and then we keep seeing automation failures. Most of Testers have been part of these failure stories as actors or audience or directors :-)

I am sharing top 3 reasons for automation failure in this post which is kind of little modified from my post on blogspot @ http://infomine-software.blogspot.com/#!/2010/10/why-does-test-automation-fail-every-now.html:-

1- How many tester's are really tester, may be 50%, rest 50% were circumferenced into testing world. (Dont beat me for that:-))

A good tester generally does not like coding, he/she likes to be always in test execution phase, hitting severity one bugs all the time..... bottom line, A good tester generally does not like creating scripts. So how do we motivate such folks to start writting automation scripts/code, well show them value of automation to find bugs, that may just fly with them.

2- Well done, we have got the automation done - date X.................................................................Scripts are not working, due to environment or other issues on date X + 2 months..........Most of us will be able to relate to this scenario, why do scripts fail, because they were probably poor in the first place. Person maintaining scripts are not deep into ocean of automation. Its hard but thats how it goes down!!!

You need deep knowledge of any tools or technology you use for automation, folks floating on surface can give few demo's or write good paper's but for a long term automation ROI, you need someone to do automation design for you whom you can call as SME in say Coded UI or Middle tier web services. Having someone in the automation team who can lead that R&D effort will save from lot of re-work in future.

3- Biggest tempation for a tester working on test automation is technology and coding. With little code understanding automation tester's make mistake of targeting low hanging fruits. So it works for a release and then die's or involves so much manual effort before running it, that there is absolutely no ROI.

Automate for ROI, without productivity gains automation is waste. Future of Automation is in ROI to start with, next level is to find bugs with automation. Keep that in mind from very begining, low hanging fruits are not always a good choice to run after especially when it comes to automation.

My vision of automation goes something like below:-

Good Tester's write testcases in a predefined manner (or using some test case writting tool) and say Run, he/she never has to worry about code. So who handles code, a dedicate person from test team who works with Dev team to understand their code structure, suggests them to insert some hooks, which will be used by automation framework. This automation framework just calls the hooks and results scripts from test cases.

From a management perspective if i have to implement above vision, i would have folks who have good knowledge of automation as directors and good testers as actors to build a successful automation story!!!

-Naren

Views or Opinions expressed in this blog represnt only author and not any company or organization or person.

Improve productivity in Testing

2011-12-01T21:13:00Z

This paper focuses on reducing the cost of software to customers and talks in detail about ‘Improving test productivity’ as one of the best practice.

Test teams are often asked, "How do you plan to improve test productivity?" AND there is an opportunity in this question for every test organization!!!

Well, there is no one answer for it!!!

On an average overall test activities effort is around 50% of development effort, which can definitely be improved. Test teams needs to start moving towards more tailored test processes which suit their business model.

A key challenge for any organization is always how to reduce COST for customer. Today the customers can be made happier

using cost reduction rather than giving ornamental features.

COST, Timelines and Quality are three most important priorities for business in today’s highly competitive market. If we don’t reduce cost for customers, they will do it in their own way!!! As the IT industry faces challenges due to economical slowdown, we need to find different ways to reduce cost for our customer so that they still see smooth ROI, and remain our customer’s.

This paper talks about causes which refrain test teams from being more productive and suggests measures to overtake them. Use of Automation right from test case preparation till test reporting along with examples/tools available at MS is explained in later sections.

Testers does spend most of their time in test case preparation, execution, and bug reporting and finally test status reporting. This paper will talk about where we lose opportunity to improve productivity at every step. Also, suggesting some productivity tools to do our job better and in more efficient manner. A suggested test lifecycle

Complete paper is attached for your further read. Please note that this paper was written with respect to my then team at Microsoft, so any mentions had to be read with this note in mind.

-Naren

Views or Opinions expressed in this blog represnt only author and not any company or organization or person.

Testing in Agile based projects

2011-11-27T01:22:00Z

This paper is going to talk about following points:-

-Why Agile?
-Agile test practices
-Challenges in testing agile projects
-Possible solutions to issues in testing agile projects
-Future of Agile…..
-Benefits and Drawbacks….

Complete paper is attached for your further read, this paper was presented by at Agile conference at Anna University in Chennai, 2007.

-Naren

Views or Opinions expressed in this blog represnt only author and not any company or organization or person.

Use Case based Testing

2011-11-27T01:17:00Z

This paper proposes a methodology to test an application with Use-cases along with its benefits & limitations.

This paper also provides roadmap for QA team/IT manager to apply Use-case testing technique to foresee the probable improvement in development process.The rest of the paper deals with some practical insights into Use-case testing technique based on industry experience, some of these being:

-Identify Testing Scenarios from Use-cases
-Path analysis using Use-casesDerive functional and negative testcasesUse of Unified Modeling Language (UML)
-‘Testing Process Map’ change with Use-case testing technique
-Extend Use-cases for User-Acceptance Testing (UAT)

Attached is the complete white paper for your reading, this paper was presented by me at a International Conference 'Testing World', Hyderabad as well in 2006.

-Naren

Views or Opinions expressed in this blog represnt only author and not any company or organization or person.

Datawarehouse testing using MiniDBs for IT

2011-11-27T01:09:00Z

In Business Intelligence(BI) applications there are Datawarehouse(DW) which holds data in Tera Bytes and sometimes go up to as many as 50-100 data sources. Testing such applications requires huge test servers and longer data testing cycles. The risk of source systems being down or in restore mode when ETL was running is not in team’s control. To address all such issues and improve test effectiveness/productivity, Microsoft India BI Team came up with a organization wide concept of using MiniDB’s across all BI projects.

Above diagram shows a typical DW architecture and storage needs, ETL Run time, Internal Job run times. MiniDB is basically a smaller data set version with exactly same schema/objects of source systems, application databases and a working model but with different topology than the production environment. Using the SQL server trimming mechanism we trim the databases to fit into a single
box, and call it as One Box environment which is used by Dev/Test teams to do most of their functional, data, and ETL testing. The entire test strategy resulted in many advantages as reducing test cycles by 50%, more test coverage and shorter regression cycles.

This paper will talk about how to create MiniDB’s, replicate working model of a DW, test strategy to do BVT in less than 2 hours in a large data warehouse applications, and all practical challenges/benefits in the course of the implementing this technique in any data warehouse application.

Attached is the complete white paper for your continued reading , feedback and comments....

-Naren

Views or Opinions expressed in this blog represnt only author and not any company or organization or person.

Data Dictionary based Testing (DDbT) for Business Intelligence Applications

2011-11-27T00:59:00Z

This Paper tries to present a new test methodology which can help us better test Business Intelligence (BI) or Data warehouse (DW) with more coverage, more data centric approach and with less time. The horizon's to cover with this idea are endless to explore.
Assume that we want to test a Business Intelligence (BI) or Data warehouse (DW) application , and we normally start to think that it’s going to be black box testing or data quality testing, or both.

Our test plans are towards how to make sure data is correct, functionality is working, jobs are completing etc. and how do we achieve it…?Normally using black box techniques. This is where the twist lies..

Do we really test a BI/DW using black box techniques? Theoretically the answer may be Yes, but practically No.What we need is essentially a data verification testing methodology to test BI/DW applications.

This is what we are calling in more technical term – ‘Data dictionary Based Testing Framework’“A test framework wherein the data dictionary is leveraged to describe each and every element of the data residing in database server objects, their relationships within the data, the data lineage between the elements, which in turn when rationalized would help us to test the data quality and functionality of the application”.

Entire paper is attached herewit for your further read and comments....

- Naren

Views or Opinions expressed in this blog represnt only author and not any company or organization or person.