As a very brief intro, I have worked as a tester in SQL Server for the past 10+ years on many different features. Along the way, we develop and test features and release them to the public only to discover some customers inevitably encounter bugs when they run the features against their databases. How can this happen when we have amazing PMs and developers, devoted and talented test teams, and thousands of automated test cases per feature? The answer often lies in the incredible complexity of customer databases running on SQL Server and the evolution of those databases as they have grown from small to very complex databases over the years. As testers, we have a few different options to try to mitigate this problem and represent "all possible databases" in our testing, but it is impossible to test every possible permutation of databases based on this complexity. In practice, we do all of these to an extent and are constantly working on improving each of them.
Some options are to:
The last option (handcrafted databases) are often our last resort, but result in the most effective method for ensuring that specific features work for specific test cases. Our dev, test, and PM team spent some time recently for a feature we are working on to come up with the list of "complex" databases that we do not have in our test environment, but would like to add. Over the next few blog posts, I'll cover some of the interesting databases and the techniques I used to create them. Here are a few of the DBs we had in mind:
Keep in mind that there are MANY more interesting DBs based on scale, max limits, relationships, etc., but for our intended goal, this list covers the P1 cases.
Hope you enjoy,Sam Lester (MSFT)