Report of activities at Tech Ed 2006
Well, after Tech Ed, catching up on e-mail, a couple of weeks of vacation, then catching up on e-mail again (gotta love the 'conveniences' of the 21st century), I'm finally getting back to the blog.
Rather than repeating the analytical exercise that resulted in my 12-page Tech Ed report to our team, I figured I'd just toss into the blog some of the sections of that report (understandably omitting sensitive, internal information).
Oh, one quick note before the report. I gave a very impromptu podcast interview to Tech Net Radio while I was on booth duty (if you listen, it will quickly become apparent just how impromptu it was :)). I'll toss in the links to the WMA and the MP3 versions. My interview is the second one done:
WMA: http://download.microsoft.com/download/B/1/5/B15D95AC-F710-43A3-962D-CB50F71483B5/TechNetRadio-060616.wma
MP3: http://download.microsoft.com/download/B/1/5/B15D95AC-F710-43A3-962D-CB50F71483B5/TechNetRadio-060616_Hi.mp3
Selections from my report on Tech Ed 2006
Summary
Every customer I spoke to, DBAs, DB devs, and managers, was excited about the direction that we are taking VSTS. They are split on whether or not we have enough functionality in V1 for them to immediately commit to our product (which is standard for a V1). Those who are not ready to commit to V1 gave specifics about the features that they need to have for them to jump in.
There were few surprises in the feedback and feature requests that we received and the feedback was remarkably narrow and consistent, in my opinion. This speaks well of our understanding of our customers and of our opportunity to increase ‘customer delight’ about our product in the next version with specific, targeted additions.
We, like most other V1 teams, have overly simplified the profile of our target customers. In fact, we have different groups of customers who are passionate about different directions that we are going, and the passions of one group sometimes directly conflict with those of another group.
The Booth and Other Customer Feedback
We spoke to hundreds of customers at the booth during the week, engaging in conversations that lasted anywhere from 30 seconds to 30 minutes or longer. As often as we could, we jotted down notes from our conversations (Robert and Jon will send out contents from their notes, too, I’m sure). Additionally, we created a feedback form for those who did our hands-on labs to gather their impressions, positive and negative, about our product. In this section I’ll go through the highlights of what we learned as well as quotes (usually paraphrased) from customers I met.
What customers like best about our product:: There is not a single ‘dud’ feature in the product. Not every customer wants to use every feature of the product, of course, but all features received a good amount of love:
- SCC/versioning as a first-class citizen in a DB tool is a huge hit. Many already use SCC, but not in the central fashion that we do; they want to use our vision of DB versioning
- “What problems will this solve? Trying to figure out who changed what and when”
- “This solves the problem of managing changes between my dev and production databases”
- “This ensures a set location for the ‘latest and greatest’ DB version and a structured method of moving DB changes from the developer to the master copy of the DB”
- “Now we can instigate a code review process for our DB changes”
- Developing in a sandbox, not against a live DB resonated well with everyone. They quickly see how this improves their development process.
- Involving DBA in the development cycle
- DBAs who talked to us were nervous about the idea of using VS until we showed them that the barrier to entry was low and that they would be working with TSQL, not .NET code.
- DBAs, and many devs, are excited that having DBA in the development cycle will increase their projects’ chances for success at deployment time
- Many described the wall that currently exists between DBA and Dev, with Devs often seeing DBAs as some kind of killjoy troll while DBAs saw Devs as wanting to keep them out of the loop on work being done that targeted their databases.
- “As a DBA, I feel, sometimes, that devs want to keep DBAs ‘in the basement’. They want to make changes on their own when they can”. He doesn’t like current VS DB tools because they’re not integrated with the rest of the development process (he says that they’re ‘shoved in a corner’ of VS). Needless to say, he very much likes the prospect of DBA being made part of the solution process.
- Heck, if our sales are as strong as we hope, we might get a Nobel Prize to go along with our Ship It! awards ;)
- “This product will give us an increase of shared knowledge between DBA and developers”
- Data Generation
- One of the women completing DAT006 in the HOL gushed about DGen for a full 20 minutes
- She’s thrilled that DGen will allow her to populate test tables with 1,000,000 rows for testing (Deep Dive Testing Target)
- She currently uses 4 tools to do what our tool does in DAT006
- They currently have a team dedicated to creating and populating their test DBs.
- There is overwhelming feedback that we need to have Select/Deselect All checkbox for DGen.
- Unit Testing
- Several of those we talked to didn’t know what to make of this feature right off the bat because they weren’t doing anything like it. Many more, though, immediately saw it as a big improvement over their current testing methodologies.
- For those already using unit testing of code, it was important that our DB unit testing story integrate with their code unit testing processes in VSTS.
- Schema Compare
- “One of the main reasons I don’t currently have a test database is because I’d have to remember the differences between my test DB and my deployment DB”
- Data Compare
- A developer from a small shop works in an environment in which each dev keeps an identical copy of a test DB. He likes our lifecycle story for maintaining a ‘true’ copy of the schema for deployment; however, he does not want to use DGen, as they have specific customer data that they use for all testing. I showed him how he could leverage Data Compare for his task. He was thrilled and felt that the product met all of their needs. This was an interesting conversation, to me, because it pointed out how heterogeneous our customers’ work environments are. This was not what we would call an ‘enterprise shop’, but we still benefit as a product by meeting their needs and those of the other small shops.
- Refactoring
- “This tool makes a really tough job for me much easier”
- T-SQL Editor
- “I love the validation of scripts on save” (Adolfo, trainer, consultant, and author)
- VSTS/TFS integration
- “Now we can have communication and tracking of bugs around our DB”
- ”I came to Tech Ed to check out this product for our group. Before I did the hand-on lab I expected to hate it. Your product changed my mind, though!” She’s very excited about it. She didn’t expect MS to give her anything new or innovative (ouch!). She found SQL 2k5 to have some added features, but that it didn’t change much for her. SSMS is a bit easier to use, but not much better.
What customers feel is missing from our product: Few surprises here, although it was disappointing to hear how a number of customers considered some of our missing features in V1 to be adoption blockers. On the bright side, though, each of those, without exception, said that they love the direction that we’re going and will be watching what we do in VNext.
Here are key missing features that we heard about:
- Support for non-SQL Server databases
- Visual database designers/modeling. This is an adoption blocker for a number of customers.
- “A dream would be to have an ERD, as well”
- “Graphical Query/Table editors”
- “I will not write TSQL in a text editor. I’ll keep trying to use the visual DB tools in VS”. Interestingly, though, a number of DBAs believe that working in a visual designer makes one ‘not really a DBA’. Go figure.
- Deeper refactoring support. Rename is exciting and they want more.
- In particular, there is a strong expectation by many we spoke to that in the future we have refactoring cascade changes into middle-tier code that calls the refactored objects.
- Intellisense
- People have come to expect this in VS and are very disappointed to not yet see it in our product. This is not an adoption blocker, but it’s a hole we want to fill as soon as we can.
- Prescriptive guidance/best practices
- “We want integration with existing standards and perhaps an automated rules engine”
- Performance tools
- SQL2k5 SSMS provides performance tools. One of our most enthusiastic HOL visitors wants to see us have integration with that.
- Team support for encryption
- Store encrypted files in SCC and have anyone on his team work on them.
- Custom TSQL templates
- Several customers want to customize the TSQL templates that we provide.
- Prescriptive guidance around security and architecture. People want help in making some tough decisions.
- After a long conversation with one gentleman at the booth, I asked him what might be missing from our product that would help him. He responded that he had been trying to come up with something during our discussion, but couldn’t. “Everything you’ve shown me is stuff that I do and need”.
In what ways can we make VSTEDP easier to use?:
- “Not many. Well done on the VS2005 framework integration”
- “Some kind of Intellisense”
- “Clicking on an error did not take me to the error line, even though the line number is displayed. I want error navigation”
- “Give us integration between the Server Explorer and Import Schema”
- “I want to use the Query Builder to make my query for the Data Bound Generator. Oh, and if I have only one result field from my query, please autobind that to the column to be populated”
- Gary, who we met at the booth, heavily relies on stored procs. In VS he wants to be able to step through his .NET code and right into the stored proc in our T-SQL editor.
How customers currently manage DB change:
- Some customers don’t really have a defined process
- “Good question. We only have CA Erwin”
- “We individually apply changes to the database”
- Others have home-grown processes, no two of which seem to be same.
- “We run manual tests against a testing instance” (yeesh!)
- “We manually track changes to the dev database and rolling changes into a master upgrade script”
- “Individuals creating the change test it, then it is tested as part of front-end application testing”
- “We use Erwin and have DBA reviews of changes”
- “We diff schema scripts manually and in VSS”
- “We have test and staging servers and use SQL Delta to compare them after changes”
- “It is a very manual process. Unit tests change and we run test harness code against stored procs for regress testing”
- “We get sql scripts, periodically, from the DB and put them into SCC”. So, their SCC version is reactive to the deployed DB, not the other way around.
The nature of the databases that our customers work with: Desiring to learn more about how we should be testing our product, we asked many customers to describe the databases on which they work. Not surprisingly, we received a broad range of answers:
- When we asked customers about the number of tables in their databases, we received answers in the range of ~15-200+, skewed toward the lower end of this range.
- Asked about the number of rows they have in tables, answers ranged from tens of thousands to tens of millions, with many reporting 1,000,000+. This highlights the need that we have to do extensive performance testing and tuning late in the cycle.
- Many customers are tasked with maintaining and testing legacy databases with poor architecture. I spoke to multiple customers who work with databases that simply are not normalized, but which cannot be changed architecturally. These customers want to be able to use DGen (Deep Dive Testing Target) successfully against these databases.
- “Our previous developer created Access DBs, then used a wizard to migrate to SQL Server. The design was terrible, but it produces 50 reports per day and cannot be taken offline to rearchitect. Your tool will let me work in a sandbox to make and test changes to the architecture and version while I fix this bad DB”
- “Our legacy DB has no PK-FK relationships. Despite this, there are inter-table and inter-column dependencies in the DB, all handled through business logic. I want to be able to declare these relationships in DGen and have population work as if a PK-FK relationship existed. Updating the schema is not an option for us”
- Lookup tables are quite common and it’s very important to those customers that our product allow them to RevEng, check into SCC, and otherwise manage such tables.
Performance expectations for DGen: Since perf has been such a concern for DGen, we took the opportunity to ask many attendees how many rows of data they might want to populate in a test server and what their performance expectations are:
- One corporate customer wants to populate 1,000,000 rows in a ‘short time’ (a few hours or less) or they won’t use it.
- Another company wants to populate 50,000 rows with test data. He expects this to complete in less than one minute, or he’d begin to wonder if something is wrong.
- Yet another (Blog note: I’m eliminating company names for the blog posting) tables with 120,000,000 rows. Our current pre-beta doesn’t work for them because it takes too long. Statistical analysis tells them that they need to populate ~1,000,000 rows with test data. He wants that population to complete within a day
- A company in The Netherlands expect to populate 100,000 rows with test data within one hour.
- A final customer, involved in our TAP program (Andrew and I spoke to them after Tech Ed) wants to populate 1,000,000 rows in a highly normalize DB at a rate of 250,000/hour
