Follow up On Feedback From Beta1 and DevLab About VC Project Conversion’s Performance

Follow up On Feedback From Beta1 and DevLab About VC Project Conversion’s Performance

  • Comments 10

Hi,

 

My name is Bogdan Mihalcea and I’m a developer on the C++ Project & Build team. In the last 2 years I worked on a new C++ project system build on top MSBuild.

 

I’m writing this blog to share some excellent news related to improvements we made to the performance of project conversion since the Beta1 build. This was possible because of the feedback we got from you, and I want to thank you for doing so!

 

During our development milestones, we had tests reporting that we have a very slow conversion for specific types of projects. They were usually containing many files (1000+) and they had a significant percentage of them containing file level configurations. After we analyzed the root cause and the costs of fixing, we made the assumption that these projects with many file level configurations were not very common, so the impact of the performance in this scenario would be low. Another assumption we made was that the conversion it is a onetime thing so the impact even if big for those rare cases it is a onetime tax. Based on these assumptions we prioritized the fix lower and decided to invest our time in making the performance of main line scenarios better.

 

Our team has a customer plan which includes the early release of the product through Beta1 and having various face to face meetings with customers. One of these events is the Visual C++ DevLab, which last time we kept in May 09. During this event we noticed that 20% of the customers were experiencing a very slow conversion.

 

We investigated and we realized that all cases were caused by less than 5% of the projects in the solution, which took the majority of conversion time. One common detail about those few projects is that they were very heavy on file level configurations. From the discussions with the customers we realized that it is unfortunately easy to get in that state due to various reasons (evolutionary codebases, easy to cause human error due to bulk select, bugs in previous conversions, not a easy bulk way to revert the file level configurations to project level).

 

That pretty much invalidated our assumption that it is not common to get in a situation where a customer can have a project with 1000+ files and more than half have file level configurations. The first assumption was also invalidated from the discussions we had with the customers and from feedback we received from Beta1’s customers, which revealed that even if the projects that have this case are not very common the unfortunate thing is that it is common for customers with hundreds of projects have 1 or 2 of these and that will impact the whole experience of the conversion for a much bigger percentage of the customers, than what we estimated earlier.

 

Furthermore the second assumption was challenged as well as we learnt from your feedback that conversion might not be a onetime tax as most of the companies prefer to stay for months in a dual state with the development done in parallel on both products and it is common to maintain only the previous version files as a baseline for changes and to reconvert always when the new product is used. Considering the new feature we added in Dev10, native multitargeting, this seems to make this approach even more popular.

 

After we got all this feedback we acted on it. We analyzed the code paths and we found places where we could optimize for this type of scenarios and made the fix. Below is an extract from our measurements before and after the fix.

 

Project

Files/vcproj

Configurations

% files in vcproj with FileConfigurations

Previous Conversion Time

Current Conversion Time

Customer

~4000

4

50%

55 min

8 sec

MVP

~5000

8

80%

1h 40min

23 sec

PerfLab test

~200

2

less than 10%

2.8s

1.4s

Customer

~200

16

100%

50 min

10 sec

 

The common case projects (PerfLab test), with only few file level configurations, were impacted too as you can see from the above table. We used pristine lab machines dedicated to performance runs and the results are specific numbers to these machines.

 

We are constantly on lookout for feedback and we are adjusting our priorities based on it, and we will try to keep you guys up to date with the progress we make.

 

Regards,

Bogdan Mihalcea

C++ Project & Build

  • Some of our older projects suffer from high numbers of per-file configuration settings. As you suggest, I suspect that this is because they've been manually tweaked over the years, and have been through successive upgrades (e.g. from VC5 all the way to VS2008).

    I wonder if, when VS2010 is done, someone could find some time to write a VS PowerToy to make it easier to look at these and maybe remove the incorrect per-file configurations? ;-)

  • Yes, in fact the conversion was really slow. But as you said, it was a one-time tax, so it really doesn't matter!

    Woow going down from 1h,40 to 20secs ! There should have been a real bootleneck! :)

  • "lpbData" uses array new operator to allocate memory in CSettingsStore::Read(), but it uses  free() to free memory.

    Sorry, I don't know where to report the bug ^_^

  • Hi cppboy,

    please use http://connect.microsoft.com/VisualStudio to report bugs and suggestions

    --

    SvenC

  • More important for us is to be able to identify which files have file-level settings. Consistency is very important for us, and I would like to be able to set options for all projects, and be able to identify which projects and files override these settings - such that I can get a report, probably from msbuild, which settings are different.

  • It's always interesting to hear about fixes that yield 2+ orders of magnitude improvement in performance.  Can you elaborate on what the key change was?

  • We would appreciate any help in detecting any different projects and file level settings in a multi-project C++ application solution.

    Our goal is to use the same project level settings for all projecs in the solution.  We would also like to not use any file level custom settings.

    We want to be able to answer the basic question:

    Will this project build from an arbritrary directory on a machine with VS version 2010 without any extra manual tweaking?

    We want to detect and avoid things like:

     - hard coded paths to DLLs, projects, solution files, h files, c++ files

     - custom pre and post build steps

     - reliance on multiple copies of the same file (H or dll) in different subdirectories of the same project

    Some real world cases we've seen:

     - Developer creates mapped drives Q:, R:, S:, etc and then hard codes them to different projects (e.g., all gui dlls are on the Q: drive).  This is a poor attempt at providing location independence.  We want to start each project at a root directory (e.g., c:\dev\abc) and the have all files for that entire solution in that directory outside of builtin win32 OS dlls, .net BCL DLLS, etc.

      - Have multiple projects in a solution divided out for purposes of TDD unit testing only with duplicate copies of the same DLL and H files in different directories.  We want to combine the projects in a large solution to remove the overhead of managing multiple projects, take read advantage of whole program optimization and greatly reduce the number of files in our source and installation.

     - Developer links in several open source dll files into an application to use a small portion of the functionality because they want to use a particular coding style (e.g., object relational mapping DLL created from 50,000+ lines of C++ that is used only to store/retrieve a simple 5 object structure).  That extra code adds significant risk and maintenance time to a large project as the benifit of using it is tiny when compared to the risk of adding 50,000+ extra unknown lines of code to our application.

  • Hello cppboy

    As SvenC correctly suggests, please use http://connect.microsoft.com/VisualStudio to submit a bug on this. On a quick look at our current codebase we cannot see this issue, so please supply as much detail on your configuration as possible.

    Thanks

    Damien

  • Greg,

    I wrote a tool for comparing vcproj settings, it's free and available on my web site here - http://www.cix.co.uk/~gort/win32.htm#VCProjCompare.

    It won't show file level overrides, but it's on the TODO list :-)

  • Hi Greg,

    Thanks for sharing feedback and suggestion on this blog. These are really good suggestions and we recognize the value that such tool would provide to the C++ developer community. Unfortunately we would not be able to deliver this functionality in the box. If you are interested you could start a community project on CodePlex and we will try to help out by contributing to this effort.

    You can contact me directly at amitmo at microsoft dot com to discuss this further.

    Thanks,

    Amit

Page 1 of 1 (10 items)