It's great that you're looking at the linker as it's a big bottleneck on larger projects. I'm in games development and the link time is easily the biggest time waster.

Parallelising the pdb generation seems like a good start, but I think it would also be good if you could review the algorithmic complexity of the debug information compilation and symbol management as well to see if there are any easy wins there. Especially when you start using templates you end up with lots and lots of redundant debug data and empirically it seems to affect link quite a bit.

Only per object file (cpp) __pragma init_seg(...) available to set the initialization order.

No way to set this per c++ object basis like:

CMyObject __declspec(linkinit(56)) MyObject;

A second thread for writing the PDB is nice, but still won't make much use of today's multi-core CPUs. Is there no way the bulk of the work can be split across many threads? It's annoying to see Task Manager reporting 12.5% CPU usage for link.exe on a Core i7... still, I'm glad it's getting some attention.

Do any of the test case include machine generated source code files?  Please provide some test performance numbers when generating a lexer/parser using yacc or another compiler generator for a decently large language (c++, c, pascal, etc).

Most comiler/linker pairs do miserably with generated code given the tens of thousands of symbol names and large amounts of goto code.

Avoid the java 1 class per file scheme.  It is quite slow over the long run considering it takes a few extra seconds over and over throughout the life of the project.

Avoid making hundreds of extra 1 liner classes just for completeness.  They can be wrapped into a large catch all class (e.g., a large class that does data type conversions from machine specific to not machine specific basic data types).

Avoid having 10 different overloaded versions of the same method.  Make 1 method that does all of the work and have optional pareters to provide the equivalent signature to the caller.  (NB: I've stopped using optional arguments in the last 2 years since I've seen offshore written code that only supplies the minimal set of parameters to a function instead of calling the function the right way for the time it is needed.  I think this is the . intellisense lazy method.)

Avoid creating unnecessary names and extra function calls just for coding style.

.NET, java, reflection and modern tools promote an explosion of names and identifiers but that, while simple in the narror scope, is detrimental in the long run.  More names/symbols means more costly and harder to progress the project in the future.

@CMWoods,

It is standard practice in most professional development shops to store PDBs for all released binaries, even if the PDBs themselves are not released, for help in debugging. Adding /debug to the default template was an effect, not a cause.

Also, how ICF (indentical COMDAT folding) affects the link time? Default BUILD script enables it by default, even in checked builds (which I believe is bad for setting breakpoints during debugging).

I am working in the team developing huge C++ applications with circa 4000 source files and 2500 obj files. VS2008. Debug and Release configuration are built with debug info without /LTCG flag. Debug use incremental linking, Release don't use it.

My machine is double core 2.3Gz 4096Mb

Our concern are unsatisfactory linking times. Linking Debug configuration takes between 20 and 1200 sec.

Linking Release lays between 270 and 1800 sec.  Besonders much time takes linking with IncredBuild also with high priority.

I have two suggestions on the issue.

First linker takes 25 minutes to link and after that its CPU link time is only 19 seconds and storage 350Mb. It would be nice if you will have a special option like /HUGEPROJECT and  read all object segments into the storage also on the first step. All modern computers have a lot of memory but it is simply doesn't used by your linker.

Second, your linker obviously lacks satisfactory prelink check in the incremental modus to know whether it is possible to update the project in proper time. Every linker starting from 7.0 fails to link in acceptable time, if the changes in files are a little more than tiny. In result non-incremental linking takes 4 minutes, incremental 15. It is something other than a satisfactory performance.

With Regards,

Boris

The VC++ team spends a lot of time investigating linker throughput issues.  The multi-threading work was done after demonstrating, through significant performance analysis of multiple scenarios, that it did actually increase overall performance

Maybe your scenario exposes unique performance problem, but overall, I don't think we have obvious bad practices in the linker.  

Maybe I am wrong, and maybe there are. All I can say is we will continue to analyze linker performance, spending time understanding IO access patterns to further optimize throughput.

We know linker throughput is one of the most important things to our customers, and we'll keep doing whatever we can to improve it.

I agree with the cpp->obj->module comment.

If I was in the compiler/linker business, I would start using RAM.

The result is a service/daemon:

- In-memory dependency graph(FindFirstChange etc.)

- Traditional dependency check will not be necessary 99.99% of the time. It's done already.

- Compiler pipes output directly to daemon

- Daemon links in-memory

- Daemon keeps result (dll/exe) in-memory

- Daemon outputs dll/exe on-demand.

Patrick