Hi, Andreas Fuchsberger here.....
It is important to understand what happens CAT.NET builds its Call Flow Super Graphs. We use a CCI object called CciControlGraph to build a Control Flow Graph for each method and each method call we find in the Common Intermediate Language (CIL) of the modules being analysed. These individual control flow graphs are collected together in a .NET List object.
Unfortunately this is where we start to encounter a shortcoming in the implementation. Even with virtual memory there are limits to how much memory a single .NET application can allocate. As reported in recent blog post a 32-bit process, such as the CAT.NET Visual Studio plug-in version can only allocate about 1200 MB, even on a 4GB RAM (32-bit) system. Moreover another shortcoming of the current implementation is that when CAT.NET runs out of memory is it exits with an unhandled OutOfMemory (OOM) exception, unfortunately this does not get reported by the Visual Studio plug-in and the plug-in just seems to hang.
There is a solution, on the same 4 GB RAM system you can more than double the amount of memory available to CAT.NET by using a 64-bit version of Vista or Windows Server and the 64-bit command line version of CAT.NET. This is why we supply the 64-bit command line version of the CAT.NET binaries: (http://www.microsoft.com/downloads/details.aspx?FamilyID=e0052bba-2d50-4214-b65b-37e5ef44f146).
For those without the option of running on a 64-bit system, one manual method for analysing larger applications that the 32-bit version of CAT.NET fails to do is to restrict the number of modules that are passed to CAT.NET. For a developer analysing their own code it is relatively easy to make sensible choices which modules should be analysed together. Typically they would choose all the their own as code that receives user input and any dependant modules, e.g. third party libraries, that process that input to produce an output, leaving all other business logic modules aside. However it must be pointed out that this method lead to some vulnerabilities being missed.
We are working on scalable long term solutions to these sorts of problems which require some relatively heavy lifting on our part. For today hopefully the above advice will at least help understand why the issue happens.