GC.Collect()

A look at service startup issues

parthas — Thu, 30 Aug 2007 15:59:00 GMT

I was discussing various ways to do a live debug of a strange issue with one of my colleagues and came up with an interesting option which I thought I should share here. Let’s consider we have a windows service which is causing some trouble during the startup. The trouble could be either a service crash or something that you expect the service to do which it isn’t doing. Let’s assume the service is crashing on startup here.

You have quite a few options if your service is running for a while and then crashing. You can attach one of the debuggers to the service, wait for a second chance exception to happen, and then proceed with analyzing further by looking at the call stack/exception or capturing a memory dump to analyze this later. But we are not left with these options if the service is crashing on startup as you would have no time to attach the debugger to the service before it crashes.

For this scenario, you can go for Image File Execution (IFE) as explained in ‘Configure a service to start with the WinDbg debugger attached’ section of the KB article – KB 824344

Note that this KB article talks about enabling “Allow Service to Interact with Desktop” option for the service. This option, as the name suggests, allows the service to interact with desktop without which the UI displayed or debugger attached through IFE would not be visible in the desktop. Your service would land up in a hung state if this UI is waiting for some user’s input.

So how would you troubleshoot an issue when the issue happens only when this option is not selected? How would you troubleshoot such a startup issue when the service is not running under the default SYSTEM account (for which you do not seem to have an option to allow service interact with desktop in the SCM)? This is exactly the scenario that we had to troubleshoot.

This is when you can make use of Remote Debugging option in WinDBG. Remote debugging might not be exactly meant to troubleshoot such a scenario but yeah, can be made use of here to achieve what we want. As you might be aware, remote debugging would have a server instance (which actually gets attached to the process) and a client instance (where you do the debugging after connecting to the server instance).

I am not getting too much into the details of WinDBG’s remote debugging as it is already nicely documented in the help file that comes along with it. But what we need to do here is attach an instance to of WinDBG as a ‘server’ to the crashing service using IFE as explained above. We would not need to take a look at this instance as this is just a server which means we would not need to allow the service to interact with desktop. Now, launch another instance of WinDBG on the same machine (this would act as the client) and connect this to the server so that we can do a live debug of the service.

Setup:

Ø Follow the instructions in ‘Configure a service to start with the WinDbg debugger attached’ section of the KB article – KB 824344. Note that you can safely ignore Step 2 in this section as you would no longer need the service to interact with desktop.

Ø Add the following command line parameters to the path of the debugger: (let’s assume debugger is installed in ‘C:\Debuggers’ instead of ‘C:\Program Files\Debugging Tools for Windows’ for convenience)

C:\Debuggers\WinDBG.exe -G -c ".server tcp:port=1234"

Ø Start your service from Service Control Manager.

Ø By now the debugger is already connected to the service as a ‘server’. You would have reached the ‘Initial Breakpoint’ waiting for a g (Go) command from the debugger.

Ø Launch another instance of WinDBG and connect to the ‘server’. There are many ways to do this; the simplest would be to go to File->Connect to Remote Session->Browse->Enter machine name->Refresh. Hit OK on your ‘server’ that is listed to connect.

You are now all set to do a live debug of the service startup issue for a service which is not allowed to interact with desktop. You can play around with the command line parameters to WinDBG depending on your needs. WinDBG -? contains a detailed description of the command line syntax that WinDBG uses.

Happy debugging!

Link for reference:

Services

Debugging 64-bit applications using Visual Studio 2005

parthas — Thu, 26 Apr 2007 21:46:00 GMT

A short note on this issue that I came across recently. Let’s say we are trying to debug a managed/unmanaged code in Visual Studio 2005 in a 64-bit machine. We complete the code and hit F5 (debug the application), and land up with the following error:

Error while trying to run project: Unable to start debugging.

The components for the 64-bit debugger are not registered. Please repair your Visual Studio 2005 Remote Debugger installation via 'Add or Remove Programs' in Control Panel.

Okay all that we are doing is debugging an application in the local machine. So why should there be a problem with the remote debugger installation? Here is some background on how debugging works in 64-bit environment that explains why you see this error message.

Even on a 64-bit machine, Visual Studio runs as a 32-bit application. This means that Visual Studio would run in the Windows-on-Windows (WoW64) layer. It would not be possible to attach a 32-bit debugger to a 64-bit process. However, if you want to debug a 64-bit application, there is something special that Visual Studio does to make this possible. We would achieve this pseudo-remotely. You are running the debugger on the same machine as the process but debugging is done through the remote debugger. All this happens behind the scenes and you as a developer would not need to do anything special.

Now, if you are seeing this error on a 64-bit machine, I would assume that the remote debugging components were not included during the Visual Studio installation process. All that we would need to do is install the remote debugging components using the link below following the instructions under the section ‘To install remote debugging components (non-SQL Server 2005)’. And yeah, we would not need to worry about the remote debugging permissions since we are just debugging an application in the local machine in this case.

Link for reference:

How to: Set Up Remote Debugging

Troubleshooting Stack Overflow Exceptions

parthas — Thu, 19 Apr 2007 21:02:00 GMT

It’s been a while since I had updated this blog and thought would discuss in general regarding the stack overflow exceptions. To troubleshoot this issue further, got hold of a simple code snippet (which would be pretty obvious once you take a look at the snippet J ) that would throw a stack overflow exception.

//Stack overflow code

void Foo(int level)

{

const int iBlockSize = 100000;

char szHugeBlock[iBlockSize];

printf("StackOverflow count: %d\n", level);

Foo(level+1);

}

int _tmain(int argc, _TCHAR* argv[])

{

Foo(0);

return 0;

}

I had deliberately used a huge block size to quicken the exception. This example might be straight forward but what would follow would help in determining issues in much complex scenarios as well, hopefully.

Let’s attach the debugger to the process to see how the call stack looks after the crash.

Attaching WinDBG to this process and letting it to crash has the following call stack:

0:000> kbL

ChildEBP RetAddr Args to Child

0003b280 00411410 0000000a 0006c1b0 00053a24 StackOverflow!_chkstk+0x27

00053a18 00411410 00000009 00084948 0006c1bc StackOverflow!Foo+0x50

0006c1b0 00411410 00000008 0009d0e0 00084954 StackOverflow!Foo+0x50

00084948 00411410 00000007 000b5878 0009d0ec StackOverflow!Foo+0x50

0009d0e0 00411410 00000006 000ce010 000b5884 StackOverflow!Foo+0x50

000b5878 00411410 00000005 000e67a8 000ce01c StackOverflow!Foo+0x50

000ce010 00411410 00000004 000fef40 000e67b4 StackOverflow!Foo+0x50

000e67a8 00411410 00000003 001176d8 000fef4c StackOverflow!Foo+0x50

000fef40 00411410 00000002 0012fe70 001176e4 StackOverflow!Foo+0x50

001176d8 00411410 00000001 0012ff48 00000000 StackOverflow!Foo+0x50

0012fe70 00411535 00000000 00000000 00000000 StackOverflow!Foo+0x50

0012ff48 00411b26 00000001 00193d30 00195678 StackOverflow!wmain+0x25

0012ff98 0041196d 0012ffac 77913833 7ffd7000 StackOverflow!__tmainCRTStartup+0x1a6

0012ffa0 77913833 7ffd7000 0012ffec 77b7a9bd StackOverflow!wmainCRTStartup+0xd

0012ffac 77b7a9bd 7ffd7000 0012d144 00000000 kernel32!BaseThreadInitThunk+0xe

0012ffec 00000000 00411082 7ffd7000 00000000 ntdll!_RtlUserThreadStart+0x23

I would assume that we already got the debugger configured with the public symbol server. Take a look at the reference section below for details on this.

The last function that seems to have got called is the StackOverflow!_chkstk. It makes more sense to proceed further now. StackOverflow is the name of the application that’s crashing.

Taking a look at the number of threads:

0:000> ~

. 0 Id: 15ac.1b14 Suspend: 1 Teb: 7ffdf000 Unfrozen

There is just one thread and this is the thread that needs to be looked at further. Before we can confirm that this is a stack overflow exception, taking a look at the current stack size and the maximum stack size would help.

To get details on the maximum stack size, we would need to look into TEB data structure. This value is stored in the DeallocationStack field.

0:000> dt _TEB 7ffdf000

ntdll!_TEB

+0x000 NtTib : _NT_TIB

+0x01c EnvironmentPointer : (null)

+0x020 ClientId : _CLIENT_ID

+0x028 ActiveRpcHandle : (null)

+0x02c ThreadLocalStoragePointer : 0x7ffdf02c

+0x030 ProcessEnvironmentBlock : 0x7ffd7000 _PEB

...

+0xc00 StaticUnicodeBuffer : [261] "kernel32.dll"

+0xe0c DeallocationStack : 0x00030000

+0xe10 TlsSlots : [64] (null)

...

+0xfe8 TotalSwitchOutTime : 0

+0xff0 WaitReasonBitMap : _LARGE_INTEGER 0x0

<Structure details cut to save space>

Get the stack base details...

0:000> !teb

TEB at 7ffdf000

ExceptionList: 0012ff88

StackBase: 00130000

StackLimit: 00031000

SubSystemTib: 00000000

FiberData: 00001e00

ArbitraryUserPointer: 00000000

Self: 7ffdf000

EnvironmentPointer: 00000000

ClientId: 000015ac . 00001b14

RpcHandle: 00000000

Tls Storage: 7ffdf02c

PEB Address: 7ffd7000

LastErrorValue: 0

LastStatusValue: c0000135

Count Owned Locks: 0

HardErrorMode: 0

And now the maximum stack size

0:000> ? 00130000-00030000

Evaluate expression: 1048576 = 00100000

0:000> ? 1048576 / 1024

Evaluate expression: 4132 = 00001024 ß 1 MB

Find out the current stack size for this thread from StackBase and StackLimit field.

StackBase: 00130000

StackLimit: 00031000

0:000> ? 00130000-00031000

Evaluate expression: 1044480 = 000ff000

0:000> ? 1044480 / 1024

Evaluate expression: 4128 = 00001020 ß Just lest that 1 MB

We can say for sure that the stack size is complete and we would need to re-look at the code to see what is going wrong.

So how do we determine which function has taken up most of the stack size? Disassembling the functions should provide details on this. From the call stack above, let’s take a look at how much of stack space wmain() function has occupied.

0:000> u StackOverflow!wmain

StackOverflow!wmain

004114a0 55 push ebp

004114a1 8bec mov ebp,esp

004114a3 81ecc0000000 sub esp,0C0h

004114a9 53 push ebx

004114aa 56 push esi

...

The sub instruction above on the stack pointer is allocating size on the stack that is required.

This would apply in most of the cases but let’s take a look at the function Foo() for the stack size details.

0:000> u StackOverflow!Foo

StackOverflow!Foo

004113c0 55 push ebp

004113c1 8bec mov ebp,esp

004113c3 b880870100 mov eax,18780h

004113c8 e883fcffff call StackOverflow!_chkstk (00411700)

...

So what is the _chkstk routine all about? This link from MSDN has description on what this routine is all about.

[MSDN] Called by the compiler when you have more than one page of local variables in your function.

From the code snippet that we have taken,

const int iBlockSize = 100000;

char szHugeBlock[iBlockSize];

the size of this variable would approximately take 100K which is way greater than 4096 bytes. The value in EAX (18780h) should give the size allocated in stack in this case. 18780h (in Hex) should be somewhere close to 100K and 10 calls to this function should end up throwing a stack overflow exception, which is the case here.

Links for reference:

Visual Studio 2005 Service Pack 1

parthas — Fri, 15 Dec 2006 19:24:00 GMT

The much awaited service pack for Visual Studio 2005 is out! It can be downloaded from this link.

Check this link for more details on VS 2005 SP1. Fresh out of the box. Happy patching everybody J

Application crash on replacing sscanf() with sscanf_s()

parthas — Wed, 06 Dec 2006 00:34:00 GMT

Here is the typical scenario. You have upgraded your code which uses sscanf() from one of the previous versions of Visual Studio to Visual Studio .Net 2005. Building this code in VS 2005 is going to throw a warning at build time suggesting you to make use of a more secure version of this function – sscanf_s() instead. You replace sscanf() with sscanf_s(), and this causes your application to crash when it is called at runtime.

Are we hitting upon a possible bug in sscanf_s()? No.

One of the things that you might want to take a look at is the ‘Remarks’ section in the MSDN documentation for sscanf_s()

The ‘Remarks’ section states:

The sscanf_s function reads data from buffer into the location given by each argument. The arguments after the format string specify pointers to variables with a type that corresponds to a type specifier in format. Unlike the less secure version sscanf_s, a buffer size parameter sizeOfBuffer is required when using the type field characters c, C, s, S and [. This parameter must be supplied as an additional parameter after each buffer which requires it. For more information, see scanf_s, _scanf_s_l, wscanf_s, _wscanf_s_l and scanf Type Field Characters.

Let’s take a look at this code snippet –

Before VS 2005:

Say VS 2003, application with this code snippet using sscanf() is not going to crash.

char szInput[] = "Sample input";

char szFormat[10];

sscanf( szInput, "%s", szFormat);

After upgrading to VS 2005:

Application with this code snippet using sscanf_s() is going to crash.

char szInput[] = "Sample input";

char szFormat[10];

sscanf_s( szInput, "%s", szFormat);

After upgrading to VS 2005 and the changes:

This one is going to work just fine.

char szInput[] = "Sample input";

char szFormat[10];

sscanf_s( szInput, "%s", szFormat, sizeof(szFormat));

Also pay attention to the line in ‘Remarks’ that this parameter (buffer size parameter) must be supplied as an additional parameter after each buffer which requires it. An example code for this is provided here.

Note: One of the warnings that you might come across at build time when using CRT functions while upgrading to VS 2005 is regarding the use of deprecated functions. Many CRT functions now have secure version to make CRT more secure. The new version has the _s ("secure") suffix while the less secure version is marked as deprecated. You may want to check this link for details on security enhancements in the CRT.

Creating Program Alias

parthas — Wed, 04 Oct 2006 17:47:00 GMT

This is one feature that I use extensively and it helps me a lot in quickly firing up the tools that I use frequently. We got desktop shortcuts, start menu shortcuts for programs, but how about making use of “Run” dialog shortcuts? That is exactly what we are talking about – creating a program alias. It makes life a lot more simpler when we got to type “n” instead of “notepad.exe” in the “Run” dialog to bring up notepad to life, doesn’t it?

Okay before we get started, here goes the warning: Registry modifications are permanent; you may consider taking a look at this link before you make any modifications to registry and take a backup if required before making any changes to it.

Here is how you do it:

· Run RegEdit.exe and navigate to the following path: [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\App Paths]

· Create sub-key and give it your alias’ name (“n.exe” in this case)

· Modify the default value of the sub-key with the path and filename of the application you wish to be fired up using this alias (Should be "C:\Windows\notepad.exe" here)

· Exit from the registry and test your new alias from “Run” dialog (Start menu -> Run command) by typing in the name of the alias (“n”) to see notepad.exe getting invoked.

Here are some of the most common aliases that I make use of:

n Notepad

i Internet Explorer

w WinDBG Debugger

f FileMon

r RegMon

p Process Explorer

Okay there are quite a few more and I am not listing them all here.

Note that you can also create a shortcut (.lnk file) to a folder that you make use of very often and get an alias created for that. This should be something like making use of an alias (say “o”) to open “C:\SampleDir” using the Windows Explorer.

Debug a process which is attached to another Debugger

parthas — Wed, 04 Oct 2006 05:09:00 GMT

For any process, we can attach only one debugger at a time. We might hit upon a need to attach more than one debugger to a process for various reasons while troubleshooting/debugging an issue, some of which that I had come across are scenarios like

1) A need to attach another debugger (WinDBG) while debugging the code in Visual Studio.

2) Troubleshooting issues when we get an exception only while having the process attached to a debugger.

We might see the following error message while trying to attach WinDBG to a process that is being debugged by Visual Studio.

“An attempt to set a processes DebugPort or ExceptionPort was made, but a port already exists in the process.”

One way to debug such a process from another debugger is to use the noninvasive debugging mode support provided by WinDBG. To debug a running process noninvasively, we need to

Select File -> Attach to a Process from WinDBG menu.
Select the required target process in the Attach to Process dialog box.
Select the Noninvasive check box.
Click OK

We can also make use of the WinDBG command line by using the –pv option and specifying the Process Name by using –pn.

Here’s an example:

WinDBG -pv -pn <Process Name>

Note that we might not have many of the functionalities of the debugger while in noninvasive mode – we might not be able to set breakpoints, step-into or step-over the code, etc. This is because while in noninvasive debugging mode, the debugger does not actually attach to the target application. The debugger will suspend all of the target process’ threads and will have access to its memory, registers, and other such information.

Some of the useful commands that we can make use of while in noninvasive debugging mode using WinDBG are

.dump /ma <DumpPath> To create a user mini dump file with full memory for the application

~ Thread status

~*kb Call stack for all the threads

There are a lot of other useful commands that come along with WinDBG, you may want to take a look at the Help that comes along with WinDBG for more information.

Incase we are just looking at creating a memory dump of the process that is being debugged using the Visual Studio debugger, we can also make use of the “Save Dump As” option that comes with Visual Studio 2005 Debugger. A very useful option which eliminates the need for making use of another debugger when we just need to create a dump of the debugged process for later analysis.

We can access this option from Debug menu -> Save Dump As option -> select Minidump or Minidump with Heap based on the requirement. Here is a link from MSDN for more information on saving and opening dump files using Visual Studio 2005 Debugger.

Links for reference:

Redistributing Visual C++ Libraries

parthas — Wed, 13 Sep 2006 21:03:00 GMT

While deploying an application developed on VS 2005, we will have to make sure that the target machine has the Visual C++ libraries that are required to run this application. If we suspect that the target machine might not have the VC++ libraries already, all we need to do is redistribute the VC++ files along with the application.

I had earlier discussed a typical scenario where we would have to redistribute the VC++ libraries. Here is the link.

Here are the various deployment options for redistributing the VC++ libraries.

Options:

Use Visual C++ Redistributable Package (VCRedist*.exe)
Use the MSM merge modules
App local deployment (Private Assembly install under the application directory)

1. Use Visual C++ Redistributable Package (VCRedist*.exe)

The Visual C++ Redistributable Package (VCRedist_x86.exe, VCRedist_x64.exe, VCRedist_ia64.exe) is a standalone exe that installs all the VC++ libraries to the WinSXS directory. It is present in the following path after you have installed VS 2005 in your machine: %PROGDIR%\Microsoft Visual Studio 8\SDK\v2.0\Bootstrapper\Packages\vcredist_x86\vcredist_x86.exe

The VC++ 2005 Redistributable Package is released to the web and can be downloaded from the Microsoft site. Refer the "Links for reference" section below for the links.

2. Use the MSM merge modules

Merge Modules can be used to install the VC++ libraries to the WinSxS directory. The various merge modules can be located at %PROGDIR%\Common Files\Merge Modules

These MSMs can be merged into the application setup project.

Here is a quick look at the various merge modules present:

ATL

X86 - Microsoft_VC80_ATL_x86.msm, policy_8_0_Microsoft_VC80_ATL_x86.msm

IA64 - Microsoft_VC80_ATL_x86_ia64.msm, policy_8_0_Microsoft_VC80_ATL_x86_ia64.msm

X64 - Microsoft_VC80_ATL_x86_x64.msm,policy_8_0_Microsoft_VC80_ATL_x86_x64.msm

CRT

X86 - Microsoft_VC80_CRT_x86.msm, policy_8_0_Microsoft_VC80_CRT_x86.msm

IA64 - Microsoft_VC80_CRT_x86_ia64.msm, policy_8_0_Microsoft_VC80_CRT_x86_ia64.msm

X64 - Microsoft_VC80_CRT_x86_x64.msm, policy_8_0_Microsoft_VC80_CRT_x86_x64.msm

MFC

X86 - Microsoft_VC80_MFC_x86.msm, policy_8_0_Microsoft_VC80_MFC_x86.msm

IA64 - Microsoft_VC80_MFC_x86_ia64.msm, policy_8_0_Microsoft_VC80_MFC_x86_ia64.msm

X64 - Microsoft_VC80_MFC_x86_x64.msm, policy_8_0_Microsoft_VC80_MFC_x86_x64.msm

OpenMP

X86 - Microsoft_VC80_OpenMP_x86.msm, policy_8_0_Microsoft_VC80_OpenMP_x86.msm

IA64 - Microsoft_VC80_OpenMP_x86_ia64.msm, policy_8_0_Microsoft_VC80_OpenMP_x86_ia64.msm

X64 - Microsoft_VC80_OpenMP_x86_x64.msm, policy_8_0_Microsoft_VC80_OpenMP_x86_x64.msm

MFCLOC

X86 - Microsoft_VC80_MFCLOC_x86.msm, policy_8_0_Microsoft_VC80_MFCLOC_x86.msm

IA64 - Microsoft_VC80_MFCLOC_x86_ia64.msm, policy_8_0_Microsoft_VC80_MFCLOC_x86_ia64.msm

X64 - Microsoft_VC80_MFCLOC_x86_x64.msm, policy_8_0_Microsoft_VC80_MFCLOC_x86_x64.msm

Here is a good link on how to deploy using a setup and deployment project.

Note that Debug DLLs cannot be redistributed. They are shipped with VS for debugging purposes. Refer this link for details on redistributing using Merge Modules.

3. App local deployment (Private Assembly install under the application directory)

The VC++ libraries can also be installed under the application directory. The required files are placed at %PROGDIR%\Microsoft Visual Studio 8\VC\Redist

The required libraries can be copied to the application directory. (we would need to copy the sub-directories)

Here is a good link for more details on app local deployment

Comparison:

A quick comparison of the three options:

	VCRedist.exe	*Merge Modules (.MSM)**	App Local Deployment
Setup	Application does not require setup project.	Application should have its own setup project.	XCopy deployment.
Required Libraries	We will have to install all the VC++ libraries irrespective of whether they are used by the application.	Deploy only the assemblies that the application depends on.	Deploy only the assemblies that the application depends on.
Administrative privileges	Requires	Requires	Does not require
Others	Package has to be installed once on the machine and all applications can make use of the libraries present in WinSxS.	MSMs have to be installed once on the machine and all applications can make use of the libraries present in WinSxS.	Duplication of the VC++ libraries in the case of multiple applications that is present in different directories.

Edit: Added new links for downloading Microsoft Visual C++ 2005 SP1 Redistributable Packages in the 'Links for reference' section.

Links for reference:

Redistributing Visual C++ Files
Choosing a Deployment Method
Determining which DLLs to Redistribute
Links for downloading the Microsoft Visual C++ 2005 Redistributable package:

Links for downloading the Microsoft Visual C++ 2005 SP1 Redistributable Package:

A closer look at Hello World developed using VS 2005

parthas — Wed, 13 Sep 2006 02:04:00 GMT

Okay now that I had spoken about the Hello World program, let's take a closer look at one of the common issues that might pop-up when we run an application developed using Visual Studio .Net 2005 in a machine that does not have VS 2005 installed in it.

To get this going, I created a simple Win32 console application with the following all-time famous "Hello World" line of code added to it.

printf("Hello World\n");

You may consider checking this link for help on creating a Win32 console application.

Running this on the machine that I used to develop this project brought the application into life displaying "Hello World". Just the way we would expect it to work.

Now starts the trouble. Taking this "Hello World" application and running it on a machine that does not have VS 2005 might not work the way we expect it to. The reason for this is probably that this machine does not have the Visual C++ libraries that are required to run the application developed using VS 2005.

Here is the error that I got when I ran this program on a machine that does not have the required Visual C++ libraries:

"This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem."

The other errors that we might come across are:

"The system cannot execute the specified program."

"The application failed to initialize properly (0xC0150002). Click on OK to terminate the application."

Here is a screen shot of the error:

It would be a good idea to look at the event log to get a better picture of what the issue is. When you look at the "System" event log for error messages with source "SideBySide", you would find the following in the "Description"

"Dependent Assembly Microsoft.VC80.CRT could not be found and Last Error was The referenced assembly is not installed on your system."

This application is dependent on Microsoft.VC80.CRT which is not present in this machine. All we need to do is to redistribute the required Visual C++ libraries to get this back to form, displaying "Hello World"!

Check this link for details on Redistributing Visual C++ Libraries.

Hello World

parthas — Tue, 12 Sep 2006 15:14:00 GMT

How else do I start this blog, other than doing it the programmer's way?

To all the non-programmers who are wondering what it is all about, a "Hello World" is a program that prints out "Hello World" on the display monitor, commonly used as an introductory tutorial for learning a programming language. It is usually the first step into the world of programming, my first step into the world of tech blogging.