Posted by: Upender Sandadi
 
(reposting since text formatting got messed up first time)

This is the second entry in the KITL series. In the first entry we learnt aboust basic KITL packet layout. In this we will look at some of the common things we have run across in debugging KITL issues on devices in-house. This covers mostly KITL over ethernet. For KITL over serial and KITL over USB, I would recommend getting a trace and analyzing the KITL traffic using the packet layout information.

Tracing tools:
Some of the tools which can be used to capture KITL and debug traffic are:

• You can use netmon.exe (from Microsoft) or Ethereal 
• You can use a null modem cable to capture all the serial output traffic from the device and use Hyperterm or Teraterm to capture output from a serial port on the desktop.

Connectivity Issues:

Image downloads to the device but KITL doesn’t connect to the desktop

• Make sure KITL is initialized on the device: Unfortunately this is very early in the boot phase. So you would have to use custom debug messages in OEMInit() to debug further. One possible problem could be a conflict of an IRQ with another device on the target. For CEPCs check the autoexec.bat to make sure IRQ is set to 0 or to a non-conflicting value. 
• Make sure the device is not connecting to a wrong Platform Builder session on the host: This mostly happens if you have multiple PB sessions and one session is already connected to a device you are trying to establish connection with. You can detect this in the serial output where KITL usually prints the IP address of the host it is connecting to.
• Make sure you are using compatible network card: Some images require you to have right set of network card in the device for KITL to establish a connection. For the most part if you use NE2000 compatible network card, this issue should be resolved.
• Some times KITL interrupt initialization can fail and image will boot in polling mode. This is something one should be aware of if there seem to be any perf issues or issues booting up. Serial output from the device will indicate if the image is running in polling mode or interrupt mode. One can also break into the debugger and look at {,,nk.exe}Kitl.Interrupt (in Windows CE 6.0 this is referenced as {,,kitl.dll}Kitl.Interrupt) value to see if the KITL interrupt is initialized or not. If it is not initialized, then that would be set to 0xFF.
• Make sure that the device name generated by the bootloader matches the device name generated by KITL inside the download image – Platform Builder looks for the same device name to identify which device to connect with.

      Best tools to debug in these scenarios are:

• Serial output from the device. If you are creating your own images then enable all KITL debug zones and one should see all the KITL activity on device serial output. If you see KitlSend messages on serial output and yet the KITL packet doesn’t show up in the netmon trace, then most likely the problem is with the network card.
• Network capture (using Ethereal or netmon) of all traffic to/from the device.
• Custom debug messages in the OAL and KITL startup code.

Image downloads but KITL hangs after printing a message which says “Loading nk.exe …”.

• One would typically see this if the image has debugger built in but the settings on desktop for debugger are not properly set. Make sure KdStub is selected as the debugger option in the Platform Builder target connectivity options for your device.

Image downloads but KITL hangs after printing a message which says “Loading relsfd.dll …”.

• If the image has debugger, then try and break in the debugger to see why KITL relfsd calls are hung. This could be for several reasons:
  
• Platform Builder version is not compatible with the OS version.
• Some high priority thread is spinning leading to starvation for the KITL threads.
• KITL Interrupts are turned off inadvertently leading to connectivity failure.
• If the image doesn’t have debugger, then get the KITL trace and analyze the KITL packets to see why the desktop and device are not in sync.

This is definitely not an exhaustive list. Please add to this list if you run into a KITL issue with your custom devices and how you solved it.