As I mentioned in Part I, index.dat files are used by various features of IE to keep track of URLs and associated information about URLs. The two primary consumers of these files are the Temporary Internet Files folder (TIF) and History.
TIF uses an index.dat file to map URLs to the associated Cache file name. Developers can use the GetUrlCacheEntryInfoEx() function to perform this lookup.
History, which I am more familiar with, uses an index.dat file to keep track of which URLs should be displayed in History. Additionally, it stores other metadata about the url, such as the number of visits. This information is consumed by many features of IE. Off-line favorites uses it to keep track of subscription information. Shortcuts use it to find the Favicon. Auto-complete uses it to create a list of options to display in the drop-down box. Link coloring in the HTML rendering engine uses the list to decide how to color any given link on the page.
Furthermore, History actually uses more then one index.dat file. If you look in the [user profile]\Local Settings\History\History.IE5\ directory, you will see index.dat and some directories of the form MSHist###################. Those directories also contain index.dat files. The root-most index.dat file is where general history information is stored. The ones in the MSHist directories are used to support alternate views of History. If you open the History Explorer Bar (CTRL+H) and click the View drop down (ALT+W) you will see the various views that IE supports.
More to follow.