In this deep dive, we continue this series to explain SharePoint crawls by using ULS to provide a detailed step-by-step of events that occur when starting a crawl. If you've ever had a crawl get stuck in starting or needed to troubleshoot the start of a crawl, then this should help explain where to look next. In this and upcoming posts, I'll again show the same overall flow, but this time using key ULS events to illustrate a deep dive through an expected path when crawling an item.
There is an undocumented assumption baked into SharePoint Search that the Default Public URL of a Web Application will be crawled. If you want everything to work auto-magically, crawl the Web Application's Public URL for the Default zone. Otherwise, when crawling a non-Default zone, expect things to break. Read here to understand why.
As I began in part 1, seemingly “sporadic” query problems are often just straightforward failures being masked by the three levels of load balancing involved with a SharePoint 2010 Search Query and my goal here has been to help unravel all the moving pieces. Here, I described the load balancing that occurs when there are multiple instances of the SQ&SS (e.g. Farm level load balancing) and from this, begin to explain why the "Internal Server Error Exception Occurred" failure is completely ambiguous.