Jie Li's GeekWorld : iFilter

Index and Search PDF Files in SharePoint Server 2010

Jie Li — Tue, 20 Oct 2009 16:38:00 GMT

Like Office SharePoint Server 2007, there’s no OOTB PDF iFilter in SharePoint Server 2010. If you add PDF as a file type for SharePoint Search, you will get the following result:

You can see that only the file attributes are indexed.

You need to install a x64 PDF iFilter for this. There’re three PDF iFilter on market, Adobe, Foxit, and TET. You can refer to my earlier post for comparison. Since the registry name is changed in 2010, you may need to manually modify it to make the iFilters registered. Foxit recently updated their installer to reflect this change.

http://www.foxitsoftware.com/pdf/ifilter/

Quote from Foxit PDF iFilter change log:

Version Number: 1.0.0.3213

* Fixes a crash issue that is caused by embedded fonts.

* Adds the following registry settings in the installation program:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\14.0\Search\Setup\Filters\.pdf]

"Extension"=".pdf"

"FileTypeBucket"=dword:00000001

"MimeTypes"="application/pdf"

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\14.0\Search\Setup\ContentIndexCommon\Filters\Extension\.pdf]

@="{987f8d1a-26e6-4554-b007-6b20e2680632}"

So run the installer, and then restart SharePoint Server Search 14 service. This service name is subject to change when RTM, but you can easily get the idea.

Recrawl the files.

It worked. Please note the installer will not get you PDF icon file, you need to follow the steps here http://www.foxitsoftware.com/pdf/ifilter/installation.html to download icon file and modify DOCICON.XML.

This also applies to Search Server 2010. FAST Search index PDF files OOTB, so you don’t need to go with these steps.

Jie Li

Technical Product Manager, SharePoint

PDF iFilter Battle, second round

Jie Li — Mon, 09 Mar 2009 22:30:34 GMT

If you still remember the last round of our PDF iFilter battle, FoxIT won it. Now in this round, we bring in another challenger: TET PDF iFIlter. It is also avaliable on x86 and x64, free for non-commercial desktop use, will need a license for Server installation.

So here's the new result for file set II:

	File Number	Total File Size(MB)	Avg File Size(MB)	Crawl Time(m:s)	Crawl Time(s)	File Per Second	Success	Error
FoxIT	2676	2406	0.90	7:46	466	5.74	2759	0
Adobe	2676	2406	0.90	40:58	2458	1.09	2757	2
TET	2676	2406	0.90	13:48	828	3.23	2752	0

I also obtained an archive copy from People's Daily, from 2001 to 2006. ~20,000 PDF files, 13.4GB total. Tested on a 8 cores XEON box.

	File Number	Total File Size(MB)	Avg File Size(MB)	Crawl Time(h:m:s)	Crawl Time(s)	File Per Second	Success	Error
FoxIT	19890	13793	0.69	00:30:53	1853	10.73	19884	7
Adobe	19890	13793	0.69	05:19:04	19144	1.03	19887	4
TET	19890	13793	0.69	01:40:09	6009	3.31	19879	12

And licensing comparsion for production(USD):

	Desktop	Server	1-2 Cores Per Server	4 Cores Per Server	8+ Cores Per Server
Adobe	Free	Free	Free	Free	Free
Foxit	Free	Not Free	329.99	589.97	1109.93
TET	$119 for commercial usage	Not Free	595	595	595

Summary

It is good to see another vendor joined this market. TET showed good performance, although still behind Foxit. But it's licensed based on servers not cores, the cost would be lower than Foxit if you have a typical 2 way quad cores box.

PDF iFilter Battle! FoxIT vs.. Adobe, 64bit version

Jie Li — Wed, 10 Dec 2008 07:46:28 GMT

After so long a time Adobe finally released its 64bit version of PDF iFilter!

http://www.adobe.com/support/downloads/detail.jsp?ftpID=4025

“In response to customer requests, Adobe is releasing Adobe PDF iFilter 9 for 64-bit platforms, which will allow searching PDF files on Microsoft® Windows® 64-bit platforms for applications such as Microsoft Office SharePoint Server 2007, Microsoft Exchange Server 2007, and Microsoft SQL Server 2005.”

But, what about performance? How does it compare with FoxIT 64bit PDF iFilter?

My friend Deb Haldar did a performance test last year for their 32bit iFilters. You can find the result here: FOXIT vs.. Adobe PDF IFilter [ 32-bit only ]. Let’s say, FoxIT 32bit PDF iFilter is more than 4 times faster than the Adobe one.

Will the story change in 64bit age?

I picked about two sets of PDF files. Set I contains ~1000 PDF files, 1.7 GB in total. Set 2 contians ~2600 files, 2.4G in total. Language is mixed by 30% Chinese, 70% US English. The hardware spec is a two-way dual core XEON at 3.4GHz, 4G Ram. SharePoint was patched with October CU. Here’s the result.

	File Set	File Number	Total File Size(MB)	Avg File Size(MB)	Crawl Time(m:s)	Crawl Time(s)	File Per Second	Success	Error
FoxIT	I	1041	1751	1.68	6:02	362	2.88	1064	0
Adobe	I	1041	1751	1.68	30:03	1803	0.58	1063	1
FoxIT	II	2676	2406	0.90	7:46	466	5.74	2759	0
Adobe	II	2676	2406	0.90	40:58	2458	1.09	2757	2

On average, FoxIT x64 PDF ifilter is still ~5 times faster than the Adobe one. But FoxIT charges 330 USD for a 2 core machine, while Adobe PDF iFilter is free. So if PDF indexing is the key to your business, go with FoxIT to get much better performance. If not, you may play with Adobe PDF iFilter to furfill some simple and basic request.