Crawling Large Library`s in SharePoint 2007

Jan 8, 2011   //   by Ronald@iDocs   //   Blog, Configuration, MOSS 2007 posts  //  1 Comment

I had been experiencing issues crawling a large document library of over 60,000 items in a SharePoint 2007 farm 64x after the index was corrupt and I had to reset the crawled content. The only error I could find in the crawl log was the error “The item may be too large or corrupt.” The crawler stopted around the 33,000 items from this document library. I have searched a lot on the internet for this problem and found a few Blogs describing this problem with different solutions. The solution for my issue was a mix of what I found on the internet. After these changes the crawler was able to index all 60,000 items from one Library.

Register changes:

  • HKLM/SOFTWARE/Microsoft/Office Server/12/Search/Global/GatheringManager/DedicatedFilterProcessMemoryQuota” –> Change the value to: 256000000 Hex
  • HKLM/SOFTWARE/Microsoft/Office Server/12/Search/Global/GatheringManager/FilterProcessMemoryQuota –> Change the value to: 256000000 Hex
  • HKLM/SOFTWARE/Microsoft/Office Server/12/Search/Global/GatheringManager/FolderHighPriority –> Change the value to: 500 Hex
  • HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Office Server/12.0/Search/Global/Gathering Manager: set DeleteOnErrorInterval –> Change the value to: 4 Decimal

Search Time Out settings:

1. Central Administration -> Application Management -> Search section -> Manage search service

2. Manage Search Service page –> “Farm-level search settings

3. Change the “Timeout Settings” both from 60 –> 500

Tags: ,

1 Comment

Leave a comment

Categories

Tags

Access Denied Beenmodeonline cache Cluster Database DisableLoopbackCheck DPM Edit in Datasheet Educations error FrieslandCampina iis Issue K2 BlackPoint manager 2007 MOSS 2007 MOSS 2010 PDF peoplepicker Permission Level PowerShell profile Reportviewer Search Search Service SharePoint Sharepoint 2010 SharePoint Designer Shred Service Provider Site Collections site theme Skills & Strenghts SQL 2008 SSP stsadm Sync Usage Statistics User alert error viadata Vs2008 WebApplications webparts WFE Windows 2008 R2 Workflow

Recent Posts

Follow me on Twitter

What I'm Doing...

Posting tweet...

Blogroll