Symptoms
Search Index gets corrupted when broken pdf file is uploaded to dms and indexed.
Solution
Remove org.apache.jackrabbit.extractor.PdfTextExtractor from the textFilterClasses list in your workspace.xml as well as from your jackrabbit configuration file.
Side effect of this fix is that PDF files will not be indexed anymore.
Another option is to wrap this indexer in your own which will perform check on every pdf file before passing it through for indexing.