Solr Configuration Options : Slow updates ( mergeFactor )

If you are experiencing slow Import / Updates there is a setting in solrconfig.xml that can dramatically affect the indexing time.

As described in the Solr Documentation mergeFactor is an important consideration. The only issue being the higher you set the mergeFactor the slower the response for a search query is going to be . The question then becomes how much that matters to you. If your index only needs updating every week then you could set the mergeFactor to a low setting and get rapid searches. On the other hand if you need your search results updated more frequently then you will have to set it higher (depending on the amount of content in your domain.) Possibly having a negative impact on the speed of your search responses.

 

mergeFactor Tradeoffs

High value merge factor (e.g., 25):

• Pro: Generally improves indexing speed
• Con: Less frequent merges, resulting in a collection with more index files which may slow searching

Low value merge factor (e.g., 2):

• Pro: Smaller number of index files, which speeds up searching.
• Con: More segment merges slow down indexing. 

Inheriting a bag of solr worms

Solr Installation & Configuration Guide

Having inherited a badly set-up and configured Solr set-up , and tried to tame it - its painfully obvious that what is needed is to bin it and start again.

Before I go any further - If you are new to this area , and looking for a search engine to use for your site do not choose Solr without taking a really good look at Sphinx (http://sphinxsearch.com/) ; In my experience its much more lightweight and easier to get to grips and doesnt have the high overheads of Java. Its a neat, fast 'C' based application with great interfaces for PHP, Ruby and friends. If i had a choice i probably wouldnt have chosen Solr for the project which I have inherited - it adds too many layers of complexity to an already complex web application running on a completely different set of technologies. Sphinx does most of what can be done with Solr so make sure know what features you will actually need and which wont be needed.

If you are experiencing performance issues and Solr is running like a dog maybe this will
be a useful point of reference for you.

Having read the "7 deadly sins of Solr"

Im going to start off by upgrading our aging Solr Search server to a newer version, then take a real, hard look at all those configuration options and see whats going wrong.

Follow these posts they might be of assistance to you.