Full Text Search

The search engine capabilities in LightSpeed provide an easy to use free text search implementation that is database engine independent. It provides developers a mechanism to perform Google-style queries and have them match any entities within a defined scope – for example, provide any entity that has the word ‘car’ in it.

To implement full text search in your LightSpeed application you need to use a search provider. Out of the box, LightSpeed ships with a single search provider which is built on top of the open-source Lucene project. Lucene provides a very high performance and scalable search infrastructure. Developers can plug in their own search provider by implementing the ISearchEngine interface.

To use a full text search you first need to instruct LightSpeed to use the search engine and where it can store the search index on disk. This is done via configuration of the LightSpeedContext by either XML configuration file or in code.

    <add name="Test"
         connectionStringName="Test"
         dataProvider="Sqlite3"
         searchEngineClass="Mindscape.LightSpeed.Search.LuceneSearchEngine"
           searchEngineFileLocation="c:\foo\bar\baz"/>

Make sure that the search engine file location directory is writable by your application process as it will be written to as new entities are saved.

Indexing Data

LightSpeed does not index the entire database – it will only index data on entities that you care about. To enable indexing, select each of the properties that you want to include in the search index and set their Include in Full Text Index option to true. To do this in hand coded entities, apply IndexedAttribute to the fields:

    [Indexed]
    private string _registration;

In advanced scenarios you can override the Entity.SearchData method to provide custom indexing data to be indexed. Note that you will still need to set Include in Full Text Index on one of the entity’s properties (or apply IndexedAttribute to one of its fields) to notify LightSpeed that you wish the entity to be indexed; however, when you override Entity.GetSearchData LightSpeed will use your custom implementation instead of getting indexing data from the attributed field.

protected override string GetSearchData()
{
  if (Warrant != null)
  {
    return Warrant.SerialNumber + " " + Registration;
  }
  return Registration;
}

Overriding the GetSearchData method provides a powerful mechanism for collapsing indexed data into one entity. For example, when searching for a Car you may want to include engine details. These are stored in an Engine entity, but you can add them to the Car index data by returning them from Car’s GetSearchData method.

If updating the fields that are being indexed or overriding GetSearchData, be sure to call a Rebuild of your search index to ensure that the search engine index is up to date with your changes.

Building the Search Index

By default LightSpeed will update the search index when you call UnitOfWork.SaveChanges. The changes that have occurred as part of the save will be updated in the search index.  However, if you are adding a search index to an existing database it’s important to rebuild the search index from the entire database, not just a single change set. LightSpeed provides the Rebuild method for rebuilding the entire search index.  Call Rebuild with the types of entities that you want to have indexed.

_context.SearchEngine.Rebuild(IsolationLevel.ReadCommitted,
  typeof(Comment), typeof(Tag), typeof(Car), typeof(Contribution));

Rebuilding the search index should not be needed often. Calling Rebuild() can be an expensive operation, and for larger database solutions you may wish to use Lucene outside of the LightSpeed framework to achieve the best performance.

Performing Searches

Performing searches using LightSpeed is done with a LightSpeed query object. This first example demonstrates searching for a comment with the word “video” in it.

Query query = new Query();
query.SearchQuery = "video";
using (var unitOfWork = _context.CreateUnitOfWork())
{
  unitOfWork.Find<Comment>(query);
}

Searching with LightSpeed is not limited to only returning one type of entity. The next example demonstrates searching for comments and tags at the same time.

Query query = new Query();
query.SearchQuery = "video OR Mindscape";
using (var unitOfWork = _context.CreateUnitOfWork())
{
  var results = unitOfWork.Search(query, typeof(Comment), typeof(Tag));
  Assert.AreEqual(3, results.Count);
}

You can combine a full text search and a normal SQL query to limit the text search.

using (var unitOfWork = _context.CreateUnitOfWork())
{
  Query query = new Query();
  query.SearchQuery = "video";
  query.QueryExpression = Entity.Attribute("Member") == unitOfWork.FindById<Member>(41);
  var count = unitOfWork.Count<Comment>(query);
  Assert.AreEqual(1, count);
}

Advanced Operator Support

As LightSpeed is using Lucene by default as a search engine developers can use any of the advanced operators that Lucene supports by specifying them in the SearchQuery string.

Further Search Reading

If you are developing a significant solution with LightSpeed and working with Lucene for search, we recommend that you read about Lucene and understand how to configure and operate it for the best performance.  See the Lucene Web site for more information.