# Apache Lucene Migration Guide

## Lucene 3.x index format no longer supported

Lucene 5 no longer supports the Lucene 3.x index format. Opening
indexes will result in `IndexFormatTooOldException`. It is recommended
to either reindex all your data, or upgrade the old indexes with
the `IndexUpgrader` tool of latest Lucene 4 version (4.10.x).
Those indexes can then be read (see next section) with Lucene 5.

## Support for previous Lucene 4.x index formats moved to new module

Lucene 5 will by default only read indexes created with Lucene 5.
To read and upgrade Lucene 4.x indexes, you must add the
`lucene-backward-codecs.jar` to the classpath. It is recommended
to upgrade the old indexes with the `IndexUpgrader` tool,
so you can remove the backward-codecs module from classpath.
This will also improve performance.

## All file handling APIs changed to Java 7 NIO.2 (LUCENE-5945)

All APIs around Directory and other file-based resources were changed to make
use of the new Java 7 NIO.2 API. It is no longer possible to pass
java.io.File onames to FSDirectory classes. FSDirectory classes now requires
java.nio.file.Path instances. This allows to place index directories also
on "virtual file systems" like ZIP or TAR files. To migrate existing code
use java.io.File#toPath().

In addition, make sure that custom directory implementations throw the new
IOException types, because Lucene cannot understand the old legacy
IOExceptions (like java.io.FileNotFoundException) instead of the new ones
like java.nio.file.NoSuchFileException.

## Directory and LockFactory APIs restructured (LUCENE-5953)

Locking is now under the responsibility of the Directory implementation.
LockFactory is only used by subclasses of BaseDirectory to delegate locking
to an impl class. LockFactories are responsible to create a Lock on behalf
of a BaseDirectory subclass.

The following changes in existing code need to be done:

- LockFactory implementations are singletons now and have no state. They only
  need to implement one method: makeLock(Directory dir, String name).
  The passed directory can be used to determine the corect file system path
  for the lock file or similar, so it knows where to create the lock.
  In addition, the factory may check with instanceof, if the lock factory
  can be used with the type of directory at all.
- It was never really supported to place lock files outside of the index
  directory and this functionality was removed. If you still rely on this,
  you can use the following trick: Use FileSwitchDirectory and delegate
  the file extension ".lock" to another Directory instance pointing to
  another path. FileSwitchDirectory also delegates lock files based on
  the extension.
- If you wrap another directory using FilterDirectory, you cannot make use
  of LockFactories anymore, because only BaseDirectory knows about them.
  To wrap locking, you must hook into FilterDirectory.makeLock(String name)
  and wrap the Lock instance returned, as needed. See MockDirectoryWrapper
  in lucene-test-framework for an example.
- It is no longer allowed to pass "null" as LockFactory to FSDirectory
  implementations. You have to explicitely pass the platform default to the
  directory (currently always NativeFSLockFactory.INSTANCE, but subject to
  change!). To get the platform default, call FSLockFactory.getDefault().

## Removed Reader from Tokenizer constructor (LUCENE-5388)

The constructor of Tokenizer no longer takes Reader, as this was a leftover
from before it was reusable. See the org.apache.lucene.analysis package
documentation for more details.

## Refactored Collector API (LUCENE-5299)

The Collector API has been refactored to use a different Collector instance
per segment. It is possible to migrate existing collectors painlessly by
extending SimpleCollector instead of Collector: SimpleCollector is a
specialization of Collector that returns itself as a per-segment Collector.

## Refactored FieldComparator API (LUCENE-5702)

Like collectors (see above), field comparators have been refactored to
produce a new comparator (called LeafFieldComparator) per segment. It is
possible to migrate existing comparators painlessly by extending
SimpleFieldComparator, which will implements both FieldComparator and
LeafFieldComparator and return itself as a per-segment comparator.

## Removed ChainedFilter (LUCENE-5984)

Users are advised to switch to BooleanFilter instead.

## Removed OpenBitSet (LUCENE-6010)

OpenBitSet only differs from LongBitSet by its ability to grow automatically.
In case growth is required, it would need to be managed externally.

## FunctionValues.exist() Behavior Changes due to ValueSource bug fixes (LUCENE-5961)

Bugs fixed in several ValueSource functions may result in different behavior in 
situations where some documents do not have values for fields wrapped in other 
ValueSources.  Users who want to preserve the previous behavior may need to wrap 
their ValueSources in a "DefFunction" along with a ConstValueSource of "0.0".

## PayloadAttributeImpl.clone() (LUCENE-6055)

PayloadAttributeImpl.clone() did a shallow clone which was incorrect, and was
fixed to do a deep clone. If you require shallow cloning of the underlying bytes,
you should override PayloadAttributeImpl.clone() to do a shallow clone instead.

## Removed out-of-order scoring (LUCENE-6179)

Bulk scorers must now always collect documents in order. If you have custom
collectors, the acceptsDocsOutOfOrder method has been removed and collectors
can safely assume that they will be collected in order.

## Renamed "Atomic" to "Leaf" for segment readers (LUCENE-5569)

AtomicReader and AtomicReaderContext are now called LeafReader and LeafReaderContext, respectively.

## Removed custom Analyzer per-document indexing APIs from IndexWriter (LUCENE-6212)

These methods were removed because they are dangerous since they let
you analyze each document arbitrarily differently, making it difficult
to properly analyze text at query time and easy to accidentally "lose"
search hits.  Instead, you should break out text into separate fields
and use a different analyzer for each field with
PerFieldAnalyzerWrapper.