Title: Apache Lucene™ 9.4.0 available category: core/news URL: save_as:

The Lucene PMC is pleased to announce the release of Apache Lucene 9.4.0.

Apache Lucene is a high-performance, full-featured search engine library written entirely in Java. It is a technology suitable for nearly any application that requires structured search, full-text search, faceting, nearest-neighbor search across high-dimensionality vectors, spell correction or query suggestions.

This release contains numerous bug fixes, optimizations, and improvements, some of which are highlighted below. The release is available for immediate download at:

https://lucene.apache.org/core/downloads.html

Lucene 9.4.0 Release Highlights:

New features

  • Added ShapeDocValues/Field, a unified abstraction to represent existing types: XY and lat/long.
  • FacetSets can now be filtered using a Query via MatchingFacetSetCounts.
  • SortField now allows control over whether to apply index-sort optimizations.
  • Support for Java 19 foreign memory access (“project Panama”) was added. Applications started with command line parameter “java --enable-preview” will automatically use the new foreign memory API of Java 19 to access indexes on disk with MMapDirectory. This is an opt-in feature and requires explicit Java command line flag passed to your application‘s Java process (e.g., modify startup parameters of Solr or Elasticsearch/Opensearch)! When enabled, Lucene logs a notice using java.util.logging. Please test thoroughly and report bugs/slowness to Lucene’s mailing list. When the new API is used, MMapDirectory will mmap Lucene indexes in chunks of 16 GiB (instead of 1 GiB) and indexes closed while queries are running can no longer crash the JVM.

Optimizations

  • Added support for dynamic pruning to queries sorted by a string field that is indexed with both terms and SORTED or SORTED_SET doc values. This can lead to dramatic speedups when applicable.
  • TermInSetQuery is optimized for the case when one of its terms matches all docs in a segment, and it now provides cost estimation, making it usable with IndexOrDocValuesQuery for better query planning.
  • KnnVector fields can now be stored with reduced (8-bit) precision, saving storage and yielding a small query latency improvement.

Other

  • KnnVector fields' HNSW graphs are now created incrementally when new documents are added, rather than all-at-once when flushing. This yields more consistent predictable behavior at the cost of an overall increase in indexing time.
  • randomizedtesting dependency upgraded to 2.8.1
  • addIndexes(CodecReader) now respects MergePolicy and MergeScheduler, enabling it to do its work concurrently.

... plus a number of helpful bug fixes!

Please read CHANGES.txt for a full list of new features and changes:

https://lucene.apache.org/core/9_4_0/changes/Changes.html