[ASTERIXDB-3955][COMP] Fix CBO cost of a pushed LIMIT

The CBO costed every operator of a plan as if it stopped at a pushed
LIMIT. A limit is pushed only into a primary-index search, so a
secondary-index search still returns every match, and the primary-key
sort after it is blocking. An index plan therefore looked near-constant
and won even when a scan that stops after a few rows was far cheaper.

A primary-index search returns the records itself, so it is no longer
charged a primary-key sort and fetch.

Measured against an index plan, a scan that a limit stops early still
runs several times slower than the rows it reads suggest, so it is
costed as reading eight times further, though never more than a full
scan. Without that weight it displaced index plans that were faster on
selective predicates with large limits.

Ext-ref: MB-70486
Change-Id: Id437d35c7c3e3f9624076464d196518da823400c
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22002
Reviewed-by: Rithwik Koul <rithwik.koul@couchbase.com>
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
10 files changed
tree: a36833076efd6822d1567781cf592baf5cb7508f
  1. asterixdb/
  2. hyracks-fullstack/
  3. .asf.yaml
  4. .gitattributes
  5. .gitignore
  6. build.xml
  7. CLAUDE.md
  8. pom.xml
  9. README.md
README.md

What is AsterixDB?

AsterixDB is a BDMS (Big Data Management System) with a rich feature set that sets it apart from other platforms. Its feature set makes it well-suited to modern needs such as web data warehousing and social data storage and analysis. AsterixDB has:

  • Data model
    A semistructured NoSQL style data model (ADM) resulting from extending JSON with object database ideas

  • Query languages
    An expressive and declarative query language (SQL++ that supports a broad range of queries and analysis over semistructured data

  • Scalability
    A parallel runtime query execution engine, Apache Hyracks, that has been scale-tested on up to 1000+ cores and 500+ disks

  • Native storage
    Partitioned, flexible LSM-based row and column storage, with indexing to support efficient ingestion and management of semistructured data

  • External storage
    Support for query access to externally stored data (e.g., data in HDFS) as well as to data stored natively by AsterixDB

  • Data types
    A rich set of primitive data types, including spatial and temporal data in addition to integer, floating point, and textual data

  • Indexing
    Secondary indexing options that include B+ trees, R trees, and inverted keyword (exact and fuzzy) index types

  • Transactions
    Basic transactional (concurrency and recovery) capabilities akin to those of a NoSQL store

Learn more about AsterixDB at its website.

Build from source

To build AsterixDB from source, you should have a platform with the following:

  • A Unix-ish environment (Linux, OS X, will all do).
  • git
  • Maven 3.3.9 or newer.
  • JDK 21 or newer.
  • Python 3.11+ with pip and venv

Instructions for building the master:

  • Checkout AsterixDB master:

      $git clone https://github.com/apache/asterixdb.git
    
  • Build AsterixDB master:

      $cd asterixdb
      $mvn clean package -DskipTests
    

Run the build on your machine

Here are steps to get AsterixDB running on your local machine:

  • Start a single-machine AsterixDB instance:

      $cd asterixdb/asterix-server/target/asterix-server-*-binary-assembly/apache-asterixdb-*-SNAPSHOT
      $./opt/local/bin/start-sample-cluster.sh
    
  • Good to go and run queries in your browser at:

      http://localhost:19006
    
  • Read more documentation to learn the data model, query language, and how to create a cluster instance.

Documentation

To generate the documentation, run asterix-doc with the generate.rr profile in maven, e.g mvn -Pgenerate.rr ... Be sure to run mvn package beforehand or run mvn site in asterix-lang-sqlpp to generate some resources that are used in the documentation that are generated directly from the grammar.

Community support