docs/src/site/twiki/Search.twiki

---+ Search

Atlas exposes search over the metadata in two ways:
   * Search using DSL
   * Full-text search

---++ Search DSL Grammar
The DSL exposes an SQL like query language for searching the metadata based on the type system.
The grammar for the DSL is below.

<verbatim>
queryWithPath: query ~ opt(WITHPATH)

query: rep1sep(singleQuery, opt(COMMA))

singleQuery: singleQrySrc ~ opt(loopExpression) ~ opt(selectClause) ~ opt(orderby) ~ opt(limitOffset)

singleQrySrc = FROM ~ fromSrc ~ opt(WHERE) ~ opt(expr ^? notIdExpression) |
        WHERE ~ (expr ^? notIdExpression) |
        expr ^? notIdExpression |
        fromSrc ~ opt(WHERE) ~ opt(expr ^? notIdExpression)

fromSrc: identifier ~ AS ~ alias | identifier

orderby: ORDERBY ~ order ~ opt (sortOrder) 
    
limitOffset: LIMIT ~ lmt ~ opt (offset)
    
offset: OFFSET ~ offsetValue 
    
sortOrder = ASC | DESC
    
loopExpression: LOOP ~ (LPAREN ~> query <~ RPAREN) ~ opt(intConstant <~ TIMES) ~ opt(AS ~> alias)

selectClause: SELECT ~ rep1sep(selectExpression, COMMA)

selectExpression:  expr ~ opt(AS ~> alias)

expr:  compE ~ opt(rep(exprRight))

exprRight: (AND | OR) ~ compE

compE:
        arithE ~ (LT | LTE | EQ | NEQ | GT | GTE) ~ arithE |
            arithE ~ (ISA | IS) ~ ident  |
            arithE ~ HAS ~ ident  |
            arithE

arithE: multiE ~ opt(rep(arithERight))

arithERight: (PLUS | MINUS) ~ multiE

multiE: atomE ~ opt(rep(multiERight))

multiERight: (STAR | DIV) ~ atomE

atomE: literal | identifier | LPAREN ~> expr <~ RPAREN

identifier: rep1sep(ident, DOT)

alias: ident | stringLit

literal: booleanConstant |
        intConstant  |
        longConstant  |
        floatConstant |
        doubleConstant  |
        stringLit
</verbatim>

Grammar language:
{noformat}
opt(a)     => a is optional
~            => a combinator. 'a ~ b' means a followed by b
rep         => zero or more
rep1sep => one or more, separated by second arg.
{noformat}

Language Notes:
   * A *!SingleQuery* expression can be used to search for entities of a _Trait_ or _Class_.
 Entities can be filtered based on a 'Where Clause' and Entity Attributes can be retrieved based on a 'Select Clause'.
   * An Entity Graph can be traversed/joined by combining one or more !SingleQueries.
   * An attempt is made to make the expressions look SQL like by accepting keywords "SELECT",
 "FROM", and "WHERE"; but these are optional and users can simply think in terms of Entity Graph Traversals.
   * The transitive closure of an Entity relationship can be expressed via the _Loop_ expression. A
  _Loop_ expression can be any traversal (recursively a query) that represents a _Path_ that ends in an Entity of the same _Type_ as the starting Entity.
   * The _!WithPath_ clause can be used with transitive closure queries to retrieve the Path that
 connects the two related Entities. (We also provide a higher level interface for Closure Queries
  see scaladoc for 'org.apache.atlas.query.ClosureQuery')
   * ORDERBY is optional. Orderby clause should be specified in single quote ('). When order by clause is specified case insensitive sorting is done in ascending order.
    For sorting in descending order specify 'DESC' after order by clause. If no order by is specified then no default sorting is applied.
   * LIMIT is optional. It limits the maximum number of objects to be fetched starting from specified optional offset. If no offset is specified count starts from beginning.
   * There are couple of Predicate functions different from SQL:
      * _is_ or _isa_can be used to filter Entities that have a particular Trait.
      * _has_ can be used to filter Entities that have a value for a particular Attribute.
   * When querying for a space delimited multiple-word identifier, it need to be enclosed within
   backquote (`)

---+++ DSL Examples

   * from DB
   * DB where name="Reporting" select name, owner
   * DB where name="Reporting" select name, owner orderby 'name'
   * DB where name="Reporting" select name limit 10
   * DB where name="Reporting" select name, owner limit 10 offset 0
   * DB where name="Reporting" select name, owner orderby 'name' limit 10 offset 5
   * DB where name="Reporting" select name, owner orderby 'name' desc limit 10 offset 5
   * DB has name
   * DB is !JdbcAccess
   * Column where Column isa PII
   * Table where name="sales_fact", columns
   * Table where name="sales_fact", columns as column select column.name, column.dataType, column.comment
   * `Log Data`


---++ Full-text Search

Atlas also exposes a lucene style full-text search capability.