docs/src/site/twiki/Search.twiki

---+ Search

Atlas exposes search over the metadata in two ways:
   * Search using DSL
   * Full-text search

---++ Search DSL Grammar
The DSL exposes an SQL like query language for searching the metadata based on the type system.
The grammar for the DSL is below.

<verbatim>
queryWithPath: query ~ opt(WITHPATH)

query: rep1sep(singleQuery, opt(COMMA))

singleQuery: singleQrySrc ~ opt(loopExpression) ~ opt(selectClause)

singleQrySrc = FROM ~ fromSrc ~ opt(WHERE) ~ opt(expr ^? notIdExpression) |
        WHERE ~ (expr ^? notIdExpression) |
        expr ^? notIdExpression |
        fromSrc ~ opt(WHERE) ~ opt(expr ^? notIdExpression)

fromSrc: identifier ~ AS ~ alias | identifier

loopExpression: LOOP ~ (LPAREN ~> query <~ RPAREN) ~ opt(intConstant <~ TIMES) ~ opt(AS ~> alias)

selectClause: SELECT ~ rep1sep(selectExpression, COMMA)

selectExpression:  expr ~ opt(AS ~> alias)

expr:  compE ~ opt(rep(exprRight))

exprRight: (AND | OR) ~ compE

compE:
        arithE ~ (LT | LTE | EQ | NEQ | GT | GTE) ~ arithE |
            arithE ~ (ISA | IS) ~ ident  |
            arithE ~ HAS ~ ident  |
            arithE

arithE: multiE ~ opt(rep(arithERight))

arithERight: (PLUS | MINUS) ~ multiE

multiE: atomE ~ opt(rep(multiERight))

multiERight: (STAR | DIV) ~ atomE

atomE: literal | identifier | LPAREN ~> expr <~ RPAREN

identifier: rep1sep(ident, DOT)

alias: ident | stringLit

literal: booleanConstant |
        intConstant  |
        longConstant  |
        floatConstant |
        doubleConstant  |
        stringLit
</verbatim>

Grammar language:
{noformat}
opt(a)     => a is optional
~            => a combinator. 'a ~ b' means a followed by b
rep         => zero or more
rep1sep => one or more, separated by second arg.
{noformat}

Language Notes:
   * A *SingleQuery* expression can be used to search for entities of a _Trait_ or _Class_.
 Entities can be filtered based on a 'Where Clause' and Entity Attributes can be retrieved based on a 'Select Clause'.
   * An Entity Graph can be traversed/joined by combining one or more SingleQueries.
   * An attempt is made to make the expressions look SQL like by accepting keywords "SELECT",
 "FROM", and "WHERE"; but these are optional and users can simply think in terms of Entity Graph Traversals.
   * The transitive closure of an Entity relationship can be expressed via the _Loop_ expression. A
  _Loop_ expression can be any traversal (recursively a query) that represents a _Path_ that ends in an Entity of the same _Type_ as the starting Entity.
   * The _WithPath_ clause can be used with transitive closure queries to retrieve the Path that
 connects the two related Entities. (We also provide a higher level interface for Closure Queries see scaladoc for 'org.apache.metadata.query.ClosureQuery')
   * There are couple of Predicate functions different from SQL:
      * _is_ or _isa_can be used to filter Entities that have a particular Trait.
      * _has_ can be used to filter Entities that have a value for a particular Attribute.

---+++ DSL Examples

   * from DB
   * DB where name="Reporting" select name, owner
   * DB has name
   * DB is JdbcAccess
   * Column where Column isa PII
   * Table where name="sales_fact", columns
   * Table where name="sales_fact", columns as column select column.name, column.dataType, column.comment


---++ Full-text Search

Atlas also exposes a lucene style full-text search capability.