tag	6004c024a8802e0812410b1f5cb386433919bf56
tagger	Andy Grove <andygrove73@gmail.com>	Sun Oct 14 12:13:34 2018 -0600
object	cc5a6c335d06b70f1be22dd4e3a3e3a3f0a480dc

commit	cc5a6c335d06b70f1be22dd4e3a3e3a3f0a480dc	[log] [tgz]
author	Andy Grove <andygrove73@gmail.com>	Sun Oct 14 12:13:25 2018 -0600
committer	Andy Grove <andygrove73@gmail.com>	Sun Oct 14 12:13:25 2018 -0600
tree	a86bb98c460cef1459f4659b117166dd8456008b
parent	1bb7149670e08cf65d1ee927a77b218cc4ed6bf4 [diff]

tree: a86bb98c460cef1459f4659b117166dd8456008b

README.md

Extensible SQL Lexer and Parser for Rust

The goal of this project is to build a SQL lexer and parser capable of parsing SQL that conforms with the ANSI SQL:2011 standard but also making it easy to support custom dialects so that this crate can be used as a foundation for vendor-specific parsers.

This parser is currently being used by the DataFusion query engine.

Example

The current code is capable of parsing some trivial SELECT and CREATE TABLE statements.

let sql = "SELECT a, b, 123, myfunc(b) \
           FROM table_1 \
           WHERE a > b AND b < 100 \
           ORDER BY a DESC, b";

let dialect = GenericSqlDialect{}; // or AnsiSqlDialect, or your own dialect ...

let ast = Parser::parse_sql(&dialect,sql.to_string()).unwrap();

println!("AST: {:?}", ast);

This outputs

AST: SQLSelect { projection: [SQLIdentifier("a"), SQLIdentifier("b"), SQLLiteralLong(123), SQLFunction { id: "myfunc", args: [SQLIdentifier("b")] }], relation: Some(SQLIdentifier("table_1")), selection: Some(SQLBinaryExpr { left: SQLBinaryExpr { left: SQLIdentifier("a"), op: Gt, right: SQLIdentifier("b") }, op: And, right: SQLBinaryExpr { left: SQLIdentifier("b"), op: Lt, right: SQLLiteralLong(100) } }), order_by: Some([SQLOrderBy { expr: SQLIdentifier("a"), asc: false }, SQLOrderBy { expr: SQLIdentifier("b"), asc: true }]), group_by: None, having: None, limit: None }

Design

This parser is implemented using the Pratt Parser design, which is a top-down operator-precedence parser.

I am a fan of this design pattern over parser generators for the following reasons:

Code is simple to write and can be concise and elegant (this is far from true for this current implementation unfortunately, but I hope to fix that using some macros)
Performance is generally better than code generated by parser generators
Debugging is much easier with hand-written code
It is far easier to extend and make dialect-specific extensions compared to using a parser generator

Supporting custom SQL dialects

This is a work in progress but I started some notes on writing a custom SQL parser.

Contributing

Contributors are welcome! Please see the current issues and feel free to file more!