tag	6c0060c251de9c049e2c7119c96ef16d848a6949
tagger	Andy Grove <andygrove73@gmail.com>	Tue Mar 24 20:06:56 2020 -0600
object	ce0a25567b1ad448f96327f10042f2a80c729240

commit	ce0a25567b1ad448f96327f10042f2a80c729240	[log] [tgz]
author	Andy Grove <andygrove@users.noreply.github.com>	Tue Mar 24 20:01:36 2020 -0600
committer	GitHub <noreply@github.com>	Tue Mar 24 20:01:36 2020 -0600
tree	c29352cb1c66eb4456192bb242df997ad28f32e8
parent	30de48c840702c68c82849b4d11c3ad5a24757eb [diff]

tree: c29352cb1c66eb4456192bb242df997ad28f32e8

README.md

Extensible SQL Lexer and Parser for Rust

The goal of this project is to build a SQL lexer and parser capable of parsing SQL that conforms with the ANSI SQL:2011 standard but also making it easy to support custom dialects so that this crate can be used as a foundation for vendor-specific parsers.

This parser is currently being used by the DataFusion query engine and LocustDB.

Example

The current code is capable of parsing some trivial SELECT and CREATE TABLE statements.

let sql = "SELECT a, b, 123, myfunc(b) \
           FROM table_1 \
           WHERE a > b AND b < 100 \
           ORDER BY a DESC, b";

let dialect = GenericSqlDialect{}; // or AnsiSqlDialect, or your own dialect ...

let ast = Parser::parse_sql(&dialect,sql.to_string()).unwrap();

println!("AST: {:?}", ast);

This outputs

AST: SQLSelect { projection: [SQLIdentifier("a"), SQLIdentifier("b"), SQLLiteralLong(123), SQLFunction { id: "myfunc", args: [SQLIdentifier("b")] }], relation: Some(SQLIdentifier("table_1")), selection: Some(SQLBinaryExpr { left: SQLBinaryExpr { left: SQLIdentifier("a"), op: Gt, right: SQLIdentifier("b") }, op: And, right: SQLBinaryExpr { left: SQLIdentifier("b"), op: Lt, right: SQLLiteralLong(100) } }), order_by: Some([SQLOrderBy { expr: SQLIdentifier("a"), asc: false }, SQLOrderBy { expr: SQLIdentifier("b"), asc: true }]), group_by: None, having: None, limit: None }

Design

This parser is implemented using the Pratt Parser design, which is a top-down operator-precedence parser.

I am a fan of this design pattern over parser generators for the following reasons:

Code is simple to write and can be concise and elegant (this is far from true for this current implementation unfortunately, but I hope to fix that using some macros)
Performance is generally better than code generated by parser generators
Debugging is much easier with hand-written code
It is far easier to extend and make dialect-specific extensions compared to using a parser generator

Supporting custom SQL dialects

This is a work in progress but I started some notes on writing a custom SQL parser.

Contributing

Contributors are welcome! Please see the current issues and feel free to file more!