tree c6443b35422eec55f524ede5e0292071542d8850
parent b800f4a9234f2005e47e056ce3d164720c20bed8
author Tamas Mate <tmate@cloudera.com> 1598957072 +0200
committer Impala Public Jenkins <impala-public-jenkins@cloudera.com> 1600401982 +0000

IMPALA-10051: impala-shell exits with ValueError with WITH clauses

When a query contains WITH clause impala-shell tries to identify whether
it is a DML query or not, so that later it can provide appropriate
result messages. Earlier shlex was used to create tokens and assess the
query type based on that. However shlex can misinterpret some query
strings where whitespace charachters are mixed with quotes, because it
splits the string based on whitespace charachters. In some scenarios
'ValueError: No closing quotation' error can occur.

This change moves the tokenization from shlex to sqlparse.

Testing:
 - Added unit test to cover queries that contain mixed whitespaces
   and strings

Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad
Reviewed-on: http://gerrit.cloudera.org:8080/16389
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>