For a full list of releases, see github. Downloads are available on the [downloads page]({{ site.baseurl }}/downloads/).
{: #v1-29-0}
This release comes two months after 1.28.0, contains contributions from 23 authors, and resolves 47 issues.
This release upgrades log4j2 to 2.17.0 to fix security vulnerabiities such as CVE-2021-44228 and CVE-2021-45105.
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using JDK/OpenJDK versions 8 to 17; Guava versions 19.0 to 31.0.1-jre; other software versions as specified in gradle.properties.
Contributors to this release: Ada Wong, Aleksey Plekhanov, Alessandro Solimando, Chunwei Lei, Francesco Gini, Jacques Nadeau, Jay Narale, Julian Hyde, liuyanze, Louis Kuang, NobiGo, Ruben Quesada Lopez, Rui Wang (release manager), Sergey Nuyanzin, Stamatis Zampetakis, Thomas Rebele, Vladimir Sitnikov, Will Noble, Zhe Hu.
{: #new-features-1-29-0}
ARRAY_CONCAT
, ARRAY_REVERSE
, ARRAY_LENGTH
functions for BigQuery dialectdecimal
column type in CSV and File adaptersAggregateReduceFunctionsRule
should accept arbitrary predicates{: #fixes-1-29-0}
ImmutableBeans
post 1.28 releaseSqlBasicCall
, make the operands
field privateAggregateExpandDistinctAggregatesRule
must infer correct data type for top aggregate callsXmlFunctions
exception handled by System.out
NVL
function incorrectly changes nullability of its operandsIN
-list that references columns is wrongly converted to Values
, and gives incorrect resultsIN
-list that includes NULL
converted to Values
throws exceptionRelJsonWriter
to allow customized JsonBuilder
EnumerableIntersect
is followed by EnumerableLimit
Exchange
is created from externalized JSON, RelDistribution
is not correctly set in its traitSet
RelFieldTrimmer
incorrectly drops filter conditionRelBuilder.groupKey(ImmutableBitSet, ImmutableList)
that clashes with newer API methodRelMetadataQuery
JdbcCalc
and remove JdbcCalcRule
SELECT
clausegetDef
on metadata handlersIterativeRuleDriver
RelOptRules.CALC_RULES
SemiJoinRule
loses hintsMAP
type in user-defined function (UDF) cannot be created from externalized JSONRelBuilder.size()
AggregateUnionTransposeRule
produces wrong groupingSets
for the top Aggregate
{: #build-1-29-0}
:core
runtime dependencies...\n" +
a IS NOT NULL AND a = b
simplificationExecution optimizations have been disabled
’ warnings{: #dependency-1-29-0}
com.github.vlsi.vlsi-release-plugins
to 1.76{: #site-1-29-0}
{: #v1-28-0}
This release comes four months after 1.27.0, contains contributions from 38 authors, and resolves 76 issues. New features include the UNIQUE sub-query predicate, the MODE aggregate function, PERCENTILE_CONT and PERCENTILE_DISC inverse distribution functions, an Exasol dialect for the JDBC adapter, and improvements to materialized view recognition.
This release contains some breaking changes due to the replacement of ImmutableBeans with Immutables; the changes concern custom planner rule configurations, in particular interface RelRule.Config
, and are fully described in the [news item]({{ site.baseurl }}/news/2021/10/19/release-1.28.0). Two APIs are deprecated and will be removed in release 1.29.
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using JDK/OpenJDK versions 8 to 15; Guava versions 19.0 to 31.0.1-jre; other software versions as specified in gradle.properties.
Contributors to this release: Alessandro Solimando, Alon Eldar, Amir Gajst, Bruce Irschick, dz, Evgeniy Stanilovskiy, Feng Zhu, Grzegorz Gierlach, Haisheng Yuan, Jack Scott, Jacky Yin, Jacques Nadeau, James Starr, Jesus Camacho Rodriguez, Jianhui Dong, Jiasen Sheng, Julian Hyde (release manager), Liu Enze, Michael Mior, Narayanan Venkateswaran, Nick Riasanovsky, NobiGo, Rafay Qureshi, Ruben Quesada Lopez, Sergey Nuyanzin, Stamatis Zampetakis, Taras Ledkov, Thomas Rebele, TJ Banghart, Ulrich Kramer, Vladimir Ozerov, Vladimir Sitnikov, Will Noble, Xurenhe, Yanjing Wang, Yingyu Wang, YuKong.
{: #to-be-removed-in-1-29-0}
class Immutables
and deprecated ImmutableBeans
; in 1.29, [CALCITE-4839] will remove ImmutableBeans
operands
field of SqlBasicCall
. Before 1.29, we will make that field private.{: #new-features-1-28-0}
RexSubQuery
that collect sub-queries into MULTISET
, ARRAY
and MAP
collectionsRelBuilder
, add methods for creating various kinds of sub-queryReduceExpressionsRule
never reduces dynamic expressions but this should be configurableIN
-list as if the user had written OR
, even if the IN
-list contains NULL
GROUP BY
clause contains literal, materialized view recognition failsUNIQUE
sub-querySOME <>
sub-queryFILTER
clause in AggregateExpandWithinDistinctRule
GROUPING SETS
, Calcite should return duplicate rowsAggregate.groupKey
to be a strict superset of Aggregate.groupKeys
Values
by generating SELECT
without FROM
SqlToRelConverter
should eliminate duplicate variablesRelDataTypeSystem
from Config
in Planner
MODE
aggregate functionRelBuilder.join
should convert Correlate
to Join
if correlation variable is unusedPERCENTILE_CONT
and PERCENTILE_DISC
functions*
” should be followed by spaceSEARCH
call to termsQuery
FilterJoinRule
misses opportunity to push Filter
to SemiJoin
input{: #fixes-1-28-0}
HAVING
condition to a query with a dynamic parameter makes the result always emptyJaninoRelMetadataProvider
API for binding methodsHAVING
clause in BigQuery dialectSqlUtil.GENERATED_EXPR_ALIAS_PREFIX
AggregateUnionTransposeRule
causes row type mismatch when some inputs have unique grouping keySqlBasicCall
, deprecated the operands
fieldSqlImplementor
fails, include the RelNode
in the exceptionNULL
in query resultsClassCastException
retrieving from ARRAY
that has mixed INTEGER
and DECIMAL
elementsClassCastException
retrieving from an ARRAY
that has DATE
, TIME
or TIMESTAMP
elementsexecuteBatch
and preparedStatement
when using RemoteMeta
Coalesce(null, row)
fails with NullPointerException
Exchange
operator deserialize fails when the RexInput
has no RelCollation
CompileException
in UnitCompiler
when using multiple class loadersJaninoRelMetadataProvider
uses hardcoded class nameSemiJoin
operator is not skipped in materialized view-based rewriting algorithmRelNode
subtypesImmutableBeans
with Immutables
in core
moduleImmutableBeans
and deprecateImmutableBeans
ConstantExpression
RelOptMaterializations
to allow registering UnifyRule
RelDecorrelator
's RemoveSingleAggregateRule
can produce result with wrong row typeMetadata
API backed by Java ReflectionPushProjector
should retain alias when handling RexCall
RelOptUtil#classifyFilters
Correlate#requiredColumns
is subset of columns in left relationRelJson
should throw if asked to deserialize a call to an unknown operatorRelBuilder.lessThan
, and use RelBuilder
shorthandsSqlValidatorImpl#performUnconditionalRewrites
for Values
nodeHepPlanner
, remove outdated graph edgesRelBuilder
creation fails with error ‘No suitable driver found for jdbc:calcite:
’ in shaded CalcitePARTITION BY
list of OVER
window causes conversion exceptionRexNode
in MutableCalc
, SubstitutionVisitor
should return right rebuild RexNode
AbstractRelNode#getId
can overflow into a negative value, causing CompileException
in the implement
methods of certain Enumerable
sub-classesAggregateExpandDistinctAggregatesRule
must cast top aggregates to original typeClassCastException
converting Sarg in RelNode
to SQLINNER JOIN ... ON TRUE
SemiJoinRule
should not match semi-joinDOUBLE
or TINYINT
datatypesCHARACTER SET
in Redshift*
” is a column aliasSimpleCalciteSchema
ignore case sensitivity parameterVolcanoPlanner
fails to recognize transformation rule correctly in the top-down modeJdbcTable.scan
throws NullPointerException
RelCollation
for aggregate functionsFULL JOIN
plan cannot be executed in MySQL{: #build-1-28-0}
HrSchema
in MaterializationTest
with HrSchema.class.getName()
ErrorProne
violations in testkit
@Override
annotationscalcite-testkit
and unpublish -test.jar
java.security.AccessController
deprecationEqualsHashCode
verification in ErrorProne
: it is already verified with Checkstyle
user.timezone
property to the test JVMCassandraAdapterDataTypesTest.testCollectionsInnerValues
fails depending on the user timezonecom.google.common.io.Files.createTempDir()
with java.nio.file.Files.createTempDirectory()
in ElasticSearch testsJdbcTest#testBushy
and update expected planRelOptRulesTest
improvementsprepareVote
draft email{: #dependency-1-28-0}
{: #site-1-28-0}
README.md
should point to app.travis-ci.com
instead of travis-ci.org
.asf.yml
{: #v1-27-0}
This release comes eight months after 1.26.0. It includes more than 150 resolved issues, comprising a few new features, three minor breaking changes, many bug-fixes and small improvements, as well as code quality enhancements and better test coverage.
Among others, it is worth highlighting the following:
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using JDK/OpenJDK versions 8 to 15; Guava versions 19.0 to 29.0-jre; other software versions as specified in gradle.properties.
{: #breaking-1-27-0}
SUBSTRING
operator comply with ISO standard SQL{: #new-features-1-27-0}
Interpreter
, support table-valued functionsRLIKE
operator (similar to LIKE
, but Hive- and Spark-specific) (Shradha Ambekar)WITHIN DISTINCT
clause for aggregate functions (experimental)MergeUnion
operator in Enumerable
conventionGROUP_CONCAT
aggregate function (MySQL's equivalent of LISTAGG
)ILIKE
operator (as LIKE
, but case-insensitive and PostgreSQL-specific) (Ondřej Štumpf)ROW
UNPIVOT
operator to SQLSUBSTR
function (James Starr)GROUP BY
columns (xzh)COUNTIF
aggregate function for BigQuery (Aryeh Hillman)ITEM
operator on ROW/STRUCT
data types (Alessandro Solimando)ARRAY_AGG
, ARRAY_CONCAT_AGG
, STRING_AGG
aggregate functions for BigQueryBOOL_AND
, BOOL_OR
, LOGICAL_AND
, LOGICAL_OR
aggregate functions (ShuMingLi)IF
function for BigQuery, Hive and Spark dialects (Vaibhav Jain)DOT
format (Liya Fan)InnoDB
adapter (neoremind){: #fixes-1-27-0}
RelBuilder
, support windowed aggregate functions (OVER)CASE
causes AssertionError
in RelToSqlConverter
SemiJoinRule
throws AssertionError
on ANTI
joinAssertionError
in RelToSqlConverter
AggregateRemoveRule
throws while handling AVG
SqlToRelConverter#convertExpression
cannot convert SqlNode
with sub query (jibiyr)DatabaseMetaData#getFunctions
(Malte Bellmann)VALUES
has zero fieldsRexLiteral
can produce wrong digest for some user defined typesEXISTS
subquery with COALESCE
in the predicateRelBuilder#join
with correlationsRelRunner
should throw SQLException if prepare failsRelRunner
(NobiGo)PigRelBuilder
with custom config.simplify()
(Jiatao Tao)RelBuilder#filter
with config.simplify()
(Jiatao Tao)STORE
commands causes the merging of multiple SQL statements (Mahesh Kumar Behera)PigConverter
with custom properties (Mahesh Kumar Behera)RANK
or FILTER
operators (Mahesh Kumar Behera)ClassCastException
if Pig Latin script contains FLATTEN
or STRSPLIT
operators (Mahesh Kumar Behera)ServerDdlExecutor
cannot execute DROP
commands with qualified object names (Vladimir Ozerov)CalciteMetaImpl#createEmptyResultSet
should not pass class to CursorFactory#deduce
(Alessandro Solimando)Sort
should be lower if sort keys are empty (huangqixiang)Interpreter
does not close resources held by its nodes on closeSqlSelect.selectList
, DataContext.getTypeFactory
)REPLACE
and IF NOT EXISTS
keywords in CREATE TABLE/SCHEMA
commands (Vladimir Ozerov)SqlSnapshot#unparse
loses the AS
keyword when the table has alias (jibiyr)MaterializedViewOnlyAggregateRule
performs invalid rewrite on query that contains join and time-rollup function (FLOOR
) (Justin Swett)UNNEST
a collection that has a field with nested data generates an Exception
RelSets
, fine-tune which set is merged into which, for efficiency (Botong Huang)Sort
should be decorrelated even though it has fetch or limit when it is not inside a Correlate
(Thomas Rebele)CAST
to unknown type (Louis Kuang)RelNode
presence in RelSubset
(Igor Lozynskyi)RelBuilder
, remove not-null arguments to COUNT
RelBuilder
throws NullPointerException
while implementing GROUP_ID()
RelTraitSets
(Jiatao Tao)SqlWriterConfig
in SqlPrettyWriter
, reduce the overhead of ImmutableBeans#create
(Jiatao Tao)Logical
nodes inside Enumerable
rules (Vladimir Ozerov)vFloat in (1.0, 2.0)
throws UnsupportedOperationException
SqlSimpleParser
inner Tokenizer should not recognize the sql of TokenType.ID or some keywords in some case (wangjie)requireNonNull(var, "var")
instead of requireNonNull(var)
for better error messagesRelTraitDef#convert
when the source trait satisfies the target trait (Vladimir Ozerov)ORDER BY
syntax (Yanjing Wang)RexExecutorImpl#compile
should use RexBuilder
's type factory if possibleVARCHAR
literal projection fails with JsonParseException
x IS NULL OR x NOT IN (1, 2)
CASE row WHEN row
DefaultEdge
lacks a proper toString
implementation (Liya Fan)RelBuilder#literal
from RexNode
to RexLiteral
RexFieldAccess
ITEM(STRUCT, INDEX)
access (Alessandro Solimando)RelOptUtil#createCastRel
should throw if source and target row types have different number of fieldsRelBuilder
RelNode
to SQL loses DISTINCT
on window aggregation (Jiatao Tao)ImmutableBeans
: make reference properties non-nullable by defaultRelDataType#getSqlTypeName
non-nullableNPE
in LoptMultiJoin
when mq#getColumnOrigin(left, i)
returns null
SqlStdOperatorTable.NOT_LIKE
has a wrong implementorRelFieldTrimmer
after trimming all the fields in an aggregate should not return a zero field Aggregate (Rafay)RelMdSelectivity#getSelectivity
for Calc
propagates predicate with wrong referencesRelBuilder
tries to create a field on a non-struct expressionExceptionInInitializerError
due to NPE
in SqlCallBinding
caused by circular dependencyCalc
and SetOp
operator in RelMdAllPredicates
(Xu Zhaohui)SqlCall#equalsDeep
does not take into account the function quantifier (Huang Qixiang)SqlTableRef
operator should create a SqlTableRef
as the callRelNode
has been removed from its subset, skip the origin rule match (Jiatao Tao)RelBuilder
, optimize VALUES ... UNION ALL ... VALUES
to a single VALUES
with multiple rowsROW
and implicit row constructor sometimes print too many spacesSqlNodeList
implement List<SqlNode>
SqlMatchRecognize
returns wrong operand list (Dawid Wysakowicz)a IN (1, 2) AND a = 1
should be simplified to a = 1
AggregateCaseToFilterRule
throws NullPointerException
when converting CASE
without ELSE
(Jiatao Tao)listCoerced
in TypeCoercionImpl#inOperationCoercion
(Jiatao Tao)RexSimplify
incorrectly drops IS NULL
and IS NOT NULL
from SEARCH
expressionsGROUP BY
ordinalRelDecorrelator
pluggableVALUES
, and UNNEST
of ARRAY
and MULTISET
constructorsSqlOperator#createCall
FILTER (WHERE ...)
expressions when generating SQL (Jeremiah Rhoads Hall)RexSimplify
incorrectly simplifies complex expressions that contain Sarg and IS NULL
SqlTypeUtil#getMaxPrecisionScaleDecimal
returns a decimal with same precision and scale (Jiatao Tao)Sort
rel should be decorrelated even though it has fetch or limit when its parent is not a Correlate
VolcanoPlanner
(Botong Huang)NPE
in RelMdUtil#checkInputForCollationAndLimit
NPE
when division includes nullsMap
& Array
annotationTrustAllSslSocketFactory
class (intrigus-lgtm)IdentityHashMapUsage
, JdkObsolete
-> JavaUtilDate
{: #build-1-27-0}
System.out.println
from DiffRepository
DiffRepository
should write a test's resource file only when it is modifiedDiffRepository
tests should fail if new XML resources are not in alphabetical orderspark.driver.bindAddress
explicitly to avoid BindException
thrown by Spark (Jiatao Tao)Redis
tests using Docker containersClassCastException
in JDBC AdapterMethodCanBeStatic
check via ErrorProneSqlToRelTestBase
to allow custom Context
in teststestCollectionsInnerValues()
for Cassandra adapter is wrong (Alessandro Solimando)ResultSetEnumerable
to avoid nested lambdas{: #dependency-1-27-0}
{: #site-1-27-0}
SchemaFactory
of CSV adapter{: #v1-26-0}
This release comes about two months after 1.25.0. It includes more than 70 resolved issues, comprising a lot of new features and bug-fixes. Among others, it is worth highlighting the following.
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using JDK/OpenJDK versions 8 to 15; Guava versions 19.0 to 29.0-jre; other software versions as specified in gradle.properties.
{: #breaking-1-26-0}
{: #new-features-1-26-0}
SEARCH
operator and Sarg
literal that represents a set of values or rangesPIVOT
operator to SQLORDER BY
computation in Enumerable convention by exploiting LIMIT
(Thomas Rebele)EnumerableMergeJoin
BIT_AND
, BIT_OR
and BIT_XOR
support binary and varbinary type (Hailong Wang)RelNode
plans (Liya Fan)LEFT JOIN
in EnumerableMergeJoin
{: #fixes-1-26-0}
Values
by generating SELECT
without FROM
(Stuti Gupta)SqlToRelConverter.Config
) to retain ORDER BY
in sub-query (Jiatao Tao)UNION ALL
) changes query semantics (Jin Xing)GROUP BY
list of materialized view (Wang Yanlin)COALESCE IN
throws NullPointerException
<>
predicatesLIKE '%'
) to TRUE
RelMdColumnOrigins
gets the wrong index of group by columns after RelNode
was optimized by AggregateProjectMergeRule
rule (FangZheng Li)RelMdUniqueKeys
should not return empty when meeting Intersect
and Minus
if its input has unique keysCURRENT_DATE
in CASE
expressionRelDecorrelator
outputs wrong plan for correlate sort with fetch limitRelBuilder
, add an option to not simplify LIMIT 0
to an empty relationJoin
ON
clauses (James Starr)RelDataType.getSqlTypeName
non-nullableRelCrossType#getFieldCount()
SqlToRelConverter
, use RelBuilder
for creating Aggregate
Mappings#asListNonNull
as a null-safe alternative for Mappings#asList
AssertionError
in SqlTypeFactoryImpl.leastRestrictive
when running slow testsImmutableBeans
should make an immutable copy of property values of type List
, Set
, or Map
NOT LIKE
in join conditionUnsupportedOperationException
if query contains range predicate on columns from sub-querySESSION
table functionSEARCH
operator cannot be pushed into DruidLists.transform
and Iterables.transform
with Util.transform
AggregateJoinRemoveRule
and ProjectJoinRemoveRule
are not fired if the last column of the join's left input is referenced (Liya Fan)SqlTumbleTableFunction
SqlOperandCountRange
throws IllegalArgumentException
(Zhenghua Gao)OR
simplification incorrectly loses termExceptionInInitializerError
when initializing DruidRulesAssertionError
when registering Druid rules due to conflict in descriptionRelMdUniqueKeys
returns wrong unique keys for Aggregate
with grouping setsRelBuilder.in
should allow duplicate valuesSqlToRelConverter
: SELECT 1 IS DISTINCT FROM NULL
fails with AssertionError
org.apache.calcite.schema.Statistic
uses null
vs emptyList
appropriatelyImmutableIntList#toArray(Integer[])
should support arguments larger than the collection itselfFlatLists.Flat6List#append
should not throw NPE if there are null elements in the listAdd Util.throwAsRuntime
and Util.causeOrSelf
to simplify exception re-throwingHOP
and SESSION
EnumerableMergeJoin#create
does not set EnumerableConvention
in the trait setSqlTableFunction
when creating table function scanIllegalArgumentException
in EnumerableBatchNestedLoopJoinRule
SqlTypeUtil#deriveType(SqlCallBinding)
to make type computation simplerorg.apache.calcite.rel.type.RelDataType#getFamily
non-nullableEnumerableBatchNestedLoopJoin
dynamic code generation can lead to variable name issues if two EBNLJ are nestedRelNode
to output its relational expression string (Jiatao Tao)SqlParser.ConfigBuilder
ArrayList
allocation from Mappings#bijection
, and add helpful message in case NPE is thrownPair.left(Iterable)
and Pair.right(Iterable)
implementationPair
comparison to use Comparator.nullsFirst
and .naturalOrder
SqlToRelConverter.ConfigBuilder
, and refactor SqlToRelConverterTest
SqlParserTest
DruidAdapterIT
and DruidAdapter2IT
{: #build-1-26-0}
MaxMetaspaceSize=512m
to avoid metaspace issues when building Calcitemerge=union
strategy to avoid false merge conflicts on CalciteResource.properties
package-info
files with importsorg.nosphere.apache.rat
plugin from 0.5.2 to 0.7.0, and print files with unapproved licenses to console{: #site-1-26-0}
{: #v1-25-0}
This release comes shortly after 1.24.0 and removes methods which were deprecated in the previous version. It also introduces other breaking changes so make sure to consult corresponding section. Notable improvements in this release are:
INTERVAL '1' HOUR
, INTERVAL -'1:2' HOUR TO MINUTE
)Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 8, 9, 10, 11, 12, 13, 14 and OpenJDK 8, 9, 10, 11, 12, 13, 14; Guava versions 19.0 to 28.2-jre; other software versions as specified in gradle.properties.
{: #breaking-1-25-0}
SqlTableFunction
and have CURSOR
as their return typeSqlDialect
can cause class initialization deadlock{: #new-features-1-25-0}
ST_MakeGrid
and ST_MakeGridPoints
ProjectAggregateMergeRule
, to merge a Project
onto an Aggregate
SqlConformance.allowCharLiteralAlias()
{: #fixes-1-25-0}
ListTransientTable
BigDecimal
(Qi Yu)AggregateMergeRule
should merge SUM0
into COUNT
even if GROUP BY
is emptyUnsupportedOperationException
when generating SQL for untyped NULL
literal (Anton Haidai)CAST
from RexNode incorrectly{: #build-1-25-0}
ReflectiveRelMetadataProvider
constructor should throw an exception (instead of assertion) when called with an empty mapINSERT
(Xu Zhaohui)RelNode
VolcanoPlannerPhase
in Planner (Jiatao Tao)CalciteAssert.forceDecorrelate
(Jiatao Tao){: #v1-24-0}
This release comes about two months after 1.23.0. It includes more than 80 resolved issues, comprising a lot of new features as well as performance improvements and bug-fixes. Among others, it is worth highlighting the following.
TUMBLE/HOP
table functionsIn this release, quite a few instance variables are deprecated and will be removed before 1.25, such as EnumerableToBindableConverterRule.INSTANCE
, CassandraToEnumerableConverterRule.INSTANCE
and so on. Besides, some methods in RelNode
are changed from ‘to removed before 2.0’ to ‘to be removed before 1.25’, including isDistinct()
, isKey(ImmutableBitSet)
, getQuery()
, getRows()
, getVariablesStopped()
, computeSelfCost()
, isValid(boolean)
, getCollationList()
, getChildExps()
. All deprecated APIs are strongly recommended to be replaced by their replacements as soon as possible(CALCITE-3923, CALCITE-4079).
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 8, 9, 10, 11, 12, 13, 14 and OpenJDK 8, 9, 10, 11, 12, 13, 14; Guava versions 19.0 to 28.2-jre; other software versions as specified in gradle.properties.
{: #breaking-1-24-0}
CalcMergeRule
as TransformationRule
. With this change, the CalcMergeRule
won't match PhysicalNode
(including EnumerableCalc
) in VolcanoPlanner
PhysicalNode
generation in TransformationRule
RelNode#recomputeDigest()
return type from String
to void
{: #new-features-1-24-0}
OFFSET
parameter in TUMBLE/HOP
table functions (Rui Wang)MULTISET/SET
and VOLATILE
modifiers in CREATE TABLE
statements (Drew Schmitt)CAST(integer AS DATE)
even though it is illegal in Calcite SQLHoist
, a utility to replace literals in a SQL string with placeholders{: #fixes-1-24-0}
RexNormalize
for more effect rex nodes normalizationRexNode-to-Expression
code generationDigest
from RelNode
and RexCall
EnumerableSortedAggregate
(Rui Wang)RelBuilder
to create RelNode
with convention (Xiening Dai)NOT IN
AbstractRelNode#getRelTypeName
VolcanoRuleCall
EnumerableMergeJoin
constructor protectedSqlDotOperator
& SqlItemOperator
(Dawid Wysakowicz)HAVING
clause with a Project
on Filter
on Aggregate
CalciteConnectionConfig
in DecorrelateProgram
& some minor code refactoring (Jiatao Tao)RexNode
's nullable state in codegenPair
with Flat2List
in RelDigestWriter
NPE
using WITH
clause without a corresponding SELECT FROM
(James Kim)SqlOperator
of SqlKind#OTHER_FUNCTION
to define a Strong.Policy
AntiJoin
in EnumerableMergeJoin
RelJson.toJson(RelDistribution)
(Jiatao Tao)SqlTypeUtil#convertTypeToSpec
cover Array/Multiset/Row
types (Jiatao Tao)SqlTypeUtil#equalSansNullability
consider Array/Map
type (Jiatao Tao)CassandraFilter
has generated wrong condition expression for filter with non string literal (Wenhui Tang)UNNEST
(Rui Wang)NPE
(Xu Zhaohui)SELECT
above a sub-query generates a bad table alias (Steven Talbot)RelTraitSet
failed to canonize traitsSqlInsert
with SqlShuttle
cause NullPointerException
(Xu ZhaoHui)ClassCastException
SqlCollation
Calc
operator in RelFieldTrimmer
(Xu Zhaohui)EnumerableBatchNestedLoopJoin
(Rui Wang)EnumerableCalc
RelFieldTrimmer
loses hintsProjectFilterTransposeRule
to push down project and filter expressions whole, not just field referencesRexVisitor
, RexBiVisitor
, RelOptUtil.InputFinder
RexSimplify
should not pass exprs containing non-const subExprs to RexExecutor
(Shuo Cheng)EnumerableValues
EnumerableCorrelate
MergeJoin
collation check should not be limited to join key's orderEnumerableHashJoin
and EnumerableNestedLoopJoin
(Rui Wang)OptimizeTask
JoinCommuteRule
must not match SEMI
/ ANTI
joinIllegalArgumentException
message in RelBuilder#field
RelSet.getOrCreateSubset()
(Botong Huang)Volcano.register
should not return stale subset (Botong Huang)SqlToRelConverter
and RelBuilder
, add option to avoid pushing down join conditionProjectSortTransposeRule
RelTrait
, RelTraitSet
and RelCollation
EnumerableProject
and EnumerableFilter
(Rui Wang)SqlInsert
with SqlShuttle
cause NullPointerException
(Xu ZhaoHui)RelOptRuleOperand
description in debugger to facilitate debuggingProjectJoinTransposeRule
DialectPool
implementation using Guava cacheProjectJoinTransposeRule
to support SemiJoin
and AntiJoin
(Liya Fan)RelMdRowCount
doesn't take into account intersect all
(Xu Zhaohui){: #build-1-24-0}
elasticsearch:test
, disable the related tests first (Jiatao Tao){: #site-1-24-0}
SqlGroupingFunction
contradicts its behavior{: #v1-23-0}
This release comes two months after 1.22.0. It includes more than 100 resolved issues, comprising a lot of new features as well as performance improvements and bug-fixes. For some complex queries, the planning speed can be 50x or more faster than previous versions with built-in default rule set. It is also worth highlighting the following.
VolcanoPlanner
supports top down trait request and trait enforcement without abstract converter (CALCITE-3896)VolcanoPlanner
performance by removing rule match and subset importance (CALCITE-3753)VolcanoPlanner
performance when abstract converter is enabled (CALCITE-2970)SESSION
and HOP
Table function (CALCITE-3780, CALCITE-3737)Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 8, 9, 10, 11, 12, 13, 14 and OpenJDK 8, 9, 10, 11, 12, 13, 14; Guava versions 19.0 to 28.2-jre; other software versions as specified in gradle.properties.
{: #breaking-1-23-0}
RexWindow
, make fields upperBound
and lowerBound
not-nullableruleSet
(protected)and ruleNames
(private) in VolcanoPlanner
VolcanoPlanner
flags impatient
and ambitious
are removed, alternatively use checkCancel()
to achieve impatient
modeVolcanoPlanner
, transformation rules won't match with Enumerable physical operatorsAbstractMaterializedViewRule
into multiple classes (addendum){: #new-features-1-23-0}
VolcanoPlanner
supports top down trait request and trait enforcement without abstract converterSESSION
Table function (Rui Wang)HOP
Table function (Rui Wang)unnest
with items alias (Will Yu)SemiJoin
in EnumerableMergeJoin
CONCAT
for variable arguments (Wenhui Tang)EnumerableMergeJoin
support non-equi join conditionsSINH
functionCOMPRESS
function (ritesh-kapoor)EVERY
, SOME
, INTERSECTION
STRCMP
function{: #fixes-1-23-0}
Exchange
operator in RelFieldTrimmer
(Xu Zhaohui)Calc
in RelMdColumnOrigins
(Xu ZhaoHui)TableModify
json serialization and deserialization (Wang Yanlin)LogicalCalc
in RelShuttle
(dz)SortExchange
support to RelFieldTrimmer
(Krisztian Kasa)RelDistribution
json serialization (Krisztian Kasa)IntersectOnCalcsToIntersectUnifyRule
for materialized view recognition (dz)CONCAT
operatorapply(Mappings.Mapping)
to RelTrait
and RelTraitSet
Calc
in RelMdSize
, RelMdSelectivity
, RelMdMaxRowCount
, RelMdMinRowCount
, RelMdTableReferences
Intersect
and Minus
in Bindables
(xzh)RexWindow
, make fields upperBound
and lowerBound
not-nullableFilterMergeRule
to rely on RelBuilder
instead of RexProgram
RelTraitSet
RedisTable
is created (Xu Zhang)VolcanoPlanner.prunedNodes
info is lost when duplicate RelNode
is discarded (Botong Huang)CannotPlanException
when an empty LogicalValues requires a certain collationFilter
and Join
conditions may not need to retain nullability during simplificationsRelSubset
when it becomes deliveredJSON_VALUE
operands varadicArrayList
with initial capacity when size is known (Xu Zhang)RelDistributions.of()
and RelCollations.of()
should canonize trait instanceRexCall
from SqlWindowTableFunction
DEBUG
log level (Xiening Dai)RelSubset
matching is not properly handled in VolcanoRuleCall
(Botong Huang)FilterTableScanRule
generate wrong mapping for filter condition when underlying is BindableTableScan
(Jin Xing)SqlValidator.Config
UnionEliminatorRule
and ProjectRemoveRule
to auto pruning SubstitutionRule
(Botong Huang)dumpSets
and dumpGraphviz
out of VolcanoPlanner
RelSubset
is not fired for rule when set gets merged (Botong Huang)ruleSet
(protected)and ruleNames
(private) in VolcanoPlannerHint
item can not parse correctly if the name is right after token /*+createStatementParserMethods
DATE
and TIMESTAMP
returns a wrong resultSqlFunctions#addMonths
yields incorrect results in some corner case (Zhenghua Gao)updateCount
when creating MetaResultSet
(Robert Yokota)TIMESTAMP
type as DATETIME
for CAST
(Vineet Garg)RelMdMinRowCount
doesn't take into account UNION
DISTINCT
FilterIntoJoinRule
UNION ALL
applied to VALUES
RelCollationTraitDef.canConvert
should always return trueVolcanoPlanner
performance when enabling abstract converterSubstitutionVisitor
to consider RexCall
of type PLUS
and TIMES
for canonicalization (Vineet Garg)Config
for SqlValidator
JoinCommuteRule
should update all input references in join conditionRelOptPredicateList
may generate incorrect map of constant valuesRelToSqlConverter
should not merge a Project
that contains a window function that references a window function in input Project
RelTraitSet
SqlTypeAssignmentRule
(Wenhui Tang)EnumerableMergeJoin
: wrong comparison of composite key with null valuesMergeJoinEnumerator
should not use inputs enumerators until it is really requiredVALUES
that has column aliases produces wrong SQL in the JDBC adapterANTI
and SEMI
join to NOT EXISTS
and EXISTS
in the JDBC adapter. Also add forgotten IS_DISTINCT_FROM
translation supportRexSimplify
doesn't simplify NOT EQUAL predicatesIndexOutOfBoundsException
(Vineet Garg)SortJoinCopyRule
Mappings$NoElementException
(Vineet Garg)CASE WHEN
expression with nullability CAST
is considered as reduced wrongly in ReduceExpressionsRule
VolcanoPlanner
performance by removing rule match and subset importanceString.replaceAll
RelBuilder.aggregate
, cannot lookup field by nameRelNode
when merging child RelSet
with parent RelSet
RexSimplify
simplifies nondeterministic function incorrectlyEnumerableDefaults#orderBy
should be lazily computed + support enumerator re-initializationRelOptUtil::containsMultisetOrWindowedAgg
doesn't really check multiset (Xiening Dai)AbstractMaterializedViewRule
into multiple classes (addendum)JoinProjectTransposeRule
should skip Projects containing windowing expression (Vineet Garg)VolcanoPlanner
does not remove the entry in ruleNames when removing a ruleEnumerableMergeJoin
is never taken{: #build-1-23-0}
DiffRepository
lock contentionTIMESTAMP
and TIMESTAMP(0)
correctly (neoReMinD)PigRelBuilderStyleTest
since it fails too often for no reason{: #site-1-23-0}
SubstitutionVisitor
(xzh){: #v1-22-0}
This release comes five months after 1.21.0. It includes more than 250 resolved issues, comprising a large number of new features as well as general improvements and bug-fixes. Among others, it is worth highlighting the following.
We have also fixed some important bugs:
RelSet
sometimes gave inconsistent state,GROUP_ID
function gave incorrect results,Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 8, 9, 10, 11, 12, 13 and OpenJDK 8, 9, 10, 11, 12, 13; Guava versions 19.0 to 27.1-jre; Apache Flink 1.10.0; other software versions as specified in gradle.properties.
Project
, TableScan
, Calc
, Aggregate
and Join
introduce new parameter named hints
(CALCITE-482)RelNode
's create
method need to pass in hints explicitly (CALCITE-3723)Project
names will not represent in RelNode
digest anymore (CALCITE-3713)RexCall
s are default to be normalized in the RelNode
digest (CALCITE-2450)RelBuilder.aggregate
now would prune the unused fields from the input, thus the plan may change (CALCITE-3763)RelBuilder.scan
and sql-to-rel conversion always invoke RelOptTable.toRel
now, so there may be some plan changes for the TableScan
node if your RelOptTable.toRel
returns a physical rel before{: #new-features-1-22-0}
TRIM
Support for HIVE/SPARK Dialect (Dhirenda Gautam)COSH
functionTANH
functionEXISTSNODE
Function Support (ritesh-kapoor)TUMBLE
as Table Value Function (Rui Wang)CBRT
function (Qianjin Xu)TRIM
function in BigQuery DialectEXTRACT(XML)
Function Support (ritesh-kapoor)XMLTRANSFORM
Function Support (ritesh-kapoor)ExtractValue
functionRepeatUnion
All=falseHintStrategy
can specify excluded rules for plannerRelBuilder
Calc
Aggregate
(Shuo Cheng)RelOptRuleCall
to customize the propagation of hints before registering into planner rulestored_fields = _none_
to prohibit FetchPhase get involved (Yunfeng,Wu)MatchQuery
in ElasticSearch adapter (Shlok Srivastava)ProjectableFilterableTable
when Project
is not simple mapping (Jin Xing)CREATE FUNCTION
and DROP FUNCTION
ddl (Zhenqiu Huang)SqlValidator
that treats statements as valid if they contain unknown functions (Ryan Fu)CLASSIFIER
and LAST
functions for MATCH_RECOGNIZE
Window
in RelToSqlConverter
(Wenhui Tang){: #fixes-1-22-0}
UPDATE
assigns wrong type to bind variablesAggregateCall
RexSimplify
to simplify ‘a>1 or (a<3 and b)’ to ‘a>1 or b’ if column a is not nullableMATCH_RECOGNIZE
SqlToRelConverter.Config#isConvertTableAccess
HepPlanner
does not clear metadata cache for the ancestors of discarded node when a transformation happensRexSimplify
should return early if there is no pulled up predicate when simplifying using predicatesRelBuilder
view expander pluggableTableScanRule
RelBuilder
and ProjectMergeRule
, prevent merges when it would increase expression complexityRelBuilder.aggregate
should prune unused fields from the input, if the input is a Project
RelBuilder.transform
, which allows you to clone a RelBuilder
with slightly different ConfigHepPlanner.belongToDag()
doesn't have to use mapDigestToVertex (Xiening Dai)PruneEmptyRules#JOIN_RIGHT_INSTANCE
wrong behavior for JoinRelType.ANTI
EnumerableMergeJoin::create
`methodVolcanoPlanner.changeTraitsUsingConverters()
has parameter that's never usedRelHint
RelBuilder.Config
to an interface, and deprecate RelBuilder.ConfigBuilder
AggregateCaseToFilterRule
handles NULL
values incorrectlyGROUP_ID
returns wrong result (DonnyZone)RelSubset
should not match operand(RelNode.class)
INSERT
statementVolcanoPlanner.dumpGraphviz
should handle exception gracefully (Qianjin Xu)TableModify
in ToLogicalConverter
(dz)VolcanoPlanner
does not match all the RelSubSet
in matchRecursiveRelSet
gets mergedBETWEEN
with RelBuilder
throws class cast exceptionImmutableBeans
, allow interfaces to have default methodsRelOptRuleCall
to customize the propagation of hints before registering into planner ruleFilter
of distinct aggregate call is lost after applying AggregateExpandDistinctAggregatesRule
(Shuo Cheng)ProjectSetOpTransposeRule
to normalize materialized view (xy2953396112)Pair.forEach(Iterable, Iterable, BiConsumer)
RelBuilder.groupKey(ImmutableBitSet, ImmutableList<ImmutableBitSet>)
RelMetadataTest
fails with NPE due to unsafe RelMetadataQuery.instance
call (Jin Xing)RuleQueue#addMatch
: skip the match if it is not required for the phaseClassCastException
if group by is used on the result of scalar valued table function (DonnyZone)Hook#PLAN_BEFORE_IMPLEMENTATION
to capture the plan after optimizationProject#digest
RexCall
predicates to a canonical form validationEnumerableTableScan
constructor to validate if the table is suitable for enumerable scanRelOptTable
BitString.createFromBytes
reverses order of the bits in each byteRexSimplify
: CAST(CAST(intExpr as BIGINT) as INT) => intExprRexBuilder
may lose decimal fraction for creating literal with DECIMAL
type (Wang Yanlin)TableModify
of Update
contains correlated variable by mistake (Jin Xing)Correlate
should override estimateRowCount
TableModify
to Sql string (JinXing)SqlPrettyWriter
should have options to fold/chop long lines, print leading commasTableFunctionScan
(Wang Yanlin)RelMdColumnUniqueness
and RelMdUniqueKeys
for Calc
SortJoinTransposeRule
must not push sort into Project
that contains OVER
expressionsSort
rel contains explicit field name instead of * (Lei Jiang)JoinCommuteRule
when both inputs are the sameReduceExpressionsRule
LogicalTableModify
in RelShuttle (xy2953396112)RexSimplify
: eliminate redundant rex calls in ORSqlXmlFunctionsTest
locale (ritesh-kapoor)RelOptUtil.createCastRel
to not create new projection if the input rel is already a projectSqlLateralOperator
's unparse add additional keyword LATERAL
when the inner operator is SqlSnapshot
RexRangeRef
to avoid null stringEnumerableTableScan
: wrong JavaRowFormat
for elementType StringRelBuilder
for projecting out expressionsEnumerableJoinRule
: remove unnecessary Filter
on top of INNER
JoinTableFunctionScan
to SqlNode
(Wang Yanlin)SqlTypeAssignmentRules
conversion mapping pluggableEnumerableDefaults
nested loop join Provide a new implementation of nested loop join that, unlike the existing one, does not require to build the complete result as a list before returning it. Instead, it iterates through the outer and inner enumerables and returns the results step by step.BinaryExpression
evaluate method (Wang Yanlin)Interpreter
(Wang Yanlin)EnumerableIntersect
and EnumerableMinus
convert to Logical (xzh_dz)RecordType
(Wang Yanlin)Coalesce
due to wrong NullAs strategy (Jin Xing)CASE
and COALESCE
may produce NULL valuesRexSimplify
does not exploit all known predicatesProjectJoinTransposeRule
caused AssertionError when creating a new Join (Wenhui Tang)EnumerableDefaults#nestedLoopJoin
returns duplicates for JoinType.SEMICalciteSystemProperty
failed to load config fileExchange
, SetOp
and TableModify
for builtin metadata query (xy2953396112)SqlHintsConverterTest
(Shuo Cheng)CompileException
in Janino when a query contains a division between a Double
and a BigDecimal
(DonnyZone)RexUtil.simplifyOrs()
throws exception if terms has 1 RexNodeBitSet
to keep inheritPath in RelHint (Shuo Cheng)VolcanoPlanner.completeConversion()
(Xiening Dai)VolcanoPlanner#isValid
when DEBUG is enabled (Xiening Dai)AggregateOnCalcToAggUnifyRule
may ignore Project incorrectly (Jin Xing)ParameterScope
should override resolveColumn interface (Jark Wu)SqlSimpleParser
toke.s.equals() (Xiucheng Qu)SubstitutionVisitor#rowTypesAreEquivalent
(Min Dai)RelMetadataQuery
class in VolcanoPlanner.isValid() (Xiening Dai)ROW
s returns false for identical literal valueIN
subqueries in same scopeCAST
operation and BOOLEAN
type value in table macro (Wang Yanlin)AbstractRelNode
RexOver
(Wang Yanlin)RexSimplify
incorrectly simplifies IS NOT NULL
operator with ITEM
callEQUALS
operator between date/timestamp types returns false if the type is nullable (DonnyZone)TableScan
into the TableModify
(dy.Zhuang)VolcanoPlanner
CannotPlanException: best rel is null even though there is an option with non-infinite cost Problem solved via CALCITE-2018, just add a unit test for this specific scenarioExchange
in RelMdMaxRowCount
,RelMdMinRowCount
,RelMdRowCount
(xy2953396112)RelMetadataQuery
extensibleITEM
expression to SqlKind
and include it in the policy map for Strong (Aman Sinha)Intersect
and Minus
in RelMdPredicates
(Jin Xing)TableModify
in RelMdNodeTypes
(xy2953396113)CalciteConnectionConfigImpl
, add isSet and unset methods (Ryan Fu)RelToSqlConverter
does not properly alias ambiguous ORDER BY
SqlTypeExplicitPrecedenceList.COMPACT_NUMERIC_TYPES
because the NULL delimiters are uselessRelMdColumnUniqueness
for Filter
by considering constant columns (Jin Xing)RelBuilder.fields(ImmutableBitSet)
RelStructuredTypeFlattener
: improve support for functions with struct input (Igor Guzenko)CAST
of STRING
operand in binary comparison for BigQuery (soma-mondal)MINUS
and INTERSECT
in interpreter (Wang Yanlin)NullPointerException
throws for implicit type coercion of nested SET
operationsRelMetadataQuery
reuse (Jin Xing)RelOptRulesTest
and HepPlannerTest
by making test methods fluentAggregateExpandDistinctAggregatesRule
, treat all the agg expressions as distinct if they have the same arguments and the non-distinct expressions distinct constraints can be ignoredTUMBLE
grouping function into SQL parser (Rui Wang)UNION
with different ‘all’ property (Jin Xing)STRING
column expression of table during sql-to-rel conversionFixNullabilityShuttle
when using FilterIntoJoinRule
(Wang Yanlin, Shuming Li)JoinUnionTransposeRule.RIGHT_UNION
should not match SEMI
/ANTI
Join (Jin Xing)LatticeSuggester
, recommend lattices based on UNION
queriesJdbcSchema
in QuerySqlStatisticProvider
(Lindsey Meyer)Calc#accept(RexShuttle shuttle)
does not update rowType. (Jin Xing)ConstantExpression
support SET
literals (xy2953396112)SqlBetweenOperator
to compare DATE
and TIMESTAMP
Union
in RelMdRowCount.java
doesn't take into account ‘union all’ (Hong Shen)Trivial
null checking in RelSet#addAbstractConverters
(Jin Xing)LatticeSuggester
, allow join conditions that use expressionsSqlValidator
throws NPE for unregistered function without implicit type coercionLogicalCorrelate
from json string (Wang Yanlin)LogicalCalc
with RelInput type parameter (Wang Yanlin){: #adapters-1-22-0}
GROUP BY
... ORDER BY
queryHAVING
clause for BigQuery (Jin Xing)GROUP BY
clause of sub-query (Wang Weidong)RelToSqlConverter
generates NULLS LAST
and NULLS FIRST
wrongly when using MysqlSqlDialect
to convert RexOver
to sql (Wenhui Tang)INTERVAL
int64 time_unit. Range time unit is not supported yet by BigQuery (amaliujia)ROW
value expression, generates the ROW
keyword only if the dialect allows it (quxiucheng)UNION ALL
on BigQuery (Steven Talbot)NULLS FIRST
using CASE
expression (Justin Swett)SELECT TOP(n)
for MSSQL 2008 and earlier, and for Sybase
ASESUBSTRING
function to correct format (Jacky Woo)INTEGER
type as INT
(huangfeng){: #build-1-22-0}
CREATE TABLE ... AS SELECT ...
in Quidem testsHydromaticFileSetCheck
, upgrade Checkstyle 7.8.2 → 8.27SqlValidatorTest
and SqlParserTest
by making test methods fluentTpcdsLatticeSuggesterTest
because it does not workClass#getResource
in FileReaderTest instead of hard-coding file nameRexProgramTest#reproducerFor3457
testRexProgramFuzzy
so the results are simpler to reason aboutSqlPrettyWriterTest
, using a fluent API for invoking testsRelMetadataQuery
in test suites{: #dependency-1-22-0}
{: #site-1-22-0}
{: #v1-21-0}
This release comes two months after 1.20.0. It includes more than 100 resolved issues, comprising a large number of new features as well as general improvements and bug-fixes.
It is worth highlighting that Calcite now:
MATCH_RECOGNIZE
in the Enumerable
convention (CALCITE-1935);ANY
/SOME
/ALL
sub-queries (CALCITE-3031);ROW
, ARRAY
, and nested collection (CALCITE-3233, CALCITE-3231, CALCITE-3250);Enumerable
convention (CALCITE-2979, CALCITE-2973, CALCITE-3284).Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 8, 9, 10, 11, 12, 13 and OpenJDK 8, 9, 10, 11, 12, 13; Guava versions 19.0 to 27.1-jre; Apache Druid version 0.14.0-incubating; other software versions as specified in pom.xml
.
{: #breaking-1-21-0}
SqlTypeNameSpec
instead of SqlIdentifier
.Rule description ... is not valid
exception. The exception can easily disappear by changing the name of the Convention
which causes the problem.{: #new-features-1-21-0}
MD5
, SHA1
SQL functions (Shuming Li)jps
command for OS adapter (Qianjin Xu)REGEXP_REPLACE
function in Oracle, MySQL libraries (Shuming Li)RelBuilder.correlate
method, and allow custom implementations of Correlate
in RelDecorrelator
(Juhwan Kim)CONVERT_TIMEZONE
, TO_DATE
and TO_TIMESTAMP
non-standard SQL functions (Lindsey Meyer)CONCAT
function for Redshift (Ryan Fu)SqlDataTypeSpec
MATCH_RECOGNIZE
(Julian Feinauer, Zhiqiang-He)::
) (Muhammad Gelbana)ROW
type for SqlDataTypeSpec
ARRAY
type for SqlDataTypeSpec
ANY
/SOME
/ALL
sub-query (Vineet Garg)CHR
function (Sergey Tsvetkov, Chunwei Lei)AggregateCaseToFilterRule
, that converts SUM(CASE WHEN b THEN x END)
to SUM(x) FILTER (WHERE b)
DAYNAME
,MONTHNAME
functions; add locale
connection property (xuqianjin)TO_BASE64
, FROM_BASE64
SQL functions (Wenhui Tang){: #fixes-1-21-0}
JdbcRule
instances that have different JdbcConvention
to same VolcanoPlanner
's RuleSet
(Wenhui Tang, Igor Guzenko)generatePredicate
method from EnumerableNestedLoopJoin
, EnumerableHashJoin
, and EnumerableBatchNestedLoopJoin
into a single locationSqlUpdate
(Jin Xing)PigToSqlAggregateRule
should be applied on multi-set projection to produce an optimal plan (Igor Guzenko)RelNode
(Wang Yanlin)Sort
rel (Juhwan Kim)RelSubset
does not contain its best RelNode
(Xiening Dai)CAST(NULL AS type)
rather than NULL
conditionally (Wang Weidong)TRIM
to TRIM
, LTRIM
or RTRIM
when target is Hive (Jacky Woo)ProjectScanRule
: interpreterRexInputRef
in the projects (Jin Xing)RelMetadataQuery
cache is not invalidated when log trace is enabled (Xiening Dai)RelStructuredTypeFlattener
doesn't restructure ROW
type fields (Igor Guzenko)BinaryExpression
evaluate method support full numeric types in Primitive
(xy2953396112)RelXmlWriter
(Wang Yanlin)equals
and hashCode
methods final in AbstractRelNode
, and remove overriding methods in EnumerableTableScan
(Jin Xing)EquiJoin
SqlDataTypeSpec#deriveType(RelDataTypefactory)
UnionToUnionRule
for materialization matching (refine rule name) (Jin Xing)UnionToUnionRule
for materialization matching (Jin Xing)UnsupportedOperationException
for UPDATE
with IN
querySqlDataTypeSpec
DISTINCT
flag from calls to MIN
, MAX
, BIT_OR
, BIT_AND
aggregate functions (xuqianjin)IndexOutOfBoundsException
when generating deeply nested Java code from linq4j (Sahith Nallapareddy)RelBuilder
doesn't keep the alias when scan
from an expanded view (Jin Xing)RexSimplify
to handle (x <> a or x <> b)
Project
below Join
IndexOutOfBoundsException
when checking candidate parent match's input ordinal in VolcanoRuleCall
DATEADD
function (which is valid on Redshift) (Lindsey Meyer)RexBuilder.constantNull()
, because it produces untyped NULL
literals that make planning difficultValues
by generating SELECT
without FROM
LatticeSuggester
, record whether columns are used as “dimensions” or “measures”AssertionError
while serializing to JSON a RexLiteral
with Enum
type (Wang Yanlin)JoinToMultiJoinRule
should not match semi- or anti-LogicalJoinNOT NULL
expressionsMutableRel
may fail to be identified by SubstitutionVisitor
during matching (Jin Xing)Join
with LIKE
condition to SQL statement (wojustme)EXTRACT(MILLI/MICRO/NANOSECOND FROM date)
(Sergey Nuyanzin, Chunwei Lei)RepeatUnion
and TableSpool
)MutableMultiRel.setInput
, exception thrown (Jin Xing)VolcanoRuleCall
should look at RelSubset
rather than RelSet
when checking child ordinal of a parent operand (Botong Huang)SqlValidator
throws exception for SQL insert target table with virtual columnsIS NOT DISTINCT FROM
condition pushed from Filter
to Join
is not collapsed (Bohdan Kazydub)RelBuilder
configurableMutableAggregate
s with different row types should match with each other (Jin Xing)RelBuilder.aggregate
throws IndexOutOfBoundsException
if groupKey
is non-empty and there are duplicate aggregate functionsSqlStdOperatorTable.GROUPING
field to public classFrameworks
, add interface BasePrepareAction
(a functional interface) and deprecate abstract class PrepareAction
Filter
is copied with wrong traitSet (Juhwan Kim)Project#accept
(RexShuttle
shuttle) does not update rowTypeIndexOutOfBoundsException
in ProjectFilterTransposeRule
when executing SELECT COUNT
SEMI
/ANTI
join in RelDecorrelator
SemiJoin
on conditions push down throws IndexOutOfBoundsException
RelBuilder#empty
does not keep aliasesVolcanoPlanner
hangs due to sub-query with dynamic starclass CorrelateJoinType
class SemiJoinType
SemiJoin
usage completelyJdbcAggregate
in SqlImplementor
(Wenhui Tang)LogicalValues
can not be converted to UNION ALL
without operands which can not be unparsed (Musbah EL FIL)AggregateUnionAggregateRule
when remove the bottom Aggregate
(Hequn Cheng)RelDataType
CACHE in RelDataTypeFactoryImpl
can't be garbage collectedMutableProject
should be generated based on INVERSE_SURJECTION mapping (DonnyZone)IndexOutOfBoundsException
for SqlInsert
when source and sink have non-equal number of fields{: #build-1-21-0}
RelMetadataTest
PigRelExTest
, PigRelOpTest
and PigScriptTest
fail on WindowsSqlFunctionsTest
, replace assertEquals
and assertNull
with assertThat
DruidAdapterIT#testCastToTimestamp
(Justin Szeluga)TpcdsTest
using assertEqual
instead of printing results{: #site-1-21-0}
SubstitutionVisitor
(Jin Xing)ConverterRule
(TANG Wen-hui){: #v1-20-0}
This release comes three months after 1.19.0. It includes a large number of bug fixes, and additional SQL functions. There is now also explicit support for anti-joins. Several new operators have been added to the algebra to allow support for recursive queries. An adapter has also been added for Apache Kafka.
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 8, 9, 10, 11, 12, 13 and OpenJDK 8, 9, 10, 11, 12, 13; Guava versions 19.0 to 27.1-jre; Apache Druid version 0.14.0-incubating; other software versions as specified in pom.xml
.
{: #breaking-1-20-0}
EnumerableMergeJoin
extend Join
instead of EquiJoin
Correlate
use JoinRelType
instead of SemiJoinType
EnumerableThetaJoin
to EnumerableNestedLoopJoin
EnumerableJoin
to EnumerableHashJoin
SemiJoinFactory
from RelBuilder
, method semiJoin
now returns a LogicalJoin
with join type JoinRelType.SEMI
instead of a SemiJoin
SemiJoinFilterTransposeRule
, SemiJoinJoinTransposeRule
, SemiJoinProjectTransposeRule
and SemiJoinRemoveRule
match LogicalJoin
with join type SEMI
instead of SemiJoin
.SemiJoin
, EnumerableSemiJoin
, SemiJoinType
and CorrelateJoinType
, and methods that use them, are deprecated for quick removal in 1.21{: #new-features-1-20-0}
MultiJoin
rules with any project/filter (Siddharth Teotia)AntiJoin
relational expressionLEFT
, RIGHT
(xuqianjin)ASCII
, REPEAT
, SPACE
, SOUNDEX
, DIFFERENC
(pingle wang)JSON_STORAGE_SIZE
function (xuqianjin)REVERSE
function (pingle wang)REPEAT
, SPACE
, SOUNDEX
, DIFFERENCE
JSON_REMOVE
function (xuqianjin)AggregateJoinTransposeRule
(Vineet Garg)LatticeSuggester
deduces foreign keysLISTAGG
function (Sergey Nuyanzin, Chunwei Lei)JSON_KEYS
function (xuqianjin)RESPECT NULLS
, IGNORE NULLS
option for LEAD
, LAG
, FIRST_VALUE
, LAST_VALUE
, NTH_VALUE
functions (Chunwei Lei)RelBuilder
, add antiJoin
method (Ruben Quesada Lopez)RelBuilder
, add functionScan
method to create TableFunctionScan
(Chunwei Lei)ExchangeRemoveConstantKeysRule
that removes constant keys from Exchange
or SortExchange
(Chunwei Lei)WindowReduceExpressionsRule
(Chunwei Lei)JSON_LENGTH
function (xuqianjin)unifyAggregates
method to work with Grouping SetsLAST_DAY
function (Chunwei Lei){: #fixes-1-20-0}
CorrelateJoinType
(in favor of JoinType
)AggregateOnProjectToAggregateUnifyRule
ignores Project incorrectly when its Mapping breaks ordering (DonnyZone)MockSqlOperatorTable
RelBuilder
, eliminate duplicate aggregate callsRelDecorrelator
use wrong output map for LogicalAggregate
decorrelate (godfreyhe and Danny Chan)USER
and SYSTEM_USER
variables from DataContext
(Siddharth Teotia, Jacques Nadeau)SqlUtil#getSelectListItem
PlannerImpl
RelBuilder
, make alias method idempotentrelNode
's rowType
and digest as unique key for cache in RelOptPlanner
(KazydubB)CUBE
&ROLLUP
queries in SparkSqlDialect
(DonnyZone)IS NOT DISTINCT FROM
expression identification in RelOptUtil#pushDownJoinConditions
()SqlDialect
and SqlParser.Config
AggregateJoinTransposeRule
throws error for unique under aggregate keys when generating merged callstestSubprogram()
does not test whether subprogram gets re-executedFLOOR&SUBSTRING
functions in SparkSqlDialect
(DonnyZone)SqlLibraryOperators
provenanceMap
if not debugMaterializedViewAggregateRule
(Project-Aggregate): ArrayIndexOutOfBoundsException
RelToSqlConverter
may incorrectly throw an AssertionError
for some decimal literalsAggregateJoinTransposeRule
(Vineet Garg)JsonValueExpressionOperator
ProjectTransposeJoinRule
messes INDF expressionsCompileException
when inserting casted value of composited user defined type into tableAssertionError
when GROUP BY nested field (Will Yu)FULL OUTER JOIN
operator (Vineet Garg)NullPointerException
when casting null literal to composite user defined typeSqlParseException
when using component identifier for setting in merge statements (Danny Chan)FilterableTable
and the sql has multiple where conditionsRexCopier
should support all rex types (Chunwei Lei, Alexander Shilov)SqlItemOperator
should throw understandable exception message for incorrect operand type (pengzhiwei)ArrayEqualityComparer
should use Arrays#deepEquals
/deepHashCode
instead of Arrays#equals
/hashCode
(Ruben Quesada Lopez)ArrayEqualityComparer
should use Arrays#deepEquals
/deepHashCode
instead of Arrays#equals
/hashCode
RexOver
is incorrectly pushed down in ProjectSetOpTransposeRule
and ProjectCorrelateTransposeRule
(Chunwei Lei)ParseException
may be thrown for legal SQL queries due to incorrect “LOOKAHEAD(1)” hintsJsonValueExpression
into ExpressionDiffRepository
should ensure that XML resource file does not contain duplicate test names= ANY
sub-query (Vineet Garg)RelStructuredTypeFlattener
generates wrong types for nested column when flattenProjection
(Will Yu)ANY
sub-query in project (Vineet Garg)FilterProjectTransposeRule
generates wrong traitSet
when copyFilter
/Project
is true (Ruben Quesada Lopez)PushProjector
with OVER expression causes infinite loop (Chunwei Lei)getMaxRowCount
should return 1 for an Aggregate with constant keys (Vineet Garg)JoinProjectTransposeRule
should not pull a literal up through the null-generating side of a join (Chunwei Lei)MATCH_RECOGNIZE
VolcanoRuleCall
doesn't match unordered child operand when the operand is not the first operand. PruneEmptyRules
UNION
and MINUS
with empty inputs cause infinite cycle. (Zuozhi Wang)SubQueryJoinRemoveRule
and SubQueryProjectRemoveRule
passing on empty set instead of set of correlation id (Vineet Garg)IllegalStateException
when FilterCorrelateRule
matches a SEMI or ANTI Correlate (Ruben Quesada Lopez)RelStructuredTypeFlattener
generates wrong types on nested columns (Will Yu)EnumerableLimitRule
on Sort with no collation creates EnumerableLimit
with wrong traitSet
and cluster
(Ruben Quesada Lopez)SemiJoin
with lazy computation of innerLookup
(Ruben Quesada Lopez)TEMPORAL TABLE
ROLLUP
on MySQL 5RelFieldCollation
, add a withX
copy methodLatticeTest.testTileAlgorithm2
and LatticeTest.testTileAlgorithm3
fail intermittentlyorg.apache.calcite.util.Bug.CALCITE_461_FIXED
RelBuilder
wrongly skips creation of Aggregate that prunes columns if input produces one row at mostgetApplicableMaterializations
each time the rule is triggeredRelMetadataQuery
each time the rule is triggered{: #build-1-20-0}
{: #site-1-20-0}
NVL
and LTRIM
, in the SQL referenceRuleQueue.computeImportance()
is inconsistent (Meng Wang){: #v1-19-0}
This release comes three months after 1.18.0. It includes more than 80 resolved issues, comprising of a few new features as well as general improvements and bug-fixes. Among others, there have been significant improvements in JSON query support.
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 8, 9, 10, 11, 12 and OpenJDK 8, 9, 10, 11, 12; Guava versions 19.0 to 27.1-jre; Druid version 0.11.0; other software versions as specified in pom.xml
.
{: #new-features-1-19-0}
FOR SYSTEM_TIME AS OF
in regular queriesJSON_ARRAYAGG
JSON_TYPE
functionJSON_DEPTH
functionJSON_PRETTY
functionBIT_AND
, BIT_OR
HAVING
clause for aggregate functions{: #fixes-1-19-0}
NOT IN
query throws AssertionError
FILTER
is left untransformedAggregateJoinTransposeRule
generates non-equivalent nodes if Aggregate
contains a DISTINCT
aggregate function$0 = null
predicate from $0 IS NULL
when $0
is not nullableEnumerableDefaults.join_()
leaks connectionsFilterMultiJoinMergeRule
doesn't combine postFilterConditionProject(x=1)
and Project(x=1)
equal when the type of 1
is int
in the first rel and long
in the secondCalciteException
, SqlValidatorException
RelNode
description includes all tree when recomputedFilterProjectTransposeRule
does not always simplify the new filter conditionASCII(string)
in SqlFunctions
POSITION
, MOD
and set operators in BigQuery and Hive SQL dialectsROW_NUMBER
, RANK
generating invalid SQLCatalogReader#getAllSchemaObjects
in CatalogScope
getMonotonocity
is slow on wide tablesSqlIdentifier
same name with built-in function but with escape character should be still resolved as an identifierEnumerableConvention
SqlImplementor.createLeftCall
method throws StackOverflowError
catalog
and schema
from JDBC connect string to retrieve tables if specifiedPI
operator is incorrectly identified as dynamic function_id
field when querying ElasticSearchClosableAllocation
, ClosableAllocationOwner
, CompoundClosableAllocation
NullPointerException
EnumerableAggregate
, wrong result produced If there are sorted aggregates and non-sorted aggregates at the same timeAggregateJoinTransposeRule
incorrectly pushes down distinct count into joinAPPROX_COUNT_DISTINCT
aggregate function in ElasticSearchORDER BY
in sub-query, provided it has no LIMIT
or OFFSET
'2010-01-01' < TIMESTAMP
leads to incorrect resultsGROUP BY
when using raw item access (e.g. _MAP*
[‘a.b.c’])CannotPlanException
more informativeCONVENTION.NONE
planning with VolcanoPlanner
IS TRUE
checksBOOLEAN
literalsRexNode
simplification logic should use more specific UnknownAs
modesRexCall
digest containing IN
expressions leads to exceptionsBOOLEAN
column as filter operandCOMMA JOIN
as CROSS JOIN
for SparkSqlDialect
_MAP
RexLiteral.toJavaString()
Class.getSimpleName()
RelTraitPropagationVisitor
and remove its usagesany_value
with other aggregation functions failedRelMdPercentageOriginalRows
throws NullPointerException
when explaining plan with all attributesAbstractRelNode.computeDigest()
IS NULL
checks are incorrectly assuming that CAST
s are possibleCannotPlanException
: make boxes shorter, print composite traits if they were simplifiedSparkHandlerImpl
singleton thread-safeuserConfig
attribute in ElasticSearch adapter{: #build-1-19-0}
maven-remote-resources-plugin
to 1.6.0PlannerTest
ignores top-level ORDER BY
clause (RootRel.collation
)maven-checkstyle-plugin
StringBuffer
RelOptListener
LogicalAggregate
maven-compiler-plugin
from calcite-plus
module pom.xml
throw new RuntimeException(e)
in testsmaven-wrapper.jar
from source distributionDruidAdapterIT
regression after 1.17 releaseCalciteAssert
log4j-over-slf4j
) from cassandra-all
{: #site-1-19-0}
maven:alpine
to maven
image for generating javadoc when building the site_MAP
{: #v1-18-0}
With over 200 commits from 36 contributors, this is the largest Calcite release ever. To the SQL dialect, we added JSON functions and linear regression functions, the WITHIN GROUP clause for aggregate functions; there is a new utility to recommend lattices based on past queries, and improvements to expression simplification, the SQL advisor, and the Elasticsearch and Apache Geode adapters.
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 8, 9, 10, 11 and OpenJDK 10, 11; Guava versions 19.0 to 27.0.1-jre; Druid version 0.11.0; other software versions as specified in pom.xml
.
{: #new-features-1-18-0}
Planner
, allow parsing a stream (Reader
) instead of a String
(Enrico Olivelli)TIMESTAMPADD
function now applies to DATE
and TIME
as well as TIMESTAMP
(xuqianjin)CREATE
and DROP FUNCTION
(ambition119)JSON_EXISTS
, JSON_VALUE
, JSON_QUERY
, JSON_OBJECT
, JSON_OBJECTAGG
, JSON_ARRAY
, JSON_ARRAYAGG
, x IS JSON
predicate (Hongze Zhang)WITHIN GROUP
clause for aggregate functions (Hongze Zhang)YEAR
, SECOND
, DESC
non-reservedTRIM
function now trims more than one character (Andrew Pilloud)COVAR_POP
, COVAR_SAMP
, REGR_COUNT
, REGR_SXX
, REGR_SYY
SqlAdvisor
):from_clause
when _suggest_
token is located in one of the FROM
sub-queries*
is present in queryMINUS
--
commentsRelBuilder
):Exchange
and SortExchange
relational expressions (Chunwei Lei)RelBuilder.scan
should expand TranslatableTable
and viewsgroupKey
method that assumes only one grouping setproject
method should combine expressions if the underlying node is a Project
DISTINCT
and GROUP BY
without aggregate functions (Siyuan Liu)DATE
and NUMBER
NOT
operatorANY_VALUE
aggregate functionAggregate
(Andrei Sereda)DATE
, TIME
, TIMESTAMP
fields (Sandeep Chada)GeodeFilter
now converts multiple OR
predicates (on same attribute) into a single IN SET
(Sandeep Chada)BOOLEAN
values as strings (Andrei Sereda){: #fixes-1-18-0}
IndexOutOfBoundsException
when trying to infer operand type from STRUCT
return type (Rong Rong)RexBuilder
gives AssertionError
when creating integer literal larger than 263 (Ruben Quesada Lopez)INTEGER
and BIGINT
(Piotr Bojko)VARCHAR
type exceeding max lengthBaz
classes immutable (Stamatis Zampetakis)RelMetadataQuery.getTableOrigin
throws IndexOutOfBoundsException
if RelNode
has no columns (Zoltan Haindrich)Interner
instead of LoadingCache
to cache traits, and so allow traits to be garbage-collected (Haisheng Yuan).field
to follow any expression, not just tables and columns (Rong Rong)BETWEEN
and AND
(Qi Yu)RexNode
and its sub-classes implement hashCode
and equals
methods (Zoltan Haindrich)RexFieldAccess
should implement equals
and hashCode
methodsBasicSqlType
immutable, and now SqlTypeFactory.createWithNullability
can reuse existing type if possible (Ruben Quesada Lopez)IS DISTINCT FROM
could lead to exceptions in ReduceExpressionsRule
(Zoltan Haindrich)SqlDialect
supports pushing of all functions by defaultReduceExpressionsRule
fails to preserve type nullability (Zoltan Haindrich)RelMdTableReferences
should check whether references inferred from input are null for Union
/Join
operatorsRelMdExpressionLineage
SqlAggFunction
(Hongze Zhang)SqlNode
to SQL conversion fails if the join condition references a BOOLEAN
column (Zoltan Haindrich)RexShuttle
, use RexCall.clone
instead of new RexCall
(Chunwei Lei)EnumerableCorrelate
(Ruben Quesada Lopez)EnumerableCorrelate
SUM0(x)
to COALESCE(SUM(x), 0)
RelBuilder.toString()
FilterProjectTransposeRule
that can push down a Filter
that contains correlated variables (Stamatis Zampetakis)RexInputRef
that points to a call to a dynamic or non-deterministic function (Danny Chan)NullPointerException
while generating GROUP BY
query for MySQLRexFieldCollation.toString()
method is not deterministicOR
contains UNKNOWN
(Zoltan Haindrich)EnumerableDefaults#mergeJoin
should throw error and not return incorrect results when inputs are not ordered (Enrico Olivelli)EnumerableMergeJoin
is never takenOR
with nullable comparisons (pengzhiwei)CompoundQueryExpression
StandardConvertletTable#convertDatetimeMinus
NullPointerException
when FlatList
contains null elementsLattice
constructorSUBSTRING
function (Andrew Pilloud)Compatible
, CompatibleGuava11
IN
expressions when converting SqlNode
to SQL (Zoltan Haindrich)VolcanoPlanner#validate
Profiler
's surprise
and cardinality
attributes to avoid floating point discrepancies (Alisha Prabhu)SqlToRelConverter
gives ClassCastException
while handling IN
inside WHERE NOT CASE
(pengzhiwei)new
expression in constant array creationmatchRecognize
field of SqlSelect
SqlIdentifier
conversion to ITEM
operator for dynamic tables in ExtendedExpander
(Arina Ielchiieva)NameSet
, NameMap
, and NameMultimap
SparkHandlerImpl#compile
silent by default, print code in calcite.debug=true
mode onlyInteger.MAX_VALUE
(Hongze Zhang)AND
, OR
, COALESCE
, IS [NOT] DISTINCT
in RexUtil#op
org.apache.calcite.util.Source
, and use it for URL → File conversion in testsNullPointerException
NameSet
assumes lower-case characters have greater codes, which does not hold for certain charactersNameSet.contains
wrongly returns false
when element in set is upper-case and seek
is lower-casePOWER
, ATAN2
functions fail with NoSuchMethodException
HepPlanner
trace message (Dylan)AssertionError
when determining monotonicity (Alina Ipatina)Map.computeIfAbsent
when possibleSqlUtil.getAncestry
throws AssertionError
when providing completion hints for sub-schemaRelDataTypeImpl.getFieldList
throws AssertionError
when SQL Advisor inspects non-struct fieldSqlValidatorImpl.lookupFieldNamespace
throws NullPointerException
when SQL Advisor observes non-existing fieldRelToSqlConverter
throws ClassCastException
with structs (Benoit Hanotte)forbiddenapis
to 2.6 for JDK 11 supportowasp-dependency-check
maven plugin to 3.3.1RexSimplify
):RexProgramBuilder
makes unsafe simplifications to CASE
expressions (Zoltan Haindrich)RelBuilder
incorrectly simplifies a Filter
with duplicate conjunction to empty (Stamatis Zampetakis)ReduceExpressionRule
may oversimplify filter conditions containing NULL
valuesCASE
FilterReduceExpressionsRule
causes ArithmeticException
at execution timeCOALESCE(NULL, x)
→ x
(pengzhiwei)NOT-AND-OR
, RexSimplify
incorrectly applies predicates deduced for operands to the same operands (Zoltan Haindrich)UNKNOWN
value will be interpreted as is, or as TRUE
or FALSE
IS NOT FALSE(FALSE)
(zhiwei.pzw) (Zoltan Haindrich)COALESCE(+ nullInt, +vInt())
results in AssertionError: result mismatch
(pengzhiwei)COALESCE(NULL > NULL, TRUE)
produces wrong result filter expressions (pengzhiwei)CASE
with repeated branches gives AssertionError
(pengzhiwei)CAST
when operand has exactly the same type as it is casted toCASE
expressionsNOT TRUE
→ FALSE
, and NOT FALSE
→ TRUE
(pengzhiwei)UNKNOWN AND NOT (UNKNOWN OR ...)
(pengzhiwei)(c IS NULL) OR (c IS ...)
might result in AssertionError: result mismatch (pengzhiwei)IS NULL(NULL)
to TRUE
(pengzhiwei)x >= NULL
to UNKNOWN
(pengzhiwei)(NOT x) IS NULL
→ x IS NULL
(pengzhiwei); also, simplify f(x, y) IS NULL
→ x IS NULL OR y IS NULL
if f
is a strong operatorAND(x, y, NOT(y))
→ AND(x, null, IS NULL(y))
x AND NOT(x)
to FALSE
for nullable x
AssertionError
when simplifying IS [NOT] DISTINCT
expressions (Haisheng Yuan){: #build-1-18-0}
RelBuilderTest#testRelBuilderToString
fails on Windows (Stamatis Zampetakis)OsAdapterTest
now checks whether required commands are availablecheckSimplify3
with expected
, expectedFalse
, expectedTrue
all the sametry/final
block in RefCountPolicy
TestKtTest
and add apache-rat:check
to Travis CImaven-wrapper.properties
forbiddenapis
and maven-compiler
use the correct JDK versionmaven-wrapper.properties
is missing a headercommons:commons-pool2
is used in tests only, so use scope=test
for itfindbugs:jsr305
dependency optionalfailOnWarnings
to maven-javadoc-plugin
configurationCsvTest.testCsvStream
failing often: add @Ignore
since the test is known to failSqlTester.checkFails
(previously it was a no-op) (Hongze Zhang)parser.java
update on each buildCassandraAdapterTest
HepPlannerTest#testRuleApplyCount
complexityPartiallyOrderedSetTest#testPosetBitsLarge
with CalciteAssert.ENABLE_SLOW
RelMetadataTest#testMetadataHandlerCacheLimit
with CalciteAssert.ENABLE_SLOW
SqlValidatorDynamicTest
to CalciteSuite
SqlValidatorDynamicTest
, and avoid reuse of MockCatalogReaderDynamic
e.printStackTrace()
from CalciteAssert#returns
StreamTest#ROW_GENERATOR
to Table.scan().iterator
to make it not shared between threads (Sergey Nuyanzin)CalciteException
, SqlValidatorException
during testsModelTest#testYamlFileDetection
when source folder has spacesMockCatalogReader
is used in testing, so cache should be disabled there to avoid thread conflicts and/or stale resultsRexProgramTest
: replace nullLiteral
→ nullInt
, unknownLiteral
→ nullBool
for brevityRexProgramTest
: move “rex building” methods to base classSqlTestFactory
: use lazy initialization of objectsSqlTestFactory
{: #site-1-18-0}
README.md
{: #v1-17-0}
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 8, 9, 10; Guava versions 19.0 to 23.0; Druid version 0.11.0; other software versions as specified in pom.xml
.
This release comes four months after 1.16.0. It includes more than 90 resolved issues, comprising a large number of new features as well as general improvements and bug-fixes. Among others:
Implemented Babel SQL parser that accepts all SQL dialects. Allowed JDK 8 language level for core module. Calcite has been upgraded to use Avatica 1.12.0
{: #new-features-1-17-0}
SortRemoveConstantKeysRule
, that removes constant keys from Sort (Atri Sharma)CREATE TYPE
(Shuyi Chen)AggregateReduceFunctionsRule
(Fabian Hueske)NATURAL JOIN
and JOIN USING
LEFT SEMI JOIN
CHAR
columns of different lengths can now (based on a conformance setting) yield a VARCHAR
column (Hequn Cheng){: #fixes-1-17-0}
LATERAL
combined with window function or table functionOVERLAPS
should match even if operands are in (high, low) orderMIN
/MAX
functions (Muhammad Gelbana)FLOOR
to MySQL (Kang Wang, Sergey Nuyanzin)CalciteStatement
should call AvaticaStatement
close_(), to avoid memory leak (Kevin Risden)Double
and BigDecimal
arguments (余启).travis.yml
UNNEST
(Chunhui Shi)DATE
/INTERVAL
arithmetic (Rahul Raj)RelBuilder
into RelDecorrelator
and RelStructuredTypeFlattener
(Volodymyr Vysotskyi)JoinPushTransitivePredicatesRule
should not create Filter
on top of equivalent Filter
(Vitalii Diravka)SqlToRelConverter
generates incorrect ordinal while flattening a record-valued field (Shuyi Chen)AggregatePullUpConstantsRule
while adjusting Aggregate
indicesCASE
statement (Zoltan Haindrich)RelBuilder
for FilterRemoveIsNotDistinctFromRule
(Vitalii Diravka)AND
and OR
conditions using predicates (Zoltan Haindrich)JdbcProjectRule
ruleAbstractRelNode.id
(Zhong Yu)NOT
condition in JOIN
(Vitalii Diravka)ROW
value constructor in SELECT
clause and elsewhere (Danny Chan)SemiJoin
operator in materialized view-based rewriting algorithmAggregateJoinTransposeRule
fails to split aggregate call if input contains an aggregate call and has distinct rows (Haisheng Yuan)TIMESTAMPADD
function has wrong precision (Sudheesh Katkam)FlatList.equals()
throws StackOverflowError
(Zhen Wang, Zhong Yu)TIMESTAMPADD
(SQL_TSI_FRAC_SECOND
) should be nanoseconds (Sergey Nuyanzin)EXTRACT
function, support MICROSECONDS
, MILLISECONDS
, EPOCH
, ISODOW
, ISOYEAR
and DECADE
time units (Sergey Nuyanzin)TINYINT
and DOUBLE
typesRexLiteral.getValue3
with null literal of type DECIMAL
(Godfrey He)NumberFormatException
while starting SQLLineFilter
to JDBCFilter
(Piotr Bojko)defaultNullCollation
” configuration parameter when translating ORDER BY
inside OVER
(John Fang)EXTRACT
function: HOUR
, MINUTE
and SECOND
parts of a DATE
must be zero (Sergey Nuyanzin)(A or B) and C
fails for Elasticsearch adapter (Andrei Sereda)FLOOR(CEIL(x))
to FLOOR(x)
CEIL
function over date typesImmutableBitSetTest
for jdk11SqlSetOption
(Andrew Pilloud)ANY_VALUE
aggregate function (Gautam Parai)misc.iq
and scalar.iq
quidem unit tests failures on WindowsOsAdapterTest
failure on windows (Sergey Nuyanzin)NTH_VALUE
window function (Sergey Nuyanzin)getPulledUpPredicates
(Zoltan Haindrich)date
/timestamp
cast expressions in Druid adaptercommons-dbcp
to commons-dbcp2
version 2.4.0UNNEST
or LATERAL
fails with ClassCastException
NATURAL JOIN
and JOIN USING
when dynamic table is usedNULL
intervals in TIMESTAMPADD
and DATETIME_PLUS
functions (James Duong)SqlSelect
must call into SqlDialect
for unparse (James Duong)SparkAdapterTest
fails on Windows when ‘/tmp’ directory does not exist (Sergey Nuyanzin){: #v1-16-0}
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 8, 9, 10; Guava versions 19.0 to 23.0; Druid version 0.11.0; other software versions as specified in pom.xml
.
This release comes three months after 1.15.0. It includes more than 80 resolved issues, comprising a large number of new features as well as general improvements and bug-fixes to Calcite core. Among others:
SCAN
queries rather than SELECT
queries for more efficient execution and it can push more work to Druid using its new expressions capabilities, and the JDBC adapter now supports the SQL dialect used by Jethro Data.{: #new-features-1-16-0}
OFFSET
and FETCH
to data sourceSCAN
query rather than SELECT
query (Nishant Bangarwa){: #fixes-1-16-0}
EXTRACT
with WHERE ... IN
clause if field has null valueEXTRACT
calls (Nishant Bangarwa)DISTINCT
applied to VALUES
returns wrong result( IN ) OR ( IN )
logicFileReaderTest
fails with path containing spacesRelOptUtil.splitJoinCondition()
could not split condition with case after applying FilterReduceExpressionsRule
(Volodymyr Vysotskyi)IN
and OR
in WHERE
clause returns wrong result (Vineet Garg)NOT IN
operator and literal throws AssertionError
: ‘Cast for just nullability not allowed’ (Volodymyr Vysotskyi)JOIN
throws NullPointerException
(Zhen Wang)SqlValidatorImpl.findTable()
method incorrectly handles table schema with few schema levels (Volodymyr Vysotskyi)floor(time)
to Druid (Nishant Bangarwa)EXTRACT
gets pushed as an interval if possibleRelBuilder
into SqlToRelConverter
OsAdapterTest
in Calcite Plus does not respect locale (Piotr Bojko)Count(*)
returns null instead of 0 when condition filters all rowsdummy_aggregator
OR
of AND
clausesGROUP BY
inside UNION
does not compile (Zhen Wang)ORDER BY
keys, and ensure RelCollation contains no duplicates (John Fang)AggregateUnionAggregateRule
to reduce potential matching cost in VolcanoPlanner
(lincoln-lil)AggregateJoinTransposeRule
fails when process aggregate without group keys (jingzhang)TimeExtractionFunction
to Druid (Nishant Bangarwa)AggregateJoinTransposeRule
fails when process aggregateCall above SqlSumEmptyIsZeroAggFunction
without groupKeys (jingzhang)ArrayIndexOutOfBoundsException
in RexSimplify when using ReduceExpressionsRule.JOIN_INSTANCE
DruidAggregateFilterTransposeRule
TIMESTAMP
or DATE
literal, according to target type (Nishant Bangarwa)AggregateExpandDistinctAggregatesRule
should make SUM
nullable if there is no GROUP BY
(Godfrey He)OVER
clause, validator should treat query as an aggregate query (Volodymyr Tkach)FLOOR
and CEIL
of TIMESTAMP WITH LOCAL TIMEZONE
in local time zone (Nishant Bangarwa)CEIL
/FLOOR
functionsRelSubset.copy
method (Alessandro Solimando)CALCITE-2170
AggregateJoinTransposeRule
fails to aggregate over unique column (Zhong Yu){: #site-1-16-0}
{: #v1-15-0}
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 7, 8, 9, 10; Guava versions 14.0 to 23.0; Druid version 0.10.0; other software versions as specified in pom.xml
.
This release comes three months after 1.14.0. It includes than 44 resolved issues, mostly modest improvements and bug-fixes, but here are some features of note:
LIMIT
and OFFSET
and clauses;Calcite now supports JDK 10 and Guava 23.0. (It continues to run on JDK 7, 8 and 9, and on versions of Guava as early as 14.0.1. The default version of Guava remains 19.0, the latest version compatible with JDK 7 and the Cassandra adapter's dependencies.)
This is the last release that will support JDK 1.7.
{: #new-features-1-15-0}
OFFSET
, FETCH
and LIMIT
clauses (Enrico Olivelli)ReduceExpressionRule
simplifies a nullable expression, allow the result to change type to NOT NULL
SELECT orders[5].color FROM t
(Shuyi Chen)APPROX_COUNT_DISTINCT
RelDistribution
extend RelMultipleTrait
(LeoWangLZ)ST_X
and ST_Y
GIS functionsDatabaseProduct
with dialect methods, and introduce a configurable SqlDialectFactory
(Christian Beikov){: #fixes-1-15-0}
OVER
clause (Liao Xintao)JaninoRelMetadataProvider
loading cache might cause OutOfMemoryError
RexSimplify.removeNullabilityCast()
always removes cast for operand with ANY
type (Volodymyr Vysotskyi)ImmutableBitSet.set(int, boolean)
UPDATE
with dynamic parameter in SET
clause (Enrico Olivelli)Aggregate
might check for simple grouping sets incorrectlySqlCallBinding.getOperandLiteralValue(int)
to specify desired value typeAssertionError
when pushing project to ProjectableFilterableTable
RelBuilder
implementation in some rules (Volodymyr Vysotskyi)BindableTableScan
to make sure Project
is pushed through Aggregate
(Luis Fernando Kauer)LocalInterval
by Interval
in Druid adapterDISTINCT
in aggregate functionsProject
is pushed through Aggregate
(Luis Fernando Kauer)SqlStmt
syntax (Roman Kulyk)COUNT(druid_time_column)
as if it were COUNT(*)
FileReaderTest
fails with path containing spaces (Marc Prud'hommeaux)SubQueryRemoveRule
should create Join
, not Correlate
, for un-correlated sub-queries (Liao Xintao)IS DISTINCT FROM
condition in WHERE
or JOIN
clause fails with AssertionError
, “Cast for just nullability not allowed” (Volodymyr Vysotskyi)ORDER BY
null values (Abbas Gadhia)saffron.properties
file in classpath rather than in working directory (Arina Ielchiieva)NullPointerException
on filtered aggregators using IN
TRIM
signatureRexSimplify
behavior when literals come firstIS NULL
and IS NOT NULL
predicates to DruidVALUES
syntaxFilter
if predicates indicate they are always true or false=
and <>
operations with numeric cast on dimensions to DruidRelMdPredicates.getPredicates
is slow if there are many equivalent columns (Rheet Wong)EXTRACT
for JDBC (Pavel Gubin)RelBuilder.match
and methods for building patterns (Dian Fu)RelBuilder.aggregate
should rename underlying fields if groupKey
contains an aliasSELECT
if dialect does not support nested aggregate functions (Pawel Ruchaj){: #site-1-15-0}
FileSchemaFactory
javadoc (Marc Prud'hommeaux){: #v1-14-0}
This release brings some big new features. The GEOMETRY
data type was added along with 35 associated functions as the start of support for Simple Feature Access. There are also two new adapters. Firstly, the Elasticsearch 5 adapter which now exists in parallel with the previous Elasticsearch 2 adapter. Additionally there is now an [OS adapter]({{ site.baseurl }}/docs/os_adapter.html) which exposes operating system metrics as relational tables. ThetaSketch
and HyperUnique
support has also been added to the Druid adapter.
Several minor improvements are added as well including improved MATCH_RECOGNIZE
support, quantified comparison predicates, and ARRAY
and MULTISET
support for UDFs. A full list of new features is given below.
There are also a few breaking changes. The return type of RANK
and other aggregate functions has been changed. There also changes to Aggregate
in order to improve compatibility with Apache Hive. Finally, the Schema#snapshot()
interface has been upgraded to allow for more flexible versioning.
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 1.7, 1.8, 9; Guava versions 14.0 to 21.0; Druid version 0.11.0; other software versions as specified in pom.xml
.
{: #new-features-1-14-0}
GEOMETRY
data type and first 35 functions Add Spatial page, document GIS functions in SQL reference (indicating which ones are implemented), and add “countries” data set for testing.MATCH_RECOGNIZE
, support WITHIN
sub-clause (Dian Fu)ThetaSketch
and HyperUnique
support to Calcite via rolled up columns (Zain Humayun)sqlsh
NULL
literal as argumentARRAY
or MULTISET
(Ankit Singhal)"LIMIT [offset,] row_count"
, per MySQL (Kaiwang Chen){: #fixes-1-14-0}
RANK
and other aggregate functions. Various aggregate functions that used to return INTEGER
now return other types: RANK
, DENSE_RANK
, and NTILE
now return BIGINT
; CUME_DIST
and PERCENT_RANK
now return DOUBLE
. (This is a breaking change.)TIME
/TIMESTAMP
with local time zone types to optimizer.sha512
and .md5
digests for release artifactsSchema#snapshot()
(This is a breaking change.)Aggregate
, deprecate indicators, and allow GROUPING
to be used as an aggregate function (This is a breaking change.)AVG
, VARIANCE
, STDDEV
and COVAR
customizable via RelDataTypeSystem"NOT (x IS FALSE)" to "x IS NOT FALSE"; "x IS TRUE"
would be wrongNavigationExpander
and NavigationReplacer
in SqlValidatorImpl
(Dian Fu)tableName
calls in Druid (Zain Humayun)AggregateExpandDistinctAggregatesRule
when there are multiple AggregateCalls
referring to the same inputROUND()
and TRUNCATE()
to take one operand, defaulting scale to 0Project.getPermutation()
should return null if not a permutation (e.g. repeated InputRef
)JaninoRelMetadataProvider
, cache null values (Ted Xu)RexSubQuery
in RelToSqlConverter
rowType
of Match should include PARTITION BY
and ORDER BY
columnsRelDataTypeFactory.FieldInfoBuilder
NullPointerException
when target in ReflectiveSchema
belongs to root package (Lim Chee Hau)ClassCastException
LIKE
must match ‘.’ (period) literallyHepPlanner
should force garbage collect whenever a root registered (Ted Xu)SqlValidatorUtil.checkIdentifierListForDuplicates()
(Rajeshbabu Chintaguntla)LAST
within PREV
is not parsed correctly for MATCH_RECOGNIZE
FLOOR
operator when in a GROUP BY
(Chris Baynes)Sort.computeSelfCost()`` calls
makeCost()`` with arguments in wrong order (Junxian Wu)SqlToRelConverter
configurableNullPointerException
in RexUtil
(Ruidong Li)VolcanoPlanner
SqlKind.DYNAMIC
(parameters) in SqlImplementor
(Dylan Adams)StructKind.PEEK_FIELDS_NO_EXPAND
, similar to PEEK_FIELDS
but is not expanded in "SELECT *"
(Shuyi Chen){: #site-1-14-0}
{: #v1-13-0}
This release comes three months after 1.12.0. It includes more than 75 resolved issues, comprising a large number of new features as well as general improvements and bug-fixes.
First, Calcite has been upgraded to use Avatica 1.10.0, which was recently released.
Moreover, Calcite core includes improvements which aim at making it more powerful, stable and robust. In addition to numerous bux-fixes, we have implemented a new materialized view rewriting algorithm and new metadata providers which should prove useful for data processing systems relying on Calcite.
In this release, we have also completed the work to support the MATCH_RECOGNIZE
clause used in complex-event processing (CEP).
In addition, more progress has been made for the different adapters. For instance, the Druid adapter now relies on Druid 0.10.0 and it can generate more efficient plans where most of the computation can be pushed to Druid, e.g., using extraction functions.
There is one minor but potentially breaking API change in [CALCITE-1788] (Simplify handling of position in the parser), requiring changes in the parameter lists of parser extension methods.
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 1.7, 1.8, 9; Guava versions 14.0 to 21.0; Druid version 0.10.0; other software versions as specified in pom.xml
.
{: #new-features-1-13-0}
MATCH_RECOGNIZE
operator, for event pattern-matchingmatch_number
syntax support for MATCH_RECOGNIZE
MATCH_RECOGNIZE
(Zhiqiang-He)MATCH_RECOGNIZE
(Zhiqiang-He)MATCH_RECOGNIZE
(Zhiqiang-He)AFTER MATCH
sub-clause of MATCH_RECOGNIZE
clause (Zhiqiang-He)MEASURES
clause in MATCH_RECOGNIZE
(Zhiqiang-He)TIME
/TIMESTAMP
/DATE
datatype handling to RexImplicationChecker
EXTRACT
for time unit DOW
, DOY
; and fix CENTURY
Union
operatorsrequired Order
and requiresOver
parameters to the constructor of SqlUserDefinedAggregate Function
(SunJincheng)GROUP BY
and HAVING
to reference SELECT
expressions by ordinal and alias (Rajeshbabu Chintaguntla)CUBE
and ROLLUP
RelNode
expressions to be executed by JDBC driverCAST
of literal values in filters pushed to Druid (Remus Rusanu)OrderBy
/Limit
operation over time dimension and additional columns (Slim Bouguerra)Extraction
filter on Year
/Month
/Day
to druid (Slim Bouguerra)RelMdColumnUniqueness
for HepRelVertex
{: #fixes-1-13-0}
UnionMergeRule
to deal correctly with EXCEPT
UnionMergeRule
to deal with more than 2 branches (Pengcheng Xiong)COUNT(c)
; Druid only supports COUNT(*)
<>
as a filter correctly (Zain Humayun)FLOOR
operator (Chris Baynes)RelMetadataQuery
from planner to rules and invalidate in transformTo
(Remus Rusanu)NOT NULL
array to NULLABLE
array (Ankit Singhal)NULL
for ARRAY
constructor (Ankit Singhal)ProcessBuilder
is security sensitive; move it to test suite to prevent accidentsJaninoRelMetadataProvider
generated classes leak ACTIVE nodes on exception (Remus Rusanu)TIMESTAMP(3)
CAST(<string> as TIMESTAMP)
adds part of sub-second fraction to the valueUNION
in FROM
clause (Viktor Batytskyi, Minji Kim)AggregateExpandDistinctAggregatesRule
should work on Aggregate
instead of LogicalAggregate
(Haohui Mai)COUNT(DISTINCT)
is a GROUP BY
columnCAST(NULL AS ...)
gives NPE (Slim Bouguerra)WHERE FALSE
causes AssertionError
(Slim Bouguerra)WHERE CASE
throws AssertionError
“Cast for just nullability not allowed”MATCH_RECOGNIZE
(Zhiqiang-He)TUMBLE
/HOP
/SESSION_START
/END
do not resolve time field correctlyPERIOD
type constructor and period operators (CONTAINS
, PRECEDES
, etc.)SubstitutionVisitor
to use generic RelBuilder
instead of Logical instances of the operators when possibleTIMESTAMPADD(MONTH, ...)
should return last day of month if the day overflowsDATE
and TIME
values to int
, and TIMESTAMP
values to long
(Hongbin Ma)PigRelBuilderStyleTest
test cases are flappingIndexOutOfBoundsException
when using LATERAL TABLE
with more than one field (Godfrey He)DECIMAL
literals (Slim Bouguerra)DruidProjectFilterTransposeRule
against DruidQuery
(Nishant Bangarwa){: #site-1-13-0}
TIMESTAMPADD
, TIMESTAMPDIFF
functions (SunJincheng){: #v1-12-0}
Features of note this release are JDK 9 support, the new file/web and Apache Pig adapters, general improvements to the Druid adapter, more helpful error messages if you get a table or column name wrong, improved the plans for correlated sub-queries, support for TUMBLE
, HOP
and SESSION
window functions in streaming and regular queries, experimental support for MATCH_RECOGNIZE
clause for complex-event processing (CEP), several new built-in functions to comply with the ODBC/JDBC standard.
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 1.7, 1.8, 9; Guava versions 14.0 to 21.0; Druid version 0.9.1.1; other software versions as specified in pom.xml
.
{: #new-features-1-12-0}
IN
filter (Slim Bouguerra)MATCH_RECOGNIZE
, a new clause for complex-event processing (CEP) (Zhiqiang-He)FINAL
and other functions inside MATCH_RECOGNIZE
(Zhiqiang-He)PATTERN_DEFINE_AS
in SqlStdOperatorTable; MATCH_RECOGNIZE
now uses AS
(Zhiqiang-He)1 = 1
to TRUE
, 1 > 2
to FALSE
(Kevin Risden)DECIMAL
columnsHOP
and SESSION
functions in the GROUP BY
clause (Julian Hyde and Haohui Mai)$x = $x
to $x is not null
sqlline
commandmatchSeq
to allow selection of nth match<TH>
elementsGROUPING
function to have multiple arguments, like GROUPING_ID
RelBuilder.distinct
no-op if input is already distinct; use it in RelDecorrelator
MinRowCount
metadataSemiJoinRule
SortProjectTransposeRule
should check for monotonicity preserving CAST
INSERT
/UPSERT
without an explicit target column list, allow fewer source columns than table (Kevin Liew)TUMBLE
window function in the GROUP BY
clause (Julian Hyde and Haohui Mai)DATABASE
, IFNULL
, USER
(Laurent Goujon)REPLACE
function, callable with and without JDBC escape syntax (Riccardo Tommasini){: #fixes-1-12-0}
DruidAggregateFilterTransposeRule
, because it causes fine-grained aggregations to be pushed to DruidRexSimplify
, providing an explicit RexExecutor
for methods to simplify RexNode
sPigTest
cluster aware of data files; hopefully this will prevent intermittent test failures (Eli Levine)RexLocalRef
for EXPLAIN PLAN AS JSON
Filter
with other relations (Nishant Bangarwa)VARCHAR
and VARBINARY
types from 1 to “unspecified” (Kevin Liew)ClassCastException
in RelOptUtil.containsNullableFields
, attempting to convert executor to RexExecutorImpl
Union
, Minus
or Intersect
HAVING
support in RelToSqlConverter
(Zhiqiang He)ORDER BY
or GROUP BY
on TIMESTAMP
column throws CompileException (Gangadhar Kairi)LIKE
does not match value that contains newline (Mark Payne)ORDER BY
DruidQuery
to encourage early column pruning (Nishant Bangarwa)CAST('<string>' as TIMESTAMP)
wrongly adds part of sub-second fraction to the valueCAST('YYYY-MM-DD hh:mm:ss.SSS' as TIMESTAMP)
should round the sub-second fraction (Remus Rusanu)RexUtil.simplify
for constant reduction (Remus Rusanu)DateRangeRules
loses OR filtersMutableFilter
)RexBuilder.makeNullLiteral(RelDataType)
EnumerableMergeJoin
Correlate.joinType
attribute in lower-case, same as Join.joinType
AssertionError
while translating query with WITH
and correlated sub-queryDATE
, TIME
and TIMESTAMP
literals have Calendar
with GMT timezoneCAST
is ignored by rules pushing operators into DruidQuery
VolcanoPlanner
timeseries
query shows all days, even if no dataJdbcSchema
throws exception when detecting nullable columns (Wu Xiang)RelBuilder
sort-combining optimization treats aliases incorrectly (Jess Balint)RelBuilder.call
throws NullPointerException
if argument types are invalid (Jess Balint)java.sql.Date
(Zhen Wang)RelToSqlConverter
doesn't handle cartesian join (Jess Balint)Util.newInternal
, .pre
, .post
, .permAssert
and Throwables.propagate
UNION
has more than two inputs (Zhiqiang He)ORDER BY
is ambiguous (Zhen Wang)ConventionTraitDef.getConversionData()
is not thread-safetopN
query returns approximate resultstopN
query limit with granularityenum Granularity
SqlToRelConverter
throws UnsupportedOperationException
if query has NOT ... NOT IN
NOT IN
sub-queries with disjunctionSimpleCalciteSchema
AggregateExpandDistinctAggregatesRule
gets field mapping wrong if group key is used in aggregate function (Zhenghua Gao)AggregateJoinTransposeRule
fails to preserve row type (Kurt Young)AssertionError
(Kurt Young){: #site-1-12-0}
defaultNullCollation
key listed as materializationsEnabled
(Kevin Liew){: #v1-11-0}
Nearly three months after the previous release, there is a long list of improvements and bug-fixes, many of them making planner rules smarter.
Several adapters have improvements:
INSERT
, UPDATE
, DELETE
), windowed aggregates (OVER
), IS NULL
and IS NOT NULL
operators.For correlated and uncorrelated sub-queries, we generate more efficient plans (for example, in some correlated queries we no longer require a sub-query to generate the values of the correlating variable), can now handle multiple correlations, and have also fixed a few correctness bugs.
New SQL syntax:
CROSS APPLY
and OUTER APPLY
;MINUS
as a synonym for EXCEPT
;AS JSON
option for the EXPLAIN
command;INSERT
, allowing you to insert into individual fields of record-valued columns (or column families if you are using the Apache Phoenix adapter).A variety of new and extended built-in functions: CONVERT
, LTRIM
, RTRIM
, 3-parameter LOCATE
and POSITION
, RAND
, RAND_INTEGER
, and SUBSTRING
applied to binary types.
There are minor but potentially breaking API changes in [CALCITE-1519] (interface SubqueryConverter
becomes SubQueryConverter
and some similar changes in the case of classes and methods) and [CALCITE-1530] (rename Shuttle
to Visitor
, and create a new class Visitor<R>
). See the cases for more details.
Compatibility: This release is tested on Linux, macOS, Microsoft Windows; using Oracle JDK 1.7, 1.8; Guava versions 14.0 to 19.0; Druid version 0.9.1.1; other software versions as specified in pom.xml
.
{: #new-features-1-11-0}
RelBuilder.project
(Jess Balint)RAND
function, returning DOUBLE
values in the range 0..1RAND_INTEGER
function, which returns a random integer modulo N (Julian Feinauer)PARTITION BY
clause of window function (Hongbin Ma)INSERT
target column listCREATE TABLE
in tests (and only in tests)RelBuilder
, add field
method to reference input to join by alias (Jess Balint)INSERT
/UPDATE
/MERGE
CROSS
/OUTER APPLY
syntax (Jark Wu)OVER
clause to underlying SQL via JDBC adapter (Christian Tzolov)NOT IN
if we can deduce that the key is never nullIS NOT NULL
, and project predicatesAggregateValuesRule
, that applies to an Aggregate
on an empty relation (Gian Merlino)INTERSECT DISTINCT
by rewriting to UNION ALL
and counting (Pengcheng Xiong)MINUS
as a synonym for EXCEPT
(enabled in Oracle10 conformance) (Chandni Singh)ANY
type with binary comparison and arithmetic operators (Jungtaek Lim)CONVERT
function (Laurent Goujon)Intersect
and Minus
; flatten set-operators if the top is DISTINCT
and bottom is ALL
Table
SqlConformance
RelBuilder
, simplify “CAST(literal TO type)” to a literal when possibleFILTER
on aggregate functions in Interpreter
SUBSTRING
function for BINARY
and VARBINARY
(Jungtaek Lim)IS NULL
and IS NOT NULL
operators in generated SQL join condition (Viktor Batytskyi)LTRIM
, RTRIM
and 3-parameter LOCATE
and POSITION
(Jungtaek Lim)AS JSON
as output option for EXPLAIN
{: #fixes-1-11-0}
avatica
directory from sqlline
's class pathEXISTS
in WHERE
clauseRelNode
validationJdbcAdapterTest
, to ensure that tests that modify data run in seriesNOT IN
correlated queriesStrong
to infer whether a predicate's inputs may be nullVisitor
to Shuttle
OFFSET
cannot be pushed through a JOIN
if the non-preserved side of outer join is not count-preservingSetOp
with incompatible args (Jess Balint)HttpUtils
, don't log HTTP requests; they may contain user name, passwordCorrelate
(Alexey Makhmutov)SemiJoinRule
should not apply to RIGHT
and FULL JOIN
, and should strip LEFT JOIN
hydromatic-resource-maven-plugin
, and re-work SaffronProperties
LIMIT
with trivial ORDER BY
being pushed through JOIN
endlesslyEnumerableUnion
should use array comparator when row format is ARRAY
(Dayue Gao)AssertionError
in ReduceExpressionsRule
when CASE
is used with optional value and literal (Serhii Harnyk)ValuesReduceRule
should ignore empty Values
ALTER
statements (Gabriel Reid)AssertionError
in ReduceExpressionsRule
on multi-column IN
sub-query (Gian Merlino)Quidem
AutoCloseable
where possible (Chinmay Kolhatkar)aggdesigner
avatica-1.9
, sqlline-1.2.0
JaninoRelMetadataProvider
breaks shading (Jark Wu)RelOptPredicateList
fromNext
when requesting rows from Druid (Jiarong Wei)pagingIdentifiers
might have more than one value (Jiarong Wei)RelDataTypeFactoryImpl.copyType()
did not copy StructKind
FLOAT
, not DOUBLE
(Jiarong Wei){: #site-1-11-0}
{: #v1-10-0}
This release comes shortly after 1.9.0. It includes mainly bug-fixes for the core and Druid adapter. For the latest, we fixed an important issue that prevented us from handling consistently time dimensions in different time zones.
Compatibility: This release is tested on Linux, Mac OS X, Microsoft Windows; using Oracle JDK 1.7, 1.8; Guava versions 14.0 to 19.0; Druid version 0.9.1.1; other software versions as specified in pom.xml
.
{: #new-features-1-10-0}
!=
as an alternative to <>
{: #fixes-1-10-0}
ArrayIndexOutOfBoundsException
in sql-to-rel conversion for two-level columnsSTAR
, expand sub-fields in RecordType
columns of StructKind.PEEK_FIELDS
and StructKind.PEEK_FIELDS_DEFAULT
ITEM
operator ignores the value type of the collection, assigns to Object variable (Jungtaek Lim)CoreMatchers.containsString
causes javadoc errorsisDeterministic
only explores top RexCall
ClassCastException
in FilterReduceExpressionsRule
RelFieldTrimmer
utility methodsAggregatePullUpConstantsRule
might adjust aggregation function parameters indices wronglyDruidAdapterIT
broken{: #site-1-10-0}
org.apache.calcite.benchmarks.generated
, org.openjdk.jmh
from javadoc{: #v1-9-0}
This release includes extensions and fixes for the Druid adapter. New features were added, such as the capability to recognize and translate Timeseries and TopN Druid queries. Moreover, this release contains multiple bug-fixes over the initial implementation of the adapter. It is worth mentioning that most of these fixes were contributed by Druid developers, which demonstrates the good reception of the adapter by that community.
We have added new SQL features too, e.g., support for LATERAL TABLE
. There are multiple interesting extensions to the planner rules that should contribute to obtain better plans, such as avoiding doing the same join twice in the presence of COUNT DISTINCT
, or being able to simplify the expressions in the plan further. In addition, we implemented a rule to convert predicates on EXTRACT
function calls into date ranges. The rule is not specific to Druid; however, in principle, it will be useful to identify filter conditions on the time dimension of Druid data sources.
Finally, the release includes more than thirty bug-fixes, minor enhancements and internal changes to planner rules and APIs.
Compatibility: This release is tested on Linux, Mac OS X, Microsoft Windows; using Oracle JDK 1.7, 1.8; Guava versions 14.0 to 19.0; other software versions as specified in pom.xml
.
{: #new-features-1-9-0}
LATERAL TABLE
(Jark Wu){: #druid-adapter-1-9-0}
SegmentMetadataQuery
is used to detect types (Gian Merlino)Timeseries
and TopN
queries in DruidQuery
{: #planner-rules-1-9-0}
RexUtil.simplifyCase()
should account for nullabilityEXTRACT
function calls into date rangesProjectPusher
should use rel factories when creating new rels, e.g. project/filterUnionPullUpConstantsRule
{: #fixes-1-9-0}
Statement.cancel
methodDATE
+INTERVAL
arithmeticsSqlFunctionCategory
values for table functions and macros (Julien Le Dem)RexOptUtil
does not support function table other than SqlStdOperatorTable
NOT
precedenceRelTraits
other than Convention
DELETE
should be same as INSERT
SqlTypeFactory.leastRestrictiveSqlType()
Quidem
tests from JdbcTest
into their own classRelBuilder
should rename fields without creating an identity Project (Jark Wu)SqlTypeName
values for each interval range, e.g. YEAR_MONTH
GROUP BY
BETWEEN
operator (Yiming Liu)TIMESTAMP_ADD
applied to a DATE
should be TIMESTAMP
if unit is smaller than DAY
ORDER BY
SqlParserTest.testGenerateKeyWords
x.y.z AS z
AggFunctions
supported by JdbcAggregate
should depend on SqlKind
, instead of operator instanceViewExpander
(Julien Le Dem)RexCall
digests (Ted Xu)BigDecimal
value is less than 1NullPointerException
GROUP BY
SqlKind
, add OTHER_DDL
to DDL
enum set (Rajeshbabu Chintaguntla){: #site-1-9-0}
{: #v1-8-0}
This release adds adapters for Elasticsearch and Druid. It is also now easier to make a JDBC connection based upon a single adapter.
There are several new SQL features: UNNEST
with multiple arguments, MAP arguments and with a JOIN; a DESCRIBE statement; and a TRANSLATE function like the one in Oracle and PostgreSQL. We also added support for SELECT without FROM (equivalent to the VALUES
clause, and widely used in MySQL and PostgreSQL), and added a [conformance]({{ site.baseurl }}/docs/adapter.html#jdbc-connect-string-parameters) parameter to allow you to selectively enable this and other SQL features.
And a couple of dozen bug-fixes and enhancements to planner rules and APIs.
Compatibility: This release is tested on Linux, Mac OS X, Microsoft Windows; using Oracle JDK 1.7, 1.8; Guava versions 14.0 to 19.0; other software versions as specified in pom.xml
.
{: #new-features-1-8-0}
EXTRACT
, CEIL
and FLOOR
functions (Venki Korukanti)UNNEST
applied to MAP
data type (Johannes Schulte)INSERT
SELECT
without FROM
(Jimmy Xiang)UNNEST
with multiple argumentsUNNEST
with JOIN
TRANSLATE
function with 3 parameters, like the one in Oracle (Javanshir Yelchiyev)DESCRIBE
statement (Arina Ielchiieva){: #planner-rules-1-8-0}
LIMIT
+ OFFSET
in CassandraFilter
-on-Scan
to materialized view (Amogh Margoor)RelOptUtil.splitJoinCondition
to handle IS NOT DISTINCT FROM
(Venki Korukanti)CassandraSort
for LIMIT
ProjectRemoveRule
to pre-processing program of materialization substitution{: #fixes-1-8-0}
int
or float
ClassCastException
git.properties
ANY
type in RexBuilder.ensureType
and TypeFactory.leastRestrictive
(Mehand Baid, Minji Kim)SqlSetOption
to correctly handle SqlOperator.createCall
(Sudheesh Katkam, Minji Kim)ProjectJoinTransposeRule
(Minji Kim)SqlSumEmptyIsZeroAggFunction
constructor public (Minji Kim)IntList
with Guava Ints
classRelBuilder.field
gets offsets wrongLitmus
argument interpolation (Chris Baynes)RelBuilder.scan
to take qualified table name (Chris Baynes)Enumerables
to EnumerableDefaults
JoinPushThroughJoinRule
(Yi Xinglu)Parser.jj
template (Venki Korukanti)SqlParserTest
of reserved keywords from various versions of the SQL standardLIMIT
but no ORDER BY
gives invalid SQL (Emmanuel Bastien)enum SqlState
in Avatica, and deprecate SqlStateCodes
in CalciteCHAR
when performing cast to VARCHAR
SqlOperatorBinding.isOperandLiteral()
(Hsuan-Yi Chu)DatabaseMetaData.getColumnLabel
returns null when query has ORDER BY
CassandraTable
IN
RelOptRuleOperand
ConsList
SqlSetOperator
to be overridden, as a regular SqlOperator
can (Hsuan-Yi Chu){: #site-1-8-0}
EnumerableTableAccessRel
to EnumerableTableScan
{: #v1-7-0}
This is the first Apache Calcite release since [Avatica became an independent project]({{ site.avaticaBaseurl }}/news/2016/03/03/separate-project/). Calcite now depends on [Avatica]({{ site.avaticaBaseurl }}/) in the same way as it does other libraries, via a Maven dependency. To see Avatica-related changes, see the [release notes for Avatica 1.7.1]({{ site.avaticaBaseurl }}/docs/history.html#v1-7-1).
We have added an [adapter]({{ site.baseurl }}/docs/adapter.html) for Apache Cassandra. You can map a Cassandra keyspace into Calcite as a schema, Cassandra CQL tables as tables, and execute SQL queries on them, which Calcite converts into CQL. Cassandra can define and maintain materialized views but the adapter goes further: it can transparently rewrite a query to use a materialized view even if the view is not mentioned in the query.
This release adds an Oracle-compatibility mode. If you add fun=oracle
to your JDBC connect string, you get all of the standard operators and functions plus Oracle-specific functions DECODE
, NVL
, LTRIM
, RTRIM
, GREATEST
and LEAST
. We look forward to adding more functions, and compatibility modes for other databases, in future releases.
We‘ve replaced our use of JUL (java.util.logging
) with SLF4J. SLF4J provides an API which Calcite can use independent of the logging implementation. This ultimately provides additional flexibility to users, allowing them to configure Calcite’s logging within their own chosen logging framework. This work was done in [CALCITE-669].
For users experienced with configuring JUL in Calcite previously, there are some differences as some the JUL logging levels do not exist in SLF4J: FINE
, FINER
, and FINEST
, specifically. To deal with this, FINE
was mapped to SLF4J‘s DEBUG
level, while FINER
and FINEST
were mapped to SLF4J’s TRACE
.
Compatibility: This release is tested on Linux, Mac OS X, Microsoft Windows; using Oracle JDK 1.7, 1.8; Guava versions 12.0.1 to 19.0; other software versions as specified in pom.xml
.
{: #new-features-1-7-0}
TIMESTAMPADD
, TIMESTAMPDIFF
functions (Arina Ielchiieva)DECODE
, NVL
, LTRIM
, RTRIM
, GREATEST
, LEAST
{: #planner-rules-1-7-0}
AggregateRemoveRule
extensiblesimplify
for reducing expressionsIS NULL
and IS NOT NULL
in RexImplicationChecker
(Amogh Margoor)JoinUnionTransposeRule
to match Union
instead of LogicalUnion
(Vasia Kalavri)Filter
through Aggregate
(Amogh Margoor)RelNode
(Hsuan-Yi Chu)RelMdDistribution
to match other metadata APIs (Ted Xu)RelBuilder
, simplify predicates, and optimize away WHERE FALSE
Aggregate
on empty to empty if its GROUP BY
key is empty{: #fixes-1-7-0}
maven-remote-resources-plugin
TimeUnit
add WEEK
, QUARTER
, MICROSECOND
values, and change type of multiplier
SqlLiteral.SqlSymbol
; SqlSymbol
can now wrap any enumgroupId
when Calcite POMs reference Avatica modulesTIMESTAMP
values that may be NULL
CsvScannableTable.scan
is called twiceSqlNode.equalsDeep
has O(n ^ 2) performanceSTDDEV_SAMP
, STDDEV_POP
(Matthew Shaer)TIMESTAMP
values after noon (Chris Albright)Stacks
{: #site-1-7-0}
{: #v1-6-0}
As usual in this release, there are new SQL features, improvements to planning rules and Avatica, and lots of bug-fixes. We'll spotlight a couple of features make it easier to handle complex queries.
[CALCITE-816] allows you to represent sub-queries (EXISTS
, IN
and scalar) as RexSubQuery
, a kind of expression in the relational algebra. Until now, the sql-to-rel converter was burdened with expanding sub-queries, and people creating relational algebra directly (or via RelBuilder) could only create ‘flat’ relational expressions. Now we have planner rules to expand and de-correlate sub-queries.
Metadata is the fuel that powers query planning. It includes traditional query-planning statistics such as cost and row-count estimates, but also information such as which columns form unique keys, unique and what predicates are known to apply to a relational expression's output rows. From the predicates we can deduce which columns are constant, and following [CALCITE-1023] we can now remove constant columns from GROUP BY
keys.
Metadata is often computed recursively, and it is hard to safely and efficiently calculate metadata on a graph of RelNode
s that is large, frequently cyclic, and constantly changing. [CALCITE-794] introduces a context to each metadata call. That context can detect cyclic metadata calls and produce a safe answer to the metadata request. It will also allow us to add finer-grained caching and further tune the metadata layer.
Compatibility: This release is tested on Linux, Mac OS X, Microsoft Windows; using Oracle JDK 1.7, 1.8; other software versions as specified in pom.xml
.
{: #new-features-1-6-0}
RexNode
UNNEST ... WITH ORDINALITY
RelNode
to SQL (Amogh Margoor)FETCH/LIMIT
and OFFSET
in RelToSqlConverter (Amogh Margoor)JdbcImplementor
and JdbcRules
to new class SqlImplementor
DatabaseMetaData
RelToSqlConverterTest
on Windowsboolean
properties means trueTable
objectDATE
or TIMESTAMP
valueDATE
or TIMESTAMP
parametersNULLS FIRST
/NULLS LAST
not specified, sort NULL
values high{: #avatica-1-6-0}
CalciteRemoteDriverTest
ErrorResponse
's protobuf representation betterequals
and hashCode
methods uniformJdbcMeta
{: #planner-rules-1-6-0}
RelMetadataProvider
parameter to standard planner Program
sSubQueryRemoveRule
should create Correlate
, not Join
, for correlated sub-queriesFilter
simplifies to false (Jason Altekruse)Executor
configurable in FrameworkConfig
(Jason Altekruse)RelBuilder.empty
, and rewrite LIMIT 0 and WHERE FALSE to itFilter
operatorRexProgram
, in particular (NOT CASE ... END) IS TRUE
, which occurs in when NOT IN
is expandedRelOptRulesTest.testPushFilterPastProject
Aggregate
keys that are constantSortJoinTransposeRule
not firing due to getMaxRowCount(RelSubset)
returning nullRelMdUtil.checkInputForCollationAndLimit()
was wrong with alreadySorted
checkgetMaxRowCount
for Aggregate
(Mike Hinchey)LIMIT 0
results in an infinite loop (Pengcheng Xiong)FilterToProjectUnifyRule.invert(MutableRel, MutableRel, MutableProject)
works incorrectlyEnumerableSort
with DESC
wrongly sorts NULL
values lowSortProjectTransposeRule
's constructor{: #fixes-1-6-0}
DriverManager
before registering AlternatingDriver
ChunkList.clear
throws AssertionError
ReflectiveRelMetadataProvider.apply
waiting for HashMap.get
MILLISECOND
in TimeUnit
(Pengcheng Xiong)AND
expressionNullPointerException
in SqlJoin.toString()
ImmutableBitSet.rebuild()
ThreadLocal
values on exitDiffRepository
should not insert new resources at the end of the repositoryLitmus
(continuation-passing style for methods that check invariants)RelBuilder.project
now does nothing if asked to project the identity with the same field namesUtil
methods, and upgrade last Maven modules to JDK 1.7RelOptPredicateList
ImmutableNullableList.copyOf(Iterable)
javac
Parser.create(sql).parseQuery()
FILTER
is BOOLEAN NOT NULL
SqlKind
value for each built-in aggregate functionModelHandler
calling SchemaPlus.setCacheEnabled()
causes UnsupportedOperationException
when using SimpleCalciteSchema
OFFSET 0
causes AssertionError
(Zhen Wang)GROUP BY *constant*
on empty relation should return 0 rows.oq
Quidem files to .iq
AND
and OR
implementation in Enumerable
conventionSelfPopulatingList
is not thread-safeCloseable
with AutoCloseable
VolcanoPlanner
, populate RelOptRuleCall.nodeInputs
for operands of type “any”VolcanoPlanner
now clears ruleNames
in order to avoid rule name conflicting errorJdbcTest
to UdfTest
, and classes into Smalls
DELETE
(Yuri Au Yong)timezone
connection property to timeZone
{: #site-1-6-0}
mvn test
needs to be invokedKEYS
files, and direct from mirrors{: #v1-5-0}
Our first release as a top-level Apache project!
Avatica has undergone major improvements, including a new RPC layer that uses protocol buffers, support for DML statements, better support for bind variables and unique identifiers for connections and statements.
There are lots of improvements to planner rules, and the logic that replaces relational expressions with equivalent materializations.
We continue to find more uses for [RelBuilder]({{ site.baseurl }}/docs/algebra.html). We now recommend that you use RelBuilder
whenever you create relational expressions within a planner rule; the rule can then be re-used to create different sub-classes of relational expression, and the builder will perform simple optimizations automatically.
Using RelBuilder
we built Piglet, a subset of the classic Hadoop language Pig. Pig is particularly interesting because it makes heavy use of nested multi-sets. You can follow this example to implement your own query language, and immediately taking advantage of Calcite‘s back-ends and optimizer rules. It’s all just algebra, after all!
{: #new-features-1-5-0}
CalciteSchema
that does not cache sub-objectsSUM
’s return type by a customizable policy (Maryann Xue)ScannableTable
DEFAULT
arguments to function calls; support named arguments when calling table functions and table macrosARRAY
, MULTISET
, STRUCT
typesCOLLECT
aggregate functionEnumerableDefaults
methods (MiNG)ALTER ... RESET
statement (Sudheesh Katkam)GROUP BY
RelRoot
, a contract for the result of a relational expression{: #avatica-1-5-0}
getString()
on binary data (Josh Elser)ColumnMetaData
for arrays (Josh Elser)JdbcMeta
, register each statement using an id from a generator (Bruno Dumon)AvaticaSqlException
to pass server-side exception information to clients (Josh Elser)OpenConnectionRequest
(Bruno Dumon)setScale
on BigDecimal
when scale is 0 (Josh Elser)Execute
out from Fetch
request (Yeong Wei)JsonSubType
for ExecuteResponse
, and fix JSON docs (Josh Elser)getTables
returns empty result in JdbcMeta
(Jan Van Besien)JdbcMeta
statement IDs are not uniqueAvaticaConnection.getAutoCommit
throws NullPointerException
{: #materializations-1-5-0}
TableScan
without Project
(Maryann Xue)ArrayTable
{: #planner-rules-1-5-0}
RelMdPredicates
to deal with RexLiteral
(Pengcheng Xiong)SortUnionTransposeRule
for order-preserving Union
(Maryann Xue)SortJoinTransposeRule
for right joins (Maryann Xue)Aggregate
applied to already unique keys (Maryann Xue)ReduceExpressionsRule
handles duplicate constraints (Pengcheng Xiong)INTERVAL
literal (Hsuan-Yi Chu)SortUnionTransposeRule
(Pengcheng Xiong)ReduceExpressionsRule
extensibleProject
CASE
... END
) = constant” inside AND
or OR
(Hsuan-Yi Chu)SortJoinTransposeRule
LogicalProject
past LogicalWindow
, adjust references to constants properly (Hsuan-Yi Chu)Project
through Window
(Hsuan-Yi Chu)Aggregate
with Filter
through Union(all)
{: #rel-builder-1-5-0}
RelBuilder.scan()
now gives a nice exception if the table does not exist (Andy Grove)DUMP
applied to multisets and structsCOLLECT
in PigletVALUES
command to PigletRelNode
{: #fixes-1-5-0}
RelFieldTrimmer
RelFieldTrimmer
is applied to Aggregate
AbstractRelNode
may result in NPELEFT JOIN
to INNER
createProject
in RelOptUtil
should uniquify field namesRelNode.isKey
and isDistinct
methodsINTEGER
’ should be BIGINT
IN
sub-query (Maryann Xue)PARTITION BY
list (Hsuan-Yi Chu)GROUP BY
clauseStack
with ArrayDeque
ReflectiveRelMetadataProvider
is not thread-safeLogicalWindow.create()
ImmutableBitSet.get(int, int)
NullPointerException
in JdbcSchema
JdbcSchema
gives NullPointerException
on non-standard column type (Marc Prud'hommeaux)AVG
window function in GROUP BY
gives AssertionError
(Hsuan-Yi Chu)OVER
query (Hsuan-Yi Chu)TableScan
does not support large/infinite scans (Jesse Yates)OVER
clause (Hsuan-Yi Chu)IN
clause to semijoin (Josh Wills){: #v1-4-0}
In addition to a large number of bug-fixes and minor enhancements, this release includes improvements to lattices and materialized views, and adds a builder API so that you can easily create relational algebra expressions.
{: #new-features-1-4-0}
RelBuilder
, builder for expressions in relational algebraCOUNT
using GROUPING SETS
BitSet
and ImmutableBitSet
utilities{: #site-1-4-0}
{: #fixes-1-4-0}
DEPENDENCIES
file includes all module dependenciesFilterJoinRule
and create a new rule for itStackOverflowError
getting predicates from the metadata providerRelOptUtil.splitJoinCondition
incorrectly splits a join condition (Hsuan-Yi Chu)LogicalAggregate
LogicalAggregate
collation inferring logic made in [CALCITE-783] (Milinda Pathirage)ProjectFactory
in AggregateJoinTranposeRule
and FilterJoinRule
Frameworks
gives NPE when FrameworkConfig
has no default schemaJoinProjectTransposeRule
with option to support outer joinsMYSQL_ANSI
Lexing policyRexBuilder
reverses precision and scale of DECIMAL
literalupdateCount
, maxRows
from int to longRelBuilder.avg
ROW_NUMBER
should emit distinct valuesProjectMergeRule
RelOptUtil.pushFilterPastProject
Project
using monotonicity (Milinda Pathirage)SqlOperator.getMonotonicity
to allow it to be used for RexNode
as well as SqlNode
GROUP BY
is present, ensure that window function operands only refer to grouping keys (Hsuan-Yi Chu)HAVING count(*) > 1
NullPointerException
using USING
on table alias with column aliasesMetaImpl.MetaCatalog
should expose TABLE_CAT
instead of TABLE_CATALOG
RelBuilder
, calling sort
then limit
has same effect as calling sortLimit
Ord.reverse
EnumerableJoin
to be sub-classed (Li Yang)BigDecimal
underflow (Li Yang)Sort
to MutableSort
(Maryann Xue)Content-Type
from the RPC server to application/json
(Lukáš Lalinský)RelBuilderTest
CASE
and NULL
Aggregate
recommender blows up if row count estimate is too highAggregate
operators may derive row types with duplicate column namesSqlTypeName
methodsOVER
clause of JOIN
queryWHERE
clause causes wrong result in MongoDB adapterClassCastException
in MongoDB adapterTableFactory
from suggested table name, so one TableFactory
can be used for several tablesMaterializationService.TableFactory
(Rajat Venkatesh)Statement.execute()
for Avatica (Xavier Leong)setMaxRows
for statement execute (Xavier Leong)LoggingLocalJsonService
, to make it easier to test that JDBC requests cause the right RPCsDatabaseMetaData.getTypeInfo
(Xavier Leong)JdbcTest.testVersion
more permissive, so that version.major
and version.minor
can be set just before a release, rather than just after as at present{: #v1-3-0}
Mainly bug-fixes, but this release adds support for modifiable views and filtered aggregate functions and various improvements to Avatica.
{: #new-features-1-3-0}
FILTER
clause for aggregate functions{: #api-1-3-0}
RelOptQuery
RelOptCluster
AggregateProjectMergeRule
in standard rule setSemiJoin
SWAP_OUTER
static instance to JoinCommuteRule
(Maryann Xue)Primitive.DOUBLE.min
should be large and negative{: #fixes-1-3-0}
splitCondition
does not behave correctly when one side of the condition references columns from different inputsCASE
statement against JDBC tables generates invalid Oracle SQL (Yeong Wei)VolcanoPlanner.ambitious
comment doc is inconsistent (Santiago M. Mola)IndexOutOfBoundsException
in ROLLUP
query on JDBC data sourceCOUNT
query gives wrong resultsClassCastException
in table from CloneSchema
SELECT
or HAVING
clause gives AssertionError
HAVING
clause for sub-queries and IN
-lists (Hsuan-Yi Chu)WHERE
clause cause AssertionError
(Sean Hsuan-Yi Chu)SINGLE_VALUE
aggregate function to a sub-query that will never return more than one row (Hsuan-Yi Chu)LIMIT
inside scalar sub-queryGROUP BY ()
, areColumnsUnique()
should return true for any keyRemoteDriverTest.testTypeHandling
fails east of GreenwichRemoteDriverTest.testStatementLifecycle
thread-safeSqlNode.unparse
produces invalid SQLCalciteAssert.assertArrayEqual
for more user-friendly assertsSQLFeatureNotSupportedException
(Ng Jiunn Jye)ByteString
does not deserialize properly as a FetchRequest
parameter valueAssertionError
in GROUPING SETS
querySemiJoinRule
mixes up fields when Aggregate.groupSet
is not field #0{: #v1-2-0}
A short release, less than a month after 1.1.
There have been many changes to Avatica, hugely improving its coverage of the JDBC API and overall robustness. A new provider, JdbcMeta
, allows you to remote an existing JDBC driver.
[CALCITE-606] improves how the planner propagates traits such as collation and distribution among relational expressions.
[CALCITE-613] and [CALCITE-307] improve implicit and explicit conversions in SQL.
{: #new-features-1-2-0}
CAST
between date-time typesORDER BY
aggregate function in SELECT DISTINCT
, provided that it occurs in SELECT
clause (Sean Hsuan-Yi Chu)firstOrDefault
, single
, and singleOrDefault
methods (Daniel Cooper)NullPointerException
when executing JdbcAggregate.implement
method (Yuri Au Yong)averageTypeValueSize
method in RelMdSize
(Jesus Camacho Rodriguez)SemiJoin
(Jesus Camacho Rodriguez)splitCondition
method in RelOptUtil
to handle multiple joins on the same key (Jesus Camacho Rodriguez){: #avatica-1-2-0}
AvaticaPreparedStatement
should support execute()
and executeUpdate()
(Nick Dimiduk)AvaticaStatement.execute
method broken over remote JDBC (Yeong Wei and Julian Hyde)ConnectionSync
RPC (Nick Dimiduk)AvaticaStatement.execute
semantics (Nick Dimiduk)avatica
into avatica-server
(Nick Dimiduk)JdbcMeta
, convert property definitions to an enum (Nick Dimiduk)CloseConnection
RPC (Nick Dimiduk)CloseStatement
RPC (Nick Dimiduk)AvaticaParameter.setObject
(Nick Dimiduk)getTableTypes
, getColumns
(Xavier FH Leong)getTables
(Julian Hyde and Nick Dimiduk){: #api-1-2-0}
CachingInvocationHandler
that MD provider is not null (Jesus Camacho Rodriguez)REAL
is 4 bytes, FLOAT
is 8 bytes{: #fixes-1-2-0}
ANY
type should be nullable (Jinfeng Ni)NullPointerException
in JdbcTest
ORDER BY
clause is used with WITH CLAUSE
ProjectMergeRule
description for new naming convention (Jinfeng Ni)README.md
linking to the wrong page of optiq-csv
(hongbin ma)SUPERCLASS_COMPARATOR
in ReflectiveRelMetadataProvider
is inconsistent (Jesus Camacho Rodriguez)Functions.adapt
WITH ... ORDER BY
cannot find tableIN
clause in CASE
in GROUP BY
gives AssertionError
{: #v1-1-0}
This Calcite release makes it possible to exploit physical properties of relational expressions to produce more efficient plans, introducing collation and distribution as traits, Exchange
relational operator, and several new forms of metadata.
We add experimental support for streaming SQL.
This release drops support for JDK 1.6; Calcite now requires 1.7 or later.
We have introduced static create
methods for many sub-classes of RelNode
. We strongly suggest that you use these rather than calling constructors directly.
{: #new-features-1-1-0}
TableMacro
to consume maps and collections||
mishandles ANY
type (Sean Hsuan-Yi Chu)ProjectableFilterableTable
SetOpFactory
in rules containing Union
operator (Jesus Camacho Rodriguez)RelMdDistribution
for Project
operator is called (Jesus Camacho Rodriguez)RelNode
metadataArrayIndexOutOfBoundsException
when deducing collation (Aman Sinha)LogicalSort
relational expression, and make Sort
abstractEnumerableMergeJoin
, which exploits sorted inputsRelCollation
trait and AbstractRelNode.getCollationList
consistentRelCollation
on aliased columns in JoinRule
ArrayIndexOutOfBoundsException
when deducing collationRelDistribution
trait and Exchange
relational expression{: #api-1-1-0}
RelNode
now have a static create
method which automatically sets up traits such as collation and distribution. The constructors are not marked deprecated, but we strongly suggest that you use the create
method if it exists.jetty-server
to 9.2.7.v20150116 and port avatica-server HttpServer
(Trevor Hartman)Context
instances by wrapping objects and by chaining contextsEquiJoin
in wrong package (Jesus Camacho Rodriguez)RelOptUtil.createProject
and createRename
Project.flags
(methods are deprecated, to be removed before 2.0){: #fixes-1-1-0}
LICENSE
file of calcite-example-csv (the former optiq-csv) and move its history into main historysetAggChildKeys
should take into account indicator columns of Aggregate
operator (Jesus Camacho Rodriguez)ReduceExpressionsRule
requires planner to have an Executor
TableScanNode.create
methodFrameworks
should expand identifiers (Jinfeng Ni)tzdata2014f
, Java 1.8.0_31Integer
column throws ClassCastException
Signature.internalParameters
{: #v1-0-0}
Calcite's first major release.
Since the previous release we have re-organized the into the org.apache.calcite
namespace. To make migration of your code easier, we have described the mapping from old to new class names as an attachment to [CALCITE-296].
The release adds SQL support for GROUPING SETS
, EXTEND
, UPSERT
and sequences; a remote JDBC driver; improvements to the planner engine and built-in planner rules; improvements to the algorithms that implement the relational algebra, including an interpreter that can evaluate queries without compilation; and fixes about 30 bugs.
{: #new-features-1-0-0}
NEXT
/CURRENT VALUE FOR
syntax for using sequencesUPSERT
statement in parserEXTEND
clause, for defining columns and their types at query/DML timeschema.table.column
LATERAL
expression{fn QUARTER(date)}
function (Benoy Antony)GROUPING SETS
, CUBE
, ROLLUP
in SQL and algebraGROUP_ID
,GROUPING_ID
, GROUPING
functionsProject
that merely renames fieldsAND(x, y, x)
to AND(x, y)
in more circumstancesAbstractConverter
Aggregate
with grouping sets in RelMdColumnOrigins
(Jesus Camacho Rodriguez)FilterAggregateTransposeRule
(Jesus Camacho Rodriguez)AggregateFilterTransposeRule
(Jesus Camacho Rodriguez)AggregateProjectMergeRule
(Jesus Camacho Rodriguez)AggregateExpandDistinctAggregatesRule
(Jesus Camacho Rodriguez)AggregateUnionTransposeRule
(Jesus Camacho Rodriguez)Aggregate
operator without grouping sets (Jesus Camacho Rodriguez)Correlate
mechanics and implement EnumerableCorrelate
(aka nested loops join)Union
in interpreterJoin
in interpreter and improve handling of scalar expressionsAggregate
(including GROUPING SETS
) in interpreter (Jacques Nadeau)BINDABLE
convention (but ENUMERABLE
is still the default), and add ArrayBindable
and Scalar
interfacesJsonHandler
and JsonService
thread-safe{: #api-1-0-0}
ImmutableBitSet
and replace uses of BitSet
RelNode.getChildExps
to RelNode.accept(RexShuttle)
rowType
field and constructor/copy argument of Calc
OneRow
and Empty
relational expressions; Values
will suffice{: #fixes-1-0-0}
NOTICE
, DEPENDENCIES
, and LICENSE
filesParser.jj
only at first buildorg-apache-calcite-jdbc.properties
git.properties
@PackageMarker
to package-info.java
so maven-compiler skips compilation when the sources are unchangedEnumerableTableScan
broken when table has single columnProjectRemoveRule
that considers a project trivial only if its field names are identical (John Pullokkaran)ReduceExpressionsRule
tries to reduce SemiJoin
condition to non-equi conditionpentaho-aggdesigner
that does not pull in servlet-api
quidem
dependency have scope “test”ReduceExpressionsRule
throws “duplicate key” exceptionFULL JOIN
CalciteAssert
to respect multiple settingsGROUP BY
on a CASE
expression containing IN
predicate fails (Aman Sinha)induce
method to return CUBE
and ROLLUP
(Jesus Camacho Rodriguez)Group.induce
by assuming that group sets are sortedOR
FilterIntoJoinRule
creates filters containing invalid RexInputRef
RelNode
, be tolerant if it is equivalent to a RelNode
with different traitsitem(any,...)
operator.m2
directoryResultSetMetaData.getColumnClassName
(Knut Forkalsrud)EnumerableRelImplementor.stash
so it is suitable for all kinds of classesEnumerable
sEnumerable
over random-access listcopy
method in LogicalAggregate
not copying the indicator value properlyRelMdColumnUniqueness
uses ImmutableBitSet.Builder
twice, gets NullPointerException
Enumerable<Holder>
does not work if where Holder
is a custom class with a single field; Calcite tries to treat it as SCALAR
due to premature JavaRowFormat.optimize
ResultSet.next()
is called after close()
Enumerable
gives NullPointerException
with NOT
on nullable expressionWITH
, LATERAL
, GROUPING SETS
, CUBE
, ROLLUP
; add descriptions for all built-in functions and operatorsEMPS.deptno
column Integer → intLinq4j.product
; the cartesian product of 0 attributes is one row of 0 attributespackage-info.java
JdbcTest.testVersion
now that version is 1.0{: #v0-9-2}
A fairly minor release, and last release before we rename all of the packages and lots of classes, in what we expect to call 1.0. If you have an existing application, it's worth upgrading to this first, before you move on to 1.0.
{: #new-features-0-9-2}
Table
{: #api-0-9-2}
RelOptRuleOperand
constructor that takes a predicate{: #fixes-0-9-2}
SELECT DISTINCT *
on reflective schema gives ClassCastException
at runtimeRelMetadataQuery
, and add some more tests for [CALCITE-443]getPredicates
from a Union is not correctFilterAggregateTransposeRule
loses conditions that cannot be pushedLoptOptimizeJoinRule
incorrectly re-orders outer joinsSqlValidatorUtil.uniquify()
may not terminate under some conditionsSemiJoinRel
LIKE ... ESCAPE
.DiffRepository
documentation.ORDER BY
. (Works already.){: #v0-9-1}
This is the first release as Calcite. (The project was previously called Optiq.)
{: #new-features-0-9-1}
RelNode
s using an interpreterSqlRun
into its own artifact, Quidem.NULL
values in NOT IN
sub-queriesREGR_SXX
and REGR_SYY
regression functionsCOVAR_POP
and COVAR_SAMP
aggregate functionsFilterAggregateTransposeRule
, that pushes a filter through an aggregateAND
factors out of OR
predicatesMergeProjectRule
should not construct RexProgram
s for simple mappingsRexUtil.toCnf()
, to convert expressions to conjunctive normal form (CNF)MergeFilterRule
should flatten AND
conditionFieldTrimmer
work with RelNode
base classesFieldTrimmer.trimFields(SetOp)
generate ProjectRel
instead of CalcRel
{: #api-0-9-1}
RelDataTypeSystem
plugin, allowing different max precision of a DECIMAL
Planner
, query de-correlation no longer requires state in a SqlToRelConverter
.RelFieldTrimmer
should use factory to create new rel nodesCalcRel.createProject
methods to RelOptUtil
JoinFactory.createJoin()
; add SemiJoinFactory
{: #fixes-0-9-1}
RemoveDistinctAggregateRule
, AggregateFilterTransposeRule
sqlline
shell scriptpackage-info.java
was causing errors in Eclipse.RelFieldTrimmer
: when trimming SortRel
, the collation and trait set don't matchRemoveTrivialProjectRule
drops child node's traitsPushFilterPastProjectRule
should not push filters past windowed aggregatesFoodMartQuerySet
unless we have to. It’s big.apache-
prefix to tarball and directory within tarballRelMdPredicates
) doesn't handle semi-joingit-commit-id-plugin
error when running in Travis-CI.<plugins>
tag in root pomgit.properties
file describing its exact versionmvn site
on JDK 1.8 and enable in Travis-CI.{: #v0-9-0}
This is the first release under the Apache incubator process.
{: #new-features-0-9-0}
JOIN
whose ON
clause contains mixed equi and thetaEnumerableSemiJoinRel
, implementation of semi-join in enumerable conventionStrong
, for detecting null-rejecting predicates.PushFilterPastJoinRule
should strengthen join typeEquiJoinRel
, base class for joins known to be equi-joins.CAST(<string> AS <datetime>)
and <datetime> + <interval>
.SqlRun
, add !plan
commandSqlRun
, match output regardless of order if ORDER BY
not presentCOUNT(DISTINCT x, y, ...)
{: #api-0-9-0}
Hook
to use Guava Function
(was linq4j Function1
).withHook
, to more easily add hooks in tests.CompoundIdentifier
in SqlParser
.SqlExplain
, SqlOrderBy
and SqlWith
to support SqlShuttle
use.SqlUnresolvedFunction.inferReturnType()
to return ANY
type so framework implementors can support late bound function implementations.ViewExpander
for Planner
in Frameworks
. Expose additional properties of ViewTable
to allow subclassing.Context
and FrameworkConfig
{: #fixes-0-9-0}
LoptOptimizeJoinRule
& PushFilterPast
* rules to use factorymaven-checkstyle-plugin
.Holder
, a mutable slot that can contain one object.CURRENT_TIME
, etc.ImmutableIntList
's iterators.WITH ... ORDER BY
querydependencyManagement
and pluginManagement
in POM filesFilterFactory
.README
file, incubation disclaimers, and how-to build and running tests.KEYS
and start how-to for making snapshots and releases.apache-rat
and git-commit-id-plugin
in “release” maven profile.BitSet.previousClearBit
.RexBuilder.makeCall
, if possible.RexProgramBuilder.addCondition
, not RexBuilder.makeCall
as previously.AssertionError
in RexToLixTranslator
comparing to date literalPushFilterPastJoinRule
to RelDecorrelator
; interim pending [CALCITE-343] fix.SqlToRelConverter
when push expressions in join conditions into ProjectRel
.RelNode
tree with a materialized view expression, switch representation to MutableRel
s.SqlTypeExplicitPrecedenceList
.RelOptUtil
ANY
type check.ANY
typeSqlRun
, allow !plan
after !ok
for same SQL statement.SqlRun
SELECT DISTINCT
would return duplicate rows.{: #v0-8}
{: #new-features-0-8}
COUNT(not-null-expression)
to COUNT()
SqlRun
, an idempotent utility for running SQL test scripts{: #api-0-8}
Program
interface, a planner phase more general than current RuleSet
IS_NOT_TRUE
and IS_NOT_FALSE
SqlKind
enums.SqlKind.IS_NOT_NULL
enum value, and use where possible, including for IS_NOT_UNKNOWN
operator.{: #fixes-0-8}
WindowRel
WITH ... ORDER BY
WITH ... ORDER BY
query gives AssertionError
MultiJoinRel
and some other planner rule tests.ImmutableNullableList
and UnmodifiableArrayList
, and remove calls to Arrays.asList
.IntPair.zip
.NOT IN
where sub-query returns NULL values.CURRENT ROW
ORDER BY
EXTRACT
function's SQL return type is BIGINT
but implemented as Java int
{: #v0-7}
{: #new-features-0-7}
JOIN LATERAL
and JOIN UNNEST
.UNNEST
relational operator, and various improvements to ARRAY
and MULTISET
data types.ARRAY
columns as Java lists.CARDINALITY(ARRAY)
SQL operator.SELECT
clause.SUM(1) OVER (ROWS 10 PRECEDING)
;UNBOUNDED PRECEDING
window range;GROUP BY
clause is empty, SUM
may return null.AvaticaConnection
; add ExceptionMessageTest
.{: #api-0-7}
OptiqSchema
wrappers. (Tobi Vollebregt and Julian Hyde){: #fixes-0-7}
SqlFunctions
as @Deterministic
;static final
constants generated by linq4j.Object[]
storage.fromTrait
of the just previously converted RelNode
instead of the original RelNode
.SplunkAdapterTest
if Splunk is not available.SplunkAdapterTest
now uses the same Foodmart database as JdbcTest
.{: #v0-6}
{: #new-features-0-6}
name
field of ReflectiveSchema
.EXTRACT
function to support DATE
, TIME
and TIMESTAMP
valuessqlline.bat
and fix issues running sqlline
under Cygwin.{: #api-0-6}
RexImpTable.AggregateImplementor
and rename AggImplementor2
. (This is a breaking change.)CombinedParser.jj
into freemarker template to allow custom parser implementations. (Venki Korukanti)Planner
to pass a custom ConvertletTable
and custom SQL parser.Frameworks
, add a way to specify list of TraitDef
s that will be used by planner. (Jinfeng Ni)RexExecutor
to evaluate projections and filtersALTER scope SET option = value
statementSqlLiteral.booleanValue(SqlNode)
, SqlLiteral.symbolValue(SqlNode)
.RelOptPlanner.clear()
; now it is safe to call transform
twice. (Jinfeng Ni){: #fixes-0-6}
FETCH
and LIMIT
are ignored during SQL-to-RelNode translation. (Venki Korukanti)RelOptTableImpl.create
always expects QueryableTable
type in OptiqSchema.TableEntry
Pair.adjacents(Iterable)
.ANY
type conditions aren't correctly being considered (Jacques Nadeau).gitignore
that shouldn't be checked in when using Eclipse. (Jacques Nadeau)ControlFlowException
, and make it base class of existing control-flow exception classes.SqlUnresolvedFunction
, to improve how return type of user-defined functions is resolved. (Vladimir Sitnikov)ORDER BY
and LIMIT
.NOTICE
and LICENSE
files in all generated JAR filesNL
in tests to Linux newline “\n”. This makes string constants simpler.Lex.MYSQL
SELECT
clause is considered an aggregate query, even if there is no GROUP BY
.XmlOutput
(based on org.eigenbase.xom.XMLOutput
) and remove dependency on eigenbase-xom.RexProgramBuilder
by using Pair
rather than String
as expression key. (Vladimir Sitnikov)SpaceList
.RelTrait.subsumes
rather than equals
in an assert. (Jinfeng Ni){: #v0-5}
{: #new-features-0-5}
quoting
, quotedCasing
, unquotedCasing
, and caseSensitive
properties to be specified explicitly (Vladimir Sitnikov)WITH
queries (common table expressions)WHEN
clause in simple CASE
expression to have multiple values{: #api-0-5}
ProjectRelBase.copy(RelTraitSet, RelNode, List<RexNode>, RelDataType)
and make ProjectRelBase.copy(RelTraitSet, RelNode)
final. (This is a breaking change for sub-classes of ProjectRelBase
.)RexBuilder.makeRangeReference
parameter type.RexBuilder.makeInputRef
replaces RelOptUtil.createInputRef
.SqlCall
that keeps operands in fields, not an operands arraySqlSelect
, SqlJoin
, SqlInsert
, and a new node type SqlOrderBy
, are now sub-classes of SqlCall
but not SqlBasicCall
.SqlSelect
has an operands
field.){: #fixes-0-5}
Casing.UNCHANGED
DATE.getLimit
now returns Calendar
in GMT time zone (Vladimir Sitnikov)en_US
locale in tests that match against error numbers, dates (Vladimir Sitnikov)RexExecutorImpl
.src/test/java
to src/test/resources
(reduces the number of ‘untracked files’ reported by git)Util.SpaceList
, fix race condition, and move into new utility class Spaces
.SQLFeatureNotSupported
when calling setQueryTimeout
(Karel Vervaeke)-Dcalcite.test.mongodb=true
.spark=true
is specified in the connect string.mongo-zips-model.json
in HOWTO. (Mariano Luna)ONE_MINUS_EPSILON
.Planner.transform()
multiple times, each with different rulesetRelTraitSet
creation and amortize RelTraitSet.toString()
.maven-source-plugin
.{: #v0-4-18}
{: #api-0-4-18}
ProjectFactory
for creating projects. (John Pullokkaran)Cursor
& Accessor
implementations to throw SQLException
. (Jacques Nadeau)RelOptCostFactory
) from planner. Allow VolcanoPlanner
to be sub-classed with different cost factory.RelOptCost
. (Harish Butani)VolcanoCost
immutable.SqlTypeStrategies
into OperandTypes
, ReturnTypes
and InferTypes
, and rename its static members to upper-case, per checkstyle. (This is a breaking change.)AvaticaDatabaseMetaData
methods.LIMIT 0
is present. Also remove sort, aggregation, join if their inputs are known to be empty, and propagate the fact that the relational expressions are known to be empty up the tree. (We already do this for union, filter, project.)RexNode
and its sub-classes are now immutable.{: #fixes-0-4-18}
Util.startsWith(List, List)
and Util.hashCode(double)
.SqlValidator
to rely on RelDataType
to do field name matching. Fix RelDataTypeImpl
to correctly use the case sensitive flag rather than ignoring it.us-ascii
.optiq-mat-plugin
to README.{: #v0-4-17}
{: #fixes-0-4-17}
Schema
and Table
SPIs simpler to implement, and make them re-usable across connections (This is a breaking change.)RelOptRuleOperandChildren
contains the children of an operand and the policy for dealing with them. Existing rules now use the new methods to construct operands: operand()
, leaf()
, any()
, none()
, unordered()
. The previous methods are now deprecated and will be removed before 0.4.18. (This is a breaking change.)Util
: add methods skipLast
, last
, skip
; remove subList
, butLast
.SqlIdentifier.names
from String[]
to ImmutableList<String>
.OptiqAssert.assertThat()
to that()
, to avoid clash with junit's Assert.assertThat()
.RelDataTypeFactory.FieldInfoBuilder
. It now has a type-factory, so you can just call build()
.HepProgramBuilder
into a fluent API.{: #tuning-0-4-17}
EnumerableAggregateRel
has no aggregate functions, generate a call to Enumerable.distinct()
, thereby saving the effort of building trivial accumulators.CalcRel
until a later phase of planning. This reduces the number of trivial projects and calcs created, merged, and elimated.Util.isDistinct
for small lists.Unit
class, which is a singleton.EnumerableJoinRel
so that join is cheaper if the larger input is on the left, and more expensive if it is a cartesian product.LinkedList
with ChunkList
, which has an O(1) remove and add, a fast O(n) get, and fast scan.{: #other-0-4-17}
VALUES
operator.RelNode
s have identical digest but different row-type.RemoveTrivialProjectRule.strip
, JdbcProjectRel.implement
and SortRel.computeSelfCost
.@author
tags.VolcanoCost
.FoodMartQuerySet
, that contains the 6,700 foodmart queries.UnregisteredDriver
CloneSchema
to be specified in terms of url, driver, user, password; not just dataSource.SQLException
.{: #v0-4-16}
{: #v0-4-15}
Optiq-csv-0.3 was the last independent release of optiq-csv. From calcite-0.9.2 onwards, the code was included as the calcite-example-csv module.
{: #csv-v0-3}
sqlline.bat
(windows) and sqlline
(windows using cygwin)sqlline.bat
copies dependencies to target/dependencies
; sqlline
constructs target/classpath.txt
{: #csv-v0-2}
sqlline.bat
, Windows SQL shell (based on fix for [DRILL-338]){: #csv-v0-1}
CsvTableFactory
, an example of a custom tableCsvTable
) that just execute and a “smart” schema that generates tables (CsvSmartTable
) that undergo optimizationCsvEnumerator
a top-level classLinq4j-0.4 was the last independent release of linq4j. From calcite-0.9.2 onwards, the code was included as calcite-linq4j, and features added to linq4j in a particular calcite release are described with the other changes in that release.
{: #linq4j-v0-4}
Enumerable
methods: any
, all
, contains
with EqualityComparer
, first
, first
with predicate.{: #linq4j-v0-3}
ExpressionType.modifiesLvalue
to avoid invalid inlining.@Deterministic
annotation to help deduce which expressions are constant.Boolean.valueOf(const)
, ‘not’ expressions (!const
, !!a
, !(a==b)
, !(a!=b)
, !(a>b)
, etc.), ‘?’ expressions coming from CASE
(a ? booleanConstant : b
and a ? b : booleanConstant
).{: #linq4j-v0-2}
Expressions.ifThenElse
does not work”.HashMap
for searching of declarations to reuse; consider both optimizing
and optimize
flags when reusing.equals
and hashCode
for expressions. Hash codes for complex expressions are cached into a field of the expression.com.example.Linq4jExample
.Statement.toString
via ExpressionWriter
.AtomicInteger
for naming of ParameterExpression
s to avoid conflicts in multithreaded usageFunctions.adapt
rather than new AbstractList
NOTICE
and LICENSE
files in generated JAR file.select()
if selector is identity.{: #linq4j-v0-1-13}
Primitive.defaultValue
.oraclejdk8
in Travis CI.{: #linq4j-v0-1-12}
Enumerable.asEnumerable
in DefaultQueryable
(inherited by most classes that implement Queryable
).{: #linq4j-v0-1-11}