[HOTFIX] Fix After alter table add column schemaOrdinal is not continuous
Why is this PR needed?
Consider a scenario with complex columns[c1 int, c2 array, c3 array] here schema ordinal of child columns are -1. so c1, c2, c3 has ordinal 0, 1, 2. After alter table add column "c4 int", its ordinal should be 3. But currently it is 6 (due to flat column schema).
This will give problem when this ordinal is referred for some other functionality like rearrange projection columns.
What changes were proposed in this PR?
make schema ordinal continuous by collecting previous schema ordinal max.
Does this PR introduce any user interface change?
No
Is any new testcase added?
No
This closes #3577
diff --git a/integration/spark-common/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala b/integration/spark-common/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
index 523ec9e..a7e6f63 100644
--- a/integration/spark-common/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
+++ b/integration/spark-common/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
@@ -240,7 +240,8 @@
def process: Seq[ColumnSchema] = {
val tableSchema = tableInfo.getFactTable
val tableCols = tableSchema.getListOfColumns.asScala
- val existingColsSize = tableCols.size
+ // previous maximum column schema ordinal + 1 is the current column schema ordinal
+ val currentSchemaOrdinal = tableCols.map(col => col.getSchemaOrdinal).max + 1
var longStringCols = Seq[ColumnSchema]()
// get all original dimension columns
// but exclude complex type columns and long string columns
@@ -263,7 +264,7 @@
isDimensionCol = true,
field.precision,
field.scale,
- field.schemaOrdinal + existingColsSize,
+ field.schemaOrdinal + currentSchemaOrdinal,
alterTableModel.highCardinalityDims,
alterTableModel.databaseName.getOrElse(dbName),
isSortColumn(field.name.getOrElse(field.column)),
@@ -296,7 +297,7 @@
isDimensionCol = false,
field.precision,
field.scale,
- field.schemaOrdinal + existingColsSize,
+ field.schemaOrdinal + currentSchemaOrdinal,
alterTableModel.highCardinalityDims,
alterTableModel.databaseName.getOrElse(dbName)
)
