tree b968005f6b8c8586c288d3481102b1f29c73d10f
parent 9e8c4aa3f43a3d99bff56cca319db623abc473ee
author Milan Dankovic <milan.dankovic@databricks.com> 1714468741 +0800
committer Wenchen Fan <wenchen@databricks.com> 1714468741 +0800

[SPARK-47566][SQL] Support SubstringIndex function to work with collated strings

### What changes were proposed in this pull request?
Extend built-in string functions to support non-binary, non-lowercase collation for: substring_index.

### Why are the changes needed?
Update collation support for built-in string functions in Spark.

### Does this PR introduce _any_ user-facing change?
Yes, users should now be able to use COLLATE within arguments for built-in string function SUBSTRING_INDEX in Spark SQL queries, using non-binary collations such as UNICODE_CI.

### How was this patch tested?
Unit tests for queries using SubstringIndex (`CollationStringExpressionsSuite.scala`).

### Was this patch authored or co-authored using generative AI tooling?
No

### To consider:
There is no check for collation match between string and delimiter, it will be introduced with Implicit Casting.

We can remove the original `public UTF8String subStringIndex(UTF8String delim, int count)` method, and get the existing behavior using `subStringIndex(delim, count, 0)`.

Closes #45725 from miland-db/miland-db/substringIndex-stringLocate.

Authored-by: Milan Dankovic <milan.dankovic@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>