Release 0.285¶
Warning
There is a bug for LIKE pattern with multiple-byte characters. (#21578)
Details¶
General Changes¶
Fix min_by(x, y, n) / max_by(x, y, n) functions to ensure n is unique.
Fix LIKE pattern to handle multi-byte characters. #21578
Improve HBO to track partial aggregation statistics. #21160
Improve PushPartialAggregationThroughExchange to use partial aggregation statistics when available. This extends to multi-key aggregations.
Improve performance of union when subqueries are empty.
Improve cost based optimizer join reordering to work with non-simple equi-join predicates. #21153
Improve history based optimizer performance when nulls are present in joins. #21203
Add
array_least_frequent()
function.Add support for HANA connector. #21034
Add task killer which is triggered when a worker is running out of memory and the garbage collector cannot reclaim sufficient memory. Two strategies are provided: full garbage collection, and frequent full garbage collection. #21254
Add support to remove redundant cast to varchar expressions in a join condition. This feature is configurable by the session property
remove_redundant_cast_to_varchar_in_join
(enabled by default). #21050Add support to use HBO for scaled writers. This feature is configurable by the session property
enable_hbo_for_scaled_writer
(disabled by default).Add support to split aggregates into partial and final based on partial aggregation statistics. This feature is configurable by the session property
use_partial_aggregation_history
(disabled by default). #21160Add optimization where values node followed by an always false filter is converted to an empty values node.
Add information about cost-based optimizers and the source of stats they use (CBO/HBO) in explain plans when session property
verbose_optimizer_info_enabled=true
. #20990Upgrade Joda-Time to 2.12.5 to use 2023c tzdata. The JVM should also be updated to ensure the timezone data is consistent. #21329
Upgrade Alluxio to 304 from 2.8.1.
Upgrade AWS SDK to 1.12.560 from 1.12.261.
Upgrade Avro version to 1.11.3 from 1.9.2.
Prestissimo (Native Execution) Changes¶
Fix task cleanup to use task termination time instead of task end time.
Add support for JWT authentication. #20290
Add session property
native_debug.validate_output_from_operators
to identify malformed output from operators. #21036
Security Changes¶
Fix critical vulnerability in Babel and related npm packages by updating to newer versions. #21322
Hive Connector Changes¶
Improve support for schema evolution of partition column types for Parquet file format. #19983
Add support for Parquet writer versions V1 and V2. See Hive Connector.
Iceberg Connector Changes¶
Add support for reading and writing distinct value count statistics as described by Iceberg’s Puffin file specification. #20993
Add support for
ANALYZE
when configured with the Hive Catalog. The table must be un-partitioned. #20720Add support for
DELETE FROM <table> [where <filter>]
. Deletes one or more partitions. #21048Add support for
ALTER TABLE <table> ADD COLUMN <column> [WITH (partitioning = '<transform_func>')]
. #21206Add support for creating tables using the AWS Glue metastore. #20699
Add support for
SHOW STATS
for tables withtimestamp
type. #21286Add support for views. See Iceberg Connector.
Add support for Parquet writer versions V1 and V2.
Remove
timestamp with time zone
type in create table and alter table statements. #21096Upgrade Iceberg version from 1.3.1 to 1.4.1.
Pinot Connector Changes¶
Fix push down of literal expression with single quotes. #21020
Credits¶
Ajay George, Ajay Gupte, Amit Dutta, Anant Aneja, Andrii Rosa, Arjun Gupta, Avinash Jain, Beinan, Bikramjeet Vig, Chandrashekhar Kumar Singh, Christian Zentgraf, Chunxu Tang, Deepak Majeti, Eduard Tudenhoefner, James Xu, Jialiang Tan, JiamingMai, Jimmy Lu, Jonathan Hehir, Karteekmurthys, Ke, Kevin Wilfong, Krishna Pai, Lyublena Antova, Mahadevuni Naveen Kumar, Masha Basmanova, Michael Shang, Miguel Blanco Godón, Nikhil Collooru, Pedro Pedreira, Pranjal Shankhdhar, Pratyush Verma, Ruslan Mardugalliamov, Sergey Pershin, Sergii Druzkin, Shrinidhi Joshi, Sotirios Delimanolis, Sreeni Viswanadha, Steve Burnett, Sudheesh, Swapnil Tailor, Tim Meehan, Xiang Fu, Yihong Wang, Zac Blanco, aditi-pandit, feilong-liu, kedia,Akanksha, kiersten-stokes, mmorgan98, pratyakshsharma, wangd, wypb, xiaoxmeng, yingsu00