Release 0.221¶
General Changes¶
Fix error during stats collection phase of query planning.
Fix a performance regression for some outer joins without equality predicates when
join_distribution_type
is set toAUTOMATIC
.Improve performance for queries that have constant
VARCHAR
predicates on join columns.Add a variant of
strpos()
that returns the position of the N-th instance of the substring.Add
strrpos()
that returns the position of the N-th instance of a substring from the back of a string.Add aggregation function
entropy()
.Add classification aggregation functions
classification_miss_rate()
,classification_precision()
,classification_recall()
,classification_thresholds()
.Add overload of
approx_set()
which takes in the maximum standard error.Add
max_tasks_per_stage
session property andstage.max-tasks-per-stage
config property to limit the number of tasks per stage for grouped execution. Setting this session property allows queries running with grouped execution to use a predictable amount of memory independent of the cluster size.Add encryption for spill files (see Spill to Disk).
Web UI Changes¶
Add information about query warnings to the web UI.
Raptor Changes¶
Revert the change introduced in 0.219 to rebalance bucket assignment after restarting the cluster. Automatic rebalancing can cause unexpected downtime when restarting the cluster to resolve emergent issues.
Hive Connector Changes¶
Improve coordinator memory utilization for Hive splits.
Improve performance of writing large ORC files.
SPI Changes¶
Add
pageSinkContext
forcreatePageSink
inPageSinkProvider
andConnectorPageSinkProvider
. It contains a booleanpartitionCommitRequired
, which is false by default. See the note below aboutcommitPartition
for more information.Add
commitPartition
toMetadata
andConnectorMetadata
. This SPI is coupled withpageSinkContext#partitionCommitRequired
and is used by the engine to commit a partition of data to the target connector. The connector that implements this SPI should ensure that ifpageSinkContext#isPartitionCommitRequired
is true inConnectorPageSinkProvider#createPageSink
, the written data is not published untilConnectorMetadata#commitPartition
is called. Also, it is expected for the connector to addSUPPORTS_PARTITION_COMMIT
inConnector#getCapabilities
.Add
ExpressionOptimizer
inRowExpressionService
.ExpressionOptimizer
simplifies aRowExpression
and prunes redundant part of it.Add
pushNegationToLeaves
method toLogicalRowExpressions
to push negation down below conjunction or disjunction for a logical expression.Replace
SplitSchedulingStrategy
withSplitSchedulingContext
inConnectorSplitManager
.SplitSchedulingContext
contains theSplitSchedulingStrategy
and a booleanschedulerUsesHostAddresses
that indicates whether the network topology is used during scheduling. If false, the connector doesn’t need to provide the host addresses for remotely accessible splits.