Redis HBO Provider¶
Redis HBO Provider supports loading a custom configured Redis Client for storing and retrieving historical stats for Historical Based Optimization (HBO). The Redis client is stateful and is based on Lettuce. Both RedisClient and RedisClusterClient are supported, RedisClusterAsyncCommandsFactory is meant to be extended by the user for custom configurations.
Configuration¶
Create etc/catalog/redis-provider.properties
to mount the Redis HBO Provider Plugin.
Edit the configuration properties as appropriate:
Configuration properties¶
The following configuration properties are available for use in etc/catalog/redis-provider.properties
:
Property Name |
Description |
---|---|
|
Boolean property whether Presto server is a coordinator |
|
Redis Server URI |
|
Maximum timeout in ms for Redis fetch requests |
|
Maximum timeout in ms for Redis set requests |
|
TTL in seconds of the Redis data to be stored |
|
Boolean property whether this plugin is enabled in production |
|
Path for Redis credentials |
|
Boolean property whether cluster mode is enabled |
Coordinator Configuration for Historical Based Optimization¶
These properties must be configured on the Presto coordinator in etc/config.properties
for tracking and using historical statistics in planning:
Property Name |
Description |
Default Value |
---|---|---|
|
Boolean property to enable the use of historical plan statistics |
false |
|
Boolean property to enable tracking of historical plan statistics |
false |
|
Integer use history based optimizations only when number of nodes in canonical plan is within this limit |
1000 |
|
Duration end to end timeout for optimizer in plan hashing and gathering statistics |
10 (seconds) |
Credentials¶
The plugin requires the Redis Server URI property hbo.redis-provider.server_uri
to access Redis.
Based on your custom Redis deployment, you may need to add additional credentials.
Local Test Setup¶
Set up a local Redis cluster following the Redis documentation to Create a Redis Cluster.
In
presto-main/etc/config.properties
, add../redis-hbo-provider/pom.xml,\
toplugin.bundles
.In
presto-main/etc/
, create the fileredis-provider.properties
with these sample properties:coordinator=true hbo.redis-provider.enabled=true hbo.redis-provider.total-fetch-timeoutms=5000 hbo.redis-provider.total-set-timeoutms=5000 hbo.redis-provider.default-ttl-seconds=4320000 hbo.redis-provider.cluster-mode-enabled=true hbo.redis-provider.server_uri=redis://localhost:7001/
Production Setup¶
You can place the plugin JARs in the production’s plugins
directory.
Alternatively, follow this method to ensure that the plugin is loaded during the Presto build.
Add the following to register the plugin in
<fileSets>
inpresto-server/src/main/assembly/presto.xml
:<fileSet> <directory>${project.build.directory}/dependency/redis-hbo-provider-${project.version}</directory> <outputDirectory>plugin/redis-hbo-provider</outputDirectory> </fileSet>
In
redis-hbo-provider/src/main/resources
, create the fileMETA-INF.services
with the Plugin entry classcom.facebook.presto.statistic.RedisProviderPlugin
.Add the dependency on the module in
presto-server/pom.xml
:<dependency> <groupId>com.facebook.presto</groupId> <artifactId>redis-hbo-provider</artifactId> <version>${project.version}</version> <type>zip</type> <scope>provided</scope> </dependency>
(Optional) Add your custom Redis client connection login in
com.facebook.presto.statistic.RedisClusterAsyncCommandsFactory
.Note: The AsyncCommands must be provided properly.