prestoadmin.util package

Submodules

prestoadmin.util.all_write_handler module

class prestoadmin.util.all_write_handler.AllWriteTimedRotatingFileHandler(filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False)

Bases: logging.handlers.TimedRotatingFileHandler

prestoadmin.util.application module

Logic at the application level for logging and exception handling

class prestoadmin.util.application.Application(name, version=None, log_file_path=None)

Bases: object

A generic application entry point. Provides logging and exception handling features. This class is expected to be used as a base class for various applications.

Parameters:

• - human readable name for the application (name) –
• - the version of the application, as a string (version) –
• - optional name of the log file including whatever (log_file_path) –
• you may want to use. For example, 'foo.log' would create (extension) –
• file called 'foo.log' in the default presto-admin logging directory (a) –
• tree. –

prestoadmin.util.application.entry_point(name, version=None, log_file_path=None, application_class=<class 'prestoadmin.util.application.Application'>)

A decorator for application entry points. The decorated function will be wrapped in an Application object and executed in that safe environment. Note that decorating a function with this decorator will not actually cause it to be invoked. You must explicitly call the function in the script.

Parameters:

• - human readable name for the application (name) –
• - the version of the application, as a string (version) –
• - optional name of the log file including whatever (log_file_path) –
• you may want to use. For example, 'foo.log' would create (extension) –
• file called 'foo.log' in the default prestoadmin logging directory (a) –
• tree. –
• - Type of application to run. The default is (application_class) –
• but there can be subclasses of that class. (Application) –

prestoadmin.util.base_config module

Module for common configuration stuff.

class prestoadmin.util.base_config.BaseConfig(config_path, config_items)

Bases: object

BaseConfig provides the common config functionality for loading configuration files for presto-admin and going through the interactive config process if a config file isn’t present.

Instances of classes that subclass BaseConfig are intended to be used with the @requires_config decorator, which is responsible for adding an attribute to the task that tells main() how to load the configuration and subsequently for enforcing that the configuration has been loaded at the time the task is actually run.

In order to be compatible with @requires_config, subclasses must define a no-arguments constructor.

get_conf_interactive()

get_config()

is_config_loaded()

read_conf()

set_config_loaded()

set_env_from_conf(conf)

write_conf(conf)

class prestoadmin.util.base_config.MultiConfigItem(items, validate, validate_keys, validate_failed_text)

Bases: object

collect_prompts(l)

prompt_user(conf)

class prestoadmin.util.base_config.SingleConfigItem(key, prompt, default=None, validate=None)

Bases: object

collect_prompts(l)

prompt_user(conf)

prestoadmin.util.base_config.requires_config(config_class)

prestoadmin.util.constants module

This modules contains read-only constants used throughout the presto admin project.

prestoadmin.util.exception module

This module defines error types relevant to the Presto administrative suite.

exception prestoadmin.util.exception.ConfigFileNotFoundError(message='', config_path='')

Bases: prestoadmin.util.exception.ConfigurationError

exception prestoadmin.util.exception.ConfigurationError(message='')

Bases: prestoadmin.util.exception.ExceptionWithCause

exception prestoadmin.util.exception.ExceptionWithCause(message='')

Bases: exceptions.Exception

static get_cause_if_supported(exception)

exception prestoadmin.util.exception.InvalidArgumentError(message='')

Bases: prestoadmin.util.exception.ExceptionWithCause

prestoadmin.util.exception.is_arguments_error(exception)

prestoadmin.util.fabric_application module

Logic for starting and stopping Fabric applications.

class prestoadmin.util.fabric_application.FabricApplication(name, version=None, log_file_path=None)

Bases: prestoadmin.util.application.Application

A Presto Fabric application entry point. Provides logging and exception handling features. Additionally cleans up Fabric network connections before exiting.

prestoadmin.util.fabricapi module

Module to add extensions and helpers for fabric api methods

prestoadmin.util.fabricapi.by_role_coordinator(host, f, *args, **kwargs)

prestoadmin.util.fabricapi.by_role_worker(host, f, *args, **kwargs)

prestoadmin.util.fabricapi.by_rolename(host, rolename, f, *args, **kwargs)

prestoadmin.util.fabricapi.get_coordinator_role()

prestoadmin.util.fabricapi.get_host_list()

prestoadmin.util.fabricapi.get_worker_role()

prestoadmin.util.fabricapi.put_secure(user_group, mode, *args, **kwargs)

prestoadmin.util.fabricapi.task_by_rolename(rolename)

prestoadmin.util.filesystem module

Filesystem tools.

prestoadmin.util.filesystem.ensure_directory_exists(path)

prestoadmin.util.filesystem.ensure_parent_directories_exist(path)

prestoadmin.util.filesystem.write_to_file_if_not_exists(content, path)

prestoadmin.util.hiddenoptgroup module

An option group for which you can hide the help text.

class prestoadmin.util.hiddenoptgroup.HiddenOptionGroup(parser, title, description=None, suppress_help=False)

Bases: optparse.OptionGroup

Optparse allows you to suppress Options from the help text, but not groups. This class allows you to suppress the help of groups.

format_help(formatter)

prestoadmin.util.httpscacertconnection module

class prestoadmin.util.httpscacertconnection.HTTPSCaCertConnection(host, port, key_file, cert_file, ca_file, strict, timeout=None)

Bases: httplib.HTTPSConnection

Class to make a HTTPS connection, with support for full client-based SSL Authentication

connect()

Connect to a host on a given (SSL) port. If ca_file is pointing somewhere, use it to check Server Certificate.

Redefined/copied and extended from httplib.py:1105 (Python 2.6.x). This is needed to pass cert_reqs=ssl.CERT_REQUIRED as parameter to ssl.wrap_socket(), which forces SSL to check server certificate against our client certificate.

prestoadmin.util.local_config_util module

prestoadmin.util.local_config_util.get_catalog_directory()

prestoadmin.util.local_config_util.get_config_directory()

prestoadmin.util.local_config_util.get_coordinator_directory()

prestoadmin.util.local_config_util.get_log_directory()

prestoadmin.util.local_config_util.get_topology_path()

prestoadmin.util.local_config_util.get_workers_directory()

prestoadmin.util.parser module

An extension to optparse for presto-admin which logs user parsing errors.

class prestoadmin.util.parser.LoggingOptionParser(usage=None, option_list=None, option_class=<class optparse.Option>, version=None, conflict_handler='error', description=None, formatter=None, add_help_option=True, prog=None, epilog=None)

Bases: optparse.OptionParser

An extension to optparse which logs exceptions via the logging module in addition to writing the out to stderr.

If used with HiddenOptionGroup, print_extended_help disables the suppress_help attribute of HiddenOptionGroup so as to print out extended helptext.

exit(status=0, msg=None)

format_epilog(formatter)

The default format_epilog strips the newlines (using textwrap), so we override format_epilog here to use its own epilog

print_extended_help(filename=None)

prestoadmin.util.presto_config module

class prestoadmin.util.presto_config.PrestoConfig(config_properties, config_path, config_host)

static coordinator_config()

default_config = {'internal-communication.authentication.ldap.user': None, 'http-server.http.enabled': 'true', 'http-server.https.enabled': 'false', 'http-server.https.port': '8443', 'internal-communication.authentication.ldap.password': None, 'http-server.http.port': '8080', 'internal-communication.https.keystore.path': None, 'internal-communication.https.keystore.key': None}

static from_file(config, config_path=None, config_host=None)

get_client_keystore_password()

get_client_keystore_path()

get_http_port()

get_https_port()

get_ldap_password()

get_ldap_user()

use_https()

use_ldap()

prestoadmin.util.remote_config_util module

prestoadmin.util.remote_config_util.lookup_catalog_directory(host)

prestoadmin.util.remote_config_util.lookup_in_config(config_key, config_file, host)

prestoadmin.util.remote_config_util.lookup_launcher_log_file(host)

prestoadmin.util.remote_config_util.lookup_port(host)

Get the http port from config.properties http-server.http.port property if available. If the property is missing return default port 8080. If the file is missing or cannot parse the port number, throw ConfigurationError :param host: :return:

prestoadmin.util.remote_config_util.lookup_server_log_file(host)

prestoadmin.util.remote_config_util.lookup_string_config(config_value, config_file, host, default='')

prestoadmin.util.validators module

Module for validating configuration information supplied by the user.

prestoadmin.util.validators.is_valid_hostname(hostname)

prestoadmin.util.validators.validate_can_connect(user, host, port)

prestoadmin.util.validators.validate_can_sudo(sudo_user, conn_user, host, port)

prestoadmin.util.validators.validate_host(host)

prestoadmin.util.validators.validate_port(port)

prestoadmin.util.validators.validate_username(username)

prestoadmin.util.version_util module

Stuff to handle version ranges.

class prestoadmin.util.version_util.VersionRange(min_version, max_version, versioned_thing=None)

Bases: object

Represents a range of version numbers [min_version, max_version). The interval is right-open so that you can construct a numerically continuous list of versions like so: l = [VersionRange((0, 0), (0, 5)), VersionRange((0, 5), (1, 0))] and for all versions v where 0.0 <= v < 1.0 is contained in exactly one VersionRange in l.

Continuity between version ranges can be checked using is_continuous.

VersionRanges understand how to check if a Teradata version is contained in a VersionRange, but do no special handling to accomodate Teradata versions in their internal min_version and max_version members. I.e., creating a VersionRange with a Teradata version will work, but __contains__ will not work correctly. We don’t currently need this, and hope not to.

Note that the right-open interval representation of a version range does NOT allow the creation of a VersionRange that contains exactly one version.

Note that empty intervals cannot be constructed as the serve no useful purpose. Specifically, we assert that min_version < max_version in the constructor.

is_continuous(next)

static pad_tuple(tup, length, pad)

static strip_td_suffix(version)

zero_pad(other)

Pad out min_version, max_version, and other with zeroes to the length of the longest of the three. This allows subsequent comparisons to work as expected when tuples are of unequal length. Returns a tuple of tuples padded out to the same length

class prestoadmin.util.version_util.VersionRangeList(*range_list)

Bases: object

A VersionRangeList is a list of continuous, non-overlapping VersionRanges. This is guaranteed by calling VersionRange.is_continuous on all pairs of VersionRanges vr[i], vr[i + 1] in the list, which ensures that the list is both sorted in order of ascending version and that the interval [vr[0].min_version, vr[n].max_version) has no discontinuities.

for_version(version)

prestoadmin.util.version_util.get_int_or_t(x)

prestoadmin.util.version_util.is_int_or_t(x)

prestoadmin.util.version_util.split_version(version_string)

prestoadmin.util.version_util.strip_tag(version)

Strip any parts of the version that are not numeric components or t’s We leave the ‘t’ on numeric components if it’s present. [‘1’, ‘2’, ‘THREE’] -> (1, 2) [‘1’, ‘TWO’, ‘3’] -> (1, 3) [‘0’, ‘115t’, ‘SNAPSHOT’] -> (0, ‘115t’) [‘ZERO’, ‘123t’] -> (123t) [‘0’, ‘148’, ‘t’] => (0, 148, ‘t’) [‘0’, ‘148’, ‘t’, 0, 1] => (0, 148, ‘t’, 0, 1) [‘0’, ‘148’, ‘t’, 0, 1, ‘SNAPSHOT’] => (0, 148, ‘t’, 0, 1) [‘0’, ‘162’, ‘SNAPSHOT’, ‘t’, ‘SNAPSHOT’] => (0, 162, ‘t’)

This checks the components of the version from least to most significant.

Parameters:version – something that can be sliced Returns:a tuple containing only integer components or the letter t

Module contents