tf.contrib.training.HParams

Class to hold a set of hyperparameters as name-value pairs.

tf.contrib.training.HParams(
    hparam_def=None, model_structure=None, **kwargs
)

A HParams object holds hyperparameters used to build and train a model, such as the number of hidden units in a neural net layer or the learning rate to use when training.

You first create a HParams object by specifying the names and values of the hyperparameters.

To make them easily accessible the parameter names are added as direct attributes of the class. A typical usage is as follows:

# Create a HParams object specifying names and values of the model
# hyperparameters:
hparams = HParams(learning_rate=0.1, num_hidden_units=100)

# The hyperparameter are available as attributes of the HParams object:
hparams.learning_rate ==> 0.1
hparams.num_hidden_units ==> 100

Hyperparameters have type, which is inferred from the type of their value passed at construction type. The currently supported types are: integer, float, boolean, string, and list of integer, float, boolean, or string.

You can override hyperparameter values by calling the parse() method, passing a string of comma separated name=value pairs. This is intended to make it possible to override any hyperparameter values from a single command-line flag to which the user passes 'hyper-param=value' pairs. It avoids having to define one flag for each hyperparameter.

The syntax expected for each value depends on the type of the parameter. See parse() for a description of the syntax.

Example:

# Define a command line flag to pass name=value pairs.
# For example using argparse:
import argparse
parser = argparse.ArgumentParser(description='Train my model.')
parser.add_argument('--hparams', type=str,
                    help='Comma separated list of "name=value" pairs.')
args = parser.parse_args()
...
def my_program():
  # Create a HParams object specifying the names and values of the
  # model hyperparameters:
  hparams = tf.contrib.training.HParams(
      learning_rate=0.1,
      num_hidden_units=100,
      activations=['relu', 'tanh'])

  # Override hyperparameters values by parsing the command line
  hparams.parse(args.hparams)

  # If the user passed `--hparams=learning_rate=0.3` on the command line
  # then 'hparams' has the following attributes:
  hparams.learning_rate ==> 0.3
  hparams.num_hidden_units ==> 100
  hparams.activations ==> ['relu', 'tanh']

  # If the hyperparameters are in json format use parse_json:
  hparams.parse_json('{"learning_rate": 0.3, "activations": "relu"}')

Args
hparam_def	Serialized hyperparameters, encoded as a hparam_pb2.HParamDef protocol buffer. If provided, this object is initialized by deserializing hparam_def. Otherwise **kwargs is used.
model_structure	An instance of ModelStructure, defining the feature crosses to be used in the Trial.
**kwargs	Key-value pairs where the key is the hyperparameter name and the value is the value for the parameter.

Raises
ValueError	If both hparam_def and initialization values are provided, or if one of the arguments is invalid.

Methods

add_hparam

View source

add_hparam(
    name, value
)

Adds {name, value} pair to hyperparameters.

Args
name	Name of the hyperparameter.
value	Value of the hyperparameter. Can be one of the following types: int, float, string, int list, float list, or string list.

Raises
ValueError	if one of the arguments is invalid.

del_hparam

View source

del_hparam(
    name
)

Removes the hyperparameter with key 'name'.

Does nothing if it isn't present.

Args
name	Name of the hyperparameter.

from_proto

View source

@staticmethod
from_proto(
    hparam_def, import_scope=None
)

get

View source

get(
    key, default=None
)

Returns the value of key if it exists, else default.

get_model_structure

View source

get_model_structure()

override_from_dict

View source

override_from_dict(
    values_dict
)

Override existing hyperparameter values, parsing new values from a dictionary.

Args
values_dict	Dictionary of name:value pairs.

Returns
The HParams instance.

Raises
KeyError	If a hyperparameter in values_dict doesn't exist.
ValueError	If values_dict cannot be parsed.

parse

View source

parse(
    values
)

Override existing hyperparameter values, parsing new values from a string.

See parse_values for more detail on the allowed format for values.

Args
values	String. Comma separated list of name=value pairs where 'value' must follow the syntax described above.

Returns
The HParams instance.

Raises
ValueError	If values cannot be parsed or a hyperparameter in values doesn't exist.

parse_json

View source

parse_json(
    values_json
)

Override existing hyperparameter values, parsing new values from a json object.

Args
values_json	String containing a json object of name:value pairs.

Returns
The HParams instance.

Raises
KeyError	If a hyperparameter in values_json doesn't exist.
ValueError	If values_json cannot be parsed.

set_from_map

View source

set_from_map(
    values_map
)

DEPRECATED. Use override_from_dict. (deprecated)

set_hparam

View source

set_hparam(
    name, value
)

Set the value of an existing hyperparameter.

This function verifies that the type of the value matches the type of the existing hyperparameter.

Args
name	Name of the hyperparameter.
value	New value of the hyperparameter.

Raises
KeyError	If the hyperparameter doesn't exist.
ValueError	If there is a type mismatch.

set_model_structure

View source

set_model_structure(
    model_structure
)

to_json

View source

to_json(
    indent=None, separators=None, sort_keys=False
)

Serializes the hyperparameters into JSON.

Args
indent	If a non-negative integer, JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0, or negative, will only insert newlines. None (the default) selects the most compact representation.
separators	Optional (item_separator, key_separator) tuple. Default is (', ', ': ').
sort_keys	If True, the output dictionaries will be sorted by key.

Returns
A JSON string.

to_proto

View source

to_proto(
    export_scope=None
)

Converts a HParams object to a HParamDef protocol buffer.

Args
export_scope	Optional string. Name scope to remove.

Returns
A HParamDef protocol buffer.

values

View source

values()

Return the hyperparameter values as a Python dictionary.

Returns
A dictionary with hyperparameter names as keys. The values are the hyperparameter values.

__contains__

View source

__contains__(
    key
)