tf.contrib.distributions.QuantizedDistribution

Distribution representing the quantization Y = ceiling(X).

Inherits From: Distribution

tf.contrib.distributions.QuantizedDistribution(
    distribution, low=None, high=None, validate_args=False,
    name='QuantizedDistribution'
)

Definition in Terms of Sampling

1. Draw X
2. Set Y <-- ceiling(X)
3. If Y < low, reset Y <-- low
4. If Y > high, reset Y <-- high
5. Return Y

Definition in Terms of the Probability Mass Function

Given scalar random variable X, we define a discrete random variable Y supported on the integers as follows:

P[Y = j] := P[X <= low],  if j == low,
         := P[X > high - 1],  j == high,
         := 0, if j < low or j > high,
         := P[j - 1 < X <= j],  all other j.

Conceptually, without cutoffs, the quantization process partitions the real line R into half open intervals, and identifies an integer j with the right endpoints:

R = ... (-2, -1](-1, 0](0, 1](1, 2](2, 3](3, 4] ...
j = ...      -1      0     1     2     3     4  ...

P[Y = j] is the mass of X within the jth interval. If low = 0, and high = 2, then the intervals are redrawn and j is re-assigned:

R = (-infty, 0](0, 1](1, infty)
j =          0     1     2

P[Y = j] is still the mass of X within the jth interval.

Examples

We illustrate a mixture of discretized logistic distributions [(Salimans et al., 2017)][1]. This is used, for example, for capturing 16-bit audio in WaveNet [(van den Oord et al., 2017)][2]. The values range in a 1-D integer domain of [0, 2**16-1], and the discretization captures P(x - 0.5 < X <= x + 0.5) for all x in the domain excluding the endpoints. The lowest value has probability P(X <= 0.5) and the highest value has probability P(2**16 - 1.5 < X).

Below we assume a wavenet function. It takes as input right-shifted audio samples of shape [..., sequence_length]. It returns a real-valued tensor of shape [..., num_mixtures * 3], i.e., each mixture component has a loc and scale parameter belonging to the logistic distribution, and a logits parameter determining the unnormalized probability of that component.

import tensorflow_probability as tfp
tfd = tfp.distributions
tfb = tfp.bijectors

net = wavenet(inputs)
loc, unconstrained_scale, logits = tf.split(net,
                                            num_or_size_splits=3,
                                            axis=-1)
scale = tf.nn.softplus(unconstrained_scale)

# Form mixture of discretized logistic distributions. Note we shift the
# logistic distribution by -0.5. This lets the quantization capture "rounding"
# intervals, `(x-0.5, x+0.5]`, and not "ceiling" intervals, `(x-1, x]`.
discretized_logistic_dist = tfd.QuantizedDistribution(
    distribution=tfd.TransformedDistribution(
        distribution=tfd.Logistic(loc=loc, scale=scale),
        bijector=tfb.AffineScalar(shift=-0.5)),
    low=0.,
    high=2**16 - 1.)
mixture_dist = tfd.MixtureSameFamily(
    mixture_distribution=tfd.Categorical(logits=logits),
    components_distribution=discretized_logistic_dist)

neg_log_likelihood = -tf.reduce_sum(mixture_dist.log_prob(targets))
train_op = tf.train.AdamOptimizer().minimize(neg_log_likelihood)

After instantiating mixture_dist, we illustrate maximum likelihood by calculating its log-probability of audio samples as target and optimizing.

References

[1]: Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P. Kingma. PixelCNN++: Improving the PixelCNN with discretized logistic mixture likelihood and other modifications. International Conference on Learning Representations, 2017. https://arxiv.org/abs/1701.05517 [2]: Aaron van den Oord et al. Parallel WaveNet: Fast High-Fidelity Speech Synthesis. arXiv preprint arXiv:1711.10433, 2017. https://arxiv.org/abs/1711.10433

Args
`distribution`	The base distribution class to transform. Typically an instance of `Distribution`.
`low`	`Tensor` with same `dtype` as this distribution and shape able to be added to samples. Should be a whole number. Default `None`. If provided, base distribution's `prob` should be defined at `low`.
`high`	`Tensor` with same `dtype` as this distribution and shape able to be added to samples. Should be a whole number. Default `None`. If provided, base distribution's `prob` should be defined at `high - 1`. `high` must be strictly greater than `low`.
`validate_args`	Python `bool`, default `False`. When `True` distribution parameters are checked for validity despite possibly degrading runtime performance. When `False` invalid inputs may silently render incorrect outputs.
`name`	Python `str` name prefixed to Ops created by this class.

Raises
`TypeError`	If `dist_cls` is not a subclass of `Distribution` or continuous.
`NotImplementedError`	If the base distribution does not implement `cdf`.

Attributes
`allow_nan_stats`	Python `bool` describing behavior when a stat is undefined. Stats return +/- infinity when it makes sense. E.g., the variance of a Cauchy distribution is infinity. However, sometimes the statistic is undefined, e.g., if a distribution's pdf does not achieve a maximum within the support of the distribution, the mode is undefined. If the mean is undefined, then by definition the variance is undefined. E.g. the mean for Student's T for df = 1 is undefined (no clear way to say it is either + or - infinity), so the variance = E[(X - mean)**2] is also undefined.
`batch_shape`	Shape of a single sample from a single event index as a `TensorShape`. May be partially defined or unknown. The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.
`distribution`	Base distribution, p(x).
`dtype`	The `DType` of `Tensor`s handled by this `Distribution`.
`event_shape`	Shape of a single sample from a single batch as a `TensorShape`. May be partially defined or unknown.
`high`	Highest value that quantization returns.
`low`	Lowest value that quantization returns.
`name`	Name prepended to all ops created by this `Distribution`.
`parameters`	Dictionary of parameters used to instantiate this `Distribution`.
`reparameterization_type`	Describes how samples from the distribution are reparameterized. Currently this is one of the static instances `distributions.FULLY_REPARAMETERIZED` or `distributions.NOT_REPARAMETERIZED`.
`validate_args`	Python `bool` indicating possibly expensive checks are enabled.

Args
`value`	`float` or `double` `Tensor`.
`name`	Python `str` prepended to names of ops created by this function.

Args
`other`	`tfp.distributions.Distribution` instance.
`name`	Python `str` prepended to names of ops created by this function.

Args
`sample_shape`	`Tensor` or python list/tuple. Desired shape of a call to `sample()`.
`name`	name to prepend ops with.

Args
`sample_shape`	0D or 1D `int32` `Tensor`. Shape of the generated samples.
`seed`	Python integer seed for RNG
`name`	name to give to the op.

tf.contrib.distributions.QuantizedDistribution

Definition in Terms of Sampling

Definition in Terms of the Probability Mass Function

Examples

References

Methods

batch_shape_tensor

cdf

copy

covariance

cross_entropy

entropy

event_shape_tensor

is_scalar_batch

is_scalar_event

kl_divergence

log_cdf

log_prob

log_survival_function

mean

mode

param_shapes

param_static_shapes

prob

quantile

sample

stddev

survival_function

variance

`batch_shape_tensor`

`cdf`

`copy`

`covariance`

`cross_entropy`

`entropy`

`event_shape_tensor`

`is_scalar_batch`

`is_scalar_event`

`kl_divergence`

`log_cdf`

`log_prob`

`log_survival_function`

`mean`

`mode`

`param_shapes`

`param_static_shapes`

`prob`

`quantile`

`sample`

`stddev`

`survival_function`

`variance`