tensorplay.distributions.kl_divergence

tensorplay.distributions.kl_divergence(p: Distribution, q: Distribution) → Tensor[source]

Compute Kullback-Leibler divergence $KL(p \| q)$ between two distributions.

$$KL(p \| q) = \int p(x) \log\frac {p(x)} {q(x)} \,dx$$

Parameters:

• p (Distribution) – A Distribution object.

• q (Distribution) – A Distribution object.

Returns:

A batch of KL divergences of shape batch_shape.

Return type:

Tensor

Raises:

NotImplementedError – If the distribution types have not been registered via register_kl().

KL divergence is currently implemented for the following distribution pairs:

• Bernoulli and Bernoulli

• Bernoulli and Poisson

• Beta and Beta

• Beta and ContinuousBernoulli

• Beta and Exponential

• Beta and Gamma

• Beta and Normal

• Beta and Pareto

• Beta and Uniform

• Binomial and Binomial

• Categorical and Categorical

• Cauchy and Cauchy

• ContinuousBernoulli and ContinuousBernoulli

• ContinuousBernoulli and Exponential

• ContinuousBernoulli and Normal

• ContinuousBernoulli and Pareto

• ContinuousBernoulli and Uniform

• Dirichlet and Dirichlet

• Exponential and Beta

• Exponential and ContinuousBernoulli

• Exponential and Exponential

• Exponential and Gamma

• Exponential and Gumbel

• Exponential and Normal

• Exponential and Pareto

• Exponential and Uniform

• ExponentialFamily and ExponentialFamily

• Gamma and Beta

• Gamma and ContinuousBernoulli

• Gamma and Exponential

• Gamma and Gamma

• Gamma and Gumbel

• Gamma and Normal

• Gamma and Pareto

• Gamma and Uniform

• Geometric and Geometric

• Gumbel and Beta

• Gumbel and ContinuousBernoulli

• Gumbel and Exponential

• Gumbel and Gamma

• Gumbel and Gumbel

• Gumbel and Normal

• Gumbel and Pareto

• Gumbel and Uniform

• HalfNormal and HalfNormal

• Independent and Independent

• Laplace and Beta

• Laplace and ContinuousBernoulli

• Laplace and Exponential

• Laplace and Gamma

• Laplace and Laplace

• Laplace and Normal

• Laplace and Pareto

• Laplace and Uniform

• LowRankMultivariateNormal and LowRankMultivariateNormal

• LowRankMultivariateNormal and MultivariateNormal

• MultivariateNormal and LowRankMultivariateNormal

• MultivariateNormal and MultivariateNormal

• Normal and Beta

• Normal and ContinuousBernoulli

• Normal and Exponential

• Normal and Gamma

• Normal and Gumbel

• Normal and Laplace

• Normal and Normal

• Normal and Pareto

• Normal and Uniform

• OneHotCategorical and OneHotCategorical

• Pareto and Beta

• Pareto and ContinuousBernoulli

• Pareto and Exponential

• Pareto and Gamma

• Pareto and Normal

• Pareto and Pareto

• Pareto and Uniform

• Poisson and Bernoulli

• Poisson and Binomial

• Poisson and Poisson

• TransformedDistribution and TransformedDistribution

• Uniform and Beta

• Uniform and ContinuousBernoulli

• Uniform and Exponential

• Uniform and Gamma

• Uniform and Gumbel

• Uniform and Normal

• Uniform and Pareto

• Uniform and Uniform