POSTERIOR_ENTROPY

This function computes Shannon entropy for a posterior probability distribution, or relative entropy (Kullback-Leibler divergence) when a reference distribution is also provided.

For posterior probabilities p_i, Shannon entropy is:

H(p) = -\sum_i p_i \log(p_i)

For posterior probabilities p_i and reference probabilities q_i, relative entropy is:

D_{\mathrm{KL}}(p\|q) = \sum_i p_i \log\left(\frac{p_i}{q_i}\right)

Inputs are normalized internally by SciPy when needed, and the logarithm base can be configured to report units such as nats or bits.

Excel Usage

=POSTERIOR_ENTROPY(pk, qk, base, axis, nan_policy, keepdims)
  • pk (list[list], required): 2D array of posterior probabilities or nonnegative weights.
  • qk (list[list], optional, default: null): 2D array of reference probabilities for relative entropy (unitless).
  • base (float, optional, default: null): Logarithm base for entropy units (unitless).
  • axis (int, optional, default: 0): Axis along which entropy is computed (dimensionless index).
  • nan_policy (str, optional, default: “propagate”): Rule for handling NaN values.
  • keepdims (bool, optional, default: false): Keep reduced dimensions in output shape.

Returns (list[list]): Entropy result returned as a 2D array for Excel compatibility.

Example 1: Shannon entropy in bits for a fair binary posterior

Inputs:

pk base
0.5 0.5 2

Excel formula:

=POSTERIOR_ENTROPY({0.5,0.5}, 2)

Expected output:

Result
0 0
Example 2: Relative entropy against a biased reference distribution

Inputs:

pk qk base
0.5 0.5 0.9 0.1 2

Excel formula:

=POSTERIOR_ENTROPY({0.5,0.5}, {0.9,0.1}, 2)

Expected output:

Result
0 0
Example 3: Entropy along columns for a two-row posterior table

Inputs:

pk axis
0.2 0.8 0
0.6 0.4

Excel formula:

=POSTERIOR_ENTROPY({0.2,0.8;0.6,0.4}, 0)

Expected output:

Result
0.562335 0.636514
Example 4: Entropy with retained dimensions for broadcast compatibility

Inputs:

pk axis keepdims
0.25 0.25 0.5 1 true

Excel formula:

=POSTERIOR_ENTROPY({0.25,0.25,0.5}, 1, TRUE)

Expected output:

1.03972

Python Code

import numpy as np
from scipy.stats import entropy as scipy_entropy

def posterior_entropy(pk, qk=None, base=None, axis=0, nan_policy='propagate', keepdims=False):
    """
    Compute Shannon or relative entropy for posterior probability tables.

    See: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.entropy.html

    This example function is provided as-is without any representation of accuracy.

    Args:
        pk (list[list]): 2D array of posterior probabilities or nonnegative weights.
        qk (list[list], optional): 2D array of reference probabilities for relative entropy (unitless). Default is None.
        base (float, optional): Logarithm base for entropy units (unitless). Default is None.
        axis (int, optional): Axis along which entropy is computed (dimensionless index). Default is 0.
        nan_policy (str, optional): Rule for handling NaN values. Valid options: Propagate, Omit, Raise. Default is 'propagate'.
        keepdims (bool, optional): Keep reduced dimensions in output shape. Default is False.

    Returns:
        list[list]: Entropy result returned as a 2D array for Excel compatibility.
    """
    try:
        def to2d(x):
            return [[x]] if not isinstance(x, list) else x

        def to_excel_2d(value):
            arr = np.asarray(value)
            if arr.ndim == 0:
                return [[float(arr)]]
            if arr.ndim == 1:
                return [[float(v) for v in arr.tolist()]]
            if arr.ndim == 2:
                return [[float(v) for v in row] for row in arr.tolist()]
            flat = arr.ravel()
            return [[float(v) for v in flat.tolist()]]

        pk = to2d(pk)
        qk = None if qk is None else to2d(qk)

        if not isinstance(pk, list) or not all(isinstance(row, list) for row in pk):
            return "Error: pk must be a 2D list"

        try:
            pk_arr = np.array(pk, dtype=float)
        except Exception:
            return "Error: pk must contain numeric values"

        qk_arr = None
        if qk is not None:
            if not isinstance(qk, list) or not all(isinstance(row, list) for row in qk):
                return "Error: qk must be a 2D list"
            try:
                qk_arr = np.array(qk, dtype=float)
            except Exception:
                return "Error: qk must contain numeric values"

        axis_arg = None if axis is None else int(axis)
        base_arg = None if base is None else float(base)

        result = scipy_entropy(
            pk_arr,
            qk=qk_arr,
            base=base_arg,
            axis=axis_arg,
            nan_policy=nan_policy,
            keepdims=bool(keepdims)
        )

        return to_excel_2d(result)
    except Exception as e:
        return f"Error: {str(e)}"

Online Calculator

2D array of posterior probabilities or nonnegative weights.
2D array of reference probabilities for relative entropy (unitless).
Logarithm base for entropy units (unitless).
Axis along which entropy is computed (dimensionless index).
Rule for handling NaN values.
Keep reduced dimensions in output shape.