In statistics, the Bhattacharyya distance is a quantity which represents a notion of similarity between two probability distributions. It is closely related to the Bhattacharyya coefficient, which is a measure of the amount of overlap between two statistical samples or populations.

It is not a metric, despite being named a "distance", since it does not obey the triangle inequality.

History

Both the Bhattacharyya distance and the Bhattacharyya coefficient are named after Anil Kumar Bhattacharyya, a statistician who worked in the 1930s at the Indian Statistical Institute. He has developed this through a series of papers. He developed the method to measure the distance between two non-normal distributions and illustrated this with the classical multinomial populations, this work despite being submitted for publication in 1941, appeared almost five years later in Sankhya. Note that the first term is a squared Mahalanobis distance.

Properties

<math>0 \le BC \le 1</math> and <math>0 \le D_B \le \infty</math>.

<math>D_B</math> does not obey the triangle inequality, though the Hellinger distance <math display="inline">\sqrt{1-BC(p,q)}</math> does.

Bounds on Bayes error

The Bhattacharyya distance can be used to upper and lower bound the Bayes error rate:

<math display='block'> \frac{1}{2} - \frac{1}{2}\sqrt{1-4\rho^2} \leq L^* \leq \rho</math>

where <math display="inline">\rho = \operatorname\mathbb E\left[\sqrt {\eta(X) \left(1-\eta(X)\right)}\right]</math> and <math>\eta(X) = \mathbb P(Y=1 | X)</math> is the posterior probability.

Applications

The Bhattacharyya coefficient quantifies the "closeness" of two random statistical samples.

Given two sequences from distributions <math>P, Q</math>, bin them into <math>n</math> buckets, and let the frequency of samples from <math>P</math> in bucket <math>i</math> be <math>p_i</math>, and similarly for <math>q_i</math>, then the sample Bhattacharyya coefficient is

<math display="block">BC(\mathbf{p},\mathbf{q}) = \sum_{i=1}^n \sqrt{p_i q_i},</math>

which is an estimator of <math>BC(P, Q)</math>. The quality of estimation depends on the choice of buckets; too few buckets would overestimate <math>BC(P, Q)</math>, while too many would underestimate.

A common task in classification is estimating the separability of classes. Up to a multiplicative factor, the squared Mahalanobis distance is a special case of the Bhattacharyya distance when the two classes are normally distributed with the same variances. When two classes have similar means but significantly different variances, the Mahalanobis distance would be close to zero, while the Bhattacharyya distance would not be.

The Bhattacharyya coefficient is used in the construction of polar codes.

The Bhattacharyya distance is used in feature extraction and selection, image processing, speaker recognition, phone clustering, and in genetics.

See also

  • Bhattacharyya angle
  • Kullback–Leibler divergence
  • Hellinger distance
  • Mahalanobis distance
  • Chernoff bound
  • Rényi entropy
  • F-divergence
  • Fidelity of quantum states

References

  • Statistical Intuition of Bhattacharyya's distance
  • Some of the properties of Bhattacharyya Distance