Remote Access Theory of Probability and Mathematical Statistics

Theory of Probability and Mathematical Statistics

ISSN 1547-7363(online) ISSN 0094-9000(print)



A Bayesian classifier

Authors: B. A. Zalessky and P. V. Lukashevich
Translated by: N. Semenov
Original publication: Teoriya Imovirnostei ta Matematichna Statistika, tom 78 (2008).
Journal: Theor. Probability and Math. Statist. 78 (2009), 23-35
MSC (2000): Primary 62C10; Secondary 90Bxx
Published electronically: August 4, 2009
MathSciNet review: 2446846
Full-text PDF Free Access

Abstract | References | Similar Articles | Additional Information

Abstract: We consider a new Bayesian classifier for the classification of multidimensional observations $ X_1,\dots,X_n$ of $ \mathbb{R}^k$ if the learning sample is known. We assume that the data are generated by two disjoint bounded sets $ \Omega_0,\Omega_1\subset \mathbb{R}^k$ and each vector $ X_i$ of the sample is a result of the observation after one of the sets $ \Omega_\ell$, $ \ell=0,1$, with a random error. In other words, we assume that a priori the Bayesian probability $ \mu$ is given on the set $ \Omega=\Omega_0\cup\Omega_1$ and that every vector of observations $ X_i$ has the density

$\displaystyle g_\ell(x)=q_\ell\int_{\Omega_\ell}f(x,y) \mu(dy),\qquad \ell=0,1, $

where the function $ f(x,y)$ is a probability density for all $ y\in \Omega$ and $ q_\ell^{-1}=\mu(\Omega_\ell)$.

The maximum a posteriori probability estimators $ \widehat{\Omega}_{\ell,n}$, $ \ell=0,1$, for the sets  $ \Omega_\ell$, $ \ell=0,1$, are constructed with the help of the learning sample. Under natural assumptions imposed on $ \Omega_0$ and $ \Omega_1$, we show that the estimators converge to some sets (possibly different from $ \Omega_0$ and $ \Omega_1$). If the mean frequencies $ \pi_\ell$ of observations of the classes $ \Omega_\ell$ are equal to $ \mu(\Omega_\ell)$, $ \ell=0,1$, then the estimators are consistent in the sense that $ \widehat{\Omega}_{\ell,n} \stackrel{n\rightarrow\infty}{\longrightarrow}\Omega_{\ell}$, $ \ell=0,1$. We also discuss some results of numerical experiments showing the applicability of our classifier for solving the problems of the statistical classification.

References [Enhancements On Off] (What's this?)

  • 1. S. A. Aivazyan, B. M. Buchshtaber, I. S. Enyukov, and L. D. Meshalkin, Applied Statistics: Classification and Reducing of Dimension, Finansy i Statistika, Moscow, 1989. (Russian)
  • 2. A. A. Borovkov, Mathematical Statistics, Nauka, Moscow, 1984; English. transl., Taylor and Francis, Amsterdam, 1999. MR 782295 (86i:62001)
  • 3. L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and Regression Trees, Wadsworth International Group, 1984. MR 726392 (86b:62101)
  • 4. L. Breiman, Random Forests, Technical report, Department of Statistics, University of California, Berkeley, CA, 1999.
  • 5. V. N. Vapnik, Estimation of Dependencies Based on Empirical Data, Nauka, Moscow, 1979; English transl., Springer-Verlag, New York, 1982. MR 672244 (84a:62043)
  • 6. V. N. Vapnik, Statistical Learning Theory, Wiley, New York, 1998. MR 1641250 (99h:62052)
  • 7. S. Haykin, Neural Networks: A Comprehensive Foundation, Wiley, New York, 2005.
  • 8. E. E. Zhuk and Yu. S. Kharin, Stability in the Cluster Analysis of Multivariate Data, Belgosuniversitet, Minsk, 1998. (Russian)
  • 9. S. Zaks, Theory of Statistical Inference, John Wiley and Sons, New York, 1971. MR 0420923 (54:8934a)
  • 10. E. Lehmann, Theory of Point Estimation, Chapman and Hall, London, 1991. MR 1143059 (93c:62003b)
  • 11. G. Matheron, Random Sets and Integral Geometry, Wiley, New York, 1975. MR 0385969 (52:6828)
  • 12. V. V. Mottl' and I. B. Muchnik, Hidden Markov Models in Structural Analysis of Signals, Fizmatlit, Moscow, 1999. (Russian) MR 1778152 (2001m:94014)
  • 13. J. Pfanzagl, On the measurability and consistency of minimum contrast estimates, Metrika 14 (1969), 249-273.
  • 14. D. Forsyth and J. Ponce, Computer Vision. A Modern Approach, Prentice Hall, New York, 2002.
  • 15. K. Fukunaga, Introduction to Statistical Pattern Recognition, Elsevier Science and Technology Books, Amsterdam, 1990. MR 1075415 (91i:68131)
  • 16. M. I. Schlesinger and V. Hlavac, Ten Lectures on Statistical and Structural Pattern Recognition, Springer-Verlag, Berlin, 2002.

Similar Articles

Retrieve articles in Theory of Probability and Mathematical Statistics with MSC (2000): 62C10, 90Bxx

Retrieve articles in all journals with MSC (2000): 62C10, 90Bxx

Additional Information

B. A. Zalessky
Affiliation: United Institute of Informatics Problems, National Academy of Sciences, Surganova Street 6, Minsk, 220012, Belarus’

P. V. Lukashevich
Affiliation: United Institute of Informatics Problems, National Academy of Sciences, Surganova Street 6, Minsk, 220012, Belarus’

Received by editor(s): October 23, 2006
Published electronically: August 4, 2009
Additional Notes: The first author was supported by the INTAS grant 04-77-7036
Article copyright: © Copyright 2009 American Mathematical Society

American Mathematical Society