Skip to Main Content
Remote Access Theory of Probability and Mathematical Statistics

Theory of Probability and Mathematical Statistics

ISSN 1547-7363(online) ISSN 0094-9000(print)

 
 

 

A Bayesian classifier


Authors: B. A. Zalessky and P. V. Lukashevich
Translated by: N. Semenov
Journal: Theor. Probability and Math. Statist. 78 (2009), 23-35
MSC (2000): Primary 62C10; Secondary 90Bxx
DOI: https://doi.org/10.1090/S0094-9000-09-00759-5
Published electronically: August 4, 2009
MathSciNet review: 2446846
Full-text PDF Free Access

Abstract | References | Similar Articles | Additional Information

Abstract:

We consider a new Bayesian classifier for the classification of multidimensional observations $X_1,\dots ,X_n$ of $\mathbb {R}^k$ if the learning sample is known. We assume that the data are generated by two disjoint bounded sets $\Omega _0,\Omega _1\subset \mathbb {R}^k$ and each vector $X_i$ of the sample is a result of the observation after one of the sets $\Omega _\ell$, $\ell =0,1$, with a random error. In other words, we assume that a priori the Bayesian probability $\mu$ is given on the set $\Omega =\Omega _0\cup \Omega _1$ and that every vector of observations $X_i$ has the density \[ g_\ell (x)=q_\ell \int _{\Omega _\ell }f(x,y) \mu (dy),\qquad \ell =0,1, \] where the function $f(x,y)$ is a probability density for all $y\in \Omega$ and $q_\ell ^{-1}=\mu (\Omega _\ell )$.

The maximum a posteriori probability estimators $\widehat {\Omega }_{\ell ,n}$, $\ell =0,1$, for the sets $\Omega _\ell$, $\ell =0,1$, are constructed with the help of the learning sample. Under natural assumptions imposed on $\Omega _0$ and $\Omega _1$, we show that the estimators converge to some sets (possibly different from $\Omega _0$ and $\Omega _1$). If the mean frequencies $\pi _\ell$ of observations of the classes $\Omega _\ell$ are equal to $\mu (\Omega _\ell )$, $\ell =0,1$, then the estimators are consistent in the sense that $\widehat {\Omega }_{\ell ,n} \stackrel {n\rightarrow \infty }{\longrightarrow }\Omega _{\ell }$, $\ell =0,1$. We also discuss some results of numerical experiments showing the applicability of our classifier for solving the problems of the statistical classification.


References [Enhancements On Off] (What's this?)

References
  • S. A. Aivazyan, B. M. Buchshtaber, I. S. Enyukov, and L. D. Meshalkin, Applied Statistics: Classification and Reducing of Dimension, Finansy i Statistika, Moscow, 1989. (Russian)
  • A. A. Borovkov, Matematicheskaya statistika, “Nauka”, Moscow, 1984 (Russian). Otsenka parametrov. Proverka gipotez. [Estimation of parameters. Testing of hypotheses]. MR 782295
  • Leo Breiman, Jerome H. Friedman, Richard A. Olshen, and Charles J. Stone, Classification and regression trees, Wadsworth Statistics/Probability Series, Wadsworth Advanced Books and Software, Belmont, CA, 1984. MR 726392
  • L. Breiman, Random Forests, Technical report, Department of Statistics, University of California, Berkeley, CA, 1999.
  • Vladimir Vapnik, Estimation of dependences based on empirical data, Springer Series in Statistics, Springer-Verlag, New York-Berlin, 1982. Translated from the Russian by Samuel Kotz. MR 672244
  • Vladimir N. Vapnik, Statistical learning theory, Adaptive and Learning Systems for Signal Processing, Communications, and Control, John Wiley & Sons, Inc., New York, 1998. A Wiley-Interscience Publication. MR 1641250
  • S. Haykin, Neural Networks: A Comprehensive Foundation, Wiley, New York, 2005.
  • E. E. Zhuk and Yu. S. Kharin, Stability in the Cluster Analysis of Multivariate Data, Belgosuniversitet, Minsk, 1998. (Russian)
  • Shelemyahu Zacks, The theory of statistical inference, John Wiley & Sons, Inc., New York-London-Sydney, 1971. Wiley Series in Probability and Mathematical Statistics. MR 0420923
  • È. Leman, Teoriya tochechnogo otsenivaniya, Teoriya Veroyatnosteĭ i Matematicheskaya Statistika [Probability Theory and Mathematical Statistics], vol. 43, “Nauka”, Moscow, 1991 (Russian). Translated from the English by Yu. V. Prokhorov. MR 1143059
  • G. Matheron, Random sets and integral geometry, John Wiley & Sons, New York-London-Sydney, 1975. With a foreword by Geoffrey S. Watson; Wiley Series in Probability and Mathematical Statistics. MR 0385969
  • V. V. Mottl′ and I. B. Muchnik, Skrytye markovskie modeli v strukturnom analize signalov, Fiziko-Matematicheskaya Literatura, Moscow, 1999 (Russian, with Russian summary). MR 1778152
  • J. Pfanzagl, On the measurability and consistency of minimum contrast estimates, Metrika 14 (1969), 249–273.
  • D. Forsyth and J. Ponce, Computer Vision. A Modern Approach, Prentice Hall, New York, 2002.
  • Keinosuke Fukunaga, Introduction to statistical pattern recognition, 2nd ed., Computer Science and Scientific Computing, Academic Press, Inc., Boston, MA, 1990. MR 1075415
  • M. I. Schlesinger and V. Hlavac, Ten Lectures on Statistical and Structural Pattern Recognition, Springer-Verlag, Berlin, 2002.

Similar Articles

Retrieve articles in Theory of Probability and Mathematical Statistics with MSC (2000): 62C10, 90Bxx

Retrieve articles in all journals with MSC (2000): 62C10, 90Bxx


Additional Information

B. A. Zalessky
Affiliation: United Institute of Informatics Problems, National Academy of Sciences, SurganovaStreet 6, Minsk, 220012, Belarus’
Email: zalesky@newman.bas-net.by

P. V. Lukashevich
Affiliation: United Institute of Informatics Problems, National Academy of Sciences, SurganovaStreet 6, Minsk, 220012, Belarus’

Received by editor(s): October 23, 2006
Published electronically: August 4, 2009
Additional Notes: The first author was supported by the INTAS grant 04-77-7036
Article copyright: © Copyright 2009 American Mathematical Society