A Bayesian classifier

Authors:
B. A. Zalessky and P. V. Lukashevich

Translated by:
N. Semenov

Journal:
Theor. Probability and Math. Statist. **78** (2009), 23-35

MSC (2000):
Primary 62C10; Secondary 90Bxx

DOI:
https://doi.org/10.1090/S0094-9000-09-00759-5

Published electronically:
August 4, 2009

MathSciNet review:
2446846

Full-text PDF Free Access

Abstract |
References |
Similar Articles |
Additional Information

Abstract:

We consider a new Bayesian classifier for the classification of multidimensional observations $X_1,\dots ,X_n$ of $\mathbb {R}^k$ if the learning sample is known. We assume that the data are generated by two disjoint bounded sets $\Omega _0,\Omega _1\subset \mathbb {R}^k$ and each vector $X_i$ of the sample is a result of the observation after one of the sets $\Omega _\ell$, $\ell =0,1$, with a random error. In other words, we assume that a priori the Bayesian probability $\mu$ is given on the set $\Omega =\Omega _0\cup \Omega _1$ and that every vector of observations $X_i$ has the density \[ g_\ell (x)=q_\ell \int _{\Omega _\ell }f(x,y) \mu (dy),\qquad \ell =0,1, \] where the function $f(x,y)$ is a probability density for all $y\in \Omega$ and $q_\ell ^{-1}=\mu (\Omega _\ell )$.

The maximum a posteriori probability estimators $\widehat {\Omega }_{\ell ,n}$, $\ell =0,1$, for the sets $\Omega _\ell$, $\ell =0,1$, are constructed with the help of the learning sample. Under natural assumptions imposed on $\Omega _0$ and $\Omega _1$, we show that the estimators converge to some sets (possibly different from $\Omega _0$ and $\Omega _1$). If the mean frequencies $\pi _\ell$ of observations of the classes $\Omega _\ell$ are equal to $\mu (\Omega _\ell )$, $\ell =0,1$, then the estimators are consistent in the sense that $\widehat {\Omega }_{\ell ,n} \stackrel {n\rightarrow \infty }{\longrightarrow }\Omega _{\ell }$, $\ell =0,1$. We also discuss some results of numerical experiments showing the applicability of our classifier for solving the problems of the statistical classification.

References
- S. A. Aivazyan, B. M. Buchshtaber, I. S. Enyukov, and L. D. Meshalkin,
*Applied Statistics: Classification and Reducing of Dimension*, Finansy i Statistika, Moscow, 1989. (Russian)
- A. A. Borovkov,
*Matematicheskaya statistika*, “Nauka”, Moscow, 1984 (Russian). Otsenka parametrov. Proverka gipotez. [Estimation of parameters. Testing of hypotheses]. MR **782295**
- Leo Breiman, Jerome H. Friedman, Richard A. Olshen, and Charles J. Stone,
*Classification and regression trees*, Wadsworth Statistics/Probability Series, Wadsworth Advanced Books and Software, Belmont, CA, 1984. MR **726392**
- L. Breiman,
*Random Forests*, Technical report, Department of Statistics, University of California, Berkeley, CA, 1999.
- Vladimir Vapnik,
*Estimation of dependences based on empirical data*, Springer Series in Statistics, Springer-Verlag, New York-Berlin, 1982. Translated from the Russian by Samuel Kotz. MR **672244**
- Vladimir N. Vapnik,
*Statistical learning theory*, Adaptive and Learning Systems for Signal Processing, Communications, and Control, John Wiley & Sons, Inc., New York, 1998. A Wiley-Interscience Publication. MR **1641250**
- S. Haykin,
*Neural Networks: A Comprehensive Foundation*, Wiley, New York, 2005.
- E. E. Zhuk and Yu. S. Kharin,
*Stability in the Cluster Analysis of Multivariate Data*, Belgosuniversitet, Minsk, 1998. (Russian)
- Shelemyahu Zacks,
*The theory of statistical inference*, John Wiley & Sons, Inc., New York-London-Sydney, 1971. Wiley Series in Probability and Mathematical Statistics. MR **0420923**
- È. Leman,
*Teoriya tochechnogo otsenivaniya*, Teoriya Veroyatnosteĭ i Matematicheskaya Statistika [Probability Theory and Mathematical Statistics], vol. 43, “Nauka”, Moscow, 1991 (Russian). Translated from the English by Yu. V. Prokhorov. MR **1143059**
- G. Matheron,
*Random sets and integral geometry*, John Wiley & Sons, New York-London-Sydney, 1975. With a foreword by Geoffrey S. Watson; Wiley Series in Probability and Mathematical Statistics. MR **0385969**
- V. V. Mottl′ and I. B. Muchnik,
*Skrytye markovskie modeli v strukturnom analize signalov*, Fiziko-Matematicheskaya Literatura, Moscow, 1999 (Russian, with Russian summary). MR **1778152**
- J. Pfanzagl,
*On the measurability and consistency of minimum contrast estimates*, Metrika **14** (1969), 249–273.
- D. Forsyth and J. Ponce,
*Computer Vision. A Modern Approach*, Prentice Hall, New York, 2002.
- Keinosuke Fukunaga,
*Introduction to statistical pattern recognition*, 2nd ed., Computer Science and Scientific Computing, Academic Press, Inc., Boston, MA, 1990. MR **1075415**
- M. I. Schlesinger and V. Hlavac,
*Ten Lectures on Statistical and Structural Pattern Recognition*, Springer-Verlag, Berlin, 2002.

References
- S. A. Aivazyan, B. M. Buchshtaber, I. S. Enyukov, and L. D. Meshalkin,
*Applied Statistics: Classification and Reducing of Dimension*, Finansy i Statistika, Moscow, 1989. (Russian)
- A. A. Borovkov,
*Mathematical Statistics*, Nauka, Moscow, 1984; English. transl., Taylor and Francis, Amsterdam, 1999. MR **782295 (86i:62001)**
- L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone,
*Classification and Regression Trees*, Wadsworth International Group, 1984. MR **726392 (86b:62101)**
- L. Breiman,
*Random Forests*, Technical report, Department of Statistics, University of California, Berkeley, CA, 1999.
- V. N. Vapnik,
*Estimation of Dependencies Based on Empirical Data*, Nauka, Moscow, 1979; English transl., Springer-Verlag, New York, 1982. MR **672244 (84a:62043)**
- V. N. Vapnik,
*Statistical Learning Theory*, Wiley, New York, 1998. MR **1641250 (99h:62052)**
- S. Haykin,
*Neural Networks: A Comprehensive Foundation*, Wiley, New York, 2005.
- E. E. Zhuk and Yu. S. Kharin,
*Stability in the Cluster Analysis of Multivariate Data*, Belgosuniversitet, Minsk, 1998. (Russian)
- S. Zaks,
*Theory of Statistical Inference*, John Wiley and Sons, New York, 1971. MR **0420923 (54:8934a)**
- E. Lehmann,
*Theory of Point Estimation*, Chapman and Hall, London, 1991. MR **1143059 (93c:62003b)**
- G. Matheron,
*Random Sets and Integral Geometry*, Wiley, New York, 1975. MR **0385969 (52:6828)**
- V. V. Mottl’ and I. B. Muchnik,
*Hidden Markov Models in Structural Analysis of Signals*, Fizmatlit, Moscow, 1999. (Russian) MR **1778152 (2001m:94014)**
- J. Pfanzagl,
*On the measurability and consistency of minimum contrast estimates*, Metrika **14** (1969), 249–273.
- D. Forsyth and J. Ponce,
*Computer Vision. A Modern Approach*, Prentice Hall, New York, 2002.
- K. Fukunaga,
*Introduction to Statistical Pattern Recognition*, Elsevier Science and Technology Books, Amsterdam, 1990. MR **1075415 (91i:68131)**
- M. I. Schlesinger and V. Hlavac,
*Ten Lectures on Statistical and Structural Pattern Recognition*, Springer-Verlag, Berlin, 2002.

Similar Articles

Retrieve articles in *Theory of Probability and Mathematical Statistics*
with MSC (2000):
62C10,
90Bxx

Retrieve articles in all journals
with MSC (2000):
62C10,
90Bxx

Additional Information

**B. A. Zalessky**

Affiliation:
United Institute of Informatics Problems, National Academy of Sciences, SurganovaStreet 6, Minsk, 220012, Belarus’

Email:
zalesky@newman.bas-net.by

**P. V. Lukashevich**

Affiliation:
United Institute of Informatics Problems, National Academy of Sciences, SurganovaStreet 6, Minsk, 220012, Belarus’

Received by editor(s):
October 23, 2006

Published electronically:
August 4, 2009

Additional Notes:
The first author was supported by the INTAS grant 04-77-7036

Article copyright:
© Copyright 2009
American Mathematical Society