A Bayesian classifier
Authors:
B. A. Zalessky and P. V. Lukashevich
Translated by:
N. Semenov
Original publication:
Teoriya Imovirnostei ta Matematichna Statistika, tom 78 (2008).
Journal:
Theor. Probability and Math. Statist. 78 (2009), 2335
MSC (2000):
Primary 62C10; Secondary 90Bxx
Published electronically:
August 4, 2009
MathSciNet review:
2446846
Fulltext PDF
Abstract 
References 
Similar Articles 
Additional Information
Abstract: We consider a new Bayesian classifier for the classification of multidimensional observations of if the learning sample is known. We assume that the data are generated by two disjoint bounded sets and each vector of the sample is a result of the observation after one of the sets , , with a random error. In other words, we assume that a priori the Bayesian probability is given on the set and that every vector of observations has the density where the function is a probability density for all and . The maximum a posteriori probability estimators , , for the sets , , are constructed with the help of the learning sample. Under natural assumptions imposed on and , we show that the estimators converge to some sets (possibly different from and ). If the mean frequencies of observations of the classes are equal to , , then the estimators are consistent in the sense that , . We also discuss some results of numerical experiments showing the applicability of our classifier for solving the problems of the statistical classification.
 1.
S. A. Aivazyan, B. M. Buchshtaber, I. S. Enyukov, and L. D. Meshalkin, Applied Statistics: Classification and Reducing of Dimension, Finansy i Statistika, Moscow, 1989. (Russian)
 2.
A.
A. Borovkov, Matematicheskaya statistika, “Nauka”,
Moscow, 1984 (Russian). Otsenka parametrov. Proverka gipotez. [Estimation
of parameters. Testing of hypotheses]. MR 782295
(86i:62001)
 3.
Leo
Breiman, Jerome
H. Friedman, Richard
A. Olshen, and Charles
J. Stone, Classification and regression trees, Wadsworth
Statistics/Probability Series, Wadsworth Advanced Books and Software,
Belmont, CA, 1984. MR 726392
(86b:62101)
 4.
L. Breiman, Random Forests, Technical report, Department of Statistics, University of California, Berkeley, CA, 1999.
 5.
Vladimir
Vapnik, Estimation of dependences based on empirical data,
Springer Series in Statistics, SpringerVerlag, New YorkBerlin, 1982.
Translated from the Russian by Samuel Kotz. MR 672244
(84a:62043)
 6.
Vladimir
N. Vapnik, Statistical learning theory, Adaptive and Learning
Systems for Signal Processing, Communications, and Control, John Wiley
& Sons, Inc., New York, 1998. A WileyInterscience Publication. MR 1641250
(99h:62052)
 7.
S. Haykin, Neural Networks: A Comprehensive Foundation, Wiley, New York, 2005.
 8.
E. E. Zhuk and Yu. S. Kharin, Stability in the Cluster Analysis of Multivariate Data, Belgosuniversitet, Minsk, 1998. (Russian)
 9.
Shelemyahu
Zacks, The theory of statistical inference, John Wiley &
Sons, Inc., New YorkLondonSydney, 1971. Wiley Series in Probability and
Mathematical Statistics. MR 0420923
(54 #8934a)
 10.
È.
Leman, Teoriya tochechnogo otsenivaniya, \cyr Teoriya
Veroyatnosteĭ\ i Matematicheskaya Statistika [Probability Theory and
Mathematical Statistics], vol. 43, “Nauka”, Moscow, 1991
(Russian). Translated from the English by Yu.\ V. Prokhorov. MR 1143059
(93c:62003b)
 11.
G.
Matheron, Random sets and integral geometry, John
Wiley\thinspace&\thinspace Sons, New YorkLondonSydney, 1975. With a
foreword by Geoffrey S. Watson; Wiley Series in Probability and
Mathematical Statistics. MR 0385969
(52 #6828)
 12.
V.
V. Mottl′ and I.
B. Muchnik, Skrytye markovskie modeli v strukturnom analize
signalov, FizikoMatematicheskaya Literatura, Moscow, 1999 (Russian,
with Russian summary). MR 1778152
(2001m:94014)
 13.
J. Pfanzagl, On the measurability and consistency of minimum contrast estimates, Metrika 14 (1969), 249273.
 14.
D. Forsyth and J. Ponce, Computer Vision. A Modern Approach, Prentice Hall, New York, 2002.
 15.
Keinosuke
Fukunaga, Introduction to statistical pattern recognition, 2nd
ed., Computer Science and Scientific Computing, Academic Press, Inc.,
Boston, MA, 1990. MR 1075415
(91i:68131)
 16.
M. I. Schlesinger and V. Hlavac, Ten Lectures on Statistical and Structural Pattern Recognition, SpringerVerlag, Berlin, 2002.
 1.
 S. A. Aivazyan, B. M. Buchshtaber, I. S. Enyukov, and L. D. Meshalkin, Applied Statistics: Classification and Reducing of Dimension, Finansy i Statistika, Moscow, 1989. (Russian)
 2.
 A. A. Borovkov, Mathematical Statistics, Nauka, Moscow, 1984; English. transl., Taylor and Francis, Amsterdam, 1999. MR 782295 (86i:62001)
 3.
 L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and Regression Trees, Wadsworth International Group, 1984. MR 726392 (86b:62101)
 4.
 L. Breiman, Random Forests, Technical report, Department of Statistics, University of California, Berkeley, CA, 1999.
 5.
 V. N. Vapnik, Estimation of Dependencies Based on Empirical Data, Nauka, Moscow, 1979; English transl., SpringerVerlag, New York, 1982. MR 672244 (84a:62043)
 6.
 V. N. Vapnik, Statistical Learning Theory, Wiley, New York, 1998. MR 1641250 (99h:62052)
 7.
 S. Haykin, Neural Networks: A Comprehensive Foundation, Wiley, New York, 2005.
 8.
 E. E. Zhuk and Yu. S. Kharin, Stability in the Cluster Analysis of Multivariate Data, Belgosuniversitet, Minsk, 1998. (Russian)
 9.
 S. Zaks, Theory of Statistical Inference, John Wiley and Sons, New York, 1971. MR 0420923 (54:8934a)
 10.
 E. Lehmann, Theory of Point Estimation, Chapman and Hall, London, 1991. MR 1143059 (93c:62003b)
 11.
 G. Matheron, Random Sets and Integral Geometry, Wiley, New York, 1975. MR 0385969 (52:6828)
 12.
 V. V. Mottl' and I. B. Muchnik, Hidden Markov Models in Structural Analysis of Signals, Fizmatlit, Moscow, 1999. (Russian) MR 1778152 (2001m:94014)
 13.
 J. Pfanzagl, On the measurability and consistency of minimum contrast estimates, Metrika 14 (1969), 249273.
 14.
 D. Forsyth and J. Ponce, Computer Vision. A Modern Approach, Prentice Hall, New York, 2002.
 15.
 K. Fukunaga, Introduction to Statistical Pattern Recognition, Elsevier Science and Technology Books, Amsterdam, 1990. MR 1075415 (91i:68131)
 16.
 M. I. Schlesinger and V. Hlavac, Ten Lectures on Statistical and Structural Pattern Recognition, SpringerVerlag, Berlin, 2002.
Similar Articles
Retrieve articles in Theory of Probability and Mathematical Statistics
with MSC (2000):
62C10,
90Bxx
Retrieve articles in all journals
with MSC (2000):
62C10,
90Bxx
Additional Information
B. A. Zalessky
Affiliation:
United Institute of Informatics Problems, National Academy of Sciences, Surganova Street 6, Minsk, 220012, Belarus’
Email:
zalesky@newman.basnet.by
P. V. Lukashevich
Affiliation:
United Institute of Informatics Problems, National Academy of Sciences, Surganova Street 6, Minsk, 220012, Belarus’
DOI:
http://dx.doi.org/10.1090/S0094900009007595
PII:
S 00949000(09)007595
Received by editor(s):
October 23, 2006
Published electronically:
August 4, 2009
Additional Notes:
The first author was supported by the INTAS grant 04777036
Article copyright:
© Copyright 2009
American Mathematical Society
