## Local differentiability of distance functions

HTML articles powered by AMS MathViewer

- by R. A. Poliquin, R. T. Rockafellar and L. Thibault
- Trans. Amer. Math. Soc.
**352**(2000), 5231-5249 - DOI: https://doi.org/10.1090/S0002-9947-00-02550-2
- Published electronically: June 9, 2000
- PDF | Request permission

## Abstract:

Recently Clarke, Stern and Wolenski characterized, in a Hilbert space, the closed subsets $C$ for which the distance function $d_{C}$ is continuously differentiable everywhere on an open “tube” of uniform thickness around $C$. Here a corresponding local theory is developed for the property of $d_{C}$ being continuously differentiable outside of $C$ on some neighborhood of a point $x\in C$. This is shown to be equivalent to the prox-regularity of $C$ at $x$, which is a condition on normal vectors that is commonly fulfilled in variational analysis and has the advantage of being verifiable by calculation. Additional characterizations are provided in terms of $d_{C}^{2}$ being locally of class $C^{1+}$ or such that $d_{C}^{2}+\sigma |\cdot |^{2}$ is convex around $x$ for some $\sigma >0$. Prox-regularity of $C$ at $x$ corresponds further to the normal cone mapping $N_{C}$ having a hypomonotone truncation around $x$, and leads to a formula for $P_{C}$ by way of $N_{C}$. The local theory also yields new insights on the global level of the Clarke-Stern-Wolenski results, and on a property of sets introduced by Shapiro, as well as on the concept of sets with positive reach considered by Federer in the finite dimensional setting.## References

- J. M. Borwein and J. R. Giles,
*The proximal normal formula in Banach space*, Trans. Amer. Math. Soc.**302**(1987), no. 1, 371–381 (English, with French summary). MR**887515**, DOI 10.1090/S0002-9947-1987-0887515-5 - F. H. Clarke, R. J. Stern, and P. R. Wolenski,
*Proximal smoothness and the lower-$C^2$ property*, J. Convex Anal.**2**(1995), no. 1-2, 117–144. MR**1363364** - Jean-Philippe Vial,
*Strong and weak convexity of sets and functions*, Math. Oper. Res.**8**(1983), no. 2, 231–259. MR**707055**, DOI 10.1287/moor.8.2.231 - Alexander Shapiro,
*Existence and differentiability of metric projections in Hilbert spaces*, SIAM J. Optim.**4**(1994), no. 1, 130–141. MR**1260410**, DOI 10.1137/0804006 - Herbert Federer,
*Curvature measures*, Trans. Amer. Math. Soc.**93**(1959), 418–491. MR**110078**, DOI 10.1090/S0002-9947-1959-0110078-1 - R. A. Poliquin and R. T. Rockafellar,
*Prox-regular functions in variational analysis*, Trans. Amer. Math. Soc.**348**(1996), no. 5, 1805–1838. MR**1333397**, DOI 10.1090/S0002-9947-96-01544-9 - R. A. Poliquin and R. T. Rockafellar,
*Generalized Hessian properties of regularized nonsmooth functions*, SIAM J. Optim.**6**(1996), no. 4, 1121–1137. MR**1416532**, DOI 10.1137/S1052623494279316 - René Poliquin and Terry Rockafellar,
*Second-order nonsmooth analysis in nonlinear programming*, Recent advances in nonsmooth optimization, World Sci. Publ., River Edge, NJ, 1995, pp. 322–349. MR**1460008** - R. Tyrrell Rockafellar and Roger J.-B. Wets,
*Variational analysis*, Grundlehren der mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 317, Springer-Verlag, Berlin, 1998. MR**1491362**, DOI 10.1007/978-3-642-02431-3 - C. Combari, A. Elhilali Alaoui, A. Levy, R. Poliquin, and L. Thibault,
*Convex composite functions in Banach spaces and the primal lower-nice property*, Proc. Amer. Math. Soc.**126**(1998), no. 12, 3701–3708. MR**1451793**, DOI 10.1090/S0002-9939-98-04324-X - René A. Poliquin,
*Integration of subdifferentials of nonconvex functions*, Nonlinear Anal.**17**(1991), no. 4, 385–398. MR**1123210**, DOI 10.1016/0362-546X(91)90078-F - René A. Poliquin,
*An extension of Attouch’s theorem and its application to second-order epi-differentiation of convexly composite functions*, Trans. Amer. Math. Soc.**332**(1992), no. 2, 861–874. MR**1145732**, DOI 10.1090/S0002-9947-1992-1145732-5 - A. B. Levy, R. Poliquin, and L. Thibault,
*Partial extensions of Attouch’s theorem with applications to proto-derivatives of subgradient mappings*, Trans. Amer. Math. Soc.**347**(1995), no. 4, 1269–1294. MR**1290725**, DOI 10.1090/S0002-9947-1995-1290725-3 - Lionel Thibault and Dariusz Zagrodny,
*Integration of subdifferentials of lower semicontinuous functions on Banach spaces*, J. Math. Anal. Appl.**189**(1995), no. 1, 33–58. MR**1312029**, DOI 10.1006/jmaa.1995.1003 - R. Correa, A. Jofré, and L. Thibault,
*Characterization of lower semicontinuous convex functions*, Proc. Amer. Math. Soc.**116**(1992), no. 1, 67–72. MR**1126193**, DOI 10.1090/S0002-9939-1992-1126193-4 - Edgar Asplund,
*Čebyšev sets in Hilbert space*, Trans. Amer. Math. Soc.**144**(1969), 235–240. MR**253023**, DOI 10.1090/S0002-9947-1969-0253023-7 - Jean-Pierre Aubin and Ivar Ekeland,
*Applied nonlinear analysis*, Pure and Applied Mathematics (New York), John Wiley & Sons, Inc., New York, 1984. A Wiley-Interscience Publication. MR**749753** - Ka Sing Lau,
*Almost Chebyshev subsets in reflexive Banach spaces*, Indiana Univ. Math. J.**27**(1978), no. 5, 791–795. MR**510772**, DOI 10.1512/iumj.1978.27.27051 - J. M. Borwein and H. M. Strójwas,
*Proximal analysis and boundaries of closed sets in Banach space. II. Applications*, Canad. J. Math.**39**(1987), no. 2, 428–472. MR**899844**, DOI 10.4153/CJM-1987-019-4 - Alfred Rosenblatt,
*Sur les points singuliers des équations différentielles*, C. R. Acad. Sci. Paris**209**(1939), 10–11 (French). MR**85** - E. Asplund and R. T. Rockafellar,
*Gradients of convex functions*, Trans. Amer. Math. Soc.**139**(1969), 443–467. MR**240621**, DOI 10.1090/S0002-9947-1969-0240621-X - T. S. Motzkin,
*Sur quelques propriétés caractéristiques des ensembles convexes*, Att. R. Acad. Lincei, Rend.**21**(1935), 562–567. - Victor Klee,
*Convexity of Chevyshev sets*, Math. Ann.**142**(1960/61), 292–304. MR**121633**, DOI 10.1007/BF01353420 - Jean-Baptiste Hiriart-Urruty,
*Ensembles de Tchebychev vs. ensembles convexes: l’état de la situation vu via l’analyse convexe non lisse*, Ann. Sci. Math. Québec**22**(1998), no. 1, 47–62 (French, with English and French summaries). MR**1626398** - Robert R. Phelps,
*Convex functions, monotone operators and differentiability*, 2nd ed., Lecture Notes in Mathematics, vol. 1364, Springer-Verlag, Berlin, 1993. MR**1238715**

## Bibliographic Information

**R. A. Poliquin**- Affiliation: Department of Mathematical Sciences, University of Alberta, Edmonton, Alberta, Canada T6G 2G1
- Email: rene.poliquin@ualberta.ca
**R. T. Rockafellar**- Affiliation: Department of Mathematics 354350, University of Washington, Seattle, Washington 98195-4350
- Email: rtr@math.washington.edu
**L. Thibault**- Affiliation: Laboratoire d’Analyse Convexe, Université Montpellier II, 34095 Montpellier, France
- Email: thibault@math.univ-montp2.fr
- Received by editor(s): June 17, 1997
- Received by editor(s) in revised form: June 10, 1998
- Published electronically: June 9, 2000
- Additional Notes: This work was supported in part by the Natural Sciences and Engineering Research Council of Canada under grant OGP41983 for the first author, by the National Science Foundation under grant DMS–9500957 for the second author, and by NATO under grant CRG 960360 for the third author. The authors are grateful for useful discussions with C. Combari, and for helpful comments made by the referee.
- © Copyright 2000 American Mathematical Society
- Journal: Trans. Amer. Math. Soc.
**352**(2000), 5231-5249 - MSC (1991): Primary 49J52, 58C06, 58C20; Secondary 90C30
- DOI: https://doi.org/10.1090/S0002-9947-00-02550-2
- MathSciNet review: 1694378