On a directionally reinforced random walk

We consider a generalized version of a directionally reinforced random walk, which was originally introduced by Mauldin, Monticino, and von Weizs\"{a}cker in \cite{drw}. Our main result is a stable limit theorem for the position of the random walk in higher dimensions. This extends a result of Horv\'{a}th and Shao \cite{limits} that was previously obtained in dimension one only (however, in a more stringent functional form).


Introduction
In this paper we study the following directionally reinforced random walk. Fix d ∈ N and a finite set U of distinct unit vectors in R d (see Remark 2.8 at the end of Section 2 below, where a suitable alternative Markovian setup in a general state space is discussed). The vectors in U serve as feasible directions for the motion of the random walk. To avoid trivialities we assume that U contains at least two elements. Let X t ∈ R d denote the position of the random walk at time t. Throughout the paper we assume that X 0 = 0. The random walk changes its direction at random times s 1 := 0 < s 2 < s 3 < s 4 < ....

We assume that the time intervals
T n := s n+1 − s n , n ∈ N, are independent and identically distributed. Let η n ∈ U be the direction of the walk during time interval [s n , s n+1 ). We assume that η := (η n ) n≥1 is an irreducible stationary Markov chain on U which is, furthermore, independent of (s n ) n∈N .

Statement of main results
We first introduce a few notations. For a vector x = (x 1 , . . . , x d ) ∈ R d let x = max i |x i |. For (possibly, random) functions f, g : R + (or N) → R, write f ∼ g and f (t) = o(g(t)) to indicate that, respectively, lim t→∞ f (t)/g(t) = 1 and lim t→∞ f (t)/g(t) = 0, a. s. Let π = (π v ) v∈U ∈ R |U | be the unique stationary distribution of the Markov chain η and let Thus µ = E(η n ) ∈ R d for each n ∈ N.
The following theorem shows that a strong law of large numbers holds for X t and that, under suitable second moment condition, the sample paths of the random walk are uniformly close to the sample paths of a drifted Brownian motion. We have: (a) Suppose that E(T p 1 ) < ∞ for some constant p ∈ (1, 2). Then, If E(T p 1 ) < ∞ for some constant p > 2, then (in an enlarged, if needed, probability space) there exist a process X = X t ) t≥0 distributed as X and a Brownian motion The results stated in Theorem 2.1 as well as in Theorem 2.3 below are essentially due to [11]. In fact, the original proofs can be adapted to our more general setup. However, the proofs we give in Section 3 are shorter and somewhat simpler than the original ones. Furthermore, our proves can easily be seen working for the general Markov chain setup described in Remark 2.8 below.
The second part of Theorem 2.1 implies the invariance principle for (X nt − µnt) with the usual normalization √ n. We next state an invariance principle and the corresponding law of iterated logarithm under a slightly more relaxed moment condition. Let D R d denote the set of R d -valued càdlàg functions on [0, 1] equipped with the Skorokhod J 1 -topology. We use notation ⇒ to denote the weak convergence in D R d . We have: If E(T 2 1 ) < ∞, then (a) S n ⇒ W, where W = (W t ) t≥0 is a (possibly degenerate, but not identically equal to zero) d-dimensional Brownian motion.
Furthermore, a similar statement holds for the lim inf .
We next consider the case when E(T 2 1 ) = ∞ and T 1 is in the domain of attraction of a stable law. Namely, for the rest of our results we impose the following assumption. Recall that a function h : R + → R is said to be regularly varying of index α ∈ R if h(t) = t α L(t) for some L : R + → R such that L(λt) ∼ L(t) for all λ > 0. We will denote the set of all regularly varying functions of index α by R α .
If h(t) ∈ R α with α ∈ (1, 2] (and hence E(T 1 ) < ∞), one can obtain the following analogue of Theorem 2.3. It turns out that also in this case the functional limit theorem and the law of iterated logarithm for X t inherit the structure of the corresponding statements for the partial sums of i.i.d. variables n k=1 T k . Theorem 2.5. Let Assumption 2.4 hold with α ∈ (1, 2]. Let We have: (a) If α ∈ (1, 2), then (i) S t converges weakly to a non-degenerate multivariate stable law in R d .
In particular, for some constant c(x) > 0, (b) If α = 2 and E(T 2 1 ) = ∞, then S t converges weakly to a non-degenerate multivariate Gaussian distribution in R d .
Remark 2.7. The limiting random law in the statement of Theorem 2.6 is specified in (29) below. The stable limit laws for X t stated in Theorems 2.5 and Theorem 2.6 are extensions of corresponding one-dimensional results in [11]. The latter however are obtained in [11] in a more stringent functional form. The law of iterated logarithm given in Theorem 2.5 appears to be new even for d = 1. (ii) The proofs of our results given in Section 3 rest on the exploiting of a regenerative (renewal) structure associated with η, i.e. on the use of random times τ n which are introduced below in Section 3.1. It is then not hard to verify that all the results stated in this section, with the only exception of the generalized law of iterated logarithm given in part (a)-(ii) of Theorem 2.5, remain true for a class of regenerative (in the sense of [3]) Markov chains η whose stationary distribution are supported on general Borel subsets of S d−1 rather than on a finite set U ⊂ S d−1 . For instance, the following strong version of the classical Doeblin's conditions is sufficient for our purposes: • There exist a constant c r > 1 and a probability measure ψ on A regenerative (renewal) structure for Markov chains which satisfies Doeblin's condition is described in [3]. Due to the fact that under the assumption (5), the kernel H(x, A) is dominated uniformly from above and below by a probability measure ψ, such Markov chains share two key features with finite-state Markov chains. Namely, 1) the exponential bound stated in (7) holds for the renewal times which are defined in [3]; and 2) c −1 r < P x (A)/P y (A) < c r for any non-null event A ∈ T d and almost every states x, y ∈ S d−1 (with respect to the stationary law). Here P x stands for the law of the Markov chain η starting from the initial state x ∈ S d−1 . Once these two crucial properties are verified, our proofs (except only the proof of part (a)-(ii) of Theorem 2.5) work nearly verbatim for directionally reinforced random walks governed by a Markov chain η which satisfies condition (5).

Proofs
This section is devoted to the proof of the results stated in Section 2 above. Some preliminary observations are stated in Section 3.1 below. The proof of Theorem 2.1 is contained in Section 3.2. Theorems 2.3 and 2.5 are proved in Section 3.3 and Section 3.4, respectively. Finally, the proof of Theorem 2.6 is given in Section 3.5.

Preliminaries
Our approach relies on the use of a renewal structure which is induced on the paths of the random walk by the cycles of the underlying Markov chain η. To define the renewal structure, set τ 0 = 0 and let Thus, for i ≥ 1, τ i are steps when the Markov chain η visits the distinguished state u 1 . Correspondingly, s τ i are successive times when the random walk chooses u 1 as the direction of its motion. Recall N t from Section 1 (see a few lines preceding (1)). Denote by c(t) the number of times that the walker chooses direction u 1 before time t > 0. That is, where 1 A stands for the indicator function of an event A. Notice that N t is the unique mapping from R + to Z + which has the following property: The strong Markov property implies that the pairs ξ i , τ i+1 − τ i i∈N form an i.i.d. sequence which is independent of (ξ 0 , τ 1 ). Furthermore, since η is an irreducible finite-state Markov chain, there exist positive constants K 1 , K 2 > 0 such that the inequality holds uniformly for all reals t ≥ 0 and all integers i ≥ 0. We next list some direct consequences of the law of large numbers that will be frequently exploited in the subsequent proofs. Let v(n) be the number of times that the Markov chain η visits u 1 during its first n steps. Thus, while c(t) is the number of visits of η to u 1 up to time t > 0 on the clock of the random walk, v(n) is the number of occurrences of u 1 among first n directions of the random walk. In particular, v(N t ) = c(t). Taking into account (7), the law of large numbers and the renewal theorem imply that and, letting Λ k := Since η and (T k ) k∈N are independent, it follows that , a. s.
We now turn to the proofs of our main results.

Proof of Theorem 2.1
Part (a) of Theorem 2.1. Recall (7) and observe that the moment condition of the theorem along with the independence of the Markov chain η and (T k ) k∈N of each other, implies that where we used Minkowski's inequality and (7). It follows that ξ k = o k 1/p . Indeed, for any ε > 0, Chebyshev's inequality implies that and hence P ξ k > k 1 p ε i. o. = 0 by the Borel-Cantelli lemma. For now we will make a simplifying assumption (to be removed later on) that µ = 0. By virtue of (9), the Marcinkiewicz-Zigmund law of large numbers implies that lim t→∞ c(t)−1 i=0 An argument similar to the one which we used to estimate the order of ξ n , shows that with probability one r n = o(n 1/p ). Then (9) implies that This completes the proof of part (a) of Theorem 2.1 for the particular case µ = 0.
We now turn to the general case of arbitrary finite µ ∈ R d . Let Then X t is a directionally reinforced random walk associated with (T n ) n∈N andη = (η n ) n∈N .
Since E(η i ) = 0, we have X t = o t 1/p . To complete the proof of part (a) of the theorem, Part (b) of Theorem 2.1. Recall (8). Let Notice that Therefore, by virtue of (9), it suffices to show that lim n→∞ n −1/p · sup 0≤k≤n r k = 0, a. s. (15) Toward this end, let g(n) = max k ≤ n : r k ≥ r i for all 1 ≤ i ≤ n , n ∈ N.
Thus g(n) ≤ n and sup 0≤k≤n r k = r g(n) . Furthermore, since r k are i.i.d. random variables, lim n→∞ g(n) = ∞ with probability one. Therefore, r n = o(n 1/p ) yields (15). The proof of Theorem 2.1 is completed.

Proof of Theorem 2.3
Part (a) of Theorem 2.3. By (10), E ξ 1 2 ) < ∞ under the conditions of the theorem. Assume first that µ = 0. Then the invariance principle for i.i.d. sequences implies that where W (t) is a d-dimensional Brownian motion. It follows then from (9) where b = π 1 ·E(T 1 ) . Under the moment condition of Theorem 2.3 we have the following counterpart of (14): ξ k is bounded above by r c(nt) , it follows that which implies the desired convergence of n −1/2 · X nt when µ = 0. To prove the general case of arbitrary µ ∈ R d one can apply the result with µ = 0 to the Markov chainη n and the random walkX t that were introduced in (13). The proof of part (a) of the theorem is completed.
Part (b) of Theorem 2.3. Suppose first that µ = 0. For x ∈ Span(U) ⊂ R d and i ∈ N define Then, in view of (9), the law of iterated logarithm for i.i.d. sequences implies that there exists a constant K(x) ∈ (0, ∞) such that lim sup t→∞ c(t)−1 i=0 By (8) and (12) lim in the case µ = 0. To obtain the general case with an arbitrary µ ∈ R d , apply this result to the random walk X t defined in (13) and recall that X t − X t = µt. The proof of part (b) of Theorem 2.3 is completed.

Proof of Theorem 2.5
Part ( [4,20]) that a random vector ξ ∈ R d is said to be regularly varying with index α > 0 if there exists a function a : R + → R, regularly varying with index 1/α, and a Radon measure ν ξ on R d 0 such that where v ⇒ denotes the vague convergence of measures. We will denote by R d,α,a the set of all random d-vectors regularly varying with index α, associated with a given function a ∈ R 1/α by (18). The measure ν is referred to as the measure of regular variation associated with ξ. We will also use the following equivalent definition of the regular variation for random vectors (see, for instance, [4,20]). Let S d−1 denote the unit sphere in R d with respect to the norm · . Then ξ ∈ R d,α,a if and only if there exists a finite Borel measure S ξ on S d−1 such that for all t > 0, where v ⇒ denotes the vague convergence of measures on S d−1 . The following well-known result is the key to the proof of the next lemma: if ξ, η ∈ R d,α,a and ξ, η are independent of each other, then ν ξ 1 +η = ν ξ + ν η and S ξ+η = S ξ + S η . We have: Proof of Lemma 3.1. It is not hard to see that the claim of part (a) can be formally deduced from that of part (b). Thus we will focus on proving the more general claim (b).
First, observe that (19) implies that T 1 u ∈ R d,α,a for any u ∈ U. Let be the transition matrix of the Markov chain η. Further, define a sub-Markovian kernel Θ by setting Fix any t > 0 and a Borel set B ⊂ S d−1 , and let Then, where we assume that the sums v 2 =u 1 · · · v k =u 1 are empty if k = 1. Let Notice that for any k ∈ N and fixed set of vectors v 2 , . . . , v k ∈ U, we have Observe that the spectral radius of the matrix Θ is strictly less than one and that S T 1 v j (B) is uniformly bounded from above by max v∈U S T 1 v S d−1 . Therefore, the dominated convergence theorem implies that the following limit exists and the identity holds: Since the spectral radius of Θ is strictly less than one, Fubini's theorem implies that the right-hand side of the above identity defines a measure on S d−1 . The proof of the lemma is therefore completed.
We are now in a position to complete the proof of the limit results stated in parts (a) and (b) of Theorem 2.5. Suppose first that µ = 0. It follows from Lemma 3.1 and the stable limit theorem for i.i.d. sequences (see, for instance, Section 1.6 in [6, p. 75 where S α (t) is a homogeneous vector-valued process in D R d with independent increments and S α (1) distributed according to a stable law of index α. Then (similarly to (16)), asymptotic equivalence (9) In particular, using t = 1, Recall r k from (11). Since an application of the renewal theorem shows that X n n ⇒ L α and hence X ⌊t⌋ t ⇒ L α .
Since X ⌊t⌋ − X t ≤ 1, the proof of part (a)-(i) of Theorem 2.5 is completed.
Part (a)-(ii) of Theorem 2.5. For V ∈ U let c v (t) be the number of occurrences of v in the set {η 1 , η 2 , . . . , η Nt }. That is, Notice that c u 1 (t) = c(t), where c(t) is introduced in Section 3.1. Similarly to (9) we have , a. s., where π v is the mass that the stationary distribution of the Markov chain η puts on v.
We then have: P G n,v ∩ E n,v and G n,u ∩ E n,u ≤ P E n,v E n,u + P c v (n) > γ n + P c u (n) > γ n = P (E n,v ) · P (E n,u ) + P c v (n) > γ n + P c u (n) > γ n .
It follows from the large deviation principle for c v (n)/n that P c v (n) > γ n < K v e −nλv for some K v > 0 and λ v > 0. Furthermore, for any A > 0 and k n = [A n ], we have P (E kn,v ) ≤ Cn −β for some constants β > 1/2 and C > 0 (see [6, p. 177]; here we exploit the constraint 2αδ > 1). The Borel-Cantelli lemma implies then that P E kn,v E kn,u i. o. = 0. Since for any n ∈ N there is a unique j(n) ∈ N such that k j(n) ≤ n < k j(n)+1 , and lim k→∞ a k+1 (ln a k+1 ) δ a k (ln a k ) δ = 1, this yields (25). The proof of part (a)-(ii) of Theorem 2.5 is therefore completed.

Proof of Theorem 2.6
Define two families of processes, (B n ) n∈N and (C n ) n∈N in D(R), by setting B n (t) = [nt] k=1 ξ k a n and C n (t) = s τ [nt] a n , t ∈ [0, 1].
Lemma 3.1 combined with [4, Theorem 1.1] implies that (ξ 1 , s τ 2 − s τ 1 ) ∈ R d+1,α,a , and hence where S α and U α are homogeneous process with independent increments in D R d and D(R), respectively, such that S α (1) and U α (1) have (multivariate in the former case) stable distributions of index α. Let U −1 n and C −1 n denote the inverse processes of U n and C n , respectively. One can define C −1 n explicitly as follows: C −1 n (t) = n −1 c(a n t), t ∈ [0, 1].
Then the same argument as in [11, pp. 380-381] shows that (alternatively, one can use the result of [26]): in D R d+1 . This along with (28) implies (see, for instance, [5, p. 151]) that Passing to the subsequence m n = ⌊a −1 n ⌋ and using basic properties of regularly varying functions, we obtain c(n)−1 i=0 To conclude the proof of the theorem one can use verbatim the argument along the lines following (21) in the concluding paragraph of the above proof of part (a)-(i) of Theorem 2.5. Namely, taking into account the inequality and using the renewal theorem which ensures the weak convergence of r c(t) to a proper random variable, (30) yields that Xt t ⇒ L α . The proof of Theorem 2.6 is completed.