# Restriction of irreducible unitary representations of to parabolic subgroups

## Abstract

In this paper, we obtain explicit branching laws for all irreducible unitary representations of when restricted to a parabolic subgroup The restriction turns out to be a finite direct sum of irreducible unitary representations of . We also verify DufloтАЩs conjecture for the branching laws of discrete series representations of . with respect to That is to show: in the framework of the orbit method, the branching law of a discrete series representation is determined by some geometric behavior of the moment map for the corresponding coadjoint orbit. .

## 1. Introduction

The branching law problem concerning the decomposition of the restriction of irreducible unitary representations to a closed Lie subgroup is an important problem in the representation theory of real Lie groups. In a series of seminal papers *discretely decomposable* if it is a direct sum of irreducible unitary representations of If moreover, all irreducible unitary representations of . have only finite multiplicities in then , is said to be *admissible*. Kobayashi established criteria for the admissibility for a large class of irreducible unitary representations with respect to reductive subgroups. Based on his work, branching laws for admissible restriction have been studied in many papers including

The orbit method of Kirillov (

Let us give some explanations for Conjecture 1.1. The notion of тАЬalmost algebraic groupтАЭ is defined in *strongly regular* if is regular (i.e., the coadjoint orbit containing is of maximal dimension) and its тАЬreductive factorтАЭ is of maximal dimension among reductive factors of all regular elements in ( is regular implies that is commutative). Here, is the Lie algebra of the stabilizer of in Let . denote the set of strongly regular elements in A coadjoint orbit . is called *strongly regular* if there exists an element (then every element in which is strongly regular. тАЬWeakly properтАЭ in (i) means that the preimage (for ) of each compact subset which is contained in ) is compact in Note that it is known that the classic properness condition is not sufficient to characterize the . when -admissibility is not reductive (see *quantization commutes with reduction* principle (see

In this paper, based on our explicit branching laws and an explicit description of the moment map, we verify Conjecture 1.1 for the restriction to a minimal parabolic subgroup of all discrete series representations of In our setting the restriction is admissible for any irreducible unitary representation . while the moment map is weakly proper for any .

For the proof of our branching laws, a key idea is to construct of certain irreducible unitary representations by taking the classical Fourier transform of the non-compact picture -models( of principal series representations of -picture) Classical Fourier transform was also used in other works studying branching laws, e.g., .

On the geometry side, let be a regular elliptic coadjoint By parametrizing the double coset space -orbit. we find explicit representatives of in -orbits By calculating the Pfaffian and the characteristic polynomial of the related skew-symmetric matrix, we are able to identify the . of the moment map image of each representative. With this, we calculate the image and show geometric properties of the moment map. -class

One might compare our branching laws with KirillovтАЩs conjecture which says that the restriction to a mirabolic subgroup of any irreducible unitary representation of (for an archimedean or non-archimedean local field) is irreducible. KirillovтАЩs conjecture was proved by Bernstein

The paper is organized as follows. In ┬з2 we introduce notation used throughout the paper, and we give a classification of irreducible unitary representations of In ┬з .3 we obtain branching laws of irreducible unitary representations of when restricted to Section .4 is devoted to the description of the moment map In ┬з .5 we verify Conjecture 1.1 in our setting. In Appendix A, we show that is determined by the types of for any unitary representation of In Appendix .B, we explain that branching laws shown in this paper are related to a case of Bessel model of the local Gan-Gross-Prasad conjecture (

## 2. Preliminaries

### 2.1. Notation and conventions

#### Indefinite orthogonal and spin groups of real rank one.

Fix a positive integer (Excluding . makes the group below connected and makes the parametrization of irreducible representations uniform. The case was treated in

Put

where is the identity component of (and of and ), is a non-trivial 2-fold covering of The Lie algebras of . are all equal to

Our results will be stated and proved for For concrete matrix calculation we will also work with groups . and The group . and its representations are used only in Appendix B.

#### Cartan decomposition

Write

Then, are maximal compact subgroups of respectively. Their Lie algebras are equal to

Write

Then, which is a Cartan decomposition for , The corresponding Cartan involution . of (or , is given by ) .

#### Restricted roots and Iwasawa decomposition

Put

which is a maximal abelian subspace in Define . by Then, the restricted root system . consists of two roots Let . be a positive restricted root. Then the associated positive nilpotent part is

Let be half the sum of positive roots in Then .

One has the *Iwasawa decomposition* .

#### Standard parabolic and opposite parabolic subalgebras

Let

Write which is a parabolic subalgebra of , We have the opposite nilradical .

and the opposite parabolic subalgebra .

#### Subgroups

Let (or , , ), (or , , ), (or , , be connected analytic subgroups of ) (or , , with Lie algebras ) respectively. In particular,

By the two-fold covering and the inclusions and we identify , , with identify , , , with and identify , , , with .

Put

Set

and

Then, the Lie algebras of (or , , ), (or , , ), (or , , are equal to ) , , respectively. Note that

and where , .

#### Nilpotent elements.

For a row vector write

The following Lie brackets will be used later

where and .

Let and Then . and more concretely, ,

and

Via the maps

#### Invariant bilinear form

For

Then

Then,

#### Roots and weights

Let