Kernel Theorems in Coorbit Theory

We prove general kernel theorems for operators acting between coorbit spaces. These are Banach spaces associated to an integrable representation of a locally compact group and contain most of the usual function spaces (Besov spaces, modulation spaces, etc.). A kernel theorem describes the form of every bounded operator between a coorbit space of test functions and distributions by means of a kernel in a coorbit space associated to the tensor product representation. As special cases we recover Feichtinger's kernel theorem for modulation spaces and the recent generalizations by Cordero and Nicola. We also obtain a kernel theorem for operators between the Besov spaces $\dot{B}^0_{1,1}$ and $\dot{B}^{0}_{\infty, \infty }$.


Introduction
Kernel theorems assert that every "reasonable" operator can written as a "generalized" integral operator. For instance, the Schwartz kernel theorem states that a continuous linear operator A : S(R d ) → S ′ (R d ) possesses a unique distributional kernel K ∈ S ′ (R 2d ), such that (1) Af, g = K, g ⊗ f , f, g ∈ S(R d ) .
If K is a locally integrable function, then K(x, y)f (y)g(x)dydx, f, g ∈ S(R d ), and thus A has indeed the form of an integral operator. Similar kernel theorems hold for continuous operators from D(R d ) → D ′ (R d ) [24,Theorem 5.2] and for Gelfand-Shilov spaces and their distribution spaces [21]. The importance of these kernel theorems stems from the fact that they offer a general formalism for the description of linear operators.
In the context of time-frequency analysis, Feichtinger's kernel theorem [12] (see also [18] and [23,Theorem 14.4.1]) states that every bounded linear operator from the modulation space M 1 (R d ) to the modulation space M ∞ (R d ) can be represented in the form (1) with a kernel in M ∞ (R 2d ). The advantage of this kernel theorem is that both the space of test functions M 1 (R d ) and the distribution space M ∞ (R d ) = M 1 (R d ) * are Banach spaces and thus technically easier than the locally convex spaces S(R d ) and S ′ (R d ).
Recently, Cordero and Nicola [8] revisited Feichtinger's kernel theorem and proved several new kernel theorems that "do not have a counterpart in distribution theory". They argue that "this reveals the superiority, in some respects, of the modulation space formalism upon distribution theory". While we agree full-heartedly with this claim, we would like to add a more abstract point of view and argue that the deeper reason for this superiority lies in the theory of coorbit spaces and in the convenience of Schur's test for integral operators. Indeed, we will prove kernel theorems similar to Feichtinger's kernel theorem for many coorbit spaces.
The main idea is to investigate operators in a transform domain, after taking a short-time Fourier transform, a wavelet transform, or an abstract wavelet transform, i.e., a continuous transform with respect to a unitary group representation. In this new representation every operator between a suitable space of test functions and distributions is an integral operator. The standard boundedness conditions of Schur's test then yield strong kernel theorems.
The technical framework for this idea is coorbit theory which was introduced and studied in [15,16,17,22] for the construction and analysis of function spaces by means of a generalized wavelet transform. The main idea is that functions in the standard function spaces, such as Besov spaces and modulation spaces, can be characterized by the decay or integrability properties of an associated transform (the wavelet transform or the short-time Fourier transform). In the abstract setting, G is a locally compact group and π : G → U(H) is an irreducible, unitary, integrable representation of G. Leaving technical details aside, the coorbit space Co π L p w (G) consists of all distributions f in a suitable distribution space, such that the representation coefficient g → f, π(g)ψ is in the weighted space L p w (G). Next, let G 1 and G 2 be two locally compact groups, and (π 1 , H 1 ) and (π 2 , H 2 ) be irreducible, unitary, integrable representations of G 1 and G 2 respectively.
Let A be a bounded linear operator between Co π 1 L 1 w 1 (G 1 ) and Co π 2 L ∞ 1/w 2 (G 2 ) . Our main insight is that such an operator can be described by a kernel in a coorbit space that is related to the tensor product representation π = π 2 ⊗π 1 of G = G 1 ×G 2 on the tensor product space H 2 ⊗ H 1 . The following non-technical formulation offers a flavor of our main result in Theorem 3: . This statement is not just a mere abstraction and generalization of the classical kernel theorem. With the choice of a specific group and representation one obtains explicit kernel theorems. For instance, using the Schrödinger representation of the Heisenberg group, one recovers Feichtinger's original kernel theorem. The added value is our insight that the conditions on the kernel of [8] in terms of mixed modulation spaces [4] amount to coorbit spaces with respect to the tensor product representation. Choosing the ax + b-group and the continuous wavelet representation, one obtains a kernel theorem for all bounded operators operators between the Besov spacesḂ 0 1,1 andḂ 0 ∞,∞ with a kernel in a space of dominating mixed smoothness. This class of function spaces has been studied extensively [32,31] and is by no means artificial.
By using suitable versions of Schur's test, it is then possible to derive characterizations for the boundedness of operators between other coorbit spaces. For example, in Theorem 7 we will prove the following, with 1 p + 1 q = 1: , where the mixed-norm Lebesgue spaces L p,q and L p,q on G 1 × G 2 are defined in (23) and (24) respectively.
The paper is organized as follows. In Section 2 we present the basics of tensor products and coorbit space theory. The theory of coorbit spaces of kernels with respect to products of integrable representations is developed in Section 3. Our main results, the kernel theorems, are proved in Section 4 and applied to particular examples of group representations in Section 5.
We note that our proofs require a meaningful formulation of coorbit theory. One can therefore prove kernel theorems also in the context of coorbit space theories [6,9], e.g., for certain reducible representations.

Preliminaries on Tensor Products and Coorbit Spaces
2.1. Tensor Products and Hilbert-Schmidt Operators. The theory of tensor products is at the heart of kernel theorems for operators. Algebraically, a simple tensor of two vectors (in two possibly different Hilbert spaces) is a formal product of two vectors f 1 ⊗ f 2 , and the tensor product H 1 ⊗ H 2 is obtained by taking the completion of all linear combinations of simple tensors with respect to the inner product This tensor product is homogeneous in the following sense: α·(f 1 ⊗f 2 ) = (αf 1 )⊗f 2 = f 1 ⊗ (αf 2 ). Note explicitly that the product f 1 ⊗ f 2 is anti-linear in the second factor. In some books this is done by introducing the dual Hilbert space H ′ 2 [25]. If each Hilbert space is an L 2 -space H 1 = L 2 (X, µ), H 2 = L 2 (Y, ν), then the simple tensor f ⊗ g is just the product (x, y) → f (x) · g(y) and the tensor product becomes the product space The connection between functions and operators arises in the analytic approach to tensor products. We interpret a function of two variables as an integral kernel for an operator. Thus a simple tensor f 1 ⊗f 2 of two functions becomes the rank one operator f → f, f 2 f 1 with integral kernel f 1 (x)f 2 (y), and a general k ∈ L 2 (X × Y, µ × ν) becomes a Hilbert-Schmidt operator from L 2 (Y, ν) to L 2 (X, µ). The systematic, analytic treatment of general tensor products of two Hilbert-spaces often defines the tensor product as a space of Hilbert-Schmidt operators between H 2 and H 1 . We note that his definition is already based on the characterization of Hilbert-Schmidt operators and thus represents a non-trivial kernel theorem [7]. Whereas the working mathematician habitually identifies an operator with its distributions kernel, we will make the conceptual distinction between tensor products and operators for our study of kernel theorems.
In the sequel we will denote the (distributional) kernel of an integral operator by k and the abstract kernel in a tensor product by K.

2.2.
Coorbit Space Theory. Let G be a locally compact group with left Haar measure G . . . dg, H be a separable Hilbert space, and U(H) the group of unitary operators acting on H. A continuous unitary group representation π : G → U(H) is called square integrable [11,1], if it is irreducible and there exist ψ ∈ H such that A non-zero vector ψ satisfying (3) is called admissible. For every square integrable representation there exist a densely defined operator T such that ∀f 1 , f 2 ∈ H, ψ 1 , ψ 2 ∈ Dom (T ), one has For fixed ψ 1 = ψ 2 = ψ the representation coefficient f → V ψ f (g) := f, π(g)ψ is interpreted as a generalized wavelet transform. The orthogonality relation (4) then implies that V ψ is a multiple of an isometry from H to L 2 (G). By using a weak interpretation of vector-valued integrals, (4) can also be recast as the inversion formula For the rest of this paper we assume without loss of generality that the chosen admissible vectors ψ are normalized, i.e. T ψ = 1.
The adjoint operator V * ψ : L 2 (G) → H is formally defined by Other domains and convergence properties will be discussed later. With this notation (5) says that V * ψ V ψ = I H for all admissible and normalized vectors ψ, which in the language of recent frame theory means that {π(g)ψ} g∈G is a continuous Parseval frame. By [5, Proposition 2.1] one can always assume that G is σ-finite since we assume H to be separable.
In coorbit theory one needs much stronger hypotheses on π. The representation π is called integrable with respect to a weight w if there exists an admissible vector ψ ∈ H such that Let g 1 , g 2 , g 3 ∈ G. We call a weight w : G → R + submultiplicative, if w(g 1 g 2 ) w(g 1 )w(g 2 ), and a function m : G → R + w-moderate, if it satisfies m(g 1 g 2 g 3 ) w(g 1 )m(g 2 )w(g 3 ). If m is w-moderate, the weighted Lebesgue space L p m (G) is then invariant under left translation L x f (y) = f (x −1 y) and under the right translation R x f (y) = f (yx). Throughout this paper, we will assume that the weight w satisfies , and ∆ denotes the modular function of G.
Our standing assumption is that the representation π of G possesses an admissible vector ψ such that V ψ ψ ∈ L 1 w (G). We denote the corresponding set by , the space of anti-linear continuous functionals on H 1 w . As H 1 w is dense in H, it follows that the inner product on H × H extends to (H 1 w ) ∼ × H 1 w and so does the generalized wavelet transform.
The coorbit space with respect to L p m (G) is then defined by , and is equipped with the natural norm . With our assumptions on π, ψ, m, the coorbit space Co π L p m (G) is a Banach space [16]. Alternatively, Co π L p m (G) for p < ∞ can be defined as the completion of H 1 w with respect to this norm. Moreover, In the context of coorbit space theory the space H 1 w serves as a space of test functions, and H ∞ 1/w is the corresponding distribution space.
We quickly recall some of the fundamental properties of coorbit spaces, see for example [16, . Then the following properties hold: where the duality is given by We furthermore need a result on the existence of atomic decompositions for the space Co π L 1 w (G), see [15,Theorem 4.7].
There exists a discrete subset {g i } i∈I ⊂ G and a collection of linear functionals λ i : and the sum converges absolutely in Co π L 1 w (G).

Frames and Coorbit Spaces via Tensor Products
Let G 1 , G 2 be two locally compact groups with unitary square integrable representations π 1 : G 1 → U(H 1 ) and π 2 : acts on a simple tensor Ψ := ψ 2 ⊗ ψ 1 ∈ H 2 ⊗ H 1 by It follows immediately that π is a unitary representation of G on H 2 ⊗ H 1 . Moreover, π is irreducible (e.g., by [34,Section 4.4,Theorem 6]). Note that the order of indices is in agreement with the formulation of the kernel theorem in Theorem 3.
If we interpret the simple tensor Ψ = ψ 2 ⊗ψ 1 as the rank one operator . In case we treat the tensor product as a space of Hilbert-Schmidt operators, π acts on A ∈ HS(H 1 , H 2 ) as π(g)A = π 2 (g 2 )Aπ 1 (g 1 ) * . The generalized wavelet transform of a simple tensor f 2 ⊗ f 1 with respect to a "wavelet" Ψ = ψ 2 ⊗ ψ 1 is given by Thus, the wavelet transform of the tensor product representation factors into the product of wavelet transforms on G 1 and G 2 . Strictly speaking, we would have to write V π i ψ i f i to indicate the underlying representation, but we omit the reference to the group to keep notation simple.

Kernel Theorems
In this section we derive the general kernel theorems for operators between coorbit spaces. The basic idea comes from linear algebra, where a linear operator is identified with its matrix with respect to a basis. In coorbit theory the basic structure consists of the vectors π(g)ψ. Thus in analogy to linear algebra we try to describe an operator A : H 1 → H 2 by the kernel (= continuous matrix) (14) k This can be seen as a continuous Galerkin like representation of the operator A [2,3]. The idea goes back to coherent state theory [30, Chpt. 1.6]. One of its goals is to associate to every operator A a function or symbol k A , and (14) is one of the many possibilities to do so. Assume that A : , and f ∈ Co π 1 L 1 w 1 (G 1 ) , i.e., A maps "test functions" to "distributions". By using the inversion formula (5) for f and applying A to it, it follows that formally be the integral operator with the kernel k A . Then (15) can be written as Using this factorization, the computation in (15) can be given a precise meaning on coorbit spaces. Identity (18) is the heart of the kernel theorems. The combination of the properties of the generalized wavelet transform (Proposition 1) and boundedness properties of integral operators yields powerful and very general kernel theorems.
We will first show the existence of a generalized kernel for operators mapping the space of test functions Co π 1 L 1 w 1 (G 1 ) into the distribution space Co π 2 L ∞ 1/w 2 (G 2 ) . Subsequently, we will characterize continuous operators in certain subclasses.
Theorem 3. Let G 1 and G 2 be two locally compact groups, and (π j , H j ) be integrable, unitary, irreducible representations of G j , such that A w j (G j ) = ∅, for j = 1, 2.
Uniqueness: Let us assume that the kernel K ∈ Co π L ∞ 1/w (G) also satisfies Aυ, ϕ = K, ϕ ⊗ υ = K, ϕ ⊗ υ , for every υ ∈ Co π 1 L 1 w 1 (G 1 ) , ϕ ∈ Co π 2 L 1 w 2 (G 2 ) . By Theorem 2, there exists a discrete set {γ i } i∈I ⊂ G such that every F ∈ Co π L 1 w (G) can be written as As this equality holds for every F ∈ Co π L 1 w (G), it follows that K = K. Surjectivity: Let us assume that A : We claim that k A is a generalized wavelet transform. Precisely, there exists K ∈ Co π L ∞ 1/w (G 1 ×G 2 ) such that k A = V Ψ K. To prove this claim, we use Proposition 1(v), which asserts that k A = V ψ K for some K ∈ Co π L ∞ 1/w (G) if and only if k A = k A * V Ψ Ψ. As k A · V Ψ (π(g)Ψ) ∈ L 1 (G 1 × G 2 ), we may choose the most convenient order of integration and apply the reproducing formula of Proposition 1 (v) consecutively to the representations π 1 and π 2 . Using (13) we obtain

At this point we note that by assumption on A there exists a unique operator
for every υ ∈ Co π 1 L 1 w 1 (G 1 ) and ϕ ∈ Co π 2 L 1 w 2 (G 2 ) . By its definition, A ′ is weak *continuous. We continue with the integration over G 1 and obtain By Proposition 1 (v) there exists a kernel K ∈ Co π L ∞ 1/w (G 1 ×G 2 ), such that k A (g 1 , g 2 ) = V Ψ K(g 1 , g 2 ). By the first part of the proof K defines an operator B : Consequently, Bπ 1 (g 1 )ψ 1 = Aπ 1 (g 1 )ψ 1 for all g 1 ∈ G 1 . This identity extends to all finite linear combinations of vectors π 1 (g 1 )ψ 1 and by Theorem 2 to Co π 1 L 1 w 1 (G 1 ). Thus B = A and we have shown that the map from kernels to operators is onto.
The map K → A is bounded and invertible. By the inverse mapping theorem we obtain that K Coπ L ∞ 1/w (G) C A Op , which proves (20). (19) correctly. For utmost precision, we would have to write

Remark 4. It is crucial to interpret the brackets in
, but we feel that this notation would distract from the analogy to distribution theory.
The injectivity of the mapping K → A from kernels to operators is closely related to an important property of the coorbit spaces Co π 1 L 1 w 1 (G 1 ) . This so-called tensor product property has gained considerable importance in certain special cases [13, Theorem 7D] and [26], we therefore state and prove a general version. Recall that the projective tensor product of two Banach spaces B 1 and B 2 is defined to be The norm is given as f ⊗ = inf i∈I φ i B 1 ψ i B 2 over all representations of f = i∈I φ i ⊗ ψ i . The following identification of the projective tensor product of Co π 1 L 1 w 1 (G 1 ) and Co π 2 L 1 w 2 (G 2 ) with the coorbit space Co π L 1 w (G 1 × G 2 ) is a generalization of Feichtinger's original result for modulation spaces [13, Theorem 7D].
Theorem 5. Under the general assumptions on the groups G i and the representations (π i , H i ) we have . Then by Theorem 2 applied to π = π 2 ⊗ π 1 , F possesses the representation Using Fubini's theorem and Proposition 1 (ii) yields The equivalence of the norms follows from the inverse mapping theorem.
Once the kernel theorem provides a general description of operators between test functions and distributions, we may try to characterize certain classes of operators by properties of their kernel. Since on the level of the generalized wavelet transform such operators correspond to integral operators (see diagram), we may translate the various versions of Schur's test to kernel theorems for operators between coorbit spaces. Following the procedure in [8,Theorem 3.3], we first formulate a general version of Schur's test and then derive abstract kernel theorem.
We introduce two classes of mixed norm spaces. For two σ-finite measure spaces (X, µ) and (Y, ν), 1 p ∞, and m : X × Y → R + , we define the spaces L p,∞ m (X × Y ), and L p,∞ m (X × Y ), by the norms In that case In this case We now characterize the boundedness of operators between certain coorbit spaces.
The following diagram shows the connection between the different operators and spaces.
Using interpolation between L p -spaces, Schur's test can also be formulated as saying that an integral operator is bounded on all L p simultaneously, if and only if its kernel belongs to L 1,∞ ∩ L 1,∞ . The corresponding version for coorbit spaces is a consequence of Theorem 7 and an interpolation argument.
Corollary 8. The following conditions are equivalent: Clearly one can now translate every boundedness result for an integral operator into a kernel theorem for coorbit spaces. As a simple, but important example we offer a sufficient condition for regularizing operators, i.e., of operators that map distributions to test functions. Theorem 9. Under the assumptions of Theorem 3, if the unique kernel of the operator A satisfies K ∈ Co π L 1 w (G), then A is bounded from Co π 1 L ∞ 1/w 1 (G 1 ) to Co π 2 L 1 w 2 (G 2 ). Proof. Consider the integral operator A as in the proof of Theorem 7 and observe for a compact neighborhood Q of e, then there exists a discrete subset Λ ⊂ G, and constants C 1 , C 2 > 0, such that Corollary 11. Let Λ = Λ 1 × Λ 2 ⊂ G a discrete set such that {π(λ)Ψ} λ∈Λ satisfies (28) for Co π L p,∞ m −1 1 ⊗m 2 (G) and Co π L p,∞ m −1 1 ⊗m 2 (G). If A is a bounded operator from with kernel K, then the following holds: Proof. (i) By Theorem 7 A has a kernel in Co π L p,∞ (G), and A Op ≍ K Coπ L p,∞ (G) . By (28), the expression in (29) is an equivalent norm for K Coπ L p,∞ (G) . The proof of (ii) works exactly the same.

Modulation spaces. The Weyl-Heisenberg group
Let T x f (t) := f (t − x) denote the translation, and M ω f (t) := e 2πiωt f (t) the modulation operator. The operator π W H (x, ω, τ ) = e 2πiτ M ω T x for (x, ω, τ ) ∈ G W H defines a unitary square-integrable representation of G W H acting on L 2 (R d ), for which every nonzero vector in L 2 (R d ) is admissible. Since the phase factor e 2πiτ is irrelevant for the definition of coorbit spaces, it is convenient to drop the trivial third component, and consider the time-frequency shift π(x, ω) = π W H (x, ω, 1) = M ω T x . Formally, we treat the projective representation π of R 2d instead of the unitary representation π W H of G W H . The transform corresponding to π is nothing else but the short-time Fourier transform The coorbit spaces associated to π W H coincide therefore with the coorbit spaces associated to π. These are the modulation spaces M p m (R d ) which were first introduced by Feichtinger in [14] as certain decomposition spaces and subsequently were identified with the coorbit spaces of the Heisenberg group [18]. We refer to the standard textbooks [20,23] for more information on time-frequency analysis.
Theorem 3 asserts that every bounded operator from M 1 Let us elaborate in detail what the kernel theorem asserts in this case: for g i = (x i , ω i , τ i ) ∈ G W H , i = 1, 2, the tensor representation π W H ⊗ π W H acts on the simple tensor ( Thus except for the phase factor e 2πi(τ 1 −τ 2 ) the tensor representation π W H ⊗ π W H is just the time-frequency shift M (ω 2 ,−ω 1 ) T (x 2 ,x 1 ) acting on L 2 (R 2d ). Consequently, the coorbit spaces with respect to the product group G W H ⊗ G W H are again modulation spaces, this time on R 2d . For the coorbit of L ∞ we compare the norms which are obviously equal. In this case Theorem 3 is therefore just Feichtinger's kernel theorem: The recent extension of Feichtinger's kernel theorem by Cordero and Nicola [8] can be seen in the same light. Let us explain the difference in the formulations. Our approach considers the generalized wavelet transform of the kernel. The conditions of Theorem 7 are formulated by mixed norms acting simultaneously on the variables (x 2 , ω 2 ) and on (x 1 , ω 1 ). The treatment in [8] uses the short-time Fourier transform on R 2d which is the same transform, except for the order of the variables. In [8] it was therefore necessary to reshuffle the order of integration of time-frequency shifts and to use the notion of mixed modulation spaces, which were studied in [4,29]. The new insight of our formulation is that the mixed modulation spaces are simply the coorbit spaces with respect to the tensor product representation.
The special case of Theorem 7 for the Weyl-Heisenberg group and the weights m s (x, ω, τ ) = (1 + |x| + |ω|) s for s ∈ R states the following: Fix σ > 0 and let A be an operator from M 1 Then for |r|, |s| σ, 1 p, q ∞ and 1/p + 1/q = 1 we have . Regularizing operators from M ∞ to M 1 were studied recently studied by Feichtinger and Jakobsen [19]: they characterized a subclass of this space of operators by an integral kernel in M 1 (R 2d ). The sufficiency of this result in a coorbit version is contained in Theorem 9.

Wavelet Coorbit Spaces and Besov
Spaces. The affine group G aff = R×R * is given by the group law (x, a)·(y, b) = (x+ay, ab) where x, y ∈ R and a, b ∈ R\{0}. Its left Haar measure is given by dxda a 2 . Let D a f (t) = a −1/2 f (t/a) denote the dilation operator. Then (x, a) → π aff (x, a) = T x D a defines an unitary, square-integrable representation of G aff on L 2 (R).
5.3. The Case of Two Distinct Representations. For most applications it suffices to consider a single group G and its product group G × G. Our formulation with two different group allows us to study operators acting between coorbit spaces associated with different group representations. Using the representations of the Weyl-Heisenberg group and the affine group of Sections 5.1 and 5.2, one can characterize the boundedness of operators between certain modulation spaces and Besov spaces by properties of their associated kernels. Theorem 7 now reads as follows: (i) A : M 1 ms (R d ) →Ḃ r p,p (R) bdd. ⇔ K ∈ Co π aff ⊗π W H L p,∞ m −s ⊗ν r+1/2−1/p (G W H × G aff ), (ii) A : M p ms (R d ) →Ḃ r ∞,∞ (R) bdd. ⇔ K ∈ Co π aff ⊗π W H L q,∞ m −s ⊗ν r+1/2 (G W H × G aff ), (iii) A :Ḃ r 1,1 (R) → M p ms (R d ) bdd. ⇔ K ∈ Co π W H ⊗π aff L p,∞ m −r+1/2 ⊗νs (G a × G W H ), (iv) A :Ḃ r p,p (R) → M ∞ ms (R d ) bdd. ⇔ K ∈ Co π WH ⊗π aff L q,∞ ν −r−1/2+1/p ⊗ms (G a × G W H ).
As a special case one obtains a characterization of the bounded operators A froṁ B r 1,1 (R) to L 2 (R d ). Since M 2 (R d ) = L 2 (R d ), they are completely characterized by the membership of their kernel in Co π aff ⊗π W H L 2,∞ 1⊗m −r+1/2 (G aff × G W H ).