Enumerating classes and characters of p-groups

We develop general formulae for the numbers of conjugacy classes and irreducible complex characters of finite p-groups of nilpotency class less than p. This allows us to unify and generalize a number of existing enumerative results, and to obtain new such results for generalizations of relatively free p-groups of exponent p. Our main tools are the Lazard correspondence and the Kirillov orbit method.


Introduction
The study of the conjugacy classes and irreducible complex characters of groups is an active area of research. The enumeration of classes and characters of finite groups of Lie type, for instance, has played an important role in the work of Liebeck, Shalev and others; see, for instance, [25]. Motivated by a conjecture of Higman [13], the classes and characters of upper-unitriangular groups have been extensively studied; see, for example, [18,30].
'Representation growth of groups' is an umbrella term for the asymptotic and arithmetic properties of group representations as a function of their dimensions. A key tool in the study of representation growth is the Kirillov orbit method. Where applicable, it provides a parameterization of the irreducible complex representations of a group in terms of co-adjoint orbits. It was pioneered by Kirillov in the realm of nilpotent Lie groups and later adapted to other classes of groups, including p-adic analytic groups, finitely generated nilpotent groups, and finite p-groups; see [10,14,15,20]. Under certain conditions the linearization achieved by this method facilitates a description of the numbers of characters of a group in terms of geometric data attached to the dual of a Lie algebra associated with the group, such as the numbers of rational points of certain algebraic subvarieties.
Let p be a prime. In this paper we employ the Kirillov orbit method to study the classes and characters of finite p-groups of nilpotency class less than p. Let G be a finite p-group. For i ≥ 0, we define cc i (G) = #{conjugacy classes of G of cardinality p i } and ch i (G) = #{irreducible complex characters of G of degree p i }.
The vectors cc(G) = (cc i (G)) i and ch(G) = (ch i (G)) i are the class vector and the character vector of G, respectively. We denote by cs(G) = {p i : cc i (G) = 0} the class sizes of G and by cd(G) = {p i : ch i (G) = 0} the character degrees of G. We write for the class number of G.
Let c be the nilpotency class of G, and assume that c < p. Let g = log(G) be the finite Lie ring associated to G by the Lazard correspondence. We associate to G a subset S(G) of g/z × g ′ , where z denotes the centre of g and g ′ = Hom Z (g ′ , C × ) the Pontryagin dual of the derived Lie ring g ′ . In Theorem A we show that the class and conjugacy vectors of G may be described in terms of the cardinalities of fibres of the natural projections from S(G) onto g/z and g ′ .
Theorem B gives a geometric description of the class and character vectors of certain p-groups and describes the variation of these vectors under 'extension of scalars'. More precisely, let o be a compact discrete valuation ring of characteristic zero with residue field k of characteristic p. Theorem B asserts that if g is a finite, nilpotent o-Lie algebra of class c < p, and g ′ or, equivalently, g/z, is a k-vector space, then computing class and character vectors of the p-group exp(g) associated to g under the Lazard correspondence is equivalent to enumerating k-rational points of degeneracy loci of certain 'commutator matrices' associated with g. Moreover, the formulae given in Theorem B are uniformly valid for groups of the form exp(g ⊗ o O), where O is a finite, unramified extension of o.
The Lie algebra g may be obtained by base change from a globally defined object, such as a nilpotent Z-Lie algebra. For some of the groups obtained from such Lie algebras, Theorem B yields formulae which are uniform under variation of both the cardinality and the characteristic of the residue field. Consider, for instance, the free F q -Lie algebras f r,c (F q ) on r generators and of nilpotency class c, where F q is a finite field of characteristic p > c. These algebras are of the form f r,c (F q ) = f r,c (Z) ⊗ Z F q , where f r,c (Z) is the free nilpotent Z-Lie algebra of class c on r generators. Theorem B applies to the groups F r,c (F q ) := exp(f r,c (F q )).
In Section 2 we state Theorems A and B, together with some applications to groups of the form F r,c (F q ). Our main tools are the Lazard correspondence for p-groups of nilpotency class c < p and the Kirillov orbit method for such groups. In Section 3 we review these tools and use them to prove Theorems A and B. In Section 4 we apply these results to uniformize a number of existing enumerative results on classes and characters of p-groups. In Section 5 we prove new results for the groups F r,c (F q ), including those stated in Section 2. They extend and generalize results of Ito and Mann [19] for the relatively free groups of exponent p.
1.1. Notation. We denote the cardinality of a set S by either #S or |S|. We write N for the set {1, 2, . . . } of natural numbers. For I ⊆ N and c ∈ R, we write I 0 for I ∪ {0} and cI 0 for {ci : i ∈ I 0 }. Given a, b ∈ N 0 we define [a] = {1, . . . , a} and [a, b] = {a, . . . , b}. For x ∈ R we set ⌊x⌋ := max{m ∈ Z : m ≤ x}. If I is any ordered set then we write I = {i 1 , . . . , i l } < to indicate that i 1 < · · · < i l . Given a proposition P , the 'Kronecker delta' δ P is 1 if P holds and 0 otherwise. If n 1 , . . . , n r ∈ N 0 and f ∈ N, we write (n 1 , . . . , n r ) f for the vector if f = 1 we drop the subscript. Given a ring R, an R-Lie algebra g is an R-algebra with a 'Lie bracket', that is to say an R-bilinear map [ , ] : g × g → g which is skew-symmetric and satisfies the Jacobi identity. A Lie ring is a Z-Lie algebra. We write [u, v, w Throughout this paper, o is a compact discrete valuation ring of characteristic zero, viz. a finite extension of the p-adic integers Z p , with maximal ideal p and residue field k = o/p of characteristic p. An arbitrary field is denoted by K.
The centre and derived group of a group G are denoted by Z (or Z(G)) and G ′ respectively; the centre and derived ring of a Lie algebra g are z (or Z(g)) and g ′ . We write [ , ] also for the induced map g/z × g/z → g ′ , (x + z, y + z) → [x, y]. Given g ∈ G and x ∈ g we write C G (g) and C g (x) for the respective centralizers.
Given a ring R and integers m and n, we write Mat(n × m, R) for the n × m-matrices over R. We abbreviate Mat(n×n, R) to Mat(n, R). We denote the transpose of a matrix A by A tr .
By a character of a group we always mean a complex irreducible character.

The main results
The Lazard correspondence establishes an order-preserving one-to-one correspondence between finite p-groups of nilpotency class c < p on the one hand and finite nilpotent Lie rings of p-power order and class c < p on the other; cf. [23,Example 10.24]. More precisely, one may define a group operation on such a Lie ring g by the formula where F i (X, Y ) is the homogeneous part of degree i of the Hausdorff series F (X, Y ), an element in the completion of the free Q-Lie algebra on variables X and Y ; cf. [23, §9.2]. Then exp(g) := (g, ⋆) is a p-group of class c. The theorem underlying the Lazard correspondence asserts that the isomorphism type of every p-group G of class c < p arises in this manner from a Lie ring g, unique up to isomorphism. We denote the map underlying a fixed isomorphism exp(g) ∼ = G by exp : g → G, and write log for its inverse. We write g ′ for the Pontryagin dual Hom Z (g ′ , C × ) of the finite abelian p-group g ′ .
Theorem A. Let G be a finite p-group of nilpotency class c < p and let g = log(G) be the corresponding Lie ring. Define with projections π 1 : S(G) → g/z and π 2 : S(G) → g ′ . For i ≥ 0, In particular, the class number For a certain family of groups, Theorem B exploits this result to provide a uniform description of the class and character vectors in terms of the numbers of rational points of rank varieties of matrices of linear forms. We now formulate this more precisely.
Assume that o is a compact discrete valuation ring of characteristic zero and residue characteristic p, and that g is a finite, nilpotent o-Lie algebra of class c < p. Set Definition 2.1. Let X = (X 1 , . . . , X a ) and Y = (Y 1 , . . . , Y b ) be independent variables. We define commutator matrices (with respect to e and f ) of o-linear forms in X and Y, namely If g is a K-algebra with K-basis B = (e 1 , . . . , e h ) such that the residue classes of the elements e 1 , . . . , e a form a K-basis e for g/z and f = (e h−b+1 , . . . , e h ) is a K-basis for g ′ then we refer to the associated commutator matrices A and B as 'with respect to B'. Assume now that g/z or, equivalently, g ′ is annihilated by p, the maximal ideal of o. We write k for the residue field o/p of characteristic p. The set of generators f for g ′ may be regarded as a k-basis for the k-vector space g ′ . Similarly, we view e as a k-basis for the k-vector space g/z.
The commutator matrices A and B may be considered as matrices of linear forms over k. Let K be an extension of k. For x = (x 1 , . . . , x a ) ∈ K a we write A(x) ∈ Mat(a × b, K) for the matrix obtained by evaluating the variables X i at x i . Likewise B(y) ∈ Mat(a, K) is defined for y = (y 1 , . . . , y b ) ∈ K b . We note that the ranks of matrices of the form B(y), for y ∈ K b , are even integers.
It is well-known that every finite field k is self-dual, i.e. (noncanonically) isomorphic to its Pontryagin dual. Indeed, let ψ : k → C × be a nontrivial additive character of k. For a ∈ k define ψ a (x) = ψ(ax) for x ∈ k. The map a → ψ a is an isomorphism between k and its Pontryagin dual k; cf., for instance, [4]. Since g ′ ∼ = k b , this yields an isomorphism between g ′ and its dual g ′ . On the other hand there is, of course, a -likewise noncanonical -isomorphism between g ′ and its linear dual Hom k (g ′ , k). We fix an isomorphism ψ 1 : g ′ → Hom k (g ′ , k). The dual k-basis f ∨ = (f ∨ k ) for Hom k (g ′ , k) gives a coordinate system Similarly, the k-basis e for g/z gives a coordinate system x j e j → x = (x 1 , . . . , x a ). Assume further that O is an unramified extension of o, with maximal ideal P. We identify the residue field O/P, a finite extension of k, with F q . The O-Lie algebra g(O) inherits the property that the derived algebra and the cocentre of g(O) are annihilated by P. We consider e and f as F q -bases for the respective F q -vector spaces of dimensions a and b. Set G(O) := exp(g(O)). Note that our assumption on g implies that both G(O) ′ and G(O)/Z(G(O)) have exponent p. Our second main result gives a uniform description of the set S(G(O)) introduced in Theorem A -and therefore for the class and character vectors of G(O) -in terms of the numbers of F q -rational points of degeneracy loci of the commutator matrices A and B.
Theorem B. Let o be a compact discrete valuation ring of characteristic zero and residue field k of characteristic p, and let g be a finite, nilpotent o-Lie algebra of class c < p. Assume that g ′ ∼ = k b and that g/z ∼ = k a as k-vector spaces. Let O be a finite, unramified extension of o, with residue field isomorphic to F q . The class sizes and character degrees of We illustrate Theorem B with a well-known example.
In Section 5 we study generalizations of the relatively free p-groups of exponent p. For integers r ≥ 2 and c ≥ 1 we consider the free F q -Lie algebra f r,c (F q ) on r generators and nilpotency class c, where q = p f is a power of a prime p > c. The Lazard correspondence associates the p-group F r,c (F q ) = exp(f r,c (F q )) to this F q -Lie algebra. Our approach yields, for instance, a simple, geometric proof of the following generalization of [19,Theorem 5] and of Example 2.3.
. Proof. We fix an F q -basis (x 1 , . . . , x r , y kℓ : 1 ≤ k < ℓ ≤ r) for f 2,r (F q ), subject to the relations [x ℓ , x k ] = y kℓ for 1 ≤ k < ℓ ≤ r. Note that a = r and b = r 2 . The commutator matrix B(Y) with respect to this basis is the generic skew-symmetric matrix in variables Recall that the 'Witt formula' is defined, for i ∈ N, by where µ denotes the Möbius function; cf., for example, [12,Chapter 11]. We define Theorem 2.5. Assume that (r, c) = (2, 3), that p > c and let q be a power of p. The character degrees of F r,c (F q ) are 1, q, q 2 , . . . , q n(r,c) .
The character vectors of the groups F 2,3 (F q ) are given in Proposition 5 Theorem 2.6. Assume p > c and let q be a power of p. The class sizes of and cc 0 (F r,c (F q )) = |Z(F r,c (F q ))| = q Wr(c) .
Observe that the function i → k(r, c, i) is injective unless r = 2 and c ∈ {3, 4}; in these cases the sum in (2.2) has at most two nonzero summands. Generically it has at most one. Theorems 2.5 and 2.6 will be proven in Section 5.

Proofs of Theorems A and B
The Lazard correspondence between p-groups and Lie rings of nilpotency class c < p allows us to linearize the problem of enumerating conjugacy classes and characters. Let G be a finite p-group of nilpotency class c < p, with associated Lie ring g = log(G).

3.1.
Counting conjugacy classes. It follows from straightforward calculations with the Hausdorff series that log induces an order-preserving correspondence between subgroups of G and subalgebras of g, and log maps normal subgroups to ideals. In particular, |G/Z| = |g/z| and |G ′ | = |g ′ |, and centralizers in G correspond to centralizers in g. Thus The last equality reflects the fact that the centralizer of an element only depends on its coset modulo the centre. For x ∈ g/z we define

3.2.
Kirillov's orbit method and counting characters. The Kirillov orbit method offers a linearization of the character theory of G in terms of co-adjoint orbits: characters of G correspond to orbits in g := Hom Z (g, C × ), the Pontryagin dual of g, under the co-adjoint action Ad ⋆ of G on g. The following is well-known; see, for example, [ Theorem 3.1. Let G = exp(g) be a finite p-group of nilpotency class c < p. Let Ω ⊆ g be a co-adjoint orbit and ω ∈ Ω.
(1) There exists a polarizing subalgebra h ⊆ g for the bi-additive, skew-symmetric (2) Viewing ω as a function on G (via log), the function ω| H is a one-dimensional representation of H. The induced representation U Ω := Ind G H ω of G is irreducible, independent of ω, and has dimension |Ω| 1/2 . All irreducible complex representations of G have this form.
(3) The character of U Ω is given by |Ω| −1/2 ω∈Ω ω(g), for g ∈ G. Remark 3.2. A Kirillov orbit method for torsion-free finitely generated nilpotent pro-p groups of class 2 that holds for all primes p is presented in [27,Section 2.4]. We expect that it can be used to prove the conclusions of Theorem 3.1 for 2-groups of class 2.
Theorem 3.1 reduces the problem of enumerating the characters of G to that of computing the indices in g of the radicals Rad(B ω ), as ω ranges over g. In fact, given ω ∈ g, the form B ω only depends on the restriction of ω to g ′ . Given ω ∈ g ′ we therefore write B ω for B ω , where ω ∈ g is any extension of ω. With this notation, Theorem 3.1 implies that Using the natural projections π 1 : S(G) → g/z and π 2 : S(G) → g ′ , we see that . We obtain two descriptions of the class number k(G): This proves Theorem A.

Proof of Theorem B.
Recall that o is a compact discrete valuation ring with residue field k = o/p of characteristic p, and that g is a finite, nilpotent o-Lie algebra of class c < p with the property that g/z and g ′ are annihilated by p. Further recall the isomorphisms ϕ : g/z → k a and ψ : g ′ → k b introduced in Section 2. Consider the p-group G = exp(g). By (3.3), The following lemma, proved analogously to [1,Lemma 3.3], characterizes membership of S(G) in terms of the above coordinate systems for g/z and g ′ . This proves Theorem B.

Consequences of Theorems A and B
4.1. Isoclinism. Recall from [11] that two p-groups G 1 and G 2 are isoclinic if there are isomorphisms θ : pair (θ, ϕ) is an isoclinism between G 1 and G 2 .
If G 1 and G 2 have nilpotency class less than p and (θ, ϕ) is an isoclinism between G 1 = exp(g 1 ) and G 2 = exp(g 2 ), then there is a pair of associated maps (Θ, . This, of course, merely reflects the well-known fact that isoclinic groups have, up to multiplication by p-powers, identical class (and character) vectors. [3] studied the class vectors of some pgroups of class 2 and exponent p. In this section we prove a generalization and extension of [3, Theorem 3.2]. We first describe our broader context. Let o be a compact, discrete valuation ring with residue field o/p, which we identify with F q , where q = p f is an odd prime power. Let g be a finite, nilpotent o-Lie algebra of class 2. Assume that g/z and g ′ are annihilated by p, so that Theorem B applies. The coordinate systems introduced in Section 2 identify g/z with F a q and g ′ with F b q , where we write a for the o-rank of g/z and b for the o-rank of g ′ . Recall from Definition 2.1 the commutator matrix B associated to g with respect to the chosen bases. We denote by P b−1 (F q ) the (b − 1)-dimensional projective space over F q . Note that rk(B( y)) is well-defined for y = ( y 1 : · · · : y b ) ∈ P b−1 (F q ). We write G = exp(g) and recall that, by Theorem B, cs(G) and ch(G) consist of powers of q = p f . Then

Pfaffian hypersurfaces. Boston and Isaacs
In particular The claim about ch(G) then follows from Theorem B which asserts that ch To prove (4.2) it suffices to show that, firstly, µ if (G) = 0 for i ∈ [b − 2] and, secondly, µ (b−1)f (G) = n(q 2 − 1). Indeed, clearly µ 0 (G) = 1 and b i=0 µ if (G) = q a , so that µ bf (G) = q a − 1 − n(q 2 − 1). The claim about cc(G) then follows from Theorem B which asserts that cc Given y ∈ F b q we view B(y) as the matrix of an endomorphism of F a q , whose kernel we denote by ker(B(y)). Likewise, given x ∈ F a q , we view A(x) as the matrix of the linear map F b q → F a q , y → yA(x) tr , whose kernel we denote by ker(A(x)). Let y ∈ F b q be one of the n(q −1) elements with rk(B(y)) = a−2, so dim(ker(B(y))) = 2. Observe that rk(A(x)) < b for all x ∈ ker(B(y)). We claim that rk(A(x)) = b − 1 for all such x which are nonzero. Indeed, assume that x = 0 with rk(A(x)) ≤ b − 2. Let V ≤ F b q be a 2-dimensional subspace of ker(A(x)). For every y ∈ V we deduce using Lemma 3.3 that yA(x) tr = xB(y) tr = 0. Therefore V defines a line in P b−1 (F q ) on which no point y satisfies rk(B( y)) = a, contradicting our hypotheses. Thus rk(A(x)) = b − 1. This shows that µ if (G) = 0 for i ∈ [b − 2], establishing the first claim.

This follows from inspection of the commutator matrices
The class vector differs from (4.2), but the character vector agrees with (4.3). The hypothesis of Theorem 4.1 regarding lines in P 3 (F q ) is not satisfied. We observe that the factor 2(q + 1) of cc 2f (G) is the number of lines on the Pfaffian hypersurface, the quadric surface defined by

Prescribing class sizes and character degrees.
It is known that every finite set of p-powers containing 1 can be realized as the class sizes or character degrees of a finite p-group; cf. [7] and [17] respectively. Such results can be obtained readily using Theorems A and B. Throughout this section let p be an odd prime.

4.3.1.
We show how to obtain the result of [17]. Let I ⊂ N be finite and let j = max(I).
To construct a p-group G such that cd(G) = {p i : i ∈ I 0 }, consider the F p -Lie algebra g, with F p -basis consisting of x 1 , . . . , x 2j and y i for i ∈ I, subject only to the relations The commutator matrix B(Y) ∈ Mat(2j, F p [Y]) in variables Y i for i ∈ I with respect to this basis is the sum of the (2j × 2j)-matrices  We show how to construct such a group G. Consider the F p -Lie algebra g, with F p -basis (x 1 , . . . , x l , x 1 , . . . , x l+n−1 , y 1 . . . , y n ), subject only to the relations: With respect to this basis the commutator matrix The commutator matrix A(X) ∈ Mat((2l + n − 1) × n, Note that {rk(B(y)) : y ∈ F n p } = {0, l} and {rk(A(x)) : x ∈ F 2l+n−1 p } = {0, 1, . . . , l, n}. Therefore cd(exp(g)) and cs(exp(g)) are as stated.

Relatively free p-groups of exponent p
Let p be a prime and let r ≥ 2, c ≥ 1. Ito and Mann [19] study the numbers of classes and characters of the relatively free p-groups in the variety of groups of exponent p and nilpotency class c on r generators. Our methods apply when c < p. We consider, more generally, the groups F r,c (F q ) := exp(f r,c (F q )), where q = p f and f r,c (F q ) is the free nilpotent F q -Lie algebra of class c on r generators. The orders of the terms of the lower central series of f r,c (F q ), and so of F r,c (F q ), may be expressed in terms of the Witt formula (2.1): for i ∈ [c], (5.1) |γ i (f r,c (F q )) : γ i+1 (f r,c (F q ))| = |γ i (F r,c (F q )) : γ i+1 (F r,c (F q ))| = q Wr(i) ; cf. [19,Proposition 1]. We often write F for F r,c (F q ), f for f r,c (F q ) and W for W r .
5.1. Proof of Theorem 2.6. We first prove a lemma about free Lie algebras over arbitrary fields. Let K be a field and L a free K-Lie algebra of rank at least 2. We fix a Lie basis B for L and, for m ∈ N, define L m as the K-linear span of m-fold Lie products of elements of B. The standard grading L = ⊕ ∞ m=1 L m determines the lower central series filtration L i := ⊕ ∞ j=i L j of L. By convention, L i := L for i ≤ 0. Let u = ∞ i=1 u i ∈ L be a nonzero element in the standard grading, i.e. u i ∈ L i for all i. We denote by u the nonzero homogeneous component of lowest degree of u in the standard grading: namely, u := u deg(u) , where deg(u) := min{i ∈ N : u i = 0} is the degree of u.
Proof. Without loss of generality, assume v ∈ Ku. By the Shirshov-Witt theorem [26,Theorem 2.5], every subalgebra of L is free, and so [u, v] If [u, v] = 0 then the Shirshov-Witt theorem implies that v = ku for some nonzero k ∈ K. Thus deg(v − ku) > deg u. Since v − ku ∈ Ku and so [u, v − kv] = 0, we may apply the argument of the previous paragraph to v − ku instead of v, deducing that v − ku ∈ L i+1−deg u .
The Lazard correspondence implies the following.
We now prove Theorem 2.6. By Lemma 5.2, the conjugacy class sizes in F = F r,c (F q ) are the indices of the subgroups g, . Thus if g ∈ γ i (F ) \ γ i+1 (F ) then the conjugacy class of g has size q k(r,c,i) . The statement that cc 0 (F ) = q W (c) follows immediately from (5.1), as Z(F ) = γ c (F ). Note that 0 = k(r, c, c). To determine cc jf (F ) where j = k(r, c, i) = 0, it suffices to count the elements in each γ i (F ) \ γ i+1 (F ) such that k(r, c, i) = j and to observe that these elements fall into conjugacy classes of equal size q j . Thus This concludes the proof of Theorem 2.6.
Corollary 5.3. Let r ≥ 2, c ≥ 1 and q a power of p > c. The entries of the class vector cc(F r,c (F q )), and hence also the class number k(F r,c (F q )), are given by a polynomial in q which depends only on r and c. Expanded in q − 1, this polynomial has nonnegative coefficients.
Proof. Theorem 2.6 shows that the relevant quantities may be written as sums of terms of the form q α and (q β − 1)q γ for nonnegative integers α, β, γ.
Remark 5.4. Corollary 5.3 may be compared to an analogous conjecture about the class vectors of the groups U n (F q ) of upper-unitriangular matrices over F q ; cf. [30]. Isaacs [18] formulates a similar conjecture for the characters vectors of these groups; it is proved in [8] for n ≤ 13. In Remark 5.12 we note that the corresponding statement for the character vectors of the groups F 2,5 (F q ) is false.

5.2.
Proof of Theorem 2.5. We recall the well-known definition of a Hall basis; cf. [21].
r } be a set of Lie algebra generators for f = f r,c (F q ). If u ∈ f is a Lie product of elements from ∆ then u has weight wt(u) = i if u ∈ γ i (f) \ γ i+1 (f). A Hall basis (on ∆) for f is a well-ordered subset H of f, satisfying the following.
1 ] ∈ H (i) . Choose a Hall basis H for f. It is well-known that the elements of c i=2 H (i) yield an F q -basis for the derived Lie algebra f ′ , and that the residue classes of the elements of c−1 i=1 H (i) yield an F q -basis for the cocentre f/Z(f). Observe that the commutator matrix B(Y) ∈ Mat c−1 j=1 W (j), F q [Y] with respect to H is a skew-symmetric matrix of Z-linear forms in b = c j=2 W (j) variables. We label the variables as follows. For [2,c] and For each k ∈ [2, c], the variables Y (k) only occur in the matrices B i,j with i + j = k. It follows from [28, Theorem 1] that • if j > i and i does not divide j then B i,j is generic: there are no linear relations among its entries; • if i = j then B i,i is generic skew-symmetric: the only linear relations between its entries are those resulting from the identity B i,i = −B tr i,i . To prove Theorem 2.5 it suffices, by Theorem B, to show that (5.4) rk(B(y)) : The containment ⊆ in (5.4) is clear, as the rank of B(y) is clearly bounded from above by 2n(r, c). We establish the containment ⊇ in (5.4) by induction on c. For c = 1 there is nothing to prove, and the case c = 2 is covered by Proposition 2.4, so let c > 2. The induction step is divided into five steps. To ensure that the induction hypothesis is applicable, we assume further that (r, c) = (2, 4). The statement of Theorem 2.5 for groups of the form F 2,4 (F q ) follows from Proposition 5.10.
Step 1: By the induction hypothesis, we can obtain every rank in 2[n(r, c − 1)] 0 by setting the 'new' variables Y (c) to zero, and arguing as for c − 1.
Step 2: Let ρ ∈ 2[n(r, c − 1) + 1, n(r, c)]. If there exists a vector y = (y (k) ) k∈ [2,c]  Step 3: We now prove that such a vector y exists. As (r, c) = (2, 3), W (i) < W (j) whenever i < j. It suffices to show that, for each i < c/2, the matrix B c−i,i has a square submatrix Step 4: For i < c/2 we now exhibit such a submatrix B c−i,i of B c−i,i . By definition of the commutator matrix B, the matrix Step 5: To find such a subset S of [W (c − i)], we distinguish three cases.
This concludes the proof of Theorem 2.5.

5.3.
Taketa bounds for F r,c (F q ). The Taketa problem asks for a bound to the derived length dl(G) of a finite solvable group G in terms of the number of its character degrees; see, for example, [22]. It is known that dl(G) is bounded by a linear function in | cd(G)|. Isaacs conjectured that the bound for p-groups is logarithmic. It cannot be better than logarithmic, as the family (U n (F q )) shows: dl(U n (F q )) = ⌈log 2 (n)⌉, but | cd(U n (F q ))| ∼ n 2 /4; cf. [16].
We also observe that there is a logarithmic bound to the derived length of the groups F r,c (F q ) in terms of their numbers of class sizes. In fact, | cs(F r,c (F q ))| = c + 1 (unless both r and c are very small), so dl(F r,c (F q )) ≤ c 3 log(| cs(F r,c (F q ))|). The (log q of the) class sizes of the groups U n (F q ) form an interval of length n−1 2 (cf. [29]), also yielding a logarithmic bound for this family. 5.4. Numbers of characters. Theorem 2.5 describes the support of the character vectors ch(F r,c (F q )), showing that the numbers ch if (F r,c (F q )), for i ∈ [n(r, c)] 0 , are nonzero. We make one observation on the order of magnitude of the number of characters of maximal degree q n(r,c) . Define 2n(r, c).  Proof. The Lang-Weil estimate (cf. [24]) for the number of rational points on varieties over finite fields implies that ch n(r,c)f (F r,c (F q )) ∼ q r−2n(r,c) |F r,c (F q ) ′ | = q N (r,c) .
By Theorem 2.5, the smallest degree of a nonlinear character of F r,c (F q ) is q = p f . We now count the number of characters of F r,c (F q ) having degree q = p f , so generalizing [19,Theorem 7]. Proposition 5.7. Let r ≥ 2, p > c > 2 and q = p f . Then Proof. Let B(Y) be the commutator matrix with respect to a Hall basis for f r,c (F q ), and recall the definition (5.3) of the matrices B i,j . For y ∈ F b q we define For s ∈ [2, c], we now compute the quantity n s := #{y ∈ F b q : u(y) = s, rk(B(y)) = 2}. This suffices, as ch f (F r,c (F q )) = q r−2 ν f (F r,c (F q )) = q r−2 c s=2 n s . For s = 2, Proposition 2.4 for k = 1 implies that n 2 = (q r − 1)(q r−1 − 1)/(q 2 − 1). For s > 2, we claim that First note that if u(y) = s and rk(B(y)) = 2, then rk(B 1,s−1 (y)) = 1 and B i,s−i (y) = 0 for i ∈ [2, s − 2]; see, for example, [28,Theorem 1]. In fact, after a suitable change of basis for f r,c (F q ), we may assume that B(y) has zero entries everywhere except the first row and column. We claim that Indeed, there are q r−1 ways to fill in a row of B 1,s−1 (Y) so that all other rows are zero. To see this, assume without loss of generality that this is the first row, and note that exactly r − 1 of the Lie products of the form [e The variables associated to these occur in some other row of B 1,s−1 (Y), or in some B i,s−i (Y) for i ≥ 2, and so have the value zero. Up to nonzero scalars, there are thus (q r−1 − 1)/(q − 1) ways to fill a row without obtaining a zero row. Every row of B 1,s−1 (Y) is a linear multiple of such a nonzero row, and only one of the q r possibilities yields the zero matrix. This establishes the claim.
We also claim that, for each y ∈ F W (s) q such that rk(B 1,s−1 (y)) = 1 and B i,s−i (y) = 0 for i ∈ [2, s − 2], there are q (r−1)(s−2) ways to choose y ′ ∈ F b q such that rk(B(y ′ , y)) = 1. Indeed, again without loss we may assume that B 1,s−1 (y) is supported only on its first row. By the arguments in the previous paragraph, each of the matrices B 1,i (Y), for i ∈ [s − 2], has exactly r − 1 variables corresponding to basic commutators in its first row. All other entries in the first row are linear combinations of variables corresponding to basic commutators occurring in other rows. This establishes the claim, and so (5.6).

5.5.
Results on F r,c (F q ) for specific values of r and c. We start with a lemma generalizing the opening remarks of [19,Section 3], thus dealing with the exceptional parameter values in Theorem 2.5.
We obtain the following generalization of [19,Lemma 14].
Proof. Note that n(2, 4) = 2. The formula for ch f (F 2,4 (F q )) is given by Proposition 5.7 and the class number k(F 2,4 (F q )) is given by Theorem 2.6.
Proposition 5.11. Let p ≥ 7 and let q = p f . The nonzero values of ch i (F 2,5 (F q )) are given as follows.
We now describe a Hall basis for f 2,5 (F q ). We choose Lie generators x and y, where y < x, and omit Lie brackets in left-normed Lie products, so, for example, [[x, y], x] is represented by xyx. It is easily verified that the following elements form a Hall basis.
We observe this phenomenon only for the family of groups F 2,5 (F q ), for p ≥ 7; in all other families we considered the corresponding coefficients are nonnegative.