On the variety of invariant subspaces of a finite-dimensional linear operator

Abstract:

If $V$ is a finite-dimensional vector space over $\mathbf {R}$ or $\mathbf {C}$ and $A \in {\operatorname {Hom}}(V)$, the set ${S_A}(k)$ of $k$-dimensional $A$-invariant subspaces is a compact subvariety of the Grassmann manifold ${G^k}(V)$, but it need not be a Schubert variety. We study the topology of ${S_A}(k)$. We reduce to the case where $A$ is nilpotent. In this case we prove that ${S_A}(k)$ is connected but need not be a manifold. However, the subset of ${S_A}(k)$ consisting of those subspaces with a fixed cyclic structure is a regular submanifold of ${G^k}(V)$.