The title refers to the fact, noted by Chebyshev in 1853, that primes congruent to 3 modulo 4 seem to predominate over those congruent to 1. We study this phenomenon and its generalizations. Assuming the Generalized Riemann Hypothesis and the Grand Simplicity Hypothesis (about the zeros of the \hbox{Dirichlet} $L$-function), we can characterize exactly those moduli and residue classes for which the bias is present. We also give results of numerical investigations on the prevalence of the bias for several moduli. Finally, we briefly discuss generalizations of the bias to the distribution to primes in ideal classes in number fields, and to prime geodesics in homology classes on hyperbolic surfaces.