Ordinary primes for some varieties with extra endomorphisms

Let A be an abelian variety defined over a number field and of dimension g. When g <= 2, by the recent work of Sawin, we know the exact (nonzero) value of the density of the set of primes which are ordinary for A. In higher dimension very little is known. We show that if g = 3 and A has multiplication by an imaginary quadratic field E, then there exists a nonzero density set of ordinary primes for A. We reach the same conclusion if g = 4 and the pair (A, E) has signature (2, 2). We also obtain partial results when g = 3 and A has multiplication by a totally real cubic field. We show that our methods also apply to certain abelian varieties of Albert type IV of higher dimension. These results are derived from an extended version of the P-adic methods of Katz, Ogus, and Serre in the presence of extra endomorphisms.