Solving constrained multi-objective optimization problems (CMOPs) is a challenging task, because they need to optimize multiple conflicting objectives and satisfy several constraints. In the past several years, though many constrained multi-objective evolutionary algorithms (CMOEAs) have been proposed to deal with CMOPs, they still have some limitations. For instance, different CMOEAs may be required for CMOPs with different characteristics. To tackle this issue, this paper proposes a Pareto fronts (PFs) guided co-evolutionary algorithm (called PFCEA) for CMOPs. Inspired by competitive multitasking (CMT), PFCEA employs two populations with specialized roles. The first population aims to use the unconstrained PF (UPF) and the most useful single-constrained PF (SPF) to find the constrained PF (CPF). To choose the most useful SPF, a novel single constraint priority (SCP) method is designed. The second population considers total constraint violation and objective values by using an improved fuzzy constraint handling technique. During the search, a reward accumulation method is used for the co-evolution of two populations. To verify the performance of PFCEA, four popular benchmark sets and six real-world CMOPs are tested. The performance of PFCEA is compared with eleven other state-of-the-art CMOEAs. Experimental results demonstrate that the proposed PFCEA is competitive in solving CMOPs with different properties.