Federated learning (FL) empowers privacy-aware consumer electronics but remains vulnerable to backdoor attacks that compromise safety-critical applications. We present SpecGuard, a server-side defense designed for communication-constrained consumer electronics and AIoT systems. SpecGuard characterizes client updates via a parameter-only sensitivity spectrum and filters malicious participants using the Wasserstein-1 distance to a benign prototype. Crucially, this approach requires no auxiliary data, trigger knowledge, client-side modification, or additional client communication. We evaluate SpecGuard on MNIST, CIFAR-10, and CIFAR-100 against representative adaptive attacks. Experiments using a MobileNet backbone, reduced-participation scenarios, dispersion-gate sensitivity analysis, anomaly-score visualization, and optimized server-side runtime measurement show that SpecGuard mitigates backdoors under the evaluated attacks while introducing no extra client communication and only limited server-side overhead.