Middleware Systems for Efficient and Robust Distributed Machine Learning

Distributed machine learning faces fundamental challenges arising from heterogeneous data, shifting client populations, and volatile inference workloads. This dissertation advances the thesis that middleware-not the models themselves-must serve as an active, adaptive decision layer that unifies training-time and inference-time robustness. On the training side, FLIPS performs privacy-preserving participant selection via label-distribution clustering within TEEs, improving fairness and convergence under non-IID data, while ShiftEx detects covariate and label shifts in streaming environments and dynamically creates or reuses experts to accelerate adaptation with minimal retraining. On the inference side, OptiSeq improves the reliability of LLM-based API sequencing by selecting example orderings using inference-time log-probability reasoning, and VOILA formulates multimodal fidelity selection as a value-of-information optimization, achieving large cost reductions with calibrated accuracy guarantees. Together, these systems establish adaptive middleware as a unifying approach for improving efficiency, robustness, and accuracy across modern distributed ML deployments.