Scalable HVAC Flexibility Aggregation with DRL and Linear Programming

Large-scale HVAC demand response can provide grid flexibility, but market participation requires fast aggregate allocation, device-level feasibility and protection of occupant comfort. Although recent research has advanced building-flexibility assessment, learning-based control and distributed service provision, evidence is still limited for allocating a post-clearing dispatch request across many lower-level HVAC aggregators while accounting for accumulated thermal comfort burden. Here we develop a hierarchical aggregation framework in which a deep reinforcement learning (DRL) allocator assigns market-requested flexibility among resource aggregators using both current conditions and accumulated comfort impact. Local linear programming (LP) controllers then schedule HVAC operation under comfort and power constraints. The framework is evaluated for supply-demand adjustment requests defined by flexibility magnitude, direction, response-time tolerance and continuous delivery period. Compared with a centralised LP benchmark and a rule-based Greedy+LP baseline, the proposed DRL+LP framework reduces computation from more than 1,000 to a few seconds at large scale. At the 3,600-room scale, it assigns requests up to 0.792 MW for three hours and keeps aggregate assigned requests within a  ± 10% tolerance in nearly all reported extraction intervals. Its comfort-violation metrics remain close to the centralised LP reference, with absolute differences no larger than 0.052 ∘C in mean absolute error (MAE) and 0.045 ∘C2 in mean squared error (MSE) in the reported cases. It also avoids the uneven comfort burden observed in greedy allocation under low-flexibility requests. In an unseen two-request daily scenario, the learned policy maintains request allocation within tolerance and median comfort deviations of approximately 0.04–0.05 ∘C. These results establish a scalable and comfort-aware allocation approach within the simulated market-dispatch and continuously modulatable HVAC setting considered here. Evidence remains limited to a single calibrated room model, cooling operation and one climatic region; performance under heating, heterogeneous buildings, diverse occupancy and field communication or model uncertainty has not yet been established.