Reinforcement Learning for Optimal HVAC Control: from Theory to Real-world Applications

The HVAC system accounted for a significant portion of the building’s energy consumption, resulting in enormous CO2 emissions. Among the numerous HVAC control methods, reinforcement learning (RL) gives the ability to control complex systems without requiring an explicit model of the building’s thermal dynamics. This study conducted a concise review of previous research on the application of RL to HVAC systems in buildings, it offered a thorough explanation of the theoretical foundations of RL and a summary of several recent RL studies that employ a particular variant of each main component of the RL: environment, state-space, action-space, rewards function, number of time steps and training episodes. Most studies construct the training environment as a stationary MDP due to the use of a predefined single sequence of transitions for non-action-controllable state vector components (e.g., outdoor temperature and occupancy schedule). This type of MDP is solved using tabular RL and DRL algorithms. Future research should focus on using the Meta-RL approach for HVAC systems, which solves the problem of non-stationarity in the environment (nonstationarity- MDP) where the sequence of transitions for (outdoor temperature and occupancy schedule) are changeable, as is the case during the real-world implementation of the RL controller in buildings.