Thesis Multi-agent Deep Rainforcement Learning for Efficient Multi-Timescale Bidding of a Hybrid Power Plant in Day-Ahead and Real-Time Markets
Date
2022
Authors
Ochoa Abett de la Torre, Tomás Felipe
Journal Title
Journal ISSN
Volume Title
Publisher
Universidad Técnica Federico Santa María
Abstract
Effective bidding on multiple electricity products under uncertainty would allow a more profitable market participation for hybrid power plants with variable energy resources and storage systems, therefore aiding the decarbonization process. This study deals with the effective bidding of a photovoltaic plant with an energy storage system (PV-ESS) participating in multi-timescale electricity markets by providing energy and ancillary services (AS) products. The energy management system (EMS) aims to maximize the plant’s profits by efficiently bidding in the day ahead and realtime markets while considering the awarded products adequate delivery. EMS’s bidding decisions are usually obtained from traditional mathematical optimization frameworks. However, since the addressed problem is a multi-stage stochastic program, it is often intractable and suffers the curse of dimensionality. This document presents a novel multi-agent deep reinforcement learning (MADRL) framework for efficient multi-timescale bidding. Two agents based on multi-view artificial neural networks with recurrent layers (MVANNs) are adjusted to map environment observations to actions. Such mappings use as inputs available information related to electricity market products, bidding decisions, solar generation, stored energy, and time representations to bid in both electricity markets. Sustained by a price-taker assumption, the physically and financially constrained EMS’s environment is simulated by employing historical data. A shared cumulative reward function with a finite time horizon is used to adjust both MVANNs’ weights simultaneously during the learning phase. We compare the proposed MADRL framework against scenariobased two-stage robust and stochastic optimization methods. Results are provided for one year round market participation of the hybrid plant at a 1-minute resolution. The proposed method achieved statistically significant higher profits, less variable incomes from both electricity markets, and better provision of awarded products by achieving smaller and less variable energy imbalances through time.
Description
Keywords
Multi-view artifical networks , multi-agent deep rainforcement learning , energy management system , solar generation , energy storage , electricity market bidding , multi-timescale electricity markets
