Closed-Loop Multi-Satellite Scheduling Based on Hierarchical MDP

In this paper, a Markov Decision Process (MDP) based closed-loop solution for the optical Earth Observing Satellites (EOSs) scheduling problem is proposed. In this MDP formulation, real-world problems, such as the communication between satellites and ground stations, the uncertainty of clouds, the constraints on energy and memory, are considered. To provide a feasible solution within a reasonable computation time, spatial and temporal approximation methods are provided to reduce the state and action space. A hierarchical MDP, which includes a sequential global MDP and local MDPs, is proposed and formulated to take full use of the computational resource on the satellite and the information gathered by the satellites about the clouds to refine the policy provided by the global MDP from the ground stations. Preliminary results are presented and discussed the feasibility of using an optimal feedback framework.