Multi-layered round robin routing for parallel servers

We study a system of several identical servers in parallel, where a routing decision must be made immediately on a job’s arrival. Jobs arrive according to a Poisson process, with their processing times following a discrete distribution with finite support. The processing time of a job is known on arrival and may be used in the routing decision. We propose a policy consisting of multi-layered round robin routing followed by shortest remaining processing time scheduling at the servers. This policy is shown to have a heavy traffic limit that is identical to one in which there is a single queue (no routing) and optimal heavy traffic scheduling. In light traffic, we show that the performance of this policy is worse than round robin routing followed by shortest remaining processing time scheduling. We also quantify the difference between round robin and multi-layered round robin routing, which in turn yields insights on the relative importance of routing versus (local) scheduling in such systems.