Delay-Join the Shortest Queue Routing for a Parallel Queueing System with Removable Servers

We introduce a new class of policies called “delay-join the shortest queue (delay-JSQ)” for use in parallel processing networks with removable servers. When jobs arrive to the system while all servers are on, jobs should be routed to the shortest queue. However, when servers are off, they take a random time to turn back on, which we allow to occur only when the number of jobs in each of the nonempty queues exceeds a fixed threshold. This new class of policies balances the load among all servers that are currently on and balances the capacity by keeping servers off until they are needed. A detailed numerical study shows that at moderate loads (where server farms and increasingly manufacturing facilities operate), delay-JSQ outperforms JSQ by up to 80%. In addition, it does so without precise knowledge of the input parameters and even when the input process is nonstationary.