Replenishment and fulfilment decisions for stores in an omni-channel retail network

•The online order fulfilment decision in a retailer’s store network is modelled.•Contrary to previous methods, we consider regular inventory replenishment decisions.•We use a decomposed Markov decision process followed by a one-step policy improvement.•The approach outperforms two existing methods on several key performance indicators.•Our method is better at balancing inventories across the stores in the network.