Magnesium alloys exhibit significant inelastic behavior during unloading, especially when twinning and detwinning are involved. It is commonly accepted that noteworthy inelastic behavior will be observed during unloading if twinning occurs during previous loading. However, this phenomenon is not always observed for Mg sheets with strong rolled texture. Therefore, the inelasticity of AZ31B rolled sheets with different rolled textures during cyclic loading-unloading are investigated by elastic viscoplastic self-consistent polycrystal plasticity model. The incorporation of the twinning and detwinning model enables the treatment of detwinning, which plays an important role for inelastic behavior during unloading. The effects of texture, deformation history, and especially twinning and detwinning on the inelastic behaviors are carefully investigated and found to be remarkable. The simulated results are in agreement with the available experimental observations, which reveals that the inelastic behavior for strongly rolled sheets is very different than the extruded bars.