In this work, we investigate binary classification under the constraints of
both differential privacy and fairness. We first propose an algorithm based on
the decoupling technique for learning a classifier with only fairness
guarantee. This algorithm takes in classifiers trained on different demographic
groups and generates a single classifier satisfying statistical parity. We then
refine this algorithm to incorporate differential privacy. The performance of
the final algorithm is rigorously examined in terms of privacy, fairness, and
utility guarantees. Empirical evaluations conducted on the Adult and Credit
Card datasets illustrate that our algorithm outperforms the state-of-the-art in
terms of fairness guarantees, while maintaining the same level of privacy and
utility.