Stakeholder participation is relevant in strengthening priority setting processes for health worldwide, since it allows for inclusion of alternative perspectives and values that can enhance the fairness, legitimacy and acceptability of decisions. Low-income countries operating within decentralized systems recognize the role played by sub-national administrative levels (such as districts) in healthcare priority setting. In Uganda, decentralization is a vehicle for facilitating stakeholder participation. Our objective was to examine district-level decision-makers’ perspectives on the participation of different stakeholders, including challenges related to their participation. We further sought to understand the leverages that allow these stakeholders to influence priority setting processes. We used an interpretive description methodology involving qualitative interviews. A total of 27 district-level decision-makers from three districts in Uganda were interviewed. Respondents identified the following stakeholder groups: politicians, technical experts, donors, non-governmental organizations (NGO)/civil society organizations (CSO), cultural and traditional leaders, and the public. Politicians, technical experts and donors are the principal contributors to district-level priority setting and the public is largely excluded. The main leverages for politicians were control over the district budget and support of their electorate. Expertise was a cross-cutting leverage for technical experts, donors and NGO/CSOs, while financial and technical resources were leverages for donors and NGO/CSOs. Cultural and traditional leaders’ leverages were cultural knowledge and influence over their followers. The public’s leverage was indirect and exerted through electoral power. Respondents made no mention of participation for vulnerable groups. The public, particularly vulnerable groups, are left out of the priority setting process for health at the district. Conflicting priorities, interests and values are the main challenges facing stakeholders engaged in district-level priority setting. Our findings have important implications for understanding how different stakeholder groups shape the prioritization process and whether representation can be an effective mechanism for participation in health-system priority setting.