The Learning Health System (LHS) aims to improve healthcare by using continuous data analysis to create equitable, patient-centered, and cost-effective care. Evaluating LHS success is challenging due to real-world variability in execution and implementation and absence of clear metrics. We conducted an international jurisdictional scan to highlight common evaluation approaches, indicators, outcomes, challenges, and assumptions related to establishing counterfactuals in LHS evaluation. Evaluation outputs were categorized into four types: description, lessons learned, efficacy, and effectiveness. Frequencies and thematic analysis were used to describe LHSs, their evaluations, indicators of change, and lessons learned. 45 papers describing 44 LHSs were included. 30 papers shared lessons on LHS progress, 14 reported on efficacy during scaling, and none reported on effectiveness of sustained systems. Ingredients perceived to contribute to a successful LHS included engagement of key individuals, establishment of a LHS culture, data considerations, and contextual factors. Future evaluations should consider LHS maturity, utilize counterfactuals, and prioritize equity. Evaluating and addressing these gaps can fuel LHS effectiveness and ensure that diverse needs of patients and providers are met. Ultimately, structured and more standardized evaluation efforts could foster a culture of continuous learning and improvement, enabling health systems to better enhance population health outcomes and deliver high-quality, equitable care.