- We developed a machine learning framework in order to establish the correlation between dose and activity distributions in proton therapy. A recurrent neural network was used to predict dose distribution in three dimensions based on the information of proton-induced positron emitters. Hounsfield Unit (HU) information from CT images and analytically derived stopping power (SP) information were incorporated as auxiliary inputs. Four different scenarios were investigated: Activity only, Activity + HU, Activity + SP and Activity + HU + SP. The performance was quantitatively studied in terms of mean absolute error (MAE) and mean relative error (MRE), under different signal-to-noise ratios (SNRs). In addition to the first dataset of mono-energetic beams, three additional datasets were validated to help evaluate the generalization capability of our proposed model: a dataset of a lower SNR, five reconstructed PET images, and a dataset of spread-out Bragg peaks. Good verification accuracy of dose verification in three dimensions is demonstrated. The inclusion of anatomical information improves both accuracy and generalization. For an activity profile with an SNR of 4 (the mono-energetic case), the framework is able to obtain an MRE of ∼ 0.99% over the whole range and a range uncertainty of ∼ 0.27 mm. The machine learning-based framework may emerge as a useful tool to allow for online dose verification and quality assurance in proton therapy.