The extent to which variation in food-related metabolites are attributable to non-dietary factors remains unclear, which may explain inconsistent food-metabolite associations observed in population studies. This study examined the association between non-dietary factors and the serum concentrations of food-related biomarkers and quantified the amount of variability in metabolite concentrations explained by non-dietary factors. Pregnant women (n = 600) from two Canadian birth cohorts completed a validated semi-quantitative food frequency questionnaire, and serum metabolites were measured by multisegment injection-capillary electrophoresis-mass spectrometry. Hierarchical linear modelling and principal component partial R-square (PC-PR2) were used for data analysis. For proline betaine and DHA (mainly exogenous), citrus foods and fish/fish oil intake, respectively, explained the highest proportion of variability relative to non-dietary factors. The unique contribution of dietary factors was similar (15:0, 17:0, hippuric acid, TMAO) or lower (14:0, tryptophan betaine, 3-methylhistidine, carnitine) compared to non-dietary factors (i.e., ethnicity, maternal age, gestational age, pre-pregnancy BMI, physical activity, and smoking) for metabolites that can either be produced endogenously, biotransformed by gut microbiota, and/or derived from multiple food sources. The results emphasize the importance of adjusting for non-dietary factors in future analyses to improve the accuracy and precision of the measures of food intake and their associations with health and disease.