Research in the last decade has clearly revealed a critical role of prostate cancer stem cells (PCSCs) in prostate cancer (PC). Prostate stem cells (PSCs) reside in both basal and luminal layers, and are the target cells of oncogenic transformation, suggesting a role of PCSCs in PC initiation. Mutations in PTEN, TP53, and RB1 commonly occur in PC, particularly in metastasis and castration-resistant PC. The loss of PTEN together with Ras activation induces partial epithelial–mesenchymal transition (EMT), which is a major mechanism that confers plasticity to cancer stem cells (CSCs) and PCSCs, which contributes to metastasis. While PTEN inactivation leads to PC, it is not sufficient for metastasis, the loss of PTEN concurrently with the inactivation of both TP53 and RB1 empower lineage plasticity in PC cells, which substantially promotes PC metastasis and the conversion to PC adenocarcinoma to neuroendocrine PC (NEPC), demonstrating the essential function of TP53 and RB1 in the suppression of PCSCs. TP53 and RB1 suppress lineage plasticity through the inhibition of SOX2 expression. In this review, we will discuss the current evidence supporting a major role of PCSCs in PC initiation and metastasis, as well as the underlying mechanisms regulating PCSCs. These discussions will be developed along with the cancer stem cell (CSC) knowledge in other cancer types.