New cancer characteristics can be discovered by focusing on the process of tumor formation. Cancer stem cells (CSCs) are a key subpopulation, as they are theorized to be at the apex of the tumor hierarchy. We can better understand their function in the tumor hierarchy by using sectioned samples to observe the growth of tumors from their origins as CSCs. In this study, we evaluated the growth of moderate differentiated colorectal cancer from LGR5-positive cells, which is a CSC marker of colorectal cancer, using xenograft and three-dimensional culture models spatiotemporally. These cells express LGR5 at high levels and show CSC phenotypes. To detect them, we used a previously generated antibody that specifically targets LGR5, and were therefore able to observe LGR5-positive cells aggregating into small clusters (sCLs) over the course of tumor growth. Because these LGR5-expressing sCLs formed continuously during growth mainly in the invasive front, we concluded that the structure must contribute significantly to the expansion of CSCs and to tumor growth overall. We confirmed the formation of sCLs from gland structures using a three-dimensional culture model. In addition, sCLs exhibited upregulated genes related to stress response and partial/hybrid epithelial-mesenchymal transition (EMT), as well as genes reported to be prognosis factors. Finally, sCLs with high LGR5 expression were identified in clinical samples. Based on these results, we elucidate how sCLs are an important contributors to tumor growth and the expansion of CSCs.