Many wild populations are suffering from the loss of genetic diversity caused by habitat fragmentation, while the degree of diversity loss differs among species and populations based on their life history characteristics. Trillium camschatcense, an understory perennial plant, has undergone intensive habitat fragmentation in the Tokachi region, Hokkaido, Japan. Although demographic deteriorations, such as reduced seed production, were already reported, genetic consequences of fragmentation have not been studied with reference to its life history. Here, we examined how life history events (e.g., growth and reproduction) and the stochasticity therein influence genetic diversity in two (each large and small) fragmented T. camschatcense populations. Genetic diversity was evaluated using genome-wide 2,008 single nucleotide polymorphisms (SNPs). In the small population, genetic diversity of newly germinated seedlings was significantly lower than that of matured life history stages, and effective number of breeders (N_b) was smaller than that of the large population. Simulations using a matrix population model showed that the diversity loss at seedlings is caused by genetic drift during reproduction, which was intensified by smaller N_b. Besides, simulations using randomly perturbed transition matrices suggested that stasis at juvenile stages, which is a common characteristics of T. camschatcense, maintains genetic diversity by buffering stochastic decrease, possibly contributing to population viability. While previous studies showed the importance to facilitate reproduction and recruitment for demographic recovery, this study highlighted the crucial roles of juvenile survival in terms of genetic diversity for the conservation of fragmented T. camschatcense populations in the Tokachi region.