Research has emphasized the important role of recalling childhood experiences on adult mental health, and also the benefits of self-compassion on well-being. This study explored self-compassion as a mediator between early memories with family and peers and quality of life, on a wide age range female sample (N = 645). Path analysis revealed that self-compassion mediated the impact of both types of memories on women’s perceived quality of life. Overall, results revealed that the recall of early positive memories, either with family and peers, associate with a more compassionate self-to-self relationship, and the perception of greater psychological and social well-being.