Previous research suggests that music listening influences individual affective responses. However, there is scant research examining how social factors might interact to influence subjective affective responses to music. This study investigates the effects of social feedback and social context on subjective affective responses to music. In a between-subjects experiment, participants (N = 120) listened to unfamiliar music from various genres either alone or with another participant. For each musical example, participants received positive and negative social feedback, derived from a pilot study, or factual album information. After listening to each musical example and reading the provided social feedback or album information, participants reported their subjective valence, arousal, subjective affective intensity, concentration, music liking and familiarity. There was no effect of social feedback on subjective valence responses. Positive and negative social feedback influenced subjective arousal responses positively and negatively, respectively. Subjective affective intensity was not influenced by social feedback. Social context did not influence subjective affective responses to the musical examples. Lower concentration was reported in social listening conditions compared to solitary conditions. Greater familiarity with the musical examples was reported when social feedback was provided. The findings of the present study suggest that social feedback can influence particular affective responses to, and familiarity with, music. However, social listening might reduce concentration, especially in the absence of social feedback. These findings highlight issues warranting consideration for how music is affectively experienced in everyday life, as well as purposely used in varied contexts.