With Sonnet 119, William Shakespeare further elaborates on his metaphor, introduced in Sonnet 118, of having taken bitter medicines to prevent himself from ever getting sick of his younger lover, these potions having been affairs, encounters, or even relationships of sorts with other people.
Who these other people were he still doesn't tell us, but he here makes it even clearer that they were fundamentally bad for him, their principal, if not sole, redeeming feature being that their experience has ultimately strengthened him and cemented his love for his young man.
The sonnet is the third of three sonnets which all attempt to explain and to some extent excuse Shakespeare's infidelities of the past, and they all do so in the wake, directly, of Sonnet 116 which famously and categorically posited, "Let me not to the marriage of true minds | Admit impediments."