When the Peter Quill's mother dying of cancer and followed this upsetting scene with Quill's abduction by a spaceship.
While that was tragic, and even played into that movie's last battle, it was only the start of the series' solemnity. All eight of the Guardians of the Galaxy have a tragic backstory, making all the heroes tragic characters who only find their place when together.
Groot
Guardians of the Galaxy loved his father dearly. However, on his own, his tragedy never matches up with other of the team.
Mantis
Mantis first appeared in Guardians of the Galaxy Vol. 2 and ed the team after they defeated Peter Quill's cosmic dad, Ego the Living Planet. The character comes across as comedy relief, especially with her interactions with Drax the Destroyer in that second movie. However, her past is tragic on a low-key level. Ego raised her since she was a child, and he used her to help ease his neurotic behavior. Ego took her in when he found her in her larva state and kept her completely sheltered from the world until the Guardians of the Galaxy arrived.
Star-Lord
Since the first Guardians of the Galaxy movie opened with a young Peter Quill watching his mother die from cancer, it seemed he might be the most tragic backstory. However, he never matches up to other team who have lost loved ones. Losing his mother as a child was tragic, but many of his teammates have somehow even endured worse familial suffering. Yondu also wasn’t the best mentor for Peter as a surrogate parent, but in the end, he learned that he still loved him.
Drax the Destroyer
Thanos tried to kill Arthur Douglas and his family. Only Arthur's wife died completely, while Arthur had his essence sent into the body of the legendary Drax to see revenge and his daughter also escaped, becoming Moondragon.
In the movies, Thanos was still responsible, but he sent Ronan to destroy half the beings on Kylos, and he killed Drax's wife and daughter in front of him, causing Drax to seek bloody vengeance. Watching Ronan kill his family was a tragedy that Peter Quill never could have imagined.
Yondu
Yondu was a minor antagonist from the first Guardians of the Galaxy movie, before eventually becoming an ally. There was nothing tragic about the character at first, but that all changed in Guardians of the Galaxy Vol. 2, as his backstory showed why he grew so hardened in adulthood.
As a child, Yondu's parents sold him to the Kree Empire, where he mostly lived in a cage. Stakar Ogord then freed him from slavery and took him into the Ravager clan. However, Yondu made the mistake of taking money to kidnap a child for Ego, which caused Stakar to banish him, leaving him alone. He never found redemption until his death.
Gamora
Gamora faced a similar fate as Drax the Destroyer but also went through an experience that resembled Rocket's, as both were turned into monsters by those who raised them. For Gamora, she lived on Zen-Whoberi and was a child when Thanos arrived to kill half the beings on the planet, including her own mother.
Thanos took her in and raised her to become a perfect soldier, something which helped her as a fighter in the Guardians of the Galaxy. He taught her to kill, instilled in her a sense of hatred and vengeance, and molded her as his prized "daughter." That last fact keeps her story less tragic than Rocket and Nebula, who never found favor with their masters. However, Gamora's story ended in tragedy as well, when Thanos killed her to gain the Soul Stone.
Rocket Raccoon
Rocket Raccoon's tragic backstory was much more horrific than losing a loved one like Peter Quill and Drax the Destroyer. It also proved more tragic than Yondu, because Rocket never had a chance from the moment of his birth.
Rocket's real name is 89P13, and he is a creature recovered by alien scientists. They enhanced his genetics to give him intelligence but did so through years of abuse. This genetic rewriting was unceasing, which explains his temperament and refusal to believe he deserves a happy ending.
Nebula
Nebula is the Guardians of the Galaxy member with the most tragic past. Much like Gamora, Nebula ed Thanos as one of his "daughters" and soldiers. Thanos took in Nebula after murdering her entire family, the same as Gamora. However, while Thanos loved Gamora as his prized daughter, he hated everything about Nebula and tortured her endlessly. While she only sought to please Thanos, he repaid her by tearing her apart and "upgrading" her to strengthen her when she'd lose battles with Gamora. The torture eventually caused Nebula to hate both Gamora and herself, but she finally found a place with the Guardians of the Galaxy in the end.