The first episode of Gallifrey, and back when s/he had a family.
The Doctor's dialogue in this episode is poignant and haunting, and she opens up in a way that's rarely seen. Attempting to reach out and comfort her friends and companions, the Doctor talks about how she carries her family in her heart. "They may be dead to the world, but they're not dead to me," she declares. It's a touching moment, but it also serves a useful narrative function; it tells viewers that the Doctor has regained all her memories at last.
Related: Doctor Who Season 11: New Cast & Character Guide
There's a strong element of tragedy to the story of the Doctor's family. When Doctor Who began back in 1963, William Hartnell's First Doctor was traveling with a teenager named Susan, who addressed him as "grandfather." That particular relationship hasn't really been explored since the 1960s - until last year, when the BBC published the canon book A Brief History of Time Lords. That strongly implied Susan was actually the daughter of the Lord President of Gallifrey; the book listed a number of things the Doctor stole when he left his homeworld, and the President's daughter was referenced alongside a photo of Susan. If that's the case, the Doctor's family affairs clearly became very complicated and quite acrimonious. It's easy to imagine a bitter dispute between the Doctor and his child, the President of Gallifrey, over whether or not Time Lords should intervene in the cosmos. That would certainly add a personal dimension to the Doctor's philosophical conflict with his own people. We can assume Susan took her grandfather's side in this, and chose to go with him.
As for where those children are, it's been known the Doctor's children died since the Patrick Troughton era. In Tomb of the Cybermen, Troughton's Second Doctor attempted to comfort his grieving ward, Victoria, who'd lost her father. The teenager was simply unable to believe that the Doctor could empathize with the pain she felt; she suggested he was so impossibly ancient that he probably couldn't even his family. The Doctor's reply was a powerful one:
"Oh yes, I can when I want to. And that's the point, really. I have to really want to, to bring them back in front of my eyes. The rest of the time they sleep in my mind, and I forget. And so will you. Oh yes, you will. You'll find there's so much else to think about. So , our lives are different to anybody else's. That's the exciting thing. There's nobody in the universe can do what we're doing."
More recently, David Tennant's Tenth Doctor confirmed that he'd had several children - and had lived to see them all die. In the episode The Doctor's Daughter, he told Donna that he'd been a father before. "When they died," he observed, "that part of me died with them. It'll never come back. Not now." Notice the plural, the only hint so far that the Doctor had more than one child. Tennant also had a "daughter", Jenny, although that was actually a product of altered DNA.
Related: Doctor Who: The Best (And Worst) Companions So Far
And what of the Doctor's parents">The End of Time, he gave viewers a brief glimpse of a Time Lady he intended to be the Doctor's mother. The episode itself deliberately left that ambiguous, though, so as not to cause problems with continuity.
We may not know the truth about the Doctor's parents, but the sad fact remains that the Doctor seems to have parted with his/her family on very bitter - and it's unlikely they were reconciled before their deaths. Given Jodie Whittaker's Thirteenth Doctor is now traveling with three grieving companions, it's likely we're going to see this idea explored some more over the rest of the season.