Thanos. However, their absence in the original The Infinity Gauntlet storyline was more than the workings of the Mad Titan - the saga's creator Jim Starlin wasn't allowed to use them.
The Infinity Gauntlet - by Jim Starlin, George Perez, and Ron Lim - is ed as one of the best Marvel crossover events of all time. The story features Thanos acquiring all of the Infinity Stones and using the Infinity Gauntlet to snap half of all life in the universe out of existence in an effort to impress Death, whose affection he desperately craves. The six-issue miniseries would have a number of sequels that featured a ton of Marvel heroes and villains. But, the reason Starlin's roster in The Infinity Gauntlet was so small despite the grandness of the event was because he only was allowed to use a certain number of characters from each title.
he was only able to use Doctor Doom.
While it hasn't been officially confirmed by Starlin, it would make sense that an editor would disprove his use of the Fantastic Four. After all, how were they to know the cultural impact that The Infinity Gauntlet series would have on both Marvel and comics is general? It's weird to think such a grand comic wouldn't feature Marvel's First Family. Thankfully, when Starlin wrote his sequel series, The Infinity War, not only was he allowed to use the Fantastic Four, but the roster of characters involved increased dramatically.
In the end, it's fair to say that things worked out for Starlin, Perez, Lim, and Marvel. Starlin had a built-in excuse in Thanos' snap to sideline some of Marvel Comics' most powerful and memorable heroes, so whether it was his choice or that of an editor, the Fantastic Four's absence at least made sense in the story. The Infinity Gauntlet would have been much different with the Fantastic Four not getting snapped, but it probably was for the best.
Source: CBR.com