Kinda' depends on the music, no? You can't do Rush covers on a 4 piece, can you (though, it would be a fun challenge)? Been reliving my youth the last few days, listening to some old Iron Maiden. Even those tunes have more drum sounds than a 4 or 5 piece can offer. I don't know if I'm helping.
I'd like the option of a slightly bigger kit, but I'll be happy finding the elusive mid-70's Gretsch 8x12 walnut tom, to complete my 2 up and over, two down kit. I think big kits are still a "Thing", just not like it used to be. Like concert toms, they fell out of flavor. I don't think there's any true theory as to why the big kits became fewer.
Interesting question and comments here. That said, girls like drummers?
B