It depends a lot more of how the Riders are used than the amount of Riders, honestly. And as said before, you can have heavily important characters that aren't Riders.
I'd say Kamen Rider has never been a team series, though. At best, you had three Riders working together at any given time, excluding crossovers. Agito is probably the best at this, even the opening puts Agito, Gills and G3(-X) in the same level, but the fourth (debatable) Rider of the series, Another Agito, is not shown, and never really joins the core characters (and has a far shorter lifespan, too). Newer series like W (W, Accel), OOO (OOO, Birth, Proto Birth) and Fourze (Fourze, Meteor) show it well too.
Even in those "teams" there was always conflict, and it wasn't unusual to see the teams barely holding together and outright breaking at times. Faiz is the most extreme example of this, because everyone in that series has serious problems with interaction, and heavily require psychologists to teach them that beating each other up Zeta Gundam-style is NOT the way to go.
Usually, Riders outside those teams tended to be very undeveloped, and basically there for the story's sake (Ryuki, of course, is the strongest example of this, with only three really developed Riders (Ryuki, Knight, Zolda), one extremely important character with no development but awesome nonetheless (Ouja), and everyone else as one-note characters, especially Scissors. It's definitely not the only example though, there's Delta, Sasword, Drake, every evil Rider ever...).
How the Riders are used is heavily important, I'd say even moreso than having them as well-developed characters. Taking Ryuki as an example, again, only Ryuki, Knight and Zolda ever get developed, but most Riders has a role in the story, since it's centered around the Rider War and all (I'd say Scissors, Gai and Verde weren't really needed). I'd say Decade does the same thing, but that'd require making sense out of Decade's non-story. In Faiz, Delta is heavily important while on the enemy's hands, especially when its used by the Dragon Orphenoch, however, when Mihara gets it, he becomes a non-entity, and it'd have been the same or even better if the Delta Gear had been destroyed, because Mihara!Delta served no purpouse and had no personality whatsoever.
Of course, one must have in mind how Rider Systems change hands and fast. Ixa is the best example, having a crapton of users. Otoya, Jirou, Yuri, Nago, Megumi, and even two Fangires, Spider and Rook. All of them are heavily important to the story at large, and being Ixa is important for their characters. In Rook's case, it exploits Ixa's weakness as an advantage, which I found a moment of genius in a series that definitely lacked those (though it happened in 1986, the best moment of Kiva...).
Basically, what I want to say is what I said at the beginning, what matters is HOW they're used, not how many there are. Double does great things with its four Riders (Double (Joker included), Accel, Skull and Eternal), exploiting its movies to fully tell the stories of the Riders that didn't get the chance to shine in the series proper (namely, Skull and Eternal). Ryuki uses the Riders to generate conflict, as the framing point of the story, to challenge the viewpoint and ideals of its leads (Namely, Shinji and Ren, though again, Kitaoka has development as well). Then you have Kabuto, which had a great amount of totally useless characters (Sasword, Drake, looking at you. TheBee, why do you even exist if everyone will just leave you for another Rider System?), Kiva which had a Rider I'll never get (Saga was totally pointless, Taiga could've easily been Dark Kiva to begin with and nothing would've changed), Den-O which keeps getting more movie Riders that are never interesting enough (then again, considering Den-O is now five Riders instead of one with multiple form changes, you can say that Den-O himself is a Sentai...of one-note jokes that stopped being funny years ago).
In short, character dynamics are everything. How sizable the cast is hardly matters if it's not used properly.