You seem to forget that the powers of Goranger to Goseiger doesn't belong to the Gokaigers.
they may not belong to them, but the legend's did except them as the 35th sentai and were therefore willing to give them there legendery power so while they may not belong to them there's no reason not to keep them.