I loved Tough Crowd, I watched it every night (even when Comedy Central gave it its test run in late '02 for a couple of weeks). It started to go down after a while -- I think when Colin knew Comedy Central was close to pulling the plug, so he tried these weirdo attempts and being different, like the elimination of chairs for the guest comedians (WTF!?!) or getting "names" to drop by (the show worked best with Colin's nobody comedian pals).
I think people rather write Colin off as an unintelligible idiot than try to listen to him -- I understood him fine (well, maybe except when he got into one of his super-pissed quick rants) and I think he's one of the best comedians out there. He was consistantly funny every night, so even when I didn't like the panel of comedians, I knew Colin would be good. I was about the only person who liked him on "Weekend Update," too.
I liked Chappelle, too...that and Tough Crowd were the only original shows on Comedy Central that actually featured "comedy." I always had a sneaking suspicion that Comedy Central killed Tough Crowd because Chappelle was becoming a monstrous hit -- and look how that bit them on the ass! They're stuck trying to make the painfully unfunny Reno 911 a hit!
What pisses me off the most is Tough Crowd was canceled, lack of viewers stated as the reason (although Colin said it became "too political" for the suits) to make room for "Blue Collar" reruns and/or "Daily Show" reruns.