I think my problem is that I do absolutely nothing to make the words and kanji stick, I just go passing by. What sticks, sticks, and what doesn't... Well, I'll probably see it again sometime. Something I need to work on, really.
Now for the grammar part, I have to thank my school. The material used studies almost exclusively grammar. I suck at remembering the rules, but seeing it so much made me get used to them, much like in english. I can't name a single grammar rule, but I can use them pretty well.