In order to emulate a system purely through software, your hardware must be exponentially more powerful than the original system in question. Computers now can emulate PS2 because they've got exponentially more powerful processors and more RAM, but the PS3 just wasn't exponentially more powerful than the PS2 at launch. For the PS3 to run PS2 games, it needed PS2 hardware shoved in the box, which was expensive. And at the time the PS3 came out, PS2 emulation on PC also didn't work very well yet, because most computers just weren't powerful enough to pull it off.
The Xbox 360 is just barely powerful enough to emulate some, but not all, Xbox games. And from what we know of the specs, the Xbox One simply is not powerful enough to software emulate 360 games at all. The PS4 also isn't powerful enough to software emulate the PS3, but there's rumors that Sony's working on some kind of crazy streaming solution to the problem that might let them sidestep their hardware limitations.