They do look absolutely brilliant on the PSP, for obvious reasons. But the screen is also a thousand times clearer than a normal CRT with composite cables. It looks incredibly clean. I think a lot of PS1 games looks excellent on my telly though, running in running upscaled to 1080p on the PS3 might have something to do with that.
I can mention a lot of Amiga 500 games that have aged incredibly well, like Full Contact, Alien Breed, Superfrog, Monkey Island 1 and 2, Indy and the Fate of Atlantis, Cruise for a Corpse, Sensible Soccer etc etc. The thing about a lot of Amiga games are that they look so incredibly clean and makes excellent use of colours and art design. That's kinda what makes a game look good today.
Aging well doesn't mean it's not possible to see that it's from a different time. Aging well means it's still aestetically pleasing to look at, and that makes it a lot easier to enjoy games from that time today. Early 3D were often overly complex with more polygons than required. Trying to push a million details into a low-res polygon model will in most cases make the model look dirty and plain ugly. When the whole game was filled with this kind of 3D models, it's easy to see why it's not so impressive to look at today. The best 3D games were actually those who blended 2D and 3D, by using gorgeous pre-rendered backgrounds (FFVII-IX) and well designed characters that weren't tough on the eyes. These characters are very detailed, but it's use of colours and art direction is what makes them work. Then we have Metal Gear Solid. If you look at the models close-up, it's not very impressive. But the game is usually played somewhat zoomed out, thus making it a lot better on the eyes - and I'm pretty sure that's what the characters were designed for. It creates an illusion that you see things that aren't really there, your mind just fills in the blanks.
Ok, enough about that. The bottom line is, there are good examples of early 3D and there are bad examples of early 3D. Heck, there are good and bad examples of graphics from every era.