Directly hitting the hardware is simply not possible anymore. How do you want to directly hit the hardware if your typical target customer may have one of a myriad of more or less up-to-date graphic cards installed? You can't code for 20 different hardware platforms (=20 different gfx cards), especially if you don't have any documentation available (nVidia). You're simply forced to stick to abstract APIs.
The situation is slightly better on consoles, as the hardware specs are fixed in this case. Compare the first batch of PSX-1 games with the later ones, and you'll see that the game developers learned how to use the hardware more efficiently.
As far as custom chips are concerned, the PC has cloned most of the Amigas big advantages by now. A modern GFX card is a collection of highly powerful custom chips, the main difference is the fact that it's geared (nearly) only towards 3D operations (as opposed to the Amiga's hardware scrolling/BOB/raster manipulation based approach). Even low budget gfx cards come with a blitter now, practically all components in a modern PC are able to do DMA.
IMHO, the PC suffers from three problems the Amiga didn't have:
- thousands of individual configurations and badly documented hardware
- a lousy OS (that's a pretty big factor)
- big game companies with no imagination but lots of business sense