Quote:
Originally Posted by Karlos
MIP: Meaningless Indicator of Performance
This is why I prefer benchmarks of actual software, i.e. the things you'll actually use.
|
This makes a lot sense.
But also this can cause confusion.Lets say for example you like mp3 and you use as your yardstick lame (mp3 converter tool). On Aminet you can find not only many different version of lame, but also different compiled versions. I saw one compile properly done with compiler optimization turned on,
and another lame compile done (probably by accident) with all optimizations turned off and with debugger enabled.
Both of the lame exe look the same to the user but the not-optimize versions runs like 2 times slower.
So if people report their results - they can vary by 100% for no apparent reason.