17 Feb
2022
17 Feb
'22
4:56 a.m.
And before that, Fortran compilers were known to recognize Whetstone.
Mashey did his wonder talk called 'Lies, Damn Lies and Benchmarks' in the
mid-80s which begat the SPEC suite. Wasn't perfect but it did help.
Marketing people in particular like to grab a single number for 'goodness'
just to try to show mine is better than yours (my current disgust has been
the clock rate of the processors which sadly, my employer was one of the
worst in using as a figure of merit).
On Wed, Feb 16, 2022 at 5:44 PM Larry McVoy <lm(a)mcvoy.com> wrote:
On Thu, Feb 17, 2022 at 09:27:39AM +1100, Dave
Horsfall wrote:
Indeed, when a lot of compilers recognised the
Sieve of Eratosthenes
being
used and optimised for it... Wasn't all
that long ago that vehicle
manufacturers also started doing the same thing :-)
Indeed. I hated when people cheated because benchmarks should teach you
the capabilities of the machine being benchmarked. They aren't useful
if people cheat.
When I did LMbench, the rules were "cc -O2" and you could not link with
any benchmarking libraries and you could only report a result if you
reported all results.
I had a beef with this guy: https://www.cs.jhu.edu/~shap/ because he
had some toy OS that didn't have a VM system, didn't have networking,
didn't have much of anything, but boy oh boy, did it context switch
fast (because there was no context to speak of). He reported those
numbers in blatant violation of my rules and I had to escalate to
get him to stop. The bummer is he is a smart guy, capable of good
work, there is no need to cheat.