In late 2009, I was doing some benchmarks for a client,
and got strange results that seemed to depend on the order in which various benchmarks were executed.
At the time, I was doing multiple calls to my
bb.util.Benchmark class from within the same JVM session.
The tasks being benchmarked were not totally independent, but were differently parameterized versions of some common functionality.
I ended up concluding that hotspot was optimizing for the first execution profile being benchmarked,
but that profile was not optimal for subsequent tasks.
There appears to be no way that you can programmatically tell the JVM to throw away all its profiling and optimizations.
So, the only way to do pristine benchmarks is to do but a single benchmark per JVM session.
As an example of how to do but a single benchmark per JVM session, I have revised the latest version of my article project so that the script files now generate all the scenarios instead of the Java code.
Augmenting the original text is some highly specialized documentation that I have placed into an article supplement. These formats are available:
If anyone knows of a free open source way to convert Microsoft Word 2007 documents to an open format which would preserve the equations in MathML as well as leave hyperlinks working, please email me. I am aware of Microsoft's pdf convertor add-on, but that requires a valid copy of MS Office which I no longer have. I am also aware that OpenOffice 3 can allegedly import from/export to MS Office 2007, however, when I tried to open my docx file with Open Office 3.1 on 2009-05-18, it garbled all of the equations.
The following may be downloaded:
|benchmarking framework (just jars):||
bb.jar, jsci-core.jar, mt-13.jar (download all these individual jar files)
Last modified: 2010-06-01
There have also been many changes to other classes in bb.jar.
The jar files above are all that you need if you merely want to use the
|benchmarking framework (complete project):||
Last modified: 2010-06-01
(Same changes described above.)
Download this if you want my complete code library project, including source code for the
Last modified: 2008-07-23
This is a version from around the time when the article was published.
|article code listings:||
Last modified: 2009-05-07
Last modified: 2008-07-06
This is a version from around when the article was published.
Warning: this project is not self-contained: to run it, must first download the benchmarking framework
Note: all the text files should be proper text files (that is, use unix '\n' line ends). So, if you are a Windows user, you will need a decent text editor to read them (NotePad/WordPad may not cut it; I highly recommend TextPad)