Reference
October 26, 2008

How SimHQ Tests
PC Hardware Components

As a web site focused almost exclusively on PC simulations, SimHQ’s hardware reviews will always attempt to reflect that focus by placing a predominant emphasis of our testing on the simulation genre. SimHQ’s benchmarking suite will therefore consist of current simulations that represent what we feel our community is currently playing in the hopes that our technology articles help inform and assist the site’s readership in their system building and hardware component purchases. The benchmark suite will naturally evolve as certain simulations age and new ones are introduced, and the suite itself will at times differ per article depending upon the component being tested. And not every application included in SimHQ’s test suite falls within the traditional definition of simulation; synthetics are useful for testing forward-looking features not yet widely supported by available games, and during lulls in the simulation genre SimHQ will make use of non-sims that we feel are still of interest among a fair slice of our readers.

Certain titles by nature are useful for different component testing and a title such as Microsoft’s Flight Simulation X is well known to be CPU-limited in its performance; i.e., installing a faster graphics board will not give you better frame rates with FSX, whereas a new, faster CPU should improve the simulation’s performance. In contrast, when testing the latest GPU from AMD or NVIDIA SimHQ tries to use gaming titles that are more graphics intensive to stress the component(s) under evaluation; this situation requires that certain simulations not be used at times, and for that reason SimHQ is careful in which applications it selects when testing different PC components.

And beyond just the title selection itself, in-game parameters are also considered not just with an eye toward subsystem performance impact but also to avoid the introduction of variables in our testing. Accurate benchmarking requires strict control over the test environment (in this case, the test machine itself and its software) to avoid bad, inaccurate test results. To this end, SimHQ uses Norton Ghost to clone the boot drives of all test systems to ensure identical OS configuration. SimHQ also makes sure the newest system component drivers are installed, and the latest patches for the test applications used are applied. If the test results produced are not consistent or appear to be outside the expected margin of error, they are deleted and the test system is then re-examined. Accurate, repeatable results are a must in a properly controlled test environment and SimHQ strives for this during all hardware performance testing.

If you have comments, questions, or suggestions, please feel free to discuss them in our Hardware / Software Forum.

SimHQs Fall 2008 Benchmark Suite

Click here to go to top of this page.