Windows 7 better – but by how much?
The first major independent tests of Windows 7’s beta edition show it outperforming both XP and Vista. However, the method of the testing only shows it is superior and doesn’t accurately tell us the margin of improvement.
The tests come from Adrian Kingsley-Hughes (pictured) of ZDNet. He explained to readers that he didn’t think it was appropriate to carry out the usual computer assessment process known as benchmarking, which uses dedicated software to time how long a particular computer set-up takes to carry out tasks. (These tests produce hard numbers which you might see expressed as, for example, a “3.8 percent improvement”.)
Kingsley-Hughes explains that such testing isn’t really suitable for a beta model, mainly because at that stage developers are more concerned with making sure everything works reliably. Performance tweaks tend to come with the final releases of an operating system, meaning you can’t fairly compare numbers for the Windows 7 beta with finished editions of Windows.
Instead he carried out 23 tests and simply noted which version of Windows performed best and worst. The test included variations on the following tasks:
- Installing the system itself and Office
- Boot up and shut down
Moving files (both from one drive to another and across a network) - Compressing and extracting files
- Opening Office and Adobe Acrobat documents
- Burning a DVD
While he produced a final score for each of the three systems (based on 1 point for ‘winning a test’, 2 for second and 3 for coming last, with the lowest score winning), these scores don’t tell us anything about the margin of victory in each category.
What is clear is that Windows 7 was the winner in almost every test. The few areas where it didn’t show an improvement were mainly tasks which users won’t carry out that often, such as installing Office and transferring huge files. (Meanwhile, on an older machine there was very little difference between Vista and XP; Vista did pull ahead with a higher-spec computer).
It will come as welcome news to Microsoft that this real-world testing shows the system does perform better across the board. But we’ll likely have to wait for the final release before we know if these improvements are big enough to be – and whether Windows 7 does better on less objective measures such as useability.
Related Posts:
January 6th, 2009
“He explained to readers that he didn’t think it was appropriate to carry out the usual computer assessment process known as benchmarking”
Not when you are engaged in shilling for the latest round of vapourware – you wouldn’t want pesky statistics and bare facts getting in the way of a good story.
“Performance tweaks tend to come with the final releases of an operating system”
When Microsoft is in the frame, it’s last-minute bloat and nasty surprises that come with the final release. Does anyone remember how good Vista was going to be? I read recently that, at the time, beta testing of Windows 95 showed it to be “brilliant”, so this has been going on for a _long_ time. Microsoft are the ultimate exponents of the technological pea and thimble trick, and country yokels are still lining up and eagerly parting with their cash.