RAJAR Lifts Lid On Audiometer Tests
Radio measurement body, RAJAR, will announce the long-awaited results of its audiometer tests in February, but brief overviews of the meters’ performance given by the ratings body’s research director suggest that the all-important tests may prove inconclusive.
In a presentation to the Media Research Group last night, Sally de la Bedoyere, RAJAR’s managing director, and Paul Kennedy, RAJAR’s research director, explained the rigorous testing procedures which have been carried out on audiometers.
However, dubbing his section of the presentation a “sneak preview” of next week’s full report, Kennedy revealed that each meter tested had performed differently under varying test conditions, with some returning “false positives” and claiming to have recorded listening which never occurred, while others returned “false negatives” and failing to register when listening actually occurred.
Kennedy also revealed that the three audiometers currently on trial: the TNS/Arbitron PPM, the GFK Radiocontrol Media Watch and the Eurisko NOP World Media Monitor, would soon be joined by a fourth meter from Ipsos-RSL, understood to employ “mobile communications technology” and be capable of measuring radio and TV consumption.
A source at Ipsos-RSL this morning revealed that the device is on course to be ready for testing within the next six months, and that the next phase of RAJAR testing would begin in the summer. However, both Sally de la Bedoyere and Paul Kennedy stated last night that the third phase of testing could start as early as next month.
Explaining that the results of RAJAR’s latest tests were waiting on just one set of results, Kennedy said: “It’s a bit too early to tell because we have two data sets from Eurisko, we have had two data sets from Arbitron and we are currently getting the second data set from Radiocontrol. We have always said that we would not announce the results until we have the sets for all three, so I can’t tell you anything sensational just yet.”
Kennedy also explained that, during the tests, the meters had performed differently when measuring radio stations, ranging from 100% accuracy across the board with a test of listening to Sunrise Radio, but with wildly different measurements, including readings of zero listening, with tests using Capital Gold, Radio 1 and LBC News.
Volume also affected the meters, with a ten minute sample of Kerrang! played at a very low level resulting in one meter registering nine minutes of listening, while the other two failed to record any listening. Meanwhile, in other tests, the meters reported listening on different radio stations, while actually subjected to the same station.
Explaining the examples, Kennedy said: “In each case, one meter doesn’t deliver… The reality of meters is that they may well be different ways of measuring the same thing.” Using the analogy of a pen, with different people measuring different aspects of it from size, to weight and capacity, Kennedy went on: “They are all true, but they’re all slightly different and expecting them to deliver the same thing is probably asking a little bit too much. However, ultimately we do hope that they would agree.”
The research boss stated that, in an ideal world, all three meters would perform in the same way, and be correct, resulting in a competitive market and cheaper equipment for the ratings body’s research. He said: “Whether they’re all right, or none of them are right remains to be seen. We will leave that decision until we have seen the final results. The argument is always that it’s just a currency, it doesn’t actually matter. As long as the data from one week is comparable with the data from the next week and you can actually see a change then that’s all that matters.”
He added: “That’s probably true up to a point, but the problem is that if you say to, for instance, Capital Gold that ‘if you measure with this meter then you’re number nine in the market, and if you measure them with that meter you’re number 16 in the market, does that matter to you?’ They’re likely to say ‘just a little bit’ before I head off to the Job Centre! So it is quite critical.”
Summarising RAJAR’s research to date, Kennedy explained that false positives were inevitable aspects of electronic measurement, with all meters recording them. However, he warned: “Two minutes extra per week will actually increase some stations’ reach by 15%. The problem that meters have got is that every time you try to get rid of false positives you increase the false negatives and to find a balance between the two is actually very difficult. You will always get one or the other because they’re dependent on the quality of samples that are taken.”
The samples previewed in last night’s meeting would suggest that RAJAR is still some way from introducing large-scale electronic measurement. However, Sally de la Bedoyere remained adamant that 2006 would see the introduction of parallel runs, comparing the new electronic system with the current diary method side-by-side, with a view to introducing a full-blown electronic measurement methodology in 2007, as previously planned (see RAJAR To Upgrade Audience Measurement System).
RAJAR: 020 7903 535 www.rajar.co.uk
Recent Radio Stories from NewsLine RAJAR Exonerated As Wireless Court Case Struck Out Ofcom Offers FM Radio Licence In Ballymena BBC Claims Success For Radio Download Service
Subscribers can access ten years of media news and analysis in the Archive