One of the hardest stories a researcher has to tell is one that the client disagrees with. They have a vested interest and research is as much art as it is science so they can contest it. It is our job to acknowledge its limitations, defend its strengths and say what we can learn from it.
We’ve just conducted a robust poll of Brighton & Hove residents on political affiliation and attitudes to legal assistance to die (for the detail people out there, it was 1,000 representative quota sample on age, gender and working status based on 2001 census data, conducted by telephone by ICM Direct in accordance with MRS code of conduct).
The political affiliation question is interesting because Caroline Lucas of the Green party is in with a good chance of taking the Labour seat of Brighton Pavilion and becoming Britain’s first Green MP. An ICM poll commissioned by the Greens in December suggested that they were ahead of Labour and Tory. Past electoral results also add weight to this. In our independent poll in February however, we asked a different question which suggested that support for the Greens was lagging behind both Labour and Tory.
The outcry from Greens that followed its appearance in the local Argus newspaper for example here, here and here are typical shoot-the-messenger protests when it comes to research:
The sample size is too small - this is counter-intuitive to most people but the key criterion determing accuracy is how representative it is (ie how respondents have been selected) not how many interviews there are. 336 is a good number for a poll and the large difference between support for each of the parties cannot be explained by sampling variance.
There is nothing for us to learn here – there is always something to learn! The key question the Greens have to ask themselves is why do only 12% of Pavilion residents say they would vote for them in a general election tomorrow compared to 26% for Labour.
Twist or ignore the interpretation – I published the full report here and made clear in the interpretation the limitations of our raw findings and the conclusion that might be drawn.
Claim that other research is right so this research must be wrong - The main factor that explains the difference between the Green party poll and the Kindle poll is the question. The Green party question told respondents first that a general election was taking place and secondly the names of 4 of the parties who would be contesting. This practice lifts mentions of lesser known parties.
Cast suspicion on the motives – independent researchers do have allegiances but what we are primarily interested in is truth and learning. Some clients may want to put a spin on research findings to serve their own interests but when they spend money they want to find out what is going on so they can invest wisely.
Lack of experience – the sample is sound, the method is sound and the questions follow standard practice.
Love it? Hate it? We want to know: Leave us a comment (20 so far)
