Polls, ‘weighting’, and misleading the public
A recent poll, apparently showing Labour has closed the Tories’ lead in the polls to just five points, has caused a bit of a stir among Vox Political readers.
It seems that some of you are unhappy that Steve Walker’s recent article referred to the ‘raw’ figures, rather than those produced by Ipsos Mori after the company added arbitrary ‘weighting’ of its own.
So Rachel Hodge wrote: “I’m sorry but your reporting of polling is woefully inaccurate, the figures you quote are before weighting. The headline figures show a 12 point lead for the Conservatives. Most of that is a new leader bounce and shouldn’t be taken seriously, it will fall back within a few months. Judging by historical precedent the new leader bounce is probably about 10 points which puts the Tories about 2 points ahead. But we will have to wait and see.
“So, why the difference between the raw figures and the headline figures and why is it important? The difference is likelihood to vote, the headline figure is discounting people who are Labour supporters but say they aren’t certain they will vote. This is a real problem for Labour and ignoring it won’t make it disappear. We could reasonably assume that some of the softness in Labour support has to do with the coup and the bad publicity it has caused but that’s not the whole story. Personally I think the softness in Labour support has its origins in the expenses scandal, no one cares if Tories cheat the public, that’s what Tories do. But Labour MPs cheating the public is much more damaging.”
And our old friend and resident Tory, Hayfords, added: “The actual figures are CON 45%(+9), LAB 34%(-1), LDEM 7%(-4), UKIP 6%(-2), GRN 4%(nc). The chart you are using is only the left hand side from MORI
“Here is the narrative that goes with it on Ipsos/MORI’s web site: ‘Voting intention figures show the Conservatives widening a lead over Labour to their highest since 2009. The Conservatives currently stand at 45%, compared with Labour at 34%, the Liberal Democrats at 7% and UKIP at 6%.’
“This is the full chart. The web address is [here].”
Before going to Steve Walker for his reply, it’s worth pointing out that politics is going through a period of extreme change. Recently there has been a large number of people who say they support Labour but don’t vote – and it’s entirely possible that this is due to the poor behaviour of Labour MPs. But we have a new leadership and a new approach. Jeremy Corbyn and his supporters are all about bringing lapsed Labour supporters back in – and throwing out any corruption, any disloyalty to Labour and its ideals. That is starting to resonate with the wider public. It has taken a while because we have hostile media trying to support the status quo – but consider the result of the Victoria Derbyshire hustings last Wednesday, when all but four of the ‘undecideds’ in the audience moved to Jeremy Corbyn’s side at the end.
Okay, over to Steve. I promised to contact him about his choices in writing the article – and did. He was happy to provide this response:
“The 4% is the result of the raw data. The 12% is the result after they play around with what they *think* it means and how they *think* people will act based on their expressed intention.
“Used to see it all the time in unemployment data etc – huge increases in unemployment would turn into a fall after ‘seasonal adjustment’ – but the figures in the same season the previous year would often have gone the opposite way.
“Even if you trust the people doing the adjustment to do it honestly, in the end effectively they’re just guessing – and we’ve all seen in recent elections how wrong the guess-based expectations have been.
“There’s a huge disconnect between the not-yet-changed paradigms of political commentators and pollsters and what’s actually happening out there, so you have to at least ask the question whether any of their adjustments are likely to be correct.
“Recently, the results of the raw data have been much more accurately reflected in actual voting results, so I know which way I tend, at least until the pollsters have a clue what’s going on in the real world!”
Some of you are undoubtedly already sharpening your keyboard fingers to provide a pointed reply. “You know what they’re going to say already, of course,” I suggested to Steve.
“Of course. But the relentlessly wilful ignoring of the significance of actual results in favour of ‘lick your finger and stick it in the wind’ (or outright skewed) polling needs to be confronted head on sometimes.”
Yes it does. With polls making all sorts of wild suggestions that aren’t supported in elections, it is right to question the accuracy of the ‘adjustments’. Isn’t it?
ADVERT
Join the Vox Political Facebook page.
If you have appreciated this article, don’t forget to share it using the buttons at the bottom of this page. Politics is about everybody – so let’s try to get everybody involved!
Vox Political needs your help!
If you want to support this site
(but don’t want to give your money to advertisers)
you can make a one-off donation here:
Buy Vox Political books so we can continue
fighting for the facts.
Health Warning: Government! is now available
in either print or eBook format here:
The first collection, Strong Words and Hard Times,
is still available in either print or eBook format here:
I would point out that the weighted figures do more closely match actual election results. I say more closely, but in fact they do tend to overestimate Labour and underestimate Conservatives. That is one of the reasons why the Conservative majority was not predicted by the polls. This would suggest that Labour is currently further behind. But based on current polling, thus the potential Parliament
UK General Election Seat Forecast
CON: 411 (+80)
LAB: 153 (-79)
SNP: 54 (-2)
PC: 6 (+3)
LD: 5 (-3)
GRN: 2 (+1)
UKIP: 0 (-1)
NI: 18 (-)
Labour should be well ahead and shouldn’t be behind now were the party on track to win the next general election. Statistically Labour needs something like a 10% swing towards it to get a majority of one (or two) based on the current constituency boundaries. (The forthcoming new boundaries require a bigger swing.) As support tends to diminish as a general election looms in reality Labour needs to be ahead by more than 10 percentage points now to have a chance of beating the Tories in 2020. Labour’s only real hope is that something truly awful happens under the Conservatives, given a shot in the arm via a new leader, prompting non-affiliated voter to transfer support away from the Tories, some of it going to Labour, but it would have to be an event of very great negative magnitude attributed to the current government to trigger a miracle like that and make Corbyn palatable as a PM to the masses.
Something that bad however would probably be worse than another Labour defeat.
Hindsight’s a wonderful thing.
Of course, here in July 2017, we know that Labour achieved that 10 per cent swing, but still didn’t get its majority.
The boundary changes we’ve all been expecting aren’t likely to happen now, as there’s no appetite for them in Parliament after the election.
I haven’t heard from Tim in a while.
I wonder where he is?
The most recent Ipsos Mori Poll does not look good for the Labour Party:-
Voting Intention in Great Britain: Recent Trends
Figures based on all those absolutely certain to vote.
Published:17th August 2016
2016 Con Lab Lib SNP/PC Green UKIP Other Lab lead
% % % % % % % %
13th – 15th August 2016 (T) 45 34 7 4 4 6 1 -11
(T) = Telephone polls
Source: https://www.ipsos-mori.com/researchpublications/researcharchive/107/Voting-Intention-in-Great-Britain-Recent-Trends.aspx?view=wide
It may be due to a “bounce” for May but looking at the series of polls, it seems to me that that whereas Labour has stayed relatively constant at around 34 to 35 per cent since March 2016, the Tory vote has risen from 36 to 45 per cent over the last month.
Part of the explanation behind the rise in declared support for the Tories can be put down to a softening in support for the other parties, i.e.: UKIP have lost around 5 per cent since the start of 2016; the nationalist parties have also lost around 2 per cent; and support for the Lib Dems has declined around 4 per cent over the last month.
Given the degree of division obviously apparent within the Labour Party, it could be said that these figures are not perhaps so bad. Looking at earlier sets of poll figures, it is apparent that the coup plotters are having an effect on the popularity of Corbyn and the Labour Party, even though public rallies show a high level of support among party members for the existing Leader and new party arrangements.
It is when we look at the individual leader poll ratings that the May “bounce” becomes apparent but also – again – the coup plotters’ “effect” – combined with a massive mass media bias against Jeremy Corbyn – becomes apparent.
Ipsos MORI Political Monitor August 2016:-
Theresa May is enjoying a warm honeymoon welcome with the public according to Ipsos MORI’s latest Political Monitor. More than half (54%) say they are satisfied with Ms May’s performance in her first month as PM while one in five (19%) say they are dissatisfied – leaving her with a net satisfaction score of +35. This is comparable to her predecessor David Cameron’s rating in June 2010 when 57% were satisfied with him doing his job as PM and 26% dissatisfied. Although higher than those of John Major and Gordon Brown in their first month as Prime Minister, May’s satisfaction ratings are still behind Tony Blair’s first month back in May 1997 (65% were satisfied and 5% dissatisfied).
The new poll reveals that Jeremy Corbyn continues to struggle with public approval. One in four (25%) say they are satisfied with Mr Corbyn doing his job as leader of the Labour party and 58% are dissatisfied – leaving him a net satisfaction score of -33. Although better than last month when the Labour leader had a net score of -41 he is still well behind Ed Miliband’s rating after his first year as leader in September 2011 (31% were satisfied and 47% dissatisfied).
More encouraging for May are the satisfaction ratings she enjoys among supporters of both her own and the Labour parties. Three in four (76%) Conservative supporters are satisfied with Ms May (4% dissatisfied) as are 45% of Labour supporters (29% dissatisfied). Jeremy Corbyn’s ratings among his own party are lower than they are for Theresa May with two in five (39%) satisfied and 47% dissatisfied.
Tim Farron has still made little ground with the public where one in five (22%) are satisfied with him doing his job as leader of the Liberal Democrats (30% dissatisfied) and half (48%) say they don’t know. Three in five (61%) Liberal Democrat supporters however say they are satisfied with Mr Farron and 15% are dissatisfied.
Technical note
Ipsos MORI interviewed a representative sample of 1,017 adults aged 18+ across Great Britain. Interviews were conducted by telephone 13th-15th August 2016.
Data are weighted to the profile of the population.
Source: https://www.ipsos-mori.com/researchpublications/researcharchive/3772/New-Prime-Minister-starts-with-a-honeymoon-among-the-public.aspx.
How representative a telephone poll of just over 1,000 respondents is may well be arguable but it is the trends that usually are relevant over time.
The Labour Party will have to carry out a great deal of work to raise Jeremy Corbyn’s profile with the public in a positive way after he is re-elected Leader of the Labour Party. Whether or not the present hierarchy is up to that job is very much open to question. There may have to be root and branch reform at all levels.
I was standing next to Diane Abbott in 1983 in a Bayswater pub on general election night after the polls had closed when the BBC announced the result of their exit poll. They announced what appeared to be a smashing victory for the Tories. Everyone in our group was stunned momentarily. Then they started swearing and muttering “The stupid bastards” – by which I assumed they meant the electorate.
I responded that we should not blame the electorate. It was our fault if we had failed to get our message across. Needless to say, what I said was not what those present wanted to hear. However, I still think I was right in what I said.
The 1983 general election outcome can still provide us with a lesson today.
If we are all serious about wanting a Jeremy Corbyn-led Government, we are all having to go out every week and work hard for it. The media will not help us. The other political parties will not help us. We may even find elements within the Labour Party will not help us. We will have to help ourselves. It is a big commitment.
If I were asked today whether I favour Tories, Labour or Green the answer would be a NO, No and Yes respectively. Until I know what kind of Labour Party JC is going to let us have, I will not be voting in favour of Labour. If right wingers remain in the party the instability will continue and no progress will be made. As long as they remain, JC cannot offer us with any degree of certainty that he will be able to do as he has promised. Whether he, or you, believe otherwise, is immaterial, it is how the electorate and labour membership view the situation. There is a saying “there is no rest for the wicked” and they will not rest. Ever. They would be willing to lose the 2020 election in order to ensure that Corbyn and his policies do not succeed, to actually believe they would play along, even to the point of winning, is a false hope. They may play along just long enough to damage JC and his followers with the intention of damaging them both beyond repair. They really do believe they can win a 2020 election with half of Labour’s supporters voting elsewhere.
I personally think JC’s policies are the answer to Britain’s woes IF he can implement them with the support of his fellow MP’s. Therein lies the problem as most people I talk to, see it.
I tend to agree with you mohandeer x
I’d far prefer the raw figures, with a commentary that shows what the person writing the piece thinks and their reasoning as a footnote. Then I can read it and see if I agree with their logic – and therefore accept their adjustments – or whether I’d prefer to take the raw figures as reasonable; or whether I’d actually prefer to make my own assumptions that are totally contradictory to those the writer makes!
The trouble is not using the raw data, but the comparison with other polls, and not on their raw data.
In the one earlier Ipsos Mori poll, on the raw data, Labour was ahead by 4%. Comparison among pollsters’ results is highly problematic as they use different kinds of adjustments. Ipsos Mori essentially uses the 2015 election turnout figures, and uses these to adjust the raw figures. The fact was that Labour sympathisers did not vote.
It is possible that it has changed (but there is no evidence for it at all).
So, using figures based on the same methodology, Labour dropped 9% in Ipsos Mori since the coup, which sounds reasonable (as the July Ipsos Mori showed Labour at the top of the margin of error – it was an outlier by all likelihood. The current one could also be).
Labour is probably around 6% behind the Tories, it is worse than before the coup, but only about 3% higher than the average since the leadership election. It is not a conspiracy, it is something that Labour has to deal with.
But isn’t the adjustment – along the methods you described – likely to create a self-fulfilling prophecy?
If Labour support overall is up to just four per cent behind the Tories, even with Theresa May’s ‘honeymoon period’, then this would indicate an increased willingness to vote Labour.
But then the ‘adjustment’ goes in and people see a poll showing Labour down to one of its lowest points – based on nothing but voters’ unwillingness to support the fragmented ‘Tory-lite’ policies on offer last year!
There is manipulation by the media without any doubt, but not by the polling companies (even if one can roll out the ownership of YouGov and alike). They had to do something about their awful performance in May 2015 (political polling is not their main income, but credibility is important).
Essentially they didn’t find enough Tory voters (as these don’t respond as willingly to a polling call or online invitation), and hence found too many Labour ones, who then didn’t turn up and didn’t vote. Their panel members did not lie, only that the panel was wrong. (There are other issues, for example that newspapers’ polling budget is simply not big enough for the polling companies to do the job properly – you see, the 3% margin of error (Ipsos’s is 4%) doesn’t really matter if it is about different brands of baked beans, but mightily important if it’s about the outcome of the elections).
Ipsos Mori essentially drove through their data from May 2015, and then checked what turnout distribution would yield the actual election outcome. It could be wrong, of course. What they found was that in every age group labour supporters are less likely to vote, and especially less likely among the age group where labour support is the highest (under 35) and the socioeconomic group (unskilled and semiskilled workers).
So, to make sure that the adjusted figures do not become sel fulfilling prophesies, the point is not criticising the polls, but getting the youngsters and the workers to register (we have a few million who aren’t) and then they actually persuading them to go and vote (quite a few will vote for other parties, but that doesn’t matter).
The most interesting group is the people who haven’t voted for ages, but they pulled themselves together and went out to vote for Leave (most of them anyway). It’s very likely that they would reduce or even eliminate the Tory lead if the right approach to them could be found.
In fact they found that Labour supporters WERE less likely to vote, in May 2015.
Far too much has changed since then for this to be an effective yardstick.
Mike,
“Far too much has changed since then for this to be an effective yardstick.”
It is true, but they don’t have anything else. Also the evidence (it’s a too strong word) is contradictory. With a higher turnout the Kent council seat wouldn’t have been lost and the two seats in the Thanet DC on Thursday would probably have been gained. But on the other hand in Tooting a relatively higher turnout (about 20%, still very low) delivered.
I agree with the commenter who said the pollsters should publish the raw figures and then produce narratives explaining what they think it means, the weightings they have applied and why they think those are relevant.
They do.
https://www.ipsos-mori.com/Assets/Docs/Polls/pm-august-2016-topline.pdf
Actually, I don’t know how much has changed. This is from May of the members who joined after May 2015. It could be wrong of course (based on a sample over 2,000).
Just over two-thirds of Labour’s post-GE2015 members and supporters (68 per cent) have retweeted, posted or forwarded a message supporting the Labour party on social media and nearly nine out of ten (88 per cent) claim to have signed a petition on behalf of the party. But only 15 per cent of them have participated in door-to-door or telephone canvassing of voters or helped out at a party function, and only 28 per cent of them claimed to have delivered leaflets. Indeed, some 63 per cent said they had put in no time at all on behalf of the party during recent local, mayoral and devolved elections.
fairy tales hay they believe they in front then let may call a election hmmm corbyn even without every body behind him will sail through lets be going back to those tory figures only 24 percent voted them in ouch then lets go Teresa
I see that Sadiq Khan has said in the Observer, ditch Corbyn or lose to Tories.
Article on the way.
To say that this is backfiring badly on him would be one of the understatements of the year.
Jeremy Corbyn supporters do sound an awful lot like Trump supporters at times.
In what way?
Considering how Trump supporters behave I find that hard to believe.