Response to CNN poll on support for Ukraine
On April 4, 2023, CNN and SSRS released the results of a poll that appeared to indicate a dramatic increase in opposition to continued US assistance to Ukraine.
A review of CNN’s poll data has led us to conclude that this poll is flawed, and the numbers it presents likely do not reflect a true shift in American public opinion as claimed. This is backed up by multiple polls taken in the same period that show no such change in US opinions on aid for Ukraine.
I’m sure we all know not to put too much stock in the results of any single poll. However, we wanted to flag some issues with this particular poll for a few reasons:
- The striking difference between the numbers in this poll and in all previous major national polls from this year;
- The wide coverage these poll numbers have gotten in the media;
- Bad question phrasing, in violation of several best practices widely accepted in the polling industry.
After reviewing the original question phrasings in the CNN/SSRS poll and comparing with past poll results, we came to the conclusion that there are serious problems with this latest poll. We believe that the CNN poll results do not justify the claims made regarding US support for aid to Ukraine.
We suggest waiting for additional poll data before drawing conclusions based on these numbers.
Among the several significant problems is the fact that a significant fraction of respondents would have been stuck without a proper way to answer at least one of the critical questions – forcing them to select between three options, each of which would provide incorrect information about their opinion.
This is not a small fraction of respondents. According to past polls, we estimate that a minimum of 29% of respondents would have fallen into this trap, with the actual share probably far higher, possibly even exceeding 50% of all poll participants.
With these irregularities, it is not surprising that the CNN/SSRS poll reported such strikingly odd results.
In this report, we lay out the original poll questions and compare them with polling industry best practices.
The poll
Q15) Do you think the United States should do more to stop Russian military actions in Ukraine, or has it already done enough? / (Do you think the United States has already done enough to stop Russian military actions in Ukraine, or should it do more?)
Should do more | Has done enough | No opinion |
48% | 51% | 1% |
Q15a) Do you think Congress should or should not authorize additional funding to support Ukraine in the war with Russia?
Should authorize additional funding | Should not authorize additional funding | No opinion |
45% | 55% | * |
The headlines
- CNN Poll: Majority of Americans oppose more US aid for Ukraine in war with Russia | CNN Politics
- Most say Congress should not authorize more funding for Ukraine: survey | The Hill
- Opinion: Why Republicans are turning against aid to Ukraine | CNN
- Polls Show Waning GOP Support for Ukraine Aid
- Poll shows most Americans oppose additional Ukraine funding – WDEF
- Poll: American support for Ukraine aid is falling – GZERO Media
The poll has also been eagerly picked up by the usual state media fronts in Russia (RT, Sputnik), and Iran (Press TV).
Problem 1: “Aid” vs “funding”
From CNN’s initial article presenting the poll results:
Overall, 55% say the US Congress should not authorize additional funding to support Ukraine vs. 45% who say Congress should authorize such funding. (Emphasis ours.
“CNN Poll: Majority of Americans oppose more US aid for Ukraine in war with Russia”, CNN, 4 Aug. 2023)
Note that in the article headline, CNN asserts that a “majority of Americans oppose more US aid for Ukraine”.
“Aid” is a term widely used to refer to assistance in the form of money and/or other goods.
However, poll question Q15a uses the word “funding” – a word that most reasonable readers would interpret as referring to the designation of money to serve a particular purpose. In fact, much of the aid supplied to Ukraine by the US is not in the form of money or funds, but rather in the form of supplies and equipment (in particular, military surplus).
Even if CNN did have firm reason to claim that most Americans oppose additional funding to support Ukraine (a claim we are skeptical of – see below), CNN lacks grounds to make such a definitive statement about the public’s attitude toward “more US aid for Ukraine” as a whole.
Breaking down the poll questions
“Do you think the United States has already done enough to stop Russian military actions in Ukraine, or should it do more?”
This poll question goes against accepted best practices for survey questions (source: Pew, AAPOR, Krosnick & Presser (2009)). Some widely accepted guidelines for composing poll questions are:
- Avoid ambiguous wording;
- Make response options exhaustive and mutually exclusive, and include all reasonable options;
- Ask about one thing at a time (avoid “double-barreled” questions).
This poll question is essentially two different questions in one:
1) Do you think the United States has already done enough to stop Russian military actions in Ukraine? | Asks for an assessment of past actions |
2) Do you think the US should do more to stop Russian military actions in Ukraine? | Asks for an opinion on desired future actions |
In addition, there are multiple possible interpretations of each half of the question:
Should do more, 48% | Has done enough, 51% | No opinion, 1% | |
Possible readings: | 1) The US should continue to aid Ukraine. 2) The US should increase its aid to Ukraine. | 1) The US has done all it should do for Ukraine (implied: with no cause for further assistance). 2) The US has been providing an appropriate amount of aid to Ukraine. | – |
The two main options are not mutually exclusive. They also do not enable respondents to accurately convey certain views that we might reasonably expect many of them to have.
For example: A respondent who is satisfied with the current level of aid to Ukraine, and wishes aid to continue at the current rate, has no way to properly answer this question. The third option – “No opinion” – is not helpful. As the respondent does hold an opinion, this option is arguably the least correct of all three options.
Respondents who support current levels of US aid to Ukraine have in fact made up the largest group in most previous 2023 polls that posed a similar question. This segment of respondents has been shown to constitute 29%-43% of the representative survey sample (see graphs below). If we extrapolate from past polls, we come to the conclusion that a third or more of responders facing this question are forced to choose between options that are all incorrect. There is therefore reason to suspect a significant impact on the accuracy of the poll results.
“Do you think Congress should or should not authorize additional funding to support Ukraine in the war with Russia?”
Again, this question leaves a considerable amount of ambiguity.
Should authorize additional funding – 45% | Should not authorize additional funding – 55% | |
Possible readings: | 1) Congress should authorize another round of aid (money AND equipment/weapons transfers) to Ukraine once the current round of funding expires. 2) Congress should authorize another round of funding to Ukraine once the current round of funding expires. 3) Congress should authorize an increase to US aid (both money-based and otherwise), in addition to the aid already approved. 4) Congress should authorize an increase to US funding, in addition to the funding already approved (without addressing other forms of US assistance). | 1) Congress should not authorize any more aid whatsoever for Ukraine once the last round of aid runs out. 2) Congress should not authorize any increase to the amount of aid that the US is sending to Ukraine (both funds and equipment/weapons transfers). 3) Congress should not authorize any more funding for aid for Ukraine once the last round of aid runs out (not addressing the question of non-funding assistance, such as transfers of military surplus). 4) Congress should not authorize an increase to the amount of funding that the US is sending to Ukraine (not addressing the question of non-funding assistance). |
A more appropriate way to frame the question would be something along the lines of the following:
In your opinion, Congress should (choose one):
- continue to authorize assistance to Ukraine in order to maintain approximately the current level of US aid to Ukraine;
- (refuse to authorize any and all future aid to Ukraine – including refusing to continue all current aid once its authorization expires);
- decrease the level of US assistance to Ukraine;
- increase the level of US assistance to Ukraine;
- Don’t know / no opinion.
Context
Did the ambiguity in the CNN/SSRS poll questions affect the poll’s final data? Let’s compare the poll results with previous polls from this year.
To view the original poll questions, please access this Google Spreadsheet. For a list of all polls examined, with links to original poll data, see this Google Doc.
* Poll lacks a neutral / don’t know option
** Poll question is different from the other poll results compared in this chart. It is presented here for a loose comparison, not as part of the same series.
Note that two other polls were conducted over segments of the same period as the CNN/SSRS poll (during the month of July).
We can see from the first graph above that the fraction of Americans who think that US aid to Ukraine is too high, has been remarkably consistent throughout spring/summer 2023 (29%-33%).
CNN’s claim that 55% of Americans support cutting off Congressional aid to Ukraine entirely, is a dramatic departure from all previous opinion surveys, and should raise immediate red flags.
Conclusions
After an examination of the CNN/SSRS poll (conducted 01-31 July, 2023):
The Ukraine Fact Check team is of the opinion that CNN’s poll results do not justify the claims made by CNN. These claims include:
- “Majority of Americans oppose more US aid for Ukraine in war with Russia”;
- “Most Americans oppose Congress authorizing additional funding to support Ukraine in its war with Russia”.
The questions presented in the poll do not follow polling industry best practices. We suspect the final numbers presented by CNN have been significantly distorted by problems with the poll questions. These problems include: ambiguous wording, “double barreled” questions, and extremely limited forced answers.
We suggest that:
- CNN retract its findings for questions Q15 and Q15a of this poll;
- news outlets that reported CNN’s findings update their published articles to reflect the lack of confidence in the poll results.
We also recommend that anyone following the subject of US public support for aid to Ukraine, wait for new poll data before drawing conclusions.