Reagan vs Obama and McCain

I thought it would be interesting to look at the level of spin in Reagan’s speeches. He shares some characteristics with Obama; not in political opinions but in his ability to motivate an audience, and to be resistant to potentially embarrassing factual issues.
Here is the plot from yesterday’s post comparing Obama and McCain since their conventions, with five campaign speeches (all I could find) by Reagan between the convention and the 1980 election.

Comparing the spin of Reagan, Obama, and McCain

The points with red stars are Reagan’s speeches. As you can see, his level of spin is much higher than either of today’s candidates. The ability to use high levels of spin without coming across as phony is, of course, what makes an actor, so this is not entirely surprising. And I’ve argued all along that high levels of spin pay off for a politician, and the ability to give high-spin speeches especially to people who do not already like you is a key asset for a politician. Reagan is a good example of this in action.

Spin scores to the end of October

Here is the analysis of levels of spin in Obama and McCain’s speeches up to a few days ago. Usual labelling (refer to previous posts for background).

Spin scores (red - McCain, blue - Obama)

The most obvious thing to see in this plot is how McCain’s speeches all tend to lie on one side of the deceptiveness axis while Obama’s tend to lie on the other side. This is because McCain has started using motion words at high rates (and Obama does not). This has a small effect on deceptiveness score, but rates of use of motion verbs are not all that important to signalling deception.

The individual levels of spin from the convention to the end of October are here:

McCain

Obama

The last time I posted was during the period where Obama’s level of spin was quite low. As you can see, it has risen sharply again in the past week. This suggests that he is not as confident of winning now as he was then — he has consistently shown a pattern of stepping out from his facade and using lower spin when he feels confident about winning.

Update — Spin in US Presidential Election

I’ve looked at the spin in the speeches in the last few weeks, more or less since the convention. The overall picture has remained very similar to earlier stages: Obama’s levels of spin are relatively high compared to McCain’s.

Here is the overall spin plot:

Spin between the conventions and the last week of October

Here are the plots over time for McCain:

McCain's spin

and Obama:

The most interesting thing here is how much Obama’s level of spin has dropped in the last few speeches. This is very similar to what happened in the weeks (late February) when it became clear that he would get the Democratic nomination. When he feels sure of himself, he steps out from behind his election facade and presents himself much more openly. The extremely low-spin speech is his comedy routine at the Al Smith dinner — presumably the expectation that he should be funny rather than serious made him feel as if he had permission to be himself.

McCain’s high-spin speeches are those when he gives economic history lessons, unlike his typical speech in which he puts more of himself.

Canadian Federal Election – 1 day to go

The results for the three English-speaking party leaders are shown below. I have not tried to track Duceppe because we don’t fully understand how to move the deception model across languages. I think there’s a fair amount of confidence that the same (types of words) are significant in other languages, but the details are difficult. In the case of French, the pronoun “on” sometimes plays the role of “I” and sometimes not, and the differences are hard to pick out, and might perhaps be especially significant as a distancing mechanism.

The Canadian federal party leaders have all tried to run presidential style campaigns (Vote the man, rather than the party, or the local member). But they can’t bring themselves to talk about themselves, so the speeches they all give are extremely abstract blue-skies policy speeches, with hardly an “I” to be seen. Relative to the U.S. election, all of the speeches would rank as high spin. This may be partly a perception (probably accurate IMHO) that Canadians are not ready for prime minister = president and so the leaders are trying to be the face of their party, but not the single leading figure. This middle-ground approach seems to be a bit clunky, and produces some odd speeches. Can you imagine an American politician at any level giving a speech titled “New Support for Apprentices”?

Here is the spin ranking for the speeches available up to today:

Spin rankings over the entire election campaign

The pattern has been reasonable consistent over the campaign. Stephen Harper’s (blue) speeches tend to be high-spin; Stephane Dion’s (red) speeches are more moderate in spin, but much more variable; and Jack Layton’s speeches (such as they are) (magenta) are relatively low spin. This would indicate that Stephen Harper will tend to do better than the other two, but these speeches play such a small role in the campaign that not much should be read into this prediction.

Obama in the rest of the world

There’s a story in the Sydney Morning Herald this morning making the point that three-quarters of Australians want Obama to win. This is consistent with the reaction to Obama in Europe and, I suppose, many other parts of the world.

But how do they know? Part of the answer at least is that they’ve heard snippets of his speeches — and this is exactly what you’d expect of blue-skies policy speeches. Such speeches reach out to those who are far away.

The danger is that increased exposure to blue-skies policy speeches backfires because the lack of connection to the actual person starts to make them seem too artificial. This doesn’t happen in far-away places because there isn’t enough exposure. But it’s a real possibility in the U.S..

Canadian election — some results

Although the party leaders are trying to run a presidential election (vote for Harper, Dion, Layton, Duceppe rather than vote for a party or (heaven forbid) a local member), they are not doing a very good job of presenting themselves as people. The speeches that are available online are little more than press releases read out loud. The contrast with the speeches of the U.S. contenders couldn’t be more stark.

However, here are the spin values for the available speeches, 12 from Harper in blue, 3 from Dion in red, and one from Layton in magenta.

Party leaders spin to Sep 29

Speech 13 from Dion has such (apparently) low spin on the strength of a few paragraphs near then end in which he becomes slightly personal. The Harper speeches are all blue-skies policy speeches. He does not tend to take credit personally, but rather attributes the policies to the government or to the Conservative Party.

High-spin speeches are usually a good strategy, so Harper is probably doing better than the other two leaders. But this kind of generality seems not to attract anyone very strongly. Maybe that’s all Harper is going for — a safe pair of hands argument. But you can’t help but think that Bill Clinton could give all three of them pointers.

Spin — the technical basis

The work on spin that I’ve written about here is based primarily on the work of Pennebaker’s group at the University of Texas, Austin. The primary reference is here. The model is based on empirical studies of deception in settings where the ground truth is known, and has, by now, been validated many times. When someone is being deceptive, there will be characteristic changes in the ways they use certain words. Deception here means, of course, saying things that the speaker does not believe to be (entirely) true, not things that are factually incorrect.

Of course, this is domain dependent, so we can’t judge how deceptive politicians are in comparison to, say, used-car salesmen or nuns.

The use of the word “spin” rather than deception acknowledges the fact that there are differences between explicit intent to deceive and implicit, unconscious desire to present oneself as better (along some set of dimensions) than one actually is. This kind of self-improvement happens in job interviews, dating, and in politics.

And we don’t condemn someone for being deceptive when they make an initial offer in a negotiation, but the same kind of signals will appear in their language use.

The interesting thing is not so much that politicians try to appeal to as many people as possible, at the expense of strict accuracy, but that there are differences in how much this happens — changing over short time scales for a single person, and at longer time scales for a campaign; and that there are systematic differences between campaigns. There is a great deal of evidence that the properties mediated by changes in language patterns are not under conscious control (see e.g. Pennebaker and Chung) so they provide an insight into campaigns that is hard for the campaigns to obscure.

Basic ideas

Because a lot of people are coming here because of the New Scientist article, and perhaps don’t want to have to read through the back issues, I thought it would be worthwhile to review the basics.

Sometimes when people talk about spin they mean attempts to reinterpret generally-agreed facts in a way that makes one or other candidate or party look better. Sometimes this involves mistating facts, but this is usually a risky thing for a politician to do.

When I talk about spin what I mean is the way in which politicians represent themselves as different (nicer) than they are. In a perfect world, politicians might get elected based entirely on their competence, but in our world people seem to prefer to vote for someone that they like, or who seems to be someone like them. Politicians, therefore, have a lot of motivation to assume a persona or facade in which they seem more likeable and attractive than their base personality.

This presentation of an ‘enhanced’ persona has to be subconscious to really work. Only a few of us can do this consciously (people called actors, often paid well if they can do it well). So when it is happening, it can be detected by changes in subconscious behavior, in this case changes in language. Because it is largely subconscious, it provides a window into individuals’ situational view which can often be illuminating, and is often interesting as well.

For a politican, then, spin is a good thing, and a good politician will tend to be good at it. We blame politicians for being hypocritical, but they do it because it works with voters.

The limiting factors, interrelated, are how well a politician can use high spin, and the content of what s/he wants to talk about. A speech about, say, putting elephants on Mars is easy to make high spin because nothing in the speech depends on the person giving it — it’s like an argument at a debating society; there’s nothing personal invested in it. A speech that depends on what a politician has done in the past, or what s/he plans to do because of some individual skill or goal is harder to give with high spin exactly because it relates to the speaker in a tighter way. A politician who can give such a speech in a high-spin way has a huge advantage. A speech that is intended primarily for those who already support a candidate and tries to relate to them in a personal way is even more difficult to give in a high-spin way (part of Bill Clinton’s success might have been because he seems to be able to do this).

Almost all of the speeches in the U.S. election campaign can be understood from this perspective. The convention speeches are especially useful because, as a set, they all try to do more or less the same thing. But the level of spin achieved by the different speakers differs substantially.

The brief history of the campaign is this: Obama uses moderately high levels of spin throughout, varying because of the kind of speeches he gives. McCain did inded start out with straight talk, but his levels of spin quickly matched Obama once it was clear that Obama was the Democratic nominee. Clinton’s level of spin started fairly low, because she loves to get into policy intricacies. When it became clear that Obama was likely to win the nomination, she moved to a much more personal-appeal strategy with even lower levels of spin. This worked well for her with her base, but not so much outside of it, which is what you would expect for this kind of strategy.

At their conventions, Obama gave a fairly typical speech. McCain tried a risky strategy in which he used very (!!) low levels of spin. It’s not clear whether it worked for him, because both speeches were so overshadowed by the choice of Palin as VP nominee.

Election spin results covered by New Scientist

There’s an article on, among other things, my work on spin the US election campaign in this week’s issue of New Scientist. You can find it online here. There are also some interesting results from the people who look at facial expressions (pioneered by Ekman) and so voice analysis.

Canadian Federal Election — first analysis

I’ve done some analysis of the party leader speeches from the first week of campaigning. I am somewhat hampered by lack of data; only one speech by Jack Layton has been posted, and there are only three by Dion.

About the only conclusion so far is that language patterns are all over the place. There is no consistency in any of the campaigns. This might be because speechwriters are still settling in (but surely Harper’s at least must have known the election was coming?). Or it may be that all of the campaigns are unsure of their strategies in some less-conscious way. But it’s interesting, and slightly puzzling.

I’ll await more data.

Comparing the Democratic and Republican convention spin

Just to round out the picture, here’s data comparing the levels of spin across the speeches from both conventions.

Spin levels in both conventions

As usual, high spin is indicated by the red end of the line. Here are the spin scores for all of the speeches analyzed (positive numbers are high spin):

1. Bush 0.40
2. Thompson 1.71
3. Lieberman -0.73
4. Romney 4.36
5. Huckabee -1.8
6. Giuliani 2.97
7. Palin -0.62
8. McCain -7.38
9. M. Obama -1.24
10. Hillary Clinton 2.43
11. Bill Clinton 0.99
12. Biden -1.35
13. Obama 0.31

The relative positions don’t change much when compared to the other party. Within each party, there’s the same division into high- and low-spin speakers. McCain’s overall spin is still so low that it makes all of the other politicians look as if they are clustered quite close together, but there are actually quite large differences between them.

Note the cluster of Bush, Palin, Bill Clinton, and Biden who have very similar speech patterns. Perhaps this is the sign of a successful politician?

RNC convention spin summary

McCain’s speech last night had extremely low spin, mostly because of the very high rate of first-person singular pronoun use, almost twice the rate of Bush and Huckabee. It’s so large that it almost washes out the differences in the rates of use of other word classes.

Here’s the plot:

RNC speech spin

The speeches are in chronological order: 1 – Bush, 2-Thompson, 3-Lieberman, 4-Romney, 5-Huckabee, 6-Giuliani, 7-Palin, 8-McCain.

Note the alternating pattern of high and low spin, which is also characteristic of speeches on the campaign trail. We suspect that this is the result of (probably subconscious) awareness of spin that feeds into the scheduling.

The actual spin scores (positive means high spin are)

Bush -0.92
Thompson 4.02
Lieberman -0.95
Romney 2.43
Huckabee -1.88
Giuliani 3.72
Palin 1.17
McCain -7.58 (!!)

Spin in the RNC convention speeches

I’ll do a complete analysis after McCain’s speech tonight, but so far:

High spin: Thompson, Romney, Giuliani — all about the same level of spin but for different reasons.

Low spin: Bush, Lieberman, Huckabee — all about the same

Palin’s level of spin is almost exactly in between these two groups.

Of course, adding McCain’s speech to the mix may change the absolute positions of these speeches, as Obama’s did to the Clintons’ speeches at the DNC (see last week’s posts).

I’ll do the combined analysis too, to get a sense of how different the two parties are in their lanaguage use — stay tuned.

Update on spin in presidential elections

I was away for three weeks, and it’s taken me three weeks to catch up and be able to post again (although sporadically for a while).

I’ve worked through the candidate speeches since the beginning of the year. There’s  a lot to look at: what was the effect of Clinto dropping out of the race, and the transition to general-election mode?

Here’s the short version: Obama continues to use high levels of spin, after a brief period (between the time he had the primary won and the Pastor Wright affair) when his levels of spin were reduced. McCain has increased his level of spin so that it’s little different from that of Obama — his straight talk levels dropped off a cliff once Obama became the presumptive nominee.

Here’s the overall plot of spin:

Spin levels to July 16th

The red dots are McCain speeches; the blue dots, Clinton speeches; and the blue stars, Obama speeches. The speeches for each candidate are numbered in time order: 1-32 Clinton, 33-77 McCain, 78-128 Obama. Note that Clinton now looks like the straight-talking candidate, because the calculation of spin is done relative to all of the speeches over the past 7 months.

Here are the spin scores for each candidate individually, in time order:

Clinton Jan 2008 to her withdrawal

McCain Jan 2008 to July 16

Obama Jan 2008 to July 16th

The change in McCain’s levels of straight talking are by far the most noticeable. Obama has had a few flashes of straight talk, but he seems most comfortable using moderate levels of spin.

Of course, spin works and that is why both candidates are using it. But neither can really claim that their campaign is not politics as usual.

Spin in the US Presidential Primaries — Summary

As we enter what looks like it might be the end phase of the primary season, I thought I would summarize what I’ve written about spin during the process.

1. What is spin? People often talk about spin as messing with the content of a communication: leaving bits out, or changing the emphasis. What I’m talking about here is a mental (unconscious) process where a person presents themselves or their content in a way that does not reflect what they know to be true about it. Politicians (and the rest of us) do this, to some extent, all of the time — trying to make a good impression. For a politician, outright lying is a poor idea (recall Clinton under sniper fire) but there is a lot of pressure to be “all things to all men”. Because the communication is not the speaker’s natural persona, this kind of spin produces a detectable signature in the communication.
2. What is the model of spin (deception)? This work is based on Pennebaker’s empirically-derived model of word-usage changes when people are being deceptive. This model is characterized by (a) reduced rates of first-person singular pronouns; (b) reduced rates of exclusive words, words that mark the beginning of a phrase or clause that qualifies or refines what has gone before; (c) increased rates of negative-emotion words; and (d) increased rates of action verbs. These changes are unconsciously produced, so cannot be directly altered by a speaker, even one who knows the model. Although the model was developed for plain deception (outright lying) it seems to detect deception across the full range from lying, through spin, to negotiating and dating.
3. Why does there have to be a context? Because the model relies on increases and decreases in word-usage rates, there must be some kind of context of similar communications or documents to be able to tell whether a given frequency represents an increase or a decrease. Therefore, absolute spin scores cannot be determined — instead we can only rank a set of communications from most to least spinful. Even within the context of the presidential primaries, underlying language use has changed, most obviously from a ”getting to know me” phase to a “getting to know my policies” phase.
4. The early primaries. From the beginning of 2008 until the 3rd week of February, all three candidates were introducing themselves. In the speeches given during this period, McCain has the least spin, followed by Clinton, followed by Obama, with noticeably higher levels of spin. McCain generally used (and uses) high rates of first-person singular pronouns, justifying his ’straight talk” claim; Clinton generally used high rates of exclusive words, adding refinement and qualification to many of her statements. Obama’s speeches were lacking in both: he used “we” at extremely high rates (and “I” hardly at all), and his statements were simple and declarative. This makes for speeches that are light on content but, when well delivered, emotionally uplifting. (Reading Obama’s speeches rather than hearing makes one wonder what all the fuss is about — the speeches themselves are rather dry, and the delivery is everything.)
5. Obama decides he’s won. Over the weekend of February 24th, Obama’s language patterns changed dramatically, becoming very similar to Clinton’s. I conclude that this weekend his campaign did the calculations and decided that Clinton could not win the nomination (which seemed, and seems, mathematically true). He cannot have consciously altered his speech patterns, so this must the result of reframing what’s going on to himself — presumably stepping out from behind the persona he had been using before that and presenting something closer to his real self.
6. The past month. In the past month, both Obama and Clinton show higher levels of spin whenever the pressure on them has increased, and they have become defensive. For example, Obama’s levels of spin jumped back to January levels when the Wright controversy became public. In such situations, Obama’s level of spin is characteristically higher than Clinton’s.
7. Responses versus statements. It is difficult to analyze and compare the debate statements of the candidates with their speeches. The question and answer form naturally changes the rates of word usage: for example, if the question is “Would you…” it’s much more likely that the answer will begin “I will…”. And, to make matters worse, debates are not really question and answer since candidates have prepared statements for likely questions and they will use them regardless of the form (and sometimes the content) of the question. It is not yet clear how applicable the deception model is in question and answer situations, so I have not analyzed the debates, except for the most-recent — where Obama still shows up having higher spin than Clinton.
8. Does spin work? Spin is a two-edged sword. On the one hand, it does work in the sense that it can make a speaker appealing to people who would otherwise not be attracted to him/her — which is why,  of course, politicians use it. On the other hand, if a candidate steps out of character, even briefly, people may realize that it is a facade and react in a strong negative way. And, to make it harder, the facade and the language usage are largely subconscious, so a candidate may misstep without realizing it.
9. “I” versus “we”. There’s a lot of (positive) discussion of Obama’s high rates of use of “we”. This pronoun is irrelevant to deception — people who are being honest or deceptive may use “we” at either high or low rates. However, the difference between using these two pronouns is partially understood. People who are being open and not status conscious typically use “I” a lot, while those who are being closed and status conscious typically use “we” a lot. In particular, “we’ is often code for “you” — commanding without creating the impression of command. In other words, “we” is a weasel word.
10. Growing into a persona. It’s possible that Clinton and McCain have been in politics so long that a persona that they originally assumed has now become so much a part of them that it has become their real personality; and that is why their levels of spin are low. By this explanation, the reason that Obama has such high levels of spin is that he’s a relative newcomer to the US national arena, and so he still “puts on” a persona. This doesn’t seem all that convincing — first, he has a long history in public life, although on a smaller stage; second, he seems to be able to step out from behind the persona when things are going well.

The analysis on which this summary is based (and the figures that go with it) can be found in earlier postings.

Clinton and Obama spin including Penn debate

Here are the results of the analysis of spin for Clinton and Obama, using speeches from 2008, and the Penn debate last night. 19 is Clinton, and 39 is Obama.

Clinton: blue dots; Obama: blue stars.

As I’ve noted before, Clinton in general uses less spin than Obama, except when she’s under pressure. It’s also clear from this figure that there is a significant difference between the two orthogonal to the line that indicates level of spin. This is almost entirely due to Clinton’s liking for action verbs, to which Obama is relatively allergic.

It’s not quite fair to compare debate content to prepared speeches, because a debate requires some level of responsiveness to questions (which changes the model of deception). However, it was clear that most of the answers last night were heavily prepped, so perhaps it’s not too different.

The striking thing is that both performances represent all-time lows in spin for both candidates. This is somewhat surprising since Clinton is presumptively in the last few days of her candidacy. Both candidates seem to have decided that they can only be who they are — which has to be a good thing.

Mind you, Obama is still in love with the pronoun “we”. He almost said “when we become president” last night!

US presidential election spin — recent speeches

As promised, I’ve updated the plot of candidate spin to reflect only the speeches given in March.

The significance of this is that spin is always relative to some set of similar documents; and, in the Democratic campaign at least, the substance and tone of the conversation has changed substantially in the past few weeks. All candidates have moved away from presenting themselves as people to presenting their policies (McCain is back talking about himself, but this isn’t captured in this set of speeches).

9 is Obama’s race speech, 10 is his Iraq speech.

McCain still shows low levels of spin. However, the position of Obama and Clinton has largely reversed. Clinton shows high levels of spin, suggesting that her policy speeches are being simplified from the way she would usually speak (which is plausible). Obama continues to increase his level of first-person singular pronoun use, suggesting that he remains confident that he has won the nomination, despite the media’s attempts to keep stirring the pot.

Update: US presidential election spin

I’ve updated the spin analysis to reflect the speeches given up to the last weekend in March.

McCain: red dots, Clinton: blue dots; Obama: blue stars.

You can see a transition in the type of speeches, as candidates move away from speeches intended to introduce themselves as people to speeches that express policy. These look inherently more spinful compared to the earlier speeches, at least for McCain and Clinton. Arguably some of Obama’s movement is also because of the same transition.

The trends in spin are shown in the following figures:

McCain:

Clinton:

Obama:

Although there is a general trend to speeches that score higher on spin than earlier in the campaign, Obama still leads the pack in levels of spin. I will pull out the recent speeches and repeat the analysis to get a new baseline from current content, and see whether this pattern holds true. Results in a day or two.

Changing spin over time

I’ve updated yesterday’s results to include the prepared text of Obama’s speech on Iraq yesterday. The new speech, numbered 39 in the figure, is one of his most spinful.

I’ve also looked at how Clinton’s and Obama’s spin has changed over time.

Clinton started out talking fairly straight, but her spin has been increasing (lower on the graph) recently.

Obama’s level of spin has always been quite high. However, after a patch of much less spin, his last two speeches, on race and on Iraq, have had very high levels of spin.

It’s also interesting how much variation there is from speech to speech. Some of this is probably due to the topic; speeches that involve telling one’s own story naturally produce lower spin scores, and all three candidates spend some time helping people get to know them (or their personas ). There may be some variation because of which speechwriter(s) actually created the text. We were surprised, initially, that there were typical levels of spin because any speech is the product of several people, not just one. But it seems like a good speechwriter must produce something that the candidate is able to say in a natural way, and so aligns with what the candidate would have said.

I don’t know why these figures render so poorly. Any suggestions appreciated.

This is the beginning of the Easter weekend. I won’t post again until Tuesday, unless something time-sensitive happens.

Obama — return of the spin

The big speech on race that Obama made yesterday stimulated me to bring my analysis of the level of spin the candidates are using up to date.

Small clips of Obama talking about issues around his pastor sounded like they contained very high level of spin, especially wonderful phrases like “we need to think about that” (code for “YOU need to think about that”)

The figure below shows projections of all of the candidates speeches (as they appear on their websites), projected onto a axis that represents spin. The red dots are McCain speeches; the blue dots are Clinton speeches; and the blue stars are Obama speeches.

For each candidate, the speeches are in data order: 1-5 McCain; 6-21 Clinton; and 22-38 Obama.

Link to a copy for better readability (maybe)

Obama had a major reduction in spin in all of his speeches around Feb 24th. Although his speeches contain more spin than either of the other two candidates, even his speeches are of two different kinds. Some, such as 37 (US Admirals and Generals) and 28 (Reclaiming the American Dream — autobiography) have quite low spin. Others, such as yesterday’s (38), his speech when he lost Texas and Ohio (36), and his Great Need of the Hour speech (23) have high levels of spin.

You can see that the amount of spin in yesterday’s speech worked. The pundits are impressed; it’s the greatest speech about race in a generation, apparently. It was a good speech. But, perhaps, it wasn’t quite the real Obama.

I stick my my interpretation of the changing spin. Around Feb 24th Obama became convinced that he had won the nomination. Every time since then that something has happened to shed doubt on that, any kind of pressure, and he retreats behind his higher-spin language.

Clinton is also feeling the pressure. 3 of her 4 most recent speeches have had much higher levels of spin than she usually uses.

Levels of spin are not consciously controlled (see earlier posts), so they provide an insight into the mental landscape and framing that is driving each of the candidates.