Update of persona deception in the US presidential election to early September

These results are somewhat limited by the low number of Romney speeches that seem to be posted on the web; and do not include any of the convention speeches.
Recall that persona deception is the attempt by politicians to appear better than they are; that this is a form of deception detectable by the Pennebaker deception model; and that, all things being equal, the candidate who presents him/herself as most wonderful will win.
This figure shows the level of persona deception in speeches, where red is Obama and blue is Romney. The straight line is the axis of deception with high sores at the red end and low scores at the green end.

The pattern is very much as it was a few months ago; and, on this evidence, Obama is still on track to win.

Here’s a great example of high scoring Obama:

“…you are aware that there is a pretty intense campaign going on right now. And the reason it’s intense is because the choice that we face in November could not be bigger. It’s not just a choice between two candidates. It’s not even just a choice between two parties. More than any election in recent memory, this is a choice between two fundamentally different paths for our country’s future.”

Not a first-person singular pronoun to be seen; hardly an exclusive word. And it could have been said by anyone.

 

Spin in the US Presidential Primaries — Summary

As we enter what looks like it might be the end phase of the primary season, I thought I would summarize what I’ve written about spin during the process.

1. What is spin? People often talk about spin as messing with the content of a communication: leaving bits out, or changing the emphasis. What I’m talking about here is a mental (unconscious) process where a person presents themselves or their content in a way that does not reflect what they know to be true about it. Politicians (and the rest of us) do this, to some extent, all of the time — trying to make a good impression. For a politician, outright lying is a poor idea (recall Clinton under sniper fire) but there is a lot of pressure to be “all things to all men”. Because the communication is not the speaker’s natural persona, this kind of spin produces a detectable signature in the communication.
2. What is the model of spin (deception)? This work is based on Pennebaker’s empirically-derived model of word-usage changes when people are being deceptive. This model is characterized by (a) reduced rates of first-person singular pronouns; (b) reduced rates of exclusive words, words that mark the beginning of a phrase or clause that qualifies or refines what has gone before; (c) increased rates of negative-emotion words; and (d) increased rates of action verbs. These changes are unconsciously produced, so cannot be directly altered by a speaker, even one who knows the model. Although the model was developed for plain deception (outright lying) it seems to detect deception across the full range from lying, through spin, to negotiating and dating.
3. Why does there have to be a context? Because the model relies on increases and decreases in word-usage rates, there must be some kind of context of similar communications or documents to be able to tell whether a given frequency represents an increase or a decrease. Therefore, absolute spin scores cannot be determined — instead we can only rank a set of communications from most to least spinful. Even within the context of the presidential primaries, underlying language use has changed, most obviously from a ”getting to know me” phase to a “getting to know my policies” phase.
4. The early primaries. From the beginning of 2008 until the 3rd week of February, all three candidates were introducing themselves. In the speeches given during this period, McCain has the least spin, followed by Clinton, followed by Obama, with noticeably higher levels of spin. McCain generally used (and uses) high rates of first-person singular pronouns, justifying his ‘straight talk” claim; Clinton generally used high rates of exclusive words, adding refinement and qualification to many of her statements. Obama’s speeches were lacking in both: he used “we” at extremely high rates (and “I” hardly at all), and his statements were simple and declarative. This makes for speeches that are light on content but, when well delivered, emotionally uplifting. (Reading Obama’s speeches rather than hearing makes one wonder what all the fuss is about — the speeches themselves are rather dry, and the delivery is everything.)
5. Obama decides he’s won. Over the weekend of February 24th, Obama’s language patterns changed dramatically, becoming very similar to Clinton’s. I conclude that this weekend his campaign did the calculations and decided that Clinton could not win the nomination (which seemed, and seems, mathematically true). He cannot have consciously altered his speech patterns, so this must the result of reframing what’s going on to himself — presumably stepping out from behind the persona he had been using before that and presenting something closer to his real self.
6. The past month. In the past month, both Obama and Clinton show higher levels of spin whenever the pressure on them has increased, and they have become defensive. For example, Obama’s levels of spin jumped back to January levels when the Wright controversy became public. In such situations, Obama’s level of spin is characteristically higher than Clinton’s.
7. Responses versus statements. It is difficult to analyze and compare the debate statements of the candidates with their speeches. The question and answer form naturally changes the rates of word usage: for example, if the question is “Would you…” it’s much more likely that the answer will begin “I will…”. And, to make matters worse, debates are not really question and answer since candidates have prepared statements for likely questions and they will use them regardless of the form (and sometimes the content) of the question. It is not yet clear how applicable the deception model is in question and answer situations, so I have not analyzed the debates, except for the most-recent — where Obama still shows up having higher spin than Clinton.
8. Does spin work? Spin is a two-edged sword. On the one hand, it does work in the sense that it can make a speaker appealing to people who would otherwise not be attracted to him/her — which is why,  of course, politicians use it. On the other hand, if a candidate steps out of character, even briefly, people may realize that it is a facade and react in a strong negative way. And, to make it harder, the facade and the language usage are largely subconscious, so a candidate may misstep without realizing it.
9. “I” versus “we”. There’s a lot of (positive) discussion of Obama’s high rates of use of “we”. This pronoun is irrelevant to deception — people who are being honest or deceptive may use “we” at either high or low rates. However, the difference between using these two pronouns is partially understood. People who are being open and not status conscious typically use “I” a lot, while those who are being closed and status conscious typically use “we” a lot. In particular, “we’ is often code for “you” — commanding without creating the impression of command. In other words, “we” is a weasel word.
10. Growing into a persona. It’s possible that Clinton and McCain have been in politics so long that a persona that they originally assumed has now become so much a part of them that it has become their real personality; and that is why their levels of spin are low. By this explanation, the reason that Obama has such high levels of spin is that he’s a relative newcomer to the US national arena, and so he still “puts on” a persona. This doesn’t seem all that convincing — first, he has a long history in public life, although on a smaller stage; second, he seems to be able to step out from behind the persona when things are going well.

The analysis on which this summary is based (and the figures that go with it) can be found in earlier postings.

Update: US presidential election spin

I’ve updated the spin analysis to reflect the speeches given up to the last weekend in March.

McCain: red dots, Clinton: blue dots; Obama: blue stars.

You can see a transition in the type of speeches, as candidates move away from speeches intended to introduce themselves as people to speeches that express policy. These look inherently more spinful compared to the earlier speeches, at least for McCain and Clinton. Arguably some of Obama’s movement is also because of the same transition.

The trends in spin are shown in the following figures:

McCain:

Clinton:

Obama:

Although there is a general trend to speeches that score higher on spin than earlier in the campaign, Obama still leads the pack in levels of spin. I will pull out the recent speeches and repeat the analysis to get a new baseline from current content, and see whether this pattern holds true. Results in a day or two.

Changing spin over time

I’ve updated yesterday’s results to include the prepared text of Obama’s speech on Iraq yesterday. The new speech, numbered 39 in the figure, is one of his most spinful.

I’ve also looked at how Clinton’s and Obama’s spin has changed over time.

Clinton started out talking fairly straight, but her spin has been increasing (lower on the graph) recently.

Obama’s level of spin has always been quite high. However, after a patch of much less spin, his last two speeches, on race and on Iraq, have had very high levels of spin.

It’s also interesting how much variation there is from speech to speech. Some of this is probably due to the topic; speeches that involve telling one’s own story naturally produce lower spin scores, and all three candidates spend some time helping people get to know them (or their personas 🙂 ). There may be some variation because of which speechwriter(s) actually created the text. We were surprised, initially, that there were typical levels of spin because any speech is the product of several people, not just one. But it seems like a good speechwriter must produce something that the candidate is able to say in a natural way, and so aligns with what the candidate would have said.

I don’t know why these figures render so poorly. Any suggestions appreciated.

This is the beginning of the Easter weekend. I won’t post again until Tuesday, unless something time-sensitive happens.

Obama — return of the spin

The big speech on race that Obama made yesterday stimulated me to bring my analysis of the level of spin the candidates are using up to date.

Small clips of Obama talking about issues around his pastor sounded like they contained very high level of spin, especially wonderful phrases like “we need to think about that” (code for “YOU need to think about that”)

The figure below shows projections of all of the candidates speeches (as they appear on their websites), projected onto a axis that represents spin. The red dots are McCain speeches; the blue dots are Clinton speeches; and the blue stars are Obama speeches.

For each candidate, the speeches are in data order: 1-5 McCain; 6-21 Clinton; and 22-38 Obama.

Link to a copy for better readability (maybe)

Obama had a major reduction in spin in all of his speeches around Feb 24th. Although his speeches contain more spin than either of the other two candidates, even his speeches are of two different kinds. Some, such as 37 (US Admirals and Generals) and 28 (Reclaiming the American Dream — autobiography) have quite low spin. Others, such as yesterday’s (38), his speech when he lost Texas and Ohio (36), and his Great Need of the Hour speech (23) have high levels of spin.

You can see that the amount of spin in yesterday’s speech worked. The pundits are impressed; it’s the greatest speech about race in a generation, apparently. It was a good speech. But, perhaps, it wasn’t quite the real Obama.

I stick my my interpretation of the changing spin. Around Feb 24th Obama became convinced that he had won the nomination. Every time since then that something has happened to shed doubt on that, any kind of pressure, and he retreats behind his higher-spin language.

Clinton is also feeling the pressure. 3 of her 4 most recent speeches have had much higher levels of spin than she usually uses.

Levels of spin are not consciously controlled (see earlier posts), so they provide an insight into the mental landscape and framing that is driving each of the candidates.

“I” versus “we” revisited

Many people intuitively think of “I” and “we” as two ends of a continuum, where the ends are something like:

“I”, Hillary Clinton, egotistical, one-person band

and

“we”, Barack Obama, inclusive, working together

In all of the work we’ve done analyzing text, the frequencies of “I” and “we” are uncorrelated, so that knowing one tells you nothing about the other. In other words, there are people who use high rates of both, some who use “I” a lot but not “we”, some who use “we” a lot but not “I”, and some who use very few of both. So any idea of a continuum is rather misleading.

From the point of view of deception, it’s the frequency of “I” that matters: high rates equal openness, low rates equal a facade.

Until a week ago, Obama was using very low rates of “I” and so I’ve argued that he was spinning hard; while Clinton uses much higher rates, and so is presenting something closer to her “real self”. (Since last weekend, Obama has changed his use of “I”.)

It also matters what kind of context the speech is made in, most importantly how much the speaker is constrained in how they can speak. The signals of deception are much clearer in free statements than they are in responses to questions. There are quite large differences in language use between the opening statements in debates, and the subsequent answers to questions, even though politicians are good at using questions as a platform for saying what they wanted to say anyway.

It doesn’t seem to matter whether the speech is scripted, as most official speeches are, or extempore (off the cuff). Obviously a formal speech is not usually written by the speaker. It seems, though, that speech writers must capture language patterns that the speaker is comfortable with to be successful at the job. So scripted speeches tend to have the language patterns of the speaker, and not of the speech writer. It may also be that a speaker adjusts the use of the the little words that are important for detecting deception on the fly; we think of this as making tiny, unimportant deviations from the script, but they may actually be extremely important.

Improving the process of detecting deception

The main problem with Pennebaker’s deception model, discussed in an earlier post, is that the signal for deception for two of the word classes is a decrease in word frequencies. But a decrease with respect to what? Without some care, a text that does not use any of certain word classes can look unusually deceptive, just because it happens not to use certain kinds of words. For example, formal business writing almost never uses first-person singular pronouns (“I”, “me”, “my”) so an occasional text that used one or two might be considered undeceptive just on that basis.

It’s better to consider the deceptiveness of a group of documents or texts from a common domain, and rank them from most to least deceptive, rather than imagining that deception is a kind of absolute property. Then it’s clear that decrease means “frequencies that are lower compared to the norms of documents in this domain”.

It also turns out to be useful to consider the correlations between word usage across the documents in a domain. There may be conventions about the way in which ideas are expressed that go beyond simple word frequencies. So deception detection is improved by considering the correlation among messages, and using it to pick out documents that are more unusual for enhanced consideration. This is the approach I’ve used to look at Enron emails and politicians’ speeches, for example
here for the US presidential election in February 2008, and
for the Canadian Federal election in 2006.

Obama thinks he has the nomination

In my previous post, I pointed out that Obama has changed the way in which he uses first-person singular pronouns — increasing them dramatically, a strong signal that he has reduced the amount of spin he is using.

I wasn’t sure why this was happening. In the debate this evening, he’s using the same increased level of first-person singular pronouns.

The change is truly dramatic.

I think perhaps he has decided that he has the nomination, and he’s stepping out from behind the careful facade that he’s used to get this far. Certainly he learned something this weekend that has changed his attitude to the campaign. I don’t think that this much of a change could be managed consciously.

It’s a big change.

“I” versus “we”

A number of people have noticed that there are substantial differences in the way Hillary Clinton and Barack Obama use pronouns. This is part, but not the whole story about deceptiveness and spin in communication, as I’ve talked about in earlier postings.

The conventional explanation goes something like this:

“Clinton says “I” a lot because she’s egotistical, or because she plans to get things done, with or without the help of other people. Obama says “we” a lot because he is inclusive and wants to develop a shared dream that all can be involved in.”

This view is completely wrong, although it’s probably true that Obama, at some time, developed his use of “we” because he thought or was told that it would create an inclusive impression.

The reality is almost exactly the opposite. People who use first-person singular pronouns (“I”, “me”, “my”) do so, unconsciously, because they are being open, warm, and low status.

People who use first-person plural pronouns (“we”,”us”,”ours”) are not being as open, especially if the speaker is a man. Men use such pronouns as a velvet glove around an iron fist, a way to command without the appearance of command.

How do I know this? The models of language have been derived empirically, much of it the work of James Pennebaker and his group. They have studied the language that people use in known situations, and derived word usage profiles that have enabled them to draw these kinds of conclusions.

Obama uses the language he does because it works — people do perceive him as inclusive. But that doesn’t mean that that’s his real view of the situation. Clinton has adopted, consciously or not, an approach that presents her real self much more directly

It’s not all about pronouns. Other important components are exclusive words (“but”,”nor” which both McCain and Clinton have used in the past week!); negative emotion words (“angry”); and action verbs (“going”).