Raila Odinga has a couple of times recently made conspicuous public mention of the Kenya 2007 IRI/USAID/UCSD exit poll results identifying him as the winning vote-getter, including in his speech at the tenth anniversary of the founding of the Orange Democratic Movement party a few days ago, as well as a significant discussion in his autobiography.

Even a year-and-a-half after the Kenyan election, in July 2009, Kenyan Ambassador to the United States Peter Ogego said at the National Endowment for Democracy in Washington that it was important to get to the bottom of the situation with the U.S.-sponsored exit poll indicating an Odinga rather than a Kibaki win.  The late Congressman Donald Payne, then Chairman of the House Foreign Affairs Subcommittee on Africa said at the same event that the poll should have been published sooner and that not releasing it had been a mistake, although IRI, he thought, had a “good reason” for not releasing it initially.  This is the basic structure of what actually happened, contra what IRI claimed in a March 29, 2009 “rebuttal” to the New York Times investigation. (My point here is still not to berate IRI for continuing to publish this defamatory material worldwide, but I have sadly come to realize that many people seem to have been, surprisingly to me, actually misled by at least some of it.)

On Monday, January 14, 2008 the International Republican Institute’s Coalition for Electoral and Political Process Strengthening (CEPPS) manager submitted by email to USAID at 6:25pm our formal Quarterly Report on the Kenya polling program.  The program had begun with an exit poll for the 2005 constitutional referendum and was scheduled to end with our final pre-election public opinion survey in September 2007, but an amendment that September added the exit poll for the 2007 general election.

Here is this January 14, 2008 report as released under the Freedom of Information Act:

CEPPS IRI Kenya 8038_Oct-Dec 2007

In the report, we at IRI wrote:

Implementation of the December 2007 General Elections Exit Poll
IRI initiated discussions on the exit poll to be conducted during the December 2007 general elections. IRI reviewed the survey instruments, deployment plans, and schedules. Discussions between IRI, USAID, and the local polling firm, Strategic Public Relations and Implementation of the December 2007 General Elections Exit Poll
Research (“Strategic”), took place. Researchers from the University of California at San Diego also partnered with IRI to advise on the sample design, methodology, and data analysis, which they are using for independent studies on polling.

Training of Researchers
In consultation with IRI, Strategic conducted training sessions for the researchers collecting exit poll data. As with the previous polls, Strategic trained a number of researchers, who later deployed to the field as trainers of trainers (TOTs) to identify and train research assistants that would be used to collect data.

The training reviewed field resource management techniques, sampling, and interviewing techniques, as well as training to ensure that all staff had a good understanding of the questionnaire. The questionnaire was then pre-tested in various constituencies of Nairobi. The interviewers later met for a debrief and assessment of the pre-test before deploying nationally.

Data Collection

The poll was fielded on election day in Kenya, December 27, 2007.  A group of 2,887 researchers from Strategic deployed in teams to 175 of 210 constituencies, covering all eight provinces of Kenya.

The interviewers were expected to carry out interviews approximately 100 meters from polling stations.  The interviews were limited to people that had just voted, and the administration of the questionnaire varied from less than five to seven minutes.  To ensure the validity of the sample, between 15 to 25 interviews were conducted at selected polling stations, and only every fifth voter was asked to participate.  Strategic supervisors accompanied researchers to ensure the accuracy of reporting on a number of questionnaires.  Researchers relayed immediate results to their direct supervisors, who then called in to Strategic’s data processing center in Nairobi.

Challenges

During the implementation of the poll, researchers encountered certain challenges, such as the inaccessibility of some areas due to poor roads; poor network coverage; and hostility from polling officials and respondents.  In one instance, a researcher’s questionnaires were confiscated by a polling official.  However, these issues did not significantly affect the data collection exercise.  (emphasis added).

Data Analysis

As data was collected, it was immediately relayed to Strategic headquarters for compilation.  However, data analysis for the exit poll was still ongoing through the end of this quarter. (through December 31)

Earlier that Monday the McClatchy newspapers ran Shashank Bengali’s story “Kenyan president lost election according to U.S. exit poll”. 

Unfortunately, IRI President Lorne Craner spoke to Mr. Bengali from Washington when Bengali and I missed each other by phone and Mr. Craner contradicted what we had reported to USAID:

The head of the International Republican Institute — a nonpartisan democracy-building organization whose work in Kenya was funded by the U.S. Agency for International Development — said the data weren’t released because of concerns about their validity.

The institute contracted an experienced Kenyan polling firm, Strategic Public Relations and Research, which had done two previous national-opinion polls for the institute last year. But on election day the institute’s staff found that pollsters weren’t gathering information in some areas.

 

Nonetheless, that next morning in Washington  IRI’s president was quoted on the American political website RealClear Politics:

— Unintended Consequence Of The Day: Kenyan President Mwai Kibaki lost his bid for a second term, exit polls from the International Republican Institute show and McClatchy reports. The polls show challenger Raila Odinga won the December 27 election by about 8%, an 11-point gap from certified results that suggested Kibaki won by three points. The IRI has been conducting exit polls in Kenya since 1992, funded by USAID, having refined them to the point of reliability. Still, with the poll’s results being leaked just a week after exit pollsters showed Barack Obama beating Hillary Clinton in New Hampshire, it is exactly the wrong time for a pollster to be calling themselves an expert. Further problems, including that interviewers were not where they were supposed to be on Election Day, hampered the survey enough to make it questionable, IRI chief Lorne Craner said.

I don’t have any knowledge at all of how Mr. Craner came to make this second erroneous report to Real Clear Politics where it was published with the coverage of that day’s Republican and Democratic presidential primary contests involving now-President Obama, later-Secretary Clinton and IRI Chairman McCain, among others.  I was busy in Nairobi and did not see it until later, after I returned to the States.

The McClatchy story was the first time anyone at IRI suggested that there was a specific problem that had arisen about the “validity” of the exit poll data–as opposed to a cautionary desire on my part, and I assumed everyone involved–to do our best to verify our work under difficult conditions during the post election violence given the stakes involved.  I thought at the time and acted in good faith in the belief–and still hope–that it was simply a mistake by Mr. Craner in spite of the fact that my boss the Africa Director assured me that she had given  him the correct information.

IRI had higher profile election observation issues going on with Pakistan and Georgia and many much bigger programs around the world at the time and Africa in general had lost staff that weren’t replaced and generally was under resourced.  As I noted previously, IRI did not want to observe the Kenyan election and did not seek to do so–it was only after taking Ambassador Williamson to see Ambassador Ranneberger while in Nairobi that August that IRI was willing to take the idea seriously.   I was told clearly that the decision to release or not release the Exit Poll was to be Mr. Craner’s alone, with the advice of whomever he chose to consult, just as the choice of each of the delegates was his and the signature on the contract with UCSD for their services for the exit poll was his.

After I reported to Washington that I had reached Mr. Bengali in Nairobi with the correction regarding the pollsters not collecting data where expected I was thanked and told that they would handle the communications side from there.  And they did.

Also on that next day, Tuesday, January 15, 2008 IRI posted a formal release on their website saying that they were not prepared to release the exit poll because of concerns that the data was “likely invalid,” which offended our colleagues and partners at UCSD and Strategic, needless to say.  IRI brought in a Washington consultant who raised on an initial summary of the data (from Washington) a list of questions and concerns.  He argued to Dr. Gibson of UCSD that the poll should not be released due to the violence.  I was told by the Washington office that Dr. Gibson agreed that the poll was not reliable due to the consultant’s list of questions, but Dr. Gibson, with whom I had been in contact directly, told me quite the opposite.  Given the list of questions and the totality of the situation at the time, I along with UCSD agreed to re-enter digitally the polling data from the original paper questionnaires as an additional clean-up/verification exercise but this was not taken up at the time.

Nonetheless, I was told that a meeting was scheduled for a final decision on release of the poll by Mr. Craner on February 1, 2008.  I was continually told that Mr. Craner “wants” to release the poll, which I believed.  I prepared a memo addressed to Mr. Craner and the IRI executives outlining the issue and concluding with my recommendation that we should release the poll.  My boss, the Africa Director approved the final draft, but instructed me to remove the concluding recommendation.

I scheduled lunch with Mr. Bengali, to meet him for the first time and with the expectation of being able to provide him the Exit Poll results, which would have been the adjusted 6% figure as released by UCSD at CSIS–against behind the scenes objections by IRI–and then by IRI in August before the UCSD researchers, Dr. Gibson and Dr. Long, testified to the Kreigler Commission about the exit poll–not the leaked 8% figure which was the preliminary figure from election day before any adjustments.  (As noted in our January 14 report to USAID, the questionnaires were seized by police at one location and we were never able to recover them–in one other place we did.)

Unfortunately, the decision at the February 1 meeting went the other way and I had to tell Mr. Bengali at lunch that the decision was not to release.  IRI’s press secretary later provided a draft of a press release asserting that the poll was not being released as it had been concluded to be “invalid” relying on the initial list of questions from the Washington consultant verbatim.  I objected strenuously to this characterization and obtained a commitment from the Africa Director that I would be given an opportunity to fact check any final release before publication and advance notice to prepare our local staff for the reaction in Nairobi.  Unfortunately, early on the morning of February 8 Nairobi time, I was called by a Washington staff member to report that the statement, with objectionable characterization of the poll as “invalid”, had been posted on line that evening Washington time following Senator Feingold’s hearings that day and a visit to IRI headquarters by ODM party leaders.

Almost three months later I turned over the original questionnaires to my predecessor as IRI East Africa Director on May 4 with the IRI residence in Lavington, IRI having declined to bring them back to the states to re-enter the data in the meantime.  I had been told that the decision not to release was final and would not be reconsidered, and was ultimately based on “the best interests of IRI”.  On my return to the States in late May, I was routed through Atlanta instead of through Washington for the policy-specified exit briefing.  When I was contacted with an interview request by Mike McIntire of the New York Times in late July, I had been given no indication whatsoever that IRI was reconsidering the release or re-evaluating the “invalid” data and I knew that they had tried to convince CSIS not to host the release of results by UCSD earlier that month in Washington.

On this blog in May 2014 I asked Why is IRI’s report on the Kenya 2007 Exit Poll missing from the USAID Development Experience Clearinghouse? (FOIA Series Part 13):

A key document that should answer a number of questions is the IRI final report on the 2005-2007 polling program, which was originally due during my tenure at IRI in early 2008. At the time I completed my IRI service to return to my permanent job in the U.S. IRI’s second extension to file the report was winding down. At that point, IRI was faced with a quandary as it had posted on its website on February 7, 2008 a statement that it was not releasing the exit poll results because it had determined that they were “invalid” the evening following a demand by Senator Russ Feingold in a hearing of his Africa subcommittee of the Senate Foreign Relations Committee that the Assistant Secretary of State for Africa and the Africa Assistant Administrator for USAID report back to him on why the exit poll had not been released. Previously, however, in January IRI had filed its quarterly performance report with USAID reporting that the poll had been successfully conducted.

According to the requirements of the Cooperative Agreement between USAID and IRI, three copies of the final report were to be submitted by IRI, one to the agreement officer in Washington, one to the Democracy and Governance lead in Kenya, and another to the USAID Development Experience Clearinghouse (DEC) in Washington. I was able to learn in a conference call with the FOIA office last week that they have been unable to find such a copy on file in Development Experience Clearinghouse. Likewise, the other copy in Washington has not yet turned up, so it is being sought through the mission in Kenya.

Strange.

I eventually learned through FOIA that IRI never did file that final report for the 2005-2007 program at all with the apparent informal acquiescence of USAID; they just ducked out of answering the inconvenient questions of how the exit poll became temporarily “invalid” from February to August.

IRI’s current president, Ambassador Mark Green, will be familiar with some of this from his political appointment as Ambassador to Tanzania during the time of the Kenyan election and its aftermath as I noted in Part Ten of this series:

As Ambassador in Tanzania, Green received the cables from Ambassador Ranneberger in Kenya that I have discussed in my FOIA Series on this blog, including Ranneberger’s pre-election description of the planned exit poll: “The Mission is funding national public opinion polling to increase the availability of objective and reliable data and to provide an independent source of verification of electoral outcomes via exit polls. The implementing partner is IRI.” [emphasis added]. Likewise Ambassador Ranneberger’s January 2 cable describing personally witnessing the altered vote tallies received at the ECK headquarters prior to the announcement of Kibaki as winner on December 30. See Part Ten–FOIA Documents From Kenya’s 2007 Election–Ranneberger at ECK: “[M]uch can happen between the casting of votes and the tabulation of ballots, and it did”.