The Obama/McCain Education Poll and the Heroic Sherman Dorn

by Matthew K. Tabor on August 22, 2008

read 12 pages, be a hero!

Yes, I know this post title sounds like an awesome children’s book, but it isn’t. Stick around - I’m going to spend a little time on this national education poll before I explain why Sherman Dorn is this week’s hero.

In its never-ending quest to be nationally relevant - or at least pretend to be of more interest to the public than it really is - the education sector jumped for joy when The 40th Annual Phi Delta Kappa/Gallup Poll of the Public’s Attitudes Toward Public Schools appeared. After all, it gave the edbloggies an excuse to talk about the presidential candidates as they relate to education.

The Center for Public Education has a summary of the poll’s findings. The real meat is that 46% think Sen. Obama is the best choice to strengthen public schools; 29% chose Sen. McCain.

In no particular order, here’s a rundown of some edublog coverage of this poll:

The National School Boards Association BoardBuzz blog points readers to the poll without judgment or details.

Class Notes, the San Antonio Express News education blog, titles their entry National Poll: Barack Obama Better for Education. They say:

“Americans — 1,002 interviews were completed — were asked a range of questions regarding education, including questions about national education standards, measuring student achievement and how U.S. schools compare with schools in other countries.”

The Education Optimists give about the same coverage. They cite the “nice summary” posted on the Milwaukee Journal Sentinel.

Democrats for Education Reform reposted the MJS piece on their blog [Joe Williams was interviewed for the article].

Komrade Klonsky klaims that “If it’s about education, Obama wins.”

Oh, Good Lord. You get the idea. There are more links, but it’s all about the same. Obama, yada yada. McCain, yada yada.

Professor Dorn, however, took the time to read all 12 pages of the poll. You guys did know there’s some methodology and sample stuff in the back, right? Right.

Dorn’s post was titled, “The 2008 Kappan poll of older, white, wealthy college-educated women east of the Rockies.” Why? Dorn explaineth:

“The most important news to come out of today’s Kappan poll release [PDF] is not the responses to any of the questions but the sample composition: 65% women, 84% white, 50% aged 50 and older, 44% college graduates (and 71% with some college experience), 43% with incomes $50K and above, and 19% from the Census West region of the country. I skimmed through the questions with some interest, and then my jaw dropped on the last page.”

You heard the man - skip to page 12 [pdf] and check out the demographic stats, or peep the table at the very end of this post.

PDK International’s Executive Director defended its processes [lifted from the comment on Dorn's blog entry]:

“For clarification, the Gallup Organization ensures that the poll sample is identified through a truly random process. This means it’s possible to oversample one portion of the population. In order to correct for this, the responses are matched and balanced against the U.S. Census population parameters. That balancing process ensures that the sample reflects the U.S. population. Of course, in all polling, there remains a sampling error, in our case, +/-3%, standard for a national sample.”

Yes, we understand. The point is that such potential variation [or an inherent mistrust of such a weird sample population] requires us to go along cum grano salis, kiddos.

Listen, I know how tempting it is to see a headline or get a press release in your inbox and copy/paste the tasty bits in lieu of actually looking at the material [I'd do more of that myself, but I'm usually too busy].

Resist that temptation, because it makes you look like a fool. It is, however, weirdly charitable; you set the stage for a Dorn-figure to receive great praise for doing what you failed to do.

Fred Klonsky also pointed to Dorn’s expose of the poll folly:

“And here I thought that was the standard research sample. Except for the woman part.”

That got a chuckle, but Fred’s wrong - when jackbooted Rethuglicans like me think about public policy [when we're done stomping on the throats of the poor, that is], we only poll the dead white males.

In all seriousness, Dorn did something that the rest didn’t: He read the damn thing.

Let this be a lesson to ye, education blogosphere.

The best summary of that lesson - and thankfully, it resonated - came during a college history class taught by a fellow blogger. In discussing how important it was to actually read the assigned stuff, as it would make the seminar discussion a bit more fruitful if we all knew the text, he made a larger point:

“Do the reading in life.”

… lest ye look like a moron, which went politely unsaid.

{ 4 comments… read them below or add one }

Stephen Downes 08.22.08 at 8:03 pm

Yep, it’s pretty bad research. I saw it yesterday, didn’t run it for precisely that reason - Tom Hoffman http://www.tuttlesvc.org/2008/08/some-good-news.html gave me the heads up before I even had a chance to open the PDF.

Sad that so many did run it uncritically. I really hate it when something so sloppy is passed off as evidence.

Matthew K. Tabor 08.22.08 at 8:20 pm

Stephen,

Agreed. I sensed that a few bloggers didn’t bother because the summary told them exactly what they wanted to hear, but they were in the minority. The bulk just didn’t bother to read it through.

My M.O. with small polls/studies like that is to read the summary, the skip right to the methodology or sample data, then go back and read it all [unless the methodology is so obviously poor, then I skim].

Really, a rotten job here.

Sherman Dorn 08.24.08 at 8:09 am

Thanks for the kind word (and the cartoon!). It’s my fault that I gave a very quick gloss in the entry: the sample statistics worry me, but there’s a broader context given some surprising year-to-year jumps in data. I’ve explained a bit more today at http://www.shermandorn.com/mt/archives/001380.html

Matthew K. Tabor 08.25.08 at 4:07 pm

Sherman,

Not a problem - it’s remarkable that you were one of the very, very few to do more than repeat the summary. We don’t have to analyze such polls to the nth degree [we shouldn't bother, I should say], but this one deserved a little attention.

Leave a Comment

You can use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>