Don’t Ask This Question, Part IV: Evidence Examined

sad but true

The “Don’t Ask This Question” series between David Warlick and myself has taken on an interesting life. What started as a discussion of blogging’s place in professional competency has become a broader evaluation of the value of blogging in education.

Here’s a quick genesis:

David’s latest piece starts with an admirable, transparent admission resulting in some strikethrough editing of his original post. It demonstrates one of the strengths of blogging: we can go back, edit and leave traces of our thoughts. I appreciate his kind words regarding this website and would like to echo the sentiments; I also appreciate the time he has given to this discussion.

As I said in Part III, one of the most important elements of determining the value of a technology is comparing it to what it’s replacing and what other solutions we’re foregoing. This is basic opportunity cost analysis.

Consider the case of University of Washington communications professor Kathy Gill. Last semester Professor Gill used the newsblog website Newsvine so her students could publish their pieces, read their peers’ work and react to it. Using blogging tools allowed Gill and her students to meet the need to collaborate in a cheap, easy and effective way. Gill identified a need and then chose the solution that fit best.

In the UW case, students researched and wrote pieces on a broad range of issues. Using the Newsvine format, they created a private group and posted that content for consumption by Newsvine users who were free to react. It not only gave Gill’s class a central space to read content and critique it alongside the public, but also provided a clean system to collect and track submissions.

Newsvine gave the UW Bloggers a combination of logistical utility, collaboration/review and exposure. Other technologies could not have met all these needs: A wiki would provide collaboration and tracking but not exposure or outside review; a forum gives the possibility of review, but would largely go unnoticed and would create some hurdles with forum/user administration; social networks like Myspace would not have provided an adequate format for review and discussion, etc.

Blogging provided Gill’s class with clear utility that other technologies couldn’t deliver. This, however, is a rare case; few classes focus wholly on the creation and management of digital media and the use of the systems being discussed. I invite everyone to detail in the comments here other cases in which blogging met an educational need when other technologies couldn’t – it will help us further define blogging utility.

But if we’re to evaluate and choose a proposed solution properly – just like Gill – we need to be fair about the comparison.

Warlick relays an anecdote about a math professor for his graduate studies:

The program director for my graduate degree was brilliant. She was a mathematician. Although she had never taught in a K-12 classroom, she had a fair handle on our challenges. However, she taught entirely from the writings of great education thinkers and from juried journals. I learned a great deal from her. I respect and revere her. But I have to say that what I learned during those years from the listservs and newgroups I was reading at the time, written to by other practitioners like me, was AT LEAST as valuable to me and my work as what I learned in the data-oriented writings that we read and were tested on in graduate classes.

It sounds as though she was a competent teacher and I agree that it doesn’t necessarily matter that she didn’t have direct K-12 classroom experience [as I wrote earlier this week, one need not be an everyday practitioner to be an authority]. However, as Warlick points out, she relied entirely on education theorists and scholarly writing, an approach that would be categorized by any serious educator as sub-optimal.

In order to determine the real value of technologies like listservs, newsgroups and now blogging, our comparative analysis can’t use the low-hanging fruits that are sub-optimal practices. We need to identify the most effective approaches and then determine the value of technologies when implemented. There’s no sense in putting a blogging band-aid on poor practice; our efforts are better spent shoring up the practice than building on a shaky foundation. We can’t justify the value of blogging when we put it up against bad practice.

Warlick continues:

“You see, I was a real teacher.”

“I taught real 7th and 8th graders in a real classroom.”

“I taught in a real community and I taught real curriculum.”

Each of my students was different. There was no other classroom exactly like mine. The community that I taught in was unique and back then I determined the curriculum that I taught. Real teachers face new situations every day. Every child has a different story, a different frame of reference to build on. Every classroom has its own tools to use, or lack of tools. A real teacher has to be resourceful, inventive, tenacious, and absolutely critical to success is a community of other real teachers and conversations.

The emphases are his, not mine.

The assumption that theory necessarily deviates from reality is flawed. It doesn’t have to. The frequent use of the world “real” to differentiate value between two things is synthetic and largely hollow. I’ve told countless students that there is no such thing as a difference between school and the “real world” – there are just people who have isolated themselves or not applied themselves beyond the normal curricula. If you’re a 22-year old college graduate who’s excited to jump into the “real world,” you most certainly haven’t made use of the past four years.

It’s also important to note that “real” isn’t easily quantifiable. Neither are invention, tenacity or conversations. They’re relative intangibles that depend on anecdotes for their justification and stretch or contract as needed.

Anecdotal learning – not to be confused with experiential learning – isn’t enough. We don’t just learn anecdotally, and anecdotes make for inefficient process. Would you take someone seriously who said, “You can learn a lot by getting a degree in physics, but I learned a lot by talking to physicists?” Would you rather a teacher enrolled in a summer enrichment program teaching scientifically-based practices or would you rather he stayed at home and surfed the web?

One can learn by talking to others, but it isn’t remotely scientific and there’s no real quality control; the content might be right, it might be wrong, it might be harmful. It feels good to communicate with other professionals and it’s useful to read about the practices of others, but are 500,000 unique blogs the ideal media to transmit that?

Warlick continues:

I know that conversation is a vague term, but the conditions that real teachers work in are shapeless and unpredictable.

This is terribly misleading and nothing short of a cop-out. Yes, conditions change – no class will ever be the same and every kid deviates from the mean in his own way. Though there are tens of thousands of tree species in the world, all of them have trunks and roots. And remember: if a kid in your class is one in a million, that means there are 1,000 kids exactly like him in China [and another thousand in India]. The point is that there are plenty of basic, common traits in all classrooms and we can use that knowledge to teach better.

Conversation is indeed a vague term, but it’s one that proves difficult to define. Let’s assume for the sake of this discussion that conversations have a clear meaning in blogging: person A publishes content and interacts with persons B through Z in a series of comments. Let’s also assume that these conversations are of great value and critical to the education of all involved.

If conversations are so important – and really, web 2.0ers hail conversations as both the means and the ends of social media – why aren’t comments indexed, searchable and tagged? Many times the wordcount of comments dwarfs the original article and the real meat of the discussion cures in the threads. When considering the immaturity of the blogging medium and the incompleteness of posts, David said:

But, it is what’s not said that leads to conversations (like this) where fuller understanding and even new and valuable ideas can be found or even grown.

The context suggests that this understanding comes in the conversations generated by the original content – conversations that are archived but inaccessible compared to other web media.

Gary Stager made several important points in a comment on David’s In Defensive of Education Blogs article. He points out that commenters – those other parties in the conversation – can’t edit their work as Warlick did in his post. He also mentions the problem of fleeting discussion; blog posts too often influence a brief moment in time to which we seldom return [yes, there are legendary threads with staying power, but they make up an infinitesimal percentage of the blogosphere]. And, if such an effect is desired, a blogger can bury a topic under new material at his discretion. In short, writers can manipulate the conversation and readers/commenters – again, the real meat of the conversation – are largely powerless.

These points are problematic on several levels from utility to basic communication theory. Blogs are useful as a cottage industry thought farm and higher-level blogs are useful for culling the best of the crops, but we’re far from proclaiming that blogging has educational value that can compete with its communicative profit.

4 Responses to “Don’t Ask This Question, Part IV: Evidence Examined”

  1. Matthew,

    This is an interesting series of posts and an interesting conversation that is taking place between yourself and David Warlick. It’s also interesting how you two are able to converse – without ever meeting face to face – and without leaving comments on each other’s blogs. Agreed, blogging has definite communicative profit.

    To define “educational profit”, however, can be much more difficult – because what is educationally profitable to you may not be worthwhile to me. To me (being perfectly honest), I find educational profit in using blogs “as a cottage industry thought farm” and find even more educational profit in “higher-level blogs” because they actually “are useful for culling the best of the crops.”

    What more can you ask from an educational tool?

    Granted, additional evidence would be nice: cold, hard data. However, in attempting to “prove” the educational value of blogging, I’m afraid you’re going to run into the same problems in providing evidence for it’s utility as researchers have encountered for the last several decades in demonstrating the educational value of technology itself: isolated, quantitative data is simply too hard to acquire, and qualitative data is usually too full of bias.

    Nevertheless, as one that is in need of a good dissertation topic, I look forward to the continued conversation – and thank blogs for the (highly educational) opportunity to do so.

  2. Jason Hando says:

    One thought from me to try and add to the already substantial debate happening here….

    Conversations are not the type of learning that readily improves one’s core subject knowledge (such as a Mathematician’s knowledge of algebra) BUT they can have an impact on one’s practice (such as a Mathematics teacher’s lessons).

    I think this is the distinction that I see in the blogging community (and skyping/twittering/etc community). People are connecting to improve their practice.

    And if you read the book Wikinomics you will get a plethora of other examples, taking this idea of connectedness into the business and scientific world.

  3. Wesley Fryer says:

    I’d like to add two points to this conversation. First, while I’d love to believe you when you write “there is no such thing as a difference between school and the ‘real world,’ I continue to work with K-12 teachers around the state of Oklahoma and elsewhere in the Southwest and I’m here to tell you, there are BIG differences between our classrooms and the world outside those walls. In the business world, collaboration is not immediately defined as “cheating,” as it still is in many educational contexts where students are expected to perform individually rather than in a group or on a team. As a knowledge worker in our attention economy, I am not filling out scantron bubble sheets and selecting the correct answer from a limited list of four choices on a daily or even monthly basis. The problems and challenges knowledge workers face are frequently nebulous, undefined, and rather “messy.” The general focus of school is to simplify ideas and present them in an ordered taxonomy (the curriculum) so students can transfer those facts into short term memory and regurgitate them on demand for an examination. The “real world” outside of classrooms rarely values that skill set, in my own work experiences outside of education in the U.S. military and in the telecommunications industry.

    The second issue I’ll take up is that of conversation and its value. I commented on this on Andrew’s blog this evening, and as of this writing my comment is awaiting moderation. One of the bases of research I recommend reviewing and citing in relation to the value of conversation for learning and transfer is Dr. Carl Wieman’s November 2005 presentation at UBC “SCIENCE EDUCATION IN THE 21ST CENTURY: USING THE TOOLS OF SCIENCE TO TEACH SCIENCE.” You’ll find a link to that presentation at the bottom of his webpage at UBC. Wieman cites research on retention for thousands of first year physics students around the world. What does the research show? Traditional instructional methods which fail to engage students in conversations about the studied material fail to result in long-term retention, transfer, and learning. Kids pass tests because they cram facts into their short-term memory banks, but “traditional school” (which does not, btw, tend to value or emphasize conversations) do not produce learning outcomes which last. I think Wieman makes a strong case for emphasizing conversations within instruction that is based on educational research.

  4. Matthew says:


    I’m sorry that I’ve taken awhile to respond to this comment – it’s been a very busy week. I’ll give some dashed responses because most of the points you raised would require a detailed explanation that I can’t get around to at this point [but hopefully can eventually].

    - A failure to align the classroom with the outside world [or a failure to understand that] does not constitute a difference. Though you can cherrypick differences like collaboration as it relates to cheating in academics, it is an apples/oranges argument that is invalid. The same invalidity can be demonstrated with equating multiple-choice tests to the average job and your other examples.

    - Your point about “traditional school” and the value it places on conversations or interactions cites a terribly short window of “traditional school.” I would consider Socrates an older, “traditional” thinker – the foundation of the Socratic method is not unlike many of the points your work stresses. There is no need to cast aside evidence to justify reinventing a wheel that quite simply only needs to be adapted slightly to fit a new vehicle.


  1. Educational blogging and its discontents « Union Street - [...] 2nd, 2007 by Andrew Over the past two weeks there’s been an ongoing debate between Matthew ...

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>