SAT and ACT Mean Nothing?
Education Sector’s Mr. Aldeman, one of the prolific writers on The Quick and the Ed, has declared dead the usefulness of the SAT/ACT. It wasn’t his idea; he read it in Crossing the Finish Line:
Crossing the Finish Line has things to say about virtually every important factor in college life, but by far the most important thing is this:
The SAT and ACT do not matter in predicting college success.
I have been an unequivocal supporter of using the SAT/ACT* in making college admissions decisions (see here and here), but this sample of students and the rigor of this study are impossible to ignore.
No one should ignore what’s in Crossing, but I’m not about to gobble it hook, line and sinker.
The conclusions are based on a ton of data:
“Crossing the Finish Line, an important new book by former Princeton president William Bowen, former Macalaster College president Michael McPherson, and Matthew Chingos, relied on two massive databases on the entering class of 1999–one on 96,000 first-time freshmen and 30,000 entering transfer students at 21 flagship universities and the other on 108,000 freshmen and 42,000 transfers at less selective state colleges and universities in four states (Maryland, North Carolina, Ohio, and Virginia)–to compile a wide-ranging book of empirical research on topics impacting American higher education.”
Aldeman details his position-reversal on the value of these tests:
I have been an unequivocal supporter of using the SAT/ACT* in making college admissions decisions (see here and here), but this sample of students and the rigor of this study are impossible to ignore. Here’s what the authors found:
- Taken separately, high school GPA is a better predictor of college graduation rates than SAT/ACT score. This findings holds true across institution type, and gets stronger the less selective an institution is. High school GPA is three to five times more important in predicting college graduation than SAT/ ACT score.
- SAT and ACT scores are proxies for high school quality. When the authors factored in which high schools students attended (i.e. high school quality), the predictive power of high school GPA went up, and the predictive power of SAT/ ACT scores fell below zero.
- High school quality mattered, but not nearly as much as the student’s GPA. Other research, most notably on Texas’ ten percent admission rule, has proven this before. It’s somewhat counter-intuitive, but it shows that a student’s initiative to succeed, complete their work, and jump any hurdles that come up matters more than the quality of their high school.
Then he asks, “What should various actors do with this information?”
Time out.
As I wrote on the Quick’s blog entry, here’s why:
“Keep in mind that Alderman’s entire argument – and the authors’ – rests on the definition of “college success” being “graduation” or “obtaining a degree/certification.”
In theory, that’s sufficient. I’d prefer to talk about reality.
In reality, some degrees are watered-down and border on useless. At some institutions, the majority of programs fall into this category. If we pretend for a second that the degrees they award are little more than certificates of attendance and good standing with the Bursar’s Office, we do higher education reform a disservice.
Bowen, McPherson and Chingos, in a roundabout way, may have just proven not that SAT/ACT scores indicate nothing, but that high school GPA-as-harbinger means higher education is increasingly mimicking the weakness of the average American public high school.”
Mr. Aldeman et al.: Get serious about what a degree means - and what it doesn’t - and then we’ll get to work on the value of the ACT/SAT. Until then, I’m not about to worship at the altar of Crossing along with the EdSectorites.
Come to think of it, Education Sector could profit a bit from ACTA’s What Will They Learn?
11 Responses to “SAT and ACT Mean Nothing?”
Trackbacks/Pingbacks
- “I Don’t Want to Ruin My GPA” — Education for the Aughts - American School Issues and Analysis - [...] week we found out via EdSector’s Chad Aldeman that the SAT/ACT are useless. The GPA, he says, as he ...
- Two good articles « Mr. Bailey on the Web - [...] The amount of money involved is particularly troubling since these tests have been shown again and again not to ...
I’m no expert on this, but the prospect of using “high school quality” and GPA makes me nervous. My understanding is that the idea of a national test was developed exactly to give kids from lousy high schools a chance to compete against kids from Dalton and Choate. An admissions officer might discount the 3.9 average of the kid from Podunk HS in Virginia and take the kid with the 3.0 from Groton.
Matthew, you’re right that we can’t measure “college success” much better than “obtaining a degree.” That’s unfortunate, but you have to remember that almost all previous studies have defined “success” as “first-year college grades” or “one-year retention rate.” Surely you’d agree that success is closer to graduation than it is to those interim measures.
Dan, one of the original purposes of the SAT was to find talented students from unknown high schools to attend Ivy League schools. But now it’s used as a giant sorting mechanism to dole out college placements. That should be enough to make anyone nervous, especially now that we have really good evidence that it’s not predicting what it says it does.
There’s also the fact that by the time students arrive at college, they have already been partially sorted by means of ACT/SAT scores. The high-scoring ones have tended to be accepted by selective institutions, middle-scoring ones by less-selective ones, and low-scoring ones by open-admission colleges and unversities. So it might be accurate to say that within a specified range of test scores, high school grades are more predictive of college graduation, but I sure hope the author isn’t saying that students who score below 19 on the ACT are likely to graduate from Cal Tech if they have good grades from that Podunk HS.
I’ve always been confused about what SAT/ACT is supposed to predict. I had read long ago that it correlated highly with grades achieved in freshman year-not whether the student graduated. I found this study (http://professionals.collegeboard.com/profdownload/pdf/RR%2088-11.PDF)
on the College Board’s website done in 1988 looking at whether high school GPA or SAT scores were better predictors of “college success”. Reading through the report, “college success” appeared to be freshman grades.
Dan,
The simplicity with which some are looking at both tests and GPA is appalling. I read today an article pointing to Aldeman’s original piece that was titled, “Idea of the day: ditch standardized tests.”
We need evaluators to commit to intellectual honesty regarding what these tests - and GPA - mean. What we’re seeing is folks sticking to evidence on meaning when it’s convenient to their argument, ignoring meaning when it isn’t, and weaving in context as they see fit.
The debate tells us a lot about how some of the most respect names/think-tanks view education.
Chad,
I’ve responded to your comment here:
http://www.matthewktabor.com/2009/11/12/i-dont-want-to-ruin-my-gpa/
Anon,
I can’t tell what Aldeman is trying to say. I suspect that we’ll have a better handle on it when he figures it out, too.
Barry,
The love affair with what the SAT predicts seems to take a backup role to what the test shows. Common sense dictates that we’d get a handle on meaning before we use it to predict with any value; unfortunately, as we well know, common sense rarely reigns.
ospiti » Porcasi Gaetano
artist
Gaetano Porcasi is a Sicilian artist and school art teacher. His paintings are considered unique not only for their social and political commitment but also for the technique and choice of typical Mediterranean colours from which a strong and deep Sicilitudine (Sicilian mood) emerges. The 2003 itinerant exhibition Portella della Ginestra Massacre is a good example: in 1947 a group of Sicilian farmers was shot and killed in Portella by the outlaw Salvatore Giuliano and his men under orders from the local Mafia mobsters and big landowners in order to stop the farmers’ attempts to occupy and plant uncultivated local land. His historical paintings which denounce the violence and oppression of the Mafia find their counterpart in his paintings which depict sunny Sicilian landscapes rich in lemon, orange and olive trees, in prickly pear, agave and broom plants. They show the wealth of a land that has been kissed by God but downtrodden by man. In painting the sky of his native Sicily Gaetano uses several different hues of blue and it’s from this sky that his pictorial journey starts. In his paintings the history of Sicily, which has always been marked by its farmers’ sweat and blood and by their struggles for freedom and democracy, finds its pictorial expression in the fusion of the red flags of the workers with the Italian flag in a sort of Italian and Mediterranean epopea. The red flags and the Italian flag stand out against the blue sky that changes its hues according to the events, the seasons, the deeds and the moods that are painted on the canvas. The luxuriant nature of Sicily with its beautiful, sunny, Mediterranean landscapes seems to remain the silent, unchangeable and unchanged witness to events and the passing of time. Here people are only accidenti, they aren’t makers of their own life. Thus Gaetano makes a clear-cut metaphysical distinction between a benign, merciful nature and Man who breaks the natural harmony to satisfy his wild, unbridled ambition and selfishness and who becomes the perpetrator of violence and crime. Gaetano is also an active environmentalist and his fight against all forms of pollution has already cost him a lot of aggravation.
Dear Friends:
I believe the disruptive innovation in the Educational industry will occur when we start implementing a Quality Assurance System.
Education, as any other continuous process industry, needs to implement a quality control system; a Total Quality Assurance. There is a big difference between having a QC system, and measuring the quality of a given education. While the second choice provides metrics after the process has been completed, the first choice offers the managers (the teachers) the possibility to act according to the responses and make the necessary changes to achieve the desired quality. A QC system requires metrics in real time; a continuous evaluation.
The other thing that the Education industry needs is a “measurement” system. Not a set of standardized tests. A QC system uses standard units to measure the different steps of a process. The concept of Learning Objects, used in the e-learning industry as standard of content, can easily be adopted by the Educational industry to measure the courses. Instead of using “credits” or “units”, LOs could be used to quantitatively and qualitatively measure the syllabus contained in a curriculum. Correlating content between institutions would be just a matter of matching the LOs of the desired curriculum. LOs also could be used for internationalization of career titles or certificates.
This Virtual platform will potentiate teachers’ capacities to become “teaching managers”. I would like to paraphrase Deming: Teachers who will work ON the system, monitoring study performance data of individuals, and correcting their weaknesses on-time, will achieve the desired knowledge (quality).
Hi Mr. Tabor,
My name is Kristen Phelps and I’m currently in Dr. Stanges’ EDM310 class at the University of South Alabama in Mobile, Alabama. I’ve been assigned to comment on your blog! I’ve actually heard several debates, in school this year, regarding the SAT/ACT’s usefulness. Personally, I think that both have advantages but in terms of measuring what your success in college is going to be, I’m not so sure. Overall, this was a very interesting post! Thanks for sharing!